From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 54307A00C5 for ; Thu, 21 Jul 2022 03:44:44 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C80BE40A87; Thu, 21 Jul 2022 03:44:43 +0200 (CEST) Received: from mail-ej1-f54.google.com (mail-ej1-f54.google.com [209.85.218.54]) by mails.dpdk.org (Postfix) with ESMTP id A70F040A7A for ; Thu, 21 Jul 2022 03:44:42 +0200 (CEST) Received: by mail-ej1-f54.google.com with SMTP id va17so702138ejb.0 for ; Wed, 20 Jul 2022 18:44:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Ha7ZsoCEMSGKauUmEkpSAEJu4nptZGLoSzvcUEHvgFM=; b=SL2AmpKcwbj3nW7extSLHrxoAo5Wwue70J+bvJftkGqyHYeS6ej/lINa33hG14+v1P BoXVcUAhx3YW/98Wvec3y/dHhkr/UYr8Mlq4s59hGJK/lpLhNtO+5YC81Qt4Q03ychoZ ey7pAtKoUONhSiKcJI6avUnGYcFSiPxwKTZ9+1g2NGNZ8qLvBY1ElRUjmIhTh3nZWsA7 cVUDoNn1A4FsLH3+En0s1tudwxEvgvOQNwFRWFpEsFbDGHOiLIS55RlTZ8Vk5+naNgJR bFUnJDM9Iq/XBcB4mVil8Qb410WxamYVYoddTx4UNJIccCTCArbRw3149Tvpvz9Wjsb/ GK8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Ha7ZsoCEMSGKauUmEkpSAEJu4nptZGLoSzvcUEHvgFM=; b=fPxQDb1nvpgPSQu67jjLZrYPnMOQ1mR+LnWoQEDMtO2YtYWCYLM9IaQKOq544Ljh1z 32e9OmDwdkfk3z7nigo2w53lqXexanmVR7d4EvO/2873kKoKMyeJxLlbF3ejipZcSy6g gySfBs6+reYGdaHRcDuuEuFmraicPmnKmRzDfMNCCkFnaQh5c2xtKaqG3pdQzUAEfWyw so9ZJPjYBmUeBj25N0w/5qNWkvjYm/IIcGpc0puovwN43fwD53AKskM5ui8iTXReUnhV ZaZ/z7LcobHYGn3okKtGpLIIbvgKBHK9u1BQt7wf54ZnDEukZ4ea0/wPTzRb5bIvtY4x hfdw== X-Gm-Message-State: AJIora9p4ITzoSViSDG/pMBYSUlryGVCRQHGW3TTeGHPdpRu61kCIj4Q 4Z2/6WAB0Ozl02sXqa6OvO7IM2/8O7Xzlf+G23QafAqcKAWt2w== X-Google-Smtp-Source: AGRyM1svt6L2cE7nzBCel2Cv4UfhW0OVymzSpMu+iGLMKSHcD3U2LTXHoxqoODBuwzOcDqH5Xod+pMcO/5mJ5uLTxqk= X-Received: by 2002:a17:906:cc4a:b0:72b:863e:ef7c with SMTP id mm10-20020a170906cc4a00b0072b863eef7cmr36926501ejb.686.1658367881926; Wed, 20 Jul 2022 18:44:41 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Nobin Mathew Date: Thu, 21 Jul 2022 07:14:31 +0530 Message-ID: Subject: Re: VF is still resetting To: "Xing, Beilei" Cc: "users@dpdk.org" , "Yang, SteveX" , "Yang, Qiming" Content-Type: text/plain; charset="UTF-8" X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org Any pointers? Is this a firmware problem? I am not seeing " dev_err(&pf->pdev->dev, "VF reset check timeout on VF %d\n", " from i40e driver anywhere in syslog. -Nobin On Wed, Jul 20, 2022 at 11:56 AM Xing, Beilei wrote: > > Hi Steve, > > Could you please help on this? Thanks. > > BR, > Beilei > > > -----Original Message----- > > From: Nobin Mathew > > Sent: Wednesday, July 20, 2022 12:18 AM > > To: users@dpdk.org > > Subject: VF is still resetting > > > > Hi, > > > > We are running a dpdk app inside a pod, and orchestrating the app very > > frequently(test app). > > > > 1/100 or so we are getting an error: > > > > 2022-07-17T22:34:24.620291289+03:00 iavf_check_vf_reset_done(): reset > > VFR value: 3 > > 2022-07-17T22:34:24.620310455+03:00 iavf_init_vf(): VF is still resetting > > 2022-07-17T22:34:24.620339697+03:00 iavf_dev_init(): Init vf failed > > 2022-07-17T22:34:24.620390802+03:00 EAL: Releasing PCI mapped resource > > for 0000:3b:0f.5 > > 2022-07-17T22:34:24.620397381+03:00 EAL: Calling pci_unmap_resource for > > 0000:3b:0f.5 at 0x2101000000 > > 2022-07-17T22:34:24.620442514+03:00 EAL: Calling pci_unmap_resource for > > 0000:3b:0f.5 at 0x2101010000 > > 2022-07-17T22:34:24.729012277+03:00 EAL: Requested device 0000:3b:0f.5 > > cannot be used > > 2022-07-17T22:34:24.729028758+03:00 EAL: Bus (pci) probe failed. > > > > we added one log in dpdk lib to print the VFGEN_RSTAT register of the VF. In > > problematic cases, we are seeing the value 3 which maps to 0xDEADBEEF > > > > / VF reset states - these are written into the RSTAT register: > > * VFGEN_RSTAT on the VF > > * When the PF initiates a reset, it writes 0 > > * When the reset is complete, it writes 1 > > * When the PF detects that the VF has recovered, it writes 2 > > * VF checks this register periodically to determine if a reset has occurred, > > * then polls it to know when the reset is complete. > > * If either the PF or VF reads the register while the hardware > > * is in a reset state, it will return DEADBEEF, which, when masked > > * will result in 3. > > / > > enum virtchnl_vfr_states { > > VIRTCHNL_VFR_INPROGRESS = 0, > > VIRTCHNL_VFR_COMPLETED, > > VIRTCHNL_VFR_VFACTIVE, > > }; > > > > We tried this patch also, increasing the poll time, no help. > > https://github.com/DPDK/dpdk/commit/be7226980c9ad4963b92b489c8afb > > 17f08899953 > > > > Details of the setup: > > > > DPDK library version > > 21.11 > > VF Driver:- > > intel-iavf version 4.0.1-3.2 > > PF driver:- > > sudo ethtool -i enp94s0f1 > > driver: i40e > > version: 2.14.13 > > firmware-version: 8.15 0x800096ca 20.0.17 > > > > Since we are seeing 0xDEADBEEF, I am assuming VF-PF reset mailbox msg is > > received by PF, and PF initiated the RESET sequence by writing VFSWR to > > VPGEN_VFRTRIG register. > > > > I am not seeing > > " dev_err(&pf->pdev->dev, "VF reset check timeout on VF %d\n", " > > anywhere in syslog. > > > > Any pointers?, why does this happen(why VF reset is not complete)?... > > > > One more question, what is the sequence of calls in the reset path? > > i40e_vc_process_vf_msg() -> VIRTCHNL_OP_RESET_VF i40e_vc_reset_vf() -> > > i40e_reset_vf() -> i40e_trigger_vf_reset() & i40e_cleanup_reset_vf() > > > > this one? > > > > -Nobin