From: Nobin Mathew <nobin.mathew@gmail.com>
To: users@dpdk.org
Subject: Re: VF is still resetting
Date: Wed, 20 Jul 2022 10:26:09 +0530 [thread overview]
Message-ID: <CAJgSC2NXcyUw6giZL8DRbtP08ioOrmkLD=b7u9mVCij5tAGyhg@mail.gmail.com> (raw)
In-Reply-To: <CAJgSC2OTZTcDM=d=+LJGNcfc2DVrhbdii8JVyftJQ6RF5XMxyA@mail.gmail.com>
Any pointers?
i40e driver forum asked to submit question here.
iavf in-kernel VF driver has a larger timeout,
#define IAVF_RESET_WAIT_COMPLETE_COUNT 2000
and also reset detection loop timeout
#define IAVF_RESET_WAIT_DETECTED_COUNT 500
We tried even
+#define IAVF_RESET_WAIT_CNT 3000
but no luck....
-Nobin
On Tue, Jul 19, 2022 at 9:48 PM Nobin Mathew <nobin.mathew@gmail.com> wrote:
>
> Hi,
>
> We are running a dpdk app inside a pod, and orchestrating the app very
> frequently(test app).
>
> 1/100 or so we are getting an error:
>
> 2022-07-17T22:34:24.620291289+03:00 iavf_check_vf_reset_done(): reset
> VFR value: 3
> 2022-07-17T22:34:24.620310455+03:00 iavf_init_vf(): VF is still resetting
> 2022-07-17T22:34:24.620339697+03:00 iavf_dev_init(): Init vf failed
> 2022-07-17T22:34:24.620390802+03:00 EAL: Releasing PCI mapped resource
> for 0000:3b:0f.5
> 2022-07-17T22:34:24.620397381+03:00 EAL: Calling pci_unmap_resource
> for 0000:3b:0f.5 at 0x2101000000
> 2022-07-17T22:34:24.620442514+03:00 EAL: Calling pci_unmap_resource
> for 0000:3b:0f.5 at 0x2101010000
> 2022-07-17T22:34:24.729012277+03:00 EAL: Requested device 0000:3b:0f.5
> cannot be used
> 2022-07-17T22:34:24.729028758+03:00 EAL: Bus (pci) probe failed.
>
> we added one log in dpdk lib to print the VFGEN_RSTAT register of the
> VF. In problematic cases, we are seeing the value 3 which maps to
> 0xDEADBEEF
>
> / VF reset states - these are written into the RSTAT register:
> * VFGEN_RSTAT on the VF
> * When the PF initiates a reset, it writes 0
> * When the reset is complete, it writes 1
> * When the PF detects that the VF has recovered, it writes 2
> * VF checks this register periodically to determine if a reset has occurred,
> * then polls it to know when the reset is complete.
> * If either the PF or VF reads the register while the hardware
> * is in a reset state, it will return DEADBEEF, which, when masked
> * will result in 3.
> /
> enum virtchnl_vfr_states {
> VIRTCHNL_VFR_INPROGRESS = 0,
> VIRTCHNL_VFR_COMPLETED,
> VIRTCHNL_VFR_VFACTIVE,
> };
>
> We tried this patch also, increasing the poll time, no help.
> https://github.com/DPDK/dpdk/commit/be7226980c9ad4963b92b489c8afb17f08899953
>
> Details of the setup:
>
> DPDK library version
> 21.11
> VF Driver:-
> intel-iavf version 4.0.1-3.2
> PF driver:-
> sudo ethtool -i enp94s0f1
> driver: i40e
> version: 2.14.13
> firmware-version: 8.15 0x800096ca 20.0.17
>
> Since we are seeing 0xDEADBEEF, I am assuming VF-PF reset mailbox msg
> is received by PF, and PF initiated the RESET sequence by writing
> VFSWR to VPGEN_VFRTRIG register.
>
> I am not seeing
> " dev_err(&pf->pdev->dev, "VF reset check timeout on VF %d\n", "
> anywhere in syslog.
>
> Any pointers?, why does this happen(why VF reset is not complete)?...
>
> One more question, what is the sequence of calls in the reset path?
> i40e_vc_process_vf_msg() -> VIRTCHNL_OP_RESET_VF i40e_vc_reset_vf() ->
> i40e_reset_vf() -> i40e_trigger_vf_reset() & i40e_cleanup_reset_vf()
>
> this one?
>
> -Nobin
next prev parent reply other threads:[~2022-07-20 4:56 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-07-19 16:18 Nobin Mathew
2022-07-20 4:56 ` Nobin Mathew [this message]
2022-07-20 6:26 ` Xing, Beilei
2022-07-21 1:44 ` Nobin Mathew
2022-07-21 9:17 ` Yang, SteveX
2022-07-22 2:19 ` Nobin Mathew
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAJgSC2NXcyUw6giZL8DRbtP08ioOrmkLD=b7u9mVCij5tAGyhg@mail.gmail.com' \
--to=nobin.mathew@gmail.com \
--cc=users@dpdk.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).