From: Luca Boccassi <lboccass@Brocade.com>
To: "wenzhuo.lu@intel.com" <wenzhuo.lu@intel.com>
Cc: "dev@dpdk.org" <dev@dpdk.org>
Subject: Re: [dpdk-dev] [PATCH v6 0/4] support reset of VF link
Date: Wed, 6 Jul 2016 16:26:37 +0000 [thread overview]
Message-ID: <1467822396.32466.35.camel@brocade.com> (raw)
In-Reply-To: <6A0DE07E22DDAD4C9103DF62FEBC09090348DB05@shsmsx102.ccr.corp.intel.com>
On Wed, 2016-07-06 at 00:45 +0000, Lu, Wenzhuo wrote:
> Hi Luca,
>
> > -----Original Message-----
> > From: Luca Boccassi [mailto:lboccass@Brocade.com]
> > Sent: Tuesday, July 5, 2016 5:53 PM
> > To: Lu, Wenzhuo
> > Cc: dev@dpdk.org
> > Subject: Re: [dpdk-dev] [PATCH v6 0/4] support reset of VF link
> >
> > On Tue, 2016-07-05 at 00:52 +0000, Lu, Wenzhuo wrote:
> > > Hi Luca,
> > >
> > >
> > > > -----Original Message-----
> > > > From: Luca Boccassi [mailto:lboccass@Brocade.com]
> > > > Sent: Monday, July 4, 2016 11:48 PM
> > > > To: Lu, Wenzhuo
> > > > Cc: dev@dpdk.org
> > > > Subject: Re: [dpdk-dev] [PATCH v6 0/4] support reset of VF link
> > > >
> > > > On Mon, 2016-06-20 at 14:24 +0800, Wenzhuo Lu wrote:
> > > > > If the PF link is down and up, VF link will not work accordingly.
> > > > > This patch set addes the support of VF link reset. So, when VF
> > > > > receices the messges of physical link down/up. APP can reset the
> > > > > VF link and let it recover.
> > > > >
> > > > > PS: This patch set is splitted from a previous patch set,
> > > > > *automatic link recovery on ixgbe/igb VF*, and it's base on the
> > > > > patch set *support mailbox interruption on ixgbe/igb VF*.
> > > > >
> > > > > Wenzhuo Lu (3):
> > > > > lib/librte_ether: support device reset
> > > > > ixgbe: implement device reset on VF
> > > > > igb: implement device reset on VF
> > > > >
> > > > > Zhe Tao (1):
> > > > > i40e: implement device reset on VF
> > > > >
> > > > > v1:
> > > > > - Added the implementation for the VF reset functionality.
> > > > > v2:
> > > > > - Changed the i40e related operations during VF reset.
> > > > > v3:
> > > > > - Resent the patches because of the mail sent issue.
> > > > > v4:
> > > > > - Removed some VF reset emulation code.
> > > > > v5:
> > > > > - Removed all the code related with lock.
> > > > > v6:
> > > > > - Updated the NIC feature overview matrix.
> > > > > - Added more explanation in the doxygen comment of reset API.
> > > > >
> > > > > doc/guides/nics/overview.rst | 1 +
> > > > > doc/guides/rel_notes/release_16_07.rst | 13 ++++++
> > > > > drivers/net/e1000/igb_ethdev.c | 59 ++++++++++++++++++++++++
> > > > > drivers/net/i40e/i40e_ethdev.h | 4 ++
> > > > > drivers/net/i40e/i40e_ethdev_vf.c | 83
> > > > ++++++++++++++++++++++++++++++++++
> > > > > drivers/net/i40e/i40e_rxtx.c | 10 ++++
> > > > > drivers/net/i40e/i40e_rxtx.h | 4 ++
> > > > > drivers/net/ixgbe/ixgbe_ethdev.c | 64 +++++++++++++++++++++++++-
> > > > > drivers/net/ixgbe/ixgbe_ethdev.h | 2 +-
> > > > > drivers/net/ixgbe/ixgbe_rxtx.c | 12 +++--
> > > > > lib/librte_ether/rte_ethdev.c | 17 +++++++
> > > > > lib/librte_ether/rte_ethdev.h | 24 ++++++++++
> > > > > lib/librte_ether/rte_ether_version.map | 7 +++
> > > > > 13 files changed, 295 insertions(+), 5 deletions(-)
> > > >
> > > > Hello Wenzhuo,
> > > >
> > > > I'm testing this patchset, but I am sporadically running into an
> > > > issue where the VFs reset fails after the PF flaps.
> > > >
> > > > I have a VM running on a KVM box with a X540-AT2, passing 2 VFs in.
> > > >
> > > > I am using calling rte_eth_dev_reset in response to a
> > > > RTE_ETH_EVENT_INTR_RESET callback, and the following errors appear
> > > > in the
> > > > log:
> > > >
> > > > PMD: ixgbevf_dev_reset(): Ixgbe VF reset: Failed to update link.
> > > > PMD: ixgbe_alloc_rx_queue_mbufs(): RX mbuf alloc failed queue_id=0
> > > > PMD: ixgbevf_dev_start(): Unable to initialize RX hardware (-12)
> > > > PMD: ixgbevf_dev_reset(): Ixgbe VF reset: Failed to start device.
> > > >
> > > > Jumping in with GDB, it seems that the rte_rxmbuf_alloc call in
> > > > ixgbe_alloc_rx_queue_mbufs returns NULL at iteration 64 out of 2048.
> > > > The application has ~500 2MB hugepages, and there's 2GB of free
> > > > memory available on top of that.
> > > >
> > > > Have you seen this before? Any pointer or suggestion for debugging?
> > > >
> > > > Thanks!
> > > >
> > > > --
> > > > Kind regards,
> > > > Luca Boccassi
> > > I think the problem is the mbuf occupied by the packets is not released. This
> > memory has to be released by the APP, so my patches haven’t covered this.
> > Actually an example is needed to show how to use the reset API. I plan to modify
> > the testpmd.
> > > You may notice this feature is postponed to 16.11. Would you like to wait for
> > the new version that will include an example?
> >
> > Hi,
> >
> > Unfortunately we need the VF reset working sooner than that, so one way or
> > the other I'll need to sort it out. Given I've got a use case where this is happening,
> > if it can be helpful for you I'm more than happy to help as a guinea pig. If you
> > could please give some guidance/guidelines with regards to which API to use to
> > sort the mbuf problem, I can try it out and give back some feedback.
> >
> > Thanks!
> I made a stupid mistake and deleted all my code. So, I have to take some time to rewrite it :(
> Attached the example I used to test the reset API. It's modified from the l2fwd example. So you can compare it with l2fwd to see what need to be added.
> Hopefully it can help :)
Thanks! That made me understand a couple of things more, and I've got
past the problem.
Unfortunately now there's a bigger issue - rte_eth_dev_reset is a
blocking call. the _RESET event callback is fired when the PF goes down,
but when I call rte_eth_dev_reset it will block until the PF goes back
up. There is no way, as far as I can see, to know if the PF is back up
before calling rte_eth_dev_reset.
This is a problem because, as far as I understand, I have to call all
the rte_eth_dev_ APIs from the same thread, in my case the master
thread, and I can't have that block potentially indefinitely.
Would it be possible to have 2 events instead of 1, one when the PF goes
down and one when it goes up? This way an application would be able to
soft-stop the port (drain queues, etc) when the PF is down, and then
call the reset API when it goes back up.
Thanks!
--
Kind regards,
Luca Boccassi
next prev parent reply other threads:[~2016-07-06 16:26 UTC|newest]
Thread overview: 72+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-06-06 5:40 [dpdk-dev] [PATCH 0/8] " Wenzhuo Lu
2016-06-06 5:40 ` [dpdk-dev] [PATCH 1/8] lib/librte_ether: support device reset Wenzhuo Lu
2016-06-06 5:40 ` [dpdk-dev] [PATCH 2/8] lib/librte_ether: defind RX/TX lock mode Wenzhuo Lu
2016-06-08 2:15 ` Stephen Hemminger
2016-06-08 7:34 ` Lu, Wenzhuo
2016-06-09 7:50 ` Olivier Matz
2016-06-12 5:25 ` Lu, Wenzhuo
2016-06-10 18:12 ` Stephen Hemminger
2016-06-12 5:27 ` Lu, Wenzhuo
2016-06-06 5:40 ` [dpdk-dev] [PATCH 3/8] ixgbe: RX/TX with lock on VF Wenzhuo Lu
2016-06-06 5:40 ` [dpdk-dev] [PATCH 4/8] ixgbe: implement device reset " Wenzhuo Lu
2016-06-06 5:40 ` [dpdk-dev] [PATCH 5/8] igb: RX/TX with lock " Wenzhuo Lu
2016-06-06 5:40 ` [dpdk-dev] [PATCH 6/8] igb: implement device reset " Wenzhuo Lu
2016-06-06 5:40 ` [dpdk-dev] [PATCH 7/8] i40e:RX/TX with lock " Wenzhuo Lu
2016-06-06 5:40 ` [dpdk-dev] [PATCH 8/8] i40e: implement device reset " Wenzhuo Lu
2016-06-15 3:03 ` [dpdk-dev] [PATCH v5 0/4] support reset of VF link Wenzhuo Lu
2016-06-15 3:03 ` [dpdk-dev] [PATCH v5 1/4] lib/librte_ether: support device reset Wenzhuo Lu
2016-06-16 15:31 ` Bruce Richardson
2016-06-16 15:36 ` Thomas Monjalon
2016-06-15 3:03 ` [dpdk-dev] [PATCH v5 2/4] ixgbe: implement device reset on VF Wenzhuo Lu
2016-06-15 3:03 ` [dpdk-dev] [PATCH v5 3/4] igb: " Wenzhuo Lu
2016-06-15 3:03 ` [dpdk-dev] [PATCH v5 4/4] i40e: " Wenzhuo Lu
2016-06-20 6:24 ` [dpdk-dev] [PATCH v6 0/4] support reset of VF link Wenzhuo Lu
2016-06-20 6:24 ` [dpdk-dev] [PATCH v6 1/4] lib/librte_ether: support device reset Wenzhuo Lu
2016-06-20 9:14 ` Jerin Jacob
2016-06-20 16:17 ` Stephen Hemminger
2016-06-21 3:51 ` Jerin Jacob
2016-06-21 6:14 ` Lu, Wenzhuo
2016-06-21 7:37 ` Jerin Jacob
2016-06-21 8:24 ` Lu, Wenzhuo
2016-06-21 8:55 ` Jerin Jacob
2016-06-21 9:26 ` Ananyev, Konstantin
2016-06-21 10:57 ` Jerin Jacob
2016-06-21 13:10 ` Ananyev, Konstantin
2016-06-21 13:30 ` Jerin Jacob
2016-06-21 14:03 ` Ananyev, Konstantin
2016-06-21 14:29 ` Jerin Jacob
2016-06-22 1:35 ` Lu, Wenzhuo
2016-06-22 2:37 ` Jerin Jacob
2016-06-22 3:32 ` Lu, Wenzhuo
2016-06-22 4:14 ` Jerin Jacob
2016-06-22 5:05 ` Lu, Wenzhuo
2016-06-22 6:10 ` Jerin Jacob
2016-06-22 6:42 ` Lu, Wenzhuo
2016-06-22 7:59 ` Jerin Jacob
2016-06-22 8:17 ` Thomas Monjalon
2016-06-22 8:25 ` Lu, Wenzhuo
2016-06-22 9:18 ` Thomas Monjalon
2016-06-22 11:06 ` Jerin Jacob
2016-06-23 0:45 ` Lu, Wenzhuo
2016-06-23 0:39 ` Lu, Wenzhuo
2016-06-21 0:51 ` Lu, Wenzhuo
2016-06-20 6:24 ` [dpdk-dev] [PATCH v6 2/4] ixgbe: implement device reset on VF Wenzhuo Lu
2016-06-20 6:24 ` [dpdk-dev] [PATCH v6 3/4] igb: " Wenzhuo Lu
2016-06-20 6:24 ` [dpdk-dev] [PATCH v6 4/4] i40e: " Wenzhuo Lu
2016-07-04 15:48 ` [dpdk-dev] [PATCH v6 0/4] support reset of VF link Luca Boccassi
2016-07-05 0:52 ` Lu, Wenzhuo
2016-07-05 9:52 ` Luca Boccassi
2016-07-06 0:45 ` Lu, Wenzhuo
2016-07-06 16:26 ` Luca Boccassi [this message]
[not found] ` <1467822182.32466.34.camel@brocade.com>
2016-07-07 1:09 ` Lu, Wenzhuo
2016-07-07 10:20 ` Luca Boccassi
2016-07-07 13:12 ` Lu, Wenzhuo
2016-07-07 16:19 ` Luca Boccassi
2016-07-08 0:14 ` Lu, Wenzhuo
2016-07-08 17:15 ` Luca Boccassi
2016-07-11 1:32 ` Lu, Wenzhuo
2016-07-11 12:02 ` Luca Boccassi
2016-07-11 15:43 ` Luca Boccassi
2016-07-12 1:19 ` Lu, Wenzhuo
2016-08-26 12:58 ` Luca Boccassi
2016-08-29 1:04 ` Lu, Wenzhuo
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1467822396.32466.35.camel@brocade.com \
--to=lboccass@brocade.com \
--cc=dev@dpdk.org \
--cc=wenzhuo.lu@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).