From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id 6F7378D3C for ; Tue, 17 May 2016 03:11:21 +0200 (CEST) Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga102.jf.intel.com with ESMTP; 16 May 2016 18:11:21 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.26,322,1459839600"; d="scan'208";a="955935578" Received: from fmsmsx104.amr.corp.intel.com ([10.18.124.202]) by orsmga001.jf.intel.com with ESMTP; 16 May 2016 18:11:20 -0700 Received: from fmsmsx156.amr.corp.intel.com (10.18.116.74) by fmsmsx104.amr.corp.intel.com (10.18.124.202) with Microsoft SMTP Server (TLS) id 14.3.248.2; Mon, 16 May 2016 18:11:19 -0700 Received: from shsmsx101.ccr.corp.intel.com (10.239.4.153) by fmsmsx156.amr.corp.intel.com (10.18.116.74) with Microsoft SMTP Server (TLS) id 14.3.248.2; Mon, 16 May 2016 18:11:19 -0700 Received: from shsmsx102.ccr.corp.intel.com ([169.254.2.104]) by SHSMSX101.ccr.corp.intel.com ([169.254.1.148]) with mapi id 14.03.0248.002; Tue, 17 May 2016 09:11:17 +0800 From: "Lu, Wenzhuo" To: Olivier Matz , "dev@dpdk.org" Thread-Topic: [dpdk-dev] [PATCH 3/4] ixgbe: automatic link recovery on VF Thread-Index: AQHRpkl4+MTW5caN1kWULHCu4LuPTJ+7AqsAgAFX2TA= Date: Tue, 17 May 2016 01:11:17 +0000 Message-ID: <6A0DE07E22DDAD4C9103DF62FEBC090903468932@shsmsx102.ccr.corp.intel.com> References: <1462396246-26517-1-git-send-email-wenzhuo.lu@intel.com> <1462396246-26517-4-git-send-email-wenzhuo.lu@intel.com> <5739B698.8010909@6wind.com> In-Reply-To: <5739B698.8010909@6wind.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH 3/4] ixgbe: automatic link recovery on VF X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 May 2016 01:11:22 -0000 Hi Olivier, > -----Original Message----- > From: Olivier Matz [mailto:olivier.matz@6wind.com] > Sent: Monday, May 16, 2016 8:01 PM > To: Lu, Wenzhuo; dev@dpdk.org > Subject: Re: [dpdk-dev] [PATCH 3/4] ixgbe: automatic link recovery on VF >=20 > Hi Wenzhuo, >=20 > On 05/04/2016 11:10 PM, Wenzhuo Lu wrote: > > When the physical link is down and recover later, the VF link cannot > > recover until the user stop and start it manually. > > This patch implements the automatic recovery of VF port. > > The automatic recovery bases on the link up/down message received from > > PF. When VF receives the link up/down message, it will replace the > > RX/TX and operation functions with fake ones to stop RX/TX and any > > future operation. Then reset the VF port. > > After successfully resetting the port, recover the RX/TX and operation > > functions. > > > > Signed-off-by: Wenzhuo Lu > > > > [...] > > > > +void > > +ixgbevf_dev_link_up_down_handler(struct rte_eth_dev *dev) { > > + struct ixgbe_hw *hw =3D IXGBE_DEV_PRIVATE_TO_HW(dev->data- > >dev_private); > > + struct ixgbe_adapter *adapter =3D > > + (struct ixgbe_adapter *)dev->data->dev_private; > > + int diag; > > + uint32_t vteiam; > > + > > + /* Only one working core need to performance VF reset */ > > + if (rte_spinlock_trylock(&adapter->vf_reset_lock)) { > > + /** > > + * When fake rec/xmit is replaced, working thread may is > running > > + * into real RX/TX func, so wait long enough to assume all > > + * working thread exit. The assumption is it will spend less > > + * than 100us for each execution of RX and TX func. > > + */ > > + rte_delay_us(100); > > + > > + do { > > + dev->data->dev_started =3D 0; > > + ixgbevf_dev_stop(dev); > > + rte_delay_us(1000000); >=20 > If I understand well, ixgbevf_dev_link_up_down_handler() is called by > ixgbevf_recv_pkts_fake() on a dataplane core. It means that the core that > acquired the lock will loop during 100us + 1sec at least. > If this core was also in charge of polling other queues of other ports, o= r timers, > many packets will be dropped (even with a 100us loop). I don't think it i= s > acceptable to actively wait inside a rx function. >=20 > I think it would avoid many issues to delegate this work to the applicati= on, > maybe by notifying it that the port is in a bad state and must be restart= ed. The > application could then properly stop polling the queues, and stop and res= tart the > port in a separate thread, without bothering the dataplane cores. Thanks for the comments. Yes, you're right. I had a wrong assumption that every queue is handled by = one core. But surely it's not right, we cannot tell how the users will deploy their s= ystem. I plan to update this patch set. The solution now is, first let the users c= hoose if they want this auto-reset feature. If so, we will apply another series rx/tx functions whi= ch have lock. So we can stop the rx/tx of the bad ports. And we also apply a reset API for users. The APPs should call this API in t= heir management thread or so. It means APPs should guarantee the thread safe for the API. You see, there're 2 things, 1, Lock the rx/tx to stop them for users. 2, Apply a resetting API for users, and every NIC can do their own job. APP= s need not to worry about the difference=20 between different NICs. Surely, it's not *automatic* now. The reason is DPDK doesn't guarantee the = thread safe. So the operations have to be left to the APPs and let them to guarantee the thread safe. And if the users choose not using auto-reset feature, we will leave this wo= rk to the APP :) >=20 >=20 > Regards, > Olivier