From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id 1CAD67272 for ; Fri, 16 Mar 2018 01:42:33 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 15 Mar 2018 17:42:32 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.48,313,1517904000"; d="scan'208";a="25100107" Received: from fmsmsx103.amr.corp.intel.com ([10.18.124.201]) by orsmga007.jf.intel.com with ESMTP; 15 Mar 2018 17:42:31 -0700 Received: from fmsmsx157.amr.corp.intel.com (10.18.116.73) by FMSMSX103.amr.corp.intel.com (10.18.124.201) with Microsoft SMTP Server (TLS) id 14.3.319.2; Thu, 15 Mar 2018 17:42:18 -0700 Received: from shsmsx104.ccr.corp.intel.com (10.239.4.70) by FMSMSX157.amr.corp.intel.com (10.18.116.73) with Microsoft SMTP Server (TLS) id 14.3.319.2; Thu, 15 Mar 2018 17:42:17 -0700 Received: from shsmsx103.ccr.corp.intel.com ([169.254.4.235]) by SHSMSX104.ccr.corp.intel.com ([169.254.5.226]) with mapi id 14.03.0319.002; Fri, 16 Mar 2018 08:42:16 +0800 From: "Zhang, Qi Z" To: "Ananyev, Konstantin" , "thomas@monjalon.net" CC: "dev@dpdk.org" , "Xing, Beilei" , "Wu, Jingjing" , "Lu, Wenzhuo" Thread-Topic: [dpdk-dev] [PATCH v2 1/4] ether: support deferred queue setup Thread-Index: AQHTsdzO/92qABmyvEqdvvZYHITpPKPPNw0AgAFIbcCAAFZxAIAAm4gA//+MDwCAARvJcA== Date: Fri, 16 Mar 2018 00:42:16 +0000 Message-ID: <039ED4275CED7440929022BC67E706115316E424@SHSMSX103.ccr.corp.intel.com> References: <20180212045314.171616-1-qi.z.zhang@intel.com> <20180302041306.90324-1-qi.z.zhang@intel.com> <20180302041306.90324-2-qi.z.zhang@intel.com> <2601191342CEEE43887BDE71AB9772589E28FA63@irsmsx105.ger.corp.intel.com> <039ED4275CED7440929022BC67E706115316DF0F@SHSMSX103.ccr.corp.intel.com> <2601191342CEEE43887BDE71AB9772589E2901FF@irsmsx105.ger.corp.intel.com> <039ED4275CED7440929022BC67E706115316E27D@SHSMSX103.ccr.corp.intel.com> <2601191342CEEE43887BDE71AB9772589E2902E9@irsmsx105.ger.corp.intel.com> In-Reply-To: <2601191342CEEE43887BDE71AB9772589E2902E9@irsmsx105.ger.corp.intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-version: 11.0.0.116 dlp-reaction: no-action x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH v2 1/4] ether: support deferred queue setup X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Mar 2018 00:42:34 -0000 > -----Original Message----- > From: Ananyev, Konstantin > Sent: Thursday, March 15, 2018 11:39 PM > To: Zhang, Qi Z ; thomas@monjalon.net > Cc: dev@dpdk.org; Xing, Beilei ; Wu, Jingjing > ; Lu, Wenzhuo > Subject: RE: [dpdk-dev] [PATCH v2 1/4] ether: support deferred queue setu= p >=20 >=20 >=20 > > -----Original Message----- > > From: Zhang, Qi Z > > Sent: Thursday, March 15, 2018 3:09 PM > > To: Ananyev, Konstantin ; > > thomas@monjalon.net > > Cc: dev@dpdk.org; Xing, Beilei ; Wu, Jingjing > > ; Lu, Wenzhuo > > Subject: RE: [dpdk-dev] [PATCH v2 1/4] ether: support deferred queue > > setup > > > > > > > > > -----Original Message----- > > > From: Ananyev, Konstantin > > > Sent: Thursday, March 15, 2018 9:17 PM > > > To: Zhang, Qi Z ; thomas@monjalon.net > > > Cc: dev@dpdk.org; Xing, Beilei ; Wu, Jingjing > > > ; Lu, Wenzhuo > > > Subject: RE: [dpdk-dev] [PATCH v2 1/4] ether: support deferred queue > > > setup > > > > > > Hi Qi, > > > > > > > -----Original Message----- > > > > From: Zhang, Qi Z > > > > Sent: Thursday, March 15, 2018 3:14 AM > > > > To: Ananyev, Konstantin ; > > > > thomas@monjalon.net > > > > Cc: dev@dpdk.org; Xing, Beilei ; Wu, > > > > Jingjing ; Lu, Wenzhuo > > > > > > > > Subject: RE: [dpdk-dev] [PATCH v2 1/4] ether: support deferred > > > > queue setup > > > > > > > > Hi Konstantin: > > > > > > > > > -----Original Message----- > > > > > From: Ananyev, Konstantin > > > > > Sent: Wednesday, March 14, 2018 8:32 PM > > > > > To: Zhang, Qi Z ; thomas@monjalon.net > > > > > Cc: dev@dpdk.org; Xing, Beilei ; Wu, > > > > > Jingjing ; Lu, Wenzhuo > > > > > ; Zhang, Qi Z > > > > > Subject: RE: [dpdk-dev] [PATCH v2 1/4] ether: support deferred > > > > > queue setup > > > > > > > > > > Hi Qi, > > > > > > > > > > > > > > > > > The patch let etherdev driver expose the capability flag > > > > > > through rte_eth_dev_info_get when it support deferred queue > > > > > > configuraiton, then base on the flag > > > > > > rte_eth_[rx|tx]_queue_setup could decide continue to setup the > > > > > > queue or just return fail when device already started. > > > > > > > > > > > > Signed-off-by: Qi Zhang > > > > > > --- > > > > > > doc/guides/nics/features.rst | 8 ++++++++ > > > > > > lib/librte_ether/rte_ethdev.c | 30 > > > > > > ++++++++++++++++++------------ lib/librte_ether/rte_ethdev.h | > > > > > > 11 +++++++++++ > > > > > > 3 files changed, 37 insertions(+), 12 deletions(-) > > > > > > > > > > > > diff --git a/doc/guides/nics/features.rst > > > > > > b/doc/guides/nics/features.rst index 1b4fb979f..36ad21a1f > > > > > > 100644 > > > > > > --- a/doc/guides/nics/features.rst > > > > > > +++ b/doc/guides/nics/features.rst > > > > > > @@ -892,7 +892,15 @@ Documentation describes performance > > > values. > > > > > > > > > > > > See ``dpdk.org/doc/perf/*``. > > > > > > > > > > > > +.. _nic_features_queue_deferred_setup_capabilities: > > > > > > > > > > > > +Queue deferred setup capabilities > > > > > > +--------------------------------- > > > > > > + > > > > > > +Supports queue setup / release after device started. > > > > > > + > > > > > > +* **[provides] rte_eth_dev_info**: > > > > > > > > > > > > > > > ``deferred_queue_config_capa:DEV_DEFERRED_RX_QUEUE_SETUP,DEV_DEFE > > > > > RRED_ > > > > > > TX_QUEUE_SETUP,DEV_DEFERRED_RX_QUEUE_RELE > > > > > > ASE,DEV_DEFERRED_TX_QUEUE_RELEASE``. > > > > > > +* **[related] API**: ``rte_eth_dev_info_get()``. > > > > > > > > > > > > .. _nic_features_other: > > > > > > > > > > > > diff --git a/lib/librte_ether/rte_ethdev.c > > > > > > b/lib/librte_ether/rte_ethdev.c index a6ce2a5ba..6c906c4df > > > > > > 100644 > > > > > > --- a/lib/librte_ether/rte_ethdev.c > > > > > > +++ b/lib/librte_ether/rte_ethdev.c > > > > > > @@ -1425,12 +1425,6 @@ rte_eth_rx_queue_setup(uint16_t > > > > > > port_id, > > > > > uint16_t rx_queue_id, > > > > > > return -EINVAL; > > > > > > } > > > > > > > > > > > > - if (dev->data->dev_started) { > > > > > > - RTE_PMD_DEBUG_TRACE( > > > > > > - "port %d must be stopped to allow configuration\n", > > > port_id); > > > > > > - return -EBUSY; > > > > > > - } > > > > > > - > > > > > > RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_infos_get, > > > > > -ENOTSUP); > > > > > > > RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_queue_setup, > > > > > -ENOTSUP); > > > > > > > > > > > > @@ -1474,10 +1468,19 @@ rte_eth_rx_queue_setup(uint16_t > > > port_id, > > > > > uint16_t rx_queue_id, > > > > > > return -EINVAL; > > > > > > } > > > > > > > > > > > > + if (dev->data->dev_started && > > > > > > + !(dev_info.deferred_queue_config_capa & > > > > > > + DEV_DEFERRED_RX_QUEUE_SETUP)) > > > > > > + return -EINVAL; > > > > > > + > > > > > > > > > > I think now you have to check here that the queue is stopped. > > > > > Otherwise you might attempt to reconfigure running queue. > > > > > > > > I'm not sure if it's necessary to let application use different > > > > API sequence > > > for a deferred configure and deferred re-configure. > > > > Can we just call dev_ops->rx_queue_stop before rx_queue_release > > > > here > > > > > > I don't follow you here. > > > Let say now inside queue_start() we do check: > > > > > > if (dev->data->rx_queue_state[rx_queue_id] !=3D > > > RTE_ETH_QUEUE_STATE_STOPPED) > > > > > > Right now it is not possible to call queue_setup() without > > > dev_stop() before it - that's why we have check if > > > (dev->data->dev_started) in queue_setup() right now. > > > Though with your patch it not the case anymore - user is able to > > > call > > > queue_setup() without stopping the whole device. > > > But he still has to stop the queue. > > > > > > > > > > > > > > > > > > > > > > > > > rxq =3D dev->data->rx_queues; > > > > > > if (rxq[rx_queue_id]) { > > > > > > > > > RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_queue_release, > > > > > > -ENOTSUP); > > > > > > > > > > I don't think it is *that* straightforward. > > > > > rx_queue_setup() parameters can imply different rx function (and > > > > > related dev > > > > > icesettings) that are already setuped by previous > > > queue_setup()/dev_start. > > > > > So I think you need to do one of 2 things: > > > > > 1. rework ethdev layer to introduce a separate rx function (and > > > > > related > > > > > settings) for each queue. > > > > > 2. at rx_queue_setup() if it is invoked after dev_start - check > > > > > that given queue settings wouldn't contradict with current > > > > > device settings (rx function, etc.). > > > > > If they do - return an error. > > > > Yes, I think what we have is option 2 here, the > > > > dev_ops->rx_queue_setup will return fail if conflict with previous > > > > setting > > > > > > Hmm and what makes you think that? > > > As I know it is not the case right now. > > > Let say I do: > > > .... > > > rx_queue_setup(port=3D0,queue=3D0, mp=3Dmb_size_2048); > > > dev_start(port=3D0); > > > ... > > > rx_queue_setup(port=3D0,queue=3D1,mp=3Dmb_size_1024); > > > > > > If current rx function doesn't support multi-segs then second > > > rx_queue_setup() should fail. > > > Though I don't think that would happen with the current > implementation. > > > > Why you think that would not happen? dev_ops->rx_queue_setup can fail, > right? > > I mean it's the responsibility of low level driver (i40e) to check the = conflict > with current implementation. >=20 > Yes it is responsibility if the PMD because only it knows its own logic o= f rx/tx > function selection. > But I don't see such changes in i40e in your patch series. > Probably I missed them? OK, I think we are aligned on this patch, and I see the problem in i40e pat= ch, will fix.=20 >=20 > > > > > > Same story for TX offloads, though it probably not that critical, as > > > for most Intel PMDs HW TX offloads will become per port in 18.05. > > > > > > As I can see you do have either of these options implemented right > > > now - that's the problem. > > > > > > > I'm also thinking about option 1, the idea is to move per queue > > > > rx/tx > > > function into driver layer, so it will not break existing API. > > > > > > > > 1. driver can expose the capability like per_queue_rx or > > > > per_queue_tx 2. application can enable this capability by > > > > dev_config with rte_eth_conf 3, if per_queue_rx is not enable, > > > > nothing change, so we are at option 2 4. if per_queue_rx is > > > > enabled, driver will set rx_pkt_burst with a hook function which > > > > redirect to an function ptr in a per queue rx function tables ( I > > > > guess performance is impacted somehow, but this is the cost if you > > > > want different offload for different queue) > > > > > > I don't think we need to overcomplicate things here. > > > It should be transparent to the user - user just calls queue_setup() > > > - based on its input parameters PMD selects a function that fits best= . > > > Pretty much what we have right now, just possibly have an array of > > > functions (one per queue). > > > > If we don't introduce a new capability or something like, but just > > take per queue functions as default way, does that mean, we need to > change all drivers to adapt this? > > Or do you mean below? > > > > If (dev->rx_pkt_burst) > > /* default way */ > > else > > /* per queue function */ >=20 > For me either way seems ok. > Second one probably a bit easier, as no changes from PMDs are required. > But again - might be even rte_ethdev layer can fill queue's rx_pkt_burst[= ] > array for the drivers that don't support it - just by copying dev->rx_pkt= _burst > into it. > Konstantin Ok, I will add this in v2 Thanks Qi