From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by dpdk.org (Postfix) with ESMTP id D80F54F9A for ; Thu, 15 Mar 2018 14:16:55 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 15 Mar 2018 06:16:53 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.48,310,1517904000"; d="scan'208";a="39149490" Received: from irsmsx103.ger.corp.intel.com ([163.33.3.157]) by orsmga001.jf.intel.com with ESMTP; 15 Mar 2018 06:16:52 -0700 Received: from irsmsx105.ger.corp.intel.com ([169.254.7.221]) by IRSMSX103.ger.corp.intel.com ([169.254.3.197]) with mapi id 14.03.0319.002; Thu, 15 Mar 2018 13:16:51 +0000 From: "Ananyev, Konstantin" To: "Zhang, Qi Z" , "thomas@monjalon.net" CC: "dev@dpdk.org" , "Xing, Beilei" , "Wu, Jingjing" , "Lu, Wenzhuo" Thread-Topic: [dpdk-dev] [PATCH v2 1/4] ether: support deferred queue setup Thread-Index: AQHTsdzYjJFpxTZzDkGudLbJsh6cp6PPudHggAD5wQCAAKFxQA== Date: Thu, 15 Mar 2018 13:16:50 +0000 Message-ID: <2601191342CEEE43887BDE71AB9772589E2901FF@irsmsx105.ger.corp.intel.com> References: <20180212045314.171616-1-qi.z.zhang@intel.com> <20180302041306.90324-1-qi.z.zhang@intel.com> <20180302041306.90324-2-qi.z.zhang@intel.com> <2601191342CEEE43887BDE71AB9772589E28FA63@irsmsx105.ger.corp.intel.com> <039ED4275CED7440929022BC67E706115316DF0F@SHSMSX103.ccr.corp.intel.com> In-Reply-To: <039ED4275CED7440929022BC67E706115316DF0F@SHSMSX103.ccr.corp.intel.com> Accept-Language: en-IE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiNDFmMjM1ZTMtMTdjNC00YjczLWI0ZWYtYzExMTJhNGIzNDQxIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE2LjUuOS4zIiwiVHJ1c3RlZExhYmVsSGFzaCI6ImJsMStnSmVXcm92QmlqbUZYWDdSbkU5OVlyYmxhMTV6NVRWaVJabW1paGM9In0= x-ctpclassification: CTP_NT dlp-product: dlpe-windows dlp-version: 11.0.0.116 dlp-reaction: no-action x-originating-ip: [163.33.239.181] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH v2 1/4] ether: support deferred queue setup X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Mar 2018 13:16:56 -0000 Hi Qi, > -----Original Message----- > From: Zhang, Qi Z > Sent: Thursday, March 15, 2018 3:14 AM > To: Ananyev, Konstantin ; thomas@monjalon.n= et > Cc: dev@dpdk.org; Xing, Beilei ; Wu, Jingjing ; Lu, Wenzhuo > Subject: RE: [dpdk-dev] [PATCH v2 1/4] ether: support deferred queue setu= p >=20 > Hi Konstantin: >=20 > > -----Original Message----- > > From: Ananyev, Konstantin > > Sent: Wednesday, March 14, 2018 8:32 PM > > To: Zhang, Qi Z ; thomas@monjalon.net > > Cc: dev@dpdk.org; Xing, Beilei ; Wu, Jingjing > > ; Lu, Wenzhuo ; Zhang, Qi = Z > > > > Subject: RE: [dpdk-dev] [PATCH v2 1/4] ether: support deferred queue se= tup > > > > Hi Qi, > > > > > > > > The patch let etherdev driver expose the capability flag through > > > rte_eth_dev_info_get when it support deferred queue configuraiton, > > > then base on the flag rte_eth_[rx|tx]_queue_setup could decide > > > continue to setup the queue or just return fail when device already > > > started. > > > > > > Signed-off-by: Qi Zhang > > > --- > > > doc/guides/nics/features.rst | 8 ++++++++ > > > lib/librte_ether/rte_ethdev.c | 30 ++++++++++++++++++------------ > > > lib/librte_ether/rte_ethdev.h | 11 +++++++++++ > > > 3 files changed, 37 insertions(+), 12 deletions(-) > > > > > > diff --git a/doc/guides/nics/features.rst > > > b/doc/guides/nics/features.rst index 1b4fb979f..36ad21a1f 100644 > > > --- a/doc/guides/nics/features.rst > > > +++ b/doc/guides/nics/features.rst > > > @@ -892,7 +892,15 @@ Documentation describes performance values. > > > > > > See ``dpdk.org/doc/perf/*``. > > > > > > +.. _nic_features_queue_deferred_setup_capabilities: > > > > > > +Queue deferred setup capabilities > > > +--------------------------------- > > > + > > > +Supports queue setup / release after device started. > > > + > > > +* **[provides] rte_eth_dev_info**: > > > > > ``deferred_queue_config_capa:DEV_DEFERRED_RX_QUEUE_SETUP,DEV_DEFE > > RRED_ > > > TX_QUEUE_SETUP,DEV_DEFERRED_RX_QUEUE_RELE > > > ASE,DEV_DEFERRED_TX_QUEUE_RELEASE``. > > > +* **[related] API**: ``rte_eth_dev_info_get()``. > > > > > > .. _nic_features_other: > > > > > > diff --git a/lib/librte_ether/rte_ethdev.c > > > b/lib/librte_ether/rte_ethdev.c index a6ce2a5ba..6c906c4df 100644 > > > --- a/lib/librte_ether/rte_ethdev.c > > > +++ b/lib/librte_ether/rte_ethdev.c > > > @@ -1425,12 +1425,6 @@ rte_eth_rx_queue_setup(uint16_t port_id, > > uint16_t rx_queue_id, > > > return -EINVAL; > > > } > > > > > > - if (dev->data->dev_started) { > > > - RTE_PMD_DEBUG_TRACE( > > > - "port %d must be stopped to allow configuration\n", port_id); > > > - return -EBUSY; > > > - } > > > - > > > RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_infos_get, > > -ENOTSUP); > > > RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_queue_setup, > > -ENOTSUP); > > > > > > @@ -1474,10 +1468,19 @@ rte_eth_rx_queue_setup(uint16_t port_id, > > uint16_t rx_queue_id, > > > return -EINVAL; > > > } > > > > > > + if (dev->data->dev_started && > > > + !(dev_info.deferred_queue_config_capa & > > > + DEV_DEFERRED_RX_QUEUE_SETUP)) > > > + return -EINVAL; > > > + > > > > I think now you have to check here that the queue is stopped. > > Otherwise you might attempt to reconfigure running queue. >=20 > I'm not sure if it's necessary to let application use different API seque= nce for a deferred configure and deferred re-configure. > Can we just call dev_ops->rx_queue_stop before rx_queue_release here I don't follow you here. Let say now inside queue_start() we do check: if (dev->data->rx_queue_state[rx_queue_id] !=3D RTE_ETH_QUEUE_STATE_STOPPED= ) Right now it is not possible to call queue_setup() without dev_stop() befor= e it - that's why we have check if (dev->data->dev_started) in queue_setup() right= now. Though with your patch it not the case anymore - user is able to call queue= _setup() without stopping the whole device. But he still has to stop the queue.=20 >=20 > > > > > > > rxq =3D dev->data->rx_queues; > > > if (rxq[rx_queue_id]) { > > > RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_queue_release, > > > -ENOTSUP); > > > > I don't think it is *that* straightforward. > > rx_queue_setup() parameters can imply different rx function (and relate= d dev > > icesettings) that are already setuped by previous queue_setup()/dev_sta= rt. > > So I think you need to do one of 2 things: > > 1. rework ethdev layer to introduce a separate rx function (and related > > settings) for each queue. > > 2. at rx_queue_setup() if it is invoked after dev_start - check that gi= ven > > queue settings wouldn't contradict with current device settings (rx fu= nction, > > etc.). > > If they do - return an error. > Yes, I think what we have is option 2 here, the dev_ops->rx_queue_setup w= ill return fail if conflict with previous setting Hmm and what makes you think that? As I know it is not the case right now. Let say I do: .... rx_queue_setup(port=3D0,queue=3D0, mp=3Dmb_size_2048); dev_start(port=3D0); ... rx_queue_setup(port=3D0,queue=3D1,mp=3Dmb_size_1024); =20 If current rx function doesn't support multi-segs then second rx_queue_set= up() should fail. Though I don't think that would happen with the current implementation.=20 Same story for TX offloads, though it probably not that critical, as for mo= st Intel PMDs HW TX offloads will become per port in 18.05. As I can see you do have either of these options implemented right now - t= hat's the problem. > I'm also thinking about option 1, the idea is to move per queue rx/tx fun= ction into driver layer, so it will not break existing API. >=20 > 1. driver can expose the capability like per_queue_rx or per_queue_tx > 2. application can enable this capability by dev_config with rte_eth_conf > 3, if per_queue_rx is not enable, nothing change, so we are at option 2 > 4. if per_queue_rx is enabled, driver will set rx_pkt_burst with a hook f= unction which redirect to an function ptr in a per queue rx function > tables ( I guess performance is impacted somehow, but this is the cost if= you want different offload for different queue) I don't think we need to overcomplicate things here. It should be transparent to the user - user just calls queue_setup() - base= d on its input parameters PMD selects a function that fits best. Pretty much what we have right now, just possibly have an array of function= s (one per queue). >=20 > > > > From my perspective - 1) is a better choice though it required more wor= k, > > and possibly ABI breakage. > > I did some work in that direction as RFC: > > http://dpdk.org/dev/patchwork/patch/31866/ >=20 > I will learn this, thanks for the heads up. > > > > 2) might be also possible, but looks a bit clumsy as rx_queue_setup() m= ight > > now fail even with valid parameters - all depends on previous queue > > configurations. > > > > Same story applies for TX. > > > > > > > + if (dev->data->dev_started && > > > + !(dev_info.deferred_queue_config_capa & > > > + DEV_DEFERRED_RX_QUEUE_RELEASE)) > > > + return -EINVAL; > > > (*dev->dev_ops->rx_queue_release)(rxq[rx_queue_id]); > > > rxq[rx_queue_id] =3D NULL; > > > } > > > @@ -1573,12 +1576,6 @@ rte_eth_tx_queue_setup(uint16_t port_id, > > uint16_t tx_queue_id, > > > return -EINVAL; > > > } > > > > > > - if (dev->data->dev_started) { > > > - RTE_PMD_DEBUG_TRACE( > > > - "port %d must be stopped to allow configuration\n", port_id); > > > - return -EBUSY; > > > - } > > > - > > > RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_infos_get, > > -ENOTSUP); > > > RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->tx_queue_setup, > > -ENOTSUP); > > > > > > @@ -1596,10 +1593,19 @@ rte_eth_tx_queue_setup(uint16_t port_id, > > uint16_t tx_queue_id, > > > return -EINVAL; > > > } > > > > > > + if (dev->data->dev_started && > > > + !(dev_info.deferred_queue_config_capa & > > > + DEV_DEFERRED_TX_QUEUE_SETUP)) > > > + return -EINVAL; > > > + > > > txq =3D dev->data->tx_queues; > > > if (txq[tx_queue_id]) { > > > RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->tx_queue_release, > > > -ENOTSUP); > > > + if (dev->data->dev_started && > > > + !(dev_info.deferred_queue_config_capa & > > > + DEV_DEFERRED_TX_QUEUE_RELEASE)) > > > + return -EINVAL; > > > (*dev->dev_ops->tx_queue_release)(txq[tx_queue_id]); > > > txq[tx_queue_id] =3D NULL; > > > } > > > diff --git a/lib/librte_ether/rte_ethdev.h > > > b/lib/librte_ether/rte_ethdev.h index 036153306..410e58c50 100644 > > > --- a/lib/librte_ether/rte_ethdev.h > > > +++ b/lib/librte_ether/rte_ethdev.h > > > @@ -981,6 +981,15 @@ struct rte_eth_conf { > > > */ > > > #define DEV_TX_OFFLOAD_SECURITY 0x00020000 > > > > > > +#define DEV_DEFERRED_RX_QUEUE_SETUP 0x00000001 /**< Deferred > > setup rx > > > +queue */ #define DEV_DEFERRED_TX_QUEUE_SETUP 0x00000002 /**< > > Deferred > > > +setup tx queue */ #define DEV_DEFERRED_RX_QUEUE_RELEASE > > 0x00000004 > > > +/**< Deferred release rx queue */ #define > > > +DEV_DEFERRED_TX_QUEUE_RELEASE 0x00000008 /**< Deferred release > > tx > > > +queue */ > > > + > > > > I don't think we do need flags for both setup a and release. > > If runtime setup is supported - surely dynamic release should be suppor= ted > > too. > > Also probably RUNTIME_RX_QUEUE_SETUP sounds a bit better. >=20 > Agree >=20 > Thanks > Qi >=20 > > > > Konstantin > > > > > /* > > > * If new Tx offload capabilities are defined, they also must be > > > * mentioned in rte_tx_offload_names in rte_ethdev.c file. > > > @@ -1029,6 +1038,8 @@ struct rte_eth_dev_info { > > > /** Configured number of rx/tx queues */ > > > uint16_t nb_rx_queues; /**< Number of RX queues. */ > > > uint16_t nb_tx_queues; /**< Number of TX queues. */ > > > + uint64_t deferred_queue_config_capa; > > > + /**< queues can be setup/release after dev_start (DEV_DEFERRED_). *= / > > > }; > > > > > > /** > > > -- > > > 2.13.6