From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-f44.google.com (mail-wm0-f44.google.com [74.125.82.44]) by dpdk.org (Postfix) with ESMTP id C977136E for ; Fri, 24 Mar 2017 13:46:37 +0100 (CET) Received: by mail-wm0-f44.google.com with SMTP id u132so12153658wmg.0 for ; Fri, 24 Mar 2017 05:46:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=oTVLqHGLIotK29K+r4KEDLDYEVFmVcD/nafMvBmYlrQ=; b=yancHgzWKfW+qGtMA+er+uvCyoOzqi0qAifzZVaGVdzxWy2Tzl0UCluHQZ9wjDoAkZ 1gCcMl1YNHO7J9mBtNsyQe/lOlOvPes2O4h1U7MeAPzamLr/Kf9ZQiUo9amRZdEhAe51 YNmQjeqj/YnmYCBjGkX9GCj3dcLosTRblRfsYfRZcUl1pBEkB6/+saNbJK/0RTP3utvc DxApcaBPZJn7IeByPhCXPq5a8ZyW6Agu+pGuArZM/ySP4SRT+w8W6Jrjn7vvDU8mfYoD QHrHEk8IAhVjjK52raUrUc/JjD7DtLurg3+6wdXeF6oTT8oCtonLmMS1Y18PX2plavZ9 UpjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=oTVLqHGLIotK29K+r4KEDLDYEVFmVcD/nafMvBmYlrQ=; b=PzhccM6CvUnFonpbkqfo5pNucOzvGKU2KkYHhRJl6DDRPZRmqFXU86hTc6D6Nc9qYl wIMTfrr7TqCtdjFCqn0HqtVHt4eLWDDYZjL5BjsNtLzdEtj3yZM9K+qk0b++NpSW+xkL NKF18bqnHICNs+ME3AZf05U9qceY0FISoSm9+HFO8DKstWw3Pir2sHxnrrlp7mYzu2dP b0HXRIOyL0BgH9Fvz7QpeY9bNeqVOapI8FzCbUbWdDzqpKrSFEFlRafn1i7/A/Cr0pWr VQ06hCunPG9q5Nd6oRLBIC6x/Xd9NzfbrbTCHddQwPQxbFwcxflClYwMVpznZqWZNI6S GIbQ== X-Gm-Message-State: AFeK/H3RcegfivezutjwqgU/KF6frieR8RUzxSxC+vQ2kb9n1Au64EPLHd3vs5+rCvcxxg4J X-Received: by 10.28.68.69 with SMTP id r66mr2748070wma.115.1490359597412; Fri, 24 Mar 2017 05:46:37 -0700 (PDT) Received: from platinum (2a01cb0c03c651000226b0fffeed02fc.ipv6.abo.wanadoo.fr. [2a01:cb0c:3c6:5100:226:b0ff:feed:2fc]) by smtp.gmail.com with ESMTPSA id l21sm2754666wrl.59.2017.03.24.05.46.37 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 24 Mar 2017 05:46:37 -0700 (PDT) Date: Fri, 24 Mar 2017 13:46:34 +0100 From: Olivier Matz To: Billy McFall Cc: thomas.monjalon@6wind.com, wenzhuo.lu@intel.com, dev@dpdk.org Message-ID: <20170324134634.3e764423@platinum> In-Reply-To: References: <20170309205119.28170-1-bmcfall@redhat.com> <20170315180226.5999-1-bmcfall@redhat.com> <20170315180226.5999-2-bmcfall@redhat.com> <20170323113716.57e27591@glumotte.dev.6wind.com> X-Mailer: Claws Mail 3.14.1 (GTK+ 2.24.31; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [PATCH v7 1/3] ethdev: new API to free consumed buffers in Tx ring X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Mar 2017 12:46:37 -0000 Hi Billy, On Thu, 23 Mar 2017 09:32:14 -0400, Billy McFall wrote: > Thank you for your comments. See inline. > > On Thu, Mar 23, 2017 at 6:37 AM, Olivier MATZ > wrote: > > > Hi Billy, > > > > On Wed, 15 Mar 2017 14:02:24 -0400, Billy McFall > > wrote: > > > Add a new API to force free consumed buffers on Tx ring. API will return > > > the number of packets freed (0-n) or error code if feature not supported > > > (-ENOTSUP) or input invalid (-ENODEV). > > > > > > Signed-off-by: Billy McFall > > > --- > > > doc/guides/conf.py | 7 +++++-- > > > doc/guides/nics/features/default.ini | 4 +++- > > > doc/guides/prog_guide/poll_mode_drv.rst | 28 > > ++++++++++++++++++++++++++++ > > > doc/guides/rel_notes/release_17_05.rst | 7 ++++++- > > > lib/librte_ether/rte_ethdev.c | 14 ++++++++++++++ > > > lib/librte_ether/rte_ethdev.h | 31 > > +++++++++++++++++++++++++++++++ > > > 6 files changed, 87 insertions(+), 4 deletions(-) > > > > > > > [...] > > > > > --- a/doc/guides/prog_guide/poll_mode_drv.rst > > > +++ b/doc/guides/prog_guide/poll_mode_drv.rst > > > @@ -249,6 +249,34 @@ One descriptor in the TX ring is used as a sentinel > > to avoid a hardware race con > > > > > > When configuring for DCB operation, at port initialization, both > > the number of transmit queues and the number of receive queues must be set > > to 128. > > > > > > +Free Tx mbuf on Demand > > > +~~~~~~~~~~~~~~~~~~~~~~ > > > + > > > +Many of the drivers don't release the mbuf back to the mempool, or > > local cache, immediately after the packet has been > > > +transmitted. > > > +Instead, they leave the mbuf in their Tx ring and either perform a bulk > > release when the ``tx_rs_thresh`` has been > > > +crossed or free the mbuf when a slot in the Tx ring is needed. > > > + > > > +An application can request the driver to release used mbufs with the > > ``rte_eth_tx_done_cleanup()`` API. > > > +This API requests the driver to release mbufs that are no longer in > > use, independent of whether or not the > > > +``tx_rs_thresh`` has been crossed. > > > +There are two scenarios when an application may want the mbuf released > > immediately: > > > + > > > +* When a given packet needs to be sent to multiple destination > > interfaces (either for Layer 2 flooding or Layer 3 > > > + multi-cast). > > > + One option is to make a copy of the packet or a copy of the header > > portion that needs to be manipulated. > > > + A second option is to transmit the packet and then poll the > > ``rte_eth_tx_done_cleanup()`` API until the reference > > > + count on the packet is decremented. > > > + Then the same packet can be transmitted to the next destination > > interface. > > > > By reading this paragraph, it's not so clear to me that the packet > > that will be transmitted on all interfaces will be different from > > one port to another. > > > > Maybe it could be reworded to insist on that? > > > > > What if I add the following sentence: > > Then the same packet can be transmitted to the next destination interface. > + The application is still responsible for managing any packet > manipulations needed between the different destination > + interfaces, but a packet copy can be avoided. looks good, thanks. > > > + > > > +* If an application is designed to make multiple runs, like a packet > > generator, and one run has completed. > > > + The application may want to reset to a clean state. > > > > I'd reword into: > > > > Some applications are designed to make multiple runs, like a packet > > generator. > > Between each run, the application may want to reset to a clean state. > > > > What do you mean by "clean state"? All mbufs returned into the mempools? > > Why would a packet generator need that? For performance? > > > > Reworded as you suggested, then attempted to explain a 'clean state'. > Also reworded the last sentence a little. > > + * Some applications are designed to make multiple runs, like a packet > generator. > + For performance reasons and consistency between runs, the application > may want to reset back to an initial state > + between each run, where all mbufs are returned to the mempool. > + In this case, it can call the ``rte_eth_tx_done_cleanup()`` API for > each destination interface it has been using > + to request it to release of all its used mbufs. ok, looks clearer to me, thanks > > Also, do we want to ensure that all packets are actually transmitted? > > > > Added an additional sentence to indicate that this API doesn't manage > whether or not the packet has been transmitted. > > Then the same packet can be transmitted to the next destination interface. > The application is still responsible for managing any packet > manipulations needed between the different destination > interface, but a packet copy can be avoided. > + This API is independent of whether the packet was transmitted or > dropped, only that the mbuf is no longer in use by > + the interface. ok > > Can we do that with this API or should we use another API like > > rte_eth_tx_descriptor_status() [1] ? > > > > [1] http://dpdk.org/dev/patchwork/patch/21549/ > > > > I read through this patch. This API doesn't indicate if the packet was > transmitted or dropped (I think that is what you were asking). This API > could be used by the application to determine if the mbuf has been > freed, as opposed to polling the rte_mbuf_refcnt_read() for a change > in value. Did I miss your point? Maybe my question was not clear :) Let me try to reword it. For a traffic generator use-case, a dummy algorithm may be: 1/ send packets in a loop until a condition is met (ex: packet count reached) 2/ call rte_eth_tx_done_cleanup() 3/ read stats for report I think there is something missing between 1/ and 2/, to ensure that all packets that were in the tx queue are processed (either transmitted or dropped). If that's not the case, both steps 2/ and 3/ will not behave as expected: - all mbufs won't be returned to the pool - statistics may be wrong Maybe a simple wait() could do the job. Using a combination of rte_eth_tx_done_cleanup() + rte_eth_tx_descriptor_status() is probably also a solution. Do you confirm rte_eth_tx_done_cleanup() does not check that? Thanks Olivier