From: Hemant Agrawal <hemant.agrawal@nxp.com>
To: Jerin Jacob <jerin.jacob@caviumnetworks.com>, <dev@dpdk.org>
Cc: <thomas@monjalon.net>, <john.mcnamara@intel.com>
Subject: Re: [dpdk-dev] [PATCH v3 ] ethdev: introduce lock-free txq capability flag
Date: Thu, 13 Jul 2017 17:32:39 +0530 [thread overview]
Message-ID: <ff0cb451-8a6b-efb2-bd60-07fb9c2224ca@nxp.com> (raw)
In-Reply-To: <20170710165946.31080-1-jerin.jacob@caviumnetworks.com>
On 7/10/2017 10:29 PM, Jerin Jacob wrote:
> Introducing the DEV_TX_OFFLOAD_MT_LOCKFREE TX capability flag.
> if a PMD advertises DEV_TX_OFFLOAD_MT_LOCKFREE capable, multiple threads
> can invoke rte_eth_tx_burst() concurrently on the same tx queue without
> SW lock. This PMD feature will be useful in the following use cases and
> found in the OCTEON family of NPUs.
>
> 1) Remove explicit spinlock in some applications where lcores
> to TX queues are not mapped 1:1.
> example: OVS has such instance
> https://github.com/openvswitch/ovs/blob/master/lib/netdev-dpdk.c#L299
> https://github.com/openvswitch/ovs/blob/master/lib/netdev-dpdk.c#L1859
> See the the usage of tx_lock spinlock.
>
> 2) In the eventdev use case, avoid dedicating a separate TX core for
> transmitting and thus enables more scaling as all workers can
> send the packets.
>
> Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> ---
> v3:
> - Addressed Thomas's documentation review comments
> http://dpdk.org/ml/archives/dev/2017-July/070672.html
>
> v2:
> - Changed the flag name to DEV_TX_OFFLOAD_MT_LOCKFREE(Thomas)
> - Updated the documentation in doc/guides/prog_guide/poll_mode_drv.rst
> and rte_eth_tx_burst() doxgen comments(Thomas)
> ---
> doc/guides/prog_guide/poll_mode_drv.rst | 15 +++++++++++++--
> lib/librte_ether/rte_ethdev.h | 8 ++++++++
> 2 files changed, 21 insertions(+), 2 deletions(-)
>
> diff --git a/doc/guides/prog_guide/poll_mode_drv.rst b/doc/guides/prog_guide/poll_mode_drv.rst
> index 4987f70a1..1ac8f7ebf 100644
> --- a/doc/guides/prog_guide/poll_mode_drv.rst
> +++ b/doc/guides/prog_guide/poll_mode_drv.rst
> @@ -84,7 +84,7 @@ Whenever needed and appropriate, asynchronous communication should be introduced
>
> Avoiding lock contention is a key issue in a multi-core environment.
> To address this issue, PMDs are designed to work with per-core private resources as much as possible.
> -For example, a PMD maintains a separate transmit queue per-core, per-port.
> +For example, a PMD maintains a separate transmit queue per-core, per-port, if the PMD is not ``DEV_TX_OFFLOAD_MT_LOCKFREE`` capable.
> In the same way, every receive queue of a port is assigned to and polled by a single logical core (lcore).
>
> To comply with Non-Uniform Memory Access (NUMA), memory management is designed to assign to each logical core
> @@ -146,6 +146,16 @@ This is also true for the pipe-line model provided all logical cores used are lo
>
> Multiple logical cores should never share receive or transmit queues for interfaces since this would require global locks and hinder performance.
>
> +If the PMD is ``DEV_TX_OFFLOAD_MT_LOCKFREE`` capable, multiple threads can invoke ``rte_eth_tx_burst()``
> +concurrently on the same tx queue without SW lock. This PMD feature found in some NICs and useful in the following use cases:
> +
> +* Remove explicit spinlock in some applications where lcores are not mapped to Tx queues with 1:1 relation.
> +
> +* In the eventdev use case, avoid dedicating a separate TX core for transmitting and thus
> + enables more scaling as all workers can send the packets.
> +
> +See `Hardware Offload`_ for ``DEV_TX_OFFLOAD_MT_LOCKFREE`` capability probing details.
> +
> Device Identification and Configuration
> ---------------------------------------
>
> @@ -290,7 +300,8 @@ Hardware Offload
>
> Depending on driver capabilities advertised by
> ``rte_eth_dev_info_get()``, the PMD may support hardware offloading
> -feature like checksumming, TCP segmentation or VLAN insertion.
> +feature like checksumming, TCP segmentation, VLAN insertion or
> +lockfree multithreaded TX burst on the same TX queue.
>
> The support of these offload features implies the addition of dedicated
> status bit(s) and value field(s) into the rte_mbuf data structure, along
> diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
> index fd6baf37a..11fe13a07 100644
> --- a/lib/librte_ether/rte_ethdev.h
> +++ b/lib/librte_ether/rte_ethdev.h
> @@ -927,6 +927,10 @@ struct rte_eth_conf {
> #define DEV_TX_OFFLOAD_IPIP_TNL_TSO 0x00000800 /**< Used for tunneling packet. */
> #define DEV_TX_OFFLOAD_GENEVE_TNL_TSO 0x00001000 /**< Used for tunneling packet. */
> #define DEV_TX_OFFLOAD_MACSEC_INSERT 0x00002000
> +#define DEV_TX_OFFLOAD_MT_LOCKFREE 0x00004000
> +/**< Multiple threads can invoke rte_eth_tx_burst() concurrently on the same
> + * tx queue without SW lock.
> + */
>
> struct rte_pci_device;
>
> @@ -2961,6 +2965,10 @@ static inline int rte_eth_tx_descriptor_status(uint8_t port_id,
> * rte_eth_tx_burst() function must [attempt to] free the *rte_mbuf* buffers
> * of those packets whose transmission was effectively completed.
> *
> + * If the PMD is DEV_TX_OFFLOAD_MT_LOCKFREE capable, multiple threads can
> + * invoke this function concurrently on the same tx queue without SW lock.
> + * @see rte_eth_dev_info_get, struct rte_eth_txconf::txq_flags
> + *
> * @param port_id
> * The port identifier of the Ethernet device.
> * @param queue_id
>
You may also like to add this capability in "doc/guides/nics/features/*.ini"
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>
next prev parent reply other threads:[~2017-07-13 12:02 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-04-21 12:22 [dpdk-dev] [PATCH] ethdev: add lock-less " Jerin Jacob
2017-04-24 12:34 ` Thomas Monjalon
2017-04-27 11:00 ` Jerin Jacob
2017-07-05 17:46 ` Thomas Monjalon
2017-07-06 6:00 ` Jerin Jacob
2017-07-06 6:21 ` [dpdk-dev] [PATCH v2] ethdev: introduce lock-free " Jerin Jacob
2017-07-08 16:08 ` Thomas Monjalon
2017-07-10 16:56 ` Jerin Jacob
2017-07-10 16:59 ` [dpdk-dev] [PATCH v3 ] " Jerin Jacob
2017-07-13 12:02 ` Hemant Agrawal [this message]
2017-07-18 13:43 ` Thomas Monjalon
2017-07-13 18:42 ` santosh
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ff0cb451-8a6b-efb2-bd60-07fb9c2224ca@nxp.com \
--to=hemant.agrawal@nxp.com \
--cc=dev@dpdk.org \
--cc=jerin.jacob@caviumnetworks.com \
--cc=john.mcnamara@intel.com \
--cc=thomas@monjalon.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).