patches for DPDK stable branches
 help / color / mirror / Atom feed
From: Kevin Traynor <ktraynor@redhat.com>
To: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
Cc: Ola Liljedahl <ola.liljedahl@arm.com>,
	Phil Yang <phil.yang@arm.com>,
	Konstantin Ananyev <konstantin.ananyev@intel.com>,
	dpdk stable <stable@dpdk.org>
Subject: [dpdk-stable] patch 'ethdev: fix memory ordering for callback functions' has been queued to LTS release 18.11.11
Date: Wed, 18 Nov 2020 16:35:03 +0000	[thread overview]
Message-ID: <20201118163558.1101823-17-ktraynor@redhat.com> (raw)
In-Reply-To: <20201118163558.1101823-1-ktraynor@redhat.com>

Hi,

FYI, your patch has been queued to LTS release 18.11.11

Note it hasn't been pushed to http://dpdk.org/browse/dpdk-stable yet.
It will be pushed if I get no objections before 11/24/20. So please
shout if anyone has objections.

Also note that after the patch there's a diff of the upstream commit vs the
patch applied to the branch. This will indicate if there was any rebasing
needed to apply to the stable branch. If there were code changes for rebasing
(ie: not only metadata diffs), please double check that the rebase was
correctly done.

Queued patches are on a temporary branch at:
https://github.com/kevintraynor/dpdk-stable-queue

This queued commit can be viewed at:
https://github.com/kevintraynor/dpdk-stable-queue/commit/79711c90de1314b9b012da0815db2e8ac57f14ed

Thanks.

Kevin.

---
From 79711c90de1314b9b012da0815db2e8ac57f14ed Mon Sep 17 00:00:00 2001
From: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
Date: Tue, 13 Oct 2020 11:25:37 -0500
Subject: [PATCH] ethdev: fix memory ordering for callback functions

[ upstream commit 2b69bd1179162f69c0a90d5b09309498d362b3d7 ]

Call back functions are registered on the control plane. They
are accessed from the data plane. Hence, correct memory orderings
should be used to avoid race conditions.

Fixes: 4dc294158cac ("ethdev: support optional Rx and Tx callbacks")
Fixes: c8231c63ddcb ("ethdev: insert Rx callback as head of list")

Signed-off-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
Reviewed-by: Ola Liljedahl <ola.liljedahl@arm.com>
Reviewed-by: Phil Yang <phil.yang@arm.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
---
 lib/librte_ethdev/rte_ethdev.c | 28 ++++++++++++++++++-----
 lib/librte_ethdev/rte_ethdev.h | 42 ++++++++++++++++++++++++++--------
 2 files changed, 55 insertions(+), 15 deletions(-)

diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index 336ad1b83a..3f3f9505b2 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -3825,10 +3825,18 @@ rte_eth_add_rx_callback(uint16_t port_id, uint16_t queue_id,
 
 	if (!tail) {
-		rte_eth_devices[port_id].post_rx_burst_cbs[queue_id] = cb;
+		/* Stores to cb->fn and cb->param should complete before
+		 * cb is visible to data plane.
+		 */
+		__atomic_store_n(
+			&rte_eth_devices[port_id].post_rx_burst_cbs[queue_id],
+			cb, __ATOMIC_RELEASE);
 
 	} else {
 		while (tail->next)
 			tail = tail->next;
-		tail->next = cb;
+		/* Stores to cb->fn and cb->param should complete before
+		 * cb is visible to data plane.
+		 */
+		__atomic_store_n(&tail->next, cb, __ATOMIC_RELEASE);
 	}
 	rte_spinlock_unlock(&rte_eth_rx_cb_lock);
@@ -3903,10 +3911,18 @@ rte_eth_add_tx_callback(uint16_t port_id, uint16_t queue_id,
 
 	if (!tail) {
-		rte_eth_devices[port_id].pre_tx_burst_cbs[queue_id] = cb;
+		/* Stores to cb->fn and cb->param should complete before
+		 * cb is visible to data plane.
+		 */
+		__atomic_store_n(
+			&rte_eth_devices[port_id].pre_tx_burst_cbs[queue_id],
+			cb, __ATOMIC_RELEASE);
 
 	} else {
 		while (tail->next)
 			tail = tail->next;
-		tail->next = cb;
+		/* Stores to cb->fn and cb->param should complete before
+		 * cb is visible to data plane.
+		 */
+		__atomic_store_n(&tail->next, cb, __ATOMIC_RELEASE);
 	}
 	rte_spinlock_unlock(&rte_eth_tx_cb_lock);
@@ -3939,5 +3955,5 @@ rte_eth_remove_rx_callback(uint16_t port_id, uint16_t queue_id,
 		if (cb == user_cb) {
 			/* Remove the user cb from the callback list. */
-			*prev_cb = cb->next;
+			__atomic_store_n(prev_cb, cb->next, __ATOMIC_RELAXED);
 			ret = 0;
 			break;
@@ -3973,5 +3989,5 @@ rte_eth_remove_tx_callback(uint16_t port_id, uint16_t queue_id,
 		if (cb == user_cb) {
 			/* Remove the user cb from the callback list. */
-			*prev_cb = cb->next;
+			__atomic_store_n(prev_cb, cb->next, __ATOMIC_RELAXED);
 			ret = 0;
 			break;
diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
index 7b4221d7b4..e1f299c993 100644
--- a/lib/librte_ethdev/rte_ethdev.h
+++ b/lib/librte_ethdev/rte_ethdev.h
@@ -3216,5 +3216,6 @@ struct rte_eth_rxtx_callback;
  * @param user_param
  *   A generic pointer parameter which will be passed to each invocation of the
- *   callback function on this port and queue.
+ *   callback function on this port and queue. Inter-thread synchronization
+ *   of any user data changes is the responsibility of the user.
  *
  * @return
@@ -3245,5 +3246,6 @@ rte_eth_add_rx_callback(uint16_t port_id, uint16_t queue_id,
  * @param user_param
  *   A generic pointer parameter which will be passed to each invocation of the
- *   callback function on this port and queue.
+ *   callback function on this port and queue. Inter-thread synchronization
+ *   of any user data changes is the responsibility of the user.
  *
  * @return
@@ -3273,5 +3275,6 @@ rte_eth_add_first_rx_callback(uint16_t port_id, uint16_t queue_id,
  * @param user_param
  *   A generic pointer parameter which will be passed to each invocation of the
- *   callback function on this port and queue.
+ *   callback function on this port and queue. Inter-thread synchronization
+ *   of any user data changes is the responsibility of the user.
  *
  * @return
@@ -3298,5 +3301,7 @@ rte_eth_add_tx_callback(uint16_t port_id, uint16_t queue_id,
  *
  * - After a short delay - where the delay is sufficient to allow any
- *   in-flight callbacks to complete.
+ *   in-flight callbacks to complete. Alternately, the RCU mechanism can be
+ *   used to detect when data plane threads have ceased referencing the
+ *   callback memory.
  *
  * @param port_id
@@ -3331,5 +3336,7 @@ int rte_eth_remove_rx_callback(uint16_t port_id, uint16_t queue_id,
  *
  * - After a short delay - where the delay is sufficient to allow any
- *   in-flight callbacks to complete.
+ *   in-flight callbacks to complete. Alternately, the RCU mechanism can be
+ *   used to detect when data plane threads have ceased referencing the
+ *   callback memory.
  *
  * @param port_id
@@ -3881,8 +3888,16 @@ rte_eth_rx_burst(uint16_t port_id, uint16_t queue_id,
 
 #ifdef RTE_ETHDEV_RXTX_CALLBACKS
-	if (unlikely(dev->post_rx_burst_cbs[queue_id] != NULL)) {
-		struct rte_eth_rxtx_callback *cb =
-				dev->post_rx_burst_cbs[queue_id];
+	struct rte_eth_rxtx_callback *cb;
 
+	/* __ATOMIC_RELEASE memory order was used when the
+	 * call back was inserted into the list.
+	 * Since there is a clear dependency between loading
+	 * cb and cb->fn/cb->next, __ATOMIC_ACQUIRE memory order is
+	 * not required.
+	 */
+	cb = __atomic_load_n(&dev->post_rx_burst_cbs[queue_id],
+				__ATOMIC_RELAXED);
+
+	if (unlikely(cb != NULL)) {
 		do {
 			nb_rx = cb->fn.rx(port_id, queue_id, rx_pkts, nb_rx,
@@ -4145,5 +4160,14 @@ rte_eth_tx_burst(uint16_t port_id, uint16_t queue_id,
 
 #ifdef RTE_ETHDEV_RXTX_CALLBACKS
-	struct rte_eth_rxtx_callback *cb = dev->pre_tx_burst_cbs[queue_id];
+	struct rte_eth_rxtx_callback *cb;
+
+	/* __ATOMIC_RELEASE memory order was used when the
+	 * call back was inserted into the list.
+	 * Since there is a clear dependency between loading
+	 * cb and cb->fn/cb->next, __ATOMIC_ACQUIRE memory order is
+	 * not required.
+	 */
+	cb = __atomic_load_n(&dev->pre_tx_burst_cbs[queue_id],
+				__ATOMIC_RELAXED);
 
 	if (unlikely(cb != NULL)) {
-- 
2.26.2

---
  Diff of the applied patch vs upstream commit (please double-check if non-empty:
---
--- -	2020-11-18 16:33:38.296348094 +0000
+++ 0017-ethdev-fix-memory-ordering-for-callback-functions.patch	2020-11-18 16:33:37.924215059 +0000
@@ -1 +1 @@
-From 2b69bd1179162f69c0a90d5b09309498d362b3d7 Mon Sep 17 00:00:00 2001
+From 79711c90de1314b9b012da0815db2e8ac57f14ed Mon Sep 17 00:00:00 2001
@@ -5,0 +6,2 @@
+[ upstream commit 2b69bd1179162f69c0a90d5b09309498d362b3d7 ]
+
@@ -12 +13,0 @@
-Cc: stable@dpdk.org
@@ -24 +25 @@
-index 2198548663..7eaeccc19d 100644
+index 336ad1b83a..3f3f9505b2 100644
@@ -27 +28 @@
-@@ -4611,10 +4611,18 @@ rte_eth_add_rx_callback(uint16_t port_id, uint16_t queue_id,
+@@ -3825,10 +3825,18 @@ rte_eth_add_rx_callback(uint16_t port_id, uint16_t queue_id,
@@ -48 +49 @@
-@@ -4701,10 +4709,18 @@ rte_eth_add_tx_callback(uint16_t port_id, uint16_t queue_id,
+@@ -3903,10 +3911,18 @@ rte_eth_add_tx_callback(uint16_t port_id, uint16_t queue_id,
@@ -69 +70 @@
-@@ -4737,5 +4753,5 @@ rte_eth_remove_rx_callback(uint16_t port_id, uint16_t queue_id,
+@@ -3939,5 +3955,5 @@ rte_eth_remove_rx_callback(uint16_t port_id, uint16_t queue_id,
@@ -76 +77 @@
-@@ -4771,5 +4787,5 @@ rte_eth_remove_tx_callback(uint16_t port_id, uint16_t queue_id,
+@@ -3973,5 +3989,5 @@ rte_eth_remove_tx_callback(uint16_t port_id, uint16_t queue_id,
@@ -84 +85 @@
-index f07dbc4d26..c37b3a1a81 100644
+index 7b4221d7b4..e1f299c993 100644
@@ -87 +88 @@
-@@ -3912,5 +3912,6 @@ struct rte_eth_rxtx_callback;
+@@ -3216,5 +3216,6 @@ struct rte_eth_rxtx_callback;
@@ -95 +96 @@
-@@ -3941,5 +3942,6 @@ rte_eth_add_rx_callback(uint16_t port_id, uint16_t queue_id,
+@@ -3245,5 +3246,6 @@ rte_eth_add_rx_callback(uint16_t port_id, uint16_t queue_id,
@@ -103 +104 @@
-@@ -3969,5 +3971,6 @@ rte_eth_add_first_rx_callback(uint16_t port_id, uint16_t queue_id,
+@@ -3273,5 +3275,6 @@ rte_eth_add_first_rx_callback(uint16_t port_id, uint16_t queue_id,
@@ -111 +112 @@
-@@ -3994,5 +3997,7 @@ rte_eth_add_tx_callback(uint16_t port_id, uint16_t queue_id,
+@@ -3298,5 +3301,7 @@ rte_eth_add_tx_callback(uint16_t port_id, uint16_t queue_id,
@@ -120 +121 @@
-@@ -4027,5 +4032,7 @@ int rte_eth_remove_rx_callback(uint16_t port_id, uint16_t queue_id,
+@@ -3331,5 +3336,7 @@ int rte_eth_remove_rx_callback(uint16_t port_id, uint16_t queue_id,
@@ -129 +130 @@
-@@ -4688,8 +4695,16 @@ rte_eth_rx_burst(uint16_t port_id, uint16_t queue_id,
+@@ -3881,8 +3888,16 @@ rte_eth_rx_burst(uint16_t port_id, uint16_t queue_id,
@@ -149 +150 @@
-@@ -4954,5 +4969,14 @@ rte_eth_tx_burst(uint16_t port_id, uint16_t queue_id,
+@@ -4145,5 +4160,14 @@ rte_eth_tx_burst(uint16_t port_id, uint16_t queue_id,


  parent reply	other threads:[~2020-11-18 16:37 UTC|newest]

Thread overview: 72+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-11-18 16:34 [dpdk-stable] patch 'net/netvsc: check for overflow on packet info from host' " Kevin Traynor
2020-11-18 16:34 ` [dpdk-stable] patch 'net/netvsc: replace compiler builtin overflow check' " Kevin Traynor
2020-11-18 16:34 ` [dpdk-stable] patch 'eventdev: check allocation in Tx adapter' " Kevin Traynor
2020-11-18 16:34 ` [dpdk-stable] patch 'event/dpaa2: fix dereference before null check' " Kevin Traynor
2020-11-18 16:34 ` [dpdk-stable] patch 'eventdev: fix adapter leak in error path' " Kevin Traynor
2020-11-18 16:34 ` [dpdk-stable] patch 'test/event: fix function arguments for crypto adapter' " Kevin Traynor
2020-11-18 16:34 ` [dpdk-stable] patch 'eal/x86: fix memcpy AVX-512 enablement' " Kevin Traynor
2020-11-18 16:34 ` [dpdk-stable] patch 'vhost: fix virtio-net header length with packed ring' " Kevin Traynor
2020-11-18 16:34 ` [dpdk-stable] patch 'net/netvsc: fix Tx queue leak in error path' " Kevin Traynor
2020-11-18 16:34 ` [dpdk-stable] patch 'net/bonding: fix possible unbalanced packet receiving' " Kevin Traynor
2020-11-18 16:34 ` [dpdk-stable] patch 'net/bonding: fix Rx queue conversion' " Kevin Traynor
2020-11-18 16:34 ` [dpdk-stable] patch 'net/failsafe: fix state synchro cleanup' " Kevin Traynor
2020-11-18 16:34 ` [dpdk-stable] patch 'net/ring: check internal arguments' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'doc: fix EF10 Rx mode name in sfc guide' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'doc: fix typo in pcap " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'net/bnx2x: add QLogic vendor id for BCM57840' " Kevin Traynor
2020-11-18 16:35 ` Kevin Traynor [this message]
2020-11-18 16:35 ` [dpdk-stable] patch 'distributor: fix handshake synchronization' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'distributor: fix handshake deadlock' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'distributor: fix buffer use after free' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'distributor: handle worker shutdown in burst mode' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'test/distributor: fix shutdown of busy worker' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'distributor: fix return pkt calls in single mode' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'test/distributor: fix freeing mbufs' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'test/distributor: fix lcores statistics' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'test/distributor: collect return mbufs' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'distributor: fix API documentation' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'test/distributor: fix race conditions on shutdown' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'distributor: fix scalar matching' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'distributor: fix flushing in flight packets' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'distributor: fix clearing returns buffer' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'test/distributor: fix quitting workers in burst mode' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'test/distributor: fix mbuf leak on failure' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'doc: add SPDX license tag header to meson guide' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'app: fix missing dependencies' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'examples/l2fwd-crypto: fix missing dependency' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'bus/pci: remove unused scan by address' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'eal/linux: fix memory leak in uevent handling' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'efd: fix tailq entry leak in error path' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'eal: fix leak on device event callback unregister' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'mem: fix config name in error logs' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'examples/vm_power: fix build on Ubuntu 20.04' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'examples/multi_process: " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'examples/performance-thread: fix build with low core count' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'baseband/turbo_sw: fix memory leak in error path' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'examples/fips_validation: fix missed version line' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'crypto/dpaa2_sec: remove dead code' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'common/qat: add missing kmod dependency info' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'app/testpmd: fix bonding xmit balance policy command' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'net/i40e: fix virtual channel conflict' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'net/ixgbe: check switch domain allocation result' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'net/i40e: fix QinQ flow pattern to allow non full mask' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'net/ixgbe: fix vector Rx' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'net/i40e: " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'net/fm10k: " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'net/vhost: fix xstats after clearing stats' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'vhost: fix virtqueues metadata allocation' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'vhost: validate index in available entries API' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'vhost: validate index in guest notification " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'net/ena: remove unused macro' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'net/mvpp2: fix memory leak in error path' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'net/netvsc: allocate contiguous physical memory for RNDIS' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'app/testpmd: fix RSS key for flow API RSS rule' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'net/i40e: fix flow director for eth + VLAN pattern' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'net/bnxt: fix resetting mbuf data offset' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'net/vdev_netvsc: fix device probing error flow' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'net/thunderx: fix memory leak on rbdr desc ring failure' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'app/testpmd: do not allow dynamic change of core number' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'ethdev: fix data type for port id' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'fix spellings that Lintian complains about' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'doc: fix typo in KNI guide' " Kevin Traynor
2020-11-18 16:35 ` [dpdk-stable] patch 'ethdev: move non-offload capabilities' " Kevin Traynor

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20201118163558.1101823-17-ktraynor@redhat.com \
    --to=ktraynor@redhat.com \
    --cc=honnappa.nagarahalli@arm.com \
    --cc=konstantin.ananyev@intel.com \
    --cc=ola.liljedahl@arm.com \
    --cc=phil.yang@arm.com \
    --cc=stable@dpdk.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).