From: Jiawen Wu <jiawenwu@trustnetic.com>
To: dev@dpdk.org
Cc: Jiawen Wu <jiawenwu@trustnetic.com>, stable@dpdk.org
Subject: [PATCH 1/2] net/txgbe: add proper memory barriers in Rx
Date: Mon, 30 Oct 2023 18:51:43 +0800 [thread overview]
Message-ID: <20231030105144.595502-1-jiawenwu@trustnetic.com> (raw)
Refer to commit 85e46c532bc7 ("net/ixgbe: add proper memory barriers in
Rx"). Fix the same issue as ixgbe.
Segmentation fault has been observed while running the
txgbe_recv_pkts_lro() function to receive packets on the Loongson 3A5000
processor. It's caused by the out-of-order execution of CPU. So add a
proper memory barrier to ensure the read ordering be correct.
We also did the same thing in the txgbe_recv_pkts() function to make the
rxd data be valid even though we did not find segmentation fault in this
function.
Fixes: 0e484278c85f ("net/txgbe: support Rx")
Cc: stable@dpdk.org
Signed-off-by: Jiawen Wu <jiawenwu@trustnetic.com>
---
drivers/net/txgbe/txgbe_rxtx.c | 47 +++++++++++++++-------------------
1 file changed, 21 insertions(+), 26 deletions(-)
diff --git a/drivers/net/txgbe/txgbe_rxtx.c b/drivers/net/txgbe/txgbe_rxtx.c
index 834ada886a..24fc34d3c4 100644
--- a/drivers/net/txgbe/txgbe_rxtx.c
+++ b/drivers/net/txgbe/txgbe_rxtx.c
@@ -1476,11 +1476,22 @@ txgbe_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
* of accesses cannot be reordered by the compiler. If they were
* not volatile, they could be reordered which could lead to
* using invalid descriptor fields when read from rxd.
+ *
+ * Meanwhile, to prevent the CPU from executing out of order, we
+ * need to use a proper memory barrier to ensure the memory
+ * ordering below.
*/
rxdp = &rx_ring[rx_id];
staterr = rxdp->qw1.lo.status;
if (!(staterr & rte_cpu_to_le_32(TXGBE_RXD_STAT_DD)))
break;
+
+ /*
+ * Use acquire fence to ensure that status_error which includes
+ * DD bit is loaded before loading of other descriptor words.
+ */
+ rte_atomic_thread_fence(__ATOMIC_ACQUIRE);
+
rxd = *rxdp;
/*
@@ -1726,32 +1737,10 @@ txgbe_recv_pkts_lro(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts,
next_desc:
/*
- * The code in this whole file uses the volatile pointer to
- * ensure the read ordering of the status and the rest of the
- * descriptor fields (on the compiler level only!!!). This is so
- * UGLY - why not to just use the compiler barrier instead? DPDK
- * even has the rte_compiler_barrier() for that.
- *
- * But most importantly this is just wrong because this doesn't
- * ensure memory ordering in a general case at all. For
- * instance, DPDK is supposed to work on Power CPUs where
- * compiler barrier may just not be enough!
- *
- * I tried to write only this function properly to have a
- * starting point (as a part of an LRO/RSC series) but the
- * compiler cursed at me when I tried to cast away the
- * "volatile" from rx_ring (yes, it's volatile too!!!). So, I'm
- * keeping it the way it is for now.
- *
- * The code in this file is broken in so many other places and
- * will just not work on a big endian CPU anyway therefore the
- * lines below will have to be revisited together with the rest
- * of the txgbe PMD.
- *
- * TODO:
- * - Get rid of "volatile" and let the compiler do its job.
- * - Use the proper memory barrier (rte_rmb()) to ensure the
- * memory ordering below.
+ * "Volatile" only prevents caching of the variable marked
+ * volatile. Most important, "volatile" cannot prevent the CPU
+ * from executing out of order. So, it is necessary to use a
+ * proper memory barrier to ensure the memory ordering below.
*/
rxdp = &rx_ring[rx_id];
staterr = rte_le_to_cpu_32(rxdp->qw1.lo.status);
@@ -1759,6 +1748,12 @@ txgbe_recv_pkts_lro(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts,
if (!(staterr & TXGBE_RXD_STAT_DD))
break;
+ /*
+ * Use acquire fence to ensure that status_error which includes
+ * DD bit is loaded before loading of other descriptor words.
+ */
+ rte_atomic_thread_fence(__ATOMIC_ACQUIRE);
+
rxd = *rxdp;
PMD_RX_LOG(DEBUG, "port_id=%u queue_id=%u rx_id=%u "
--
2.27.0
next reply other threads:[~2023-10-30 10:40 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-10-30 10:51 Jiawen Wu [this message]
2023-10-30 10:51 ` [PATCH 2/2] net/ngbe: " Jiawen Wu
2023-10-31 12:17 ` [PATCH 1/2] net/txgbe: " Ferruh Yigit
2023-11-01 3:32 ` [PATCH v2 " Jiawen Wu
2023-11-01 3:32 ` [PATCH v2 2/2] net/ngbe: " Jiawen Wu
2023-11-01 16:55 ` [PATCH v2 1/2] net/txgbe: " Ferruh Yigit
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20231030105144.595502-1-jiawenwu@trustnetic.com \
--to=jiawenwu@trustnetic.com \
--cc=dev@dpdk.org \
--cc=stable@dpdk.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).