DPDK patches and discussions
 help / color / mirror / Atom feed
From: Rasesh Mody <rmody@marvell.com>
To: <dev@dpdk.org>, <jerinj@marvell.com>, <ferruh.yigit@intel.com>
Cc: Rasesh Mody <rmody@marvell.com>,
	<GR-Everest-DPDK-Dev@marvell.com>, <stable@dpdk.org>
Subject: [dpdk-dev] [PATCH 3/3] net/bnx2x: fix to sync fastpath Rx queue access
Date: Mon, 13 Jan 2020 16:39:20 -0800	[thread overview]
Message-ID: <20200114003920.17705-3-rmody@marvell.com> (raw)
In-Reply-To: <20200114003920.17705-1-rmody@marvell.com>

PMD handles fast path completions in the Rx handler and control path
completions in the interrupt handler. They both are processing
completions from the same fastpath completion queue. There is a
potential for race condition when these two paths are processing
the completions from the same queue and trying to updating Rx Producer.

Add a fastpath Rx lock between these two paths to close this race.

Fixes: 540a211084a7 ("bnx2x: driver core")
Cc: stable@dpdk.org

Signed-off-by: Rasesh Mody <rmody@marvell.com>
---
 drivers/net/bnx2x/bnx2x.c      | 12 ++++++++++++
 drivers/net/bnx2x/bnx2x.h      |  3 +++
 drivers/net/bnx2x/bnx2x_rxtx.c |  8 +++++++-
 3 files changed, 22 insertions(+), 1 deletion(-)

diff --git a/drivers/net/bnx2x/bnx2x.c b/drivers/net/bnx2x/bnx2x.c
index d38da4f60..7ea98b936 100644
--- a/drivers/net/bnx2x/bnx2x.c
+++ b/drivers/net/bnx2x/bnx2x.c
@@ -1167,6 +1167,10 @@ static int bnx2x_has_rx_work(struct bnx2x_fastpath *fp)
 	if (unlikely((rx_cq_cons_sb & MAX_RCQ_ENTRIES(rxq)) ==
 		     MAX_RCQ_ENTRIES(rxq)))
 		rx_cq_cons_sb++;
+
+	PMD_RX_LOG(DEBUG, "hw CQ cons = %d, sw CQ cons = %d",
+		   rx_cq_cons_sb, rxq->rx_cq_head);
+
 	return rxq->rx_cq_head != rx_cq_cons_sb;
 }
 
@@ -1249,9 +1253,12 @@ static uint8_t bnx2x_rxeof(struct bnx2x_softc *sc, struct bnx2x_fastpath *fp)
 	uint16_t bd_cons, bd_prod, bd_prod_fw, comp_ring_cons;
 	uint16_t hw_cq_cons, sw_cq_cons, sw_cq_prod;
 
+	rte_spinlock_lock(&(fp)->rx_mtx);
+
 	rxq = sc->rx_queues[fp->index];
 	if (!rxq) {
 		PMD_RX_LOG(ERR, "RX queue %d is NULL", fp->index);
+		rte_spinlock_unlock(&(fp)->rx_mtx);
 		return 0;
 	}
 
@@ -1326,9 +1333,14 @@ static uint8_t bnx2x_rxeof(struct bnx2x_softc *sc, struct bnx2x_fastpath *fp)
 	rxq->rx_cq_head = sw_cq_cons;
 	rxq->rx_cq_tail = sw_cq_prod;
 
+	PMD_RX_LOG(DEBUG, "BD prod = %d, sw CQ prod = %d",
+		   bd_prod_fw, sw_cq_prod);
+
 	/* Update producers */
 	bnx2x_update_rx_prod(sc, fp, bd_prod_fw, sw_cq_prod);
 
+	rte_spinlock_unlock(&(fp)->rx_mtx);
+
 	return sw_cq_cons != hw_cq_cons;
 }
 
diff --git a/drivers/net/bnx2x/bnx2x.h b/drivers/net/bnx2x/bnx2x.h
index 3383c7675..1dbc98197 100644
--- a/drivers/net/bnx2x/bnx2x.h
+++ b/drivers/net/bnx2x/bnx2x.h
@@ -360,6 +360,9 @@ struct bnx2x_fastpath {
 	/* pointer back to parent structure */
 	struct bnx2x_softc *sc;
 
+	/* Used to synchronize fastpath Rx access */
+	rte_spinlock_t rx_mtx;
+
 	/* status block */
 	struct bnx2x_dma                 sb_dma;
 	union bnx2x_host_hc_status_block status_block;
diff --git a/drivers/net/bnx2x/bnx2x_rxtx.c b/drivers/net/bnx2x/bnx2x_rxtx.c
index b52f023ea..c8bb202d6 100644
--- a/drivers/net/bnx2x/bnx2x_rxtx.c
+++ b/drivers/net/bnx2x/bnx2x_rxtx.c
@@ -357,6 +357,8 @@ bnx2x_recv_pkts(void *p_rxq, struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
 	uint16_t len, pad;
 	struct rte_mbuf *rx_mb = NULL;
 
+	rte_spinlock_lock(&(fp)->rx_mtx);
+
 	/* Add memory barrier as status block fields can change. This memory
 	 * barrier will flush out all the read/write operations to status block
 	 * generated before the barrier. It will ensure stale data is not read
@@ -379,8 +381,10 @@ bnx2x_recv_pkts(void *p_rxq, struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
 	 */
 	rmb();
 
-	if (sw_cq_cons == hw_cq_cons)
+	if (sw_cq_cons == hw_cq_cons) {
+		rte_spinlock_unlock(&(fp)->rx_mtx);
 		return 0;
+	}
 
 	while (nb_rx < nb_pkts && sw_cq_cons != hw_cq_cons) {
 
@@ -461,6 +465,8 @@ bnx2x_recv_pkts(void *p_rxq, struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
 
 	bnx2x_upd_rx_prod_fast(sc, fp, bd_prod, sw_cq_prod);
 
+	rte_spinlock_unlock(&(fp)->rx_mtx);
+
 	return nb_rx;
 }
 
-- 
2.18.0


  parent reply	other threads:[~2020-01-14  0:39 UTC|newest]

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-01-14  0:39 [dpdk-dev] [PATCH 1/3] net/bnx2x: fix to use required mem barriers in Rx path Rasesh Mody
2020-01-14  0:39 ` [dpdk-dev] [PATCH 2/3] net/bnx2x: fix reset of scan FP flag Rasesh Mody
2020-01-14  0:39 ` Rasesh Mody [this message]
2020-01-14 13:49 ` [dpdk-dev] [PATCH 1/3] net/bnx2x: fix to use required mem barriers in Rx path Jerin Jacob
2020-01-15  1:57   ` [dpdk-dev] [EXT] " Rasesh Mody
2020-01-26 22:55     ` Rasesh Mody
2020-01-26 22:54 ` [dpdk-dev] [PATCH v2 1/2] net/bnx2x: fix reset of scan FP flag Rasesh Mody
2020-01-28  9:59   ` Jerin Jacob
2020-01-26 22:54 ` [dpdk-dev] [PATCH v2 2/2] net/bnx2x: fix to sync fastpath Rx queue access Rasesh Mody

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200114003920.17705-3-rmody@marvell.com \
    --to=rmody@marvell.com \
    --cc=GR-Everest-DPDK-Dev@marvell.com \
    --cc=dev@dpdk.org \
    --cc=ferruh.yigit@intel.com \
    --cc=jerinj@marvell.com \
    --cc=stable@dpdk.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).