From: Rasesh Mody <rmody@marvell.com> To: <dev@dpdk.org>, <jerinj@marvell.com>, <ferruh.yigit@intel.com> Cc: Rasesh Mody <rmody@marvell.com>, <GR-Everest-DPDK-Dev@marvell.com>, <stable@dpdk.org> Subject: [dpdk-dev] [PATCH 3/3] net/bnx2x: fix to sync fastpath Rx queue access Date: Mon, 13 Jan 2020 16:39:20 -0800 Message-ID: <20200114003920.17705-3-rmody@marvell.com> (raw) In-Reply-To: <20200114003920.17705-1-rmody@marvell.com> PMD handles fast path completions in the Rx handler and control path completions in the interrupt handler. They both are processing completions from the same fastpath completion queue. There is a potential for race condition when these two paths are processing the completions from the same queue and trying to updating Rx Producer. Add a fastpath Rx lock between these two paths to close this race. Fixes: 540a211084a7 ("bnx2x: driver core") Cc: stable@dpdk.org Signed-off-by: Rasesh Mody <rmody@marvell.com> --- drivers/net/bnx2x/bnx2x.c | 12 ++++++++++++ drivers/net/bnx2x/bnx2x.h | 3 +++ drivers/net/bnx2x/bnx2x_rxtx.c | 8 +++++++- 3 files changed, 22 insertions(+), 1 deletion(-) diff --git a/drivers/net/bnx2x/bnx2x.c b/drivers/net/bnx2x/bnx2x.c index d38da4f60..7ea98b936 100644 --- a/drivers/net/bnx2x/bnx2x.c +++ b/drivers/net/bnx2x/bnx2x.c @@ -1167,6 +1167,10 @@ static int bnx2x_has_rx_work(struct bnx2x_fastpath *fp) if (unlikely((rx_cq_cons_sb & MAX_RCQ_ENTRIES(rxq)) == MAX_RCQ_ENTRIES(rxq))) rx_cq_cons_sb++; + + PMD_RX_LOG(DEBUG, "hw CQ cons = %d, sw CQ cons = %d", + rx_cq_cons_sb, rxq->rx_cq_head); + return rxq->rx_cq_head != rx_cq_cons_sb; } @@ -1249,9 +1253,12 @@ static uint8_t bnx2x_rxeof(struct bnx2x_softc *sc, struct bnx2x_fastpath *fp) uint16_t bd_cons, bd_prod, bd_prod_fw, comp_ring_cons; uint16_t hw_cq_cons, sw_cq_cons, sw_cq_prod; + rte_spinlock_lock(&(fp)->rx_mtx); + rxq = sc->rx_queues[fp->index]; if (!rxq) { PMD_RX_LOG(ERR, "RX queue %d is NULL", fp->index); + rte_spinlock_unlock(&(fp)->rx_mtx); return 0; } @@ -1326,9 +1333,14 @@ static uint8_t bnx2x_rxeof(struct bnx2x_softc *sc, struct bnx2x_fastpath *fp) rxq->rx_cq_head = sw_cq_cons; rxq->rx_cq_tail = sw_cq_prod; + PMD_RX_LOG(DEBUG, "BD prod = %d, sw CQ prod = %d", + bd_prod_fw, sw_cq_prod); + /* Update producers */ bnx2x_update_rx_prod(sc, fp, bd_prod_fw, sw_cq_prod); + rte_spinlock_unlock(&(fp)->rx_mtx); + return sw_cq_cons != hw_cq_cons; } diff --git a/drivers/net/bnx2x/bnx2x.h b/drivers/net/bnx2x/bnx2x.h index 3383c7675..1dbc98197 100644 --- a/drivers/net/bnx2x/bnx2x.h +++ b/drivers/net/bnx2x/bnx2x.h @@ -360,6 +360,9 @@ struct bnx2x_fastpath { /* pointer back to parent structure */ struct bnx2x_softc *sc; + /* Used to synchronize fastpath Rx access */ + rte_spinlock_t rx_mtx; + /* status block */ struct bnx2x_dma sb_dma; union bnx2x_host_hc_status_block status_block; diff --git a/drivers/net/bnx2x/bnx2x_rxtx.c b/drivers/net/bnx2x/bnx2x_rxtx.c index b52f023ea..c8bb202d6 100644 --- a/drivers/net/bnx2x/bnx2x_rxtx.c +++ b/drivers/net/bnx2x/bnx2x_rxtx.c @@ -357,6 +357,8 @@ bnx2x_recv_pkts(void *p_rxq, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) uint16_t len, pad; struct rte_mbuf *rx_mb = NULL; + rte_spinlock_lock(&(fp)->rx_mtx); + /* Add memory barrier as status block fields can change. This memory * barrier will flush out all the read/write operations to status block * generated before the barrier. It will ensure stale data is not read @@ -379,8 +381,10 @@ bnx2x_recv_pkts(void *p_rxq, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) */ rmb(); - if (sw_cq_cons == hw_cq_cons) + if (sw_cq_cons == hw_cq_cons) { + rte_spinlock_unlock(&(fp)->rx_mtx); return 0; + } while (nb_rx < nb_pkts && sw_cq_cons != hw_cq_cons) { @@ -461,6 +465,8 @@ bnx2x_recv_pkts(void *p_rxq, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) bnx2x_upd_rx_prod_fast(sc, fp, bd_prod, sw_cq_prod); + rte_spinlock_unlock(&(fp)->rx_mtx); + return nb_rx; } -- 2.18.0
next prev parent reply other threads:[~2020-01-14 0:39 UTC|newest] Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-01-14 0:39 [dpdk-dev] [PATCH 1/3] net/bnx2x: fix to use required mem barriers in Rx path Rasesh Mody 2020-01-14 0:39 ` [dpdk-dev] [PATCH 2/3] net/bnx2x: fix reset of scan FP flag Rasesh Mody 2020-01-14 0:39 ` Rasesh Mody [this message] 2020-01-14 13:49 ` [dpdk-dev] [PATCH 1/3] net/bnx2x: fix to use required mem barriers in Rx path Jerin Jacob 2020-01-15 1:57 ` [dpdk-dev] [EXT] " Rasesh Mody 2020-01-26 22:55 ` Rasesh Mody 2020-01-26 22:54 ` [dpdk-dev] [PATCH v2 1/2] net/bnx2x: fix reset of scan FP flag Rasesh Mody 2020-01-28 9:59 ` Jerin Jacob 2020-01-26 22:54 ` [dpdk-dev] [PATCH v2 2/2] net/bnx2x: fix to sync fastpath Rx queue access Rasesh Mody
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20200114003920.17705-3-rmody@marvell.com \ --to=rmody@marvell.com \ --cc=GR-Everest-DPDK-Dev@marvell.com \ --cc=dev@dpdk.org \ --cc=ferruh.yigit@intel.com \ --cc=jerinj@marvell.com \ --cc=stable@dpdk.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
DPDK patches and discussions This inbox may be cloned and mirrored by anyone: git clone --mirror https://inbox.dpdk.org/dev/0 dev/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 dev dev/ https://inbox.dpdk.org/dev \ dev@dpdk.org public-inbox-index dev Example config snippet for mirrors. Newsgroup available over NNTP: nntp://inbox.dpdk.org/inbox.dpdk.dev AGPL code for this site: git clone https://public-inbox.org/public-inbox.git