From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 70062A0C47 for ; Mon, 26 Jul 2021 15:54:11 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6120C40DDA; Mon, 26 Jul 2021 15:54:11 +0200 (CEST) Received: from mail-wm1-f43.google.com (mail-wm1-f43.google.com [209.85.128.43]) by mails.dpdk.org (Postfix) with ESMTP id CFABD410E1 for ; Mon, 26 Jul 2021 15:54:08 +0200 (CEST) Received: by mail-wm1-f43.google.com with SMTP id k14-20020a05600c1c8eb02901f13dd1672aso53130wms.0 for ; Mon, 26 Jul 2021 06:54:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=xSMV3N0e3VapcaWDlVW+z16SAGAtqENQA1Xsk01of+0=; b=s4Brv4WTuxAzCruxvTzaWLUVGJrBHx617IEvM16VM0pcJWE2FI3PWfdABJMqm8cNDh cx1Spd/iW7ssxZ0N+szx+HZMCc62RoCiVBjoyMdh34X/LU0/11BzwRYr3YKWEvdirQDq uwkNodpSXvBujbnxbI4DiAI19+Gbh9WJMK+jISigTLdfaBYCSxNKx+D3gIU1OxN+ZTeA D40MVe/L3tFIeHqzwYXa0jHpPgETXmfSVlP4bcCkk2ufJUv/cTVLTFw2rhgBE+L5O8yN QVKcoeTSAnB5lDnGyNbksJ/38uyB+ePQuRnbcEskpqs6fVvAEbcoN4ynUhT+Fxd1mqqC Q+1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xSMV3N0e3VapcaWDlVW+z16SAGAtqENQA1Xsk01of+0=; b=FN6CaSsN0Iv1EyL+Wws2PdSxqSlQCBbq/ZMxx7vdaRuRzl1xJTUITTM/Y1qxNplHHh 2bxGjHWUFfAop6IcXNpLVhiF0WVlqh0rGyHk77TRKkFalYNGSopkGspRtsglQlUTillk eDe3ic943hzqZ7YZS5v/SDO2C1I3iIery7rJ/Ec71F23XEQgtKEhnUVUWy/RWwT6eZGu 09NZ8BOSdvM3c4ijE0UxNvqwrYEq06BySKughiyUbETRETy3NC670GinmrZXAzfV8wTK 4iKOI9TabqWF1DTiUqAPOYC3LdTqc6SoWttguylgg+aoAIHkaRxH4BJKYROLBGjudh1z c41Q== X-Gm-Message-State: AOAM533O7/fmFwLK+gFGQI0QZ8G3YpIfpkPeLdCzfi/gydEhqtLWZq9j PYIy65vrek4MblsNHY3jtac= X-Google-Smtp-Source: ABdhPJxdyyEsglmrRAL4Z4vAt1BaBgE1p91REtJIvyKQfOUwJSx5y9rUgxWSN5OBXRLoGyDL7Acf+g== X-Received: by 2002:a1c:7fc5:: with SMTP id a188mr17587634wmd.133.1627307648584; Mon, 26 Jul 2021 06:54:08 -0700 (PDT) Received: from localhost ([137.220.125.106]) by smtp.gmail.com with ESMTPSA id v5sm5171867wrd.74.2021.07.26.06.54.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 26 Jul 2021 06:54:08 -0700 (PDT) From: luca.boccassi@gmail.com To: Lance Richardson Cc: Ajit Khaparde , Ruifeng Wang , dpdk stable Date: Mon, 26 Jul 2021 14:52:36 +0100 Message-Id: <20210726135322.149850-13-luca.boccassi@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210726135322.149850-1-luca.boccassi@gmail.com> References: <20210712130551.2462159-1-luca.boccassi@gmail.com> <20210726135322.149850-1-luca.boccassi@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [dpdk-stable] patch 'net/bnxt: fix missing barriers in completion handling' has been queued to stable release 20.11.3 X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Sender: "stable" Hi, FYI, your patch has been queued to stable release 20.11.3 Note it hasn't been pushed to http://dpdk.org/browse/dpdk-stable yet. It will be pushed if I get no objections before 07/28/21. So please shout if anyone has objections. Also note that after the patch there's a diff of the upstream commit vs the patch applied to the branch. This will indicate if there was any rebasing needed to apply to the stable branch. If there were code changes for rebasing (ie: not only metadata diffs), please double check that the rebase was correctly done. Queued patches are on a temporary branch at: https://github.com/bluca/dpdk-stable This queued commit can be viewed at: https://github.com/bluca/dpdk-stable/commit/5584a03fe99d651f57b600048189c38a1a3a3b34 Thanks. Luca Boccassi --- >From 5584a03fe99d651f57b600048189c38a1a3a3b34 Mon Sep 17 00:00:00 2001 From: Lance Richardson Date: Fri, 9 Jul 2021 12:38:48 -0400 Subject: [PATCH] net/bnxt: fix missing barriers in completion handling [ upstream commit 5ed30db87fa810e210fec27a417a3f0d69f7b425 ] Ensure that Rx/Tx/Async completion entry fields are accessed only after the completion's valid flag has been loaded and verified. This is needed for correct operation on systems that use relaxed memory consistency models. Fixes: 2eb53b134aae ("net/bnxt: add initial Rx code") Fixes: 6eb3cc2294fd ("net/bnxt: add initial Tx code") Signed-off-by: Lance Richardson Reviewed-by: Ajit Khaparde Reviewed-by: Ruifeng Wang --- drivers/net/bnxt/bnxt_cpr.h | 36 ++++++++++++++++++++++++--- drivers/net/bnxt/bnxt_ethdev.c | 16 ++++++------ drivers/net/bnxt/bnxt_irq.c | 7 +++--- drivers/net/bnxt/bnxt_rxr.c | 9 ++++--- drivers/net/bnxt/bnxt_rxtx_vec_neon.c | 2 +- drivers/net/bnxt/bnxt_rxtx_vec_sse.c | 2 +- drivers/net/bnxt/bnxt_txr.c | 2 +- 7 files changed, 53 insertions(+), 21 deletions(-) diff --git a/drivers/net/bnxt/bnxt_cpr.h b/drivers/net/bnxt/bnxt_cpr.h index c769bde619..fedfb47f2f 100644 --- a/drivers/net/bnxt/bnxt_cpr.h +++ b/drivers/net/bnxt/bnxt_cpr.h @@ -8,13 +8,10 @@ #include #include +#include "hsi_struct_def_dpdk.h" struct bnxt_db_info; -#define CMP_VALID(cmp, raw_cons, ring) \ - (!!(rte_le_to_cpu_32(((struct cmpl_base *)(cmp))->info3_v) & \ - CMPL_BASE_V) == !((raw_cons) & ((ring)->ring_size))) - #define CMPL_VALID(cmp, v) \ (!!(rte_le_to_cpu_32(((struct cmpl_base *)(cmp))->info3_v) & \ CMPL_BASE_V) == !(v)) @@ -131,4 +128,35 @@ bool bnxt_is_recovery_enabled(struct bnxt *bp); bool bnxt_is_master_func(struct bnxt *bp); void bnxt_stop_rxtx(struct bnxt *bp); + +/** + * Check validity of a completion ring entry. If the entry is valid, include a + * C11 __ATOMIC_ACQUIRE fence to ensure that subsequent loads of fields in the + * completion are not hoisted by the compiler or by the CPU to come before the + * loading of the "valid" field. + * + * Note: the caller must not access any fields in the specified completion + * entry prior to calling this function. + * + * @param cmpl + * Pointer to an entry in the completion ring. + * @param raw_cons + * Raw consumer index of entry in completion ring. + * @param ring_size + * Size of completion ring. + */ +static __rte_always_inline bool +bnxt_cpr_cmp_valid(const void *cmpl, uint32_t raw_cons, uint32_t ring_size) +{ + const struct cmpl_base *c = cmpl; + bool expected, valid; + + expected = !(raw_cons & ring_size); + valid = !!(rte_le_to_cpu_32(c->info3_v) & CMPL_BASE_V); + if (valid == expected) { + rte_atomic_thread_fence(__ATOMIC_ACQUIRE); + return true; + } + return false; +} #endif diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c index 25df887bfb..fce5e3a628 100644 --- a/drivers/net/bnxt/bnxt_ethdev.c +++ b/drivers/net/bnxt/bnxt_ethdev.c @@ -3041,7 +3041,7 @@ bnxt_rx_queue_count_op(struct rte_eth_dev *dev, uint16_t rx_queue_id) { struct bnxt *bp = (struct bnxt *)dev->data->dev_private; struct bnxt_cp_ring_info *cpr; - uint32_t desc = 0, raw_cons; + uint32_t desc = 0, raw_cons, cp_ring_size; struct bnxt_rx_queue *rxq; struct rx_pkt_cmpl *rxcmp; int rc; @@ -3053,6 +3053,7 @@ bnxt_rx_queue_count_op(struct rte_eth_dev *dev, uint16_t rx_queue_id) rxq = dev->data->rx_queues[rx_queue_id]; cpr = rxq->cp_ring; raw_cons = cpr->cp_raw_cons; + cp_ring_size = cpr->cp_ring_struct->ring_size; while (1) { uint32_t agg_cnt, cons, cmpl_type; @@ -3060,7 +3061,7 @@ bnxt_rx_queue_count_op(struct rte_eth_dev *dev, uint16_t rx_queue_id) cons = RING_CMP(cpr->cp_ring_struct, raw_cons); rxcmp = (struct rx_pkt_cmpl *)&cpr->cp_desc_ring[cons]; - if (!CMP_VALID(rxcmp, raw_cons, cpr->cp_ring_struct)) + if (!bnxt_cpr_cmp_valid(rxcmp, raw_cons, cp_ring_size)) break; cmpl_type = CMP_TYPE(rxcmp); @@ -3104,7 +3105,7 @@ bnxt_rx_descriptor_status_op(void *rx_queue, uint16_t offset) struct bnxt_rx_queue *rxq = rx_queue; struct bnxt_cp_ring_info *cpr; struct bnxt_rx_ring_info *rxr; - uint32_t desc, raw_cons; + uint32_t desc, raw_cons, cp_ring_size; struct bnxt *bp = rxq->bp; struct rx_pkt_cmpl *rxcmp; int rc; @@ -3118,6 +3119,7 @@ bnxt_rx_descriptor_status_op(void *rx_queue, uint16_t offset) rxr = rxq->rx_ring; cpr = rxq->cp_ring; + cp_ring_size = cpr->cp_ring_struct->ring_size; /* * For the vector receive case, the completion at the requested @@ -3134,7 +3136,7 @@ bnxt_rx_descriptor_status_op(void *rx_queue, uint16_t offset) cons = RING_CMP(cpr->cp_ring_struct, raw_cons); rxcmp = (struct rx_pkt_cmpl *)&cpr->cp_desc_ring[cons]; - if (CMP_VALID(rxcmp, raw_cons, cpr->cp_ring_struct)) + if (bnxt_cpr_cmp_valid(rxcmp, raw_cons, cp_ring_size)) return RTE_ETH_RX_DESC_DONE; /* Check whether rx desc has an mbuf attached. */ @@ -3160,7 +3162,7 @@ bnxt_rx_descriptor_status_op(void *rx_queue, uint16_t offset) cons = RING_CMP(cpr->cp_ring_struct, raw_cons); rxcmp = (struct rx_pkt_cmpl *)&cpr->cp_desc_ring[cons]; - if (!CMP_VALID(rxcmp, raw_cons, cpr->cp_ring_struct)) + if (!bnxt_cpr_cmp_valid(rxcmp, raw_cons, cp_ring_size)) break; cmpl_type = CMP_TYPE(rxcmp); @@ -3214,7 +3216,6 @@ bnxt_tx_descriptor_status_op(void *tx_queue, uint16_t offset) struct bnxt_tx_queue *txq = (struct bnxt_tx_queue *)tx_queue; struct bnxt_cp_ring_info *cpr = txq->cp_ring; uint32_t ring_mask, raw_cons, nb_tx_pkts = 0; - struct bnxt_ring *cp_ring_struct; struct cmpl_base *cp_desc_ring; int rc; @@ -3231,7 +3232,6 @@ bnxt_tx_descriptor_status_op(void *tx_queue, uint16_t offset) raw_cons = cpr->cp_raw_cons; cp_desc_ring = cpr->cp_desc_ring; - cp_ring_struct = cpr->cp_ring_struct; ring_mask = cpr->cp_ring_struct->ring_mask; /* Check to see if hw has posted a completion for the descriptor. */ @@ -3242,7 +3242,7 @@ bnxt_tx_descriptor_status_op(void *tx_queue, uint16_t offset) cons = RING_CMPL(ring_mask, raw_cons); txcmp = (struct tx_cmpl *)&cp_desc_ring[cons]; - if (!CMP_VALID(txcmp, raw_cons, cp_ring_struct)) + if (!bnxt_cpr_cmp_valid(txcmp, raw_cons, ring_mask + 1)) break; if (CMP_TYPE(txcmp) == TX_CMPL_TYPE_TX_L2) diff --git a/drivers/net/bnxt/bnxt_irq.c b/drivers/net/bnxt/bnxt_irq.c index 93779f4e0c..14075e41d2 100644 --- a/drivers/net/bnxt/bnxt_irq.c +++ b/drivers/net/bnxt/bnxt_irq.c @@ -21,10 +21,10 @@ void bnxt_int_handler(void *param) { struct rte_eth_dev *eth_dev = (struct rte_eth_dev *)param; struct bnxt *bp = eth_dev->data->dev_private; + uint32_t cons, raw_cons, cp_ring_size; struct bnxt_cp_ring_info *cpr; struct cmpl_base *cmp; - uint32_t raw_cons; - uint32_t cons; + if (bp == NULL) return; @@ -33,6 +33,7 @@ void bnxt_int_handler(void *param) return; raw_cons = cpr->cp_raw_cons; + cp_ring_size = cpr->cp_ring_struct->ring_size; pthread_mutex_lock(&bp->def_cp_lock); while (1) { if (!cpr || !cpr->cp_ring_struct || !cpr->cp_db.doorbell) { @@ -48,7 +49,7 @@ void bnxt_int_handler(void *param) cons = RING_CMP(cpr->cp_ring_struct, raw_cons); cmp = &cpr->cp_desc_ring[cons]; - if (!CMP_VALID(cmp, raw_cons, cpr->cp_ring_struct)) + if (!bnxt_cpr_cmp_valid(cmp, raw_cons, cp_ring_size)) break; bnxt_event_hwrm_resp_handler(bp, cmp); diff --git a/drivers/net/bnxt/bnxt_rxr.c b/drivers/net/bnxt/bnxt_rxr.c index 54c9df06e1..41be74284b 100644 --- a/drivers/net/bnxt/bnxt_rxr.c +++ b/drivers/net/bnxt/bnxt_rxr.c @@ -242,7 +242,8 @@ static int bnxt_agg_bufs_valid(struct bnxt_cp_ring_info *cpr, cpr->valid = FLIP_VALID(raw_cp_cons, cpr->cp_ring_struct->ring_mask, cpr->valid); - return CMP_VALID(agg_cmpl, raw_cp_cons, cpr->cp_ring_struct); + return bnxt_cpr_cmp_valid(agg_cmpl, raw_cp_cons, + cpr->cp_ring_struct->ring_size); } /* TPA consume agg buffer out of order, allocate connected data only */ @@ -827,7 +828,8 @@ static int bnxt_rx_pkt(struct rte_mbuf **rx_pkt, cp_cons = RING_CMP(cpr->cp_ring_struct, tmp_raw_cons); rxcmp1 = (struct rx_pkt_cmpl_hi *)&cpr->cp_desc_ring[cp_cons]; - if (!CMP_VALID(rxcmp1, tmp_raw_cons, cpr->cp_ring_struct)) + if (!bnxt_cpr_cmp_valid(rxcmp1, tmp_raw_cons, + cpr->cp_ring_struct->ring_size)) return -EBUSY; cpr->valid = FLIP_VALID(cp_cons, @@ -1006,7 +1008,8 @@ uint16_t bnxt_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, cons = RING_CMP(cpr->cp_ring_struct, raw_cons); rxcmp = (struct rx_pkt_cmpl *)&cpr->cp_desc_ring[cons]; - if (!CMP_VALID(rxcmp, raw_cons, cpr->cp_ring_struct)) + if (!bnxt_cpr_cmp_valid(rxcmp, raw_cons, + cpr->cp_ring_struct->ring_size)) break; cpr->valid = FLIP_VALID(cons, cpr->cp_ring_struct->ring_mask, diff --git a/drivers/net/bnxt/bnxt_rxtx_vec_neon.c b/drivers/net/bnxt/bnxt_rxtx_vec_neon.c index 3292852c42..3cb94926fe 100644 --- a/drivers/net/bnxt/bnxt_rxtx_vec_neon.c +++ b/drivers/net/bnxt/bnxt_rxtx_vec_neon.c @@ -347,7 +347,7 @@ bnxt_handle_tx_cp_vec(struct bnxt_tx_queue *txq) cons = RING_CMPL(ring_mask, raw_cons); txcmp = (struct tx_cmpl *)&cp_desc_ring[cons]; - if (!CMP_VALID(txcmp, raw_cons, cp_ring_struct)) + if (!bnxt_cpr_cmp_valid(txcmp, raw_cons, ring_mask + 1)) break; if (likely(CMP_TYPE(txcmp) == TX_CMPL_TYPE_TX_L2)) diff --git a/drivers/net/bnxt/bnxt_rxtx_vec_sse.c b/drivers/net/bnxt/bnxt_rxtx_vec_sse.c index ae73455c63..5974b475bf 100644 --- a/drivers/net/bnxt/bnxt_rxtx_vec_sse.c +++ b/drivers/net/bnxt/bnxt_rxtx_vec_sse.c @@ -329,7 +329,7 @@ bnxt_handle_tx_cp_vec(struct bnxt_tx_queue *txq) cons = RING_CMPL(ring_mask, raw_cons); txcmp = (struct tx_cmpl *)&cp_desc_ring[cons]; - if (!CMP_VALID(txcmp, raw_cons, cp_ring_struct)) + if (!bnxt_cpr_cmp_valid(txcmp, raw_cons, ring_mask + 1)) break; if (likely(CMP_TYPE(txcmp) == TX_CMPL_TYPE_TX_L2)) diff --git a/drivers/net/bnxt/bnxt_txr.c b/drivers/net/bnxt/bnxt_txr.c index 412bc59f3b..ad78fa1bea 100644 --- a/drivers/net/bnxt/bnxt_txr.c +++ b/drivers/net/bnxt/bnxt_txr.c @@ -443,7 +443,7 @@ static int bnxt_handle_tx_cp(struct bnxt_tx_queue *txq) cons = RING_CMPL(ring_mask, raw_cons); txcmp = (struct tx_cmpl *)&cpr->cp_desc_ring[cons]; - if (!CMP_VALID(txcmp, raw_cons, cp_ring_struct)) + if (!bnxt_cpr_cmp_valid(txcmp, raw_cons, ring_mask + 1)) break; opaque = rte_le_to_cpu_32(txcmp->opaque); -- 2.30.2 --- Diff of the applied patch vs upstream commit (please double-check if non-empty: --- --- - 2021-07-26 13:53:16.581330845 +0100 +++ 0013-net-bnxt-fix-missing-barriers-in-completion-handling.patch 2021-07-26 13:53:15.821292034 +0100 @@ -1 +1 @@ -From 5ed30db87fa810e210fec27a417a3f0d69f7b425 Mon Sep 17 00:00:00 2001 +From 5584a03fe99d651f57b600048189c38a1a3a3b34 Mon Sep 17 00:00:00 2001 @@ -5,0 +6,2 @@ +[ upstream commit 5ed30db87fa810e210fec27a417a3f0d69f7b425 ] + @@ -13 +14,0 @@ -Cc: stable@dpdk.org @@ -23 +23,0 @@ - drivers/net/bnxt/bnxt_rxtx_vec_avx2.c | 2 +- @@ -27 +27 @@ - 8 files changed, 54 insertions(+), 22 deletions(-) + 7 files changed, 53 insertions(+), 21 deletions(-) @@ -30 +30 @@ -index 2a56ec52ce..4095c8c40d 100644 +index c769bde619..fedfb47f2f 100644 @@ -45,4 +45,4 @@ - #define CMP_TYPE(cmp) \ - (((struct cmpl_base *)cmp)->type & CMPL_BASE_TYPE_MASK) - -@@ -121,4 +118,35 @@ bool bnxt_is_recovery_enabled(struct bnxt *bp); + #define CMPL_VALID(cmp, v) \ + (!!(rte_le_to_cpu_32(((struct cmpl_base *)(cmp))->info3_v) & \ + CMPL_BASE_V) == !(v)) +@@ -131,4 +128,35 @@ bool bnxt_is_recovery_enabled(struct bnxt *bp); @@ -85 +85 @@ -index ed09f1bf52..ee69296926 100644 +index 25df887bfb..fce5e3a628 100644 @@ -88 +88 @@ -@@ -3126,7 +3126,7 @@ bnxt_rx_queue_count_op(struct rte_eth_dev *dev, uint16_t rx_queue_id) +@@ -3041,7 +3041,7 @@ bnxt_rx_queue_count_op(struct rte_eth_dev *dev, uint16_t rx_queue_id) @@ -97 +97 @@ -@@ -3138,6 +3138,7 @@ bnxt_rx_queue_count_op(struct rte_eth_dev *dev, uint16_t rx_queue_id) +@@ -3053,6 +3053,7 @@ bnxt_rx_queue_count_op(struct rte_eth_dev *dev, uint16_t rx_queue_id) @@ -105 +105 @@ -@@ -3145,7 +3146,7 @@ bnxt_rx_queue_count_op(struct rte_eth_dev *dev, uint16_t rx_queue_id) +@@ -3060,7 +3061,7 @@ bnxt_rx_queue_count_op(struct rte_eth_dev *dev, uint16_t rx_queue_id) @@ -114 +114 @@ -@@ -3189,7 +3190,7 @@ bnxt_rx_descriptor_status_op(void *rx_queue, uint16_t offset) +@@ -3104,7 +3105,7 @@ bnxt_rx_descriptor_status_op(void *rx_queue, uint16_t offset) @@ -123 +123 @@ -@@ -3203,6 +3204,7 @@ bnxt_rx_descriptor_status_op(void *rx_queue, uint16_t offset) +@@ -3118,6 +3119,7 @@ bnxt_rx_descriptor_status_op(void *rx_queue, uint16_t offset) @@ -131 +131 @@ -@@ -3219,7 +3221,7 @@ bnxt_rx_descriptor_status_op(void *rx_queue, uint16_t offset) +@@ -3134,7 +3136,7 @@ bnxt_rx_descriptor_status_op(void *rx_queue, uint16_t offset) @@ -140 +140 @@ -@@ -3245,7 +3247,7 @@ bnxt_rx_descriptor_status_op(void *rx_queue, uint16_t offset) +@@ -3160,7 +3162,7 @@ bnxt_rx_descriptor_status_op(void *rx_queue, uint16_t offset) @@ -149 +149 @@ -@@ -3299,7 +3301,6 @@ bnxt_tx_descriptor_status_op(void *tx_queue, uint16_t offset) +@@ -3214,7 +3216,6 @@ bnxt_tx_descriptor_status_op(void *tx_queue, uint16_t offset) @@ -157 +157 @@ -@@ -3316,7 +3317,6 @@ bnxt_tx_descriptor_status_op(void *tx_queue, uint16_t offset) +@@ -3231,7 +3232,6 @@ bnxt_tx_descriptor_status_op(void *tx_queue, uint16_t offset) @@ -165 +165 @@ -@@ -3327,7 +3327,7 @@ bnxt_tx_descriptor_status_op(void *tx_queue, uint16_t offset) +@@ -3242,7 +3242,7 @@ bnxt_tx_descriptor_status_op(void *tx_queue, uint16_t offset) @@ -175 +175 @@ -index fd8b8fac98..ebdac8385d 100644 +index 93779f4e0c..14075e41d2 100644 @@ -209 +209 @@ -index 0dee73af86..aea71703d1 100644 +index 54c9df06e1..41be74284b 100644 @@ -212,4 +212,4 @@ -@@ -297,7 +297,8 @@ static int bnxt_agg_bufs_valid(struct bnxt_cp_ring_info *cpr, - raw_cp_cons = ADV_RAW_CMP(raw_cp_cons, agg_bufs); - last_cp_cons = RING_CMP(cpr->cp_ring_struct, raw_cp_cons); - agg_cmpl = (struct rx_pkt_cmpl *)&cpr->cp_desc_ring[last_cp_cons]; +@@ -242,7 +242,8 @@ static int bnxt_agg_bufs_valid(struct bnxt_cp_ring_info *cpr, + cpr->valid = FLIP_VALID(raw_cp_cons, + cpr->cp_ring_struct->ring_mask, + cpr->valid); @@ -222 +222 @@ -@@ -892,7 +893,8 @@ static int bnxt_rx_pkt(struct rte_mbuf **rx_pkt, +@@ -827,7 +828,8 @@ static int bnxt_rx_pkt(struct rte_mbuf **rx_pkt, @@ -231,2 +231,2 @@ - if (cmp_type == RX_TPA_START_CMPL_TYPE_RX_TPA_START || -@@ -1077,7 +1079,8 @@ uint16_t bnxt_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, + cpr->valid = FLIP_VALID(cp_cons, +@@ -1006,7 +1008,8 @@ uint16_t bnxt_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, @@ -240,15 +240,2 @@ - if (CMP_TYPE(rxcmp) == CMPL_BASE_TYPE_HWRM_DONE) { - PMD_DRV_LOG(ERR, "Rx flush done\n"); -diff --git a/drivers/net/bnxt/bnxt_rxtx_vec_avx2.c b/drivers/net/bnxt/bnxt_rxtx_vec_avx2.c -index a06dfec90e..d08854ff61 100644 ---- a/drivers/net/bnxt/bnxt_rxtx_vec_avx2.c -+++ b/drivers/net/bnxt/bnxt_rxtx_vec_avx2.c -@@ -408,7 +408,7 @@ bnxt_handle_tx_cp_vec(struct bnxt_tx_queue *txq) - cons = RING_CMPL(ring_mask, raw_cons); - txcmp = (struct tx_cmpl *)&cp_desc_ring[cons]; - -- if (!CMP_VALID(txcmp, raw_cons, cp_ring_struct)) -+ if (!bnxt_cpr_cmp_valid(txcmp, raw_cons, ring_mask + 1)) - break; - - nb_tx_pkts += txcmp->opaque; + cpr->valid = FLIP_VALID(cons, + cpr->cp_ring_struct->ring_mask, @@ -256 +243 @@ -index 263e6ec3c1..13211060cf 100644 +index 3292852c42..3cb94926fe 100644 @@ -259 +246 @@ -@@ -339,7 +339,7 @@ bnxt_handle_tx_cp_vec(struct bnxt_tx_queue *txq) +@@ -347,7 +347,7 @@ bnxt_handle_tx_cp_vec(struct bnxt_tx_queue *txq) @@ -269 +256 @@ -index 9a53d1fba0..6e56305326 100644 +index ae73455c63..5974b475bf 100644 @@ -272 +259 @@ -@@ -320,7 +320,7 @@ bnxt_handle_tx_cp_vec(struct bnxt_tx_queue *txq) +@@ -329,7 +329,7 @@ bnxt_handle_tx_cp_vec(struct bnxt_tx_queue *txq) @@ -282 +269 @@ -index 9a6b96e04a..47824334ae 100644 +index 412bc59f3b..ad78fa1bea 100644 @@ -285 +272 @@ -@@ -461,7 +461,7 @@ static int bnxt_handle_tx_cp(struct bnxt_tx_queue *txq) +@@ -443,7 +443,7 @@ static int bnxt_handle_tx_cp(struct bnxt_tx_queue *txq)