From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 476ECA0527 for ; Mon, 9 Nov 2020 19:44:40 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 3BF4C6883; Mon, 9 Nov 2020 19:44:39 +0100 (CET) Received: from mail-wm1-f41.google.com (mail-wm1-f41.google.com [209.85.128.41]) by dpdk.org (Postfix) with ESMTP id D76926883 for ; Mon, 9 Nov 2020 19:44:38 +0100 (CET) Received: by mail-wm1-f41.google.com with SMTP id p22so457168wmg.3 for ; Mon, 09 Nov 2020 10:44:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=2B8dgdbxPcylUi4UI2fsNubRRLq8IUZsMcMq4Kr2I+c=; b=Yv6MYJFFTVvcF9nyH+FqBg03mG2HVtX5mipL8jbc0iY9dEw8QTTLSekzPLTTUcQYO4 RLsLCrbMsovYQnOR59XWekL4PXAJc2Ckonz13Gwr1D1bUZ8SXZpH+feVzgEhGVeIGwiC XzzZaTXjJDngPz7TeUgktWhKCKOfIH8gfd9BcP4SGrqNLWEA1PVnIr2hwoer5kpQgoC/ K9FZIpMz8iw9EmWFbLtILHl6Ac2CB/VjhtWrbZmJoYec+uXdDaOoUh5crw03yxequtxr N71WRF3Cn0f0iSsEeCpGtGsOcki46Oj7+lZ4kFKpw4/CsgmUedusj6ZDNPrlDOTiKV6m cLbQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2B8dgdbxPcylUi4UI2fsNubRRLq8IUZsMcMq4Kr2I+c=; b=C8mzX4GnYBp+yN+1/f42ihB5P10dUHn+OYZkENb/XNdDF5DS3rKozHd9OC97fMsYTA S2QzHd3uYc48htfl0aehI4S7DlgW7ExP/5uLf3+qLegzEZR261F+kBRo8czIXwDPjcuz LlIAjCH7mg8bjbTA4gDxjo8OvFuA9zdXQVNhiD6QGWz5EiywVAgzsFvu7sh4R7kQfrfb xjTcbIVw231LVKNjz9AFewrbsotGXVrYOQa+1GIXNqeVBB6yHB3/c4eRApVQpD8Jo/J8 4fcoYAGCoOrRxodoezpcLnMEFA5Hw1x3hP7ruO3PoZNq2RUuuuVrz+ZzKJGzXLwu8bEJ 3x6g== X-Gm-Message-State: AOAM532HU0X5Zl6+DCzzzcFH+7004BUMkN5QiSZkmbz9SQ/jno5JS4+Q 3a9xjLEV2RrTOQtmN13fIGU= X-Google-Smtp-Source: ABdhPJxHPRkhNofWxZHqKkZgQVZUixH4C0u2Cj4DO0/hBjgrx+/LP7w3ItenjseubKz6RYlolX2JoQ== X-Received: by 2002:a1c:9804:: with SMTP id a4mr549393wme.158.1604947477631; Mon, 09 Nov 2020 10:44:37 -0800 (PST) Received: from localhost ([88.98.246.218]) by smtp.gmail.com with ESMTPSA id b14sm14447393wrq.47.2020.11.09.10.44.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Nov 2020 10:44:36 -0800 (PST) From: luca.boccassi@gmail.com To: Rahul Gupta Cc: Ajit Khaparde , Somnath Kotur , dpdk stable Date: Mon, 9 Nov 2020 18:40:48 +0000 Message-Id: <20201109184111.3463090-60-luca.boccassi@gmail.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201109184111.3463090-1-luca.boccassi@gmail.com> References: <20201028104606.3504127-207-luca.boccassi@gmail.com> <20201109184111.3463090-1-luca.boccassi@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [dpdk-stable] patch 'net/bnxt: fix Rx performance by removing spinlock' has been queued to stable release 19.11.6 X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Sender: "stable" Hi, FYI, your patch has been queued to stable release 19.11.6 Note it hasn't been pushed to http://dpdk.org/browse/dpdk-stable yet. It will be pushed if I get no objections before 11/11/20. So please shout if anyone has objections. Also note that after the patch there's a diff of the upstream commit vs the patch applied to the branch. This will indicate if there was any rebasing needed to apply to the stable branch. If there were code changes for rebasing (ie: not only metadata diffs), please double check that the rebase was correctly done. Queued patches are on a temporary branch at: https://github.com/bluca/dpdk-stable This queued commit can be viewed at: https://github.com/bluca/dpdk-stable/commit/08dcf505033ab43e18b60e0fb8a07279c19a26ca Thanks. Luca Boccassi --- >From 08dcf505033ab43e18b60e0fb8a07279c19a26ca Mon Sep 17 00:00:00 2001 From: Rahul Gupta Date: Sun, 25 Oct 2020 20:56:16 -0700 Subject: [PATCH] net/bnxt: fix Rx performance by removing spinlock [ upstream commit 33ac72d741b72c03350f89215df4eceb73cb61d8 ] The spinlock was trying to protect scenarios where rx_queue stop/start could be initiated dynamically. Assigning bnxt_dummy_recv_pkts and bnxt_dummy_xmit_pkts immediately to avoid concurrent access of mbuf in Rx and cleanup path should help achieve the same result. Fixes: 14255b351537 ("net/bnxt: fix queue start/stop operations") Reviewed-by: Ajit Khaparde Reviewed-by: Somnath Kotur Signed-off-by: Rahul Gupta --- drivers/net/bnxt/bnxt.h | 5 +++++ drivers/net/bnxt/bnxt_cpr.c | 12 ++++++++++++ drivers/net/bnxt/bnxt_cpr.h | 1 + drivers/net/bnxt/bnxt_rxq.c | 4 ---- drivers/net/bnxt/bnxt_rxq.h | 3 --- drivers/net/bnxt/bnxt_rxr.c | 5 +---- drivers/net/bnxt/bnxt_rxr.h | 2 -- drivers/net/bnxt/bnxt_txr.h | 2 -- 8 files changed, 19 insertions(+), 15 deletions(-) diff --git a/drivers/net/bnxt/bnxt.h b/drivers/net/bnxt/bnxt.h index 78249be6c7..3700a277c7 100644 --- a/drivers/net/bnxt/bnxt.h +++ b/drivers/net/bnxt/bnxt.h @@ -683,6 +683,11 @@ bool is_bnxt_supported(struct rte_eth_dev *dev); bool bnxt_stratus_device(struct bnxt *bp); int bnxt_link_update_op(struct rte_eth_dev *eth_dev, int wait_to_complete); +uint16_t bnxt_dummy_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, + uint16_t nb_pkts); +uint16_t bnxt_dummy_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, + uint16_t nb_pkts); + extern const struct rte_flow_ops bnxt_flow_ops; #define bnxt_acquire_flow_lock(bp) \ pthread_mutex_lock(&(bp)->flow_lock) diff --git a/drivers/net/bnxt/bnxt_cpr.c b/drivers/net/bnxt/bnxt_cpr.c index ba9933dc9b..26c7dae88f 100644 --- a/drivers/net/bnxt/bnxt_cpr.c +++ b/drivers/net/bnxt/bnxt_cpr.c @@ -76,6 +76,12 @@ void bnxt_handle_async_event(struct bnxt *bp, PMD_DRV_LOG(INFO, "Port conn async event\n"); break; case HWRM_ASYNC_EVENT_CMPL_EVENT_ID_RESET_NOTIFY: + /* + * Avoid any rx/tx packet processing during firmware reset + * operation. + */ + bnxt_stop_rxtx(bp); + /* Ignore reset notify async events when stopping the port */ if (!bp->eth_dev->data->dev_started) { bp->flags |= BNXT_FLAG_FATAL_ERROR; @@ -282,3 +288,9 @@ bool bnxt_is_recovery_enabled(struct bnxt *bp) return false; } + +void bnxt_stop_rxtx(struct bnxt *bp) +{ + bp->eth_dev->rx_pkt_burst = &bnxt_dummy_recv_pkts; + bp->eth_dev->tx_pkt_burst = &bnxt_dummy_xmit_pkts; +} diff --git a/drivers/net/bnxt/bnxt_cpr.h b/drivers/net/bnxt/bnxt_cpr.h index cccd6cdbe0..ff9697f4c8 100644 --- a/drivers/net/bnxt/bnxt_cpr.h +++ b/drivers/net/bnxt/bnxt_cpr.h @@ -126,4 +126,5 @@ void bnxt_wait_for_device_shutdown(struct bnxt *bp); bool bnxt_is_recovery_enabled(struct bnxt *bp); bool bnxt_is_master_func(struct bnxt *bp); +void bnxt_stop_rxtx(struct bnxt *bp); #endif diff --git a/drivers/net/bnxt/bnxt_rxq.c b/drivers/net/bnxt/bnxt_rxq.c index bb0196670a..d08ba6c3ca 100644 --- a/drivers/net/bnxt/bnxt_rxq.c +++ b/drivers/net/bnxt/bnxt_rxq.c @@ -204,8 +204,6 @@ void bnxt_rx_queue_release_mbufs(struct bnxt_rx_queue *rxq) if (!rxq || !rxq->rx_ring) return; - rte_spinlock_lock(&rxq->lock); - sw_ring = rxq->rx_ring->rx_buf_ring; if (sw_ring) { for (i = 0; @@ -241,7 +239,6 @@ void bnxt_rx_queue_release_mbufs(struct bnxt_rx_queue *rxq) } } - rte_spinlock_unlock(&rxq->lock); } void bnxt_free_rx_mbufs(struct bnxt *bp) @@ -381,7 +378,6 @@ int bnxt_rx_queue_setup_op(struct rte_eth_dev *eth_dev, rxq->rx_started = true; } eth_dev->data->rx_queue_state[queue_idx] = queue_state; - rte_spinlock_init(&rxq->lock); /* Configure mtu if it is different from what was configured before */ if (!queue_idx) diff --git a/drivers/net/bnxt/bnxt_rxq.h b/drivers/net/bnxt/bnxt_rxq.h index b722eeb2c8..ae3badb7a0 100644 --- a/drivers/net/bnxt/bnxt_rxq.h +++ b/drivers/net/bnxt/bnxt_rxq.h @@ -13,9 +13,6 @@ struct bnxt; struct bnxt_rx_ring_info; struct bnxt_cp_ring_info; struct bnxt_rx_queue { - rte_spinlock_t lock; /* Synchronize between rx_queue_stop - * and fast path - */ struct rte_mempool *mb_pool; /* mbuf pool for RX ring */ struct rte_mbuf *pkt_first_seg; /* 1st seg of pkt */ struct rte_mbuf *pkt_last_seg; /* Last seg of pkt */ diff --git a/drivers/net/bnxt/bnxt_rxr.c b/drivers/net/bnxt/bnxt_rxr.c index 5afe5c4f36..136f0a2742 100644 --- a/drivers/net/bnxt/bnxt_rxr.c +++ b/drivers/net/bnxt/bnxt_rxr.c @@ -619,8 +619,7 @@ uint16_t bnxt_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, return 0; /* If Rx Q was stopped return */ - if (unlikely(!rxq->rx_started || - !rte_spinlock_trylock(&rxq->lock))) + if (unlikely(!rxq->rx_started)) return 0; /* Handle RX burst request */ @@ -702,8 +701,6 @@ uint16_t bnxt_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, } done: - rte_spinlock_unlock(&rxq->lock); - return nb_rx_pkts; } diff --git a/drivers/net/bnxt/bnxt_rxr.h b/drivers/net/bnxt/bnxt_rxr.h index 76bf88d707..410b46016b 100644 --- a/drivers/net/bnxt/bnxt_rxr.h +++ b/drivers/net/bnxt/bnxt_rxr.h @@ -212,8 +212,6 @@ struct bnxt_rx_ring_info { uint16_t bnxt_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts); -uint16_t bnxt_dummy_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, - uint16_t nb_pkts); void bnxt_free_rx_rings(struct bnxt *bp); int bnxt_init_rx_ring_struct(struct bnxt_rx_queue *rxq, unsigned int socket_id); int bnxt_init_one_rx_ring(struct bnxt_rx_queue *rxq); diff --git a/drivers/net/bnxt/bnxt_txr.h b/drivers/net/bnxt/bnxt_txr.h index e7f43f9d1d..08fd2e0142 100644 --- a/drivers/net/bnxt/bnxt_txr.h +++ b/drivers/net/bnxt/bnxt_txr.h @@ -57,8 +57,6 @@ int bnxt_init_one_tx_ring(struct bnxt_tx_queue *txq); int bnxt_init_tx_ring_struct(struct bnxt_tx_queue *txq, unsigned int socket_id); uint16_t bnxt_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts); -uint16_t bnxt_dummy_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, - uint16_t nb_pkts); #ifdef RTE_ARCH_X86 uint16_t bnxt_xmit_pkts_vec(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts); -- 2.27.0 --- Diff of the applied patch vs upstream commit (please double-check if non-empty: --- --- - 2020-11-09 18:40:13.474882719 +0000 +++ 0060-net-bnxt-fix-Rx-performance-by-removing-spinlock.patch 2020-11-09 18:40:11.195312278 +0000 @@ -1 +1 @@ -From 33ac72d741b72c03350f89215df4eceb73cb61d8 Mon Sep 17 00:00:00 2001 +From 08dcf505033ab43e18b60e0fb8a07279c19a26ca Mon Sep 17 00:00:00 2001 @@ -5,0 +6,2 @@ +[ upstream commit 33ac72d741b72c03350f89215df4eceb73cb61d8 ] + @@ -17 +19 @@ - drivers/net/bnxt/bnxt.h | 4 ++++ + drivers/net/bnxt/bnxt.h | 5 +++++ @@ -25 +27 @@ - 8 files changed, 18 insertions(+), 15 deletions(-) + 8 files changed, 19 insertions(+), 15 deletions(-) @@ -28 +30 @@ -index 57178192d2..90ced972c0 100644 +index 78249be6c7..3700a277c7 100644 @@ -31,2 +33,2 @@ -@@ -890,6 +890,10 @@ void bnxt_print_link_info(struct rte_eth_dev *eth_dev); - uint16_t bnxt_rss_hash_tbl_size(const struct bnxt *bp); +@@ -683,6 +683,11 @@ bool is_bnxt_supported(struct rte_eth_dev *dev); + bool bnxt_stratus_device(struct bnxt *bp); @@ -39 +41 @@ - ++ @@ -41 +43,2 @@ - + #define bnxt_acquire_flow_lock(bp) \ + pthread_mutex_lock(&(bp)->flow_lock) @@ -43 +46 @@ -index 91d1ffe46c..ee96ae81bf 100644 +index ba9933dc9b..26c7dae88f 100644 @@ -46 +49 @@ -@@ -121,6 +121,12 @@ void bnxt_handle_async_event(struct bnxt *bp, +@@ -76,6 +76,12 @@ void bnxt_handle_async_event(struct bnxt *bp, @@ -59 +62 @@ -@@ -337,3 +343,9 @@ bool bnxt_is_recovery_enabled(struct bnxt *bp) +@@ -282,3 +288,9 @@ bool bnxt_is_recovery_enabled(struct bnxt *bp) @@ -80 +83 @@ -index 78514143e5..e0ec342162 100644 +index bb0196670a..d08ba6c3ca 100644 @@ -83 +86 @@ -@@ -210,8 +210,6 @@ void bnxt_rx_queue_release_mbufs(struct bnxt_rx_queue *rxq) +@@ -204,8 +204,6 @@ void bnxt_rx_queue_release_mbufs(struct bnxt_rx_queue *rxq) @@ -92 +95 @@ -@@ -248,7 +246,6 @@ void bnxt_rx_queue_release_mbufs(struct bnxt_rx_queue *rxq) +@@ -241,7 +239,6 @@ void bnxt_rx_queue_release_mbufs(struct bnxt_rx_queue *rxq) @@ -100 +103 @@ -@@ -389,7 +386,6 @@ int bnxt_rx_queue_setup_op(struct rte_eth_dev *eth_dev, +@@ -381,7 +378,6 @@ int bnxt_rx_queue_setup_op(struct rte_eth_dev *eth_dev, @@ -109 +112 @@ -index 201bda2269..c72105cf06 100644 +index b722eeb2c8..ae3badb7a0 100644 @@ -112 +115 @@ -@@ -16,9 +16,6 @@ struct bnxt; +@@ -13,9 +13,6 @@ struct bnxt; @@ -120,2 +123,2 @@ - uint64_t mbuf_initializer; /* val to init mbuf */ - uint16_t nb_rx_desc; /* num of RX desc */ + struct rte_mbuf *pkt_first_seg; /* 1st seg of pkt */ + struct rte_mbuf *pkt_last_seg; /* Last seg of pkt */ @@ -123 +126 @@ -index 4d0ab604f3..e375b31ed8 100644 +index 5afe5c4f36..136f0a2742 100644 @@ -126 +129 @@ -@@ -843,8 +843,7 @@ uint16_t bnxt_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, +@@ -619,8 +619,7 @@ uint16_t bnxt_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, @@ -135,2 +138,2 @@ - #if defined(RTE_ARCH_X86) || defined(RTE_ARCH_ARM64) -@@ -946,8 +945,6 @@ uint16_t bnxt_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, + /* Handle RX burst request */ +@@ -702,8 +701,6 @@ uint16_t bnxt_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, @@ -146 +149 @@ -index f9e60d01c2..3fc901fdf0 100644 +index 76bf88d707..410b46016b 100644 @@ -149 +152 @@ -@@ -77,8 +77,6 @@ struct bnxt_rx_ring_info { +@@ -212,8 +212,6 @@ struct bnxt_rx_ring_info { @@ -159 +162 @@ -index d241227d4c..3dfc8ef9b4 100644 +index e7f43f9d1d..08fd2e0142 100644 @@ -162 +165 @@ -@@ -49,8 +49,6 @@ int bnxt_init_one_tx_ring(struct bnxt_tx_queue *txq); +@@ -57,8 +57,6 @@ int bnxt_init_one_tx_ring(struct bnxt_tx_queue *txq); @@ -168 +171 @@ - #if defined(RTE_ARCH_X86) || defined(RTE_ARCH_ARM64) + #ifdef RTE_ARCH_X86