From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0B2C8455AD for ; Mon, 15 Jul 2024 17:30:28 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 04E7440E01; Mon, 15 Jul 2024 17:30:28 +0200 (CEST) Received: from mail-wm1-f47.google.com (mail-wm1-f47.google.com [209.85.128.47]) by mails.dpdk.org (Postfix) with ESMTP id 8EE4640A71 for ; Mon, 15 Jul 2024 17:30:26 +0200 (CEST) Received: by mail-wm1-f47.google.com with SMTP id 5b1f17b1804b1-426636ef8c9so29866375e9.2 for ; Mon, 15 Jul 2024 08:30:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1721057426; x=1721662226; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=E9kDsIhzpIAQNxJdPAjaQ48/Fzg73yoY0UhlMeBSeiA=; b=bn2L1nkc+Ydv39KFmO5ZR3sGwU4ebMrsm16Y2B23NBPYDZWU5jggvbwj24pYYVlTNN Odfb/7Pzyem7KK3EUbLVUeA0F+FxUP7bGICaMQtg9ad60G6rk5uJ58+zZ6PISvUdz9HY i9zvvhQVGix65k5Y6tM9ZBmn1b/FOlafoUpi27mrhjEu+PX3JixMiGdErMYVGP/uxzZk 3ICIyTE/gvmS9geRTyntyHcpACzEhaBuqXX2H3ojxF+ynWJLRFxWH2N+3xsB17zo0C8l kHLK1PDVm1pO6fF/NEEpyLvghlraf37n5QmzLRnbYtQzUNNI5u2DWLSwNukmkQr6Zmr8 KApg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721057426; x=1721662226; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=E9kDsIhzpIAQNxJdPAjaQ48/Fzg73yoY0UhlMeBSeiA=; b=UZa2TqxKTLg1aW7Znp/N4YxbeuoYfe8JotEl3dfd9tCGixIUMsXl5BQehhW6meQPR9 lAWto/Ut2b1ba+O3cPxiA9YMhJNNJlU9HRlBMEPbcw7D98VeXUreFxzWEc5oGIoRM7RF 7AgGOzY1VtTG1Jp+HlsS5I0a3FIi31YMJIsTEytm4VFupFz9eXtgw0yJC6b1vXt/6AXj uaVfP8dhd6zUYTNzHD/d0FbYuj8lEhaIMO3H3a3uCEbPN1Z9A4Xw+DAIBSIfDAIknS8q RqDreV1yXLYtLDMOLzZt2XZwcHvF6WE/8EPWqrOEsF13l+ENSeCZ7iRIv/eZeg3r7ung oXWg== X-Gm-Message-State: AOJu0YziSVAuvlsMg5Ikl0eMUNf133tuoHsr2qGoCh0Zn8fW9IlA9T0Z s952bNiPHsmWFx4utVQTzbcePm6/JmfoyCrjmbI8qnCKeMjZfHgU X-Google-Smtp-Source: AGHT+IF6CB5BoqKl+uEmpxOVCwEIV0r02cMBexppOw/KKimygGD8G5FKmBBkfuNomn72zTNEQs9TDg== X-Received: by 2002:a05:600c:4a16:b0:426:6eac:8314 with SMTP id 5b1f17b1804b1-426707ce69emr145742615e9.1.1721057426166; Mon, 15 Jul 2024 08:30:26 -0700 (PDT) Received: from localhost ([2a01:4b00:d036:ae00:7aef:1aaa:3dff:d546]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-427a0fa55afsm118743355e9.16.2024.07.15.08.30.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Jul 2024 08:30:25 -0700 (PDT) From: luca.boccassi@gmail.com To: Jiawen Wu Cc: dpdk stable Subject: patch 'net/txgbe: fix Tx hang on queue disable' has been queued to stable release 22.11.6 Date: Mon, 15 Jul 2024 16:26:40 +0100 Message-Id: <20240715152704.2229503-62-luca.boccassi@gmail.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240715152704.2229503-1-luca.boccassi@gmail.com> References: <20240624235907.885628-81-luca.boccassi@gmail.com> <20240715152704.2229503-1-luca.boccassi@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Hi, FYI, your patch has been queued to stable release 22.11.6 Note it hasn't been pushed to http://dpdk.org/browse/dpdk-stable yet. It will be pushed if I get no objections before 07/17/24. So please shout if anyone has objections. Also note that after the patch there's a diff of the upstream commit vs the patch applied to the branch. This will indicate if there was any rebasing needed to apply to the stable branch. If there were code changes for rebasing (ie: not only metadata diffs), please double check that the rebase was correctly done. Queued patches are on a temporary branch at: https://github.com/bluca/dpdk-stable This queued commit can be viewed at: https://github.com/bluca/dpdk-stable/commit/a96f251defbcc1501cca4da1b7ed6a44137216c6 Thanks. Luca Boccassi --- >From a96f251defbcc1501cca4da1b7ed6a44137216c6 Mon Sep 17 00:00:00 2001 From: Jiawen Wu Date: Tue, 18 Jun 2024 15:11:34 +0800 Subject: [PATCH] net/txgbe: fix Tx hang on queue disable [ upstream commit 8a31f4bbbbb73baf77297f00887f4b267feb7b8e ] The problem of Tx hang also occurs on Wangxun 10Gb NICs, when stop device under heavy traffic. Refer to commit ac6c5e9af56a ("net/ngbe: fix Tx hang on queue disable") Disable PCIe bus master to clear BME when stop hardware, and verify there are no pending requests. Move disabling Tx queue after disabling PCIe bus master to ensure that there are no packets left to cause Tx hang. Fixes: b1f596677d8e ("net/txgbe: support device start") Signed-off-by: Jiawen Wu --- drivers/net/txgbe/base/meson.build | 2 +- drivers/net/txgbe/base/txgbe_hw.c | 60 +++++++++++++++++++++++----- drivers/net/txgbe/base/txgbe_hw.h | 1 + drivers/net/txgbe/base/txgbe_osdep.h | 1 + drivers/net/txgbe/base/txgbe_regs.h | 3 ++ drivers/net/txgbe/base/txgbe_type.h | 1 + drivers/net/txgbe/txgbe_ethdev.c | 7 ++++ 7 files changed, 65 insertions(+), 10 deletions(-) diff --git a/drivers/net/txgbe/base/meson.build b/drivers/net/txgbe/base/meson.build index a81d6890fe..4cf90a394a 100644 --- a/drivers/net/txgbe/base/meson.build +++ b/drivers/net/txgbe/base/meson.build @@ -22,6 +22,6 @@ foreach flag: error_cflags endforeach base_lib = static_library('txgbe_base', sources, - dependencies: [static_rte_eal, static_rte_net], + dependencies: [static_rte_eal, static_rte_net, static_rte_bus_pci], c_args: c_args) base_objs = base_lib.extract_all_objects(recursive: true) diff --git a/drivers/net/txgbe/base/txgbe_hw.c b/drivers/net/txgbe/base/txgbe_hw.c index de96549ae8..b59667f1d7 100644 --- a/drivers/net/txgbe/base/txgbe_hw.c +++ b/drivers/net/txgbe/base/txgbe_hw.c @@ -462,7 +462,7 @@ void txgbe_set_lan_id_multi_port(struct txgbe_hw *hw) **/ s32 txgbe_stop_hw(struct txgbe_hw *hw) { - u32 reg_val; + s32 status = 0; u16 i; /* @@ -484,16 +484,26 @@ s32 txgbe_stop_hw(struct txgbe_hw *hw) wr32(hw, TXGBE_ICR(0), TXGBE_ICR_MASK); wr32(hw, TXGBE_ICR(1), TXGBE_ICR_MASK); + wr32(hw, TXGBE_BMECTL, 0x3); + + /* Disable the receive unit by stopping each queue */ + for (i = 0; i < hw->mac.max_rx_queues; i++) + wr32(hw, TXGBE_RXCFG(i), 0); + + /* flush all queues disables */ + txgbe_flush(hw); + msec_delay(2); + + /* Prevent the PCI-E bus from hanging by disabling PCI-E master + * access and verify no pending requests + */ + status = txgbe_set_pcie_master(hw, false); + if (status) + return status; + /* Disable the transmit unit. Each queue must be disabled. */ for (i = 0; i < hw->mac.max_tx_queues; i++) - wr32(hw, TXGBE_TXCFG(i), TXGBE_TXCFG_FLUSH); - - /* Disable the receive unit by stopping each queue */ - for (i = 0; i < hw->mac.max_rx_queues; i++) { - reg_val = rd32(hw, TXGBE_RXCFG(i)); - reg_val &= ~TXGBE_RXCFG_ENA; - wr32(hw, TXGBE_RXCFG(i), reg_val); - } + wr32(hw, TXGBE_TXCFG(i), 0); /* flush all queues disables */ txgbe_flush(hw); @@ -1174,6 +1184,38 @@ out: } } +s32 txgbe_set_pcie_master(struct txgbe_hw *hw, bool enable) +{ + struct rte_pci_device *pci_dev = (struct rte_pci_device *)hw->back; + s32 status = 0; + u32 i; + + if (rte_pci_set_bus_master(pci_dev, enable) < 0) { + DEBUGOUT("Cannot configure PCI bus master."); + return -1; + } + + if (enable) + goto out; + + /* Exit if master requests are blocked */ + if (!(rd32(hw, TXGBE_BMEPEND))) + goto out; + + /* Poll for master request bit to clear */ + for (i = 0; i < TXGBE_PCI_MASTER_DISABLE_TIMEOUT; i++) { + usec_delay(100); + if (!(rd32(hw, TXGBE_BMEPEND))) + goto out; + } + + DEBUGOUT("PCIe transaction pending bit also did not clear."); + status = TXGBE_ERR_MASTER_REQUESTS_PENDING; + +out: + return status; +} + /** * txgbe_acquire_swfw_sync - Acquire SWFW semaphore * @hw: pointer to hardware structure diff --git a/drivers/net/txgbe/base/txgbe_hw.h b/drivers/net/txgbe/base/txgbe_hw.h index 7031589f7c..4bf9da2d4c 100644 --- a/drivers/net/txgbe/base/txgbe_hw.h +++ b/drivers/net/txgbe/base/txgbe_hw.h @@ -40,6 +40,7 @@ s32 txgbe_setup_fc(struct txgbe_hw *hw); s32 txgbe_validate_mac_addr(u8 *mac_addr); s32 txgbe_acquire_swfw_sync(struct txgbe_hw *hw, u32 mask); void txgbe_release_swfw_sync(struct txgbe_hw *hw, u32 mask); +s32 txgbe_set_pcie_master(struct txgbe_hw *hw, bool enable); s32 txgbe_get_san_mac_addr(struct txgbe_hw *hw, u8 *san_mac_addr); s32 txgbe_set_san_mac_addr(struct txgbe_hw *hw, u8 *san_mac_addr); diff --git a/drivers/net/txgbe/base/txgbe_osdep.h b/drivers/net/txgbe/base/txgbe_osdep.h index b62c0b0824..0d9492c3cb 100644 --- a/drivers/net/txgbe/base/txgbe_osdep.h +++ b/drivers/net/txgbe/base/txgbe_osdep.h @@ -19,6 +19,7 @@ #include #include #include +#include #include "../txgbe_logs.h" diff --git a/drivers/net/txgbe/base/txgbe_regs.h b/drivers/net/txgbe/base/txgbe_regs.h index 79290a7afe..86896d11dc 100644 --- a/drivers/net/txgbe/base/txgbe_regs.h +++ b/drivers/net/txgbe/base/txgbe_regs.h @@ -1236,6 +1236,9 @@ enum txgbe_5tuple_protocol { #define TXGBE_TCPTMR 0x000170 #define TXGBE_ITRSEL 0x000180 +#define TXGBE_BMECTL 0x012020 +#define TXGBE_BMEPEND 0x000168 + /* P2V Mailbox */ #define TXGBE_MBMEM(i) (0x005000 + 0x40 * (i)) /* 0-63 */ #define TXGBE_MBCTL(i) (0x000600 + 4 * (i)) /* 0-63 */ diff --git a/drivers/net/txgbe/base/txgbe_type.h b/drivers/net/txgbe/base/txgbe_type.h index 75e839b7de..f52736cae9 100644 --- a/drivers/net/txgbe/base/txgbe_type.h +++ b/drivers/net/txgbe/base/txgbe_type.h @@ -29,6 +29,7 @@ #define TXGBE_FDIRCMD_CMD_POLL 10 #define TXGBE_VF_INIT_TIMEOUT 200 /* Number of retries to clear RSTI */ #define TXGBE_SPI_TIMEOUT 10000 +#define TXGBE_PCI_MASTER_DISABLE_TIMEOUT 800 #define TXGBE_ALIGN 128 /* as intel did */ diff --git a/drivers/net/txgbe/txgbe_ethdev.c b/drivers/net/txgbe/txgbe_ethdev.c index 30a2d6495a..862cdc55cc 100644 --- a/drivers/net/txgbe/txgbe_ethdev.c +++ b/drivers/net/txgbe/txgbe_ethdev.c @@ -600,6 +600,7 @@ eth_txgbe_dev_init(struct rte_eth_dev *eth_dev, void *init_params __rte_unused) hw->hw_addr = (void *)pci_dev->mem_resource[0].addr; /* Vendor and Device ID need to be set before init of shared code */ + hw->back = pci_dev; hw->device_id = pci_dev->id.device_id; hw->vendor_id = pci_dev->id.vendor_id; if (pci_dev->id.subsystem_vendor_id == PCI_VENDOR_ID_WANGXUN) { @@ -1704,6 +1705,8 @@ txgbe_dev_start(struct rte_eth_dev *dev) hw->mac.get_link_status = true; hw->dev_start = true; + txgbe_set_pcie_master(hw, true); + /* workaround for GPIO intr lost when mng_veto bit is set */ if (txgbe_check_reset_blocked(hw)) txgbe_reinit_gpio_intr(hw); @@ -1966,6 +1969,8 @@ txgbe_dev_stop(struct rte_eth_dev *dev) adapter->rss_reta_updated = 0; wr32m(hw, TXGBE_LEDCTL, 0xFFFFFFFF, TXGBE_LEDCTL_SEL_MASK); + txgbe_set_pcie_master(hw, true); + hw->adapter_stopped = true; dev->data->dev_started = 0; hw->dev_start = false; @@ -2048,6 +2053,8 @@ txgbe_dev_close(struct rte_eth_dev *dev) txgbe_dev_free_queues(dev); + txgbe_set_pcie_master(hw, false); + /* reprogram the RAR[0] in case user changed it. */ txgbe_set_rar(hw, 0, hw->mac.addr, 0, true); -- 2.39.2 --- Diff of the applied patch vs upstream commit (please double-check if non-empty: --- --- - 2024-07-15 16:19:37.715791505 +0100 +++ 0062-net-txgbe-fix-Tx-hang-on-queue-disable.patch 2024-07-15 16:19:34.668208598 +0100 @@ -1 +1 @@ -From 8a31f4bbbbb73baf77297f00887f4b267feb7b8e Mon Sep 17 00:00:00 2001 +From a96f251defbcc1501cca4da1b7ed6a44137216c6 Mon Sep 17 00:00:00 2001 @@ -5,0 +6,2 @@ +[ upstream commit 8a31f4bbbbb73baf77297f00887f4b267feb7b8e ] + @@ -17 +18,0 @@ -Cc: stable@dpdk.org @@ -43 +44 @@ -index d19fd0065d..7094551fee 100644 +index de96549ae8..b59667f1d7 100644 @@ -142 +143 @@ -index 4fce355000..62d16a6abb 100644 +index b62c0b0824..0d9492c3cb 100644 @@ -180 +181 @@ -index fa68a5d2ca..121dccb5eb 100644 +index 30a2d6495a..862cdc55cc 100644 @@ -183 +184 @@ -@@ -601,6 +601,7 @@ eth_txgbe_dev_init(struct rte_eth_dev *eth_dev, void *init_params __rte_unused) +@@ -600,6 +600,7 @@ eth_txgbe_dev_init(struct rte_eth_dev *eth_dev, void *init_params __rte_unused) @@ -191 +192 @@ -@@ -1717,6 +1718,8 @@ txgbe_dev_start(struct rte_eth_dev *dev) +@@ -1704,6 +1705,8 @@ txgbe_dev_start(struct rte_eth_dev *dev) @@ -200 +201 @@ -@@ -1980,6 +1983,8 @@ txgbe_dev_stop(struct rte_eth_dev *dev) +@@ -1966,6 +1969,8 @@ txgbe_dev_stop(struct rte_eth_dev *dev) @@ -209 +210 @@ -@@ -2062,6 +2067,8 @@ txgbe_dev_close(struct rte_eth_dev *dev) +@@ -2048,6 +2053,8 @@ txgbe_dev_close(struct rte_eth_dev *dev)