From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C8D66431D5 for ; Sun, 22 Oct 2023 16:25:40 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C0CBD40A71; Sun, 22 Oct 2023 16:25:40 +0200 (CEST) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2075.outbound.protection.outlook.com [40.107.93.75]) by mails.dpdk.org (Postfix) with ESMTP id AFD1840698 for ; Sun, 22 Oct 2023 16:25:38 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=BmxiCTGbiWqkbl4qyWI7YRO53Ti1TfGLZUsLVSGSJqjyx1mIvPsbsLRE2TE2ERJfmr6dDVfetLLivW8/5HDXjmLmNJ0zUjADyj2tV3FInb9+v0WSvfsyrrI6NobPq20wv3TVqM+Ds2M+AJwrMJGA4R8z3TCaiSPFP5Y6I8LpSIT3EDY/ZnStX0rnD0Hs5WYR+pjFO7PetzJ4m60mAKcriwKayMG+kXHaUtuSsQxLsbmjugA8cr6xVXFqaZyGz6LddSibfPA9D1B+jiiUW1hrOuGkkNh7wg3SwWnM1yA6XfXhcvZidZ+/kcXqvyqQWrzWGxOOTy8H1nnbDJ1uSiATVQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=k8GX00qa5syTn2htUSKlrclhM/zG0TM3iUkrgzPjM1k=; b=Tqc/dSmVFfGqOpCb8/O31ya9EyB0ihqcH4+iXC/We8zWc+og4Eu4fkW8seuB8bPmxV7Gg9sg0bmMZfbiTpVc4shqROhkAfzmB+XyPIqn+rvdYePFZpyYD0GBoKzP3MG+6130is4oLY0amzMP81qVEKRVgb5va7P1wEgxS15Sf976AOHONkhoVJHomTSDOYBGB5xVcQyMn2mNX0ApD/hA42OAciqrHPJVrfpD/q68ifqsL44DUfmuMwTJh2BNanammOV2Q6T7lawvnXVFTXrs+ZyK4yQfvtjj5SDVWYHshXkH7yQQ6FZ5mCBzP0XqvRui4J12qflHXwlGJLyxEj0pgg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=trustnetic.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=k8GX00qa5syTn2htUSKlrclhM/zG0TM3iUkrgzPjM1k=; b=UFwZFvuN/aAgP18nhFtm0E4IC2zxsXO7pDpCCY1A1tPuFKPTleOKtztKIj+ziWH28AYb9VAi0hoOhlgR5uSK52rb80scwdXDHTCjPrdeFOcM9zo4BIhmoovjbiuu6KKFGXcL2qtGCar9c36CD9EQYSQk7QCsk1wI1B8IDbUOWvxG8ZQiI2PalJQ4v9jRfb2q/fkoY4/Hh9ZZxTXEwcAQMRWEFn3ZjSQ7GtlSQERy4aEFQBGMU0NQjlobo53M76g8N+vFAV1BUab20+Rtvc0wU5pifJYPUzvhUODNzZ7TVsOHXercmbSwKGIDuSP7wABcI3eKBb2GM6xtSzkpuSYlLQ== Received: from MW4PR04CA0091.namprd04.prod.outlook.com (2603:10b6:303:83::6) by MN2PR12MB4568.namprd12.prod.outlook.com (2603:10b6:208:260::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6907.33; Sun, 22 Oct 2023 14:25:35 +0000 Received: from CO1PEPF000044EF.namprd05.prod.outlook.com (2603:10b6:303:83:cafe::4c) by MW4PR04CA0091.outlook.office365.com (2603:10b6:303:83::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6907.33 via Frontend Transport; Sun, 22 Oct 2023 14:25:35 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by CO1PEPF000044EF.mail.protection.outlook.com (10.167.241.69) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6933.15 via Frontend Transport; Sun, 22 Oct 2023 14:25:35 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Sun, 22 Oct 2023 07:25:25 -0700 Received: from nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Sun, 22 Oct 2023 07:25:24 -0700 From: Xueming Li To: Jiawen Wu CC: dpdk stable Subject: patch 'net/ngbe: fix flow control' has been queued to stable release 22.11.4 Date: Sun, 22 Oct 2023 22:20:56 +0800 Message-ID: <20231022142250.10324-28-xuemingl@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231022142250.10324-1-xuemingl@nvidia.com> References: <20231022142250.10324-1-xuemingl@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail201.nvidia.com (10.129.68.8) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000044EF:EE_|MN2PR12MB4568:EE_ X-MS-Office365-Filtering-Correlation-Id: 4c1b481e-3316-4e24-f891-08dbd30ac19c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: jdmp3YHD3gEXdBcD0QYqUvHCqB7bWbd5hP3sa6nhMksC23gz7KNayUFlHl7+b6qj19IQu+MztGJAaCfp9arnYb6vR2yeaG1zrZceO6J0RKqGj6d41eVzqiB0kzWk5AnCVrI8Ld1rRgWmfiUivD58zQFoeqmsAZXi+6/VoMiRK2eVSIwpuAGhBGhgXYP3LIt4Qa4D34DZ5ebjgN5pcK34p89WYeBhPnfxDIhcqCXPAFk06truJvOn4SV66r7rVSKbAkagwlvhVcsA5sw0UMYwFe2zVGLncNlUa+kziwdirriDoAvw6FYrU7Ac9Ynkh2yaG77hlv3zI6tlgbCIrN9utt/CSCRf9Ocjf4B4fAlvdSPt/iaQdlICCi3Fd9O4BZccG+E+056drvL9l/RtcEOnhsSpp1Jr7P+G3mCd7qUqPKbf9a1szV9SDlqrXOkaZ/WEfG7kQ4rt3zq7+zLo0c+7Zbap3WfU474JCRJ6YSRqlVIg2g/be/1G3JymVHn2Zlj66zkmtZZhLZVppuzbKhWrsecicyb0pdc8ExewpaVKruhs2DcxAABOUe1zEAIfG7qdJMpYcAGtC1gSfLQozEJhuoYt2fk/4A4AT69f+9qC6VYIb8RAhDqjD1wDOJEJ8tC8VHgqJZo4n/p6NgrD2vBuxv4FZNlSy+15tFkvLO9WMrwxubzGn7Wdldl1izkmt5LMl99ONElxITasH4LEb/hmtZkKSv5WBmKsbqezR632CYGsdLryhcIdPSF8irUMHSvlqmUjoJjNolGV9L5O7XrGZQTovfIavSKcLI1JrqeVU88= X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230031)(4636009)(346002)(396003)(39850400004)(376002)(136003)(230922051799003)(451199024)(1800799009)(82310400011)(186009)(64100799003)(36840700001)(40470700004)(46966006)(55016003)(2906002)(36860700001)(316002)(6916009)(70586007)(70206006)(82740400003)(356005)(7636003)(2616005)(26005)(6286002)(16526019)(1076003)(478600001)(7696005)(6666004)(53546011)(966005)(40480700001)(47076005)(336012)(426003)(83380400001)(40460700003)(41300700001)(5660300002)(36756003)(86362001)(4326008)(8936002)(8676002)(4001150100001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Oct 2023 14:25:35.3543 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 4c1b481e-3316-4e24-f891-08dbd30ac19c X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000044EF.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4568 X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Hi, FYI, your patch has been queued to stable release 22.11.4 Note it hasn't been pushed to http://dpdk.org/browse/dpdk-stable yet. It will be pushed if I get no objections before 11/15/23. So please shout if anyone has objections. Also note that after the patch there's a diff of the upstream commit vs the patch applied to the branch. This will indicate if there was any rebasing needed to apply to the stable branch. If there were code changes for rebasing (ie: not only metadata diffs), please double check that the rebase was correctly done. Queued patches are on a temporary branch at: https://git.dpdk.org/dpdk-stable/log/?h=22.11-staging This queued commit can be viewed at: https://git.dpdk.org/dpdk-stable/commit/?h=22.11-staging&id=401b94367fa51b0a31c7f6bee1c86db0dce852d8 Thanks. Xueming Li --- >From 401b94367fa51b0a31c7f6bee1c86db0dce852d8 Mon Sep 17 00:00:00 2001 From: Jiawen Wu Date: Thu, 28 Sep 2023 17:47:50 +0800 Subject: [PATCH] net/ngbe: fix flow control Cc: Xueming Li [ upstream commit d19fa5a1f447244aa961a3cc8c52c9024f9109ab ] Fix flow control high/low water limit. Fixes: f40e9f0e2278 ("net/ngbe: support flow control") Signed-off-by: Jiawen Wu --- drivers/net/ngbe/base/ngbe_type.h | 40 ++++++++++++++ drivers/net/ngbe/ngbe_ethdev.c | 89 +++++++++++++++++++++++++++++++ 2 files changed, 129 insertions(+) diff --git a/drivers/net/ngbe/base/ngbe_type.h b/drivers/net/ngbe/base/ngbe_type.h index 37be288a74..8a7d2cd331 100644 --- a/drivers/net/ngbe/base/ngbe_type.h +++ b/drivers/net/ngbe/base/ngbe_type.h @@ -116,6 +116,46 @@ struct ngbe_fc_info { enum ngbe_fc_mode requested_mode; /* FC mode requested by caller */ }; +/* Flow Control Data Sheet defined values + * Calculation and defines taken from 802.1bb Annex O + */ +/* BitTimes (BT) conversion */ +#define NGBE_BT2KB(BT) (((BT) + (8 * 1024 - 1)) / (8 * 1024)) +#define NGBE_B2BT(BT) ((BT) * 8) + +/* Calculate Delay to respond to PFC */ +#define NGBE_PFC_D 672 + +/* Calculate Cable Delay */ +#define NGBE_CABLE_DC 5556 /* Delay Copper */ + +/* Calculate Interface Delay */ +#define NGBE_PHY_D 12800 +#define NGBE_MAC_D 4096 +#define NGBE_XAUI_D (2 * 1024) + +#define NGBE_ID (NGBE_MAC_D + NGBE_XAUI_D + NGBE_PHY_D) + +/* Calculate Delay incurred from higher layer */ +#define NGBE_HD 6144 + +/* Calculate PCI Bus delay for low thresholds */ +#define NGBE_PCI_DELAY 10000 + +/* Calculate delay value in bit times */ +#define NGBE_DV(_max_frame_link, _max_frame_tc) \ + ((36 * \ + (NGBE_B2BT(_max_frame_link) + \ + NGBE_PFC_D + \ + (2 * NGBE_CABLE_DC) + \ + (2 * NGBE_ID) + \ + NGBE_HD) / 25 + 1) + \ + 2 * NGBE_B2BT(_max_frame_tc)) + +#define NGBE_LOW_DV(_max_frame_tc) \ + (2 * ((2 * NGBE_B2BT(_max_frame_tc) + \ + (36 * NGBE_PCI_DELAY / 25) + 1))) + /* Statistics counters collected by the MAC */ /* PB[] RxTx */ struct ngbe_pb_stats { diff --git a/drivers/net/ngbe/ngbe_ethdev.c b/drivers/net/ngbe/ngbe_ethdev.c index cb643c6eba..87c7b89d57 100644 --- a/drivers/net/ngbe/ngbe_ethdev.c +++ b/drivers/net/ngbe/ngbe_ethdev.c @@ -90,6 +90,7 @@ static int ngbe_dev_misc_interrupt_setup(struct rte_eth_dev *dev); static int ngbe_dev_rxq_interrupt_setup(struct rte_eth_dev *dev); static void ngbe_dev_interrupt_handler(void *param); static void ngbe_configure_msix(struct rte_eth_dev *dev); +static void ngbe_pbthresh_set(struct rte_eth_dev *dev); #define NGBE_SET_HWSTRIP(h, q) do {\ uint32_t idx = (q) / (sizeof((h)->bitmap[0]) * NBBY); \ @@ -1037,6 +1038,7 @@ ngbe_dev_start(struct rte_eth_dev *dev) } hw->mac.setup_pba(hw); + ngbe_pbthresh_set(dev); ngbe_configure_port(dev); err = ngbe_dev_rxtx_start(dev); @@ -2356,6 +2358,93 @@ ngbe_flow_ctrl_set(struct rte_eth_dev *dev, struct rte_eth_fc_conf *fc_conf) return -EIO; } +/* Additional bittime to account for NGBE framing */ +#define NGBE_ETH_FRAMING 20 + +/* + * ngbe_fc_hpbthresh_set - calculate high water mark for flow control + * + * @dv_id: device interface delay + * @pb: packet buffer to calculate + */ +static s32 +ngbe_fc_hpbthresh_set(struct rte_eth_dev *dev) +{ + struct ngbe_hw *hw = ngbe_dev_hw(dev); + struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev); + u32 max_frame_size, tc, dv_id, rx_pb; + s32 kb, marker; + + /* Calculate max LAN frame size */ + max_frame_size = rd32m(hw, NGBE_FRMSZ, NGBE_FRMSZ_MAX_MASK); + tc = max_frame_size + NGBE_ETH_FRAMING; + + /* Calculate delay value for device */ + dv_id = NGBE_DV(tc, tc); + + /* Loopback switch introduces additional latency */ + if (pci_dev->max_vfs) + dv_id += NGBE_B2BT(tc); + + /* Delay value is calculated in bit times convert to KB */ + kb = NGBE_BT2KB(dv_id); + rx_pb = rd32(hw, NGBE_PBRXSIZE) >> 10; + + marker = rx_pb - kb; + + /* It is possible that the packet buffer is not large enough + * to provide required headroom. In this case throw an error + * to user and do the best we can. + */ + if (marker < 0) { + PMD_DRV_LOG(WARNING, "Packet Buffer can not provide enough headroom to support flow control."); + marker = tc + 1; + } + + return marker; +} + +/* + * ngbe_fc_lpbthresh_set - calculate low water mark for flow control + * + * @dv_id: device interface delay + */ +static s32 +ngbe_fc_lpbthresh_set(struct rte_eth_dev *dev) +{ + struct ngbe_hw *hw = ngbe_dev_hw(dev); + u32 max_frame_size, tc, dv_id; + s32 kb; + + /* Calculate max LAN frame size */ + max_frame_size = rd32m(hw, NGBE_FRMSZ, NGBE_FRMSZ_MAX_MASK); + tc = max_frame_size + NGBE_ETH_FRAMING; + + /* Calculate delay value for device */ + dv_id = NGBE_LOW_DV(tc); + + /* Delay value is calculated in bit times convert to KB */ + kb = NGBE_BT2KB(dv_id); + + return kb; +} + +/* + * ngbe_pbthresh_setup - calculate and setup high low water marks + */ +static void +ngbe_pbthresh_set(struct rte_eth_dev *dev) +{ + struct ngbe_hw *hw = ngbe_dev_hw(dev); + + hw->fc.high_water = ngbe_fc_hpbthresh_set(dev); + hw->fc.low_water = ngbe_fc_lpbthresh_set(dev); + + /* Low water marks must not be larger than high water marks */ + if (hw->fc.low_water > hw->fc.high_water) + hw->fc.low_water = 0; +} + int ngbe_dev_rss_reta_update(struct rte_eth_dev *dev, struct rte_eth_rss_reta_entry64 *reta_conf, -- 2.25.1 --- Diff of the applied patch vs upstream commit (please double-check if non-empty: --- --- - 2023-10-22 22:17:35.440993700 +0800 +++ 0027-net-ngbe-fix-flow-control.patch 2023-10-22 22:17:34.176723700 +0800 @@ -1 +1 @@ -From d19fa5a1f447244aa961a3cc8c52c9024f9109ab Mon Sep 17 00:00:00 2001 +From 401b94367fa51b0a31c7f6bee1c86db0dce852d8 Mon Sep 17 00:00:00 2001 @@ -4,0 +5,3 @@ +Cc: Xueming Li + +[ upstream commit d19fa5a1f447244aa961a3cc8c52c9024f9109ab ] @@ -9 +11,0 @@ -Cc: stable@dpdk.org @@ -69 +71 @@ -index af77081d9a..a83fdc002a 100644 +index cb643c6eba..87c7b89d57 100644 @@ -88 +90 @@ -@@ -2386,6 +2388,93 @@ ngbe_flow_ctrl_set(struct rte_eth_dev *dev, struct rte_eth_fc_conf *fc_conf) +@@ -2356,6 +2358,93 @@ ngbe_flow_ctrl_set(struct rte_eth_dev *dev, struct rte_eth_fc_conf *fc_conf)