From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id CFFF2457A1 for ; Mon, 12 Aug 2024 15:02:15 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CB13040666; Mon, 12 Aug 2024 15:02:15 +0200 (CEST) Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2056.outbound.protection.outlook.com [40.107.223.56]) by mails.dpdk.org (Postfix) with ESMTP id 9EBE8402C3 for ; Mon, 12 Aug 2024 15:02:14 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=GHLx/+LQz0AI/HfEoFl7ztvfMM1K/MsNZgtzy96GUr+501nSJTa5er06hkmbiDXyQpjsFgHROIIpCrFfhiH7je7mpYSlN3xeA3rvkQv2R/m04WhGChqfcNiCvxQznLSkCnSwAScXFOw9ZgLzrc8epmXJAIJdCp0R7xiHVKOySAXAx9EaUPdctkoE4vyDgR7f5mIogsd2bIEiOOmq14WkstW9RD7Jq8lqAIYMAaA6mIgTvqJoLsENGLnksg3nNy+cFvDpAbFD2gNthB3lsv2dNVAxrTGWDa8Lpf0h+r4wy1vBHuEZnxocTzVGgp9NylT0fnOJ6ubg1JM5x8WJosBZFA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=eNlmzsvP5aEQmPnfAWOQqkXhpJdxgQup/mWYdAfxmyg=; b=pJVbQck5B1WS4WZUN5iX55xKpz0YGqHTzxGXKi1khuk40fe3N5nP7eiE3FsPFB5Efr60OOAO3ue052+WYSzZqjspTE/HPX/NNY1g2taedfrw9/elEKcSSmcfobEkOQRdWhXdefhnXPvL158C+H+qcSwxkwWZjXNi3m0NPDW8NZ1KwFijE57Wi7OaH83ObFjbxxKiE/+Y+LVdAzSmwNGeWD2gukBoJjQifo45kF3t6W0o7GYOmuho0sJ3nHlnzJdw8VyAgG9bhHBfrqkJT+IaBpLjj3AJWGd2PeO80oT/kMh8KyOTaps7i8HCRoxnL9eghvRs3zz4M75zu0E0CbN55A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=eNlmzsvP5aEQmPnfAWOQqkXhpJdxgQup/mWYdAfxmyg=; b=aBUKEJ25hguf78TD8iq5huq6vNQHEUx8U4F2NmLyqw+yPo65GbRzbukzrW0KdzImGJjoyfwXY04UNKxa06mTlTWciMnu1MxQU5/3PHiwpdK+5hCh4ADp/vRAkAx1/3CoKu10WlSj23fCy+3ymND9ogUFKYusTCcniiRJrsLtzDpd2Olr9yoVMDm9UAzZeAoSHJFScdMRAcffBlPAdYUshwjwWHeXyNe+wiS3z1w5R8rcB2hgOy85+WZzwcnZ3s5Iz5t6vU4leTVHQeb8Hss11jjM4tZ+Er07W090ei7HLYSPg+FWn4mtDRS7zo+acb+wjV8KvT5t+QFx7adOFsnQvw== Received: from SN7PR04CA0092.namprd04.prod.outlook.com (2603:10b6:806:122::7) by SA0PR12MB7464.namprd12.prod.outlook.com (2603:10b6:806:24b::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.22; Mon, 12 Aug 2024 13:02:11 +0000 Received: from SN1PEPF0002636E.namprd02.prod.outlook.com (2603:10b6:806:122:cafe::e6) by SN7PR04CA0092.outlook.office365.com (2603:10b6:806:122::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.20 via Frontend Transport; Mon, 12 Aug 2024 13:02:11 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by SN1PEPF0002636E.mail.protection.outlook.com (10.167.241.139) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.8 via Frontend Transport; Mon, 12 Aug 2024 13:02:10 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Mon, 12 Aug 2024 06:01:46 -0700 Received: from nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Mon, 12 Aug 2024 06:01:44 -0700 From: Xueming Li To: Jiawei Wang CC: , Bing Zhao , Viacheslav Ovsiienko , dpdk stable Subject: patch 'net/mlx5: fix shared Rx queue data access race' has been queued to stable release 23.11.2 Date: Mon, 12 Aug 2024 20:49:54 +0800 Message-ID: <20240812125035.389667-118-xuemingl@nvidia.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240812125035.389667-1-xuemingl@nvidia.com> References: <20240712110153.309690-23-xuemingl@nvidia.com> <20240812125035.389667-1-xuemingl@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail201.nvidia.com (10.129.68.8) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN1PEPF0002636E:EE_|SA0PR12MB7464:EE_ X-MS-Office365-Filtering-Correlation-Id: ab6ba2aa-dd0f-404e-b346-08dcbacefa96 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|1800799024|82310400026|36860700013|376014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?kxhQfkA2xH2WifZHmXK4SyMgX9XuhJAxrtJL3pOKwV8mqHYcgc6DHWeI03CR?= =?us-ascii?Q?LQ6/zo3241mUpxkmQ2bOl3XMUa02gzhKHC6E/2Wm02II6OOWqZv37HQUED5r?= =?us-ascii?Q?wSw8acZewOFLDkkT3cVTgI4SHNeHYJB3teazpDdx0v12DS5ke1VyvzFxT/ow?= =?us-ascii?Q?eCWEAqgFKeNpm1owPiFM+qIgan+MPfPGc+77UV3iHWYarxhyyvXZH+o7n8CS?= =?us-ascii?Q?9qv5seB7V6aBOaM5ObuRvqmK2xIe2aUAWD4txC3YnzhSiZuh310Lh3kFvw5j?= =?us-ascii?Q?jA83qL/atgGuehnh0E44gClCWmKtWyhJzNJE1s14/N1YPSaA5SLgcP0vWJYI?= =?us-ascii?Q?maFsQPu6PuaYBjoQCxrQWysWu14nfvCMY1QnEHdWYaC5gQVNSaZdJviyM8tY?= =?us-ascii?Q?YaWZVa6rd+I2TRBTlg6HjC5+S7djWe1BG3ZmUJU6VtT1c0PSVx7VD0jtwSRb?= =?us-ascii?Q?ORQV1oiBoxJD5OCdqOOpPlQad5uw3n9GtgbQdyEPBF2n0eL0Qo8ByOTVnQxM?= =?us-ascii?Q?Cao2Zy1/3OkVl2b/rvjW/luMMcfohgrjtM1tbpfgzRkU6tZH9de6m9teuzZn?= =?us-ascii?Q?Hp8+WAGG2SWpzkfc/HRojgVIp5E30tjhevjYfMQBA8xizRYYsronKiJJFRNO?= =?us-ascii?Q?2we6UdbvGR/BDQZ3ZHvSmIlhQQFHi07Tino1XB1I4G1Cm5Q0y4uRdApfyr35?= =?us-ascii?Q?x4hg1OvdCcXHLF/zlVRANyjcljKn5r5f/HvzNTjGa10t3bkaKZ2+OacLShZ4?= =?us-ascii?Q?bLhb3Pd/S7BS9SAMK8Au/OAXl4Wav845iLx7PHZqJtBsHvO28J+tsb9gNkqR?= =?us-ascii?Q?t8Fy4PyPv/v7vu5MNH1E0PQtcJMPx5FdJ9Md8/N+2Wv//fdgVqvxstowADyi?= =?us-ascii?Q?0t9xoBN/zM5XTpJ4A0xcWJTKQe+UMcVRTgjO5+lwEsxcTJcHWrJNivvoMrXI?= =?us-ascii?Q?JxlKUA92xdrFqkMiMdv53VrrF+CDk3pcTY5AwILdI6RsaDP1HcF0HCLj4hls?= =?us-ascii?Q?heA4he3T2NZ3FPZjUHuTMCyjs9zVH2axle+wCooXtBds76iloQKMLBHqS5Ft?= =?us-ascii?Q?fSZEwmwIXBhO2nx5/qEVEGT6+PO8anJHEYrmTsX5jx3heSz2TtSomtVXYCQU?= =?us-ascii?Q?dNt9TwMwR2V3WWIu10LsNOIct0oPRIFAWdnKpTSQ3ZMzFzXLKxBNWO/o0XUl?= =?us-ascii?Q?lMc6erH+pgir4qhpx91htDAask+xMHmv0iVH1vCYnmuJ6f6ZP8/GJad5Jvz2?= =?us-ascii?Q?XaTA3MVT+3W724Fb5sVIC7RGLruTvJ/jGoW4I+fLzH82/lczVMyGm4Eg0O7Q?= =?us-ascii?Q?OPJA3w7CWk6cchTmjOHH50SAuNVyoYs9Xw7L1fdimto4FtcL6BK996Tmb3md?= =?us-ascii?Q?kP1lB4hl19s5XNG+OhT0EAxfltE344KfkDBMT/zm5vCE2PdOog=3D=3D?= X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230040)(1800799024)(82310400026)(36860700013)(376014); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Aug 2024 13:02:10.8414 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ab6ba2aa-dd0f-404e-b346-08dcbacefa96 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SN1PEPF0002636E.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB7464 X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Hi, FYI, your patch has been queued to stable release 23.11.2 Note it hasn't been pushed to http://dpdk.org/browse/dpdk-stable yet. It will be pushed if I get no objections before 08/14/24. So please shout if anyone has objections. Also note that after the patch there's a diff of the upstream commit vs the patch applied to the branch. This will indicate if there was any rebasing needed to apply to the stable branch. If there were code changes for rebasing (ie: not only metadata diffs), please double check that the rebase was correctly done. Queued patches are on a temporary branch at: https://git.dpdk.org/dpdk-stable/log/?h=23.11-staging This queued commit can be viewed at: https://git.dpdk.org/dpdk-stable/commit/?h=23.11-staging&id=a8331ab8b2e9d930af3dfc94d723f8deaad22be9 Thanks. Xueming Li --- >From a8331ab8b2e9d930af3dfc94d723f8deaad22be9 Mon Sep 17 00:00:00 2001 From: Jiawei Wang Date: Fri, 5 Jul 2024 16:05:46 +0300 Subject: [PATCH] net/mlx5: fix shared Rx queue data access race Cc: Xueming Li [ upstream commit 2d8763432add2efd8b10b7718d41181a2633c733 ] The rxq_data resources were shared for shared Rx queue with the same group and queue ID. The cq_ci:24 of rxq_data was unalignment with other fields in the one 32-bit data, like the dynf_meta and delay_drop. 32bit: xxxx xxxI IIII IIII IIII IIII IIII IIIx ^ .... .... .... .... ...^ | cq_ci | The issue is that while the control thread updates the dynf_meta:1 or delay_drop:1 value during port start, another data thread updates the cq_ci at the same time, it causes the bytes race condition with different thread, and cq_ci value may be overwritten and updated the abnormal value into HW CQ DB. This patch separates the cq_ci from the configuration data spaces, and adds checking for delay_drop and dynf_meta if shared Rx queue if started. Fixes: 02a6195cbeaa ("net/mlx5: support enhanced CQE compression in Rx burst") Signed-off-by: Jiawei Wang Acked-by: Bing Zhao Acked-by: Viacheslav Ovsiienko --- drivers/net/mlx5/mlx5_devx.c | 3 ++- drivers/net/mlx5/mlx5_flow.c | 24 +++++++++++++----------- drivers/net/mlx5/mlx5_rx.h | 4 ++-- 3 files changed, 17 insertions(+), 14 deletions(-) diff --git a/drivers/net/mlx5/mlx5_devx.c b/drivers/net/mlx5/mlx5_devx.c index 9fa400fc48..4f08ddf899 100644 --- a/drivers/net/mlx5/mlx5_devx.c +++ b/drivers/net/mlx5/mlx5_devx.c @@ -592,7 +592,8 @@ mlx5_rxq_devx_obj_new(struct mlx5_rxq_priv *rxq) DRV_LOG(ERR, "Failed to create CQ."); goto error; } - rxq_data->delay_drop = priv->config.std_delay_drop; + if (!rxq_data->shared || !rxq_ctrl->started) + rxq_data->delay_drop = priv->config.std_delay_drop; /* Create RQ using DevX API. */ ret = mlx5_rxq_create_devx_rq_resources(rxq); if (ret) { diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index 85b9ae3901..fdc7c3ea54 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -1953,18 +1953,20 @@ mlx5_flow_rxq_dynf_set(struct rte_eth_dev *dev) if (rxq == NULL || rxq->ctrl == NULL) continue; data = &rxq->ctrl->rxq; - if (!rte_flow_dynf_metadata_avail()) { - data->dynf_meta = 0; - data->flow_meta_mask = 0; - data->flow_meta_offset = -1; - data->flow_meta_port_mask = 0; - } else { - data->dynf_meta = 1; - data->flow_meta_mask = rte_flow_dynf_metadata_mask; - data->flow_meta_offset = rte_flow_dynf_metadata_offs; - data->flow_meta_port_mask = priv->sh->dv_meta_mask; + if (!data->shared || !rxq->ctrl->started) { + if (!rte_flow_dynf_metadata_avail()) { + data->dynf_meta = 0; + data->flow_meta_mask = 0; + data->flow_meta_offset = -1; + data->flow_meta_port_mask = 0; + } else { + data->dynf_meta = 1; + data->flow_meta_mask = rte_flow_dynf_metadata_mask; + data->flow_meta_offset = rte_flow_dynf_metadata_offs; + data->flow_meta_port_mask = priv->sh->dv_meta_mask; + } + data->mark_flag = mark_flag; } - data->mark_flag = mark_flag; } } diff --git a/drivers/net/mlx5/mlx5_rx.h b/drivers/net/mlx5/mlx5_rx.h index 2fce908499..d0ceae72ea 100644 --- a/drivers/net/mlx5/mlx5_rx.h +++ b/drivers/net/mlx5/mlx5_rx.h @@ -101,14 +101,14 @@ struct mlx5_rxq_data { unsigned int shared:1; /* Shared RXQ. */ unsigned int delay_drop:1; /* Enable delay drop. */ unsigned int cqe_comp_layout:1; /* CQE Compression Layout*/ - unsigned int cq_ci:24; + uint16_t port_id; volatile uint32_t *rq_db; volatile uint32_t *cq_db; - uint16_t port_id; uint32_t elts_ci; uint32_t rq_ci; uint16_t consumed_strd; /* Number of consumed strides in WQE. */ uint32_t rq_pi; + uint32_t cq_ci:24; uint16_t rq_repl_thresh; /* Threshold for buffer replenishment. */ uint32_t byte_mask; union { -- 2.34.1 --- Diff of the applied patch vs upstream commit (please double-check if non-empty: --- --- - 2024-08-12 20:44:06.209327705 +0800 +++ 0117-net-mlx5-fix-shared-Rx-queue-data-access-race.patch 2024-08-12 20:44:02.465069367 +0800 @@ -1 +1 @@ -From 2d8763432add2efd8b10b7718d41181a2633c733 Mon Sep 17 00:00:00 2001 +From a8331ab8b2e9d930af3dfc94d723f8deaad22be9 Mon Sep 17 00:00:00 2001 @@ -4,0 +5,3 @@ +Cc: Xueming Li + +[ upstream commit 2d8763432add2efd8b10b7718d41181a2633c733 ] @@ -26 +28,0 @@ -Cc: stable@dpdk.org @@ -38 +40 @@ -index 7db271acb4..8ebe784000 100644 +index 9fa400fc48..4f08ddf899 100644 @@ -41 +43 @@ -@@ -684,7 +684,8 @@ mlx5_rxq_devx_obj_new(struct mlx5_rxq_priv *rxq) +@@ -592,7 +592,8 @@ mlx5_rxq_devx_obj_new(struct mlx5_rxq_priv *rxq) @@ -52 +54 @@ -index 5b3f2b9119..72fb3a55ba 100644 +index 85b9ae3901..fdc7c3ea54 100644 @@ -55 +57 @@ -@@ -1853,18 +1853,20 @@ mlx5_flow_rxq_dynf_set(struct rte_eth_dev *dev) +@@ -1953,18 +1953,20 @@ mlx5_flow_rxq_dynf_set(struct rte_eth_dev *dev) @@ -88 +90 @@ -index 1485556d89..7d144921ab 100644 +index 2fce908499..d0ceae72ea 100644 @@ -91 +93 @@ -@@ -101,14 +101,14 @@ struct __rte_cache_aligned mlx5_rxq_data { +@@ -101,14 +101,14 @@ struct mlx5_rxq_data {