From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2FEAD4639F; Thu, 13 Mar 2025 09:34:36 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5B59940ED6; Thu, 13 Mar 2025 09:34:33 +0100 (CET) Received: from NAM02-DM3-obe.outbound.protection.outlook.com (mail-dm3nam02on2084.outbound.protection.outlook.com [40.107.95.84]) by mails.dpdk.org (Postfix) with ESMTP id CDCEB40EAB for ; Thu, 13 Mar 2025 09:34:31 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Eq6kwREC3KTGI53xFthIBkMZM9tkcwFYNXUwathL3n0SWjKVvAIvZReTLXeSqpCWUgTjYgTqWcExuDKuNO2DUzd0y66yhVG6El6cQUHqx4x8gzQAxpsU54wy5nIbXL7JjfTxZvIbbASFCtMGSLXVwgfnigCHM7eeDPCjyVYOkvhh5SUhKm3d84VTga5PnFXj4hsoKR5F2Co+gKZlezGp31er+cbbwuohqzk44X/7CvnffQk8MO9n3LnEZgpTRNLOlDs/J/Lx74kGG0IGKl3btgNJQpwbcI+KzSub9/CVH4CYqZIxsAr8hmdy8eHKXIVb8d/2xJSw318Hq6maCRJkgQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=11owmUlVn4q/TuUSioT/1Hdq8z5PLHpbx3aPkCjJkbI=; b=gOJ4oPcg6XGcYC6F6G8Epwu19MQr95/rJScW4QYAFHBmsQPdFj9LZsahaYrciQG+1HTsPzkqQ2aVEaPrjyWo8CgfVnnx50u9HabmI5zhWVDy3ezZNCsdAu+rE/KNx1idMme3amWmkvPUMjtKdr3D/0fWtH7p6KT5p9SYILgT2TC9qLABo2TJafDGnFdwwHzfmx/cbSSrP+87pddI7elPCRmJEAQOXdHT5m4JMcMbDcBvgy86+NOkjt4YCpe9ZIDSDF4kpgegshjQXwv9VAoKEimVzGTuOLs+d5ZManZNn5aCl8O40+GHsvH6kT4l0N0fO3waFlhivj3Q67B8A71Iww== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=11owmUlVn4q/TuUSioT/1Hdq8z5PLHpbx3aPkCjJkbI=; b=RvW3l8DbPW7c+7DAA+S92TFUzuvq7kWSyoSNieDkq+yWneSvRcBqtInppKM2n1OaeLyfwYdjLxB9oNFiV5G5nnV07Q8mgc/cFxL97o9A5yi0RY+v4Gwcin3AVK+cXJd0GAEnhrZK7cj18W++Ggl1iQQOT3vxOr0jHU8CCgx94zHRZQiIQCC5girJI7RpRwoPw13j1sSPr+bh2G5NcBrA4t/SLJHDI9ToqoP1VIRsmUpeU78hVJvumUhlw66bYgA4S9HJbEFFRwnYPoDW4RjWh1jhAW7sjvdXSdBE3CHIEy3DcjHG3x7qJA4e/aJXouDqdTOozF6ubV41aUVx5UDqVg== Received: from CH2PR07CA0038.namprd07.prod.outlook.com (2603:10b6:610:5b::12) by SA3PR12MB9105.namprd12.prod.outlook.com (2603:10b6:806:382::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8511.27; Thu, 13 Mar 2025 08:34:28 +0000 Received: from CH1PEPF0000AD7B.namprd04.prod.outlook.com (2603:10b6:610:5b:cafe::d8) by CH2PR07CA0038.outlook.office365.com (2603:10b6:610:5b::12) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8534.21 via Frontend Transport; Thu, 13 Mar 2025 08:34:28 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by CH1PEPF0000AD7B.mail.protection.outlook.com (10.167.244.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8534.20 via Frontend Transport; Thu, 13 Mar 2025 08:34:28 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Thu, 13 Mar 2025 01:34:17 -0700 Received: from nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Thu, 13 Mar 2025 01:34:12 -0700 From: Gregory Etelson To: CC: , =?UTF-8?q?=C2=A0?= , , Michael Baum , Dariusz Sosnowski , Viacheslav Ovsiienko , "Bing Zhao" , Ori Kam , Suanming Mou , Matan Azrad Subject: [PATCH 2/2] net/mlx5: fix GENEVE parser cleanup Date: Thu, 13 Mar 2025 10:33:51 +0200 Message-ID: <20250313083351.25559-2-getelson@nvidia.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250313083351.25559-1-getelson@nvidia.com> References: <20250313083351.25559-1-getelson@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH1PEPF0000AD7B:EE_|SA3PR12MB9105:EE_ X-MS-Office365-Filtering-Correlation-Id: cdd320c8-7177-407a-41cb-08dd6209deb4 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|1800799024|82310400026|36860700013|376014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?dusKPt9+Andp24LEATlH8MOEPdwHKOZd6nkkCihMXtiiPvlhFLfWSxj53k8N?= =?us-ascii?Q?AdjT7JNnoGN1ZP5pN6QBIHhPTn8LIHw43VEQS4bmOl1PHrT+HHSra45kPX81?= =?us-ascii?Q?iz5ifR9HxM6MjLP1CMF1Vnx4TCpbWV7rwSDaW8s64u4O1P6JxtftAlZC33Ay?= =?us-ascii?Q?mDzdNu02OTFqxrra1LTJtnTpuNE18YE5Jz4oekXpkh8J4CcJUh6yxdSRBZSp?= =?us-ascii?Q?2EfErj95UFs0Wzjg7cI6ozi1kVq5mL+iEDtsbhjZm/bZAH+h7qkq18Yui1Hm?= =?us-ascii?Q?jckzreQTCjTWwxMpDdL7+RqZXfLldqBpokJgVk/OFgVKAxDPuL/oBLl1rcUk?= =?us-ascii?Q?ISVGp66gKIh9E0MTiIii6U4t6/LKHiU2x0UCA9yfIyU1gX21drlMM90yTXhl?= =?us-ascii?Q?5QnZLGx7MV6p7i73WavE1BMuR4babDEoebaxFYziy46UTE4nGONZYocZkFxy?= =?us-ascii?Q?/89kmPjgDkSI1MSSYbrkQPBI5lgitfSorHAD2Z/7AcwqCdsN+FvOUGei9cuf?= =?us-ascii?Q?jZlSPt2sHisYK9hIota/c6LOszQMMC7Fw1nJIGueouv62KQbkH/L7TiL/qbx?= =?us-ascii?Q?3uoAkLNRcVY6np6A4f4lILR/CHdZ7uqcUw1mk4hxggGcywhGxcAvQLd+JSCq?= =?us-ascii?Q?6FrdwxfSynyLyKIGYJXgJcbsgpGRZg/9RYl7TnhaeyuDKOlqBFD80jGfEt5B?= =?us-ascii?Q?fRrJvfpXtMg6bzbvUcNudS9SAM6pbhhNP3FhuyVTf1aDzy+OenvjyBrH9EpL?= =?us-ascii?Q?SL4es2eZaJHsB6xIrvK79MKkml1ygARDB53w95NDQyuUsQHZHHwuGeGYj1Xr?= =?us-ascii?Q?RTnN+B7lriTnagBt/uR9245Bi8n5Y8uaNFfg0eI8beUt+oDWpSkk1a/dFYwQ?= =?us-ascii?Q?E28nSHGTMABZTlhw0P7CPT+izgw01uGUJyVQn8r3y1H/V/KFLX8jVpdeCQ0E?= =?us-ascii?Q?H08FWpjcccdu/Tn2sdtXzaawr0Eoc1Ovf0OQlywnn6vF9LAm2t1Q5QOkWZw6?= =?us-ascii?Q?u3K2iMN3kbIC8UyE1S7hqCYZD7g27yC4C079P+3bvWembWZmHsVpERckLZzz?= =?us-ascii?Q?ZCk+19rsboB5DEIWnG+OV7t0qcCZsIH4/svHbLf3yfC8OMaAILjhCqJ++623?= =?us-ascii?Q?u1AiOFl6zC18fXIKxdaQDyhzX6b0Z02Xmw2dpnTCYop54sT1CqcNASmFhN3C?= =?us-ascii?Q?+kd1ws1hfM7Gywf8V136tj7fedkhyMhGzgthv2RhztDX3UlqnoxpnMRDXvXT?= =?us-ascii?Q?k794o8cAk3TzV8Hp7zz80/vPcl9PE1PoNDPdTyFCmVpkpGnqPILrHkDY+6Jv?= =?us-ascii?Q?cSbStWD7BK9yfIvT5rB0oJh3dX6Y2v2NFtfZ5l/chHOOEoLpzFS5FKTTFKQs?= =?us-ascii?Q?HF4gR3Pr7NebJwjIvnr3RQcLEXBWxVV4nByL/XqIPIgfRPWgokt4MRO8ntm/?= =?us-ascii?Q?JWUzPiR++tC0sXmnOigeMpMQJ0cUHRNld/sdLpqQxDuN9drJCsweTw=3D=3D?= X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230040)(1800799024)(82310400026)(36860700013)(376014); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Mar 2025 08:34:28.5498 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: cdd320c8-7177-407a-41cb-08dd6209deb4 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CH1PEPF0000AD7B.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA3PR12MB9105 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Michael Baum The GENEVE parser is shared across ports on the same physical device. It is created once for the first port and increments a reference counter for each additional port. The parser is created using the InfiniBand (IBV) context (CTX) of the first port, and cleanup should use the same context. Previously, if the port owning the context closed while another port still used the parser, the port closure would fail due to the shared parser dependency. This patch addresses the issue by changing the approach: the physical device now creates its own distinct context by importing the context from the first initialized port. This ensures that parser creation and cleanup use a consistent context, independent of individual port lifecycles. Fixes: f5177bdc8b76 ("net/mlx5: add GENEVE TLV options parser API") Cc: michaelba@nvidia.com Signed-off-by: Michael Baum --- drivers/net/mlx5/mlx5.c | 21 ++++++------ drivers/net/mlx5/mlx5.h | 2 +- drivers/net/mlx5/mlx5_flow.h | 2 -- drivers/net/mlx5/mlx5_flow_geneve.c | 50 +++-------------------------- 4 files changed, 18 insertions(+), 57 deletions(-) diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 0f49cb5e5b..a16568ff22 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -1724,6 +1724,16 @@ mlx5_get_physical_device(struct mlx5_common_device *cdev) rte_errno = ENOMEM; return NULL; } + /* + * The same CTX create the physical device objects should destroy them. + * Since we can't be sure it will be done by same CTX, we prepare for + * the physical device a special CTX used by objects creation. + */ + phdev->ctx = mlx5_os_get_physical_device_ctx(cdev); + if (!phdev->ctx) { + mlx5_free(phdev); + return NULL; + } phdev->guid = attr->system_image_guid; phdev->refcnt = 1; LIST_INSERT_HEAD(&phdev_list, phdev, next); @@ -1767,6 +1777,8 @@ mlx5_physical_device_destroy(struct mlx5_physical_device *phdev) return; /* Remove physical device from the global device list. */ LIST_REMOVE(phdev, next); + MLX5_ASSERT(phdev->ctx); + claim_zero(mlx5_glue->close_device(phdev->ctx)); mlx5_free(phdev); } @@ -2323,15 +2335,6 @@ mlx5_dev_close(struct rte_eth_dev *dev) rte_errno = EBUSY; return -EBUSY; } -#ifdef HAVE_MLX5_HWS_SUPPORT - /* Check if shared GENEVE options created on context being closed. */ - ret = mlx5_geneve_tlv_options_check_busy(priv); - if (ret) { - DRV_LOG(ERR, "port %u maintains shared GENEVE TLV options", - dev->data->port_id); - return ret; - } -#endif DRV_LOG(DEBUG, "port %u closing device \"%s\"", dev->data->port_id, sh->ibdev_name); /* diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 6df99c25e2..0194887a8b 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -1527,7 +1527,7 @@ struct mlx5_common_nic_config { */ struct mlx5_physical_device { LIST_ENTRY(mlx5_physical_device) next; - struct mlx5_dev_ctx_shared *sh; /* Created on sherd context. */ + void *ctx; /* CTX for creation of options. */ uint64_t guid; /* System image guid, the uniq ID of physical device. */ struct mlx5_geneve_tlv_options *tlv_options; struct mlx5_common_nic_config config; diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h index a5bde158ca..9816ed9238 100644 --- a/drivers/net/mlx5/mlx5_flow.h +++ b/drivers/net/mlx5/mlx5_flow.h @@ -3469,8 +3469,6 @@ flow_hw_resource_release(struct rte_eth_dev *dev); int mlx5_geneve_tlv_options_destroy(struct mlx5_geneve_tlv_options *options, struct mlx5_physical_device *phdev); -int -mlx5_geneve_tlv_options_check_busy(struct mlx5_priv *priv); void flow_hw_rxq_flag_set(struct rte_eth_dev *dev, bool enable); int flow_dv_action_validate(struct rte_eth_dev *dev, diff --git a/drivers/net/mlx5/mlx5_flow_geneve.c b/drivers/net/mlx5/mlx5_flow_geneve.c index 6bf53e1270..4d57bb763f 100644 --- a/drivers/net/mlx5/mlx5_flow_geneve.c +++ b/drivers/net/mlx5/mlx5_flow_geneve.c @@ -590,8 +590,8 @@ mlx5_geneve_tlv_option_copy(struct rte_pmd_mlx5_geneve_tlv *dst, /** * Create list of GENEVE TLV options according to user configuration list. * - * @param sh - * Shared context the options are being created on. + * @param ctx + * Context returned from mlx5 open_device() glue function. * @param tlv_list * A list of GENEVE TLV options to create parser for them. * @param nb_options @@ -604,7 +604,7 @@ mlx5_geneve_tlv_option_copy(struct rte_pmd_mlx5_geneve_tlv *dst, * NULL otherwise and rte_errno is set. */ static struct mlx5_geneve_tlv_options * -mlx5_geneve_tlv_options_create(struct mlx5_dev_ctx_shared *sh, +mlx5_geneve_tlv_options_create(void *ctx, const struct rte_pmd_mlx5_geneve_tlv tlv_list[], uint8_t nb_options, uint8_t sample_id) { @@ -625,7 +625,7 @@ mlx5_geneve_tlv_options_create(struct mlx5_dev_ctx_shared *sh, } for (i = 0; i < nb_options; ++i) { spec = &tlv_list[i]; - ret = mlx5_geneve_tlv_option_create(sh->cdev->ctx, spec, + ret = mlx5_geneve_tlv_option_create(ctx, spec, &options->options[i], sample_id); if (ret < 0) goto error; @@ -633,8 +633,6 @@ mlx5_geneve_tlv_options_create(struct mlx5_dev_ctx_shared *sh, data_mask = options->buffer + i * MAX_GENEVE_OPTION_DATA_SIZE; mlx5_geneve_tlv_option_copy(&options->spec[i], spec, data_mask); } - MLX5_ASSERT(sh->phdev->sh == NULL); - sh->phdev->sh = sh; options->nb_options = nb_options; options->refcnt = 1; return options; @@ -676,41 +674,9 @@ mlx5_geneve_tlv_options_destroy(struct mlx5_geneve_tlv_options *options, } mlx5_free(options); phdev->tlv_options = NULL; - phdev->sh = NULL; return 0; } -/** - * Check if GENEVE TLV options are hosted on the current port - * and the port can be closed - * - * @param priv - * Device private data. - * - * @return - * 0 on success, a negative EBUSY and rte_errno is set. - */ -int -mlx5_geneve_tlv_options_check_busy(struct mlx5_priv *priv) -{ - struct mlx5_physical_device *phdev = mlx5_get_locked_physical_device(priv); - struct mlx5_dev_ctx_shared *sh = priv->sh; - - if (!phdev || phdev->sh != sh) { - mlx5_unlock_physical_device(); - return 0; - } - if (!sh->phdev->tlv_options || sh->phdev->tlv_options->refcnt == 1) { - /* Mark port as being closed one */ - sh->phdev->sh = NULL; - mlx5_unlock_physical_device(); - return 0; - } - mlx5_unlock_physical_device(); - rte_errno = EBUSY; - return -EBUSY; -} - /** * Validate GENEVE TLV option user request structure. * @@ -955,18 +921,12 @@ mlx5_geneve_tlv_parser_create(uint16_t port_id, rte_errno = EEXIST; return NULL; } - if (phdev->sh == NULL) { - mlx5_unlock_physical_device(); - DRV_LOG(ERR, "GENEVE TLV options are hosted on port being closed."); - rte_errno = EBUSY; - return NULL; - } /* Use existing options. */ options->refcnt++; goto exit; } /* Create GENEVE TLV options for this physical device. */ - options = mlx5_geneve_tlv_options_create(priv->sh, tlv_list, nb_options, sample_id); + options = mlx5_geneve_tlv_options_create(phdev->ctx, tlv_list, nb_options, sample_id); if (!options) { mlx5_unlock_physical_device(); return NULL; -- 2.45.2