From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2118145DAB; Tue, 26 Nov 2024 10:26:15 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id ED7A24025F; Tue, 26 Nov 2024 10:26:14 +0100 (CET) Received: from NAM02-BN1-obe.outbound.protection.outlook.com (mail-bn1nam02on2046.outbound.protection.outlook.com [40.107.212.46]) by mails.dpdk.org (Postfix) with ESMTP id A55FD400EF for ; Tue, 26 Nov 2024 10:26:13 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=uG2TH7uFQaBND9fRBhbTBigJDizPNXSeDC9Xc1GDRKov2otOSoPP92lgAFgYxfUS2jBcbwX4iPwhHRU+4Bvh7PPWjXx6DcMtSvckfpj3MIXwz4ULUjfaOmXPA+8iiASuLliH8+vG8l/FIlH3WQmnVQQ1aGrVYAMWp43micAQUkbjKzQXxblNEZnrN26Xx0z6OuokW55M8O7aah3ZAdGa7bRXncBogZl0fe1Avu1jsmofwXvZ/exyU+WaH56BLAPwMCPEYBhJPrbnaF1DsoJvrIL2suP0HOANLO9gUzo8GS2Pn0J749ItdKfuWo6GUDF/RScxeCj+P5FHef5SqCqckw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=mWmmnF/c7UO9WHXwd//puRyrfo04nt7T2VTi95KjsvY=; b=Hb9nCyth5H5CkUXSBn5A16KII02DTwTV9oO7la9mub1EPueQe2DZwxfSls8YEJvJv8uNA6c3Nqnxf02mRO1cfgX3ik+a/VMHR4bITzDXJrAN+LOGmOSZX8HBhrP90UFjnkf+diBiVV/uNOBARfOYNPFdmePylALnrHu37fe6lFbFd+oMP5Bc/7Op0mtOiul3R8O6ZHyznLUQJ86Gp2Wu4ISI5a9ceaEy5Z5ID72wO13VhkT3yEmwddqntptKtWi62MPGhp25lUhhi1ZE2t45BfCMD4jtmmTJPazFPkdo/WMjX7TccGZ6mGtsW+86CtG4OYBrW25IoTK+J40jSfFgRw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mWmmnF/c7UO9WHXwd//puRyrfo04nt7T2VTi95KjsvY=; b=AM5ackU/wArpQkNAijQSDVpAduuqHWv4/Vbnwb3Ir62oIynoRp7y526GvNLTIj9bPukct+ZWpfSSmpHfY1bjwi2wrcMs7JnJ/sIuGT41J+PakuAZkoA9zXsbN9lOnUfOL5TKAZwiujsux7cyLgoXurNKU4sAFQ2rRa4oeolRshJaLUIh3E4IdcwtTD0s8gqNfKpgcFEhcCNwsxVWBJpGyTSUg1YGm3kpeZiUkELTlldUe+O0gkVIZ0wBUAxRurl4cj8ZzC8JxqwZ2ZoJ1sVGg8xPNAFQiHeeVZF5CCljBeSkNyVN/he4ulr5gDuY0ncdgz4Ae32Gc2rqupVA7QT9XA== Received: from MN0P222CA0012.NAMP222.PROD.OUTLOOK.COM (2603:10b6:208:531::20) by DM4PR12MB8522.namprd12.prod.outlook.com (2603:10b6:8:18f::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8182.18; Tue, 26 Nov 2024 09:26:09 +0000 Received: from MN1PEPF0000F0E0.namprd04.prod.outlook.com (2603:10b6:208:531:cafe::92) by MN0P222CA0012.outlook.office365.com (2603:10b6:208:531::20) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8182.20 via Frontend Transport; Tue, 26 Nov 2024 09:26:09 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by MN1PEPF0000F0E0.mail.protection.outlook.com (10.167.242.38) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8207.12 via Frontend Transport; Tue, 26 Nov 2024 09:26:09 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 26 Nov 2024 01:25:55 -0800 Received: from nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 26 Nov 2024 01:25:53 -0800 From: Bing Zhao To: , , , CC: , , , Subject: [PATCH] net/mlx5: fix the unneeded stub table allocation Date: Tue, 26 Nov 2024 11:25:39 +0200 Message-ID: <20241126092539.307780-1-bingz@nvidia.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN1PEPF0000F0E0:EE_|DM4PR12MB8522:EE_ X-MS-Office365-Filtering-Correlation-Id: 8f85245e-2a8a-485c-3f52-08dd0dfc5cda X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|1800799024|376014|36860700013|82310400026; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?oT0EfaWy5lNGYKbAn+FkVlOgGrjxqE8bYTlhvr0SX0XjhjBIPYEBU/ESPW2d?= =?us-ascii?Q?2UCHZKg5PVY/Dgp3e00H1egz9caf3kc6/QQlqfXhrLD0ngJa5TfoK21M0MmG?= =?us-ascii?Q?jvJjIQQ5/CcQH2rZOPvHksz88kffBcvTgksj/bfqLljqU4sk/x2sjnz4UB/W?= =?us-ascii?Q?RKJGj69ubLbAAJtYNnHBb0ppuutk0oImCIZhzvXQGpb/KISsXRzWp2J+sauv?= =?us-ascii?Q?rCpfa1s766QqXoirI6uK/S4XloD24G4YlSPgqp8byv7s4q6LURVDR+YYn8MN?= =?us-ascii?Q?WpmroXe0opiP0ByVRJSt6RuE2jTBnyH4Ub/fWcAa5pGREMZo8lOaPPNRiFJW?= =?us-ascii?Q?k5tVQIsDVx7/uttfjSI9d1cOsoa3omOh0Qvz9dBJ6wdgWn7MVXrn6JFLsgoD?= =?us-ascii?Q?aphYWbU7TGYMUus1FrxfSEpmTINCeQVSSPixGZdnZ8oR+xgpweaQ7cuhU2M4?= =?us-ascii?Q?7uXagzKxVXNS6xKsy7gOsNGIYCHswCKNynwEqN7RYrNIDR5eBjNN8Es6c7Tm?= =?us-ascii?Q?wqXxdMr9kuZFqtp5nwR9wDhJXXbrsx27vq1hM4KfG5QQItFJIjKgLq7RJcLR?= =?us-ascii?Q?kkPwNnZVVahIKusIikQCUHUncn8Tpek5Vi2sEWNAmq1140VHqkH01AoHphku?= =?us-ascii?Q?Qw2bVHXCtXv9TIGS2zl0fieHvMisted4NHDrSl2KIODr34Ra9Ub3gGCse6zg?= =?us-ascii?Q?FVUtqLkoSgn4WQphtYPtXzC08tN/kv0XaBbjEUh88OAl1rlxbxUkUw2omuTb?= =?us-ascii?Q?PIuN+a35hJ8gVdrHr4MYZZ6urV3Jua9MKjj+ELzElRStFbrJ1sFnNLFn6Yf9?= =?us-ascii?Q?A3rK2IUGjsRiHMnBjsp/cV5d6RMb0KBR36Ik7FhSzvdyCypxH4HYv36R1Wr6?= =?us-ascii?Q?w+nPvNebhKW3JiQP17Al8u8jcE3Y2DAqy9ccTAwtIJSLuabtWlv5xKomQoPH?= =?us-ascii?Q?8LJsIsQhyFuUtzO0T/wU247U/efvH0GewuKql/7plz4P5Bi1LeakLvf/1u3u?= =?us-ascii?Q?jzci9Dv2xm9oGcKbWpy2NkSpo3cCLBb28BAFdwj5ZBB+IG5miryzseYNjruL?= =?us-ascii?Q?A08QINjRDjvk6BvGEtToDBGKH0zl32PpRvYUu2+SEa/ZVhoVsmdSuJ6qA5N5?= =?us-ascii?Q?n7eMU6x09RoN4nOhpVbmpwy2FnDtIZ4kAthi0lprX3Cb0nH8E41cpeQWv0GO?= =?us-ascii?Q?RDRgNak4hiVgYLQ9zeqJpOa9I3Gj0rUBNiWCb+xqWwXr/3htMjLPkipPA+Bd?= =?us-ascii?Q?OUNQGNajn8q/AX2uCXY5UA6yIpWx7+ocXmVeVcO1v6veZWmiIzqM4iu2QzT/?= =?us-ascii?Q?m3FjkCJVneJHuqRQlWF0DIIWWfBdFAkXqmCxarjLFN0daxkVmuoaRhMisJP8?= =?us-ascii?Q?RGK++KgaDaQIs1mCv7Sb8IHtLp/dVyKpeZH5/8sjZg4Yd9y8b1Ox1S79mqcF?= =?us-ascii?Q?8i/sqxbcanXa70r4w0rHJWwS76z4P0XN?= X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230040)(1800799024)(376014)(36860700013)(82310400026); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Nov 2024 09:26:09.5330 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 8f85245e-2a8a-485c-3f52-08dd0dfc5cda X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: MN1PEPF0000F0E0.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB8522 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The HWS non-template flow API is reusing some implementation of template API to unifiy code logic. So for each rule creation, a stub / temporary table is used in order to reuse the actions construction. Since this is temporary and used only internally, there is no need to save the table permanently. Only parts of them are mandatory, so the allocation / free from the heap of RTE memory is a waste and causes a lot of overhead. By using the pre-allocated workspace and set the needed fields expliticly will save the overhead and help to speed up the rule insertion rate. Fixes: 27d171b88031 ("net/mlx5: abstract flow action and enable reconfigure") Cc: mkashani@nvidia.com Signed-off-by: Bing Zhao --- drivers/net/mlx5/mlx5_flow.c | 11 +++++++++-- drivers/net/mlx5/mlx5_flow.h | 3 +++ drivers/net/mlx5/mlx5_flow_hw.c | 19 +++++++++++-------- 3 files changed, 23 insertions(+), 10 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index 16ddd05448..9203643300 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -8270,14 +8270,21 @@ flow_alloc_thread_workspace(void) { size_t data_size = RTE_ALIGN(sizeof(struct mlx5_flow_workspace), sizeof(long)); size_t rss_queue_array_size = sizeof(uint16_t) * RTE_ETH_RSS_RETA_SIZE_512; - struct mlx5_flow_workspace *data = calloc(1, data_size + - rss_queue_array_size); + size_t alloc_size = data_size + rss_queue_array_size; +#ifdef HAVE_MLX5_HWS_SUPPORT + /* Dummy table size for the non-template API. */ + alloc_size += sizeof(struct rte_flow_template_table); +#endif + struct mlx5_flow_workspace *data = calloc(1, alloc_size); if (!data) { DRV_LOG(ERR, "Failed to allocate flow workspace memory."); return NULL; } data->rss_desc.queue = RTE_PTR_ADD(data, data_size); +#ifdef HAVE_MLX5_HWS_SUPPORT + data->table = RTE_PTR_ADD(data->rss_desc.queue, rss_queue_array_size); +#endif return data; } diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h index bcc2782460..757bbf73c1 100644 --- a/drivers/net/mlx5/mlx5_flow.h +++ b/drivers/net/mlx5/mlx5_flow.h @@ -1919,6 +1919,9 @@ struct mlx5_flow_workspace { /* The meter policy used by meter in flow. */ struct mlx5_flow_meter_policy *final_policy; /* The final policy when meter policy is hierarchy. */ +#ifdef HAVE_MLX5_HWS_SUPPORT + struct rte_flow_template_table *table; +#endif uint32_t skip_matcher_reg:1; /* Indicates if need to skip matcher register in translate. */ uint32_t mark:1; /* Indicates if flow contains mark action. */ diff --git a/drivers/net/mlx5/mlx5_flow_hw.c b/drivers/net/mlx5/mlx5_flow_hw.c index 7233ac46c4..f383d87337 100644 --- a/drivers/net/mlx5/mlx5_flow_hw.c +++ b/drivers/net/mlx5/mlx5_flow_hw.c @@ -13517,7 +13517,6 @@ flow_hw_translate_flow_actions(struct rte_eth_dev *dev, int ret = 0; uint32_t src_group = 0; enum mlx5dr_table_type table_type; - struct rte_flow_template_table *table = NULL; struct mlx5_flow_group grp; struct rte_flow_actions_template *at = NULL; struct rte_flow_actions_template_attr template_attr = { @@ -13531,6 +13530,10 @@ flow_hw_translate_flow_actions(struct rte_eth_dev *dev, RTE_SET_USED(action_flags); memset(masks, 0, sizeof(masks)); memset(mask_conf, 0, sizeof(mask_conf)); + /* Only set the needed fields explicitly. */ + struct mlx5_flow_workspace *wks = mlx5_flow_push_thread_workspace(); + struct rte_flow_template_table *table; + /* * Notice All direct actions will be unmasked, * except for modify header and encap, @@ -13540,6 +13543,12 @@ flow_hw_translate_flow_actions(struct rte_eth_dev *dev, * shared actions will be parsed as part of template translation * and not during action construct. */ + if (!wks) + return rte_flow_error_set(error, ENOMEM, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, + "failed to push flow workspace"); + table = wks->table; flow_nta_build_template_mask(actions, masks, mask_conf); /* The group in the attribute translation was done in advance. */ ret = __translate_group(dev, attr, external, attr->group, &src_group, error); @@ -13551,11 +13560,6 @@ flow_hw_translate_flow_actions(struct rte_eth_dev *dev, table_type = MLX5DR_TABLE_TYPE_NIC_TX; else table_type = MLX5DR_TABLE_TYPE_NIC_RX; - /* TODO: consider to reuse the workspace per thread. */ - table = mlx5_malloc(MLX5_MEM_ZERO, sizeof(*table), 0, SOCKET_ID_ANY); - if (!table) - return rte_flow_error_set(error, ENOMEM, RTE_FLOW_ERROR_TYPE_ACTION, - actions, "Failed to allocate dummy table"); at = __flow_hw_actions_template_create(dev, &template_attr, actions, masks, true, error); if (!at) { ret = -rte_errno; @@ -13592,10 +13596,9 @@ flow_hw_translate_flow_actions(struct rte_eth_dev *dev, __flow_hw_action_template_destroy(dev, hw_acts); else __flow_hw_act_data_flush(dev, hw_acts); - if (table) - mlx5_free(table); if (at) mlx5_free(at); + mlx5_flow_pop_thread_workspace(); return ret; } -- 2.34.1