From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4FF6242E2D; Sun, 9 Jul 2023 16:21:13 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CC27B410FB; Sun, 9 Jul 2023 16:21:12 +0200 (CEST) Received: from NAM02-BN1-obe.outbound.protection.outlook.com (mail-bn1nam02on2047.outbound.protection.outlook.com [40.107.212.47]) by mails.dpdk.org (Postfix) with ESMTP id B796C410FA for ; Sun, 9 Jul 2023 16:21:10 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=T+6prqbUqcn4fJBSEFnr3ZuG9UbRzb9DHFXYthQMEnRRm1dabSRNNBsANKctBCfZbhnbBpZbSbiZSBrsnN/GMnuapvafZPtIyl0syJnmMfxBL8IdLXQizsMRG0OWDt9SdtPNMz7jz/SFTqE6TYS0cOINNqVRmeD1Yo+pO+kqm6k1h0mlqUI3LZ//buAdxTa2ohLlcIbu1SUtegGOmZMJWjCt6aVfIKr4f41hWNZAPhtLZbr1CCe/U2JCC6UFc1vT3BQCwvQQ7oz1wk4/4K/+jw0oo3H+lZRDxH3ledm+OToiQXVfaDOj1Lk+zEKDPqWXPoWEWSaSUXv8wTAIoP0qwQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=d669zg6Mv1MCjHsYx81Xo4b4A2FI/+4L3Zk33F+6OOc=; b=DqiwC+mT0XOhjqgSq5dy+RXyOurOi6Rnq1sB3iZFGnQLQ5JAxa3mqPUb8GhO3JgvGSC7mDgJbMsXctvzqfDKDnEqgKIdk+9kiHKCFegOYmTFDAZeS3L2zkKndxeBJxoHm2Ph+EHu8QiECwjzNt6u+X4KQsCHIhFvQrX9Aimg5vDMquGgSHZKVv/SifWb+TbvJ73RmKodRjYOPOaD1YOGTijLN3i+qYoJNa6pVeSvOpT9qz22v3a+Ee/61+mF1FJrMk18re1a35K/BKxRfqwyeihcq2FQKr5uTWD0Umaiu1Y+d5n2UWTCPHnr3YLU4dxw6OPHl6BxrwASLPtAKe41GA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=monjalon.net smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=d669zg6Mv1MCjHsYx81Xo4b4A2FI/+4L3Zk33F+6OOc=; b=T+wIhBA4TcNA8svcHlv48HwmMDUeyJEtuYtag4gRYp7kvNB/xaWTg9FzLny14vywzF6aNMehNkHXlwmTn9CGI1JVBPA6Nk1zJF4uUCXGXir90k1FzwhzMvWstnmUmEMhw5kVOOlUV9Y1PJmKo5sVbwMlkum/Aam3i30cmLk+pwmDwdv5xulgzJxxXQQFJPnJhzSjGv/7Hf5NTUCVCsziWpjQTkegSHlxPIprcMlxG2VJ9rbqCrXJDV5LLrwPD+4eoCoG9aLktAgVr7YPcfIEYzKIEAtTeZoBcRP5bLvXBXj08B57k8ctw/lzBlSX+Np0zCIUE2gF+9Rf//rmjervHw== Received: from BN9PR03CA0766.namprd03.prod.outlook.com (2603:10b6:408:13a::21) by CYYPR12MB8962.namprd12.prod.outlook.com (2603:10b6:930:c4::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6565.26; Sun, 9 Jul 2023 14:21:07 +0000 Received: from BN8NAM11FT115.eop-nam11.prod.protection.outlook.com (2603:10b6:408:13a:cafe::aa) by BN9PR03CA0766.outlook.office365.com (2603:10b6:408:13a::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6565.30 via Frontend Transport; Sun, 9 Jul 2023 14:21:07 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by BN8NAM11FT115.mail.protection.outlook.com (10.13.177.151) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6588.18 via Frontend Transport; Sun, 9 Jul 2023 14:21:07 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.5; Sun, 9 Jul 2023 07:20:52 -0700 Received: from nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.37; Sun, 9 Jul 2023 07:20:49 -0700 From: Alex Vesker To: , , , , Matan Azrad , Ori Kam CC: Subject: [PATCH] net/mlx5/hws: add support for multi pattern Date: Sun, 9 Jul 2023 17:20:28 +0300 Message-ID: <20230709142028.16892-1-valex@nvidia.com> X-Mailer: git-send-email 2.18.1 MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT115:EE_|CYYPR12MB8962:EE_ X-MS-Office365-Filtering-Correlation-Id: cdc4f6ad-914a-4676-3487-08db8087bc98 X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 2HZn/UP73P/Zot9J1P021KPJvtsc56CM9YyilHpK5uyTacvohxAJgMRVz2EfhzdDqIhpkD6k0ogIFf0+I11MeMt6IfrZndSOFekziyCrxBtiuvzLUr0ZeuM2/UlA0hMg10CqxqluZJ0Zq9jruPE20qXZBNTtnntKUT2JN/ZBLNx2YVaFyAuKDl+DkTAlkeQVzOpDdV/Rwn/tAy30Kjp8YU9+r7MlkLppRpJUCNl5lL3KfED/t59GyFGQHo45/3Kt04BjCl37pfNDRvx6dp/QAclSL+DiqUckc+nX+H5n1hUjUyZqTdQV5Fj2hy2Oqz/lsIWyfVW5eczvHwNQOl/GdiPoRvaRXeu58JPBmPVT48juBLEgx1LA62QTSJw3OtU/RSmy8h7Oirx95PJDGBzFXqZBl9f20BZ26JfEAsHPD+8k5Ev6GI8liNXAOmtCf0BEYMmGURN9OzytRTpWP3NmLuou5AvuSarWq7LDFUkV0lop1hh6qXieL5xBLiwlBQar2bcni5BZel0iajWrWmZlYCoq0/9sN+nRuKDVQfDeE2fOSRWjkggTUJJWP9lGxhwYsN7VDey81XcR3+wWJgfIU8deum7eHQiaoFCkF49eRwftrTKOxQHUreLeoV+3vQf23vEm+8Z3cShcyVAaqXcfH34pb3OV2LcV6QzcQ8Db3jTj++lQ4bJxeveKtJ9lcvlFj918v3biMXXR2FQyTaSbC9tL4HH+TeGxfuI+l6DT6IgJhXJDBobl+Usfn5JzIhCC X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230028)(4636009)(396003)(376002)(346002)(39860400002)(136003)(451199021)(40470700004)(46966006)(36840700001)(41300700001)(5660300002)(110136005)(40480700001)(55016003)(30864003)(316002)(8676002)(8936002)(2906002)(7696005)(4326008)(6636002)(70586007)(70206006)(6666004)(40460700003)(36756003)(478600001)(426003)(336012)(86362001)(6286002)(16526019)(186003)(82740400003)(82310400005)(2616005)(356005)(7636003)(83380400001)(36860700001)(1076003)(26005)(47076005)(579004); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Jul 2023 14:21:07.4057 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: cdc4f6ad-914a-4676-3487-08db8087bc98 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT115.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYYPR12MB8962 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org When creating an action it can contain a bulk of objects, but the objects are fixed to a specific pattern and cannot be reused. This can lead to inefficient usage of the HW resources, This support allows creating multiple patterns over a single mlx5dr action. Signed-off-by: Alex Vesker Reviewed-by: Erez Shitrit Acked-by: Matan Azrad --- drivers/net/mlx5/hws/mlx5dr.h | 45 ++- drivers/net/mlx5/hws/mlx5dr_action.c | 538 +++++++++++++++----------- drivers/net/mlx5/hws/mlx5dr_action.h | 8 +- drivers/net/mlx5/hws/mlx5dr_pat_arg.c | 252 +++++------- drivers/net/mlx5/hws/mlx5dr_pat_arg.h | 34 +- drivers/net/mlx5/mlx5_flow_hw.c | 11 +- 6 files changed, 483 insertions(+), 405 deletions(-) diff --git a/drivers/net/mlx5/hws/mlx5dr.h b/drivers/net/mlx5/hws/mlx5dr.h index ec2230d136..6b96a120f7 100644 --- a/drivers/net/mlx5/hws/mlx5dr.h +++ b/drivers/net/mlx5/hws/mlx5dr.h @@ -156,8 +156,21 @@ struct mlx5dr_devx_obj { uint32_t id; }; -/* In actions that take offset, the offset is unique, and the user should not - * reuse the same index because data changing is not atomic. +struct mlx5dr_action_reformat_header { + size_t sz; + void *data; +}; + +struct mlx5dr_action_mh_pattern { + /* Byte size of modify actions provided by "data" */ + size_t sz; + /* PRM format modify actions pattern */ + __be64 *data; +}; + +/* In actions that take offset, the offset is unique, pointing to a single + * resource and the user should not reuse the same index because data changing + * is not atomic. */ struct mlx5dr_rule_action { struct mlx5dr_action *action; @@ -172,11 +185,13 @@ struct mlx5dr_rule_action { struct { uint32_t offset; + uint8_t pattern_idx; uint8_t *data; } modify_header; struct { uint32_t offset; + uint8_t hdr_idx; uint8_t *data; } reformat; @@ -481,12 +496,12 @@ mlx5dr_action_create_counter(struct mlx5dr_context *ctx, * The context in which the new action will be created. * @param[in] reformat_type * Type of reformat prefixed with MLX5DR_ACTION_TYP_REFORMAT. - * @param[in] data_sz - * Size in bytes of data. - * @param[in] inline_data - * Header data array in case of inline action. + * @param[in] num_of_hdrs + * Number of provided headers in "hdrs" array. + * @param[in] hdrs + * Headers array containing header information. * @param[in] log_bulk_size - * Number of unique values used with this pattern. + * Number of unique values used with this reformat. * @param[in] flags * Action creation flags. (enum mlx5dr_action_flags) * @return pointer to mlx5dr_action on success NULL otherwise. @@ -494,8 +509,8 @@ mlx5dr_action_create_counter(struct mlx5dr_context *ctx, struct mlx5dr_action * mlx5dr_action_create_reformat(struct mlx5dr_context *ctx, enum mlx5dr_action_type reformat_type, - size_t data_sz, - void *inline_data, + uint8_t num_of_hdrs, + struct mlx5dr_action_reformat_header *hdrs, uint32_t log_bulk_size, uint32_t flags); @@ -503,10 +518,10 @@ mlx5dr_action_create_reformat(struct mlx5dr_context *ctx, * * @param[in] ctx * The context in which the new action will be created. - * @param[in] pattern_sz - * Byte size of the pattern array. - * @param[in] pattern - * PRM format modify pattern action array. + * @param[in] num_of_patterns + * Number of provided patterns in "patterns" array. + * @param[in] patterns + * Patterns array containing pattern information. * @param[in] log_bulk_size * Number of unique values used with this pattern. * @param[in] flags @@ -515,8 +530,8 @@ mlx5dr_action_create_reformat(struct mlx5dr_context *ctx, */ struct mlx5dr_action * mlx5dr_action_create_modify_header(struct mlx5dr_context *ctx, - size_t pattern_sz, - __be64 pattern[], + uint8_t num_of_patterns, + struct mlx5dr_action_mh_pattern *patterns, uint32_t log_bulk_size, uint32_t flags); diff --git a/drivers/net/mlx5/hws/mlx5dr_action.c b/drivers/net/mlx5/hws/mlx5dr_action.c index 920099ba5b..48fb6d3eaa 100644 --- a/drivers/net/mlx5/hws/mlx5dr_action.c +++ b/drivers/net/mlx5/hws/mlx5dr_action.c @@ -529,7 +529,7 @@ static void mlx5dr_action_fill_stc_attr(struct mlx5dr_action *action, } else { attr->action_type = MLX5_IFC_STC_ACTION_TYPE_ACC_MODIFY_LIST; attr->modify_header.arg_id = action->modify_header.arg_obj->id; - attr->modify_header.pattern_id = action->modify_header.pattern_obj->id; + attr->modify_header.pattern_id = action->modify_header.pat_obj->id; } break; case MLX5DR_ACTION_TYP_TBL: @@ -705,11 +705,13 @@ mlx5dr_action_is_hws_flags(uint32_t flags) } static struct mlx5dr_action * -mlx5dr_action_create_generic(struct mlx5dr_context *ctx, - uint32_t flags, - enum mlx5dr_action_type action_type) +mlx5dr_action_create_generic_bulk(struct mlx5dr_context *ctx, + uint32_t flags, + enum mlx5dr_action_type action_type, + uint8_t bulk_sz) { struct mlx5dr_action *action; + int i; if (!mlx5dr_action_is_root_flags(flags) && !mlx5dr_action_is_hws_flags(flags)) { @@ -725,20 +727,30 @@ mlx5dr_action_create_generic(struct mlx5dr_context *ctx, return NULL; } - action = simple_calloc(1, sizeof(*action)); + action = simple_calloc(bulk_sz, sizeof(*action)); if (!action) { DR_LOG(ERR, "Failed to allocate memory for action [%d]", action_type); rte_errno = ENOMEM; return NULL; } - action->ctx = ctx; - action->flags = flags; - action->type = action_type; + for (i = 0; i < bulk_sz; i++) { + action[i].ctx = ctx; + action[i].flags = flags; + action[i].type = action_type; + } return action; } +static struct mlx5dr_action * +mlx5dr_action_create_generic(struct mlx5dr_context *ctx, + uint32_t flags, + enum mlx5dr_action_type action_type) +{ + return mlx5dr_action_create_generic_bulk(ctx, flags, action_type, 1); +} + struct mlx5dr_action * mlx5dr_action_create_dest_table(struct mlx5dr_context *ctx, struct mlx5dr_table *tbl, @@ -1141,7 +1153,7 @@ mlx5dr_action_create_pop_vlan(struct mlx5dr_context *ctx, uint32_t flags) return NULL; } -static void +static int mlx5dr_action_conv_reformat_to_verbs(uint32_t action_type, uint32_t *verb_reformat_type) { @@ -1149,19 +1161,23 @@ mlx5dr_action_conv_reformat_to_verbs(uint32_t action_type, case MLX5DR_ACTION_TYP_REFORMAT_TNL_L2_TO_L2: *verb_reformat_type = MLX5DV_FLOW_ACTION_PACKET_REFORMAT_TYPE_L2_TUNNEL_TO_L2; - break; + return 0; case MLX5DR_ACTION_TYP_REFORMAT_L2_TO_TNL_L2: *verb_reformat_type = MLX5DV_FLOW_ACTION_PACKET_REFORMAT_TYPE_L2_TO_L2_TUNNEL; - break; + return 0; case MLX5DR_ACTION_TYP_REFORMAT_TNL_L3_TO_L2: *verb_reformat_type = MLX5DV_FLOW_ACTION_PACKET_REFORMAT_TYPE_L3_TUNNEL_TO_L2; - break; + return 0; case MLX5DR_ACTION_TYP_REFORMAT_L2_TO_TNL_L3: *verb_reformat_type = MLX5DV_FLOW_ACTION_PACKET_REFORMAT_TYPE_L2_TO_L3_TUNNEL; - break; + return 0; + default: + DR_LOG(ERR, "Invalid root reformat action type"); + rte_errno = EINVAL; + return rte_errno; } } @@ -1199,7 +1215,9 @@ mlx5dr_action_create_reformat_root(struct mlx5dr_action *action, if (ret) return rte_errno; - mlx5dr_action_conv_reformat_to_verbs(action->type, &verb_reformat_type); + ret = mlx5dr_action_conv_reformat_to_verbs(action->type, &verb_reformat_type); + if (ret) + return rte_errno; /* Create the reformat type for root table */ ibv_ctx = mlx5dr_context_get_local_ibv(action->ctx); @@ -1210,6 +1228,7 @@ mlx5dr_action_create_reformat_root(struct mlx5dr_action *action, verb_reformat_type, ft_type); if (!action->flow_action) { + DR_LOG(ERR, "Failed to create dv_create_flow reformat"); rte_errno = errno; return rte_errno; } @@ -1217,132 +1236,84 @@ mlx5dr_action_create_reformat_root(struct mlx5dr_action *action, return 0; } -static int mlx5dr_action_handle_reformat_args(struct mlx5dr_context *ctx, - size_t data_sz, - void *data, - uint32_t bulk_size, - struct mlx5dr_action *action) -{ - uint32_t args_log_size; - int ret; - - if (data_sz % 2 != 0) { - DR_LOG(ERR, "Data size should be multiply of 2"); - rte_errno = EINVAL; - return rte_errno; - } - action->reformat.header_size = data_sz; - - args_log_size = mlx5dr_arg_data_size_to_arg_log_size(data_sz); - if (args_log_size >= MLX5DR_ARG_CHUNK_SIZE_MAX) { - DR_LOG(ERR, "Data size is bigger than supported"); - rte_errno = EINVAL; - return rte_errno; +static int +mlx5dr_action_handle_l2_to_tunnel_l2(struct mlx5dr_action *action, + uint8_t num_of_hdrs, + struct mlx5dr_action_reformat_header *hdrs, + uint32_t log_bulk_sz) +{ + struct mlx5dr_devx_obj *arg_obj; + size_t max_sz = 0; + int ret, i; + + for (i = 0; i < num_of_hdrs; i++) { + if (hdrs[i].sz % 2 != 0) { + DR_LOG(ERR, "Header data size should be multiply of 2"); + rte_errno = EINVAL; + return rte_errno; + } + max_sz = RTE_MAX(hdrs[i].sz, max_sz); } - args_log_size += bulk_size; - if (!mlx5dr_arg_is_valid_arg_request_size(ctx, args_log_size)) { - DR_LOG(ERR, "Arg size %d does not fit FW requests", - args_log_size); - rte_errno = EINVAL; + /* Allocate single shared arg object for all headers */ + arg_obj = mlx5dr_arg_create(action->ctx, + hdrs->data, + max_sz, + log_bulk_sz, + action->flags & MLX5DR_ACTION_FLAG_SHARED); + if (!arg_obj) return rte_errno; - } - action->reformat.arg_obj = mlx5dr_cmd_arg_create(ctx->ibv_ctx, - args_log_size, - ctx->pd_num); - if (!action->reformat.arg_obj) { - DR_LOG(ERR, "Failed to create arg for reformat"); - return rte_errno; - } + for (i = 0; i < num_of_hdrs; i++) { + action[i].reformat.arg_obj = arg_obj; + action[i].reformat.header_size = hdrs[i].sz; + action[i].reformat.num_of_hdrs = num_of_hdrs; + action[i].reformat.max_hdr_sz = max_sz; - /* When INLINE need to write the arg data */ - if (action->flags & MLX5DR_ACTION_FLAG_SHARED) { - ret = mlx5dr_arg_write_inline_arg_data(ctx, - action->reformat.arg_obj->id, - data, - data_sz); + ret = mlx5dr_action_create_stcs(&action[i], NULL); if (ret) { - DR_LOG(ERR, "Failed to write inline arg for reformat"); - goto free_arg; + DR_LOG(ERR, "Failed to create stc for reformat"); + goto free_stc; } } return 0; -free_arg: - mlx5dr_cmd_destroy_obj(action->reformat.arg_obj); - return ret; -} - -static int mlx5dr_action_handle_l2_to_tunnel_l2(struct mlx5dr_context *ctx, - size_t data_sz, - void *data, - uint32_t bulk_size, - struct mlx5dr_action *action) -{ - int ret; +free_stc: + while (i--) + mlx5dr_action_destroy_stcs(&action[i]); - ret = mlx5dr_action_handle_reformat_args(ctx, data_sz, data, bulk_size, - action); - if (ret) { - DR_LOG(ERR, "Failed to create args for reformat"); - return ret; - } - - ret = mlx5dr_action_create_stcs(action, NULL); - if (ret) { - DR_LOG(ERR, "Failed to create stc for reformat"); - goto free_arg; - } - - return 0; - -free_arg: - mlx5dr_cmd_destroy_obj(action->reformat.arg_obj); + mlx5dr_cmd_destroy_obj(arg_obj); return ret; } -static int mlx5dr_action_get_shared_stc_offset(struct mlx5dr_context_common_res *common_res, - enum mlx5dr_context_shared_stc_type stc_type) -{ - return common_res->shared_stc[stc_type]->remove_header.offset; -} - -static int mlx5dr_action_handle_l2_to_tunnel_l3(struct mlx5dr_context *ctx, - size_t data_sz, - void *data, - uint32_t bulk_size, - struct mlx5dr_action *action) +static int +mlx5dr_action_handle_l2_to_tunnel_l3(struct mlx5dr_action *action, + uint8_t num_of_hdrs, + struct mlx5dr_action_reformat_header *hdrs, + uint32_t log_bulk_sz) { int ret; - ret = mlx5dr_action_handle_reformat_args(ctx, data_sz, data, bulk_size, - action); - if (ret) { - DR_LOG(ERR, "Failed to create args for reformat"); - return ret; - } - /* The action is remove-l2-header + insert-l3-header */ ret = mlx5dr_action_get_shared_stc(action, MLX5DR_CONTEXT_SHARED_STC_DECAP); if (ret) { DR_LOG(ERR, "Failed to create remove stc for reformat"); - goto free_arg; + return ret; } - ret = mlx5dr_action_create_stcs(action, NULL); - if (ret) { - DR_LOG(ERR, "Failed to create insert stc for reformat"); - goto down_shared; - } + /* Reuse the insert with pointer for the L2L3 header */ + ret = mlx5dr_action_handle_l2_to_tunnel_l2(action, + num_of_hdrs, + hdrs, + log_bulk_sz); + if (ret) + goto put_shared_stc; return 0; -down_shared: +put_shared_stc: mlx5dr_action_put_shared_stc(action, MLX5DR_CONTEXT_SHARED_STC_DECAP); -free_arg: - mlx5dr_cmd_destroy_obj(action->reformat.arg_obj); return ret; } @@ -1393,67 +1364,81 @@ static void mlx5dr_action_prepare_decap_l3_actions(size_t data_sz, } static int -mlx5dr_action_handle_tunnel_l3_to_l2(struct mlx5dr_context *ctx, - size_t data_sz, - void *data, - uint32_t bulk_size, - struct mlx5dr_action *action) +mlx5dr_action_handle_tunnel_l3_to_l2(struct mlx5dr_action *action, + uint8_t num_of_hdrs, + struct mlx5dr_action_reformat_header *hdrs, + uint32_t log_bulk_sz) { uint8_t mh_data[MLX5DR_ACTION_REFORMAT_DATA_SIZE] = {0}; + struct mlx5dr_devx_obj *arg_obj, *pat_obj; + struct mlx5dr_context *ctx = action->ctx; int num_of_actions; int mh_data_size; - int ret; + int ret, i; - if (data_sz != MLX5DR_ACTION_HDR_LEN_L2 && - data_sz != MLX5DR_ACTION_HDR_LEN_L2_W_VLAN) { - DR_LOG(ERR, "Data size is not supported for decap-l3"); - rte_errno = EINVAL; - return rte_errno; + for (i = 0; i < num_of_hdrs; i++) { + if (hdrs[i].sz != MLX5DR_ACTION_HDR_LEN_L2 && + hdrs[i].sz != MLX5DR_ACTION_HDR_LEN_L2_W_VLAN) { + DR_LOG(ERR, "Data size is not supported for decap-l3"); + rte_errno = EINVAL; + return rte_errno; + } } - mlx5dr_action_prepare_decap_l3_actions(data_sz, mh_data, &num_of_actions); + /* Create a full modify header action list in case shared */ + mlx5dr_action_prepare_decap_l3_actions(hdrs->sz, mh_data, &num_of_actions); + mlx5dr_action_prepare_decap_l3_data(hdrs->data, mh_data, num_of_actions); - mh_data_size = num_of_actions * MLX5DR_MODIFY_ACTION_SIZE; + /* All DecapL3 cases require the same max arg size */ + arg_obj = mlx5dr_arg_create_modify_header_arg(ctx, + (__be64 *)mh_data, + num_of_actions, + log_bulk_sz, + action->flags & MLX5DR_ACTION_FLAG_SHARED); + if (!arg_obj) + return rte_errno; - ret = mlx5dr_pat_arg_create_modify_header(ctx, action, mh_data_size, - (__be64 *)mh_data, bulk_size); - if (ret) { - DR_LOG(ERR, "Failed allocating modify-header for decap-l3"); - return ret; - } + for (i = 0; i < num_of_hdrs; i++) { + memset(mh_data, 0, MLX5DR_ACTION_REFORMAT_DATA_SIZE); + mlx5dr_action_prepare_decap_l3_actions(hdrs[i].sz, mh_data, &num_of_actions); + mh_data_size = num_of_actions * MLX5DR_MODIFY_ACTION_SIZE; - ret = mlx5dr_action_create_stcs(action, NULL); - if (ret) - goto free_mh_obj; - - if (action->flags & MLX5DR_ACTION_FLAG_SHARED) { - mlx5dr_action_prepare_decap_l3_data(data, mh_data, num_of_actions); - ret = mlx5dr_arg_write_inline_arg_data(ctx, - action->modify_header.arg_obj->id, - (uint8_t *)mh_data, - num_of_actions * - MLX5DR_MODIFY_ACTION_SIZE); + pat_obj = mlx5dr_pat_get_pattern(ctx, (__be64 *)mh_data, mh_data_size); + if (!pat_obj) { + DR_LOG(ERR, "Failed to allocate pattern for DecapL3"); + goto free_stc_and_pat; + } + + action[i].modify_header.max_num_of_actions = num_of_actions; + action[i].modify_header.num_of_actions = num_of_actions; + action[i].modify_header.arg_obj = arg_obj; + action[i].modify_header.pat_obj = pat_obj; + + ret = mlx5dr_action_create_stcs(&action[i], NULL); if (ret) { - DR_LOG(ERR, "Failed writing INLINE arg decap_l3"); - goto clean_stc; + mlx5dr_pat_put_pattern(ctx, pat_obj); + goto free_stc_and_pat; } } return 0; -clean_stc: - mlx5dr_action_destroy_stcs(action); -free_mh_obj: - mlx5dr_pat_arg_destroy_modify_header(ctx, action); - return ret; + +free_stc_and_pat: + while (i--) { + mlx5dr_action_destroy_stcs(&action[i]); + mlx5dr_pat_put_pattern(ctx, action[i].modify_header.pat_obj); + } + + mlx5dr_cmd_destroy_obj(arg_obj); + return 0; } static int -mlx5dr_action_create_reformat_hws(struct mlx5dr_context *ctx, - size_t data_sz, - void *data, - uint32_t bulk_size, - struct mlx5dr_action *action) +mlx5dr_action_create_reformat_hws(struct mlx5dr_action *action, + uint8_t num_of_hdrs, + struct mlx5dr_action_reformat_header *hdrs, + uint32_t bulk_size) { int ret; @@ -1462,18 +1447,17 @@ mlx5dr_action_create_reformat_hws(struct mlx5dr_context *ctx, ret = mlx5dr_action_create_stcs(action, NULL); break; case MLX5DR_ACTION_TYP_REFORMAT_L2_TO_TNL_L2: - ret = mlx5dr_action_handle_l2_to_tunnel_l2(ctx, data_sz, data, bulk_size, action); + ret = mlx5dr_action_handle_l2_to_tunnel_l2(action, num_of_hdrs, hdrs, bulk_size); break; case MLX5DR_ACTION_TYP_REFORMAT_L2_TO_TNL_L3: - ret = mlx5dr_action_handle_l2_to_tunnel_l3(ctx, data_sz, data, bulk_size, action); + ret = mlx5dr_action_handle_l2_to_tunnel_l3(action, num_of_hdrs, hdrs, bulk_size); break; case MLX5DR_ACTION_TYP_REFORMAT_TNL_L3_TO_L2: - ret = mlx5dr_action_handle_tunnel_l3_to_l2(ctx, data_sz, data, bulk_size, action); + ret = mlx5dr_action_handle_tunnel_l3_to_l2(action, num_of_hdrs, hdrs, bulk_size); break; - default: - assert(false); - rte_errno = ENOTSUP; + DR_LOG(ERR, "Invalid HWS reformat action type"); + rte_errno = EINVAL; return rte_errno; } @@ -1483,15 +1467,20 @@ mlx5dr_action_create_reformat_hws(struct mlx5dr_context *ctx, struct mlx5dr_action * mlx5dr_action_create_reformat(struct mlx5dr_context *ctx, enum mlx5dr_action_type reformat_type, - size_t data_sz, - void *inline_data, + uint8_t num_of_hdrs, + struct mlx5dr_action_reformat_header *hdrs, uint32_t log_bulk_size, uint32_t flags) { struct mlx5dr_action *action; int ret; - action = mlx5dr_action_create_generic(ctx, flags, reformat_type); + if (!num_of_hdrs) { + DR_LOG(ERR, "Reformat num_of_hdrs cannot be zero"); + return NULL; + } + + action = mlx5dr_action_create_generic_bulk(ctx, flags, reformat_type, num_of_hdrs); if (!action) return NULL; @@ -1502,24 +1491,27 @@ mlx5dr_action_create_reformat(struct mlx5dr_context *ctx, goto free_action; } - ret = mlx5dr_action_create_reformat_root(action, data_sz, inline_data); - if (ret) + ret = mlx5dr_action_create_reformat_root(action, + hdrs ? hdrs->sz : 0, + hdrs ? hdrs->data : NULL); + if (ret) { + DR_LOG(ERR, "Failed to create root reformat action"); goto free_action; + } return action; } if (!mlx5dr_action_is_hws_flags(flags) || - ((flags & MLX5DR_ACTION_FLAG_SHARED) && log_bulk_size)) { - DR_LOG(ERR, "Reformat flags don't fit HWS (flags: %x0x)", - flags); + ((flags & MLX5DR_ACTION_FLAG_SHARED) && (log_bulk_size || num_of_hdrs > 1))) { + DR_LOG(ERR, "Reformat flags don't fit HWS (flags: %x0x)", flags); rte_errno = EINVAL; goto free_action; } - ret = mlx5dr_action_create_reformat_hws(ctx, data_sz, inline_data, log_bulk_size, action); + ret = mlx5dr_action_create_reformat_hws(action, num_of_hdrs, hdrs, log_bulk_size); if (ret) { - DR_LOG(ERR, "Failed to create reformat."); + DR_LOG(ERR, "Failed to create HWS reformat action"); rte_errno = EINVAL; goto free_action; } @@ -1559,17 +1551,104 @@ mlx5dr_action_create_modify_header_root(struct mlx5dr_action *action, return 0; } +static int +mlx5dr_action_create_modify_header_hws(struct mlx5dr_action *action, + uint8_t num_of_patterns, + struct mlx5dr_action_mh_pattern *pattern, + uint32_t log_bulk_size) +{ + struct mlx5dr_devx_obj *pat_obj, *arg_obj = NULL; + struct mlx5dr_context *ctx = action->ctx; + uint16_t max_mh_actions = 0; + int i, ret; + + /* Caclulate maximum number of mh actions for shared arg allocation */ + for (i = 0; i < num_of_patterns; i++) + max_mh_actions = RTE_MAX(max_mh_actions, pattern[i].sz / MLX5DR_MODIFY_ACTION_SIZE); + + /* Allocate single shared arg for all patterns based on the max size */ + if (max_mh_actions > 1) { + arg_obj = mlx5dr_arg_create_modify_header_arg(ctx, + pattern->data, + max_mh_actions, + log_bulk_size, + action->flags & + MLX5DR_ACTION_FLAG_SHARED); + if (!arg_obj) + return rte_errno; + } + + for (i = 0; i < num_of_patterns; i++) { + if (!mlx5dr_pat_verify_actions(pattern[i].data, pattern[i].sz)) { + DR_LOG(ERR, "Fail to verify pattern modify actions"); + rte_errno = EINVAL; + goto free_stc_and_pat; + } + + action[i].modify_header.num_of_patterns = num_of_patterns; + action[i].modify_header.max_num_of_actions = max_mh_actions; + action[i].modify_header.num_of_actions = pattern[i].sz / MLX5DR_MODIFY_ACTION_SIZE; + + if (action[i].modify_header.num_of_actions == 1) { + pat_obj = NULL; + /* Optimize single modify action to be used inline */ + action[i].modify_header.single_action = pattern[i].data[0]; + action[i].modify_header.single_action_type = + MLX5_GET(set_action_in, pattern[i].data, action_type); + } else { + /* Multiple modify actions require a pattern */ + pat_obj = mlx5dr_pat_get_pattern(ctx, pattern[i].data, pattern[i].sz); + if (!pat_obj) { + DR_LOG(ERR, "Failed to allocate pattern for modify header"); + goto free_stc_and_pat; + } + + action[i].modify_header.arg_obj = arg_obj; + action[i].modify_header.pat_obj = pat_obj; + } + /* Allocate STC for each action representing a header */ + ret = mlx5dr_action_create_stcs(&action[i], NULL); + if (ret) { + if (pat_obj) + mlx5dr_pat_put_pattern(ctx, pat_obj); + goto free_stc_and_pat; + } + } + + return 0; + +free_stc_and_pat: + while (i--) { + mlx5dr_action_destroy_stcs(&action[i]); + if (action[i].modify_header.pat_obj) + mlx5dr_pat_put_pattern(ctx, action[i].modify_header.pat_obj); + } + + if (arg_obj) + mlx5dr_cmd_destroy_obj(arg_obj); + + return rte_errno; +} + struct mlx5dr_action * mlx5dr_action_create_modify_header(struct mlx5dr_context *ctx, - size_t pattern_sz, - __be64 pattern[], + uint8_t num_of_patterns, + struct mlx5dr_action_mh_pattern *patterns, uint32_t log_bulk_size, uint32_t flags) { struct mlx5dr_action *action; int ret; - action = mlx5dr_action_create_generic(ctx, flags, MLX5DR_ACTION_TYP_MODIFY_HDR); + if (!num_of_patterns) { + DR_LOG(ERR, "Invalid number of patterns"); + rte_errno = ENOTSUP; + return NULL; + } + + action = mlx5dr_action_create_generic_bulk(ctx, flags, + MLX5DR_ACTION_TYP_MODIFY_HDR, + num_of_patterns); if (!action) return NULL; @@ -1579,52 +1658,37 @@ mlx5dr_action_create_modify_header(struct mlx5dr_context *ctx, rte_errno = ENOTSUP; goto free_action; } - ret = mlx5dr_action_create_modify_header_root(action, pattern_sz, pattern); + + if (num_of_patterns != 1) { + DR_LOG(ERR, "Only a single pattern supported over root"); + rte_errno = ENOTSUP; + goto free_action; + } + + ret = mlx5dr_action_create_modify_header_root(action, + patterns->sz, + patterns->data); if (ret) goto free_action; return action; } - if (!mlx5dr_action_is_hws_flags(flags) || - ((flags & MLX5DR_ACTION_FLAG_SHARED) && log_bulk_size)) { - DR_LOG(ERR, "Flags don't fit hws (flags: %x0x, log_bulk_size: %d)", - flags, log_bulk_size); + if ((flags & MLX5DR_ACTION_FLAG_SHARED) && (log_bulk_size || num_of_patterns > 1)) { + DR_LOG(ERR, "Action cannot be shared with requested pattern or size"); rte_errno = EINVAL; goto free_action; } - if (!mlx5dr_pat_arg_verify_actions(pattern, pattern_sz / MLX5DR_MODIFY_ACTION_SIZE)) { - DR_LOG(ERR, "One of the actions is not supported"); - rte_errno = EINVAL; - goto free_action; - } - - if (pattern_sz / MLX5DR_MODIFY_ACTION_SIZE == 1) { - /* Optimize single modiy action to be used inline */ - action->modify_header.single_action = pattern[0]; - action->modify_header.num_of_actions = 1; - action->modify_header.single_action_type = - MLX5_GET(set_action_in, pattern, action_type); - } else { - /* Use multi action pattern and argument */ - ret = mlx5dr_pat_arg_create_modify_header(ctx, action, pattern_sz, - pattern, log_bulk_size); - if (ret) { - DR_LOG(ERR, "Failed allocating modify-header"); - goto free_action; - } - } - - ret = mlx5dr_action_create_stcs(action, NULL); + ret = mlx5dr_action_create_modify_header_hws(action, + num_of_patterns, + patterns, + log_bulk_size); if (ret) - goto free_mh_obj; + goto free_action; return action; -free_mh_obj: - if (action->modify_header.num_of_actions > 1) - mlx5dr_pat_arg_destroy_modify_header(ctx, action); free_action: simple_free(action); return NULL; @@ -1684,6 +1748,9 @@ mlx5dr_action_create_dest_root(struct mlx5dr_context *ctx, static void mlx5dr_action_destroy_hws(struct mlx5dr_action *action) { + struct mlx5dr_devx_obj *obj = NULL; + uint32_t i; + switch (action->type) { case MLX5DR_ACTION_TYP_TIR: mlx5dr_action_destroy_stcs(action); @@ -1711,17 +1778,28 @@ static void mlx5dr_action_destroy_hws(struct mlx5dr_action *action) break; case MLX5DR_ACTION_TYP_REFORMAT_TNL_L3_TO_L2: case MLX5DR_ACTION_TYP_MODIFY_HDR: - mlx5dr_action_destroy_stcs(action); - if (action->modify_header.num_of_actions > 1) - mlx5dr_pat_arg_destroy_modify_header(action->ctx, action); + for (i = 0; i < action->modify_header.num_of_patterns; i++) { + mlx5dr_action_destroy_stcs(&action[i]); + if (action[i].modify_header.num_of_actions > 1) { + mlx5dr_pat_put_pattern(action[i].ctx, + action[i].modify_header.pat_obj); + /* Save shared arg object if was used to free */ + if (action[i].modify_header.arg_obj) + obj = action[i].modify_header.arg_obj; + } + } + if (obj) + mlx5dr_cmd_destroy_obj(obj); break; case MLX5DR_ACTION_TYP_REFORMAT_L2_TO_TNL_L3: - mlx5dr_action_destroy_stcs(action); mlx5dr_action_put_shared_stc(action, MLX5DR_CONTEXT_SHARED_STC_DECAP); + for (i = 0; i < action->reformat.num_of_hdrs; i++) + mlx5dr_action_destroy_stcs(&action[i]); mlx5dr_cmd_destroy_obj(action->reformat.arg_obj); break; case MLX5DR_ACTION_TYP_REFORMAT_L2_TO_TNL_L2: - mlx5dr_action_destroy_stcs(action); + for (i = 0; i < action->reformat.num_of_hdrs; i++) + mlx5dr_action_destroy_stcs(&action[i]); mlx5dr_cmd_destroy_obj(action->reformat.arg_obj); break; } @@ -1903,6 +1981,12 @@ mlx5dr_action_prepare_decap_l3_data(uint8_t *src, uint8_t *dst, memcpy(dst, e_src, 2); } +static int mlx5dr_action_get_shared_stc_offset(struct mlx5dr_context_common_res *common_res, + enum mlx5dr_context_shared_stc_type stc_type) +{ + return common_res->shared_stc[stc_type]->remove_header.offset; +} + static struct mlx5dr_actions_wqe_setter * mlx5dr_action_setter_find_first(struct mlx5dr_actions_wqe_setter *setter, uint8_t req_flags) @@ -1945,13 +2029,15 @@ mlx5dr_action_setter_modify_header(struct mlx5dr_actions_apply_data *apply, struct mlx5dr_actions_wqe_setter *setter) { struct mlx5dr_rule_action *rule_action; + uint32_t stc_idx, arg_sz, arg_idx; struct mlx5dr_action *action; - uint32_t arg_sz, arg_idx; uint8_t *single_action; rule_action = &apply->rule_action[setter->idx_double]; - action = rule_action->action; - mlx5dr_action_apply_stc(apply, MLX5DR_ACTION_STC_IDX_DW6, setter->idx_double); + action = rule_action->action + rule_action->modify_header.pattern_idx; + + stc_idx = htobe32(action->stc[apply->tbl_type].offset); + apply->wqe_ctrl->stc_ix[MLX5DR_ACTION_STC_IDX_DW6] = stc_idx; apply->wqe_ctrl->stc_ix[MLX5DR_ACTION_STC_IDX_DW7] = 0; apply->wqe_data[MLX5DR_ACTION_OFFSET_DW6] = 0; @@ -1972,7 +2058,7 @@ mlx5dr_action_setter_modify_header(struct mlx5dr_actions_apply_data *apply, *(__be32 *)MLX5_ADDR_OF(set_action_in, single_action, data); } else { /* Argument offset multiple with number of args per these actions */ - arg_sz = mlx5dr_arg_get_arg_size(action->modify_header.num_of_actions); + arg_sz = mlx5dr_arg_get_arg_size(action->modify_header.max_num_of_actions); arg_idx = rule_action->modify_header.offset * arg_sz; apply->wqe_data[MLX5DR_ACTION_OFFSET_DW7] = htobe32(arg_idx); @@ -1992,26 +2078,29 @@ mlx5dr_action_setter_insert_ptr(struct mlx5dr_actions_apply_data *apply, struct mlx5dr_actions_wqe_setter *setter) { struct mlx5dr_rule_action *rule_action; - uint32_t arg_idx, arg_sz; + uint32_t stc_idx, arg_idx, arg_sz; + struct mlx5dr_action *action; rule_action = &apply->rule_action[setter->idx_double]; + action = rule_action->action + rule_action->reformat.hdr_idx; /* Argument offset multiple on args required for header size */ - arg_sz = mlx5dr_arg_data_size_to_arg_size(rule_action->action->reformat.header_size); + arg_sz = mlx5dr_arg_data_size_to_arg_size(action->reformat.max_hdr_sz); arg_idx = rule_action->reformat.offset * arg_sz; apply->wqe_data[MLX5DR_ACTION_OFFSET_DW6] = 0; apply->wqe_data[MLX5DR_ACTION_OFFSET_DW7] = htobe32(arg_idx); - mlx5dr_action_apply_stc(apply, MLX5DR_ACTION_STC_IDX_DW6, setter->idx_double); + stc_idx = htobe32(action->stc[apply->tbl_type].offset); + apply->wqe_ctrl->stc_ix[MLX5DR_ACTION_STC_IDX_DW6] = stc_idx; apply->wqe_ctrl->stc_ix[MLX5DR_ACTION_STC_IDX_DW7] = 0; - if (!(rule_action->action->flags & MLX5DR_ACTION_FLAG_SHARED)) { + if (!(action->flags & MLX5DR_ACTION_FLAG_SHARED)) { apply->require_dep = 1; mlx5dr_arg_write(apply->queue, NULL, - rule_action->action->reformat.arg_obj->id + arg_idx, + action->reformat.arg_obj->id + arg_idx, rule_action->reformat.data, - rule_action->action->reformat.header_size); + action->reformat.header_size); } } @@ -2020,20 +2109,21 @@ mlx5dr_action_setter_tnl_l3_to_l2(struct mlx5dr_actions_apply_data *apply, struct mlx5dr_actions_wqe_setter *setter) { struct mlx5dr_rule_action *rule_action; + uint32_t stc_idx, arg_sz, arg_idx; struct mlx5dr_action *action; - uint32_t arg_sz, arg_idx; rule_action = &apply->rule_action[setter->idx_double]; - action = rule_action->action; + action = rule_action->action + rule_action->reformat.hdr_idx; /* Argument offset multiple on args required for num of actions */ - arg_sz = mlx5dr_arg_get_arg_size(action->modify_header.num_of_actions); + arg_sz = mlx5dr_arg_get_arg_size(action->modify_header.max_num_of_actions); arg_idx = rule_action->reformat.offset * arg_sz; apply->wqe_data[MLX5DR_ACTION_OFFSET_DW6] = 0; apply->wqe_data[MLX5DR_ACTION_OFFSET_DW7] = htobe32(arg_idx); - mlx5dr_action_apply_stc(apply, MLX5DR_ACTION_STC_IDX_DW6, setter->idx_double); + stc_idx = htobe32(action->stc[apply->tbl_type].offset); + apply->wqe_ctrl->stc_ix[MLX5DR_ACTION_STC_IDX_DW6] = stc_idx; apply->wqe_ctrl->stc_ix[MLX5DR_ACTION_STC_IDX_DW7] = 0; if (!(action->flags & MLX5DR_ACTION_FLAG_SHARED)) { diff --git a/drivers/net/mlx5/hws/mlx5dr_action.h b/drivers/net/mlx5/hws/mlx5dr_action.h index a85f3b0139..314e289780 100644 --- a/drivers/net/mlx5/hws/mlx5dr_action.h +++ b/drivers/net/mlx5/hws/mlx5dr_action.h @@ -120,15 +120,19 @@ struct mlx5dr_action { struct mlx5dr_pool_chunk stc[MLX5DR_TABLE_TYPE_MAX]; union { struct { - struct mlx5dr_devx_obj *pattern_obj; + struct mlx5dr_devx_obj *pat_obj; struct mlx5dr_devx_obj *arg_obj; __be64 single_action; + uint8_t num_of_patterns; uint8_t single_action_type; - uint16_t num_of_actions; + uint8_t num_of_actions; + uint8_t max_num_of_actions; } modify_header; struct { struct mlx5dr_devx_obj *arg_obj; uint32_t header_size; + uint8_t num_of_hdrs; + uint16_t max_hdr_sz; } reformat; struct { struct mlx5dr_devx_obj *devx_obj; diff --git a/drivers/net/mlx5/hws/mlx5dr_pat_arg.c b/drivers/net/mlx5/hws/mlx5dr_pat_arg.c index 309a61d477..50da90626c 100644 --- a/drivers/net/mlx5/hws/mlx5dr_pat_arg.c +++ b/drivers/net/mlx5/hws/mlx5dr_pat_arg.c @@ -60,22 +60,16 @@ void mlx5dr_pat_uninit_pattern_cache(struct mlx5dr_pattern_cache *cache) simple_free(cache); } -static bool mlx5dr_pat_compare_pattern(enum mlx5dr_action_type cur_type, - int cur_num_of_actions, +static bool mlx5dr_pat_compare_pattern(int cur_num_of_actions, __be64 cur_actions[], - enum mlx5dr_action_type type, int num_of_actions, __be64 actions[]) { int i; - if (cur_num_of_actions != num_of_actions || cur_type != type) + if (cur_num_of_actions != num_of_actions) return false; - /* All decap-l3 look the same, only change is the num of actions */ - if (type == MLX5DR_ACTION_TYP_REFORMAT_TNL_L3_TO_L2) - return true; - for (i = 0; i < num_of_actions; i++) { u8 action_id = MLX5_GET(set_action_in, &actions[i], action_type); @@ -96,17 +90,14 @@ static bool mlx5dr_pat_compare_pattern(enum mlx5dr_action_type cur_type, static struct mlx5dr_pattern_cache_item * mlx5dr_pat_find_cached_pattern(struct mlx5dr_pattern_cache *cache, - struct mlx5dr_action *action, uint16_t num_of_actions, __be64 *actions) { struct mlx5dr_pattern_cache_item *cached_pat; LIST_FOREACH(cached_pat, &cache->head, next) { - if (mlx5dr_pat_compare_pattern(cached_pat->type, - cached_pat->mh_data.num_of_actions, + if (mlx5dr_pat_compare_pattern(cached_pat->mh_data.num_of_actions, (__be64 *)cached_pat->mh_data.data, - action->type, num_of_actions, actions)) return cached_pat; @@ -117,13 +108,12 @@ mlx5dr_pat_find_cached_pattern(struct mlx5dr_pattern_cache *cache, static struct mlx5dr_pattern_cache_item * mlx5dr_pat_get_existing_cached_pattern(struct mlx5dr_pattern_cache *cache, - struct mlx5dr_action *action, uint16_t num_of_actions, __be64 *actions) { struct mlx5dr_pattern_cache_item *cached_pattern; - cached_pattern = mlx5dr_pat_find_cached_pattern(cache, action, num_of_actions, actions); + cached_pattern = mlx5dr_pat_find_cached_pattern(cache, num_of_actions, actions); if (cached_pattern) { /* LRU: move it to be first in the list */ LIST_REMOVE(cached_pattern, next); @@ -134,24 +124,9 @@ mlx5dr_pat_get_existing_cached_pattern(struct mlx5dr_pattern_cache *cache, return cached_pattern; } -static struct mlx5dr_pattern_cache_item * -mlx5dr_pat_get_cached_pattern_by_action(struct mlx5dr_pattern_cache *cache, - struct mlx5dr_action *action) -{ - struct mlx5dr_pattern_cache_item *cached_pattern; - - LIST_FOREACH(cached_pattern, &cache->head, next) { - if (cached_pattern->mh_data.pattern_obj->id == action->modify_header.pattern_obj->id) - return cached_pattern; - } - - return NULL; -} - static struct mlx5dr_pattern_cache_item * mlx5dr_pat_add_pattern_to_cache(struct mlx5dr_pattern_cache *cache, struct mlx5dr_devx_obj *pattern_obj, - enum mlx5dr_action_type type, uint16_t num_of_actions, __be64 *actions) { @@ -164,7 +139,6 @@ mlx5dr_pat_add_pattern_to_cache(struct mlx5dr_pattern_cache *cache, return NULL; } - cached_pattern->type = type; cached_pattern->mh_data.num_of_actions = num_of_actions; cached_pattern->mh_data.pattern_obj = pattern_obj; cached_pattern->mh_data.data = @@ -188,6 +162,20 @@ mlx5dr_pat_add_pattern_to_cache(struct mlx5dr_pattern_cache *cache, return NULL; } +static struct mlx5dr_pattern_cache_item * +mlx5dr_pat_find_cached_pattern_by_obj(struct mlx5dr_pattern_cache *cache, + struct mlx5dr_devx_obj *pat_obj) +{ + struct mlx5dr_pattern_cache_item *cached_pattern; + + LIST_FOREACH(cached_pattern, &cache->head, next) { + if (cached_pattern->mh_data.pattern_obj->id == pat_obj->id) + return cached_pattern; + } + + return NULL; +} + static void mlx5dr_pat_remove_pattern(struct mlx5dr_pattern_cache_item *cached_pattern) { @@ -196,14 +184,14 @@ mlx5dr_pat_remove_pattern(struct mlx5dr_pattern_cache_item *cached_pattern) simple_free(cached_pattern); } -static void -mlx5dr_pat_put_pattern(struct mlx5dr_pattern_cache *cache, - struct mlx5dr_action *action) +void mlx5dr_pat_put_pattern(struct mlx5dr_context *ctx, + struct mlx5dr_devx_obj *pat_obj) { + struct mlx5dr_pattern_cache *cache = ctx->pattern_cache; struct mlx5dr_pattern_cache_item *cached_pattern; pthread_spin_lock(&cache->lock); - cached_pattern = mlx5dr_pat_get_cached_pattern_by_action(cache, action); + cached_pattern = mlx5dr_pat_find_cached_pattern_by_obj(cache, pat_obj); if (!cached_pattern) { DR_LOG(ERR, "Failed to find pattern according to action with pt"); assert(false); @@ -214,62 +202,56 @@ mlx5dr_pat_put_pattern(struct mlx5dr_pattern_cache *cache, goto out; mlx5dr_pat_remove_pattern(cached_pattern); + mlx5dr_cmd_destroy_obj(pat_obj); out: pthread_spin_unlock(&cache->lock); } -static int mlx5dr_pat_get_pattern(struct mlx5dr_context *ctx, - struct mlx5dr_action *action, - uint16_t num_of_actions, - size_t pattern_sz, - __be64 *pattern) +struct mlx5dr_devx_obj * +mlx5dr_pat_get_pattern(struct mlx5dr_context *ctx, + __be64 *pattern, size_t pattern_sz) { + uint16_t num_of_actions = pattern_sz / MLX5DR_MODIFY_ACTION_SIZE; struct mlx5dr_pattern_cache_item *cached_pattern; - int ret = 0; + struct mlx5dr_devx_obj *pat_obj = NULL; pthread_spin_lock(&ctx->pattern_cache->lock); cached_pattern = mlx5dr_pat_get_existing_cached_pattern(ctx->pattern_cache, - action, num_of_actions, pattern); if (cached_pattern) { - action->modify_header.pattern_obj = cached_pattern->mh_data.pattern_obj; + pat_obj = cached_pattern->mh_data.pattern_obj; goto out_unlock; } - action->modify_header.pattern_obj = - mlx5dr_cmd_header_modify_pattern_create(ctx->ibv_ctx, - pattern_sz, - (uint8_t *)pattern); - if (!action->modify_header.pattern_obj) { + pat_obj = mlx5dr_cmd_header_modify_pattern_create(ctx->ibv_ctx, + pattern_sz, + (uint8_t *)pattern); + if (!pat_obj) { DR_LOG(ERR, "Failed to create pattern FW object"); - - ret = rte_errno; goto out_unlock; } - cached_pattern = - mlx5dr_pat_add_pattern_to_cache(ctx->pattern_cache, - action->modify_header.pattern_obj, - action->type, - num_of_actions, - pattern); + cached_pattern = mlx5dr_pat_add_pattern_to_cache(ctx->pattern_cache, + pat_obj, + num_of_actions, + pattern); if (!cached_pattern) { DR_LOG(ERR, "Failed to add pattern to cache"); - ret = rte_errno; goto clean_pattern; } -out_unlock: pthread_spin_unlock(&ctx->pattern_cache->lock); - return ret; + return pat_obj; clean_pattern: - mlx5dr_cmd_destroy_obj(action->modify_header.pattern_obj); + mlx5dr_cmd_destroy_obj(pat_obj); + pat_obj = NULL; +out_unlock: pthread_spin_unlock(&ctx->pattern_cache->lock); - return ret; + return pat_obj; } static void @@ -388,64 +370,80 @@ bool mlx5dr_arg_is_valid_arg_request_size(struct mlx5dr_context *ctx, return true; } -static int -mlx5dr_arg_create_modify_header_arg(struct mlx5dr_context *ctx, - struct mlx5dr_action *action, - uint16_t num_of_actions, - __be64 *pattern, - uint32_t bulk_size) +struct mlx5dr_devx_obj * +mlx5dr_arg_create(struct mlx5dr_context *ctx, + uint8_t *data, + size_t data_sz, + uint32_t log_bulk_sz, + bool write_data) { - uint32_t flags = action->flags; - uint16_t args_log_size; - int ret = 0; + struct mlx5dr_devx_obj *arg_obj; + uint16_t single_arg_log_sz; + uint16_t multi_arg_log_sz; + int ret; - /* Alloc bulk of args */ - args_log_size = mlx5dr_arg_get_arg_log_size(num_of_actions); - if (args_log_size >= MLX5DR_ARG_CHUNK_SIZE_MAX) { - DR_LOG(ERR, "Exceed number of allowed actions %u", - num_of_actions); - rte_errno = EINVAL; - return rte_errno; + single_arg_log_sz = mlx5dr_arg_data_size_to_arg_log_size(data_sz); + multi_arg_log_sz = single_arg_log_sz + log_bulk_sz; + + if (single_arg_log_sz >= MLX5DR_ARG_CHUNK_SIZE_MAX) { + DR_LOG(ERR, "Requested single arg %u not supported", single_arg_log_sz); + rte_errno = ENOTSUP; + return NULL; } - if (!mlx5dr_arg_is_valid_arg_request_size(ctx, args_log_size + bulk_size)) { - DR_LOG(ERR, "Arg size %d does not fit FW capability", - args_log_size + bulk_size); - rte_errno = EINVAL; - return rte_errno; + if (!mlx5dr_arg_is_valid_arg_request_size(ctx, multi_arg_log_sz)) { + DR_LOG(ERR, "Argument log size %d not supported by FW", multi_arg_log_sz); + rte_errno = ENOTSUP; + return NULL; } - action->modify_header.arg_obj = - mlx5dr_cmd_arg_create(ctx->ibv_ctx, args_log_size + bulk_size, - ctx->pd_num); - if (!action->modify_header.arg_obj) { - DR_LOG(ERR, "Failed allocating arg in order: %d", - args_log_size + bulk_size); - return rte_errno; + /* Alloc bulk of args */ + arg_obj = mlx5dr_cmd_arg_create(ctx->ibv_ctx, multi_arg_log_sz, ctx->pd_num); + if (!arg_obj) { + DR_LOG(ERR, "Failed allocating arg in order: %d", multi_arg_log_sz); + return NULL; } - /* When INLINE need to write the arg data */ - if (flags & MLX5DR_ACTION_FLAG_SHARED) + if (write_data) { ret = mlx5dr_arg_write_inline_arg_data(ctx, - action->modify_header.arg_obj->id, - (uint8_t *)pattern, - num_of_actions * - MLX5DR_MODIFY_ACTION_SIZE); - if (ret) { - DR_LOG(ERR, "Failed writing INLINE arg in order: %d", - args_log_size + bulk_size); - mlx5dr_cmd_destroy_obj(action->modify_header.arg_obj); - return rte_errno; + arg_obj->id, + data, data_sz); + if (ret) { + DR_LOG(ERR, "Failed writing arg data"); + mlx5dr_cmd_destroy_obj(arg_obj); + return NULL; + } } - return 0; + return arg_obj; } -bool mlx5dr_pat_arg_verify_actions(__be64 pattern[], uint16_t num_of_actions) +struct mlx5dr_devx_obj * +mlx5dr_arg_create_modify_header_arg(struct mlx5dr_context *ctx, + __be64 *data, + uint8_t num_of_actions, + uint32_t log_bulk_sz, + bool write_data) { - int i; + size_t data_sz = num_of_actions * MLX5DR_MODIFY_ACTION_SIZE; + struct mlx5dr_devx_obj *arg_obj; + + arg_obj = mlx5dr_arg_create(ctx, + (uint8_t *)data, + data_sz, + log_bulk_sz, + write_data); + if (!arg_obj) + DR_LOG(ERR, "Failed creating modify header arg"); + + return arg_obj; +} - for (i = 0; i < num_of_actions; i++) { +bool mlx5dr_pat_verify_actions(__be64 pattern[], size_t sz) +{ + size_t i; + + for (i = 0; i < sz / MLX5DR_MODIFY_ACTION_SIZE; i++) { u8 action_id = MLX5_GET(set_action_in, &pattern[i], action_type); if (action_id >= MLX5_MODIFICATION_TYPE_MAX) { @@ -456,51 +454,3 @@ bool mlx5dr_pat_arg_verify_actions(__be64 pattern[], uint16_t num_of_actions) return true; } - -int mlx5dr_pat_arg_create_modify_header(struct mlx5dr_context *ctx, - struct mlx5dr_action *action, - size_t pattern_sz, - __be64 pattern[], - uint32_t bulk_size) -{ - uint16_t num_of_actions; - int ret; - - num_of_actions = pattern_sz / MLX5DR_MODIFY_ACTION_SIZE; - if (num_of_actions == 0) { - DR_LOG(ERR, "Invalid number of actions %u", num_of_actions); - rte_errno = EINVAL; - return rte_errno; - } - - action->modify_header.num_of_actions = num_of_actions; - - ret = mlx5dr_arg_create_modify_header_arg(ctx, action, - num_of_actions, - pattern, - bulk_size); - if (ret) { - DR_LOG(ERR, "Failed to allocate arg"); - return ret; - } - - ret = mlx5dr_pat_get_pattern(ctx, action, num_of_actions, pattern_sz, - pattern); - if (ret) { - DR_LOG(ERR, "Failed to allocate pattern"); - goto free_arg; - } - - return 0; - -free_arg: - mlx5dr_cmd_destroy_obj(action->modify_header.arg_obj); - return rte_errno; -} - -void mlx5dr_pat_arg_destroy_modify_header(struct mlx5dr_context *ctx, - struct mlx5dr_action *action) -{ - mlx5dr_cmd_destroy_obj(action->modify_header.arg_obj); - mlx5dr_pat_put_pattern(ctx->pattern_cache, action); -} diff --git a/drivers/net/mlx5/hws/mlx5dr_pat_arg.h b/drivers/net/mlx5/hws/mlx5dr_pat_arg.h index ec467dbb4b..2a38891c4d 100644 --- a/drivers/net/mlx5/hws/mlx5dr_pat_arg.h +++ b/drivers/net/mlx5/hws/mlx5dr_pat_arg.h @@ -28,7 +28,6 @@ struct mlx5dr_pattern_cache { }; struct mlx5dr_pattern_cache_item { - enum mlx5dr_action_type type; struct { struct mlx5dr_devx_obj *pattern_obj; struct dr_icm_chunk *chunk; @@ -53,16 +52,29 @@ int mlx5dr_pat_init_pattern_cache(struct mlx5dr_pattern_cache **cache); void mlx5dr_pat_uninit_pattern_cache(struct mlx5dr_pattern_cache *cache); -bool mlx5dr_pat_arg_verify_actions(__be64 pattern[], uint16_t num_of_actions); - -int mlx5dr_pat_arg_create_modify_header(struct mlx5dr_context *ctx, - struct mlx5dr_action *action, - size_t pattern_sz, - __be64 pattern[], - uint32_t bulk_size); - -void mlx5dr_pat_arg_destroy_modify_header(struct mlx5dr_context *ctx, - struct mlx5dr_action *action); +bool mlx5dr_pat_verify_actions(__be64 pattern[], size_t sz); + +struct mlx5dr_devx_obj * +mlx5dr_arg_create(struct mlx5dr_context *ctx, + uint8_t *data, + size_t data_sz, + uint32_t log_bulk_sz, + bool write_data); + +struct mlx5dr_devx_obj * +mlx5dr_arg_create_modify_header_arg(struct mlx5dr_context *ctx, + __be64 *data, + uint8_t num_of_actions, + uint32_t log_bulk_sz, + bool write_data); + +struct mlx5dr_devx_obj * +mlx5dr_pat_get_pattern(struct mlx5dr_context *ctx, + __be64 *pattern, + size_t pattern_sz); + +void mlx5dr_pat_put_pattern(struct mlx5dr_context *ctx, + struct mlx5dr_devx_obj *pat_obj); bool mlx5dr_arg_is_valid_arg_request_size(struct mlx5dr_context *ctx, uint32_t arg_size); diff --git a/drivers/net/mlx5/mlx5_flow_hw.c b/drivers/net/mlx5/mlx5_flow_hw.c index 8e146d99e2..271e4aae10 100644 --- a/drivers/net/mlx5/mlx5_flow_hw.c +++ b/drivers/net/mlx5/mlx5_flow_hw.c @@ -1770,6 +1770,7 @@ __flow_hw_actions_translate(struct rte_eth_dev *dev, } } if (mhdr.pos != UINT16_MAX) { + struct mlx5dr_action_mh_pattern pattern; uint32_t flags; uint32_t bulk_size; size_t mhdr_len; @@ -1791,14 +1792,17 @@ __flow_hw_actions_translate(struct rte_eth_dev *dev, } else { bulk_size = rte_log2_u32(table_attr->nb_flows); } + pattern.data = (__be64 *)acts->mhdr->mhdr_cmds; + pattern.sz = mhdr_len; acts->mhdr->action = mlx5dr_action_create_modify_header - (priv->dr_ctx, mhdr_len, (__be64 *)acts->mhdr->mhdr_cmds, + (priv->dr_ctx, 1, &pattern, bulk_size, flags); if (!acts->mhdr->action) goto err; acts->rule_acts[acts->mhdr->pos].action = acts->mhdr->action; } if (reformat_used) { + struct mlx5dr_action_reformat_header hdr; uint8_t buf[MLX5_ENCAP_MAX_LEN]; bool shared_rfmt = true; @@ -1822,9 +1826,12 @@ __flow_hw_actions_translate(struct rte_eth_dev *dev, acts->encap_decap->data_size = data_size; memcpy(acts->encap_decap->data, encap_data, data_size); } + + hdr.sz = data_size; + hdr.data = encap_data; acts->encap_decap->action = mlx5dr_action_create_reformat (priv->dr_ctx, refmt_type, - data_size, encap_data, + 1, &hdr, shared_rfmt ? 0 : rte_log2_u32(table_attr->nb_flows), mlx5_hw_act_flag[!!attr->group][type] | (shared_rfmt ? MLX5DR_ACTION_FLAG_SHARED : 0)); -- 2.18.1