From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 96470A0C57; Mon, 1 Nov 2021 10:16:12 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 10689410FF; Mon, 1 Nov 2021 10:15:58 +0100 (CET) Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2047.outbound.protection.outlook.com [40.107.92.47]) by mails.dpdk.org (Postfix) with ESMTP id B926540DF6 for ; Mon, 1 Nov 2021 10:15:56 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WDioZ38rhEH3lnNqX59F2VAu7lo9db0ey1R/GLW6CBLu/VIDMWkBEqnq6T6w9obtaDm3eH+0vwOmiAgFTBbJgjBodgFL+a55c9JsMoSPXqV6Jv05URW+26Z3kS8B4W47qJvY1e9tg7I7eNn77dGRnQ/UhoNOeHx2uJda5yGal/DbVLUKRLlOprZsuMRvjqxYxOskwWjFwd5WmAvdtlaNyGDoZ3ZPSX3BT96Hgmvq8wn6EpT7zJMaVj7zMGgV73ES5oxAZXfAtou8hVUvKlDvGuzC9jZS/fSABptWHkIoiJjwoGiipagmqCNSHo7zkY09Lpadm01J8ZnjhZc9OiGkOQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=8xgZgzR3+iXmgXvO7FiGQ4jCfuQO3NAtjWRUedn+/OM=; b=eFPZKMFg18rkZHZzmgwLFW2ZrHzjrEYYp3VRskey5X8hU56A8Kaf/tEOhzJ6cMcwnFUK+BJjDhyxGJp/gCbvEIN4TP0K0082fl69pqONbNoimcud3mEO3ossu0FjiwXM8Y2bslnZJtu45sYYeySEDKRfUHIpWJ/tISxBJsKL0SYEUgvswHKmosWQcwPT9QCcTBouVZrDourB6BKTCgVuYwX9G8YSyio6TlNgtJT5x7zBXhKw1PiLJifUQcOFwaGQRtxHKB3a9Yt+h+ymafUSlyfMrrPwX4U4IioUggZqNRx/K/z6wC4NRUfH9Ta/RoDso3peZFQ9t36qA3UqS2sB1A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=intel.com smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8xgZgzR3+iXmgXvO7FiGQ4jCfuQO3NAtjWRUedn+/OM=; b=CiRGA46NfAnhspT1SPFzEBo0Rvlv6Wp5BgeQMC/Ltjiipw1jDvXEq4wK83ynYHXEtcbikcjcDbIwDchwBEusegJ5yiWtC0bqsvms5zcUepPrFW/lIo0zmoa2I0cmYEzjBICu+6DI7STxvYxmyAuCW4Y4LlJB6pREYV1nfc1NSldeF351qLmHzGtAlQqqsGshrv/4e2LsKmX3TYPuAvdaX81RrxO71Pyh8skAwh2UfYe1iuJFiLTvDOUmbfZvniiZDaA0ft4vrrvd5L4dLdv3qPMWudAANuxPSmkv28IelNE3/osy32BZ19PpLFLxzMtumvdRo8SYzIzUfXFbSS66tQ== Received: from BN9PR03CA0284.namprd03.prod.outlook.com (2603:10b6:408:f5::19) by DM6PR12MB2875.namprd12.prod.outlook.com (2603:10b6:5:15c::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4628.20; Mon, 1 Nov 2021 09:15:54 +0000 Received: from BN8NAM11FT029.eop-nam11.prod.protection.outlook.com (2603:10b6:408:f5:cafe::67) by BN9PR03CA0284.outlook.office365.com (2603:10b6:408:f5::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4649.15 via Frontend Transport; Mon, 1 Nov 2021 09:15:53 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT029.mail.protection.outlook.com (10.13.177.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4649.14 via Frontend Transport; Mon, 1 Nov 2021 09:15:52 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Mon, 1 Nov 2021 09:15:38 +0000 From: Gregory Etelson To: , , CC: , Anatoly Burakov Date: Mon, 1 Nov 2021 11:15:10 +0200 Message-ID: <20211101091514.3891-6-getelson@nvidia.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20211101091514.3891-1-getelson@nvidia.com> References: <20211101091514.3891-1-getelson@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL111.nvidia.com (172.20.187.18) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: ddd6631d-6962-4fdd-13c8-08d99d183465 X-MS-TrafficTypeDiagnostic: DM6PR12MB2875: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:326; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 1zerx2XN7zltlqu1YIYp8WphQ5mCK1wntpuKXA48+orNtLDs2H5LUD4dGJCp4J3/ah+B2ktRdYSuw9j+rV3L2yITJ8Hr+EQzthbBFgPiIir159SVWAfPz1FPMN75ur/YIYIEl53CT3n1XoaK2fo6DNWCjk8uNnuSFw8d423GcPzU7OzkdElNQ5FpxMQSwPnywcVSk6oQM4JFLXppZFpdlFYPq/fRUgWIq7Ck73fMm5CyoOLGrsF/hz13BDhH0Jq8ZGdkXPbbmuem5dhnokRX5pY/pN3BWVtdMmnEcPahsAm1n6MT+NgrTOtucB88BWUtZS/A/qr0N2e+DL/t8GMO55Dm5A+NZ5y89x+1BSgGYcucxyzy7zccGuRohdL9ZpRHtSVa2dgqRTLXb4VhnK0IB5eoAqmA6R4puou6bailq80F245xfalYGzQKA7RlMytyDj51TrHyIvR09bwa8Vd0KSF2Xdmbbw7o4LZvypXUWTkg1ADI0YnMRp01366pOgw5ux3g9K98yRoelifIWCxDBCcVYSLcNllGDDvy4vvI3/W++e83lz+9Gcb619QQbUluXPs+c9pwqQBDakDeFOtgX0BENjpt2y2RnHZAJKEjw6PRkAuI/ULUfWP03lB+09dR1iDIv6A7hz9kuj7gtrCfK6rCcO//IvBETyCmN2oqN2C0+CcU0HUmxJT3hC4dZSr+K2eywFetAI0kPmgpMgRFIkJthzd3rY/Yd53AG6/LvNg= X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(46966006)(36840700001)(336012)(30864003)(47076005)(6286002)(8676002)(54906003)(110136005)(356005)(36756003)(86362001)(316002)(7696005)(6636002)(70586007)(1076003)(4326008)(70206006)(6666004)(55016002)(508600001)(2616005)(7636003)(82310400003)(8936002)(26005)(186003)(83380400001)(2906002)(426003)(16526019)(5660300002)(36860700001)(21314003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Nov 2021 09:15:52.8564 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ddd6631d-6962-4fdd-13c8-08d99d183465 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT029.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB2875 Subject: [dpdk-dev] [PATCH 5/9] net/mlx5: add flex item API X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Viacheslav Ovsiienko This patch is a preparation step of implementing flex item feature in driver and it provides: - external entry point routines for flex item creation/deletion - flex item objects management over the ports. The flex item object keeps information about the item created over the port - reference counter to track whether item is in use by some active flows and the pointer to underlying shared DevX object, providing all the data needed to translate the flow flex pattern into matcher fields according hardware configuration. There is not too many flex items supposed to be created on the port, the design is optimized rather for flow insertion rate than memory savings. Signed-off-by: Gregory Etelson Signed-off-by: Viacheslav Ovsiienko --- drivers/net/mlx5/linux/mlx5_os.c | 4 + drivers/net/mlx5/meson.build | 1 + drivers/net/mlx5/mlx5.c | 2 +- drivers/net/mlx5/mlx5.h | 24 ++++ drivers/net/mlx5/mlx5_flow.c | 49 ++++++++ drivers/net/mlx5/mlx5_flow.h | 18 ++- drivers/net/mlx5/mlx5_flow_dv.c | 3 +- drivers/net/mlx5/mlx5_flow_flex.c | 189 ++++++++++++++++++++++++++++++ 8 files changed, 286 insertions(+), 4 deletions(-) create mode 100644 drivers/net/mlx5/mlx5_flow_flex.c diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index 72bbb665cf..cf5c5b9722 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -1682,6 +1682,8 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, err = mlx5_alloc_shared_dr(priv); if (err) goto error; + if (mlx5_flex_item_port_init(eth_dev) < 0) + goto error; } if (sh->devx && config->dv_flow_en && config->dest_tir) { priv->obj_ops = devx_obj_ops; @@ -1810,6 +1812,8 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, claim_zero(rte_eth_switch_domain_free(priv->domain_id)); if (priv->hrxqs) mlx5_list_destroy(priv->hrxqs); + if (eth_dev && priv->flex_item_map) + mlx5_flex_item_port_cleanup(eth_dev); mlx5_free(priv); if (eth_dev != NULL) eth_dev->data->dev_private = NULL; diff --git a/drivers/net/mlx5/meson.build b/drivers/net/mlx5/meson.build index 636a1be890..2f6d8cbb3d 100644 --- a/drivers/net/mlx5/meson.build +++ b/drivers/net/mlx5/meson.build @@ -17,6 +17,7 @@ sources = files( 'mlx5_flow_meter.c', 'mlx5_flow_dv.c', 'mlx5_flow_aso.c', + 'mlx5_flow_flex.c', 'mlx5_mac.c', 'mlx5_rss.c', 'mlx5_rx.c', diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index aee4fbb5ed..8166d6272c 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -378,7 +378,6 @@ static const struct mlx5_indexed_pool_config mlx5_ipool_cfg[] = { }, }; - #define MLX5_FLOW_MIN_ID_POOL_SIZE 512 #define MLX5_ID_GENERATION_ARRAY_FACTOR 16 @@ -1682,6 +1681,7 @@ mlx5_dev_close(struct rte_eth_dev *dev) mlx5_mp_os_req_stop_rxtx(dev); /* Free the eCPRI flex parser resource. */ mlx5_flex_parser_ecpri_release(dev); + mlx5_flex_item_port_cleanup(dev); if (priv->rxqs != NULL) { /* XXX race condition if mlx5_rx_burst() is still running. */ rte_delay_us_sleep(1000); diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 8ae66c4f34..75906da2c0 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -50,6 +50,9 @@ #define MLX5_MAX_MODIFY_NUM 32 #define MLX5_ROOT_TBL_MODIFY_NUM 16 +/* Maximal number of flex items created on the port.*/ +#define MLX5_PORT_FLEX_ITEM_NUM 4 + enum mlx5_ipool_index { #if defined(HAVE_IBV_FLOW_DV_SUPPORT) || !defined(HAVE_INFINIBAND_VERBS_H) MLX5_IPOOL_DECAP_ENCAP = 0, /* Pool for encap/decap resource. */ @@ -1098,6 +1101,12 @@ struct mlx5_lag { uint8_t affinity_mode; /* TIS or hash based affinity */ }; +/* Port flex item context. */ +struct mlx5_flex_item { + struct mlx5_flex_parser_devx *devx_fp; /* DevX flex parser object. */ + uint32_t refcnt; /* Atomically accessed refcnt by flows. */ +}; + /* * Shared Infiniband device context for Master/Representors * which belong to same IB device with multiple IB ports. @@ -1420,6 +1429,10 @@ struct mlx5_priv { struct mlx5_devx_obj *q_counters; /* DevX queue counter object. */ uint32_t counter_set_id; /* Queue counter ID to set in DevX objects. */ uint32_t lag_affinity_idx; /* LAG mode queue 0 affinity starting. */ + rte_spinlock_t flex_item_sl; /* Flex item list spinlock. */ + struct mlx5_flex_item flex_item[MLX5_PORT_FLEX_ITEM_NUM]; + /* Flex items have been created on the port. */ + uint32_t flex_item_map; /* Map of allocated flex item elements. */ }; #define PORT_ID(priv) ((priv)->dev_data->port_id) @@ -1799,4 +1812,15 @@ mlx5_get_supported_sw_parsing_offloads(const struct mlx5_hca_attr *attr); uint32_t mlx5_get_supported_tunneling_offloads(const struct mlx5_hca_attr *attr); +/* mlx5_flow_flex.c */ + +struct rte_flow_item_flex_handle * +flow_dv_item_create(struct rte_eth_dev *dev, + const struct rte_flow_item_flex_conf *conf, + struct rte_flow_error *error); +int flow_dv_item_release(struct rte_eth_dev *dev, + const struct rte_flow_item_flex_handle *flex_handle, + struct rte_flow_error *error); +int mlx5_flex_item_port_init(struct rte_eth_dev *dev); +void mlx5_flex_item_port_cleanup(struct rte_eth_dev *dev); #endif /* RTE_PMD_MLX5_H_ */ diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index 5d19ef1e82..25ffc57f99 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -741,6 +741,14 @@ mlx5_flow_tunnel_get_restore_info(struct rte_eth_dev *dev, struct rte_mbuf *m, struct rte_flow_restore_info *info, struct rte_flow_error *err); +static struct rte_flow_item_flex_handle * +mlx5_flow_flex_item_create(struct rte_eth_dev *dev, + const struct rte_flow_item_flex_conf *conf, + struct rte_flow_error *error); +static int +mlx5_flow_flex_item_release(struct rte_eth_dev *dev, + const struct rte_flow_item_flex_handle *handle, + struct rte_flow_error *error); static const struct rte_flow_ops mlx5_flow_ops = { .validate = mlx5_flow_validate, @@ -760,6 +768,8 @@ static const struct rte_flow_ops mlx5_flow_ops = { .tunnel_action_decap_release = mlx5_flow_tunnel_action_release, .tunnel_item_release = mlx5_flow_tunnel_item_release, .get_restore_info = mlx5_flow_tunnel_get_restore_info, + .flex_item_create = mlx5_flow_flex_item_create, + .flex_item_release = mlx5_flow_flex_item_release, }; /* Tunnel information. */ @@ -9514,6 +9524,45 @@ mlx5_release_tunnel_hub(__rte_unused struct mlx5_dev_ctx_shared *sh, } #endif /* HAVE_IBV_FLOW_DV_SUPPORT */ +/* Flex flow item API */ +static struct rte_flow_item_flex_handle * +mlx5_flow_flex_item_create(struct rte_eth_dev *dev, + const struct rte_flow_item_flex_conf *conf, + struct rte_flow_error *error) +{ + static const char err_msg[] = "flex item creation unsupported"; + struct rte_flow_attr attr = { .transfer = 0 }; + const struct mlx5_flow_driver_ops *fops = + flow_get_drv_ops(flow_get_drv_type(dev, &attr)); + + if (!fops->item_create) { + DRV_LOG(ERR, "port %u %s.", dev->data->port_id, err_msg); + rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ACTION, + NULL, err_msg); + return NULL; + } + return fops->item_create(dev, conf, error); +} + +static int +mlx5_flow_flex_item_release(struct rte_eth_dev *dev, + const struct rte_flow_item_flex_handle *handle, + struct rte_flow_error *error) +{ + static const char err_msg[] = "flex item release unsupported"; + struct rte_flow_attr attr = { .transfer = 0 }; + const struct mlx5_flow_driver_ops *fops = + flow_get_drv_ops(flow_get_drv_type(dev, &attr)); + + if (!fops->item_release) { + DRV_LOG(ERR, "port %u %s.", dev->data->port_id, err_msg); + rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ACTION, + NULL, err_msg); + return -rte_errno; + } + return fops->item_release(dev, handle, error); +} + static void mlx5_dbg__print_pattern(const struct rte_flow_item *item) { diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h index 4a16f30fb7..c1b6198adf 100644 --- a/drivers/net/mlx5/mlx5_flow.h +++ b/drivers/net/mlx5/mlx5_flow.h @@ -1229,6 +1229,19 @@ typedef int (*mlx5_flow_create_def_policy_t) (struct rte_eth_dev *dev); typedef void (*mlx5_flow_destroy_def_policy_t) (struct rte_eth_dev *dev); +typedef struct rte_flow_item_flex_handle *(*mlx5_flow_item_create_t) + (struct rte_eth_dev *dev, + const struct rte_flow_item_flex_conf *conf, + struct rte_flow_error *error); +typedef int (*mlx5_flow_item_release_t) + (struct rte_eth_dev *dev, + const struct rte_flow_item_flex_handle *handle, + struct rte_flow_error *error); +typedef int (*mlx5_flow_item_update_t) + (struct rte_eth_dev *dev, + const struct rte_flow_item_flex_handle *handle, + const struct rte_flow_item_flex_conf *conf, + struct rte_flow_error *error); struct mlx5_flow_driver_ops { mlx5_flow_validate_t validate; @@ -1263,6 +1276,9 @@ struct mlx5_flow_driver_ops { mlx5_flow_action_update_t action_update; mlx5_flow_action_query_t action_query; mlx5_flow_sync_domain_t sync_domain; + mlx5_flow_item_create_t item_create; + mlx5_flow_item_release_t item_release; + mlx5_flow_item_update_t item_update; }; /* mlx5_flow.c */ @@ -1712,6 +1728,4 @@ const struct mlx5_flow_tunnel * mlx5_get_tof(const struct rte_flow_item *items, const struct rte_flow_action *actions, enum mlx5_tof_rule_type *rule_type); - - #endif /* RTE_PMD_MLX5_FLOW_H_ */ diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c index ff8f2b75e1..1b92dd75ff 100644 --- a/drivers/net/mlx5/mlx5_flow_dv.c +++ b/drivers/net/mlx5/mlx5_flow_dv.c @@ -17942,7 +17942,8 @@ const struct mlx5_flow_driver_ops mlx5_flow_dv_drv_ops = { .action_update = flow_dv_action_update, .action_query = flow_dv_action_query, .sync_domain = flow_dv_sync_domain, + .item_create = flow_dv_item_create, + .item_release = flow_dv_item_release, }; - #endif /* HAVE_IBV_FLOW_DV_SUPPORT */ diff --git a/drivers/net/mlx5/mlx5_flow_flex.c b/drivers/net/mlx5/mlx5_flow_flex.c new file mode 100644 index 0000000000..b7bc4af6fb --- /dev/null +++ b/drivers/net/mlx5/mlx5_flow_flex.c @@ -0,0 +1,189 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2021 NVIDIA Corporation & Affiliates + */ +#include +#include +#include +#include "mlx5.h" +#include "mlx5_flow.h" + +static_assert(sizeof(uint32_t) * CHAR_BIT >= MLX5_PORT_FLEX_ITEM_NUM, + "Flex item maximal number exceeds uint32_t bit width"); + +/** + * Routine called once on port initialization to init flex item + * related infrastructure initialization + * + * @param dev + * Ethernet device to perform flex item initialization + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +mlx5_flex_item_port_init(struct rte_eth_dev *dev) +{ + struct mlx5_priv *priv = dev->data->dev_private; + + rte_spinlock_init(&priv->flex_item_sl); + MLX5_ASSERT(!priv->flex_item_map); + return 0; +} + +/** + * Routine called once on port close to perform flex item + * related infrastructure cleanup. + * + * @param dev + * Ethernet device to perform cleanup + */ +void +mlx5_flex_item_port_cleanup(struct rte_eth_dev *dev) +{ + struct mlx5_priv *priv = dev->data->dev_private; + uint32_t i; + + for (i = 0; i < MLX5_PORT_FLEX_ITEM_NUM && priv->flex_item_map ; i++) { + if (priv->flex_item_map & (1 << i)) { + /* DevX object dereferencing should be provided here. */ + priv->flex_item_map &= ~(1 << i); + } + } +} + +static int +mlx5_flex_index(struct mlx5_priv *priv, struct mlx5_flex_item *item) +{ + uintptr_t start = (uintptr_t)&priv->flex_item[0]; + uintptr_t entry = (uintptr_t)item; + uintptr_t idx = (entry - start) / sizeof(struct mlx5_flex_item); + + if (entry < start || + idx >= MLX5_PORT_FLEX_ITEM_NUM || + (entry - start) % sizeof(struct mlx5_flex_item) || + !(priv->flex_item_map & (1u << idx))) + return -1; + return (int)idx; +} + +static struct mlx5_flex_item * +mlx5_flex_alloc(struct mlx5_priv *priv) +{ + struct mlx5_flex_item *item = NULL; + + rte_spinlock_lock(&priv->flex_item_sl); + if (~priv->flex_item_map) { + uint32_t idx = rte_bsf32(~priv->flex_item_map); + + if (idx < MLX5_PORT_FLEX_ITEM_NUM) { + item = &priv->flex_item[idx]; + MLX5_ASSERT(!item->refcnt); + MLX5_ASSERT(!item->devx_fp); + item->devx_fp = NULL; + __atomic_store_n(&item->refcnt, 0, __ATOMIC_RELEASE); + priv->flex_item_map |= 1u << idx; + } + } + rte_spinlock_unlock(&priv->flex_item_sl); + return item; +} + +static void +mlx5_flex_free(struct mlx5_priv *priv, struct mlx5_flex_item *item) +{ + int idx = mlx5_flex_index(priv, item); + + MLX5_ASSERT(idx >= 0 && + idx < MLX5_PORT_FLEX_ITEM_NUM && + (priv->flex_item_map & (1u << idx))); + if (idx >= 0) { + rte_spinlock_lock(&priv->flex_item_sl); + MLX5_ASSERT(!item->refcnt); + MLX5_ASSERT(!item->devx_fp); + item->devx_fp = NULL; + __atomic_store_n(&item->refcnt, 0, __ATOMIC_RELEASE); + priv->flex_item_map &= ~(1u << idx); + rte_spinlock_unlock(&priv->flex_item_sl); + } +} + +/** + * Create the flex item with specified configuration over the Ethernet device. + * + * @param dev + * Ethernet device to create flex item on. + * @param[in] conf + * Flex item configuration. + * @param[out] error + * Perform verbose error reporting if not NULL. PMDs initialize this + * structure in case of error only. + * + * @return + * Non-NULL opaque pointer on success, NULL otherwise and rte_errno is set. + */ +struct rte_flow_item_flex_handle * +flow_dv_item_create(struct rte_eth_dev *dev, + const struct rte_flow_item_flex_conf *conf, + struct rte_flow_error *error) +{ + struct mlx5_priv *priv = dev->data->dev_private; + struct mlx5_flex_item *flex; + + MLX5_ASSERT(rte_eal_process_type() == RTE_PROC_PRIMARY); + flex = mlx5_flex_alloc(priv); + if (!flex) { + rte_flow_error_set(error, ENOMEM, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL, + "too many flex items created on the port"); + return NULL; + } + RTE_SET_USED(conf); + /* Mark initialized flex item valid. */ + __atomic_add_fetch(&flex->refcnt, 1, __ATOMIC_RELEASE); + return (struct rte_flow_item_flex_handle *)flex; +} + +/** + * Release the flex item on the specified Ethernet device. + * + * @param dev + * Ethernet device to destroy flex item on. + * @param[in] handle + * Handle of the item existing on the specified device. + * @param[out] error + * Perform verbose error reporting if not NULL. PMDs initialize this + * structure in case of error only. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +flow_dv_item_release(struct rte_eth_dev *dev, + const struct rte_flow_item_flex_handle *handle, + struct rte_flow_error *error) +{ + struct mlx5_priv *priv = dev->data->dev_private; + struct mlx5_flex_item *flex = + (struct mlx5_flex_item *)(uintptr_t)handle; + uint32_t old_refcnt = 1; + + MLX5_ASSERT(rte_eal_process_type() == RTE_PROC_PRIMARY); + rte_spinlock_lock(&priv->flex_item_sl); + if (mlx5_flex_index(priv, flex) < 0) { + rte_spinlock_unlock(&priv->flex_item_sl); + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ITEM, NULL, + "invalid flex item handle value"); + } + if (!__atomic_compare_exchange_n(&flex->refcnt, &old_refcnt, 0, 0, + __ATOMIC_ACQUIRE, __ATOMIC_RELAXED)) { + rte_spinlock_unlock(&priv->flex_item_sl); + return rte_flow_error_set(error, EBUSY, + RTE_FLOW_ERROR_TYPE_ITEM, NULL, + "flex item has flow references"); + } + /* Flex item is marked as invalid, we can leave locked section. */ + rte_spinlock_unlock(&priv->flex_item_sl); + mlx5_flex_free(priv, flex); + return 0; +} -- 2.33.1