From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C122845B96; Mon, 21 Oct 2024 23:06:30 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B3CF940A77; Mon, 21 Oct 2024 23:05:50 +0200 (CEST) Received: from egress-ip11a.ess.de.barracuda.com (egress-ip11a.ess.de.barracuda.com [18.184.203.234]) by mails.dpdk.org (Postfix) with ESMTP id C185740669 for ; Mon, 21 Oct 2024 23:05:41 +0200 (CEST) Received: from EUR02-VI1-obe.outbound.protection.outlook.com (mail-vi1eur02lp2045.outbound.protection.outlook.com [104.47.11.45]) by mx-outbound8-87.eu-central-1a.ess.aws.cudaops.com (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 21 Oct 2024 21:05:39 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=jvaM6uQXbkQeC07H8wUb29HczKnCa69O/w7G6h+pT6KAs6lC1M54YD2Dom4WseLLvpp83UGTzwghVzzgLNrVQ7/fINX6tJQV7fqiBZHSxO6FiYEHqbHOUjym5rWR9mRlNEJvp4lgLuLGfPKD9h8rh5wvbNE3O5kgnuWwfbNoMCtzLgHR+e07MW6XpRgju5SEsaI35Xi9bHoHdLi9/IreYtPy2t37vYomvzm/DXRwGT6UPTRLTHqJECbMC+D/7YwRy8kf8SDsZp0ZDq9nA3WMIdWHtCuQRJcHlTQeaD64dTlfDxs68oLhApVLJGj7eDf5VtIQ6OJnIqLjw9Bo0jgQCA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=E4KqjCSNCOJJlGj+tIEkNpkFi2IQjMcniGMq6N0BG8c=; b=tbc0FyG//XqeUyVIYfa8fEwdw9uwJe9mhgMY2tLz3rGEituS8hSobsDXsyj6fIQq8TXhvbpIZVYPoNkP6/Kqr7+PyWQtL3m6tN8DejNBBEM4wRjAaYg7BWhXgKGmQxra8udf9m2hC2u255QpnGHoD6pmXjxEcBRIELJrV3YQ2Xm/WTNsfvzu6bWT0+IviAnQWgI/o8QxBfmIQNm2dEeSJYdu3+ozCVAeAFHhCRb69HtvVhoaY4LzT8Gvit4NJM6CGGQvH8We7Te9QKXX9NJTay3IzzmorbX8v7PzQy3xMAJd+F/i+rfKt//FWhN8nyC5K4yWyea0P2MInH0+OMb3Cw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=fail (sender ip is 178.72.21.4) smtp.rcpttodomain=dpdk.org smtp.mailfrom=napatech.com; dmarc=fail (p=reject sp=reject pct=100) action=oreject header.from=napatech.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=napatech.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=E4KqjCSNCOJJlGj+tIEkNpkFi2IQjMcniGMq6N0BG8c=; b=ABhqBm1uQP0o/D2AccsQ+3Z5I4KP4iU3L1UsAJ4JA2R+WGLzMFNMj1SqIVW6knp+0mzZR3CSyG7p8A7qMInibxeMjD3NfMj7m10svtjaB7M8L7iODaNZdO8NgIaJH80NKYUyjC17I7x0G6G1OaXIGp4FL4e2OiIca/GDdcE2Y0M= Received: from AS9PR04CA0145.eurprd04.prod.outlook.com (2603:10a6:20b:48a::15) by FRZP190MB2184.EURP190.PROD.OUTLOOK.COM (2603:10a6:d10:13a::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8069.27; Mon, 21 Oct 2024 21:05:37 +0000 Received: from AMS0EPF000001A8.eurprd05.prod.outlook.com (2603:10a6:20b:48a:cafe::6a) by AS9PR04CA0145.outlook.office365.com (2603:10a6:20b:48a::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8069.29 via Frontend Transport; Mon, 21 Oct 2024 21:05:36 +0000 X-MS-Exchange-Authentication-Results: spf=fail (sender IP is 178.72.21.4) smtp.mailfrom=napatech.com; dkim=none (message not signed) header.d=none;dmarc=fail action=oreject header.from=napatech.com; Received-SPF: Fail (protection.outlook.com: domain of napatech.com does not designate 178.72.21.4 as permitted sender) receiver=protection.outlook.com; client-ip=178.72.21.4; helo=localhost.localdomain; Received: from localhost.localdomain (178.72.21.4) by AMS0EPF000001A8.mail.protection.outlook.com (10.167.16.148) with Microsoft SMTP Server id 15.20.8093.14 via Frontend Transport; Mon, 21 Oct 2024 21:05:36 +0000 From: Serhii Iliushyk To: dev@dpdk.org Cc: mko-plv@napatech.com, sil-plv@napatech.com, ckm@napatech.com, andrew.rybchenko@oktetlabs.ru, ferruh.yigit@amd.com Subject: [PATCH v1 08/73] net/ntnic: add create/destroy implementation for NT flows Date: Mon, 21 Oct 2024 23:04:10 +0200 Message-ID: <20241021210527.2075431-9-sil-plv@napatech.com> X-Mailer: git-send-email 2.45.0 In-Reply-To: <20241021210527.2075431-1-sil-plv@napatech.com> References: <20241021210527.2075431-1-sil-plv@napatech.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AMS0EPF000001A8:EE_|FRZP190MB2184:EE_ Content-Type: text/plain X-MS-Office365-Filtering-Correlation-Id: 4a4dacbe-3d62-42e9-befc-08dcf2141c62 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|1800799024|36860700013|376014|82310400026; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?n2g29obkUbP/S3iajqH/ABxrb3APgzFjiGJ0kNoCsdJSasDxxB7n/2iYiwl9?= =?us-ascii?Q?ExivhF321AYCN5G0KzxRg2TdeVsHxKyKuJNQ5A2MxEUZnSHqOx7JICcsYjoW?= =?us-ascii?Q?ntdmDdogknaU/wl029wGtbyOT3P5TRtoPyozCKd1Kz97ZQXBtmdytMKCqY4x?= =?us-ascii?Q?VhC9u6r30lffjPzan+2ntmPlowDCYPfFH6dxd4tUUKsN5yxMNCzaTP+vw9ov?= =?us-ascii?Q?yeBeQYOybctaN4SkmHtVnbuDroirSECHBAxOCPhw/v9RkHllsRlg249Ln9Mb?= =?us-ascii?Q?T1u3vRvoJhvyRjzCfUMfPo9Nr39O0XMmDa9aG9QXCY49qwIkrRUF4GWlxP5C?= =?us-ascii?Q?0UfstdidjZ1itiT8uE1ouNKwQaaNQrVQYyvAf09Js9oPoTiFYmyC5sJwNuW3?= =?us-ascii?Q?crO4KG8StMJeIhtX2SiVOkzMGAfBdSzONBRVYhwiNGUQG0UZ1H/ZWqWYWyUS?= =?us-ascii?Q?IhcPFOXzrcmCo7jH/AYCNeaZpFpqAnXgJLRYlme0iW1gHy1W7nz2QM4C95mo?= =?us-ascii?Q?V+u4Wz1TOuO95yxyf9KoUbqHXSjJ8u1SVE5XaX4CwohDnQiL0YMuvyejYm5k?= =?us-ascii?Q?cTA2qoYgpOp9YJJLrKifRiFYT+qk12rYndk0z0vhzyYZ/DyeefsZs77RTHqi?= =?us-ascii?Q?MsYPZihstJ7g5eHcJ4hHU9o/lFtXb6igKWyb0cTwjZhkNFsAxCd158egZFRr?= =?us-ascii?Q?tjFvnXmuP3fB9ZjtFBdCkw1FwLKYEyDzv2ffWz9KV6al3iU5Wgaxsp77/bOl?= =?us-ascii?Q?1tqrIhtCor5jyFmTz42rpOgUkr5Uz1iMK6xG62xL3zcZo8BdiLiTd9ggJmCM?= =?us-ascii?Q?Z+bcmrDoXJv1WKqt1tIhK6K+wZVzA4NLWlebQqUcXffPJEfHhSi6lHKurOkJ?= =?us-ascii?Q?JgsjVtmk52GA0SRIvvEEgQADz2F0pnb419rmFeHUja7scNFsCHZQSPaDye9J?= =?us-ascii?Q?RgW4JlN/Vr8Qtzd56vav2Ppn//RDQYVymjfBIxgMzs7y8jxR8kAluSzATLp7?= =?us-ascii?Q?kh0qE8BQWITZ/lJupHOH8Iij7v+kUCtRFmYxFYcn17gZpUhuu0PMS0cTv1VX?= =?us-ascii?Q?U1PWL6Ha6mHz7o6de6TJe4Pt18yG3KJteir06KVaup5FIWiKYjUEMlVRwtP6?= =?us-ascii?Q?DuZ8iDL1wD9f3OFfS5svOM9sgP7CVnfgL3T90TXT6zudiNGZ24wtldNfH+Yi?= =?us-ascii?Q?7THhxJrjEso8atU4oqtr7TNX7nE0wZguz6IXxe79WpFZ5IppW5oby2RaZ04P?= =?us-ascii?Q?TizUGO5DoAxCIyi+X2d1BNROI/PmzPvBWx43yim8tgvAcKiShAxRByaYl2RJ?= =?us-ascii?Q?DtP2poBXuiUHgxyDLICMqtgRAEMaR4IO7hRcrk5miQt/46PsSGKdRfjma7nz?= =?us-ascii?Q?SlP+lu9WzoJvKlFQQLU6oCIlksTNPPM/lU17xzzaVvg+yISK6A=3D=3D?= X-Forefront-Antispam-Report: CIP:178.72.21.4; CTRY:DK; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:localhost.localdomain; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(1800799024)(36860700013)(376014)(82310400026); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: qvEVoz3TtCQDuo7Jo8WJlPZphzLWjfTM88sKeqUcXpzwJjm/gWegXwaRp6lHwvIPj7dMHjateiy/VPuih1Ufp9MiYPem/FUPsq4JE+BxakKTIZs8fKkysQwEXDMqrVfT0x1LNR7cdVmiYu8880mTeichopSC6wAPnnOBPq03jhQEKpHvXMZAczwelUjSPngsdRGMhv2M7Q/+WICzOKH7YrpvN6TGrjDwlDa6vD+b3rW1Wck41u/UZfobjDWs5phQC1k+eZpmzZNQNB0M161hlQZykSd7O3gWo9HW1u9I71M1a6VCNoK0GXQKMMzNWVytAxDeC9kimriPgc+56I34xPBzeR3URrHyeAKzh1VFxP0GO1S/nCYQ5n0FYW2WNPKSvTf39T8+MeRMLuRsganxoE+KtY7ILyUoNJ4Pm73K+mHgLkK/ejwEWY2llI3sjiXehycxvThgEWUbf3HQXgMob718T5SPUW+TJmLl+5V445g7nS8hap0NjN3wgo//xBxnty3SQAaRI5WF/dNPlMiooffEzYYx6vW6fRgGEU17trDy3UT0m8R/Tm0rLj/Rf5EIcTpancnVd79i3qBiCWjTLl5uLF1Gj4RiZPm67b4ZrSaa1Zm27HRoxOfplsM03obNvOAzY5pepOaRL+OD8EydWbhIsZY6CuIA92YQ3v78jFw= X-OriginatorOrg: napatech.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Oct 2024 21:05:36.7862 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 4a4dacbe-3d62-42e9-befc-08dcf2141c62 X-MS-Exchange-CrossTenant-Id: c4540d0b-728a-4233-9da5-9ea30c7ec3ed X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=c4540d0b-728a-4233-9da5-9ea30c7ec3ed; Ip=[178.72.21.4]; Helo=[localhost.localdomain] X-MS-Exchange-CrossTenant-AuthSource: AMS0EPF000001A8.eurprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: FRZP190MB2184 X-BESS-ID: 1729544739-302135-12910-43220-1 X-BESS-VER: 2019.1_20241018.1852 X-BESS-Apparent-Source-IP: 104.47.11.45 X-BESS-Parts: H4sIAAAAAAACA4uuVkqtKFGyUioBkjpK+cVKVsbmxqamQGYGUDTRMMkkJSklxd giKTk5ycLcIM3C1DAtKS0t0cjU1MTMRKk2FgC2NL8SQgAAAA== X-BESS-Outbound-Spam-Score: 0.00 X-BESS-Outbound-Spam-Report: Code version 3.2, rules version 3.2.2.259883 [from cloudscan9-212.eu-central-1a.ess.aws.cudaops.com] Rule breakdown below pts rule name description ---- ---------------------- -------------------------------- 0.00 BSF_BESS_OUTBOUND META: BESS Outbound X-BESS-Outbound-Spam-Status: SCORE=0.00 using account:ESS113687 scores of KILL_LEVEL=7.0 tests=BSF_BESS_OUTBOUND X-BESS-BRTS-Status: 1 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Implements flow create/destroy functions with minimal capabilities item any action port id Signed-off-by: Serhii Iliushyk --- doc/guides/nics/features/ntnic.ini | 6 + drivers/net/ntnic/include/flow_api.h | 3 + drivers/net/ntnic/include/flow_api_engine.h | 105 +++ .../ntnic/include/stream_binary_flow_api.h | 4 + drivers/net/ntnic/meson.build | 2 + drivers/net/ntnic/nthw/flow_api/flow_group.c | 44 ++ .../net/ntnic/nthw/flow_api/flow_id_table.c | 79 ++ .../net/ntnic/nthw/flow_api/flow_id_table.h | 4 + .../flow_api/profile_inline/flm_lrn_queue.c | 28 + .../flow_api/profile_inline/flm_lrn_queue.h | 14 + .../profile_inline/flow_api_hw_db_inline.c | 92 +++ .../profile_inline/flow_api_hw_db_inline.h | 64 ++ .../profile_inline/flow_api_profile_inline.c | 684 ++++++++++++++++++ 13 files changed, 1129 insertions(+) create mode 100644 drivers/net/ntnic/nthw/flow_api/profile_inline/flm_lrn_queue.c create mode 100644 drivers/net/ntnic/nthw/flow_api/profile_inline/flm_lrn_queue.h diff --git a/doc/guides/nics/features/ntnic.ini b/doc/guides/nics/features/ntnic.ini index 8b9b87bdfe..1c653fd5a0 100644 --- a/doc/guides/nics/features/ntnic.ini +++ b/doc/guides/nics/features/ntnic.ini @@ -12,3 +12,9 @@ Unicast MAC filter = Y Multicast MAC filter = Y Linux = Y x86-64 = Y + +[rte_flow items] +any = Y + +[rte_flow actions] +port_id = Y diff --git a/drivers/net/ntnic/include/flow_api.h b/drivers/net/ntnic/include/flow_api.h index 748da89262..667dad6d5f 100644 --- a/drivers/net/ntnic/include/flow_api.h +++ b/drivers/net/ntnic/include/flow_api.h @@ -68,6 +68,9 @@ struct flow_nic_dev { uint32_t flow_unique_id_counter; /* linked list of all flows created on this NIC */ struct flow_handle *flow_base; + /* linked list of all FLM flows created on this NIC */ + struct flow_handle *flow_base_flm; + pthread_mutex_t flow_mtx; /* NIC backend API */ struct flow_api_backend_s be; diff --git a/drivers/net/ntnic/include/flow_api_engine.h b/drivers/net/ntnic/include/flow_api_engine.h index 2497c31a08..b8da5eafba 100644 --- a/drivers/net/ntnic/include/flow_api_engine.h +++ b/drivers/net/ntnic/include/flow_api_engine.h @@ -7,6 +7,10 @@ #define _FLOW_API_ENGINE_H_ #include +#include + +#include "hw_mod_backend.h" +#include "stream_binary_flow_api.h" /* * Resource management @@ -50,10 +54,107 @@ enum res_type_e { #define MAX_CPY_WRITERS_SUPPORTED 8 +enum flow_port_type_e { + PORT_NONE, /* not defined or drop */ + PORT_INTERNAL, /* no queues attached */ + PORT_PHY, /* MAC phy output queue */ + PORT_VIRT, /* Memory queues to Host */ +}; + +struct output_s { + uint32_t owning_port_id;/* the port who owns this output destination */ + enum flow_port_type_e type; + int id; /* depending on port type: queue ID or physical port id or not used */ + int active; /* activated */ +}; + +struct nic_flow_def { + /* + * Frame Decoder match info collected + */ + int l2_prot; + int l3_prot; + int l4_prot; + int tunnel_prot; + int tunnel_l3_prot; + int tunnel_l4_prot; + int vlans; + int fragmentation; + int ip_prot; + int tunnel_ip_prot; + /* + * Additional meta data for various functions + */ + int in_port_override; + int non_empty; /* default value is -1; value 1 means flow actions update */ + struct output_s dst_id[MAX_OUTPUT_DEST];/* define the output to use */ + /* total number of available queues defined for all outputs - i.e. number of dst_id's */ + int dst_num_avail; + + /* + * Mark or Action info collection + */ + uint32_t mark; + + uint32_t jump_to_group; + + int full_offload; +}; + +enum flow_handle_type { + FLOW_HANDLE_TYPE_FLOW, + FLOW_HANDLE_TYPE_FLM, +}; struct flow_handle { + enum flow_handle_type type; + uint32_t flm_id; + uint16_t caller_id; + uint16_t learn_ignored; + struct flow_eth_dev *dev; struct flow_handle *next; + struct flow_handle *prev; + + void *user_data; + + union { + struct { + /* + * 1st step conversion and validation of flow + * verified and converted flow match + actions structure + */ + struct nic_flow_def *fd; + /* + * 2nd step NIC HW resource allocation and configuration + * NIC resource management structures + */ + struct { + uint32_t db_idx_counter; + uint32_t db_idxs[RES_COUNT]; + }; + uint32_t port_id; /* MAC port ID or override of virtual in_port */ + }; + + struct { + uint32_t flm_db_idx_counter; + uint32_t flm_db_idxs[RES_COUNT]; + + uint32_t flm_data[10]; + uint8_t flm_prot; + uint8_t flm_kid; + uint8_t flm_prio; + uint8_t flm_ft; + + uint16_t flm_rpl_ext_ptr; + uint32_t flm_nat_ipv4; + uint16_t flm_nat_port; + uint8_t flm_dscp; + uint32_t flm_teid; + uint8_t flm_rqi; + uint8_t flm_qfi; + }; + }; }; void km_free_ndev_resource_management(void **handle); @@ -65,4 +166,8 @@ void kcc_free_ndev_resource_management(void **handle); */ int flow_group_handle_create(void **handle, uint32_t group_count); int flow_group_handle_destroy(void **handle); + +int flow_group_translate_get(void *handle, uint8_t owner_id, uint8_t port_id, uint32_t group_in, + uint32_t *group_out); + #endif /* _FLOW_API_ENGINE_H_ */ diff --git a/drivers/net/ntnic/include/stream_binary_flow_api.h b/drivers/net/ntnic/include/stream_binary_flow_api.h index a6244d4082..d878b848c2 100644 --- a/drivers/net/ntnic/include/stream_binary_flow_api.h +++ b/drivers/net/ntnic/include/stream_binary_flow_api.h @@ -8,6 +8,10 @@ #include "rte_flow.h" #include "rte_flow_driver.h" + +/* Max RSS hash key length in bytes */ +#define MAX_RSS_KEY_LEN 40 + /* * Flow frontend for binary programming interface */ diff --git a/drivers/net/ntnic/meson.build b/drivers/net/ntnic/meson.build index f7292144ac..e1fef37ccb 100644 --- a/drivers/net/ntnic/meson.build +++ b/drivers/net/ntnic/meson.build @@ -50,6 +50,8 @@ sources = files( 'nthw/flow_api/flow_api.c', 'nthw/flow_api/flow_group.c', 'nthw/flow_api/flow_id_table.c', + 'nthw/flow_api/hw_mod/hw_mod_backend.c', + 'nthw/flow_api/profile_inline/flm_lrn_queue.c', 'nthw/flow_api/profile_inline/flow_api_profile_inline.c', 'nthw/flow_api/profile_inline/flow_api_hw_db_inline.c', 'nthw/flow_api/flow_backend/flow_backend.c', diff --git a/drivers/net/ntnic/nthw/flow_api/flow_group.c b/drivers/net/ntnic/nthw/flow_api/flow_group.c index a7371f3aad..f76986b178 100644 --- a/drivers/net/ntnic/nthw/flow_api/flow_group.c +++ b/drivers/net/ntnic/nthw/flow_api/flow_group.c @@ -53,3 +53,47 @@ int flow_group_handle_destroy(void **handle) return 0; } + +int flow_group_translate_get(void *handle, uint8_t owner_id, uint8_t port_id, uint32_t group_in, + uint32_t *group_out) +{ + struct group_handle_s *group_handle = (struct group_handle_s *)handle; + uint32_t *table_ptr; + uint32_t lookup; + + if (group_handle == NULL || group_in >= group_handle->group_count || port_id >= PORT_COUNT) + return -1; + + /* Don't translate group 0 */ + if (group_in == 0) { + *group_out = 0; + return 0; + } + + table_ptr = &group_handle->translation_table[port_id * OWNER_ID_COUNT * PORT_COUNT + + owner_id * OWNER_ID_COUNT + group_in]; + lookup = *table_ptr; + + if (lookup == 0) { + for (lookup = 1; lookup < group_handle->group_count && + group_handle->lookup_entries[lookup].ref_counter > 0; + ++lookup) + ; + + if (lookup < group_handle->group_count) { + group_handle->lookup_entries[lookup].reverse_lookup = table_ptr; + group_handle->lookup_entries[lookup].ref_counter += 1; + + *table_ptr = lookup; + + } else { + return -1; + } + + } else { + group_handle->lookup_entries[lookup].ref_counter += 1; + } + + *group_out = lookup; + return 0; +} diff --git a/drivers/net/ntnic/nthw/flow_api/flow_id_table.c b/drivers/net/ntnic/nthw/flow_api/flow_id_table.c index 9b46848e59..5635ac4524 100644 --- a/drivers/net/ntnic/nthw/flow_api/flow_id_table.c +++ b/drivers/net/ntnic/nthw/flow_api/flow_id_table.c @@ -4,6 +4,7 @@ */ #include +#include #include #include @@ -11,6 +12,10 @@ #define NTNIC_ARRAY_BITS 14 #define NTNIC_ARRAY_SIZE (1 << NTNIC_ARRAY_BITS) +#define NTNIC_ARRAY_MASK (NTNIC_ARRAY_SIZE - 1) +#define NTNIC_MAX_ID (NTNIC_ARRAY_SIZE * NTNIC_ARRAY_SIZE) +#define NTNIC_MAX_ID_MASK (NTNIC_MAX_ID - 1) +#define NTNIC_MIN_FREE 1000 struct ntnic_id_table_element { union flm_handles handle; @@ -29,6 +34,36 @@ struct ntnic_id_table_data { uint32_t free_count; }; +static inline struct ntnic_id_table_element * +ntnic_id_table_array_find_element(struct ntnic_id_table_data *handle, uint32_t id) +{ + uint32_t idx_d1 = id & NTNIC_ARRAY_MASK; + uint32_t idx_d2 = (id >> NTNIC_ARRAY_BITS) & NTNIC_ARRAY_MASK; + + if (handle->arrays[idx_d2] == NULL) { + handle->arrays[idx_d2] = + calloc(NTNIC_ARRAY_SIZE, sizeof(struct ntnic_id_table_element)); + } + + return &handle->arrays[idx_d2][idx_d1]; +} + +static inline uint32_t ntnic_id_table_array_pop_free_id(struct ntnic_id_table_data *handle) +{ + uint32_t id = 0; + + if (handle->free_count > NTNIC_MIN_FREE) { + struct ntnic_id_table_element *element = + ntnic_id_table_array_find_element(handle, handle->free_tail); + id = handle->free_tail; + + handle->free_tail = element->handle.idx & NTNIC_MAX_ID_MASK; + handle->free_count -= 1; + } + + return id; +} + void *ntnic_id_table_create(void) { struct ntnic_id_table_data *handle = calloc(1, sizeof(struct ntnic_id_table_data)); @@ -50,3 +85,47 @@ void ntnic_id_table_destroy(void *id_table) free(id_table); } + +uint32_t ntnic_id_table_get_id(void *id_table, union flm_handles flm_h, uint8_t caller_id, + uint8_t type) +{ + struct ntnic_id_table_data *handle = id_table; + + pthread_mutex_lock(&handle->mtx); + + uint32_t new_id = ntnic_id_table_array_pop_free_id(handle); + + if (new_id == 0) + new_id = handle->next_id++; + + struct ntnic_id_table_element *element = ntnic_id_table_array_find_element(handle, new_id); + element->caller_id = caller_id; + element->type = type; + memcpy(&element->handle, &flm_h, sizeof(union flm_handles)); + + pthread_mutex_unlock(&handle->mtx); + + return new_id; +} + +void ntnic_id_table_free_id(void *id_table, uint32_t id) +{ + struct ntnic_id_table_data *handle = id_table; + + pthread_mutex_lock(&handle->mtx); + + struct ntnic_id_table_element *current_element = + ntnic_id_table_array_find_element(handle, id); + memset(current_element, 0, sizeof(struct ntnic_id_table_element)); + + struct ntnic_id_table_element *element = + ntnic_id_table_array_find_element(handle, handle->free_head); + element->handle.idx = id; + handle->free_head = id; + handle->free_count += 1; + + if (handle->free_tail == 0) + handle->free_tail = handle->free_head; + + pthread_mutex_unlock(&handle->mtx); +} diff --git a/drivers/net/ntnic/nthw/flow_api/flow_id_table.h b/drivers/net/ntnic/nthw/flow_api/flow_id_table.h index 13455f1165..e190fe4a11 100644 --- a/drivers/net/ntnic/nthw/flow_api/flow_id_table.h +++ b/drivers/net/ntnic/nthw/flow_api/flow_id_table.h @@ -16,4 +16,8 @@ union flm_handles { void *ntnic_id_table_create(void); void ntnic_id_table_destroy(void *id_table); +uint32_t ntnic_id_table_get_id(void *id_table, union flm_handles flm_h, uint8_t caller_id, + uint8_t type); +void ntnic_id_table_free_id(void *id_table, uint32_t id); + #endif /* FLOW_ID_TABLE_H_ */ diff --git a/drivers/net/ntnic/nthw/flow_api/profile_inline/flm_lrn_queue.c b/drivers/net/ntnic/nthw/flow_api/profile_inline/flm_lrn_queue.c new file mode 100644 index 0000000000..ad7efafe08 --- /dev/null +++ b/drivers/net/ntnic/nthw/flow_api/profile_inline/flm_lrn_queue.c @@ -0,0 +1,28 @@ +/* + * SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2024 Napatech A/S + */ + +#include +#include +#include + +#include + +#include "hw_mod_flm_v25.h" + +#include "flm_lrn_queue.h" + +#define ELEM_SIZE sizeof(struct flm_v25_lrn_data_s) + +uint32_t *flm_lrn_queue_get_write_buffer(void *q) +{ + struct rte_ring_zc_data zcd; + unsigned int n = rte_ring_enqueue_zc_burst_elem_start(q, ELEM_SIZE, 1, &zcd, NULL); + return (n == 0) ? NULL : zcd.ptr1; +} + +void flm_lrn_queue_release_write_buffer(void *q) +{ + rte_ring_enqueue_zc_elem_finish(q, 1); +} diff --git a/drivers/net/ntnic/nthw/flow_api/profile_inline/flm_lrn_queue.h b/drivers/net/ntnic/nthw/flow_api/profile_inline/flm_lrn_queue.h new file mode 100644 index 0000000000..8cee0c8e78 --- /dev/null +++ b/drivers/net/ntnic/nthw/flow_api/profile_inline/flm_lrn_queue.h @@ -0,0 +1,14 @@ +/* + * SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2024 Napatech A/S + */ + +#ifndef _FLM_LRN_QUEUE_H_ +#define _FLM_LRN_QUEUE_H_ + +#include + +uint32_t *flm_lrn_queue_get_write_buffer(void *q); +void flm_lrn_queue_release_write_buffer(void *q); + +#endif /* _FLM_LRN_QUEUE_H_ */ diff --git a/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_hw_db_inline.c b/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_hw_db_inline.c index 5fda11183c..eb6bad07b8 100644 --- a/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_hw_db_inline.c +++ b/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_hw_db_inline.c @@ -3,6 +3,9 @@ */ +#include "hw_mod_backend.h" +#include "flow_api_engine.h" + #include "flow_api_hw_db_inline.h" /******************************************************************************/ @@ -57,3 +60,92 @@ void hw_db_inline_destroy(void *db_handle) free(db); } + +void hw_db_inline_deref_idxs(struct flow_nic_dev *ndev, void *db_handle, struct hw_db_idx *idxs, + uint32_t size) +{ + for (uint32_t i = 0; i < size; ++i) { + switch (idxs[i].type) { + case HW_DB_IDX_TYPE_NONE: + break; + + case HW_DB_IDX_TYPE_COT: + hw_db_inline_cot_deref(ndev, db_handle, *(struct hw_db_cot_idx *)&idxs[i]); + break; + + default: + break; + } + } +} + +/******************************************************************************/ +/* COT */ +/******************************************************************************/ + +static int hw_db_inline_cot_compare(const struct hw_db_inline_cot_data *data1, + const struct hw_db_inline_cot_data *data2) +{ + return data1->matcher_color_contrib == data2->matcher_color_contrib && + data1->frag_rcp == data2->frag_rcp; +} + +struct hw_db_cot_idx hw_db_inline_cot_add(struct flow_nic_dev *ndev, void *db_handle, + const struct hw_db_inline_cot_data *data) +{ + struct hw_db_inline_resource_db *db = (struct hw_db_inline_resource_db *)db_handle; + struct hw_db_cot_idx idx = { .raw = 0 }; + int found = 0; + + idx.type = HW_DB_IDX_TYPE_COT; + + for (uint32_t i = 1; i < db->nb_cot; ++i) { + int ref = db->cot[i].ref; + + if (ref > 0 && hw_db_inline_cot_compare(data, &db->cot[i].data)) { + idx.ids = i; + hw_db_inline_cot_ref(ndev, db, idx); + return idx; + } + + if (!found && ref <= 0) { + found = 1; + idx.ids = i; + } + } + + if (!found) { + idx.error = 1; + return idx; + } + + db->cot[idx.ids].ref = 1; + memcpy(&db->cot[idx.ids].data, data, sizeof(struct hw_db_inline_cot_data)); + + return idx; +} + +void hw_db_inline_cot_ref(struct flow_nic_dev *ndev, void *db_handle, struct hw_db_cot_idx idx) +{ + (void)ndev; + struct hw_db_inline_resource_db *db = (struct hw_db_inline_resource_db *)db_handle; + + if (!idx.error) + db->cot[idx.ids].ref += 1; +} + +void hw_db_inline_cot_deref(struct flow_nic_dev *ndev, void *db_handle, struct hw_db_cot_idx idx) +{ + (void)ndev; + struct hw_db_inline_resource_db *db = (struct hw_db_inline_resource_db *)db_handle; + + if (idx.error) + return; + + db->cot[idx.ids].ref -= 1; + + if (db->cot[idx.ids].ref <= 0) { + memset(&db->cot[idx.ids].data, 0x0, sizeof(struct hw_db_inline_cot_data)); + db->cot[idx.ids].ref = 0; + } +} diff --git a/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_hw_db_inline.h b/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_hw_db_inline.h index 23caf73cf3..0116af015d 100644 --- a/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_hw_db_inline.h +++ b/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_hw_db_inline.h @@ -9,15 +9,79 @@ #include "flow_api.h" +#define HW_DB_INLINE_MAX_QST_PER_QSL 128 +#define HW_DB_INLINE_MAX_ENCAP_SIZE 128 + +#define HW_DB_IDX \ + union { \ + struct { \ + uint32_t id1 : 8; \ + uint32_t id2 : 8; \ + uint32_t id3 : 8; \ + uint32_t type : 7; \ + uint32_t error : 1; \ + }; \ + struct { \ + uint32_t ids : 24; \ + }; \ + uint32_t raw; \ + } + +/* Strongly typed int types */ +struct hw_db_idx { + HW_DB_IDX; +}; + +struct hw_db_cot_idx { + HW_DB_IDX; +}; + +enum hw_db_idx_type { + HW_DB_IDX_TYPE_NONE = 0, + HW_DB_IDX_TYPE_COT, +}; + +/* Functionality data types */ +struct hw_db_inline_qsl_data { + uint32_t discard : 1; + uint32_t drop : 1; + uint32_t table_size : 7; + uint32_t retransmit : 1; + uint32_t padding : 22; + + struct { + uint16_t queue : 7; + uint16_t queue_en : 1; + uint16_t tx_port : 3; + uint16_t tx_port_en : 1; + uint16_t padding : 4; + } table[HW_DB_INLINE_MAX_QST_PER_QSL]; +}; + struct hw_db_inline_cot_data { uint32_t matcher_color_contrib : 4; uint32_t frag_rcp : 4; uint32_t padding : 24; }; +struct hw_db_inline_hsh_data { + uint32_t func; + uint64_t hash_mask; + uint8_t key[MAX_RSS_KEY_LEN]; +}; + /**/ int hw_db_inline_create(struct flow_nic_dev *ndev, void **db_handle); void hw_db_inline_destroy(void *db_handle); +void hw_db_inline_deref_idxs(struct flow_nic_dev *ndev, void *db_handle, struct hw_db_idx *idxs, + uint32_t size); + +/**/ +struct hw_db_cot_idx hw_db_inline_cot_add(struct flow_nic_dev *ndev, void *db_handle, + const struct hw_db_inline_cot_data *data); +void hw_db_inline_cot_ref(struct flow_nic_dev *ndev, void *db_handle, struct hw_db_cot_idx idx); +void hw_db_inline_cot_deref(struct flow_nic_dev *ndev, void *db_handle, struct hw_db_cot_idx idx); + #endif /* _FLOW_API_HW_DB_INLINE_H_ */ diff --git a/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_profile_inline.c b/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_profile_inline.c index 986196b408..eb1f3227ed 100644 --- a/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_profile_inline.c +++ b/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_profile_inline.c @@ -4,13 +4,573 @@ */ #include "ntlog.h" +#include "nt_util.h" + +#include "hw_mod_backend.h" +#include "flm_lrn_queue.h" +#include "flow_api.h" #include "flow_api_engine.h" #include "flow_api_hw_db_inline.h" #include "flow_id_table.h" +#include "stream_binary_flow_api.h" #include "flow_api_profile_inline.h" #include "ntnic_mod_reg.h" +#define NT_FLM_OP_UNLEARN 0 +#define NT_FLM_OP_LEARN 1 + +static void *flm_lrn_queue_arr; + +struct flm_flow_key_def_s { + union { + struct { + uint64_t qw0_dyn : 7; + uint64_t qw0_ofs : 8; + uint64_t qw4_dyn : 7; + uint64_t qw4_ofs : 8; + uint64_t sw8_dyn : 7; + uint64_t sw8_ofs : 8; + uint64_t sw9_dyn : 7; + uint64_t sw9_ofs : 8; + uint64_t outer_proto : 1; + uint64_t inner_proto : 1; + uint64_t pad : 2; + }; + uint64_t data; + }; + uint32_t mask[10]; +}; + +/* + * Flow Matcher functionality + */ +static uint8_t get_port_from_port_id(const struct flow_nic_dev *ndev, uint32_t port_id) +{ + struct flow_eth_dev *dev = ndev->eth_base; + + while (dev) { + if (dev->port_id == port_id) + return dev->port; + + dev = dev->next; + } + + return UINT8_MAX; +} + +static void nic_insert_flow(struct flow_nic_dev *ndev, struct flow_handle *fh) +{ + pthread_mutex_lock(&ndev->flow_mtx); + + if (ndev->flow_base) + ndev->flow_base->prev = fh; + + fh->next = ndev->flow_base; + fh->prev = NULL; + ndev->flow_base = fh; + + pthread_mutex_unlock(&ndev->flow_mtx); +} + +static void nic_remove_flow(struct flow_nic_dev *ndev, struct flow_handle *fh) +{ + struct flow_handle *next = fh->next; + struct flow_handle *prev = fh->prev; + + pthread_mutex_lock(&ndev->flow_mtx); + + if (next && prev) { + prev->next = next; + next->prev = prev; + + } else if (next) { + ndev->flow_base = next; + next->prev = NULL; + + } else if (prev) { + prev->next = NULL; + + } else if (ndev->flow_base == fh) { + ndev->flow_base = NULL; + } + + pthread_mutex_unlock(&ndev->flow_mtx); +} + +static void nic_insert_flow_flm(struct flow_nic_dev *ndev, struct flow_handle *fh) +{ + pthread_mutex_lock(&ndev->flow_mtx); + + if (ndev->flow_base_flm) + ndev->flow_base_flm->prev = fh; + + fh->next = ndev->flow_base_flm; + fh->prev = NULL; + ndev->flow_base_flm = fh; + + pthread_mutex_unlock(&ndev->flow_mtx); +} + +static void nic_remove_flow_flm(struct flow_nic_dev *ndev, struct flow_handle *fh_flm) +{ + struct flow_handle *next = fh_flm->next; + struct flow_handle *prev = fh_flm->prev; + + pthread_mutex_lock(&ndev->flow_mtx); + + if (next && prev) { + prev->next = next; + next->prev = prev; + + } else if (next) { + ndev->flow_base_flm = next; + next->prev = NULL; + + } else if (prev) { + prev->next = NULL; + + } else if (ndev->flow_base_flm == fh_flm) { + ndev->flow_base_flm = NULL; + } + + pthread_mutex_unlock(&ndev->flow_mtx); +} + +static inline struct nic_flow_def *prepare_nic_flow_def(struct nic_flow_def *fd) +{ + if (fd) { + fd->full_offload = -1; + fd->in_port_override = -1; + fd->mark = UINT32_MAX; + fd->jump_to_group = UINT32_MAX; + + fd->l2_prot = -1; + fd->l3_prot = -1; + fd->l4_prot = -1; + fd->vlans = 0; + fd->tunnel_prot = -1; + fd->tunnel_l3_prot = -1; + fd->tunnel_l4_prot = -1; + fd->fragmentation = -1; + fd->ip_prot = -1; + fd->tunnel_ip_prot = -1; + + fd->non_empty = -1; + } + + return fd; +} + +static inline struct nic_flow_def *allocate_nic_flow_def(void) +{ + return prepare_nic_flow_def(calloc(1, sizeof(struct nic_flow_def))); +} + +static bool fd_has_empty_pattern(const struct nic_flow_def *fd) +{ + return fd && fd->vlans == 0 && fd->l2_prot < 0 && fd->l3_prot < 0 && fd->l4_prot < 0 && + fd->tunnel_prot < 0 && fd->tunnel_l3_prot < 0 && fd->tunnel_l4_prot < 0 && + fd->ip_prot < 0 && fd->tunnel_ip_prot < 0 && fd->non_empty < 0; +} + +static inline const void *memcpy_mask_if(void *dest, const void *src, const void *mask, + size_t count) +{ + if (mask == NULL) + return src; + + unsigned char *dest_ptr = (unsigned char *)dest; + const unsigned char *src_ptr = (const unsigned char *)src; + const unsigned char *mask_ptr = (const unsigned char *)mask; + + for (size_t i = 0; i < count; ++i) + dest_ptr[i] = src_ptr[i] & mask_ptr[i]; + + return dest; +} + +static int flm_flow_programming(struct flow_handle *fh, uint32_t flm_op) +{ + struct flm_v25_lrn_data_s *learn_record = NULL; + + if (fh->type != FLOW_HANDLE_TYPE_FLM) + return -1; + + if (flm_op == NT_FLM_OP_LEARN) { + union flm_handles flm_h; + flm_h.p = fh; + fh->flm_id = ntnic_id_table_get_id(fh->dev->ndev->id_table_handle, flm_h, + fh->caller_id, 1); + } + + uint32_t flm_id = fh->flm_id; + + if (flm_op == NT_FLM_OP_UNLEARN) { + ntnic_id_table_free_id(fh->dev->ndev->id_table_handle, flm_id); + + if (fh->learn_ignored == 1) + return 0; + } + + learn_record = + (struct flm_v25_lrn_data_s *) + flm_lrn_queue_get_write_buffer(flm_lrn_queue_arr); + + while (learn_record == NULL) { + nt_os_wait_usec(1); + learn_record = + (struct flm_v25_lrn_data_s *) + flm_lrn_queue_get_write_buffer(flm_lrn_queue_arr); + } + + memset(learn_record, 0x0, sizeof(struct flm_v25_lrn_data_s)); + + learn_record->id = flm_id; + + learn_record->qw0[0] = fh->flm_data[9]; + learn_record->qw0[1] = fh->flm_data[8]; + learn_record->qw0[2] = fh->flm_data[7]; + learn_record->qw0[3] = fh->flm_data[6]; + learn_record->qw4[0] = fh->flm_data[5]; + learn_record->qw4[1] = fh->flm_data[4]; + learn_record->qw4[2] = fh->flm_data[3]; + learn_record->qw4[3] = fh->flm_data[2]; + learn_record->sw8 = fh->flm_data[1]; + learn_record->sw9 = fh->flm_data[0]; + learn_record->prot = fh->flm_prot; + + /* Last non-zero mtr is used for statistics */ + uint8_t mbrs = 0; + + learn_record->vol_idx = mbrs; + + learn_record->nat_ip = fh->flm_nat_ipv4; + learn_record->nat_port = fh->flm_nat_port; + learn_record->nat_en = fh->flm_nat_ipv4 || fh->flm_nat_port ? 1 : 0; + + learn_record->dscp = fh->flm_dscp; + learn_record->teid = fh->flm_teid; + learn_record->qfi = fh->flm_qfi; + learn_record->rqi = fh->flm_rqi; + /* Lower 10 bits used for RPL EXT PTR */ + learn_record->color = fh->flm_rpl_ext_ptr & 0x3ff; + + learn_record->ent = 0; + learn_record->op = flm_op & 0xf; + /* Suppress generation of statistics INF_DATA */ + learn_record->nofi = 1; + learn_record->prio = fh->flm_prio & 0x3; + learn_record->ft = fh->flm_ft; + learn_record->kid = fh->flm_kid; + learn_record->eor = 1; + learn_record->scrub_prof = 0; + + flm_lrn_queue_release_write_buffer(flm_lrn_queue_arr); + return 0; +} + +/* + * This function must be callable without locking any mutexes + */ +static int interpret_flow_actions(const struct flow_eth_dev *dev, + const struct rte_flow_action action[], + const struct rte_flow_action *action_mask, + struct nic_flow_def *fd, + struct rte_flow_error *error, + uint32_t *num_dest_port, + uint32_t *num_queues) +{ + unsigned int encap_decap_order = 0; + + *num_dest_port = 0; + *num_queues = 0; + + if (action == NULL) { + flow_nic_set_error(ERR_FAILED, error); + NT_LOG(ERR, FILTER, "Flow actions missing"); + return -1; + } + + /* + * Gather flow match + actions and convert into internal flow definition structure (struct + * nic_flow_def_s) This is the 1st step in the flow creation - validate, convert and + * prepare + */ + for (int aidx = 0; action[aidx].type != RTE_FLOW_ACTION_TYPE_END; ++aidx) { + switch (action[aidx].type) { + case RTE_FLOW_ACTION_TYPE_PORT_ID: + NT_LOG(DBG, FILTER, "Dev:%p: RTE_FLOW_ACTION_TYPE_PORT_ID", dev); + + if (action[aidx].conf) { + struct rte_flow_action_port_id port_id_tmp; + const struct rte_flow_action_port_id *port_id = + memcpy_mask_if(&port_id_tmp, action[aidx].conf, + action_mask ? action_mask[aidx].conf : NULL, + sizeof(struct rte_flow_action_port_id)); + + if (*num_dest_port > 0) { + NT_LOG(ERR, FILTER, + "Multiple port_id actions for one flow is not supported"); + flow_nic_set_error(ERR_ACTION_MULTIPLE_PORT_ID_UNSUPPORTED, + error); + return -1; + } + + uint8_t port = get_port_from_port_id(dev->ndev, port_id->id); + + if (fd->dst_num_avail == MAX_OUTPUT_DEST) { + NT_LOG(ERR, FILTER, "Too many output destinations"); + flow_nic_set_error(ERR_OUTPUT_TOO_MANY, error); + return -1; + } + + if (port >= dev->ndev->be.num_phy_ports) { + NT_LOG(ERR, FILTER, "Phy port out of range"); + flow_nic_set_error(ERR_OUTPUT_INVALID, error); + return -1; + } + + /* New destination port to add */ + fd->dst_id[fd->dst_num_avail].owning_port_id = port_id->id; + fd->dst_id[fd->dst_num_avail].type = PORT_PHY; + fd->dst_id[fd->dst_num_avail].id = (int)port; + fd->dst_id[fd->dst_num_avail].active = 1; + fd->dst_num_avail++; + + if (fd->full_offload < 0) + fd->full_offload = 1; + + *num_dest_port += 1; + + NT_LOG(DBG, FILTER, "Phy port ID: %i", (int)port); + } + + break; + + default: + NT_LOG(ERR, FILTER, "Invalid or unsupported flow action received - %i", + action[aidx].type); + flow_nic_set_error(ERR_ACTION_UNSUPPORTED, error); + return -1; + } + } + + if (!(encap_decap_order == 0 || encap_decap_order == 2)) { + NT_LOG(ERR, FILTER, "Invalid encap/decap actions"); + return -1; + } + + return 0; +} + +static int interpret_flow_elements(const struct flow_eth_dev *dev, + const struct rte_flow_item elem[], + struct nic_flow_def *fd, + struct rte_flow_error *error, + uint16_t implicit_vlan_vid, + uint32_t *in_port_id, + uint32_t *packet_data, + uint32_t *packet_mask, + struct flm_flow_key_def_s *key_def) +{ + (void)fd; + (void)implicit_vlan_vid; + + *in_port_id = UINT32_MAX; + + memset(packet_data, 0x0, sizeof(uint32_t) * 10); + memset(packet_mask, 0x0, sizeof(uint32_t) * 10); + memset(key_def, 0x0, sizeof(struct flm_flow_key_def_s)); + + if (elem == NULL) { + flow_nic_set_error(ERR_FAILED, error); + NT_LOG(ERR, FILTER, "Flow items missing"); + return -1; + } + + int qw_reserved_mac = 0; + int qw_reserved_ipv6 = 0; + + int qw_free = 2 - qw_reserved_mac - qw_reserved_ipv6; + + if (qw_free < 0) { + NT_LOG(ERR, FILTER, "Key size too big. Out of QW resources."); + flow_nic_set_error(ERR_FAILED, error); + return -1; + } + + for (int eidx = 0; elem[eidx].type != RTE_FLOW_ITEM_TYPE_END; ++eidx) { + switch (elem[eidx].type) { + case RTE_FLOW_ITEM_TYPE_ANY: + NT_LOG(DBG, FILTER, "Adap %i, Port %i: RTE_FLOW_ITEM_TYPE_ANY", + dev->ndev->adapter_no, dev->port); + break; + + default: + NT_LOG(ERR, FILTER, "Invalid or unsupported flow request: %d", + (int)elem[eidx].type); + flow_nic_set_error(ERR_MATCH_INVALID_OR_UNSUPPORTED_ELEM, error); + return -1; + } + } + + return 0; +} + +static int convert_fh_to_fh_flm(struct flow_handle *fh, const uint32_t *packet_data, + uint32_t flm_key_id, uint32_t flm_ft, uint16_t rpl_ext_ptr, + uint32_t flm_scrub, uint32_t priority) +{ + (void)packet_data; + (void)flm_key_id; + (void)flm_ft; + (void)rpl_ext_ptr; + (void)flm_scrub; + (void)priority; + + struct nic_flow_def *fd; + struct flow_handle fh_copy; + + if (fh->type != FLOW_HANDLE_TYPE_FLOW) + return -1; + + memcpy(&fh_copy, fh, sizeof(struct flow_handle)); + memset(fh, 0x0, sizeof(struct flow_handle)); + fd = fh_copy.fd; + + fh->type = FLOW_HANDLE_TYPE_FLM; + fh->caller_id = fh_copy.caller_id; + fh->dev = fh_copy.dev; + fh->next = fh_copy.next; + fh->prev = fh_copy.prev; + fh->user_data = fh_copy.user_data; + + fh->flm_db_idx_counter = fh_copy.db_idx_counter; + + for (int i = 0; i < RES_COUNT; ++i) + fh->flm_db_idxs[i] = fh_copy.db_idxs[i]; + + free(fd); + + return 0; +} + +static int setup_flow_flm_actions(struct flow_eth_dev *dev, + const struct nic_flow_def *fd, + const struct hw_db_inline_qsl_data *qsl_data, + const struct hw_db_inline_hsh_data *hsh_data, + uint32_t group, + uint32_t local_idxs[], + uint32_t *local_idx_counter, + uint16_t *flm_rpl_ext_ptr, + uint32_t *flm_ft, + uint32_t *flm_scrub, + struct rte_flow_error *error) +{ + (void)dev; + (void)fd; + (void)group; + (void)local_idxs; + (void)local_idx_counter; + (void)flm_rpl_ext_ptr; + (void)flm_ft; + (void)flm_scrub; + (void)qsl_data; + (void)hsh_data; + (void)error; + + return 0; +} + +static struct flow_handle *create_flow_filter(struct flow_eth_dev *dev, struct nic_flow_def *fd, + const struct rte_flow_attr *attr, + uint16_t forced_vlan_vid, uint16_t caller_id, + struct rte_flow_error *error, uint32_t port_id, + uint32_t num_dest_port, uint32_t num_queues, + uint32_t *packet_data, uint32_t *packet_mask, + struct flm_flow_key_def_s *key_def) +{ + (void)packet_mask; + (void)key_def; + (void)forced_vlan_vid; + (void)num_dest_port; + (void)num_queues; + (void)packet_data; + + struct flow_handle *fh = calloc(1, sizeof(struct flow_handle)); + + fh->type = FLOW_HANDLE_TYPE_FLOW; + fh->port_id = port_id; + fh->dev = dev; + fh->fd = fd; + fh->caller_id = caller_id; + + struct hw_db_inline_qsl_data qsl_data; + + struct hw_db_inline_hsh_data hsh_data; + + if (attr->group > 0 && fd_has_empty_pattern(fd)) { + /* + * Default flow for group 1..32 + */ + + if (setup_flow_flm_actions(dev, fd, &qsl_data, &hsh_data, attr->group, fh->db_idxs, + &fh->db_idx_counter, NULL, NULL, NULL, error)) { + goto error_out; + } + + nic_insert_flow(dev->ndev, fh); + + } else if (attr->group > 0) { + /* + * Flow for group 1..32 + */ + + /* Setup Actions */ + uint16_t flm_rpl_ext_ptr = 0; + uint32_t flm_ft = 0; + uint32_t flm_scrub = 0; + + if (setup_flow_flm_actions(dev, fd, &qsl_data, &hsh_data, attr->group, fh->db_idxs, + &fh->db_idx_counter, &flm_rpl_ext_ptr, &flm_ft, + &flm_scrub, error)) { + goto error_out; + } + + /* Program flow */ + convert_fh_to_fh_flm(fh, packet_data, 2, flm_ft, flm_rpl_ext_ptr, + flm_scrub, attr->priority & 0x3); + flm_flow_programming(fh, NT_FLM_OP_LEARN); + + nic_insert_flow_flm(dev->ndev, fh); + + } else { + /* + * Flow for group 0 + */ + nic_insert_flow(dev->ndev, fh); + } + + return fh; + +error_out: + + if (fh->type == FLOW_HANDLE_TYPE_FLM) { + hw_db_inline_deref_idxs(dev->ndev, dev->ndev->hw_db_handle, + (struct hw_db_idx *)fh->flm_db_idxs, + fh->flm_db_idx_counter); + + } else { + hw_db_inline_deref_idxs(dev->ndev, dev->ndev->hw_db_handle, + (struct hw_db_idx *)fh->db_idxs, fh->db_idx_counter); + } + + free(fh); + + return NULL; +} + /* * Public functions */ @@ -82,6 +642,92 @@ struct flow_handle *flow_create_profile_inline(struct flow_eth_dev *dev, const struct rte_flow_action action[], struct rte_flow_error *error) { + struct flow_handle *fh = NULL; + int res; + + uint32_t port_id = UINT32_MAX; + uint32_t num_dest_port; + uint32_t num_queues; + + uint32_t packet_data[10]; + uint32_t packet_mask[10]; + struct flm_flow_key_def_s key_def; + + struct rte_flow_attr attr_local; + memcpy(&attr_local, attr, sizeof(struct rte_flow_attr)); + uint16_t forced_vlan_vid_local = forced_vlan_vid; + uint16_t caller_id_local = caller_id; + + if (attr_local.group > 0) + forced_vlan_vid_local = 0; + + flow_nic_set_error(ERR_SUCCESS, error); + + struct nic_flow_def *fd = allocate_nic_flow_def(); + + if (fd == NULL) + goto err_exit; + + res = interpret_flow_actions(dev, action, NULL, fd, error, &num_dest_port, &num_queues); + + if (res) + goto err_exit; + + res = interpret_flow_elements(dev, elem, fd, error, forced_vlan_vid_local, &port_id, + packet_data, packet_mask, &key_def); + + if (res) + goto err_exit; + + pthread_mutex_lock(&dev->ndev->mtx); + + /* Translate group IDs */ + if (fd->jump_to_group != UINT32_MAX && + flow_group_translate_get(dev->ndev->group_handle, caller_id_local, dev->port, + fd->jump_to_group, &fd->jump_to_group)) { + NT_LOG(ERR, FILTER, "ERROR: Could not get group resource"); + flow_nic_set_error(ERR_MATCH_RESOURCE_EXHAUSTION, error); + goto err_exit; + } + + if (attr_local.group > 0 && + flow_group_translate_get(dev->ndev->group_handle, caller_id_local, dev->port, + attr_local.group, &attr_local.group)) { + NT_LOG(ERR, FILTER, "ERROR: Could not get group resource"); + flow_nic_set_error(ERR_MATCH_RESOURCE_EXHAUSTION, error); + goto err_exit; + } + + if (port_id == UINT32_MAX) + port_id = dev->port_id; + + /* Create and flush filter to NIC */ + fh = create_flow_filter(dev, fd, &attr_local, forced_vlan_vid_local, + caller_id_local, error, port_id, num_dest_port, num_queues, packet_data, + packet_mask, &key_def); + + if (!fh) + goto err_exit; + + NT_LOG(DBG, FILTER, "New FlOW: fh (flow handle) %p, fd (flow definition) %p", fh, fd); + NT_LOG(DBG, FILTER, ">>>>> [Dev %p] Nic %i, Port %i: fh %p fd %p - implementation <<<<<", + dev, dev->ndev->adapter_no, dev->port, fh, fd); + + pthread_mutex_unlock(&dev->ndev->mtx); + + return fh; + +err_exit: + + if (fh) + flow_destroy_locked_profile_inline(dev, fh, NULL); + + else + free(fd); + + pthread_mutex_unlock(&dev->ndev->mtx); + + NT_LOG(ERR, FILTER, "ERR: %s", __func__); return NULL; } @@ -96,6 +742,44 @@ int flow_destroy_locked_profile_inline(struct flow_eth_dev *dev, flow_nic_set_error(ERR_SUCCESS, error); + /* take flow out of ndev list - may not have been put there yet */ + if (fh->type == FLOW_HANDLE_TYPE_FLM) + nic_remove_flow_flm(dev->ndev, fh); + + else + nic_remove_flow(dev->ndev, fh); + +#ifdef FLOW_DEBUG + dev->ndev->be.iface->set_debug_mode(dev->ndev->be.be_dev, FLOW_BACKEND_DEBUG_MODE_WRITE); +#endif + + NT_LOG(DBG, FILTER, "removing flow :%p", fh); + if (fh->type == FLOW_HANDLE_TYPE_FLM) { + hw_db_inline_deref_idxs(dev->ndev, dev->ndev->hw_db_handle, + (struct hw_db_idx *)fh->flm_db_idxs, + fh->flm_db_idx_counter); + + flm_flow_programming(fh, NT_FLM_OP_UNLEARN); + + } else { + NT_LOG(DBG, FILTER, "removing flow :%p", fh); + + hw_db_inline_deref_idxs(dev->ndev, dev->ndev->hw_db_handle, + (struct hw_db_idx *)fh->db_idxs, fh->db_idx_counter); + free(fh->fd); + } + + if (err) { + NT_LOG(ERR, FILTER, "FAILED removing flow: %p", fh); + flow_nic_set_error(ERR_REMOVE_FLOW_FAILED, error); + } + + free(fh); + +#ifdef FLOW_DEBUG + dev->ndev->be.iface->set_debug_mode(dev->ndev->be.be_dev, FLOW_BACKEND_DEBUG_MODE_NONE); +#endif + return err; } -- 2.45.0