From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9ACFA45BCC; Wed, 30 Oct 2024 22:41:17 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 632F34346D; Wed, 30 Oct 2024 22:40:21 +0100 (CET) Received: from egress-ip42a.ess.de.barracuda.com (egress-ip42a.ess.de.barracuda.com [18.185.115.201]) by mails.dpdk.org (Postfix) with ESMTP id F3568433B5 for ; Wed, 30 Oct 2024 22:40:07 +0100 (CET) Received: from EUR02-DB5-obe.outbound.protection.outlook.com (mail-db5eur02lp2113.outbound.protection.outlook.com [104.47.11.113]) by mx-outbound11-85.eu-central-1a.ess.aws.cudaops.com (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 30 Oct 2024 21:40:04 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=C7wDzvy4N+66XMibLnthezOdcVlk5UFH8L7IJM2qg5jv5dYsjZ8XZPd6LE+gmghlmwXQXVEFWP4v5nz7Epf6rmc7voAjzhMtIrlq8It2b5oHAwwOMob5pluz2nOzhAesDVg8Vgl4e0oUCRWPW152stPWbPWui6x749leREDEqhIi0lxCbj5RMSetefEx2N5+2brK7eqkXW7IlOpId1D/Eo1ct8xXqonRkp3bQDCZReu+UeIDOI2PYIbyufQrhUlIYTCjKVbQ6rFoiLmEq8TiZTnjkJJeLTTKMtpEAIH8b7iy01lsj8KTbouU7+GHMzXzfE1GFZkX3ElFjcd3RQ6qVg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=pVf+tLUcrmjV6OtP/sUVT6pD+61d1HZJMa1Al0enLwE=; b=ius/cSvTlFKcza4oF4s1tXZH0cILG9cgoOkf37b6allQZJaiuzw5iKV43sRIL2r0+0patzmH+usQbYCfC26earFCbwPgmk/LJQl9zsEBJFXfUEnYJMSqtrRfBI90zxh600ZCtEYyE8IKm1EnEu99C97pQBFS8JHSgoIggRVgPeQNGMjXPtcf+4zj9MdfvJ5DYu4USJ8StzLh4rOyLQdYpfiY91ju/YyBL93lICvCLDaWV/P/wvVPNux4UCwI+SAvle4BpBwNz81Wu/lvGMJkBjq0f3sV36seL0T9E93fZoHkFgO6g2kuraIBFJ5Bjuodi9CDo/GEQSLtGhOFkt5QKg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=fail (sender ip is 178.72.21.4) smtp.rcpttodomain=dpdk.org smtp.mailfrom=napatech.com; dmarc=fail (p=reject sp=reject pct=100) action=oreject header.from=napatech.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=napatech.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=pVf+tLUcrmjV6OtP/sUVT6pD+61d1HZJMa1Al0enLwE=; b=F0tEFuZItdt4sJX3VlWuAunUuGDbsTh4h+0ejLxS/gbqno6nlyz5HN3T1vHJIh8tMJLgvXsPJXbru8MGaqL82jqabnAtYc4oWk7SZ8kWPNYGDWx56W0GMKyljYyw4W1Wl/wDZwPXE0Ev1wNY5lVmvI8/ebscS/lf0RD5Dt1GUrA= Received: from DUZPR01CA0015.eurprd01.prod.exchangelabs.com (2603:10a6:10:3c3::18) by GVXP190MB2015.EURP190.PROD.OUTLOOK.COM (2603:10a6:150:2::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.25; Wed, 30 Oct 2024 21:40:00 +0000 Received: from DU2PEPF00028D13.eurprd03.prod.outlook.com (2603:10a6:10:3c3:cafe::a5) by DUZPR01CA0015.outlook.office365.com (2603:10a6:10:3c3::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8114.20 via Frontend Transport; Wed, 30 Oct 2024 21:39:59 +0000 X-MS-Exchange-Authentication-Results: spf=fail (sender IP is 178.72.21.4) smtp.mailfrom=napatech.com; dkim=none (message not signed) header.d=none;dmarc=fail action=oreject header.from=napatech.com; Received-SPF: Fail (protection.outlook.com: domain of napatech.com does not designate 178.72.21.4 as permitted sender) receiver=protection.outlook.com; client-ip=178.72.21.4; helo=localhost.localdomain; Received: from localhost.localdomain (178.72.21.4) by DU2PEPF00028D13.mail.protection.outlook.com (10.167.242.27) with Microsoft SMTP Server id 15.20.8114.16 via Frontend Transport; Wed, 30 Oct 2024 21:39:59 +0000 From: Serhii Iliushyk To: dev@dpdk.org Cc: mko-plv@napatech.com, sil-plv@napatech.com, ckm@napatech.com, andrew.rybchenko@oktetlabs.ru, ferruh.yigit@amd.com, stephen@networkplumber.org Subject: [PATCH v5 08/80] net/ntnic: add create/destroy implementation for NT flows Date: Wed, 30 Oct 2024 22:38:15 +0100 Message-ID: <20241030213940.3470062-9-sil-plv@napatech.com> X-Mailer: git-send-email 2.45.0 In-Reply-To: <20241030213940.3470062-1-sil-plv@napatech.com> References: <20241021210527.2075431-1-sil-plv@napatech.com> <20241030213940.3470062-1-sil-plv@napatech.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DU2PEPF00028D13:EE_|GVXP190MB2015:EE_ Content-Type: text/plain X-MS-Office365-Filtering-Correlation-Id: 4fc6ee00-20a3-42c2-8917-08dcf92b67ba X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|82310400026|376014|36860700013|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?Fp+SkrqaCy9tboCxVMeEMW3Kua7kSXfa+M7HheTcYT+Uk17dDjH8w4ttAlpa?= =?us-ascii?Q?8AJxkQQ1+4qnrpVVyGufAPWx4+Ud4dlR0/WUFfgsZA8cbAYIqiwErTc2XRjX?= =?us-ascii?Q?wtSodNfXtbXgiilP8J5zyXy2svm8LfIzz9cRoB1nXI+MZMvMx6g0bZcs2NwE?= =?us-ascii?Q?yaHN05B6WD+e5MweBzqjwPeD5k+Ugv2WkMCyj+u6FUPODanwIDnvqzg+TYyH?= =?us-ascii?Q?QtGL14ZAVQbL7Ze91pmvIW/WLfKbHsRDUCYdacln0T9BXzehsW02no0kc/TC?= =?us-ascii?Q?N7RszTNkL6weaFDV6DCu3y5eym84GEZ0t4wipDUoDnfh/Y0Xa2blCfV+o9vx?= =?us-ascii?Q?AhM1wdd0pseCf5xTEXMwKW54XrIcLFYx6tRU6kfyQbOKuu3XgdcTWQWFh2VN?= =?us-ascii?Q?lUL78nAwgrYayvtWjZ4rf+bpJ11L0VNbykm8IDPE7ft9+Djuek5b+9laP0Sj?= =?us-ascii?Q?9YRSnv1Qo6fiTt+u2rxFnToCSFRUWrLSP/Mw+P1tAmYRkL9HfOd1VaE7seCh?= =?us-ascii?Q?QJQqlUI0G5HCtAbimXHTAuLJzut+DMYcD+Y3u7bH4a/lyU8/eUX4cb04vY/8?= =?us-ascii?Q?m37aatk0bj7D5PUUccSTx2nJba9Nyu0L2ZIjCDv+GQa1IJQMR+IILLqwYmVv?= =?us-ascii?Q?6tHSse+cx3Jvb62smGtNk3vquB8NTEv/XBqZGpwITCLqLHiXF7zPYzUhS9zS?= =?us-ascii?Q?BpbifRuVAHetPtNPprfqhM/Vq29T0p/ZcJsW9NHizyzJeySqxkoGKax0dTY7?= =?us-ascii?Q?PHRqPai8R9hs9jRLpExRcGTnW2gtvkTgK575lfjkJVJWHvgcY1Jk7SxXXGKX?= =?us-ascii?Q?LCFrbqNTsYtW/0J4upolQis0gHbSBRbN0q8MIx5bF9wO/UWF+hbj+Ugrzebf?= =?us-ascii?Q?MASIFTp9wilbD1QUZxM4jnpi8u0i4FCHO835XT6HjhG/STdBl8ddCQ52leSm?= =?us-ascii?Q?3N/SHGljK1Xk9KeBUWZ0aaL31RlLB3RZotkTO8d498zP+Zhon55B8OIzButo?= =?us-ascii?Q?Q0+wGvwFy41g8cVanEANn5AoMZVigBhxjLS/cg2xURFHlPlKTM9YjXwp9V+4?= =?us-ascii?Q?k42t0Dtma3sDaSNYTrslgCJ8qjOF1sxIcT4PknJAdTmz3sWopmSEYZkWpr61?= =?us-ascii?Q?6eZWUrMBmQwg9u7IZSJzn8aj84ZKmDW/rp366bF9eGj+hojCkkAjqWyiPIzi?= =?us-ascii?Q?2Kfn/8DmNYfs8eVszXA3cK1fAz0nndVuNK81LQcqshBKwAl1ViB1sU7Fyf+j?= =?us-ascii?Q?CaKplqJpr9l4KAuceSukzSk0AJZN1zOLBD4oKZgULlhtLNz3TqjdiQJHReuQ?= =?us-ascii?Q?ow+H8PsXm5kXMfvTL+lg8BiT+Wq/Sgwyug9ryLVoEn0plwcoZyk+6+pqdt3T?= =?us-ascii?Q?+qEaH+6ZG9qXouEg0b/n88zkXLX8l/SkTC7NZI+dq2gBFw80og=3D=3D?= X-Forefront-Antispam-Report: CIP:178.72.21.4; CTRY:DK; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:localhost.localdomain; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(82310400026)(376014)(36860700013)(1800799024); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 1QaJgcdjCNpbEH2pydO8Ln3AxRKLq5PPcPR23Xb5n52BpbGRSyBSU76mksBJRhMlg6DGjv37jufDnWhTk/YxO7ZekfOb6qzFoh/7X5s1cf7k7Ze72S4vA7e3XZRMMsdapDFoZ1CCbfUSWkNr0mfo5mXeEVVuvPRJnCezeimQWzPOT8JfvqnqioCrAF4WoY8HIDijU9TWdQ4EKpD/Blo1a3zwNGu1y0U3akmS0X7SaA1WsLZ5qWiVHjJ3+mju9E36/LJ6q7e/b8uLlwNR6GaA707va65THPBenyMcNd+qNZvCE9QFUfX5FMwnYX7fIylthucQwYfVkkWxfNLatcqX5RERvcQtqUJL7MOPOE0tErIYVQ/iNOA9nvvMHZ1tYIYt3bpM2pMIcc3/CWjFdS+WWIWKqZe407/GUP4vy2iL+xms8ua3zfLCuWtc9ZbTTn7w0UmnOywOEt4X5++cHoL+vvs24zQtsfga8k021EG47zRG1sF1mTNm2WZijWK1wyCyg9tmNIpnzAg9Q1TzbJ/UF05IiUzRlFiAaTqjfUxllZvQFV7uA7R0/lIQC8JZPiyS/FOEyrm45q3n0Vwd+inJaV2o2pIO7OJFKACJlBpetWNZxKvkou+Vt2XGyvye4yVjMIWEHzDXi8/tJI5ZrjLtZZEGVoludtHXoAmtL4irC5Q= X-OriginatorOrg: napatech.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Oct 2024 21:39:59.6299 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 4fc6ee00-20a3-42c2-8917-08dcf92b67ba X-MS-Exchange-CrossTenant-Id: c4540d0b-728a-4233-9da5-9ea30c7ec3ed X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=c4540d0b-728a-4233-9da5-9ea30c7ec3ed; Ip=[178.72.21.4]; Helo=[localhost.localdomain] X-MS-Exchange-CrossTenant-AuthSource: DU2PEPF00028D13.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: GVXP190MB2015 X-BESS-ID: 1730324403-302901-12799-43451-1 X-BESS-VER: 2019.1_20241018.1852 X-BESS-Apparent-Source-IP: 104.47.11.113 X-BESS-Parts: H4sIAAAAAAACA4uuVkqtKFGyUioBkjpK+cVKVsYWxuYmQGYGUDTF2DLRwDDFKN UkzTzRwjIp1djMKM0g0dzIMNHALNXCUqk2FgBbmuCIQgAAAA== X-BESS-Outbound-Spam-Score: 0.00 X-BESS-Outbound-Spam-Report: Code version 3.2, rules version 3.2.2.260091 [from cloudscan16-249.eu-central-1b.ess.aws.cudaops.com] Rule breakdown below pts rule name description ---- ---------------------- -------------------------------- 0.00 BSF_BESS_OUTBOUND META: BESS Outbound X-BESS-Outbound-Spam-Status: SCORE=0.00 using account:ESS113687 scores of KILL_LEVEL=7.0 tests=BSF_BESS_OUTBOUND X-BESS-BRTS-Status: 1 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Implements flow create/destroy functions with minimal capabilities item any action port id Signed-off-by: Serhii Iliushyk --- doc/guides/nics/features/ntnic.ini | 6 + doc/guides/nics/ntnic.rst | 2 + doc/guides/rel_notes/release_24_11.rst | 1 + drivers/net/ntnic/include/flow_api.h | 3 + drivers/net/ntnic/include/flow_api_engine.h | 105 +++ .../ntnic/include/stream_binary_flow_api.h | 4 + drivers/net/ntnic/meson.build | 2 + drivers/net/ntnic/nthw/flow_api/flow_group.c | 44 ++ .../net/ntnic/nthw/flow_api/flow_id_table.c | 79 +++ .../net/ntnic/nthw/flow_api/flow_id_table.h | 4 + .../flow_api/profile_inline/flm_lrn_queue.c | 28 + .../flow_api/profile_inline/flm_lrn_queue.h | 14 + .../profile_inline/flow_api_hw_db_inline.c | 93 +++ .../profile_inline/flow_api_hw_db_inline.h | 64 ++ .../profile_inline/flow_api_profile_inline.c | 657 ++++++++++++++++++ 15 files changed, 1106 insertions(+) create mode 100644 drivers/net/ntnic/nthw/flow_api/profile_inline/flm_lrn_queue.c create mode 100644 drivers/net/ntnic/nthw/flow_api/profile_inline/flm_lrn_queue.h diff --git a/doc/guides/nics/features/ntnic.ini b/doc/guides/nics/features/ntnic.ini index 8b9b87bdfe..1c653fd5a0 100644 --- a/doc/guides/nics/features/ntnic.ini +++ b/doc/guides/nics/features/ntnic.ini @@ -12,3 +12,9 @@ Unicast MAC filter = Y Multicast MAC filter = Y Linux = Y x86-64 = Y + +[rte_flow items] +any = Y + +[rte_flow actions] +port_id = Y diff --git a/doc/guides/nics/ntnic.rst b/doc/guides/nics/ntnic.rst index 2c160ae592..a6568cba4e 100644 --- a/doc/guides/nics/ntnic.rst +++ b/doc/guides/nics/ntnic.rst @@ -40,6 +40,8 @@ Features - Unicast MAC filter - Multicast MAC filter - Promiscuous mode (Enable only. The device always run promiscuous mode) +- Flow API support. +- Support for multiple rte_flow groups. Limitations ~~~~~~~~~~~ diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst index 15b64a1829..a235ce59d1 100644 --- a/doc/guides/rel_notes/release_24_11.rst +++ b/doc/guides/rel_notes/release_24_11.rst @@ -161,6 +161,7 @@ New Features * Added NT flow backend initialization. * Added initialization of FPGA modules related to flow HW offload. * Added basic handling of the virtual queues. + * Added flow handling support * **Added cryptodev queue pair reset support.** diff --git a/drivers/net/ntnic/include/flow_api.h b/drivers/net/ntnic/include/flow_api.h index 748da89262..667dad6d5f 100644 --- a/drivers/net/ntnic/include/flow_api.h +++ b/drivers/net/ntnic/include/flow_api.h @@ -68,6 +68,9 @@ struct flow_nic_dev { uint32_t flow_unique_id_counter; /* linked list of all flows created on this NIC */ struct flow_handle *flow_base; + /* linked list of all FLM flows created on this NIC */ + struct flow_handle *flow_base_flm; + pthread_mutex_t flow_mtx; /* NIC backend API */ struct flow_api_backend_s be; diff --git a/drivers/net/ntnic/include/flow_api_engine.h b/drivers/net/ntnic/include/flow_api_engine.h index 2497c31a08..b8da5eafba 100644 --- a/drivers/net/ntnic/include/flow_api_engine.h +++ b/drivers/net/ntnic/include/flow_api_engine.h @@ -7,6 +7,10 @@ #define _FLOW_API_ENGINE_H_ #include +#include + +#include "hw_mod_backend.h" +#include "stream_binary_flow_api.h" /* * Resource management @@ -50,10 +54,107 @@ enum res_type_e { #define MAX_CPY_WRITERS_SUPPORTED 8 +enum flow_port_type_e { + PORT_NONE, /* not defined or drop */ + PORT_INTERNAL, /* no queues attached */ + PORT_PHY, /* MAC phy output queue */ + PORT_VIRT, /* Memory queues to Host */ +}; + +struct output_s { + uint32_t owning_port_id;/* the port who owns this output destination */ + enum flow_port_type_e type; + int id; /* depending on port type: queue ID or physical port id or not used */ + int active; /* activated */ +}; + +struct nic_flow_def { + /* + * Frame Decoder match info collected + */ + int l2_prot; + int l3_prot; + int l4_prot; + int tunnel_prot; + int tunnel_l3_prot; + int tunnel_l4_prot; + int vlans; + int fragmentation; + int ip_prot; + int tunnel_ip_prot; + /* + * Additional meta data for various functions + */ + int in_port_override; + int non_empty; /* default value is -1; value 1 means flow actions update */ + struct output_s dst_id[MAX_OUTPUT_DEST];/* define the output to use */ + /* total number of available queues defined for all outputs - i.e. number of dst_id's */ + int dst_num_avail; + + /* + * Mark or Action info collection + */ + uint32_t mark; + + uint32_t jump_to_group; + + int full_offload; +}; + +enum flow_handle_type { + FLOW_HANDLE_TYPE_FLOW, + FLOW_HANDLE_TYPE_FLM, +}; struct flow_handle { + enum flow_handle_type type; + uint32_t flm_id; + uint16_t caller_id; + uint16_t learn_ignored; + struct flow_eth_dev *dev; struct flow_handle *next; + struct flow_handle *prev; + + void *user_data; + + union { + struct { + /* + * 1st step conversion and validation of flow + * verified and converted flow match + actions structure + */ + struct nic_flow_def *fd; + /* + * 2nd step NIC HW resource allocation and configuration + * NIC resource management structures + */ + struct { + uint32_t db_idx_counter; + uint32_t db_idxs[RES_COUNT]; + }; + uint32_t port_id; /* MAC port ID or override of virtual in_port */ + }; + + struct { + uint32_t flm_db_idx_counter; + uint32_t flm_db_idxs[RES_COUNT]; + + uint32_t flm_data[10]; + uint8_t flm_prot; + uint8_t flm_kid; + uint8_t flm_prio; + uint8_t flm_ft; + + uint16_t flm_rpl_ext_ptr; + uint32_t flm_nat_ipv4; + uint16_t flm_nat_port; + uint8_t flm_dscp; + uint32_t flm_teid; + uint8_t flm_rqi; + uint8_t flm_qfi; + }; + }; }; void km_free_ndev_resource_management(void **handle); @@ -65,4 +166,8 @@ void kcc_free_ndev_resource_management(void **handle); */ int flow_group_handle_create(void **handle, uint32_t group_count); int flow_group_handle_destroy(void **handle); + +int flow_group_translate_get(void *handle, uint8_t owner_id, uint8_t port_id, uint32_t group_in, + uint32_t *group_out); + #endif /* _FLOW_API_ENGINE_H_ */ diff --git a/drivers/net/ntnic/include/stream_binary_flow_api.h b/drivers/net/ntnic/include/stream_binary_flow_api.h index a6244d4082..d878b848c2 100644 --- a/drivers/net/ntnic/include/stream_binary_flow_api.h +++ b/drivers/net/ntnic/include/stream_binary_flow_api.h @@ -8,6 +8,10 @@ #include "rte_flow.h" #include "rte_flow_driver.h" + +/* Max RSS hash key length in bytes */ +#define MAX_RSS_KEY_LEN 40 + /* * Flow frontend for binary programming interface */ diff --git a/drivers/net/ntnic/meson.build b/drivers/net/ntnic/meson.build index f7292144ac..e1fef37ccb 100644 --- a/drivers/net/ntnic/meson.build +++ b/drivers/net/ntnic/meson.build @@ -50,6 +50,8 @@ sources = files( 'nthw/flow_api/flow_api.c', 'nthw/flow_api/flow_group.c', 'nthw/flow_api/flow_id_table.c', + 'nthw/flow_api/hw_mod/hw_mod_backend.c', + 'nthw/flow_api/profile_inline/flm_lrn_queue.c', 'nthw/flow_api/profile_inline/flow_api_profile_inline.c', 'nthw/flow_api/profile_inline/flow_api_hw_db_inline.c', 'nthw/flow_api/flow_backend/flow_backend.c', diff --git a/drivers/net/ntnic/nthw/flow_api/flow_group.c b/drivers/net/ntnic/nthw/flow_api/flow_group.c index a7371f3aad..f76986b178 100644 --- a/drivers/net/ntnic/nthw/flow_api/flow_group.c +++ b/drivers/net/ntnic/nthw/flow_api/flow_group.c @@ -53,3 +53,47 @@ int flow_group_handle_destroy(void **handle) return 0; } + +int flow_group_translate_get(void *handle, uint8_t owner_id, uint8_t port_id, uint32_t group_in, + uint32_t *group_out) +{ + struct group_handle_s *group_handle = (struct group_handle_s *)handle; + uint32_t *table_ptr; + uint32_t lookup; + + if (group_handle == NULL || group_in >= group_handle->group_count || port_id >= PORT_COUNT) + return -1; + + /* Don't translate group 0 */ + if (group_in == 0) { + *group_out = 0; + return 0; + } + + table_ptr = &group_handle->translation_table[port_id * OWNER_ID_COUNT * PORT_COUNT + + owner_id * OWNER_ID_COUNT + group_in]; + lookup = *table_ptr; + + if (lookup == 0) { + for (lookup = 1; lookup < group_handle->group_count && + group_handle->lookup_entries[lookup].ref_counter > 0; + ++lookup) + ; + + if (lookup < group_handle->group_count) { + group_handle->lookup_entries[lookup].reverse_lookup = table_ptr; + group_handle->lookup_entries[lookup].ref_counter += 1; + + *table_ptr = lookup; + + } else { + return -1; + } + + } else { + group_handle->lookup_entries[lookup].ref_counter += 1; + } + + *group_out = lookup; + return 0; +} diff --git a/drivers/net/ntnic/nthw/flow_api/flow_id_table.c b/drivers/net/ntnic/nthw/flow_api/flow_id_table.c index 9b46848e59..5635ac4524 100644 --- a/drivers/net/ntnic/nthw/flow_api/flow_id_table.c +++ b/drivers/net/ntnic/nthw/flow_api/flow_id_table.c @@ -4,6 +4,7 @@ */ #include +#include #include #include @@ -11,6 +12,10 @@ #define NTNIC_ARRAY_BITS 14 #define NTNIC_ARRAY_SIZE (1 << NTNIC_ARRAY_BITS) +#define NTNIC_ARRAY_MASK (NTNIC_ARRAY_SIZE - 1) +#define NTNIC_MAX_ID (NTNIC_ARRAY_SIZE * NTNIC_ARRAY_SIZE) +#define NTNIC_MAX_ID_MASK (NTNIC_MAX_ID - 1) +#define NTNIC_MIN_FREE 1000 struct ntnic_id_table_element { union flm_handles handle; @@ -29,6 +34,36 @@ struct ntnic_id_table_data { uint32_t free_count; }; +static inline struct ntnic_id_table_element * +ntnic_id_table_array_find_element(struct ntnic_id_table_data *handle, uint32_t id) +{ + uint32_t idx_d1 = id & NTNIC_ARRAY_MASK; + uint32_t idx_d2 = (id >> NTNIC_ARRAY_BITS) & NTNIC_ARRAY_MASK; + + if (handle->arrays[idx_d2] == NULL) { + handle->arrays[idx_d2] = + calloc(NTNIC_ARRAY_SIZE, sizeof(struct ntnic_id_table_element)); + } + + return &handle->arrays[idx_d2][idx_d1]; +} + +static inline uint32_t ntnic_id_table_array_pop_free_id(struct ntnic_id_table_data *handle) +{ + uint32_t id = 0; + + if (handle->free_count > NTNIC_MIN_FREE) { + struct ntnic_id_table_element *element = + ntnic_id_table_array_find_element(handle, handle->free_tail); + id = handle->free_tail; + + handle->free_tail = element->handle.idx & NTNIC_MAX_ID_MASK; + handle->free_count -= 1; + } + + return id; +} + void *ntnic_id_table_create(void) { struct ntnic_id_table_data *handle = calloc(1, sizeof(struct ntnic_id_table_data)); @@ -50,3 +85,47 @@ void ntnic_id_table_destroy(void *id_table) free(id_table); } + +uint32_t ntnic_id_table_get_id(void *id_table, union flm_handles flm_h, uint8_t caller_id, + uint8_t type) +{ + struct ntnic_id_table_data *handle = id_table; + + pthread_mutex_lock(&handle->mtx); + + uint32_t new_id = ntnic_id_table_array_pop_free_id(handle); + + if (new_id == 0) + new_id = handle->next_id++; + + struct ntnic_id_table_element *element = ntnic_id_table_array_find_element(handle, new_id); + element->caller_id = caller_id; + element->type = type; + memcpy(&element->handle, &flm_h, sizeof(union flm_handles)); + + pthread_mutex_unlock(&handle->mtx); + + return new_id; +} + +void ntnic_id_table_free_id(void *id_table, uint32_t id) +{ + struct ntnic_id_table_data *handle = id_table; + + pthread_mutex_lock(&handle->mtx); + + struct ntnic_id_table_element *current_element = + ntnic_id_table_array_find_element(handle, id); + memset(current_element, 0, sizeof(struct ntnic_id_table_element)); + + struct ntnic_id_table_element *element = + ntnic_id_table_array_find_element(handle, handle->free_head); + element->handle.idx = id; + handle->free_head = id; + handle->free_count += 1; + + if (handle->free_tail == 0) + handle->free_tail = handle->free_head; + + pthread_mutex_unlock(&handle->mtx); +} diff --git a/drivers/net/ntnic/nthw/flow_api/flow_id_table.h b/drivers/net/ntnic/nthw/flow_api/flow_id_table.h index 13455f1165..e190fe4a11 100644 --- a/drivers/net/ntnic/nthw/flow_api/flow_id_table.h +++ b/drivers/net/ntnic/nthw/flow_api/flow_id_table.h @@ -16,4 +16,8 @@ union flm_handles { void *ntnic_id_table_create(void); void ntnic_id_table_destroy(void *id_table); +uint32_t ntnic_id_table_get_id(void *id_table, union flm_handles flm_h, uint8_t caller_id, + uint8_t type); +void ntnic_id_table_free_id(void *id_table, uint32_t id); + #endif /* FLOW_ID_TABLE_H_ */ diff --git a/drivers/net/ntnic/nthw/flow_api/profile_inline/flm_lrn_queue.c b/drivers/net/ntnic/nthw/flow_api/profile_inline/flm_lrn_queue.c new file mode 100644 index 0000000000..ad7efafe08 --- /dev/null +++ b/drivers/net/ntnic/nthw/flow_api/profile_inline/flm_lrn_queue.c @@ -0,0 +1,28 @@ +/* + * SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2024 Napatech A/S + */ + +#include +#include +#include + +#include + +#include "hw_mod_flm_v25.h" + +#include "flm_lrn_queue.h" + +#define ELEM_SIZE sizeof(struct flm_v25_lrn_data_s) + +uint32_t *flm_lrn_queue_get_write_buffer(void *q) +{ + struct rte_ring_zc_data zcd; + unsigned int n = rte_ring_enqueue_zc_burst_elem_start(q, ELEM_SIZE, 1, &zcd, NULL); + return (n == 0) ? NULL : zcd.ptr1; +} + +void flm_lrn_queue_release_write_buffer(void *q) +{ + rte_ring_enqueue_zc_elem_finish(q, 1); +} diff --git a/drivers/net/ntnic/nthw/flow_api/profile_inline/flm_lrn_queue.h b/drivers/net/ntnic/nthw/flow_api/profile_inline/flm_lrn_queue.h new file mode 100644 index 0000000000..8cee0c8e78 --- /dev/null +++ b/drivers/net/ntnic/nthw/flow_api/profile_inline/flm_lrn_queue.h @@ -0,0 +1,14 @@ +/* + * SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2024 Napatech A/S + */ + +#ifndef _FLM_LRN_QUEUE_H_ +#define _FLM_LRN_QUEUE_H_ + +#include + +uint32_t *flm_lrn_queue_get_write_buffer(void *q); +void flm_lrn_queue_release_write_buffer(void *q); + +#endif /* _FLM_LRN_QUEUE_H_ */ diff --git a/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_hw_db_inline.c b/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_hw_db_inline.c index 5fda11183c..4ea9387c80 100644 --- a/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_hw_db_inline.c +++ b/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_hw_db_inline.c @@ -3,7 +3,11 @@ */ +#include "hw_mod_backend.h" +#include "flow_api_engine.h" + #include "flow_api_hw_db_inline.h" +#include "rte_common.h" /******************************************************************************/ /* Handle */ @@ -57,3 +61,92 @@ void hw_db_inline_destroy(void *db_handle) free(db); } + +void hw_db_inline_deref_idxs(struct flow_nic_dev *ndev, void *db_handle, struct hw_db_idx *idxs, + uint32_t size) +{ + for (uint32_t i = 0; i < size; ++i) { + switch (idxs[i].type) { + case HW_DB_IDX_TYPE_NONE: + break; + + case HW_DB_IDX_TYPE_COT: + hw_db_inline_cot_deref(ndev, db_handle, *(struct hw_db_cot_idx *)&idxs[i]); + break; + + default: + break; + } + } +} + +/******************************************************************************/ +/* COT */ +/******************************************************************************/ + +static int hw_db_inline_cot_compare(const struct hw_db_inline_cot_data *data1, + const struct hw_db_inline_cot_data *data2) +{ + return data1->matcher_color_contrib == data2->matcher_color_contrib && + data1->frag_rcp == data2->frag_rcp; +} + +struct hw_db_cot_idx hw_db_inline_cot_add(struct flow_nic_dev *ndev, void *db_handle, + const struct hw_db_inline_cot_data *data) +{ + struct hw_db_inline_resource_db *db = (struct hw_db_inline_resource_db *)db_handle; + struct hw_db_cot_idx idx = { .raw = 0 }; + int found = 0; + + idx.type = HW_DB_IDX_TYPE_COT; + + for (uint32_t i = 1; i < db->nb_cot; ++i) { + int ref = db->cot[i].ref; + + if (ref > 0 && hw_db_inline_cot_compare(data, &db->cot[i].data)) { + idx.ids = i; + hw_db_inline_cot_ref(ndev, db, idx); + return idx; + } + + if (!found && ref <= 0) { + found = 1; + idx.ids = i; + } + } + + if (!found) { + idx.error = 1; + return idx; + } + + db->cot[idx.ids].ref = 1; + memcpy(&db->cot[idx.ids].data, data, sizeof(struct hw_db_inline_cot_data)); + + return idx; +} + +void hw_db_inline_cot_ref(struct flow_nic_dev *ndev __rte_unused, void *db_handle, + struct hw_db_cot_idx idx) +{ + struct hw_db_inline_resource_db *db = (struct hw_db_inline_resource_db *)db_handle; + + if (!idx.error) + db->cot[idx.ids].ref += 1; +} + +void hw_db_inline_cot_deref(struct flow_nic_dev *ndev __rte_unused, void *db_handle, + struct hw_db_cot_idx idx) +{ + struct hw_db_inline_resource_db *db = (struct hw_db_inline_resource_db *)db_handle; + + if (idx.error) + return; + + db->cot[idx.ids].ref -= 1; + + if (db->cot[idx.ids].ref <= 0) { + memset(&db->cot[idx.ids].data, 0x0, sizeof(struct hw_db_inline_cot_data)); + db->cot[idx.ids].ref = 0; + } +} diff --git a/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_hw_db_inline.h b/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_hw_db_inline.h index 23caf73cf3..0116af015d 100644 --- a/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_hw_db_inline.h +++ b/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_hw_db_inline.h @@ -9,15 +9,79 @@ #include "flow_api.h" +#define HW_DB_INLINE_MAX_QST_PER_QSL 128 +#define HW_DB_INLINE_MAX_ENCAP_SIZE 128 + +#define HW_DB_IDX \ + union { \ + struct { \ + uint32_t id1 : 8; \ + uint32_t id2 : 8; \ + uint32_t id3 : 8; \ + uint32_t type : 7; \ + uint32_t error : 1; \ + }; \ + struct { \ + uint32_t ids : 24; \ + }; \ + uint32_t raw; \ + } + +/* Strongly typed int types */ +struct hw_db_idx { + HW_DB_IDX; +}; + +struct hw_db_cot_idx { + HW_DB_IDX; +}; + +enum hw_db_idx_type { + HW_DB_IDX_TYPE_NONE = 0, + HW_DB_IDX_TYPE_COT, +}; + +/* Functionality data types */ +struct hw_db_inline_qsl_data { + uint32_t discard : 1; + uint32_t drop : 1; + uint32_t table_size : 7; + uint32_t retransmit : 1; + uint32_t padding : 22; + + struct { + uint16_t queue : 7; + uint16_t queue_en : 1; + uint16_t tx_port : 3; + uint16_t tx_port_en : 1; + uint16_t padding : 4; + } table[HW_DB_INLINE_MAX_QST_PER_QSL]; +}; + struct hw_db_inline_cot_data { uint32_t matcher_color_contrib : 4; uint32_t frag_rcp : 4; uint32_t padding : 24; }; +struct hw_db_inline_hsh_data { + uint32_t func; + uint64_t hash_mask; + uint8_t key[MAX_RSS_KEY_LEN]; +}; + /**/ int hw_db_inline_create(struct flow_nic_dev *ndev, void **db_handle); void hw_db_inline_destroy(void *db_handle); +void hw_db_inline_deref_idxs(struct flow_nic_dev *ndev, void *db_handle, struct hw_db_idx *idxs, + uint32_t size); + +/**/ +struct hw_db_cot_idx hw_db_inline_cot_add(struct flow_nic_dev *ndev, void *db_handle, + const struct hw_db_inline_cot_data *data); +void hw_db_inline_cot_ref(struct flow_nic_dev *ndev, void *db_handle, struct hw_db_cot_idx idx); +void hw_db_inline_cot_deref(struct flow_nic_dev *ndev, void *db_handle, struct hw_db_cot_idx idx); + #endif /* _FLOW_API_HW_DB_INLINE_H_ */ diff --git a/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_profile_inline.c b/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_profile_inline.c index 6d91678c56..d61912d49d 100644 --- a/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_profile_inline.c +++ b/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_profile_inline.c @@ -4,12 +4,545 @@ */ #include "ntlog.h" +#include "nt_util.h" + +#include "hw_mod_backend.h" +#include "flm_lrn_queue.h" +#include "flow_api.h" #include "flow_api_engine.h" #include "flow_api_hw_db_inline.h" #include "flow_id_table.h" +#include "stream_binary_flow_api.h" #include "flow_api_profile_inline.h" #include "ntnic_mod_reg.h" +#include + +#define NT_FLM_OP_UNLEARN 0 +#define NT_FLM_OP_LEARN 1 + +static void *flm_lrn_queue_arr; + +struct flm_flow_key_def_s { + union { + struct { + uint64_t qw0_dyn : 7; + uint64_t qw0_ofs : 8; + uint64_t qw4_dyn : 7; + uint64_t qw4_ofs : 8; + uint64_t sw8_dyn : 7; + uint64_t sw8_ofs : 8; + uint64_t sw9_dyn : 7; + uint64_t sw9_ofs : 8; + uint64_t outer_proto : 1; + uint64_t inner_proto : 1; + uint64_t pad : 2; + }; + uint64_t data; + }; + uint32_t mask[10]; +}; + +/* + * Flow Matcher functionality + */ +static uint8_t get_port_from_port_id(const struct flow_nic_dev *ndev, uint32_t port_id) +{ + struct flow_eth_dev *dev = ndev->eth_base; + + while (dev) { + if (dev->port_id == port_id) + return dev->port; + + dev = dev->next; + } + + return UINT8_MAX; +} + +static void nic_insert_flow(struct flow_nic_dev *ndev, struct flow_handle *fh) +{ + pthread_mutex_lock(&ndev->flow_mtx); + + if (ndev->flow_base) + ndev->flow_base->prev = fh; + + fh->next = ndev->flow_base; + fh->prev = NULL; + ndev->flow_base = fh; + + pthread_mutex_unlock(&ndev->flow_mtx); +} + +static void nic_remove_flow(struct flow_nic_dev *ndev, struct flow_handle *fh) +{ + struct flow_handle *next = fh->next; + struct flow_handle *prev = fh->prev; + + pthread_mutex_lock(&ndev->flow_mtx); + + if (next && prev) { + prev->next = next; + next->prev = prev; + + } else if (next) { + ndev->flow_base = next; + next->prev = NULL; + + } else if (prev) { + prev->next = NULL; + + } else if (ndev->flow_base == fh) { + ndev->flow_base = NULL; + } + + pthread_mutex_unlock(&ndev->flow_mtx); +} + +static void nic_insert_flow_flm(struct flow_nic_dev *ndev, struct flow_handle *fh) +{ + pthread_mutex_lock(&ndev->flow_mtx); + + if (ndev->flow_base_flm) + ndev->flow_base_flm->prev = fh; + + fh->next = ndev->flow_base_flm; + fh->prev = NULL; + ndev->flow_base_flm = fh; + + pthread_mutex_unlock(&ndev->flow_mtx); +} + +static void nic_remove_flow_flm(struct flow_nic_dev *ndev, struct flow_handle *fh_flm) +{ + struct flow_handle *next = fh_flm->next; + struct flow_handle *prev = fh_flm->prev; + + pthread_mutex_lock(&ndev->flow_mtx); + + if (next && prev) { + prev->next = next; + next->prev = prev; + + } else if (next) { + ndev->flow_base_flm = next; + next->prev = NULL; + + } else if (prev) { + prev->next = NULL; + + } else if (ndev->flow_base_flm == fh_flm) { + ndev->flow_base_flm = NULL; + } + + pthread_mutex_unlock(&ndev->flow_mtx); +} + +static inline struct nic_flow_def *prepare_nic_flow_def(struct nic_flow_def *fd) +{ + if (fd) { + fd->full_offload = -1; + fd->in_port_override = -1; + fd->mark = UINT32_MAX; + fd->jump_to_group = UINT32_MAX; + + fd->l2_prot = -1; + fd->l3_prot = -1; + fd->l4_prot = -1; + fd->vlans = 0; + fd->tunnel_prot = -1; + fd->tunnel_l3_prot = -1; + fd->tunnel_l4_prot = -1; + fd->fragmentation = -1; + fd->ip_prot = -1; + fd->tunnel_ip_prot = -1; + + fd->non_empty = -1; + } + + return fd; +} + +static inline struct nic_flow_def *allocate_nic_flow_def(void) +{ + return prepare_nic_flow_def(calloc(1, sizeof(struct nic_flow_def))); +} + +static bool fd_has_empty_pattern(const struct nic_flow_def *fd) +{ + return fd && fd->vlans == 0 && fd->l2_prot < 0 && fd->l3_prot < 0 && fd->l4_prot < 0 && + fd->tunnel_prot < 0 && fd->tunnel_l3_prot < 0 && fd->tunnel_l4_prot < 0 && + fd->ip_prot < 0 && fd->tunnel_ip_prot < 0 && fd->non_empty < 0; +} + +static inline const void *memcpy_mask_if(void *dest, const void *src, const void *mask, + size_t count) +{ + if (mask == NULL) + return src; + + unsigned char *dest_ptr = (unsigned char *)dest; + const unsigned char *src_ptr = (const unsigned char *)src; + const unsigned char *mask_ptr = (const unsigned char *)mask; + + for (size_t i = 0; i < count; ++i) + dest_ptr[i] = src_ptr[i] & mask_ptr[i]; + + return dest; +} + +static int flm_flow_programming(struct flow_handle *fh, uint32_t flm_op) +{ + struct flm_v25_lrn_data_s *learn_record = NULL; + + if (fh->type != FLOW_HANDLE_TYPE_FLM) + return -1; + + if (flm_op == NT_FLM_OP_LEARN) { + union flm_handles flm_h; + flm_h.p = fh; + fh->flm_id = ntnic_id_table_get_id(fh->dev->ndev->id_table_handle, flm_h, + fh->caller_id, 1); + } + + uint32_t flm_id = fh->flm_id; + + if (flm_op == NT_FLM_OP_UNLEARN) { + ntnic_id_table_free_id(fh->dev->ndev->id_table_handle, flm_id); + + if (fh->learn_ignored == 1) + return 0; + } + + learn_record = + (struct flm_v25_lrn_data_s *) + flm_lrn_queue_get_write_buffer(flm_lrn_queue_arr); + + while (learn_record == NULL) { + nt_os_wait_usec(1); + learn_record = + (struct flm_v25_lrn_data_s *) + flm_lrn_queue_get_write_buffer(flm_lrn_queue_arr); + } + + memset(learn_record, 0x0, sizeof(struct flm_v25_lrn_data_s)); + + learn_record->id = flm_id; + + learn_record->qw0[0] = fh->flm_data[9]; + learn_record->qw0[1] = fh->flm_data[8]; + learn_record->qw0[2] = fh->flm_data[7]; + learn_record->qw0[3] = fh->flm_data[6]; + learn_record->qw4[0] = fh->flm_data[5]; + learn_record->qw4[1] = fh->flm_data[4]; + learn_record->qw4[2] = fh->flm_data[3]; + learn_record->qw4[3] = fh->flm_data[2]; + learn_record->sw8 = fh->flm_data[1]; + learn_record->sw9 = fh->flm_data[0]; + learn_record->prot = fh->flm_prot; + + /* Last non-zero mtr is used for statistics */ + uint8_t mbrs = 0; + + learn_record->vol_idx = mbrs; + + learn_record->nat_ip = fh->flm_nat_ipv4; + learn_record->nat_port = fh->flm_nat_port; + learn_record->nat_en = fh->flm_nat_ipv4 || fh->flm_nat_port ? 1 : 0; + + learn_record->dscp = fh->flm_dscp; + learn_record->teid = fh->flm_teid; + learn_record->qfi = fh->flm_qfi; + learn_record->rqi = fh->flm_rqi; + /* Lower 10 bits used for RPL EXT PTR */ + learn_record->color = fh->flm_rpl_ext_ptr & 0x3ff; + + learn_record->ent = 0; + learn_record->op = flm_op & 0xf; + /* Suppress generation of statistics INF_DATA */ + learn_record->nofi = 1; + learn_record->prio = fh->flm_prio & 0x3; + learn_record->ft = fh->flm_ft; + learn_record->kid = fh->flm_kid; + learn_record->eor = 1; + learn_record->scrub_prof = 0; + + flm_lrn_queue_release_write_buffer(flm_lrn_queue_arr); + return 0; +} + +/* + * This function must be callable without locking any mutexes + */ +static int interpret_flow_actions(const struct flow_eth_dev *dev, + const struct rte_flow_action action[], + const struct rte_flow_action *action_mask, + struct nic_flow_def *fd, + struct rte_flow_error *error, + uint32_t *num_dest_port, + uint32_t *num_queues) +{ + unsigned int encap_decap_order = 0; + + *num_dest_port = 0; + *num_queues = 0; + + if (action == NULL) { + flow_nic_set_error(ERR_FAILED, error); + NT_LOG(ERR, FILTER, "Flow actions missing"); + return -1; + } + + /* + * Gather flow match + actions and convert into internal flow definition structure (struct + * nic_flow_def_s) This is the 1st step in the flow creation - validate, convert and + * prepare + */ + for (int aidx = 0; action[aidx].type != RTE_FLOW_ACTION_TYPE_END; ++aidx) { + switch (action[aidx].type) { + case RTE_FLOW_ACTION_TYPE_PORT_ID: + NT_LOG(DBG, FILTER, "Dev:%p: RTE_FLOW_ACTION_TYPE_PORT_ID", dev); + + if (action[aidx].conf) { + struct rte_flow_action_port_id port_id_tmp; + const struct rte_flow_action_port_id *port_id = + memcpy_mask_if(&port_id_tmp, action[aidx].conf, + action_mask ? action_mask[aidx].conf : NULL, + sizeof(struct rte_flow_action_port_id)); + + if (*num_dest_port > 0) { + NT_LOG(ERR, FILTER, + "Multiple port_id actions for one flow is not supported"); + flow_nic_set_error(ERR_ACTION_MULTIPLE_PORT_ID_UNSUPPORTED, + error); + return -1; + } + + uint8_t port = get_port_from_port_id(dev->ndev, port_id->id); + + if (fd->dst_num_avail == MAX_OUTPUT_DEST) { + NT_LOG(ERR, FILTER, "Too many output destinations"); + flow_nic_set_error(ERR_OUTPUT_TOO_MANY, error); + return -1; + } + + if (port >= dev->ndev->be.num_phy_ports) { + NT_LOG(ERR, FILTER, "Phy port out of range"); + flow_nic_set_error(ERR_OUTPUT_INVALID, error); + return -1; + } + + /* New destination port to add */ + fd->dst_id[fd->dst_num_avail].owning_port_id = port_id->id; + fd->dst_id[fd->dst_num_avail].type = PORT_PHY; + fd->dst_id[fd->dst_num_avail].id = (int)port; + fd->dst_id[fd->dst_num_avail].active = 1; + fd->dst_num_avail++; + + if (fd->full_offload < 0) + fd->full_offload = 1; + + *num_dest_port += 1; + + NT_LOG(DBG, FILTER, "Phy port ID: %i", (int)port); + } + + break; + + default: + NT_LOG(ERR, FILTER, "Invalid or unsupported flow action received - %i", + action[aidx].type); + flow_nic_set_error(ERR_ACTION_UNSUPPORTED, error); + return -1; + } + } + + if (!(encap_decap_order == 0 || encap_decap_order == 2)) { + NT_LOG(ERR, FILTER, "Invalid encap/decap actions"); + return -1; + } + + return 0; +} + +static int interpret_flow_elements(const struct flow_eth_dev *dev, + const struct rte_flow_item elem[], + struct nic_flow_def *fd __rte_unused, + struct rte_flow_error *error, + uint16_t implicit_vlan_vid __rte_unused, + uint32_t *in_port_id, + uint32_t *packet_data, + uint32_t *packet_mask, + struct flm_flow_key_def_s *key_def) +{ + *in_port_id = UINT32_MAX; + + memset(packet_data, 0x0, sizeof(uint32_t) * 10); + memset(packet_mask, 0x0, sizeof(uint32_t) * 10); + memset(key_def, 0x0, sizeof(struct flm_flow_key_def_s)); + + if (elem == NULL) { + flow_nic_set_error(ERR_FAILED, error); + NT_LOG(ERR, FILTER, "Flow items missing"); + return -1; + } + + int qw_reserved_mac = 0; + int qw_reserved_ipv6 = 0; + + int qw_free = 2 - qw_reserved_mac - qw_reserved_ipv6; + + if (qw_free < 0) { + NT_LOG(ERR, FILTER, "Key size too big. Out of QW resources."); + flow_nic_set_error(ERR_FAILED, error); + return -1; + } + + for (int eidx = 0; elem[eidx].type != RTE_FLOW_ITEM_TYPE_END; ++eidx) { + switch (elem[eidx].type) { + case RTE_FLOW_ITEM_TYPE_ANY: + NT_LOG(DBG, FILTER, "Adap %i, Port %i: RTE_FLOW_ITEM_TYPE_ANY", + dev->ndev->adapter_no, dev->port); + break; + + default: + NT_LOG(ERR, FILTER, "Invalid or unsupported flow request: %d", + (int)elem[eidx].type); + flow_nic_set_error(ERR_MATCH_INVALID_OR_UNSUPPORTED_ELEM, error); + return -1; + } + } + + return 0; +} + +static int convert_fh_to_fh_flm(struct flow_handle *fh, const uint32_t *packet_data __rte_unused, + uint32_t flm_key_id __rte_unused, uint32_t flm_ft __rte_unused, + uint16_t rpl_ext_ptr __rte_unused, uint32_t flm_scrub __rte_unused, + uint32_t priority __rte_unused) +{ + struct nic_flow_def *fd; + struct flow_handle fh_copy; + + if (fh->type != FLOW_HANDLE_TYPE_FLOW) + return -1; + + memcpy(&fh_copy, fh, sizeof(struct flow_handle)); + memset(fh, 0x0, sizeof(struct flow_handle)); + fd = fh_copy.fd; + + fh->type = FLOW_HANDLE_TYPE_FLM; + fh->caller_id = fh_copy.caller_id; + fh->dev = fh_copy.dev; + fh->next = fh_copy.next; + fh->prev = fh_copy.prev; + fh->user_data = fh_copy.user_data; + + fh->flm_db_idx_counter = fh_copy.db_idx_counter; + + for (int i = 0; i < RES_COUNT; ++i) + fh->flm_db_idxs[i] = fh_copy.db_idxs[i]; + + free(fd); + + return 0; +} + +static int setup_flow_flm_actions(struct flow_eth_dev *dev __rte_unused, + const struct nic_flow_def *fd __rte_unused, + const struct hw_db_inline_qsl_data *qsl_data __rte_unused, + const struct hw_db_inline_hsh_data *hsh_data __rte_unused, + uint32_t group __rte_unused, + uint32_t local_idxs[] __rte_unused, + uint32_t *local_idx_counter __rte_unused, + uint16_t *flm_rpl_ext_ptr __rte_unused, + uint32_t *flm_ft __rte_unused, + uint32_t *flm_scrub __rte_unused, + struct rte_flow_error *error __rte_unused) +{ + return 0; +} + +static struct flow_handle *create_flow_filter(struct flow_eth_dev *dev, struct nic_flow_def *fd, + const struct rte_flow_attr *attr, + uint16_t forced_vlan_vid __rte_unused, uint16_t caller_id, + struct rte_flow_error *error, uint32_t port_id, + uint32_t num_dest_port __rte_unused, uint32_t num_queues __rte_unused, + uint32_t *packet_data __rte_unused, uint32_t *packet_mask __rte_unused, + struct flm_flow_key_def_s *key_def __rte_unused) +{ + struct flow_handle *fh = calloc(1, sizeof(struct flow_handle)); + + fh->type = FLOW_HANDLE_TYPE_FLOW; + fh->port_id = port_id; + fh->dev = dev; + fh->fd = fd; + fh->caller_id = caller_id; + + struct hw_db_inline_qsl_data qsl_data; + + struct hw_db_inline_hsh_data hsh_data; + + if (attr->group > 0 && fd_has_empty_pattern(fd)) { + /* + * Default flow for group 1..32 + */ + + if (setup_flow_flm_actions(dev, fd, &qsl_data, &hsh_data, attr->group, fh->db_idxs, + &fh->db_idx_counter, NULL, NULL, NULL, error)) { + goto error_out; + } + + nic_insert_flow(dev->ndev, fh); + + } else if (attr->group > 0) { + /* + * Flow for group 1..32 + */ + + /* Setup Actions */ + uint16_t flm_rpl_ext_ptr = 0; + uint32_t flm_ft = 0; + uint32_t flm_scrub = 0; + + if (setup_flow_flm_actions(dev, fd, &qsl_data, &hsh_data, attr->group, fh->db_idxs, + &fh->db_idx_counter, &flm_rpl_ext_ptr, &flm_ft, + &flm_scrub, error)) { + goto error_out; + } + + /* Program flow */ + convert_fh_to_fh_flm(fh, packet_data, 2, flm_ft, flm_rpl_ext_ptr, + flm_scrub, attr->priority & 0x3); + flm_flow_programming(fh, NT_FLM_OP_LEARN); + + nic_insert_flow_flm(dev->ndev, fh); + + } else { + /* + * Flow for group 0 + */ + nic_insert_flow(dev->ndev, fh); + } + + return fh; + +error_out: + + if (fh->type == FLOW_HANDLE_TYPE_FLM) { + hw_db_inline_deref_idxs(dev->ndev, dev->ndev->hw_db_handle, + (struct hw_db_idx *)fh->flm_db_idxs, + fh->flm_db_idx_counter); + + } else { + hw_db_inline_deref_idxs(dev->ndev, dev->ndev->hw_db_handle, + (struct hw_db_idx *)fh->db_idxs, fh->db_idx_counter); + } + + free(fh); + + return NULL; +} /* * Public functions @@ -82,6 +615,92 @@ struct flow_handle *flow_create_profile_inline(struct flow_eth_dev *dev __rte_un const struct rte_flow_action action[] __rte_unused, struct rte_flow_error *error __rte_unused) { + struct flow_handle *fh = NULL; + int res; + + uint32_t port_id = UINT32_MAX; + uint32_t num_dest_port; + uint32_t num_queues; + + uint32_t packet_data[10]; + uint32_t packet_mask[10]; + struct flm_flow_key_def_s key_def; + + struct rte_flow_attr attr_local; + memcpy(&attr_local, attr, sizeof(struct rte_flow_attr)); + uint16_t forced_vlan_vid_local = forced_vlan_vid; + uint16_t caller_id_local = caller_id; + + if (attr_local.group > 0) + forced_vlan_vid_local = 0; + + flow_nic_set_error(ERR_SUCCESS, error); + + struct nic_flow_def *fd = allocate_nic_flow_def(); + + if (fd == NULL) + goto err_exit; + + res = interpret_flow_actions(dev, action, NULL, fd, error, &num_dest_port, &num_queues); + + if (res) + goto err_exit; + + res = interpret_flow_elements(dev, elem, fd, error, forced_vlan_vid_local, &port_id, + packet_data, packet_mask, &key_def); + + if (res) + goto err_exit; + + pthread_mutex_lock(&dev->ndev->mtx); + + /* Translate group IDs */ + if (fd->jump_to_group != UINT32_MAX && + flow_group_translate_get(dev->ndev->group_handle, caller_id_local, dev->port, + fd->jump_to_group, &fd->jump_to_group)) { + NT_LOG(ERR, FILTER, "ERROR: Could not get group resource"); + flow_nic_set_error(ERR_MATCH_RESOURCE_EXHAUSTION, error); + goto err_exit; + } + + if (attr_local.group > 0 && + flow_group_translate_get(dev->ndev->group_handle, caller_id_local, dev->port, + attr_local.group, &attr_local.group)) { + NT_LOG(ERR, FILTER, "ERROR: Could not get group resource"); + flow_nic_set_error(ERR_MATCH_RESOURCE_EXHAUSTION, error); + goto err_exit; + } + + if (port_id == UINT32_MAX) + port_id = dev->port_id; + + /* Create and flush filter to NIC */ + fh = create_flow_filter(dev, fd, &attr_local, forced_vlan_vid_local, + caller_id_local, error, port_id, num_dest_port, num_queues, packet_data, + packet_mask, &key_def); + + if (!fh) + goto err_exit; + + NT_LOG(DBG, FILTER, "New FlOW: fh (flow handle) %p, fd (flow definition) %p", fh, fd); + NT_LOG(DBG, FILTER, ">>>>> [Dev %p] Nic %i, Port %i: fh %p fd %p - implementation <<<<<", + dev, dev->ndev->adapter_no, dev->port, fh, fd); + + pthread_mutex_unlock(&dev->ndev->mtx); + + return fh; + +err_exit: + + if (fh) + flow_destroy_locked_profile_inline(dev, fh, NULL); + + else + free(fd); + + pthread_mutex_unlock(&dev->ndev->mtx); + + NT_LOG(ERR, FILTER, "ERR: %s", __func__); return NULL; } @@ -96,6 +715,44 @@ int flow_destroy_locked_profile_inline(struct flow_eth_dev *dev, flow_nic_set_error(ERR_SUCCESS, error); + /* take flow out of ndev list - may not have been put there yet */ + if (fh->type == FLOW_HANDLE_TYPE_FLM) + nic_remove_flow_flm(dev->ndev, fh); + + else + nic_remove_flow(dev->ndev, fh); + +#ifdef FLOW_DEBUG + dev->ndev->be.iface->set_debug_mode(dev->ndev->be.be_dev, FLOW_BACKEND_DEBUG_MODE_WRITE); +#endif + + NT_LOG(DBG, FILTER, "removing flow :%p", fh); + if (fh->type == FLOW_HANDLE_TYPE_FLM) { + hw_db_inline_deref_idxs(dev->ndev, dev->ndev->hw_db_handle, + (struct hw_db_idx *)fh->flm_db_idxs, + fh->flm_db_idx_counter); + + flm_flow_programming(fh, NT_FLM_OP_UNLEARN); + + } else { + NT_LOG(DBG, FILTER, "removing flow :%p", fh); + + hw_db_inline_deref_idxs(dev->ndev, dev->ndev->hw_db_handle, + (struct hw_db_idx *)fh->db_idxs, fh->db_idx_counter); + free(fh->fd); + } + + if (err) { + NT_LOG(ERR, FILTER, "FAILED removing flow: %p", fh); + flow_nic_set_error(ERR_REMOVE_FLOW_FAILED, error); + } + + free(fh); + +#ifdef FLOW_DEBUG + dev->ndev->be.iface->set_debug_mode(dev->ndev->be.be_dev, FLOW_BACKEND_DEBUG_MODE_NONE); +#endif + return err; } -- 2.45.0