From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E022645BA3; Tue, 22 Oct 2024 18:57:11 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 636C840BA4; Tue, 22 Oct 2024 18:56:23 +0200 (CEST) Received: from egress-ip11a.ess.de.barracuda.com (egress-ip11a.ess.de.barracuda.com [18.184.203.234]) by mails.dpdk.org (Postfix) with ESMTP id 149F940654 for ; Tue, 22 Oct 2024 18:56:15 +0200 (CEST) Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05lp2110.outbound.protection.outlook.com [104.47.17.110]) by mx-outbound40-26.eu-central-1c.ess.aws.cudaops.com (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 22 Oct 2024 16:56:13 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=iRG4uDV8hlbaC+hHQeZ3seHwuw+pLbo7T9kzXaJ5Ki1N8k+Gf3BdcjNDSgUpNIrZv+dM2Xm4CBJr/Vr2GW9SmVoR6sDN+wLWm7uMIhPWFtO5JXnuXIL19sD/FnKiiY+mdwCj8smYdRy6DXDrttAO8IcpJHpM4hBUMYIJAgsDjKYpyWh6qOJ0ZvwxKtxVq1/Y8vbizU0j1WnzJWnJnk921/mAET8IU+YbV/pajKrute3QiJCdvtaQOkcdhchy6K9zanBx05Jek3ebPVG8/EAStkPRHZ5ts25zLispsq/pNofV1652xgqFULd+g0sDICf5pjYticL6VZQOuDqtnVl+KQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=BR93xM6rdHsRmG+9g5CEPnLoW011HvJEAeH9OS7Dw9g=; b=gqOcPP3rafuMviEJEe49kP5zGFdrWsizcwsFthgViK3dejRCkzHyNIYXBVMn+ZxNYQlwaKPCBrFsasEO9z4UKu98diHOhCHdWbCab5AhXXUJF/BrFmijZuzA/ZQ9Re0PwTD7+ckIWRFgb2dbJOrJQamvhDGB5BlhihgBrdjF6/NankHGcWgNq69BbJqFr6Xbdfqt4RbINlWlREYM0DhaNOLo/FBoiE469ZofgMk45xcKflQI+FiT7/nq6Nqwy1wyE6Id/rJM7+oPM1oGF+r8jYdEhzQhhekKMYwU2eq/Hymwm/iGk6LNM2y5bCfO1bOHwPRwCS4widBP8IM20bHLZw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=fail (sender ip is 178.72.21.4) smtp.rcpttodomain=dpdk.org smtp.mailfrom=napatech.com; dmarc=fail (p=reject sp=reject pct=100) action=oreject header.from=napatech.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=napatech.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BR93xM6rdHsRmG+9g5CEPnLoW011HvJEAeH9OS7Dw9g=; b=WSFGUvGy42ubDLznvYTfPtmRpNxjATwGy6FW8XPkoVFYw7ezpmgYYLQq/mc1Osz3nKebMNfwkvDBDEtG94LcfKVagN+ezFCG3NtsVMxM46IbSPS2e21QwmvuEOmbNLi0hNQyWHCoKna+tTCv4pfwsTqSaPY/ITNK/pZQ5Fzndns= Received: from DB9PR06CA0023.eurprd06.prod.outlook.com (2603:10a6:10:1db::28) by AM9P190MB1075.EURP190.PROD.OUTLOOK.COM (2603:10a6:20b:265::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.16; Tue, 22 Oct 2024 16:56:06 +0000 Received: from DU2PEPF0001E9C2.eurprd03.prod.outlook.com (2603:10a6:10:1db:cafe::d8) by DB9PR06CA0023.outlook.office365.com (2603:10a6:10:1db::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.16 via Frontend Transport; Tue, 22 Oct 2024 16:56:06 +0000 X-MS-Exchange-Authentication-Results: spf=fail (sender IP is 178.72.21.4) smtp.mailfrom=napatech.com; dkim=none (message not signed) header.d=none;dmarc=fail action=oreject header.from=napatech.com; Received-SPF: Fail (protection.outlook.com: domain of napatech.com does not designate 178.72.21.4 as permitted sender) receiver=protection.outlook.com; client-ip=178.72.21.4; helo=localhost.localdomain; Received: from localhost.localdomain (178.72.21.4) by DU2PEPF0001E9C2.mail.protection.outlook.com (10.167.8.71) with Microsoft SMTP Server id 15.20.8093.14 via Frontend Transport; Tue, 22 Oct 2024 16:56:06 +0000 From: Serhii Iliushyk To: dev@dpdk.org Cc: mko-plv@napatech.com, sil-plv@napatech.com, ckm@napatech.com, andrew.rybchenko@oktetlabs.ru, ferruh.yigit@amd.com, stephen@networkplumber.org Subject: [PATCH v2 08/73] net/ntnic: add create/destroy implementation for NT flows Date: Tue, 22 Oct 2024 18:54:25 +0200 Message-ID: <20241022165541.3186140-9-sil-plv@napatech.com> X-Mailer: git-send-email 2.45.0 In-Reply-To: <20241022165541.3186140-1-sil-plv@napatech.com> References: <20241021210527.2075431-1-sil-plv@napatech.com> <20241022165541.3186140-1-sil-plv@napatech.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DU2PEPF0001E9C2:EE_|AM9P190MB1075:EE_ Content-Type: text/plain X-MS-Office365-Filtering-Correlation-Id: 15e9bbcc-ce38-471a-3b50-08dcf2ba6bae X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|376014|82310400026|1800799024|36860700013; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?ZlLVhTK47OwRIKlfILlNapVDtI8VVigpxJpv1+uKUj8E5Npd6ym6KpucOLPv?= =?us-ascii?Q?TxuRhN5Cip+1gq+oajRdP06PoYz2iW/z3N3EH1CYDA5Bi4gsjPRDpO0QLCZz?= =?us-ascii?Q?plxs4VdCjKm9xF1eJiDbJr4CYRCFQhT0hsXdxAOTJZFO0TO2J3B6YqvSxS+2?= =?us-ascii?Q?d9qPxmH7WbZSZkTaTvSibRiCSJhHsQ6hTXqP/f2QaAMdH2FVYQTz4bSO7yhx?= =?us-ascii?Q?1yrHFHazOe+taI7cT1sQC0CnI7plZvItzg45hpq0ICb2lKBjc3n2I5ZoMkGb?= =?us-ascii?Q?VdMnQsn+28lYM+lKAM6n/QwBqmynLXpgRrzzf03gIhtJ9E3O2hQ0Y7j1a3FU?= =?us-ascii?Q?OQXm0ryXclcH7yIxUj1WYqiruQnuiA0EaecqltG0IB/UsEKTSKIm9ihVRJ6m?= =?us-ascii?Q?nEscvbhEh481oD0FwBOZvdUlx7jY8ipzI6+eJAcOnJi80UuuPv1eNi37mdJM?= =?us-ascii?Q?2ijW1x5O+r4Uww9nTmUTp7p8X7lO1mblN4UF6zs34R24O3AY6Tzk0h2gyEhK?= =?us-ascii?Q?/f/VMweZlZf18CC8FdaKsdoHTcuYCAuGAYGUx6+uqSIY2Gr9aZGJADjxsEdk?= =?us-ascii?Q?u+DHxwgbDADZMdDZGkSh3XInjqPbFHd2VqmPQrKvTzzZQZFLYK62CniiqpHR?= =?us-ascii?Q?6o5QneuIPLJEdHAW/0H4KYevBrv6AqmIN+Ldxdbebre/WrQ6+cJfeRmG6n+X?= =?us-ascii?Q?0gdwFmNrVSnNzwvr+L7GYxoTFsuF5rTXSxzMUVC7o/vzM/Qjfs9JL38BBTYB?= =?us-ascii?Q?QS3lxTQvw+CX/gSE/PnR6ejVjwLGDPzxtYoPo+WyQK16J7AYRMyECYpAikI8?= =?us-ascii?Q?oTAWJEwDCwYDzOoa9Nh2F5cnbwUS4VBjBsRpicHYOsi2UIXm9gHIo6ZQ2x1O?= =?us-ascii?Q?awQ8QVGcdHWhMbqmc2iOsMGlDiLbAg1WeXFNa4ofIK1Zq8EXVBrLc5Cv2sOc?= =?us-ascii?Q?ajwP2VMqdPUVGJzNHExoPgg2qfyOJ2VlVxNripdbyK7AW1mmfLTG2SbIkCdJ?= =?us-ascii?Q?LCRoHuCBitsZfciTqfxM1eSmLPKqzmtfEvHdnxX4rb+y2iEqr+b3XbrkVfym?= =?us-ascii?Q?4lLC9m6xKZPgTdyQFxM8hJlp1jg4MxoSLpVzcFD+0jJ33+UMnx1tVHqCgeBY?= =?us-ascii?Q?7rs5zhxxJ/VBQ8nc6pRnfRj0XTP7esQ3m1FIeKtGYJoglyWGE/ENW1I9w3Cv?= =?us-ascii?Q?I96u3NxlmVgFKlt6y493TB2ZaliAy8oCprG1kb55xdfucU7LTyohNKhA5nz5?= =?us-ascii?Q?VE0Ir3jTFaMvlrEa4EQgP5JfMvpa6YbIeUV5YJjDD7YYeBF2U4y6t+pqGQRz?= =?us-ascii?Q?FOJ1UtBjo2xgXcLW9i44kZ1IS10JW2jkcZkABSCzWgSgFMNZpNt3bDzsnZ0h?= =?us-ascii?Q?aj+64q2WBh4bpdNNEKuwi3Qdt/LMY1sazj/eZp5DgxHwsYBQgQ=3D=3D?= X-Forefront-Antispam-Report: CIP:178.72.21.4; CTRY:DK; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:localhost.localdomain; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(376014)(82310400026)(1800799024)(36860700013); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: rHcbSL9ha8k7p8zZCIPydMqyANsajEyLXgiIdBoreN+1jXs8J8hHMdhE6Ke9KkkgfQ17dTYnRfxWmva++o/T68+x+0ZLUqG4zDxZnczQEcPj/Pl/EZbkWOqIwr2GQDheE3bo3OOoWdFFcaQfVKBaMs+jn4o2C+o+ZQxKs3wwTEz+XPjD4eTz7T+Yp56dzIW4pEpow58pkhJ2ddhW0VIUkNrT60XZCuASwDy3lDLAEY1PxqN/ZxGHAzSWlYJjLDoutDn5/nzhioUlAVv0lDGK2uMIOCyXouQjjOcEpAmwo6+9v1ACqP0a0oU/Hn4pNmjwNYZzSPVkVZmo4NaTtr0wBKpvDbYu6CZf9Ob0tMXo7a1eTcpluNDzFONKBHLkUzHNgmSi6R2BolnnTCqTlRNcrT3kOQXKEGXxCLu20ZIPsS2h71tteOtBunSRFchwrjE0tgiy98WpYy0w47mp1vIkth2m3AcF1ucTRp9jIKS8aM8Jy/xzlua5PMc2vOawBoI4LxSieITIbdVe/lwGlIi/5/QbxdDZawW6HIGYNethsxexM6WfuCjFD8F+AT8iUy5mqfwR6a+V17yted7xiUK7XV0QJpw7l/oueeDJ/NI7VAH1nO2sDcLjn/+kinoGNgKSV4L58JnFCok88mWSfq2Aq/+QkOMmYWDRcEs73lKRWyY= X-OriginatorOrg: napatech.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Oct 2024 16:56:06.1374 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 15e9bbcc-ce38-471a-3b50-08dcf2ba6bae X-MS-Exchange-CrossTenant-Id: c4540d0b-728a-4233-9da5-9ea30c7ec3ed X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=c4540d0b-728a-4233-9da5-9ea30c7ec3ed; Ip=[178.72.21.4]; Helo=[localhost.localdomain] X-MS-Exchange-CrossTenant-AuthSource: DU2PEPF0001E9C2.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9P190MB1075 X-BESS-ID: 1729616168-310266-12658-12976-2 X-BESS-VER: 2019.1_20241018.1852 X-BESS-Apparent-Source-IP: 104.47.17.110 X-BESS-Parts: H4sIAAAAAAACA4uuVkqtKFGyUioBkjpK+cVKVsbmxkZmQGYGUDTF3CzJ0NjA2M woOTE52SzN0MLAxDTFwNTIKNHcIs00Sak2FgAFFXowQgAAAA== X-BESS-Outbound-Spam-Score: 0.00 X-BESS-Outbound-Spam-Report: Code version 3.2, rules version 3.2.2.259902 [from cloudscan23-115.eu-central-1b.ess.aws.cudaops.com] Rule breakdown below pts rule name description ---- ---------------------- -------------------------------- 0.00 BSF_BESS_OUTBOUND META: BESS Outbound X-BESS-Outbound-Spam-Status: SCORE=0.00 using account:ESS113687 scores of KILL_LEVEL=7.0 tests=BSF_BESS_OUTBOUND X-BESS-BRTS-Status: 1 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Implements flow create/destroy functions with minimal capabilities item any action port id Signed-off-by: Serhii Iliushyk --- doc/guides/nics/features/ntnic.ini | 6 + drivers/net/ntnic/include/flow_api.h | 3 + drivers/net/ntnic/include/flow_api_engine.h | 105 +++ .../ntnic/include/stream_binary_flow_api.h | 4 + drivers/net/ntnic/meson.build | 2 + drivers/net/ntnic/nthw/flow_api/flow_group.c | 44 ++ .../net/ntnic/nthw/flow_api/flow_id_table.c | 79 +++ .../net/ntnic/nthw/flow_api/flow_id_table.h | 4 + .../flow_api/profile_inline/flm_lrn_queue.c | 28 + .../flow_api/profile_inline/flm_lrn_queue.h | 14 + .../profile_inline/flow_api_hw_db_inline.c | 93 +++ .../profile_inline/flow_api_hw_db_inline.h | 64 ++ .../profile_inline/flow_api_profile_inline.c | 657 ++++++++++++++++++ 13 files changed, 1103 insertions(+) create mode 100644 drivers/net/ntnic/nthw/flow_api/profile_inline/flm_lrn_queue.c create mode 100644 drivers/net/ntnic/nthw/flow_api/profile_inline/flm_lrn_queue.h diff --git a/doc/guides/nics/features/ntnic.ini b/doc/guides/nics/features/ntnic.ini index 8b9b87bdfe..1c653fd5a0 100644 --- a/doc/guides/nics/features/ntnic.ini +++ b/doc/guides/nics/features/ntnic.ini @@ -12,3 +12,9 @@ Unicast MAC filter = Y Multicast MAC filter = Y Linux = Y x86-64 = Y + +[rte_flow items] +any = Y + +[rte_flow actions] +port_id = Y diff --git a/drivers/net/ntnic/include/flow_api.h b/drivers/net/ntnic/include/flow_api.h index 748da89262..667dad6d5f 100644 --- a/drivers/net/ntnic/include/flow_api.h +++ b/drivers/net/ntnic/include/flow_api.h @@ -68,6 +68,9 @@ struct flow_nic_dev { uint32_t flow_unique_id_counter; /* linked list of all flows created on this NIC */ struct flow_handle *flow_base; + /* linked list of all FLM flows created on this NIC */ + struct flow_handle *flow_base_flm; + pthread_mutex_t flow_mtx; /* NIC backend API */ struct flow_api_backend_s be; diff --git a/drivers/net/ntnic/include/flow_api_engine.h b/drivers/net/ntnic/include/flow_api_engine.h index 2497c31a08..b8da5eafba 100644 --- a/drivers/net/ntnic/include/flow_api_engine.h +++ b/drivers/net/ntnic/include/flow_api_engine.h @@ -7,6 +7,10 @@ #define _FLOW_API_ENGINE_H_ #include +#include + +#include "hw_mod_backend.h" +#include "stream_binary_flow_api.h" /* * Resource management @@ -50,10 +54,107 @@ enum res_type_e { #define MAX_CPY_WRITERS_SUPPORTED 8 +enum flow_port_type_e { + PORT_NONE, /* not defined or drop */ + PORT_INTERNAL, /* no queues attached */ + PORT_PHY, /* MAC phy output queue */ + PORT_VIRT, /* Memory queues to Host */ +}; + +struct output_s { + uint32_t owning_port_id;/* the port who owns this output destination */ + enum flow_port_type_e type; + int id; /* depending on port type: queue ID or physical port id or not used */ + int active; /* activated */ +}; + +struct nic_flow_def { + /* + * Frame Decoder match info collected + */ + int l2_prot; + int l3_prot; + int l4_prot; + int tunnel_prot; + int tunnel_l3_prot; + int tunnel_l4_prot; + int vlans; + int fragmentation; + int ip_prot; + int tunnel_ip_prot; + /* + * Additional meta data for various functions + */ + int in_port_override; + int non_empty; /* default value is -1; value 1 means flow actions update */ + struct output_s dst_id[MAX_OUTPUT_DEST];/* define the output to use */ + /* total number of available queues defined for all outputs - i.e. number of dst_id's */ + int dst_num_avail; + + /* + * Mark or Action info collection + */ + uint32_t mark; + + uint32_t jump_to_group; + + int full_offload; +}; + +enum flow_handle_type { + FLOW_HANDLE_TYPE_FLOW, + FLOW_HANDLE_TYPE_FLM, +}; struct flow_handle { + enum flow_handle_type type; + uint32_t flm_id; + uint16_t caller_id; + uint16_t learn_ignored; + struct flow_eth_dev *dev; struct flow_handle *next; + struct flow_handle *prev; + + void *user_data; + + union { + struct { + /* + * 1st step conversion and validation of flow + * verified and converted flow match + actions structure + */ + struct nic_flow_def *fd; + /* + * 2nd step NIC HW resource allocation and configuration + * NIC resource management structures + */ + struct { + uint32_t db_idx_counter; + uint32_t db_idxs[RES_COUNT]; + }; + uint32_t port_id; /* MAC port ID or override of virtual in_port */ + }; + + struct { + uint32_t flm_db_idx_counter; + uint32_t flm_db_idxs[RES_COUNT]; + + uint32_t flm_data[10]; + uint8_t flm_prot; + uint8_t flm_kid; + uint8_t flm_prio; + uint8_t flm_ft; + + uint16_t flm_rpl_ext_ptr; + uint32_t flm_nat_ipv4; + uint16_t flm_nat_port; + uint8_t flm_dscp; + uint32_t flm_teid; + uint8_t flm_rqi; + uint8_t flm_qfi; + }; + }; }; void km_free_ndev_resource_management(void **handle); @@ -65,4 +166,8 @@ void kcc_free_ndev_resource_management(void **handle); */ int flow_group_handle_create(void **handle, uint32_t group_count); int flow_group_handle_destroy(void **handle); + +int flow_group_translate_get(void *handle, uint8_t owner_id, uint8_t port_id, uint32_t group_in, + uint32_t *group_out); + #endif /* _FLOW_API_ENGINE_H_ */ diff --git a/drivers/net/ntnic/include/stream_binary_flow_api.h b/drivers/net/ntnic/include/stream_binary_flow_api.h index a6244d4082..d878b848c2 100644 --- a/drivers/net/ntnic/include/stream_binary_flow_api.h +++ b/drivers/net/ntnic/include/stream_binary_flow_api.h @@ -8,6 +8,10 @@ #include "rte_flow.h" #include "rte_flow_driver.h" + +/* Max RSS hash key length in bytes */ +#define MAX_RSS_KEY_LEN 40 + /* * Flow frontend for binary programming interface */ diff --git a/drivers/net/ntnic/meson.build b/drivers/net/ntnic/meson.build index f7292144ac..e1fef37ccb 100644 --- a/drivers/net/ntnic/meson.build +++ b/drivers/net/ntnic/meson.build @@ -50,6 +50,8 @@ sources = files( 'nthw/flow_api/flow_api.c', 'nthw/flow_api/flow_group.c', 'nthw/flow_api/flow_id_table.c', + 'nthw/flow_api/hw_mod/hw_mod_backend.c', + 'nthw/flow_api/profile_inline/flm_lrn_queue.c', 'nthw/flow_api/profile_inline/flow_api_profile_inline.c', 'nthw/flow_api/profile_inline/flow_api_hw_db_inline.c', 'nthw/flow_api/flow_backend/flow_backend.c', diff --git a/drivers/net/ntnic/nthw/flow_api/flow_group.c b/drivers/net/ntnic/nthw/flow_api/flow_group.c index a7371f3aad..f76986b178 100644 --- a/drivers/net/ntnic/nthw/flow_api/flow_group.c +++ b/drivers/net/ntnic/nthw/flow_api/flow_group.c @@ -53,3 +53,47 @@ int flow_group_handle_destroy(void **handle) return 0; } + +int flow_group_translate_get(void *handle, uint8_t owner_id, uint8_t port_id, uint32_t group_in, + uint32_t *group_out) +{ + struct group_handle_s *group_handle = (struct group_handle_s *)handle; + uint32_t *table_ptr; + uint32_t lookup; + + if (group_handle == NULL || group_in >= group_handle->group_count || port_id >= PORT_COUNT) + return -1; + + /* Don't translate group 0 */ + if (group_in == 0) { + *group_out = 0; + return 0; + } + + table_ptr = &group_handle->translation_table[port_id * OWNER_ID_COUNT * PORT_COUNT + + owner_id * OWNER_ID_COUNT + group_in]; + lookup = *table_ptr; + + if (lookup == 0) { + for (lookup = 1; lookup < group_handle->group_count && + group_handle->lookup_entries[lookup].ref_counter > 0; + ++lookup) + ; + + if (lookup < group_handle->group_count) { + group_handle->lookup_entries[lookup].reverse_lookup = table_ptr; + group_handle->lookup_entries[lookup].ref_counter += 1; + + *table_ptr = lookup; + + } else { + return -1; + } + + } else { + group_handle->lookup_entries[lookup].ref_counter += 1; + } + + *group_out = lookup; + return 0; +} diff --git a/drivers/net/ntnic/nthw/flow_api/flow_id_table.c b/drivers/net/ntnic/nthw/flow_api/flow_id_table.c index 9b46848e59..5635ac4524 100644 --- a/drivers/net/ntnic/nthw/flow_api/flow_id_table.c +++ b/drivers/net/ntnic/nthw/flow_api/flow_id_table.c @@ -4,6 +4,7 @@ */ #include +#include #include #include @@ -11,6 +12,10 @@ #define NTNIC_ARRAY_BITS 14 #define NTNIC_ARRAY_SIZE (1 << NTNIC_ARRAY_BITS) +#define NTNIC_ARRAY_MASK (NTNIC_ARRAY_SIZE - 1) +#define NTNIC_MAX_ID (NTNIC_ARRAY_SIZE * NTNIC_ARRAY_SIZE) +#define NTNIC_MAX_ID_MASK (NTNIC_MAX_ID - 1) +#define NTNIC_MIN_FREE 1000 struct ntnic_id_table_element { union flm_handles handle; @@ -29,6 +34,36 @@ struct ntnic_id_table_data { uint32_t free_count; }; +static inline struct ntnic_id_table_element * +ntnic_id_table_array_find_element(struct ntnic_id_table_data *handle, uint32_t id) +{ + uint32_t idx_d1 = id & NTNIC_ARRAY_MASK; + uint32_t idx_d2 = (id >> NTNIC_ARRAY_BITS) & NTNIC_ARRAY_MASK; + + if (handle->arrays[idx_d2] == NULL) { + handle->arrays[idx_d2] = + calloc(NTNIC_ARRAY_SIZE, sizeof(struct ntnic_id_table_element)); + } + + return &handle->arrays[idx_d2][idx_d1]; +} + +static inline uint32_t ntnic_id_table_array_pop_free_id(struct ntnic_id_table_data *handle) +{ + uint32_t id = 0; + + if (handle->free_count > NTNIC_MIN_FREE) { + struct ntnic_id_table_element *element = + ntnic_id_table_array_find_element(handle, handle->free_tail); + id = handle->free_tail; + + handle->free_tail = element->handle.idx & NTNIC_MAX_ID_MASK; + handle->free_count -= 1; + } + + return id; +} + void *ntnic_id_table_create(void) { struct ntnic_id_table_data *handle = calloc(1, sizeof(struct ntnic_id_table_data)); @@ -50,3 +85,47 @@ void ntnic_id_table_destroy(void *id_table) free(id_table); } + +uint32_t ntnic_id_table_get_id(void *id_table, union flm_handles flm_h, uint8_t caller_id, + uint8_t type) +{ + struct ntnic_id_table_data *handle = id_table; + + pthread_mutex_lock(&handle->mtx); + + uint32_t new_id = ntnic_id_table_array_pop_free_id(handle); + + if (new_id == 0) + new_id = handle->next_id++; + + struct ntnic_id_table_element *element = ntnic_id_table_array_find_element(handle, new_id); + element->caller_id = caller_id; + element->type = type; + memcpy(&element->handle, &flm_h, sizeof(union flm_handles)); + + pthread_mutex_unlock(&handle->mtx); + + return new_id; +} + +void ntnic_id_table_free_id(void *id_table, uint32_t id) +{ + struct ntnic_id_table_data *handle = id_table; + + pthread_mutex_lock(&handle->mtx); + + struct ntnic_id_table_element *current_element = + ntnic_id_table_array_find_element(handle, id); + memset(current_element, 0, sizeof(struct ntnic_id_table_element)); + + struct ntnic_id_table_element *element = + ntnic_id_table_array_find_element(handle, handle->free_head); + element->handle.idx = id; + handle->free_head = id; + handle->free_count += 1; + + if (handle->free_tail == 0) + handle->free_tail = handle->free_head; + + pthread_mutex_unlock(&handle->mtx); +} diff --git a/drivers/net/ntnic/nthw/flow_api/flow_id_table.h b/drivers/net/ntnic/nthw/flow_api/flow_id_table.h index 13455f1165..e190fe4a11 100644 --- a/drivers/net/ntnic/nthw/flow_api/flow_id_table.h +++ b/drivers/net/ntnic/nthw/flow_api/flow_id_table.h @@ -16,4 +16,8 @@ union flm_handles { void *ntnic_id_table_create(void); void ntnic_id_table_destroy(void *id_table); +uint32_t ntnic_id_table_get_id(void *id_table, union flm_handles flm_h, uint8_t caller_id, + uint8_t type); +void ntnic_id_table_free_id(void *id_table, uint32_t id); + #endif /* FLOW_ID_TABLE_H_ */ diff --git a/drivers/net/ntnic/nthw/flow_api/profile_inline/flm_lrn_queue.c b/drivers/net/ntnic/nthw/flow_api/profile_inline/flm_lrn_queue.c new file mode 100644 index 0000000000..ad7efafe08 --- /dev/null +++ b/drivers/net/ntnic/nthw/flow_api/profile_inline/flm_lrn_queue.c @@ -0,0 +1,28 @@ +/* + * SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2024 Napatech A/S + */ + +#include +#include +#include + +#include + +#include "hw_mod_flm_v25.h" + +#include "flm_lrn_queue.h" + +#define ELEM_SIZE sizeof(struct flm_v25_lrn_data_s) + +uint32_t *flm_lrn_queue_get_write_buffer(void *q) +{ + struct rte_ring_zc_data zcd; + unsigned int n = rte_ring_enqueue_zc_burst_elem_start(q, ELEM_SIZE, 1, &zcd, NULL); + return (n == 0) ? NULL : zcd.ptr1; +} + +void flm_lrn_queue_release_write_buffer(void *q) +{ + rte_ring_enqueue_zc_elem_finish(q, 1); +} diff --git a/drivers/net/ntnic/nthw/flow_api/profile_inline/flm_lrn_queue.h b/drivers/net/ntnic/nthw/flow_api/profile_inline/flm_lrn_queue.h new file mode 100644 index 0000000000..8cee0c8e78 --- /dev/null +++ b/drivers/net/ntnic/nthw/flow_api/profile_inline/flm_lrn_queue.h @@ -0,0 +1,14 @@ +/* + * SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2024 Napatech A/S + */ + +#ifndef _FLM_LRN_QUEUE_H_ +#define _FLM_LRN_QUEUE_H_ + +#include + +uint32_t *flm_lrn_queue_get_write_buffer(void *q); +void flm_lrn_queue_release_write_buffer(void *q); + +#endif /* _FLM_LRN_QUEUE_H_ */ diff --git a/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_hw_db_inline.c b/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_hw_db_inline.c index 5fda11183c..4ea9387c80 100644 --- a/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_hw_db_inline.c +++ b/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_hw_db_inline.c @@ -3,7 +3,11 @@ */ +#include "hw_mod_backend.h" +#include "flow_api_engine.h" + #include "flow_api_hw_db_inline.h" +#include "rte_common.h" /******************************************************************************/ /* Handle */ @@ -57,3 +61,92 @@ void hw_db_inline_destroy(void *db_handle) free(db); } + +void hw_db_inline_deref_idxs(struct flow_nic_dev *ndev, void *db_handle, struct hw_db_idx *idxs, + uint32_t size) +{ + for (uint32_t i = 0; i < size; ++i) { + switch (idxs[i].type) { + case HW_DB_IDX_TYPE_NONE: + break; + + case HW_DB_IDX_TYPE_COT: + hw_db_inline_cot_deref(ndev, db_handle, *(struct hw_db_cot_idx *)&idxs[i]); + break; + + default: + break; + } + } +} + +/******************************************************************************/ +/* COT */ +/******************************************************************************/ + +static int hw_db_inline_cot_compare(const struct hw_db_inline_cot_data *data1, + const struct hw_db_inline_cot_data *data2) +{ + return data1->matcher_color_contrib == data2->matcher_color_contrib && + data1->frag_rcp == data2->frag_rcp; +} + +struct hw_db_cot_idx hw_db_inline_cot_add(struct flow_nic_dev *ndev, void *db_handle, + const struct hw_db_inline_cot_data *data) +{ + struct hw_db_inline_resource_db *db = (struct hw_db_inline_resource_db *)db_handle; + struct hw_db_cot_idx idx = { .raw = 0 }; + int found = 0; + + idx.type = HW_DB_IDX_TYPE_COT; + + for (uint32_t i = 1; i < db->nb_cot; ++i) { + int ref = db->cot[i].ref; + + if (ref > 0 && hw_db_inline_cot_compare(data, &db->cot[i].data)) { + idx.ids = i; + hw_db_inline_cot_ref(ndev, db, idx); + return idx; + } + + if (!found && ref <= 0) { + found = 1; + idx.ids = i; + } + } + + if (!found) { + idx.error = 1; + return idx; + } + + db->cot[idx.ids].ref = 1; + memcpy(&db->cot[idx.ids].data, data, sizeof(struct hw_db_inline_cot_data)); + + return idx; +} + +void hw_db_inline_cot_ref(struct flow_nic_dev *ndev __rte_unused, void *db_handle, + struct hw_db_cot_idx idx) +{ + struct hw_db_inline_resource_db *db = (struct hw_db_inline_resource_db *)db_handle; + + if (!idx.error) + db->cot[idx.ids].ref += 1; +} + +void hw_db_inline_cot_deref(struct flow_nic_dev *ndev __rte_unused, void *db_handle, + struct hw_db_cot_idx idx) +{ + struct hw_db_inline_resource_db *db = (struct hw_db_inline_resource_db *)db_handle; + + if (idx.error) + return; + + db->cot[idx.ids].ref -= 1; + + if (db->cot[idx.ids].ref <= 0) { + memset(&db->cot[idx.ids].data, 0x0, sizeof(struct hw_db_inline_cot_data)); + db->cot[idx.ids].ref = 0; + } +} diff --git a/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_hw_db_inline.h b/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_hw_db_inline.h index 23caf73cf3..0116af015d 100644 --- a/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_hw_db_inline.h +++ b/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_hw_db_inline.h @@ -9,15 +9,79 @@ #include "flow_api.h" +#define HW_DB_INLINE_MAX_QST_PER_QSL 128 +#define HW_DB_INLINE_MAX_ENCAP_SIZE 128 + +#define HW_DB_IDX \ + union { \ + struct { \ + uint32_t id1 : 8; \ + uint32_t id2 : 8; \ + uint32_t id3 : 8; \ + uint32_t type : 7; \ + uint32_t error : 1; \ + }; \ + struct { \ + uint32_t ids : 24; \ + }; \ + uint32_t raw; \ + } + +/* Strongly typed int types */ +struct hw_db_idx { + HW_DB_IDX; +}; + +struct hw_db_cot_idx { + HW_DB_IDX; +}; + +enum hw_db_idx_type { + HW_DB_IDX_TYPE_NONE = 0, + HW_DB_IDX_TYPE_COT, +}; + +/* Functionality data types */ +struct hw_db_inline_qsl_data { + uint32_t discard : 1; + uint32_t drop : 1; + uint32_t table_size : 7; + uint32_t retransmit : 1; + uint32_t padding : 22; + + struct { + uint16_t queue : 7; + uint16_t queue_en : 1; + uint16_t tx_port : 3; + uint16_t tx_port_en : 1; + uint16_t padding : 4; + } table[HW_DB_INLINE_MAX_QST_PER_QSL]; +}; + struct hw_db_inline_cot_data { uint32_t matcher_color_contrib : 4; uint32_t frag_rcp : 4; uint32_t padding : 24; }; +struct hw_db_inline_hsh_data { + uint32_t func; + uint64_t hash_mask; + uint8_t key[MAX_RSS_KEY_LEN]; +}; + /**/ int hw_db_inline_create(struct flow_nic_dev *ndev, void **db_handle); void hw_db_inline_destroy(void *db_handle); +void hw_db_inline_deref_idxs(struct flow_nic_dev *ndev, void *db_handle, struct hw_db_idx *idxs, + uint32_t size); + +/**/ +struct hw_db_cot_idx hw_db_inline_cot_add(struct flow_nic_dev *ndev, void *db_handle, + const struct hw_db_inline_cot_data *data); +void hw_db_inline_cot_ref(struct flow_nic_dev *ndev, void *db_handle, struct hw_db_cot_idx idx); +void hw_db_inline_cot_deref(struct flow_nic_dev *ndev, void *db_handle, struct hw_db_cot_idx idx); + #endif /* _FLOW_API_HW_DB_INLINE_H_ */ diff --git a/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_profile_inline.c b/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_profile_inline.c index 986196b408..7f9869a511 100644 --- a/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_profile_inline.c +++ b/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_profile_inline.c @@ -4,12 +4,545 @@ */ #include "ntlog.h" +#include "nt_util.h" + +#include "hw_mod_backend.h" +#include "flm_lrn_queue.h" +#include "flow_api.h" #include "flow_api_engine.h" #include "flow_api_hw_db_inline.h" #include "flow_id_table.h" +#include "stream_binary_flow_api.h" #include "flow_api_profile_inline.h" #include "ntnic_mod_reg.h" +#include + +#define NT_FLM_OP_UNLEARN 0 +#define NT_FLM_OP_LEARN 1 + +static void *flm_lrn_queue_arr; + +struct flm_flow_key_def_s { + union { + struct { + uint64_t qw0_dyn : 7; + uint64_t qw0_ofs : 8; + uint64_t qw4_dyn : 7; + uint64_t qw4_ofs : 8; + uint64_t sw8_dyn : 7; + uint64_t sw8_ofs : 8; + uint64_t sw9_dyn : 7; + uint64_t sw9_ofs : 8; + uint64_t outer_proto : 1; + uint64_t inner_proto : 1; + uint64_t pad : 2; + }; + uint64_t data; + }; + uint32_t mask[10]; +}; + +/* + * Flow Matcher functionality + */ +static uint8_t get_port_from_port_id(const struct flow_nic_dev *ndev, uint32_t port_id) +{ + struct flow_eth_dev *dev = ndev->eth_base; + + while (dev) { + if (dev->port_id == port_id) + return dev->port; + + dev = dev->next; + } + + return UINT8_MAX; +} + +static void nic_insert_flow(struct flow_nic_dev *ndev, struct flow_handle *fh) +{ + pthread_mutex_lock(&ndev->flow_mtx); + + if (ndev->flow_base) + ndev->flow_base->prev = fh; + + fh->next = ndev->flow_base; + fh->prev = NULL; + ndev->flow_base = fh; + + pthread_mutex_unlock(&ndev->flow_mtx); +} + +static void nic_remove_flow(struct flow_nic_dev *ndev, struct flow_handle *fh) +{ + struct flow_handle *next = fh->next; + struct flow_handle *prev = fh->prev; + + pthread_mutex_lock(&ndev->flow_mtx); + + if (next && prev) { + prev->next = next; + next->prev = prev; + + } else if (next) { + ndev->flow_base = next; + next->prev = NULL; + + } else if (prev) { + prev->next = NULL; + + } else if (ndev->flow_base == fh) { + ndev->flow_base = NULL; + } + + pthread_mutex_unlock(&ndev->flow_mtx); +} + +static void nic_insert_flow_flm(struct flow_nic_dev *ndev, struct flow_handle *fh) +{ + pthread_mutex_lock(&ndev->flow_mtx); + + if (ndev->flow_base_flm) + ndev->flow_base_flm->prev = fh; + + fh->next = ndev->flow_base_flm; + fh->prev = NULL; + ndev->flow_base_flm = fh; + + pthread_mutex_unlock(&ndev->flow_mtx); +} + +static void nic_remove_flow_flm(struct flow_nic_dev *ndev, struct flow_handle *fh_flm) +{ + struct flow_handle *next = fh_flm->next; + struct flow_handle *prev = fh_flm->prev; + + pthread_mutex_lock(&ndev->flow_mtx); + + if (next && prev) { + prev->next = next; + next->prev = prev; + + } else if (next) { + ndev->flow_base_flm = next; + next->prev = NULL; + + } else if (prev) { + prev->next = NULL; + + } else if (ndev->flow_base_flm == fh_flm) { + ndev->flow_base_flm = NULL; + } + + pthread_mutex_unlock(&ndev->flow_mtx); +} + +static inline struct nic_flow_def *prepare_nic_flow_def(struct nic_flow_def *fd) +{ + if (fd) { + fd->full_offload = -1; + fd->in_port_override = -1; + fd->mark = UINT32_MAX; + fd->jump_to_group = UINT32_MAX; + + fd->l2_prot = -1; + fd->l3_prot = -1; + fd->l4_prot = -1; + fd->vlans = 0; + fd->tunnel_prot = -1; + fd->tunnel_l3_prot = -1; + fd->tunnel_l4_prot = -1; + fd->fragmentation = -1; + fd->ip_prot = -1; + fd->tunnel_ip_prot = -1; + + fd->non_empty = -1; + } + + return fd; +} + +static inline struct nic_flow_def *allocate_nic_flow_def(void) +{ + return prepare_nic_flow_def(calloc(1, sizeof(struct nic_flow_def))); +} + +static bool fd_has_empty_pattern(const struct nic_flow_def *fd) +{ + return fd && fd->vlans == 0 && fd->l2_prot < 0 && fd->l3_prot < 0 && fd->l4_prot < 0 && + fd->tunnel_prot < 0 && fd->tunnel_l3_prot < 0 && fd->tunnel_l4_prot < 0 && + fd->ip_prot < 0 && fd->tunnel_ip_prot < 0 && fd->non_empty < 0; +} + +static inline const void *memcpy_mask_if(void *dest, const void *src, const void *mask, + size_t count) +{ + if (mask == NULL) + return src; + + unsigned char *dest_ptr = (unsigned char *)dest; + const unsigned char *src_ptr = (const unsigned char *)src; + const unsigned char *mask_ptr = (const unsigned char *)mask; + + for (size_t i = 0; i < count; ++i) + dest_ptr[i] = src_ptr[i] & mask_ptr[i]; + + return dest; +} + +static int flm_flow_programming(struct flow_handle *fh, uint32_t flm_op) +{ + struct flm_v25_lrn_data_s *learn_record = NULL; + + if (fh->type != FLOW_HANDLE_TYPE_FLM) + return -1; + + if (flm_op == NT_FLM_OP_LEARN) { + union flm_handles flm_h; + flm_h.p = fh; + fh->flm_id = ntnic_id_table_get_id(fh->dev->ndev->id_table_handle, flm_h, + fh->caller_id, 1); + } + + uint32_t flm_id = fh->flm_id; + + if (flm_op == NT_FLM_OP_UNLEARN) { + ntnic_id_table_free_id(fh->dev->ndev->id_table_handle, flm_id); + + if (fh->learn_ignored == 1) + return 0; + } + + learn_record = + (struct flm_v25_lrn_data_s *) + flm_lrn_queue_get_write_buffer(flm_lrn_queue_arr); + + while (learn_record == NULL) { + nt_os_wait_usec(1); + learn_record = + (struct flm_v25_lrn_data_s *) + flm_lrn_queue_get_write_buffer(flm_lrn_queue_arr); + } + + memset(learn_record, 0x0, sizeof(struct flm_v25_lrn_data_s)); + + learn_record->id = flm_id; + + learn_record->qw0[0] = fh->flm_data[9]; + learn_record->qw0[1] = fh->flm_data[8]; + learn_record->qw0[2] = fh->flm_data[7]; + learn_record->qw0[3] = fh->flm_data[6]; + learn_record->qw4[0] = fh->flm_data[5]; + learn_record->qw4[1] = fh->flm_data[4]; + learn_record->qw4[2] = fh->flm_data[3]; + learn_record->qw4[3] = fh->flm_data[2]; + learn_record->sw8 = fh->flm_data[1]; + learn_record->sw9 = fh->flm_data[0]; + learn_record->prot = fh->flm_prot; + + /* Last non-zero mtr is used for statistics */ + uint8_t mbrs = 0; + + learn_record->vol_idx = mbrs; + + learn_record->nat_ip = fh->flm_nat_ipv4; + learn_record->nat_port = fh->flm_nat_port; + learn_record->nat_en = fh->flm_nat_ipv4 || fh->flm_nat_port ? 1 : 0; + + learn_record->dscp = fh->flm_dscp; + learn_record->teid = fh->flm_teid; + learn_record->qfi = fh->flm_qfi; + learn_record->rqi = fh->flm_rqi; + /* Lower 10 bits used for RPL EXT PTR */ + learn_record->color = fh->flm_rpl_ext_ptr & 0x3ff; + + learn_record->ent = 0; + learn_record->op = flm_op & 0xf; + /* Suppress generation of statistics INF_DATA */ + learn_record->nofi = 1; + learn_record->prio = fh->flm_prio & 0x3; + learn_record->ft = fh->flm_ft; + learn_record->kid = fh->flm_kid; + learn_record->eor = 1; + learn_record->scrub_prof = 0; + + flm_lrn_queue_release_write_buffer(flm_lrn_queue_arr); + return 0; +} + +/* + * This function must be callable without locking any mutexes + */ +static int interpret_flow_actions(const struct flow_eth_dev *dev, + const struct rte_flow_action action[], + const struct rte_flow_action *action_mask, + struct nic_flow_def *fd, + struct rte_flow_error *error, + uint32_t *num_dest_port, + uint32_t *num_queues) +{ + unsigned int encap_decap_order = 0; + + *num_dest_port = 0; + *num_queues = 0; + + if (action == NULL) { + flow_nic_set_error(ERR_FAILED, error); + NT_LOG(ERR, FILTER, "Flow actions missing"); + return -1; + } + + /* + * Gather flow match + actions and convert into internal flow definition structure (struct + * nic_flow_def_s) This is the 1st step in the flow creation - validate, convert and + * prepare + */ + for (int aidx = 0; action[aidx].type != RTE_FLOW_ACTION_TYPE_END; ++aidx) { + switch (action[aidx].type) { + case RTE_FLOW_ACTION_TYPE_PORT_ID: + NT_LOG(DBG, FILTER, "Dev:%p: RTE_FLOW_ACTION_TYPE_PORT_ID", dev); + + if (action[aidx].conf) { + struct rte_flow_action_port_id port_id_tmp; + const struct rte_flow_action_port_id *port_id = + memcpy_mask_if(&port_id_tmp, action[aidx].conf, + action_mask ? action_mask[aidx].conf : NULL, + sizeof(struct rte_flow_action_port_id)); + + if (*num_dest_port > 0) { + NT_LOG(ERR, FILTER, + "Multiple port_id actions for one flow is not supported"); + flow_nic_set_error(ERR_ACTION_MULTIPLE_PORT_ID_UNSUPPORTED, + error); + return -1; + } + + uint8_t port = get_port_from_port_id(dev->ndev, port_id->id); + + if (fd->dst_num_avail == MAX_OUTPUT_DEST) { + NT_LOG(ERR, FILTER, "Too many output destinations"); + flow_nic_set_error(ERR_OUTPUT_TOO_MANY, error); + return -1; + } + + if (port >= dev->ndev->be.num_phy_ports) { + NT_LOG(ERR, FILTER, "Phy port out of range"); + flow_nic_set_error(ERR_OUTPUT_INVALID, error); + return -1; + } + + /* New destination port to add */ + fd->dst_id[fd->dst_num_avail].owning_port_id = port_id->id; + fd->dst_id[fd->dst_num_avail].type = PORT_PHY; + fd->dst_id[fd->dst_num_avail].id = (int)port; + fd->dst_id[fd->dst_num_avail].active = 1; + fd->dst_num_avail++; + + if (fd->full_offload < 0) + fd->full_offload = 1; + + *num_dest_port += 1; + + NT_LOG(DBG, FILTER, "Phy port ID: %i", (int)port); + } + + break; + + default: + NT_LOG(ERR, FILTER, "Invalid or unsupported flow action received - %i", + action[aidx].type); + flow_nic_set_error(ERR_ACTION_UNSUPPORTED, error); + return -1; + } + } + + if (!(encap_decap_order == 0 || encap_decap_order == 2)) { + NT_LOG(ERR, FILTER, "Invalid encap/decap actions"); + return -1; + } + + return 0; +} + +static int interpret_flow_elements(const struct flow_eth_dev *dev, + const struct rte_flow_item elem[], + struct nic_flow_def *fd __rte_unused, + struct rte_flow_error *error, + uint16_t implicit_vlan_vid __rte_unused, + uint32_t *in_port_id, + uint32_t *packet_data, + uint32_t *packet_mask, + struct flm_flow_key_def_s *key_def) +{ + *in_port_id = UINT32_MAX; + + memset(packet_data, 0x0, sizeof(uint32_t) * 10); + memset(packet_mask, 0x0, sizeof(uint32_t) * 10); + memset(key_def, 0x0, sizeof(struct flm_flow_key_def_s)); + + if (elem == NULL) { + flow_nic_set_error(ERR_FAILED, error); + NT_LOG(ERR, FILTER, "Flow items missing"); + return -1; + } + + int qw_reserved_mac = 0; + int qw_reserved_ipv6 = 0; + + int qw_free = 2 - qw_reserved_mac - qw_reserved_ipv6; + + if (qw_free < 0) { + NT_LOG(ERR, FILTER, "Key size too big. Out of QW resources."); + flow_nic_set_error(ERR_FAILED, error); + return -1; + } + + for (int eidx = 0; elem[eidx].type != RTE_FLOW_ITEM_TYPE_END; ++eidx) { + switch (elem[eidx].type) { + case RTE_FLOW_ITEM_TYPE_ANY: + NT_LOG(DBG, FILTER, "Adap %i, Port %i: RTE_FLOW_ITEM_TYPE_ANY", + dev->ndev->adapter_no, dev->port); + break; + + default: + NT_LOG(ERR, FILTER, "Invalid or unsupported flow request: %d", + (int)elem[eidx].type); + flow_nic_set_error(ERR_MATCH_INVALID_OR_UNSUPPORTED_ELEM, error); + return -1; + } + } + + return 0; +} + +static int convert_fh_to_fh_flm(struct flow_handle *fh, const uint32_t *packet_data __rte_unused, + uint32_t flm_key_id __rte_unused, uint32_t flm_ft __rte_unused, + uint16_t rpl_ext_ptr __rte_unused, uint32_t flm_scrub __rte_unused, + uint32_t priority __rte_unused) +{ + struct nic_flow_def *fd; + struct flow_handle fh_copy; + + if (fh->type != FLOW_HANDLE_TYPE_FLOW) + return -1; + + memcpy(&fh_copy, fh, sizeof(struct flow_handle)); + memset(fh, 0x0, sizeof(struct flow_handle)); + fd = fh_copy.fd; + + fh->type = FLOW_HANDLE_TYPE_FLM; + fh->caller_id = fh_copy.caller_id; + fh->dev = fh_copy.dev; + fh->next = fh_copy.next; + fh->prev = fh_copy.prev; + fh->user_data = fh_copy.user_data; + + fh->flm_db_idx_counter = fh_copy.db_idx_counter; + + for (int i = 0; i < RES_COUNT; ++i) + fh->flm_db_idxs[i] = fh_copy.db_idxs[i]; + + free(fd); + + return 0; +} + +static int setup_flow_flm_actions(struct flow_eth_dev *dev __rte_unused, + const struct nic_flow_def *fd __rte_unused, + const struct hw_db_inline_qsl_data *qsl_data __rte_unused, + const struct hw_db_inline_hsh_data *hsh_data __rte_unused, + uint32_t group __rte_unused, + uint32_t local_idxs[] __rte_unused, + uint32_t *local_idx_counter __rte_unused, + uint16_t *flm_rpl_ext_ptr __rte_unused, + uint32_t *flm_ft __rte_unused, + uint32_t *flm_scrub __rte_unused, + struct rte_flow_error *error __rte_unused) +{ + return 0; +} + +static struct flow_handle *create_flow_filter(struct flow_eth_dev *dev, struct nic_flow_def *fd, + const struct rte_flow_attr *attr, + uint16_t forced_vlan_vid __rte_unused, uint16_t caller_id, + struct rte_flow_error *error, uint32_t port_id, + uint32_t num_dest_port __rte_unused, uint32_t num_queues __rte_unused, + uint32_t *packet_data __rte_unused, uint32_t *packet_mask __rte_unused, + struct flm_flow_key_def_s *key_def __rte_unused) +{ + struct flow_handle *fh = calloc(1, sizeof(struct flow_handle)); + + fh->type = FLOW_HANDLE_TYPE_FLOW; + fh->port_id = port_id; + fh->dev = dev; + fh->fd = fd; + fh->caller_id = caller_id; + + struct hw_db_inline_qsl_data qsl_data; + + struct hw_db_inline_hsh_data hsh_data; + + if (attr->group > 0 && fd_has_empty_pattern(fd)) { + /* + * Default flow for group 1..32 + */ + + if (setup_flow_flm_actions(dev, fd, &qsl_data, &hsh_data, attr->group, fh->db_idxs, + &fh->db_idx_counter, NULL, NULL, NULL, error)) { + goto error_out; + } + + nic_insert_flow(dev->ndev, fh); + + } else if (attr->group > 0) { + /* + * Flow for group 1..32 + */ + + /* Setup Actions */ + uint16_t flm_rpl_ext_ptr = 0; + uint32_t flm_ft = 0; + uint32_t flm_scrub = 0; + + if (setup_flow_flm_actions(dev, fd, &qsl_data, &hsh_data, attr->group, fh->db_idxs, + &fh->db_idx_counter, &flm_rpl_ext_ptr, &flm_ft, + &flm_scrub, error)) { + goto error_out; + } + + /* Program flow */ + convert_fh_to_fh_flm(fh, packet_data, 2, flm_ft, flm_rpl_ext_ptr, + flm_scrub, attr->priority & 0x3); + flm_flow_programming(fh, NT_FLM_OP_LEARN); + + nic_insert_flow_flm(dev->ndev, fh); + + } else { + /* + * Flow for group 0 + */ + nic_insert_flow(dev->ndev, fh); + } + + return fh; + +error_out: + + if (fh->type == FLOW_HANDLE_TYPE_FLM) { + hw_db_inline_deref_idxs(dev->ndev, dev->ndev->hw_db_handle, + (struct hw_db_idx *)fh->flm_db_idxs, + fh->flm_db_idx_counter); + + } else { + hw_db_inline_deref_idxs(dev->ndev, dev->ndev->hw_db_handle, + (struct hw_db_idx *)fh->db_idxs, fh->db_idx_counter); + } + + free(fh); + + return NULL; +} /* * Public functions @@ -82,6 +615,92 @@ struct flow_handle *flow_create_profile_inline(struct flow_eth_dev *dev, const struct rte_flow_action action[], struct rte_flow_error *error) { + struct flow_handle *fh = NULL; + int res; + + uint32_t port_id = UINT32_MAX; + uint32_t num_dest_port; + uint32_t num_queues; + + uint32_t packet_data[10]; + uint32_t packet_mask[10]; + struct flm_flow_key_def_s key_def; + + struct rte_flow_attr attr_local; + memcpy(&attr_local, attr, sizeof(struct rte_flow_attr)); + uint16_t forced_vlan_vid_local = forced_vlan_vid; + uint16_t caller_id_local = caller_id; + + if (attr_local.group > 0) + forced_vlan_vid_local = 0; + + flow_nic_set_error(ERR_SUCCESS, error); + + struct nic_flow_def *fd = allocate_nic_flow_def(); + + if (fd == NULL) + goto err_exit; + + res = interpret_flow_actions(dev, action, NULL, fd, error, &num_dest_port, &num_queues); + + if (res) + goto err_exit; + + res = interpret_flow_elements(dev, elem, fd, error, forced_vlan_vid_local, &port_id, + packet_data, packet_mask, &key_def); + + if (res) + goto err_exit; + + pthread_mutex_lock(&dev->ndev->mtx); + + /* Translate group IDs */ + if (fd->jump_to_group != UINT32_MAX && + flow_group_translate_get(dev->ndev->group_handle, caller_id_local, dev->port, + fd->jump_to_group, &fd->jump_to_group)) { + NT_LOG(ERR, FILTER, "ERROR: Could not get group resource"); + flow_nic_set_error(ERR_MATCH_RESOURCE_EXHAUSTION, error); + goto err_exit; + } + + if (attr_local.group > 0 && + flow_group_translate_get(dev->ndev->group_handle, caller_id_local, dev->port, + attr_local.group, &attr_local.group)) { + NT_LOG(ERR, FILTER, "ERROR: Could not get group resource"); + flow_nic_set_error(ERR_MATCH_RESOURCE_EXHAUSTION, error); + goto err_exit; + } + + if (port_id == UINT32_MAX) + port_id = dev->port_id; + + /* Create and flush filter to NIC */ + fh = create_flow_filter(dev, fd, &attr_local, forced_vlan_vid_local, + caller_id_local, error, port_id, num_dest_port, num_queues, packet_data, + packet_mask, &key_def); + + if (!fh) + goto err_exit; + + NT_LOG(DBG, FILTER, "New FlOW: fh (flow handle) %p, fd (flow definition) %p", fh, fd); + NT_LOG(DBG, FILTER, ">>>>> [Dev %p] Nic %i, Port %i: fh %p fd %p - implementation <<<<<", + dev, dev->ndev->adapter_no, dev->port, fh, fd); + + pthread_mutex_unlock(&dev->ndev->mtx); + + return fh; + +err_exit: + + if (fh) + flow_destroy_locked_profile_inline(dev, fh, NULL); + + else + free(fd); + + pthread_mutex_unlock(&dev->ndev->mtx); + + NT_LOG(ERR, FILTER, "ERR: %s", __func__); return NULL; } @@ -96,6 +715,44 @@ int flow_destroy_locked_profile_inline(struct flow_eth_dev *dev, flow_nic_set_error(ERR_SUCCESS, error); + /* take flow out of ndev list - may not have been put there yet */ + if (fh->type == FLOW_HANDLE_TYPE_FLM) + nic_remove_flow_flm(dev->ndev, fh); + + else + nic_remove_flow(dev->ndev, fh); + +#ifdef FLOW_DEBUG + dev->ndev->be.iface->set_debug_mode(dev->ndev->be.be_dev, FLOW_BACKEND_DEBUG_MODE_WRITE); +#endif + + NT_LOG(DBG, FILTER, "removing flow :%p", fh); + if (fh->type == FLOW_HANDLE_TYPE_FLM) { + hw_db_inline_deref_idxs(dev->ndev, dev->ndev->hw_db_handle, + (struct hw_db_idx *)fh->flm_db_idxs, + fh->flm_db_idx_counter); + + flm_flow_programming(fh, NT_FLM_OP_UNLEARN); + + } else { + NT_LOG(DBG, FILTER, "removing flow :%p", fh); + + hw_db_inline_deref_idxs(dev->ndev, dev->ndev->hw_db_handle, + (struct hw_db_idx *)fh->db_idxs, fh->db_idx_counter); + free(fh->fd); + } + + if (err) { + NT_LOG(ERR, FILTER, "FAILED removing flow: %p", fh); + flow_nic_set_error(ERR_REMOVE_FLOW_FAILED, error); + } + + free(fh); + +#ifdef FLOW_DEBUG + dev->ndev->be.iface->set_debug_mode(dev->ndev->be.be_dev, FLOW_BACKEND_DEBUG_MODE_NONE); +#endif + return err; } -- 2.45.0