From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0294E45B96; Mon, 21 Oct 2024 23:08:54 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 169F240E64; Mon, 21 Oct 2024 23:06:21 +0200 (CEST) Received: from egress-ip42a.ess.de.barracuda.com (egress-ip42a.ess.de.barracuda.com [18.185.115.201]) by mails.dpdk.org (Postfix) with ESMTP id C8DAE40DCA for ; Mon, 21 Oct 2024 23:05:59 +0200 (CEST) Received: from EUR03-AM7-obe.outbound.protection.outlook.com (mail-am7eur03lp2237.outbound.protection.outlook.com [104.47.51.237]) by mx-outbound46-162.eu-central-1c.ess.aws.cudaops.com (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 21 Oct 2024 21:05:56 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=gCMnhNthe/VPsuWqQdv0qKaPuMBiEp+lPWAIh9CckHgICPq9+73/9XTVQYDr63vbP1DNVu7CJaBDlD0ZB7m2T6aMLeJNWnCvnyFVenzLY/AD3+Vunk/DmkslBr/FQr/0P9EUVoTSeA5vpTr++F49HlCbMCT7tgrxmzS67fLmvVdLwVuXojAp4ta2nS55ofJ71a4NmCQC+YiXkykFgSqYhEACaCa19HO+JsIPfVZfQRzJb4JS6mvVian9G95HfQnalrVFw1g1jdFvnCzonbI0LWTXnP9LkrQnHPpif+JijMpsBOpQogmt0k4jYPKHXogilAhOj+AcynY7bWldRoUj6Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=2KkWreTChxn8ZEWJi+lsZ1YLMjSELZqN9BQJw5aNIF4=; b=c0Sshitv3vnu3DCYldkSRwWtvp2CJ+DRoGwZtjgAAW6FtK88ftC5SROYQZXP+GR/iso3VC6CRCGHcYZ4w6ARxc0qbah3M5b2o3UcdZOXLsbw3j8MREZ6/vnCjxaHQSa5Xe4gUNc7nho09N/me94ik0HuO1nNiM9VJGiqLe0OnDwOpdmifCewUSa+Zw9859lVgqB+JQEH8WL5lIWl7PzqbFf200YTPP5gTB2/A0qBSBr6Dgk2aUa1NgpgDsGXtFrk0zVOvbjF2kX+JCF/LYUuzLoRfV9X88neUfDDc5mHXaGqVcylR720DVogWtjlQvZR/ihJzOvophXOhvXNAhKjpw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=fail (sender ip is 178.72.21.4) smtp.rcpttodomain=dpdk.org smtp.mailfrom=napatech.com; dmarc=fail (p=reject sp=reject pct=100) action=oreject header.from=napatech.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=napatech.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2KkWreTChxn8ZEWJi+lsZ1YLMjSELZqN9BQJw5aNIF4=; b=jw9LspxTMeRe3hDAwOUX+W17Vt3ImdhakgA3gBLkaKsx2wLU1ukgUrIiuV7PrM37OlImic/7h0yojjLwpObtB06/DMqF7tu7GjfOAOo9QSkvlH3OHZZ9/YXgljmbmd2qgq3FpR14cNz0yPZYVJEuoEjKZxvIA0q8XiDWqUh4mNM= Received: from AS9PR04CA0144.eurprd04.prod.outlook.com (2603:10a6:20b:48a::25) by AM9P190MB1347.EURP190.PROD.OUTLOOK.COM (2603:10a6:20b:262::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8069.18; Mon, 21 Oct 2024 21:05:54 +0000 Received: from AMS0EPF000001A8.eurprd05.prod.outlook.com (2603:10a6:20b:48a:cafe::f9) by AS9PR04CA0144.outlook.office365.com (2603:10a6:20b:48a::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8069.29 via Frontend Transport; Mon, 21 Oct 2024 21:05:54 +0000 X-MS-Exchange-Authentication-Results: spf=fail (sender IP is 178.72.21.4) smtp.mailfrom=napatech.com; dkim=none (message not signed) header.d=none;dmarc=fail action=oreject header.from=napatech.com; Received-SPF: Fail (protection.outlook.com: domain of napatech.com does not designate 178.72.21.4 as permitted sender) receiver=protection.outlook.com; client-ip=178.72.21.4; helo=localhost.localdomain; Received: from localhost.localdomain (178.72.21.4) by AMS0EPF000001A8.mail.protection.outlook.com (10.167.16.148) with Microsoft SMTP Server id 15.20.8093.14 via Frontend Transport; Mon, 21 Oct 2024 21:05:54 +0000 From: Serhii Iliushyk To: dev@dpdk.org Cc: mko-plv@napatech.com, sil-plv@napatech.com, ckm@napatech.com, andrew.rybchenko@oktetlabs.ru, ferruh.yigit@amd.com, Danylo Vodopianov Subject: [PATCH v1 30/73] net/ntnic: add KM module Date: Mon, 21 Oct 2024 23:04:32 +0200 Message-ID: <20241021210527.2075431-31-sil-plv@napatech.com> X-Mailer: git-send-email 2.45.0 In-Reply-To: <20241021210527.2075431-1-sil-plv@napatech.com> References: <20241021210527.2075431-1-sil-plv@napatech.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AMS0EPF000001A8:EE_|AM9P190MB1347:EE_ Content-Type: text/plain X-MS-Office365-Filtering-Correlation-Id: 1fb04ed2-1303-4c14-48fc-08dcf21426ad X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|376014|36860700013|1800799024|82310400026; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?bU5EEsaAgHmrD1WYSuDVEPaLcpulM+Ku/jTIvH5Dvd7vIzRwfOUYvfm6blII?= =?us-ascii?Q?0iAe/Ji+HTzhDSCt/Ah1HGHFtg9W7FWMMFzy4skricWR7XkpgQvjkHLJwA4d?= =?us-ascii?Q?KZcsLbyEBcEOrrBMo+gORQ/NI0gaQcSaX7+kwHopBnvMEKPu5MPgrxwCCGJs?= =?us-ascii?Q?e2GFIIuDr5XmyUGa2rUeWaaOAeRtAoicVzH7bSpyMjkmmrnLhPCL2RLPyYvp?= =?us-ascii?Q?w12FL8ee+jcJ9zC+oymCBWoGrsjIblWu5A34aTbhk9fmdZwsKx22Py5VZWWU?= =?us-ascii?Q?EZYIjRapIRvtaAt1/S2raOZAUvwhq2F1rgqZoz1rlqdGyaYZx3jD4s6iG44s?= =?us-ascii?Q?cJJ3buki233DH5UqBJQqhe0jgIgo/JC5rHLRs7AVzFH2jdsFE8WaX0jktVJ3?= =?us-ascii?Q?foouS0Y3KPClVPyC5KA3ve3SVDK2XOJMFJ4541aG1qfnOIwUdOOsnt/jBuJ1?= =?us-ascii?Q?YlyQ2AXJg3VpwoFVcz1dFjKWjdMGS+oQhMXuVdGD+tVQhhvgi/nblPAesbR0?= =?us-ascii?Q?Ld418U89rgKsYECLGk+4VT+BclWhahOr1gP6PODVPiL42Rk80uRe/g6I6ZPj?= =?us-ascii?Q?NsbArjr4a/EtDtV+l4thidTYMUCH0QleuwuskjuXIHXVRIjk6dss35ikhi0v?= =?us-ascii?Q?UQ16mOSEhNpGCGfFH55UUqnTv79Vcx4DQml80GRoj64YC8aNdFawRW62Cy1T?= =?us-ascii?Q?6+/edQu7VWaKRuHXqOGtUmNQzFvRcCfn5T057lHxGG/mETRpsvLf0p4Q5ABo?= =?us-ascii?Q?hCnD+v6AA9rX9Lx+58juPESBsgEorXWCZUE0RPRc6iQofj2PS/Wflyqmc1Xb?= =?us-ascii?Q?pgsYlhvZXchXBqxM8GkFzjcRZVdnyHv61o/4XVvHhz3rTsE5VhLHzDnpsi7d?= =?us-ascii?Q?ag4jgI1IYd/m5Hheu40hez4mQ522j/56L2Z2gXqONCYrp/uG1zmM7sqzH8jA?= =?us-ascii?Q?OsW8onm21KQu68qWfFp8WFWoiKa0hhLkWkEoPxSbDcYYKWtoq3RyN21c0Pqf?= =?us-ascii?Q?NN9KmZyliwp9/rME8EKjSGBsH1SwTdfeCXIzca8J9OJZIwWoRoMErnvplqKj?= =?us-ascii?Q?uSlkPxm79i/8Z4nY59HWJmk4t0A1q7Kx0TxaRZprtu5oo3MMiIqqdh2hcthI?= =?us-ascii?Q?GVxt2KHfiMUbtfQKogovS3IgC9sm8FQ3/EeeYAxyjpgsZ6Gf4dpDGnu15xeo?= =?us-ascii?Q?4wz/ANavgpN0oa6j+KSK05dEFNd83kO1uebfoHPSBD/GwQPB45zdkaZH1tW9?= =?us-ascii?Q?UDFBA06lZbdpkZgDbv2qSOBklN0pCES5a+NORTy2uZZ/9TTPmU4RzA3gfKqL?= =?us-ascii?Q?gfqJ+EsTZrQpOYL4xff3pR68IRia5ZOF3TS2bcBJomr/wCL+WFf627eUwJhc?= =?us-ascii?Q?5dA0QFn2OYuitv0c7ylfZZZUvtUwxN3b9f/AqdTPYf45zvQDCg=3D=3D?= X-Forefront-Antispam-Report: CIP:178.72.21.4; CTRY:DK; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:localhost.localdomain; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(376014)(36860700013)(1800799024)(82310400026); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: Nq5gEJvz3ZX1ux+0QjoSlYyQ5kNf3b9oRFfB58JQwUI3ExYYaG44syu/CvFBW+SRhXhjYbEPpPmHTEtEPaqXrZI0BNptPwN4/3Wwpg/dpdDylBzJJRFQc0AU75sf8Fu+z8ef7mhCV8YTC+YBbuliNVgEmERkxz6cx1myyFxN8iB+dU1hvDB2m9h/3x1HHfsbRyPtyfet/NLHbYlP7Rf101EOTvcA4/mFefHyQG9iI5Bhd6mHRFYnlEMQnp1Mu/D53CtPoFGzI/qcehHcPE/jEgJoWI7WwVTu4iPzb2RAi4Qb6C9SfXromo5WArRi5f/EB/vG881FXbALvYqT3RZGgHhQNVBnvnedMzi7v+WNH0IPPPXT9DjiirG7lctFsJ292rke4jyCNkQJBc/Ln9wSWUTrBELN0tCSF6sReFfy+wnwvIMyDw3uuKINsGrVbf0c4K/8lO1YczYMom6/yY0sCphjn0vPEn8wwT98i9E6lojAU5mGTl8aHQKhZ9dJnnUscQ9+QQ7mYzfTp61gBCDzwHsiu2AxYGlWkUqhJga1yASNMSDro//VKIV1Y0DWjt62p+xGbA6ZwITDexf1bBQwrlPiJeMSAalJzUeFAywgOZ6Yal110n5nLY2FiOhz6gBCIl81y2g1qtonVDANexrWhJnwUqO5NF4inbKnQhdZExY= X-OriginatorOrg: napatech.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Oct 2024 21:05:54.0363 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1fb04ed2-1303-4c14-48fc-08dcf21426ad X-MS-Exchange-CrossTenant-Id: c4540d0b-728a-4233-9da5-9ea30c7ec3ed X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=c4540d0b-728a-4233-9da5-9ea30c7ec3ed; Ip=[178.72.21.4]; Helo=[localhost.localdomain] X-MS-Exchange-CrossTenant-AuthSource: AMS0EPF000001A8.eurprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9P190MB1347 X-BESS-ID: 1729544756-311938-13028-41197-1 X-BESS-VER: 2019.1_20241018.1852 X-BESS-Apparent-Source-IP: 104.47.51.237 X-BESS-Parts: H4sIAAAAAAACA4uuVkqtKFGyUioBkjpK+cVKVmbGZiZGQGYGUNTCIM0o1TjVIt nAJNHSyCjR0MI0xdw0KTXN1MwkLdncXKk2FgBpLKvWQgAAAA== X-BESS-Outbound-Spam-Score: 0.50 X-BESS-Outbound-Spam-Report: Code version 3.2, rules version 3.2.2.259883 [from cloudscan16-126.eu-central-1b.ess.aws.cudaops.com] Rule breakdown below pts rule name description ---- ---------------------- -------------------------------- 0.00 BSF_BESS_OUTBOUND META: BESS Outbound 0.50 BSF_RULE_7582B META: Custom Rule 7582B X-BESS-Outbound-Spam-Status: SCORE=0.50 using account:ESS113687 scores of KILL_LEVEL=7.0 tests=BSF_BESS_OUTBOUND, BSF_RULE_7582B X-BESS-BRTS-Status: 1 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Danylo Vodopianov The Key Matcher module checks the values of individual fields of a packet. It supports both exact match which is implemented with a CAM, and wildcards which is implemented with a TCAM. Signed-off-by: Danylo Vodopianov --- drivers/net/ntnic/include/flow_api_engine.h | 110 +- drivers/net/ntnic/include/hw_mod_backend.h | 64 +- drivers/net/ntnic/nthw/flow_api/flow_km.c | 1065 +++++++++++++++++ .../ntnic/nthw/flow_api/hw_mod/hw_mod_km.c | 380 ++++++ .../profile_inline/flow_api_hw_db_inline.c | 234 ++++ .../profile_inline/flow_api_hw_db_inline.h | 38 + .../profile_inline/flow_api_profile_inline.c | 168 ++- 7 files changed, 2024 insertions(+), 35 deletions(-) diff --git a/drivers/net/ntnic/include/flow_api_engine.h b/drivers/net/ntnic/include/flow_api_engine.h index b1d39b919b..a0f02f4e8a 100644 --- a/drivers/net/ntnic/include/flow_api_engine.h +++ b/drivers/net/ntnic/include/flow_api_engine.h @@ -52,34 +52,32 @@ enum res_type_e { */ #define MAX_OUTPUT_DEST (128) +#define MAX_WORD_NUM 24 +#define MAX_BANKS 6 + +#define MAX_TCAM_START_OFFSETS 4 + #define MAX_CPY_WRITERS_SUPPORTED 8 #define MAX_MATCH_FIELDS 16 /* - * Tunnel encapsulation header definition + * 128 128 32 32 32 + * Have | QW0 || QW4 || SW8 || SW9 | SWX in FPGA + * + * Each word may start at any offset, though + * they are combined in chronological order, with all enabled to + * build the extracted match data, thus that is how the match key + * must be build */ -#define MAX_TUN_HDR_SIZE 128 -struct tunnel_header_s { - union { - uint8_t hdr8[MAX_TUN_HDR_SIZE]; - uint32_t hdr32[(MAX_TUN_HDR_SIZE + 3) / 4]; - } d; - uint32_t user_port_id; - uint8_t len; - - uint8_t nb_vlans; - - uint8_t ip_version; /* 4: v4, 6: v6 */ - uint16_t ip_csum_precalc; - - uint8_t new_outer; - uint8_t l2_len; - uint8_t l3_len; - uint8_t l4_len; +enum extractor_e { + KM_USE_EXTRACTOR_UNDEF, + KM_USE_EXTRACTOR_QWORD, + KM_USE_EXTRACTOR_SWORD, }; struct match_elem_s { + enum extractor_e extr; int masked_for_tcam; /* if potentially selected for TCAM */ uint32_t e_word[4]; uint32_t e_mask[4]; @@ -89,16 +87,76 @@ struct match_elem_s { uint32_t word_len; }; +enum cam_tech_use_e { + KM_CAM, + KM_TCAM, + KM_SYNERGY +}; + struct km_flow_def_s { struct flow_api_backend_s *be; + /* For keeping track of identical entries */ + struct km_flow_def_s *reference; + struct km_flow_def_s *root; + /* For collect flow elements and sorting */ struct match_elem_s match[MAX_MATCH_FIELDS]; + struct match_elem_s *match_map[MAX_MATCH_FIELDS]; int num_ftype_elem; + /* Finally formatted CAM/TCAM entry */ + enum cam_tech_use_e target; + uint32_t entry_word[MAX_WORD_NUM]; + uint32_t entry_mask[MAX_WORD_NUM]; + int key_word_size; + + /* TCAM calculated possible bank start offsets */ + int start_offsets[MAX_TCAM_START_OFFSETS]; + int num_start_offsets; + /* Flow information */ /* HW input port ID needed for compare. In port must be identical on flow types */ uint32_t port_id; + uint32_t info; /* used for color (actions) */ + int info_set; + int flow_type; /* 0 is illegal and used as unset */ + int flushed_to_target; /* if this km entry has been finally programmed into NIC hw */ + + /* CAM specific bank management */ + int cam_paired; + int record_indexes[MAX_BANKS]; + int bank_used; + uint32_t *cuckoo_moves; /* for CAM statistics only */ + struct cam_distrib_s *cam_dist; + + /* TCAM specific bank management */ + struct tcam_distrib_s *tcam_dist; + int tcam_start_bank; + int tcam_record; +}; + +/* + * Tunnel encapsulation header definition + */ +#define MAX_TUN_HDR_SIZE 128 + +struct tunnel_header_s { + union { + uint8_t hdr8[MAX_TUN_HDR_SIZE]; + uint32_t hdr32[(MAX_TUN_HDR_SIZE + 3) / 4]; + } d; + + uint8_t len; + + uint8_t nb_vlans; + + uint8_t ip_version; /* 4: v4, 6: v6 */ + + uint8_t new_outer; + uint8_t l2_len; + uint8_t l3_len; + uint8_t l4_len; }; enum flow_port_type_e { @@ -247,11 +305,25 @@ struct flow_handle { }; }; +void km_attach_ndev_resource_management(struct km_flow_def_s *km, void **handle); void km_free_ndev_resource_management(void **handle); int km_add_match_elem(struct km_flow_def_s *km, uint32_t e_word[4], uint32_t e_mask[4], uint32_t word_len, enum frame_offs_e start, int8_t offset); +int km_key_create(struct km_flow_def_s *km, uint32_t port_id); +/* + * Compares 2 KM key definitions after first collect validate and optimization. + * km is compared against an existing km1. + * if identical, km1 flow_type is returned + */ +int km_key_compare(struct km_flow_def_s *km, struct km_flow_def_s *km1); + +int km_rcp_set(struct km_flow_def_s *km, int index); + +int km_write_data_match_entry(struct km_flow_def_s *km, uint32_t color); +int km_clear_data_match_entry(struct km_flow_def_s *km); + void kcc_free_ndev_resource_management(void **handle); /* diff --git a/drivers/net/ntnic/include/hw_mod_backend.h b/drivers/net/ntnic/include/hw_mod_backend.h index 6fa2a3d94f..26903f2183 100644 --- a/drivers/net/ntnic/include/hw_mod_backend.h +++ b/drivers/net/ntnic/include/hw_mod_backend.h @@ -132,6 +132,22 @@ static inline int is_non_zero(const void *addr, size_t n) return 0; } +/* Sideband info bit indicator */ +#define SWX_INFO (1 << 6) + +enum km_flm_if_select_e { + KM_FLM_IF_FIRST = 0, + KM_FLM_IF_SECOND = 1 +}; + +#define FIELD_START_INDEX 100 + +#define COMMON_FUNC_INFO_S \ + int ver; \ + void *base; \ + unsigned int alloced_size; \ + int debug + enum frame_offs_e { DYN_L2 = 1, DYN_FIRST_VLAN = 2, @@ -141,22 +157,39 @@ enum frame_offs_e { DYN_TUN_L3 = 13, DYN_TUN_L4 = 16, DYN_TUN_L4_PAYLOAD = 17, + SB_VNI = SWX_INFO | 1, + SB_MAC_PORT = SWX_INFO | 2, + SB_KCC_ID = SWX_INFO | 3 }; -/* Sideband info bit indicator */ +enum { + QW0_SEL_EXCLUDE = 0, + QW0_SEL_FIRST32 = 1, + QW0_SEL_FIRST64 = 3, + QW0_SEL_ALL128 = 4, +}; -enum km_flm_if_select_e { - KM_FLM_IF_FIRST = 0, - KM_FLM_IF_SECOND = 1 +enum { + QW4_SEL_EXCLUDE = 0, + QW4_SEL_FIRST32 = 1, + QW4_SEL_FIRST64 = 2, + QW4_SEL_ALL128 = 3, }; -#define FIELD_START_INDEX 100 +enum { + DW8_SEL_EXCLUDE = 0, + DW8_SEL_FIRST32 = 3, +}; -#define COMMON_FUNC_INFO_S \ - int ver; \ - void *base; \ - unsigned int alloced_size; \ - int debug +enum { + DW10_SEL_EXCLUDE = 0, + DW10_SEL_FIRST32 = 2, +}; + +enum { + SWX_SEL_EXCLUDE = 0, + SWX_SEL_ALL32 = 1, +}; enum { PROT_OTHER = 0, @@ -440,13 +473,24 @@ int hw_mod_km_alloc(struct flow_api_backend_s *be); void hw_mod_km_free(struct flow_api_backend_s *be); int hw_mod_km_reset(struct flow_api_backend_s *be); int hw_mod_km_rcp_flush(struct flow_api_backend_s *be, int start_idx, int count); +int hw_mod_km_rcp_set(struct flow_api_backend_s *be, enum hw_km_e field, int index, int word_off, + uint32_t value); +int hw_mod_km_rcp_get(struct flow_api_backend_s *be, enum hw_km_e field, int index, int word_off, + uint32_t *value); int hw_mod_km_cam_flush(struct flow_api_backend_s *be, int start_bank, int start_record, int count); +int hw_mod_km_cam_set(struct flow_api_backend_s *be, enum hw_km_e field, int bank, int record, + uint32_t value); + int hw_mod_km_tcam_flush(struct flow_api_backend_s *be, int start_bank, int count); int hw_mod_km_tcam_set(struct flow_api_backend_s *be, enum hw_km_e field, int bank, int byte, int byte_val, uint32_t *value_set); +int hw_mod_km_tcam_get(struct flow_api_backend_s *be, enum hw_km_e field, int bank, int byte, + int byte_val, uint32_t *value_set); int hw_mod_km_tci_flush(struct flow_api_backend_s *be, int start_bank, int start_record, int count); +int hw_mod_km_tci_set(struct flow_api_backend_s *be, enum hw_km_e field, int bank, int record, + uint32_t value); int hw_mod_km_tcq_flush(struct flow_api_backend_s *be, int start_bank, int start_record, int count); diff --git a/drivers/net/ntnic/nthw/flow_api/flow_km.c b/drivers/net/ntnic/nthw/flow_api/flow_km.c index 237e9f7b4e..30d6ea728e 100644 --- a/drivers/net/ntnic/nthw/flow_api/flow_km.c +++ b/drivers/net/ntnic/nthw/flow_api/flow_km.c @@ -10,8 +10,34 @@ #include "flow_api_engine.h" #include "nt_util.h" +#define MAX_QWORDS 2 +#define MAX_SWORDS 2 + +#define CUCKOO_MOVE_MAX_DEPTH 8 + #define NUM_CAM_MASKS (ARRAY_SIZE(cam_masks)) +#define CAM_DIST_IDX(bnk, rec) ((bnk) * km->be->km.nb_cam_records + (rec)) +#define CAM_KM_DIST_IDX(bnk) \ + ({ \ + int _temp_bnk = (bnk); \ + CAM_DIST_IDX(_temp_bnk, km->record_indexes[_temp_bnk]); \ + }) + +#define TCAM_DIST_IDX(bnk, rec) ((bnk) * km->be->km.nb_tcam_bank_width + (rec)) + +#define CAM_ENTRIES \ + (km->be->km.nb_cam_banks * km->be->km.nb_cam_records * sizeof(struct cam_distrib_s)) +#define TCAM_ENTRIES \ + (km->be->km.nb_tcam_bank_width * km->be->km.nb_tcam_banks * sizeof(struct tcam_distrib_s)) + +/* + * CAM structures and defines + */ +struct cam_distrib_s { + struct km_flow_def_s *km_owner; +}; + static const struct cam_match_masks_s { uint32_t word_len; uint32_t key_mask[4]; @@ -36,6 +62,25 @@ static const struct cam_match_masks_s { { 1, { 0x00300000, 0x00000000, 0x00000000, 0x00000000 } }, }; +static int cam_addr_reserved_stack[CUCKOO_MOVE_MAX_DEPTH]; + +/* + * TCAM structures and defines + */ +struct tcam_distrib_s { + struct km_flow_def_s *km_owner; +}; + +static int tcam_find_mapping(struct km_flow_def_s *km); + +void km_attach_ndev_resource_management(struct km_flow_def_s *km, void **handle) +{ + km->cam_dist = (struct cam_distrib_s *)*handle; + km->cuckoo_moves = (uint32_t *)((char *)km->cam_dist + CAM_ENTRIES); + km->tcam_dist = + (struct tcam_distrib_s *)((char *)km->cam_dist + CAM_ENTRIES + sizeof(uint32_t)); +} + void km_free_ndev_resource_management(void **handle) { if (*handle) { @@ -98,3 +143,1023 @@ int km_add_match_elem(struct km_flow_def_s *km, uint32_t e_word[4], uint32_t e_m km->num_ftype_elem++; return 0; } + +static int get_word(struct km_flow_def_s *km, uint32_t size, int marked[]) +{ + for (int i = 0; i < km->num_ftype_elem; i++) + if (!marked[i] && !(km->match[i].extr_start_offs_id & SWX_INFO) && + km->match[i].word_len == size) + return i; + + return -1; +} + +int km_key_create(struct km_flow_def_s *km, uint32_t port_id) +{ + /* + * Create combined extractor mappings + * if key fields may be changed to cover un-mappable otherwise? + * split into cam and tcam and use synergy mode when available + */ + int match_marked[MAX_MATCH_FIELDS]; + int idx = 0; + int next = 0; + int m_idx; + int size; + + memset(match_marked, 0, sizeof(match_marked)); + + /* build QWords */ + for (int qwords = 0; qwords < MAX_QWORDS; qwords++) { + size = 4; + m_idx = get_word(km, size, match_marked); + + if (m_idx < 0) { + size = 2; + m_idx = get_word(km, size, match_marked); + + if (m_idx < 0) { + size = 1; + m_idx = get_word(km, 1, match_marked); + } + } + + if (m_idx < 0) { + /* no more defined */ + break; + } + + match_marked[m_idx] = 1; + + /* build match map list and set final extractor to use */ + km->match_map[next] = &km->match[m_idx]; + km->match[m_idx].extr = KM_USE_EXTRACTOR_QWORD; + + /* build final entry words and mask array */ + for (int i = 0; i < size; i++) { + km->entry_word[idx + i] = km->match[m_idx].e_word[i]; + km->entry_mask[idx + i] = km->match[m_idx].e_mask[i]; + } + + idx += size; + next++; + } + + m_idx = get_word(km, 4, match_marked); + + if (m_idx >= 0) { + /* cannot match more QWords */ + return -1; + } + + /* + * On km v6+ we have DWORDs here instead. However, we only use them as SWORDs for now + * No match would be able to exploit these as DWORDs because of maximum length of 12 words + * in CAM The last 2 words are taken by KCC-ID/SWX and Color. You could have one or none + * QWORDs where then both these DWORDs were possible in 10 words, but we don't have such + * use case built in yet + */ + /* build SWords */ + for (int swords = 0; swords < MAX_SWORDS; swords++) { + m_idx = get_word(km, 1, match_marked); + + if (m_idx < 0) { + /* no more defined */ + break; + } + + match_marked[m_idx] = 1; + /* build match map list and set final extractor to use */ + km->match_map[next] = &km->match[m_idx]; + km->match[m_idx].extr = KM_USE_EXTRACTOR_SWORD; + + /* build final entry words and mask array */ + km->entry_word[idx] = km->match[m_idx].e_word[0]; + km->entry_mask[idx] = km->match[m_idx].e_mask[0]; + idx++; + next++; + } + + /* + * Make sure we took them all + */ + m_idx = get_word(km, 1, match_marked); + + if (m_idx >= 0) { + /* cannot match more SWords */ + return -1; + } + + /* + * Handle SWX words specially + */ + int swx_found = 0; + + for (int i = 0; i < km->num_ftype_elem; i++) { + if (km->match[i].extr_start_offs_id & SWX_INFO) { + km->match_map[next] = &km->match[i]; + km->match[i].extr = KM_USE_EXTRACTOR_SWORD; + /* build final entry words and mask array */ + km->entry_word[idx] = km->match[i].e_word[0]; + km->entry_mask[idx] = km->match[i].e_mask[0]; + idx++; + next++; + swx_found = 1; + } + } + + assert(next == km->num_ftype_elem); + + km->key_word_size = idx; + km->port_id = port_id; + + km->target = KM_CAM; + + /* + * Finally decide if we want to put this match->action into the TCAM + * When SWX word used we need to put it into CAM always, no matter what mask pattern + * Later, when synergy mode is applied, we can do a split + */ + if (!swx_found && km->key_word_size <= 6) { + for (int i = 0; i < km->num_ftype_elem; i++) { + if (km->match_map[i]->masked_for_tcam) { + /* At least one */ + km->target = KM_TCAM; + } + } + } + + NT_LOG(DBG, FILTER, "This flow goes into %s", (km->target == KM_TCAM) ? "TCAM" : "CAM"); + + if (km->target == KM_TCAM) { + if (km->key_word_size > 10) { + /* do not support SWX in TCAM */ + return -1; + } + + /* + * adjust for unsupported key word size in TCAM + */ + if ((km->key_word_size == 5 || km->key_word_size == 7 || km->key_word_size == 9)) { + km->entry_mask[km->key_word_size] = 0; + km->key_word_size++; + } + + /* + * 1. the fact that the length of a key cannot change among the same used banks + * + * calculate possible start indexes + * unfortunately restrictions in TCAM lookup + * makes it hard to handle key lengths larger than 6 + * when other sizes should be possible too + */ + switch (km->key_word_size) { + case 1: + for (int i = 0; i < 4; i++) + km->start_offsets[0] = 8 + i; + + km->num_start_offsets = 4; + break; + + case 2: + km->start_offsets[0] = 6; + km->num_start_offsets = 1; + break; + + case 3: + km->start_offsets[0] = 0; + km->num_start_offsets = 1; + /* enlarge to 6 */ + km->entry_mask[km->key_word_size++] = 0; + km->entry_mask[km->key_word_size++] = 0; + km->entry_mask[km->key_word_size++] = 0; + break; + + case 4: + km->start_offsets[0] = 0; + km->num_start_offsets = 1; + /* enlarge to 6 */ + km->entry_mask[km->key_word_size++] = 0; + km->entry_mask[km->key_word_size++] = 0; + break; + + case 6: + km->start_offsets[0] = 0; + km->num_start_offsets = 1; + break; + + default: + NT_LOG(DBG, FILTER, "Final Key word size too large: %i", + km->key_word_size); + return -1; + } + } + + return 0; +} + +int km_key_compare(struct km_flow_def_s *km, struct km_flow_def_s *km1) +{ + if (km->target != km1->target || km->num_ftype_elem != km1->num_ftype_elem || + km->key_word_size != km1->key_word_size || km->info_set != km1->info_set) + return 0; + + /* + * before KCC-CAM: + * if port is added to match, then we can have different ports in CAT + * that reuses this flow type + */ + int port_match_included = 0, kcc_swx_used = 0; + + for (int i = 0; i < km->num_ftype_elem; i++) { + if (km->match[i].extr_start_offs_id == SB_MAC_PORT) { + port_match_included = 1; + break; + } + + if (km->match_map[i]->extr_start_offs_id == SB_KCC_ID) { + kcc_swx_used = 1; + break; + } + } + + /* + * If not using KCC and if port match is not included in CAM, + * we need to have same port_id to reuse + */ + if (!kcc_swx_used && !port_match_included && km->port_id != km1->port_id) + return 0; + + for (int i = 0; i < km->num_ftype_elem; i++) { + /* using same extractor types in same sequence */ + if (km->match_map[i]->extr_start_offs_id != + km1->match_map[i]->extr_start_offs_id || + km->match_map[i]->rel_offs != km1->match_map[i]->rel_offs || + km->match_map[i]->extr != km1->match_map[i]->extr || + km->match_map[i]->word_len != km1->match_map[i]->word_len) { + return 0; + } + } + + if (km->target == KM_CAM) { + /* in CAM must exactly match on all masks */ + for (int i = 0; i < km->key_word_size; i++) + if (km->entry_mask[i] != km1->entry_mask[i]) + return 0; + + /* Would be set later if not reusing from km1 */ + km->cam_paired = km1->cam_paired; + + } else if (km->target == KM_TCAM) { + /* + * If TCAM, we must make sure Recipe Key Mask does not + * mask out enable bits in masks + * Note: it is important that km1 is the original creator + * of the KM Recipe, since it contains its true masks + */ + for (int i = 0; i < km->key_word_size; i++) + if ((km->entry_mask[i] & km1->entry_mask[i]) != km->entry_mask[i]) + return 0; + + km->tcam_start_bank = km1->tcam_start_bank; + km->tcam_record = -1; /* needs to be found later */ + + } else { + NT_LOG(DBG, FILTER, "ERROR - KM target not defined or supported"); + return 0; + } + + /* + * Check for a flow clash. If already programmed return with -1 + */ + int double_match = 1; + + for (int i = 0; i < km->key_word_size; i++) { + if ((km->entry_word[i] & km->entry_mask[i]) != + (km1->entry_word[i] & km1->entry_mask[i])) { + double_match = 0; + break; + } + } + + if (double_match) + return -1; + + /* + * Note that TCAM and CAM may reuse same RCP and flow type + * when this happens, CAM entry wins on overlap + */ + + /* Use same KM Recipe and same flow type - return flow type */ + return km1->flow_type; +} + +int km_rcp_set(struct km_flow_def_s *km, int index) +{ + int qw = 0; + int sw = 0; + int swx = 0; + + hw_mod_km_rcp_set(km->be, HW_KM_RCP_PRESET_ALL, index, 0, 0); + + /* set extractor words, offs, contrib */ + for (int i = 0; i < km->num_ftype_elem; i++) { + switch (km->match_map[i]->extr) { + case KM_USE_EXTRACTOR_SWORD: + if (km->match_map[i]->extr_start_offs_id & SWX_INFO) { + if (km->target == KM_CAM && swx == 0) { + /* SWX */ + if (km->match_map[i]->extr_start_offs_id == SB_VNI) { + NT_LOG(DBG, FILTER, "Set KM SWX sel A - VNI"); + hw_mod_km_rcp_set(km->be, HW_KM_RCP_SWX_CCH, index, + 0, 1); + hw_mod_km_rcp_set(km->be, HW_KM_RCP_SWX_SEL_A, + index, 0, SWX_SEL_ALL32); + + } else if (km->match_map[i]->extr_start_offs_id == + SB_MAC_PORT) { + NT_LOG(DBG, FILTER, + "Set KM SWX sel A - PTC + MAC"); + hw_mod_km_rcp_set(km->be, HW_KM_RCP_SWX_SEL_A, + index, 0, SWX_SEL_ALL32); + + } else if (km->match_map[i]->extr_start_offs_id == + SB_KCC_ID) { + NT_LOG(DBG, FILTER, "Set KM SWX sel A - KCC ID"); + hw_mod_km_rcp_set(km->be, HW_KM_RCP_SWX_CCH, index, + 0, 1); + hw_mod_km_rcp_set(km->be, HW_KM_RCP_SWX_SEL_A, + index, 0, SWX_SEL_ALL32); + + } else { + return -1; + } + + } else { + return -1; + } + + swx++; + + } else { + if (sw == 0) { + /* DW8 */ + hw_mod_km_rcp_set(km->be, HW_KM_RCP_DW8_DYN, index, 0, + km->match_map[i]->extr_start_offs_id); + hw_mod_km_rcp_set(km->be, HW_KM_RCP_DW8_OFS, index, 0, + km->match_map[i]->rel_offs); + hw_mod_km_rcp_set(km->be, HW_KM_RCP_DW8_SEL_A, index, 0, + DW8_SEL_FIRST32); + NT_LOG(DBG, FILTER, + "Set KM DW8 sel A: dyn: %i, offs: %i", + km->match_map[i]->extr_start_offs_id, + km->match_map[i]->rel_offs); + + } else if (sw == 1) { + /* DW10 */ + hw_mod_km_rcp_set(km->be, HW_KM_RCP_DW10_DYN, index, 0, + km->match_map[i]->extr_start_offs_id); + hw_mod_km_rcp_set(km->be, HW_KM_RCP_DW10_OFS, index, 0, + km->match_map[i]->rel_offs); + hw_mod_km_rcp_set(km->be, HW_KM_RCP_DW10_SEL_A, index, 0, + DW10_SEL_FIRST32); + NT_LOG(DBG, FILTER, + "Set KM DW10 sel A: dyn: %i, offs: %i", + km->match_map[i]->extr_start_offs_id, + km->match_map[i]->rel_offs); + + } else { + return -1; + } + + sw++; + } + + break; + + case KM_USE_EXTRACTOR_QWORD: + if (qw == 0) { + hw_mod_km_rcp_set(km->be, HW_KM_RCP_QW0_DYN, index, 0, + km->match_map[i]->extr_start_offs_id); + hw_mod_km_rcp_set(km->be, HW_KM_RCP_QW0_OFS, index, 0, + km->match_map[i]->rel_offs); + + switch (km->match_map[i]->word_len) { + case 1: + hw_mod_km_rcp_set(km->be, HW_KM_RCP_QW0_SEL_A, index, 0, + QW0_SEL_FIRST32); + break; + + case 2: + hw_mod_km_rcp_set(km->be, HW_KM_RCP_QW0_SEL_A, index, 0, + QW0_SEL_FIRST64); + break; + + case 4: + hw_mod_km_rcp_set(km->be, HW_KM_RCP_QW0_SEL_A, index, 0, + QW0_SEL_ALL128); + break; + + default: + return -1; + } + + NT_LOG(DBG, FILTER, + "Set KM QW0 sel A: dyn: %i, offs: %i, size: %i", + km->match_map[i]->extr_start_offs_id, + km->match_map[i]->rel_offs, km->match_map[i]->word_len); + + } else if (qw == 1) { + hw_mod_km_rcp_set(km->be, HW_KM_RCP_QW4_DYN, index, 0, + km->match_map[i]->extr_start_offs_id); + hw_mod_km_rcp_set(km->be, HW_KM_RCP_QW4_OFS, index, 0, + km->match_map[i]->rel_offs); + + switch (km->match_map[i]->word_len) { + case 1: + hw_mod_km_rcp_set(km->be, HW_KM_RCP_QW4_SEL_A, index, 0, + QW4_SEL_FIRST32); + break; + + case 2: + hw_mod_km_rcp_set(km->be, HW_KM_RCP_QW4_SEL_A, index, 0, + QW4_SEL_FIRST64); + break; + + case 4: + hw_mod_km_rcp_set(km->be, HW_KM_RCP_QW4_SEL_A, index, 0, + QW4_SEL_ALL128); + break; + + default: + return -1; + } + + NT_LOG(DBG, FILTER, + "Set KM QW4 sel A: dyn: %i, offs: %i, size: %i", + km->match_map[i]->extr_start_offs_id, + km->match_map[i]->rel_offs, km->match_map[i]->word_len); + + } else { + return -1; + } + + qw++; + break; + + default: + return -1; + } + } + + /* set mask A */ + for (int i = 0; i < km->key_word_size; i++) { + hw_mod_km_rcp_set(km->be, HW_KM_RCP_MASK_A, index, + (km->be->km.nb_km_rcp_mask_a_word_size - 1) - i, + km->entry_mask[i]); + NT_LOG(DBG, FILTER, "Set KM mask A: %08x", km->entry_mask[i]); + } + + if (km->target == KM_CAM) { + /* set info - Color */ + if (km->info_set) { + hw_mod_km_rcp_set(km->be, HW_KM_RCP_INFO_A, index, 0, 1); + NT_LOG(DBG, FILTER, "Set KM info A"); + } + + /* set key length A */ + hw_mod_km_rcp_set(km->be, HW_KM_RCP_EL_A, index, 0, + km->key_word_size + !!km->info_set - 1); /* select id is -1 */ + /* set Flow Type for Key A */ + NT_LOG(DBG, FILTER, "Set KM EL A: %i", km->key_word_size + !!km->info_set - 1); + + hw_mod_km_rcp_set(km->be, HW_KM_RCP_FTM_A, index, 0, 1 << km->flow_type); + + NT_LOG(DBG, FILTER, "Set KM FTM A - ft: %i", km->flow_type); + + /* Set Paired - only on the CAM part though... TODO split CAM and TCAM */ + if ((uint32_t)(km->key_word_size + !!km->info_set) > + km->be->km.nb_cam_record_words) { + hw_mod_km_rcp_set(km->be, HW_KM_RCP_PAIRED, index, 0, 1); + NT_LOG(DBG, FILTER, "Set KM CAM Paired"); + km->cam_paired = 1; + } + + } else if (km->target == KM_TCAM) { + uint32_t bank_bm = 0; + + if (tcam_find_mapping(km) < 0) { + /* failed mapping into TCAM */ + NT_LOG(DBG, FILTER, "INFO: TCAM mapping flow failed"); + return -1; + } + + assert((uint32_t)(km->tcam_start_bank + km->key_word_size) <= + km->be->km.nb_tcam_banks); + + for (int i = 0; i < km->key_word_size; i++) { + bank_bm |= + (1 << (km->be->km.nb_tcam_banks - 1 - (km->tcam_start_bank + i))); + } + + /* Set BANK_A */ + hw_mod_km_rcp_set(km->be, HW_KM_RCP_BANK_A, index, 0, bank_bm); + /* Set Kl_A */ + hw_mod_km_rcp_set(km->be, HW_KM_RCP_KL_A, index, 0, km->key_word_size - 1); + + } else { + return -1; + } + + return 0; +} + +static int cam_populate(struct km_flow_def_s *km, int bank) +{ + int res = 0; + int cnt = km->key_word_size + !!km->info_set; + + for (uint32_t i = 0; i < km->be->km.nb_cam_record_words && cnt; i++, cnt--) { + res |= hw_mod_km_cam_set(km->be, HW_KM_CAM_W0 + i, bank, km->record_indexes[bank], + km->entry_word[i]); + res |= hw_mod_km_cam_set(km->be, HW_KM_CAM_FT0 + i, bank, km->record_indexes[bank], + km->flow_type); + } + + km->cam_dist[CAM_KM_DIST_IDX(bank)].km_owner = km; + + if (cnt) { + assert(km->cam_paired); + + for (uint32_t i = 0; i < km->be->km.nb_cam_record_words && cnt; i++, cnt--) { + res |= hw_mod_km_cam_set(km->be, HW_KM_CAM_W0 + i, bank, + km->record_indexes[bank] + 1, + km->entry_word[km->be->km.nb_cam_record_words + i]); + res |= hw_mod_km_cam_set(km->be, HW_KM_CAM_FT0 + i, bank, + km->record_indexes[bank] + 1, km->flow_type); + } + + km->cam_dist[CAM_KM_DIST_IDX(bank) + 1].km_owner = km; + } + + res |= hw_mod_km_cam_flush(km->be, bank, km->record_indexes[bank], km->cam_paired ? 2 : 1); + + return res; +} + +static int cam_reset_entry(struct km_flow_def_s *km, int bank) +{ + int res = 0; + int cnt = km->key_word_size + !!km->info_set; + + for (uint32_t i = 0; i < km->be->km.nb_cam_record_words && cnt; i++, cnt--) { + res |= hw_mod_km_cam_set(km->be, HW_KM_CAM_W0 + i, bank, km->record_indexes[bank], + 0); + res |= hw_mod_km_cam_set(km->be, HW_KM_CAM_FT0 + i, bank, km->record_indexes[bank], + 0); + } + + km->cam_dist[CAM_KM_DIST_IDX(bank)].km_owner = NULL; + + if (cnt) { + assert(km->cam_paired); + + for (uint32_t i = 0; i < km->be->km.nb_cam_record_words && cnt; i++, cnt--) { + res |= hw_mod_km_cam_set(km->be, HW_KM_CAM_W0 + i, bank, + km->record_indexes[bank] + 1, 0); + res |= hw_mod_km_cam_set(km->be, HW_KM_CAM_FT0 + i, bank, + km->record_indexes[bank] + 1, 0); + } + + km->cam_dist[CAM_KM_DIST_IDX(bank) + 1].km_owner = NULL; + } + + res |= hw_mod_km_cam_flush(km->be, bank, km->record_indexes[bank], km->cam_paired ? 2 : 1); + return res; +} + +static int move_cuckoo_index(struct km_flow_def_s *km) +{ + assert(km->cam_dist[CAM_KM_DIST_IDX(km->bank_used)].km_owner); + + for (uint32_t bank = 0; bank < km->be->km.nb_cam_banks; bank++) { + /* It will not select itself */ + if (km->cam_dist[CAM_KM_DIST_IDX(bank)].km_owner == NULL) { + if (km->cam_paired) { + if (km->cam_dist[CAM_KM_DIST_IDX(bank) + 1].km_owner != NULL) + continue; + } + + /* + * Populate in new position + */ + int res = cam_populate(km, bank); + + if (res) { + NT_LOG(DBG, FILTER, + "Error: failed to write to KM CAM in cuckoo move"); + return 0; + } + + /* + * Reset/free entry in old bank + * HW flushes are really not needed, the old addresses are always taken + * over by the caller If you change this code in future updates, this may + * no longer be true then! + */ + km->cam_dist[CAM_KM_DIST_IDX(km->bank_used)].km_owner = NULL; + + if (km->cam_paired) + km->cam_dist[CAM_KM_DIST_IDX(km->bank_used) + 1].km_owner = NULL; + + NT_LOG(DBG, FILTER, + "KM Cuckoo hash moved from bank %i to bank %i (%04X => %04X)", + km->bank_used, bank, CAM_KM_DIST_IDX(km->bank_used), + CAM_KM_DIST_IDX(bank)); + km->bank_used = bank; + (*km->cuckoo_moves)++; + return 1; + } + } + + return 0; +} + +static int move_cuckoo_index_level(struct km_flow_def_s *km_parent, int bank_idx, int levels, + int cam_adr_list_len) +{ + struct km_flow_def_s *km = km_parent->cam_dist[bank_idx].km_owner; + + assert(levels <= CUCKOO_MOVE_MAX_DEPTH); + + /* + * Only move if same pairness + * Can be extended later to handle both move of paired and single entries + */ + if (!km || km_parent->cam_paired != km->cam_paired) + return 0; + + if (move_cuckoo_index(km)) + return 1; + + if (levels <= 1) + return 0; + + assert(cam_adr_list_len < CUCKOO_MOVE_MAX_DEPTH); + + cam_addr_reserved_stack[cam_adr_list_len++] = bank_idx; + + for (uint32_t i = 0; i < km->be->km.nb_cam_banks; i++) { + int reserved = 0; + int new_idx = CAM_KM_DIST_IDX(i); + + for (int i_reserved = 0; i_reserved < cam_adr_list_len; i_reserved++) { + if (cam_addr_reserved_stack[i_reserved] == new_idx) { + reserved = 1; + break; + } + } + + if (reserved) + continue; + + int res = move_cuckoo_index_level(km, new_idx, levels - 1, cam_adr_list_len); + + if (res) { + if (move_cuckoo_index(km)) + return 1; + + assert(0); + } + } + + return 0; +} + +static int km_write_data_to_cam(struct km_flow_def_s *km) +{ + int res = 0; + assert(km->be->km.nb_cam_banks <= MAX_BANKS); + assert(km->cam_dist); + + NT_LOG(DBG, FILTER, "KM HASH [%03X, %03X, %03X]", km->record_indexes[0], + km->record_indexes[1], km->record_indexes[2]); + + if (km->info_set) + km->entry_word[km->key_word_size] = km->info; /* finally set info */ + + int bank = -1; + + /* + * first step, see if any of the banks are free + */ + for (uint32_t i_bank = 0; i_bank < km->be->km.nb_cam_banks; i_bank++) { + if (km->cam_dist[CAM_KM_DIST_IDX(i_bank)].km_owner == NULL) { + if (km->cam_paired == 0 || + km->cam_dist[CAM_KM_DIST_IDX(i_bank) + 1].km_owner == NULL) { + bank = i_bank; + break; + } + } + } + + if (bank < 0) { + /* + * Second step - cuckoo move existing flows if possible + */ + for (uint32_t i_bank = 0; i_bank < km->be->km.nb_cam_banks; i_bank++) { + if (move_cuckoo_index_level(km, CAM_KM_DIST_IDX(i_bank), 4, 0)) { + bank = i_bank; + break; + } + } + } + + if (bank < 0) + return -1; + + /* populate CAM */ + NT_LOG(DBG, FILTER, "KM Bank = %i (addr %04X)", bank, CAM_KM_DIST_IDX(bank)); + res = cam_populate(km, bank); + + if (res == 0) { + km->flushed_to_target = 1; + km->bank_used = bank; + } + + return res; +} + +/* + * TCAM + */ +static int tcam_find_free_record(struct km_flow_def_s *km, int start_bank) +{ + for (uint32_t rec = 0; rec < km->be->km.nb_tcam_bank_width; rec++) { + if (km->tcam_dist[TCAM_DIST_IDX(start_bank, rec)].km_owner == NULL) { + int pass = 1; + + for (int ii = 1; ii < km->key_word_size; ii++) { + if (km->tcam_dist[TCAM_DIST_IDX(start_bank + ii, rec)].km_owner != + NULL) { + pass = 0; + break; + } + } + + if (pass) { + km->tcam_record = rec; + return 1; + } + } + } + + return 0; +} + +static int tcam_find_mapping(struct km_flow_def_s *km) +{ + /* Search record and start index for this flow */ + for (int bs_idx = 0; bs_idx < km->num_start_offsets; bs_idx++) { + if (tcam_find_free_record(km, km->start_offsets[bs_idx])) { + km->tcam_start_bank = km->start_offsets[bs_idx]; + NT_LOG(DBG, FILTER, "Found space in TCAM start bank %i, record %i", + km->tcam_start_bank, km->tcam_record); + return 0; + } + } + + return -1; +} + +static int tcam_write_word(struct km_flow_def_s *km, int bank, int record, uint32_t word, + uint32_t mask) +{ + int err = 0; + uint32_t all_recs[3]; + + int rec_val = record / 32; + int rec_bit_shft = record % 32; + uint32_t rec_bit = (1 << rec_bit_shft); + + assert((km->be->km.nb_tcam_bank_width + 31) / 32 <= 3); + + for (int byte = 0; byte < 4; byte++) { + uint8_t a = (uint8_t)((word >> (24 - (byte * 8))) & 0xff); + uint8_t a_m = (uint8_t)((mask >> (24 - (byte * 8))) & 0xff); + /* calculate important value bits */ + a = a & a_m; + + for (int val = 0; val < 256; val++) { + err |= hw_mod_km_tcam_get(km->be, HW_KM_TCAM_T, bank, byte, val, all_recs); + + if ((val & a_m) == a) + all_recs[rec_val] |= rec_bit; + else + all_recs[rec_val] &= ~rec_bit; + + err |= hw_mod_km_tcam_set(km->be, HW_KM_TCAM_T, bank, byte, val, all_recs); + + if (err) + break; + } + } + + /* flush bank */ + err |= hw_mod_km_tcam_flush(km->be, bank, ALL_BANK_ENTRIES); + + if (err == 0) { + assert(km->tcam_dist[TCAM_DIST_IDX(bank, record)].km_owner == NULL); + km->tcam_dist[TCAM_DIST_IDX(bank, record)].km_owner = km; + } + + return err; +} + +static int km_write_data_to_tcam(struct km_flow_def_s *km) +{ + int err = 0; + + if (km->tcam_record < 0) { + tcam_find_free_record(km, km->tcam_start_bank); + + if (km->tcam_record < 0) { + NT_LOG(DBG, FILTER, "FAILED to find space in TCAM for flow"); + return -1; + } + + NT_LOG(DBG, FILTER, "Reused RCP: Found space in TCAM start bank %i, record %i", + km->tcam_start_bank, km->tcam_record); + } + + /* Write KM_TCI */ + err |= hw_mod_km_tci_set(km->be, HW_KM_TCI_COLOR, km->tcam_start_bank, km->tcam_record, + km->info); + err |= hw_mod_km_tci_set(km->be, HW_KM_TCI_FT, km->tcam_start_bank, km->tcam_record, + km->flow_type); + err |= hw_mod_km_tci_flush(km->be, km->tcam_start_bank, km->tcam_record, 1); + + for (int i = 0; i < km->key_word_size && !err; i++) { + err = tcam_write_word(km, km->tcam_start_bank + i, km->tcam_record, + km->entry_word[i], km->entry_mask[i]); + } + + if (err == 0) + km->flushed_to_target = 1; + + return err; +} + +static int tcam_reset_bank(struct km_flow_def_s *km, int bank, int record) +{ + int err = 0; + uint32_t all_recs[3]; + + int rec_val = record / 32; + int rec_bit_shft = record % 32; + uint32_t rec_bit = (1 << rec_bit_shft); + + assert((km->be->km.nb_tcam_bank_width + 31) / 32 <= 3); + + for (int byte = 0; byte < 4; byte++) { + for (int val = 0; val < 256; val++) { + err = hw_mod_km_tcam_get(km->be, HW_KM_TCAM_T, bank, byte, val, all_recs); + + if (err) + break; + + all_recs[rec_val] &= ~rec_bit; + err = hw_mod_km_tcam_set(km->be, HW_KM_TCAM_T, bank, byte, val, all_recs); + + if (err) + break; + } + } + + if (err) + return err; + + /* flush bank */ + err = hw_mod_km_tcam_flush(km->be, bank, ALL_BANK_ENTRIES); + km->tcam_dist[TCAM_DIST_IDX(bank, record)].km_owner = NULL; + + NT_LOG(DBG, FILTER, "Reset TCAM bank %i, rec_val %i rec bit %08x", bank, rec_val, + rec_bit); + + return err; +} + +static int tcam_reset_entry(struct km_flow_def_s *km) +{ + int err = 0; + + if (km->tcam_start_bank < 0 || km->tcam_record < 0) { + NT_LOG(DBG, FILTER, "FAILED to find space in TCAM for flow"); + return -1; + } + + /* Write KM_TCI */ + hw_mod_km_tci_set(km->be, HW_KM_TCI_COLOR, km->tcam_start_bank, km->tcam_record, 0); + hw_mod_km_tci_set(km->be, HW_KM_TCI_FT, km->tcam_start_bank, km->tcam_record, 0); + hw_mod_km_tci_flush(km->be, km->tcam_start_bank, km->tcam_record, 1); + + for (int i = 0; i < km->key_word_size && !err; i++) + err = tcam_reset_bank(km, km->tcam_start_bank + i, km->tcam_record); + + return err; +} + +int km_write_data_match_entry(struct km_flow_def_s *km, uint32_t color) +{ + int res = -1; + + km->info = color; + NT_LOG(DBG, FILTER, "Write Data entry Color: %08x", color); + + switch (km->target) { + case KM_CAM: + res = km_write_data_to_cam(km); + break; + + case KM_TCAM: + res = km_write_data_to_tcam(km); + break; + + case KM_SYNERGY: + default: + break; + } + + return res; +} + +int km_clear_data_match_entry(struct km_flow_def_s *km) +{ + int res = 0; + + if (km->root) { + struct km_flow_def_s *km1 = km->root; + + while (km1->reference != km) + km1 = km1->reference; + + km1->reference = km->reference; + + km->flushed_to_target = 0; + km->bank_used = 0; + + } else if (km->reference) { + km->reference->root = NULL; + + switch (km->target) { + case KM_CAM: + km->cam_dist[CAM_KM_DIST_IDX(km->bank_used)].km_owner = km->reference; + + if (km->key_word_size + !!km->info_set > 1) { + assert(km->cam_paired); + km->cam_dist[CAM_KM_DIST_IDX(km->bank_used) + 1].km_owner = + km->reference; + } + + break; + + case KM_TCAM: + for (int i = 0; i < km->key_word_size; i++) { + km->tcam_dist[TCAM_DIST_IDX(km->tcam_start_bank + i, + km->tcam_record)] + .km_owner = km->reference; + } + + break; + + case KM_SYNERGY: + default: + res = -1; + break; + } + + km->flushed_to_target = 0; + km->bank_used = 0; + + } else if (km->flushed_to_target) { + switch (km->target) { + case KM_CAM: + res = cam_reset_entry(km, km->bank_used); + break; + + case KM_TCAM: + res = tcam_reset_entry(km); + break; + + case KM_SYNERGY: + default: + res = -1; + break; + } + + km->flushed_to_target = 0; + km->bank_used = 0; + } + + return res; +} diff --git a/drivers/net/ntnic/nthw/flow_api/hw_mod/hw_mod_km.c b/drivers/net/ntnic/nthw/flow_api/hw_mod/hw_mod_km.c index 532884ca01..b8a30671c3 100644 --- a/drivers/net/ntnic/nthw/flow_api/hw_mod/hw_mod_km.c +++ b/drivers/net/ntnic/nthw/flow_api/hw_mod/hw_mod_km.c @@ -165,6 +165,240 @@ int hw_mod_km_rcp_flush(struct flow_api_backend_s *be, int start_idx, int count) return be->iface->km_rcp_flush(be->be_dev, &be->km, start_idx, count); } +static int hw_mod_km_rcp_mod(struct flow_api_backend_s *be, enum hw_km_e field, int index, + int word_off, uint32_t *value, int get) +{ + if ((unsigned int)index >= be->km.nb_categories) { + INDEX_TOO_LARGE_LOG; + return INDEX_TOO_LARGE; + } + + switch (_VER_) { + case 7: + switch (field) { + case HW_KM_RCP_PRESET_ALL: + if (get) { + UNSUP_FIELD_LOG; + return UNSUP_FIELD; + } + + memset(&be->km.v7.rcp[index], (uint8_t)*value, sizeof(struct km_v7_rcp_s)); + break; + + case HW_KM_RCP_QW0_DYN: + GET_SET(be->km.v7.rcp[index].qw0_dyn, value); + break; + + case HW_KM_RCP_QW0_OFS: + GET_SET_SIGNED(be->km.v7.rcp[index].qw0_ofs, value); + break; + + case HW_KM_RCP_QW0_SEL_A: + GET_SET(be->km.v7.rcp[index].qw0_sel_a, value); + break; + + case HW_KM_RCP_QW0_SEL_B: + GET_SET(be->km.v7.rcp[index].qw0_sel_b, value); + break; + + case HW_KM_RCP_QW4_DYN: + GET_SET(be->km.v7.rcp[index].qw4_dyn, value); + break; + + case HW_KM_RCP_QW4_OFS: + GET_SET_SIGNED(be->km.v7.rcp[index].qw4_ofs, value); + break; + + case HW_KM_RCP_QW4_SEL_A: + GET_SET(be->km.v7.rcp[index].qw4_sel_a, value); + break; + + case HW_KM_RCP_QW4_SEL_B: + GET_SET(be->km.v7.rcp[index].qw4_sel_b, value); + break; + + case HW_KM_RCP_DW8_DYN: + GET_SET(be->km.v7.rcp[index].dw8_dyn, value); + break; + + case HW_KM_RCP_DW8_OFS: + GET_SET_SIGNED(be->km.v7.rcp[index].dw8_ofs, value); + break; + + case HW_KM_RCP_DW8_SEL_A: + GET_SET(be->km.v7.rcp[index].dw8_sel_a, value); + break; + + case HW_KM_RCP_DW8_SEL_B: + GET_SET(be->km.v7.rcp[index].dw8_sel_b, value); + break; + + case HW_KM_RCP_DW10_DYN: + GET_SET(be->km.v7.rcp[index].dw10_dyn, value); + break; + + case HW_KM_RCP_DW10_OFS: + GET_SET_SIGNED(be->km.v7.rcp[index].dw10_ofs, value); + break; + + case HW_KM_RCP_DW10_SEL_A: + GET_SET(be->km.v7.rcp[index].dw10_sel_a, value); + break; + + case HW_KM_RCP_DW10_SEL_B: + GET_SET(be->km.v7.rcp[index].dw10_sel_b, value); + break; + + case HW_KM_RCP_SWX_CCH: + GET_SET(be->km.v7.rcp[index].swx_cch, value); + break; + + case HW_KM_RCP_SWX_SEL_A: + GET_SET(be->km.v7.rcp[index].swx_sel_a, value); + break; + + case HW_KM_RCP_SWX_SEL_B: + GET_SET(be->km.v7.rcp[index].swx_sel_b, value); + break; + + case HW_KM_RCP_MASK_A: + if (word_off > KM_RCP_MASK_D_A_SIZE) { + WORD_OFF_TOO_LARGE_LOG; + return WORD_OFF_TOO_LARGE; + } + + GET_SET(be->km.v7.rcp[index].mask_d_a[word_off], value); + break; + + case HW_KM_RCP_MASK_B: + if (word_off > KM_RCP_MASK_B_SIZE) { + WORD_OFF_TOO_LARGE_LOG; + return WORD_OFF_TOO_LARGE; + } + + GET_SET(be->km.v7.rcp[index].mask_b[word_off], value); + break; + + case HW_KM_RCP_DUAL: + GET_SET(be->km.v7.rcp[index].dual, value); + break; + + case HW_KM_RCP_PAIRED: + GET_SET(be->km.v7.rcp[index].paired, value); + break; + + case HW_KM_RCP_EL_A: + GET_SET(be->km.v7.rcp[index].el_a, value); + break; + + case HW_KM_RCP_EL_B: + GET_SET(be->km.v7.rcp[index].el_b, value); + break; + + case HW_KM_RCP_INFO_A: + GET_SET(be->km.v7.rcp[index].info_a, value); + break; + + case HW_KM_RCP_INFO_B: + GET_SET(be->km.v7.rcp[index].info_b, value); + break; + + case HW_KM_RCP_FTM_A: + GET_SET(be->km.v7.rcp[index].ftm_a, value); + break; + + case HW_KM_RCP_FTM_B: + GET_SET(be->km.v7.rcp[index].ftm_b, value); + break; + + case HW_KM_RCP_BANK_A: + GET_SET(be->km.v7.rcp[index].bank_a, value); + break; + + case HW_KM_RCP_BANK_B: + GET_SET(be->km.v7.rcp[index].bank_b, value); + break; + + case HW_KM_RCP_KL_A: + GET_SET(be->km.v7.rcp[index].kl_a, value); + break; + + case HW_KM_RCP_KL_B: + GET_SET(be->km.v7.rcp[index].kl_b, value); + break; + + case HW_KM_RCP_KEYWAY_A: + GET_SET(be->km.v7.rcp[index].keyway_a, value); + break; + + case HW_KM_RCP_KEYWAY_B: + GET_SET(be->km.v7.rcp[index].keyway_b, value); + break; + + case HW_KM_RCP_SYNERGY_MODE: + GET_SET(be->km.v7.rcp[index].synergy_mode, value); + break; + + case HW_KM_RCP_DW0_B_DYN: + GET_SET(be->km.v7.rcp[index].dw0_b_dyn, value); + break; + + case HW_KM_RCP_DW0_B_OFS: + GET_SET_SIGNED(be->km.v7.rcp[index].dw0_b_ofs, value); + break; + + case HW_KM_RCP_DW2_B_DYN: + GET_SET(be->km.v7.rcp[index].dw2_b_dyn, value); + break; + + case HW_KM_RCP_DW2_B_OFS: + GET_SET_SIGNED(be->km.v7.rcp[index].dw2_b_ofs, value); + break; + + case HW_KM_RCP_SW4_B_DYN: + GET_SET(be->km.v7.rcp[index].sw4_b_dyn, value); + break; + + case HW_KM_RCP_SW4_B_OFS: + GET_SET_SIGNED(be->km.v7.rcp[index].sw4_b_ofs, value); + break; + + case HW_KM_RCP_SW5_B_DYN: + GET_SET(be->km.v7.rcp[index].sw5_b_dyn, value); + break; + + case HW_KM_RCP_SW5_B_OFS: + GET_SET_SIGNED(be->km.v7.rcp[index].sw5_b_ofs, value); + break; + + default: + UNSUP_FIELD_LOG; + return UNSUP_FIELD; + } + + break; + + /* end case 7 */ + default: + UNSUP_VER_LOG; + return UNSUP_VER; + } + + return 0; +} + +int hw_mod_km_rcp_set(struct flow_api_backend_s *be, enum hw_km_e field, int index, int word_off, + uint32_t value) +{ + return hw_mod_km_rcp_mod(be, field, index, word_off, &value, 0); +} + +int hw_mod_km_rcp_get(struct flow_api_backend_s *be, enum hw_km_e field, int index, int word_off, + uint32_t *value) +{ + return hw_mod_km_rcp_mod(be, field, index, word_off, value, 1); +} + int hw_mod_km_cam_flush(struct flow_api_backend_s *be, int start_bank, int start_record, int count) { if (count == ALL_ENTRIES) @@ -180,6 +414,103 @@ int hw_mod_km_cam_flush(struct flow_api_backend_s *be, int start_bank, int start return be->iface->km_cam_flush(be->be_dev, &be->km, start_bank, start_record, count); } +static int hw_mod_km_cam_mod(struct flow_api_backend_s *be, enum hw_km_e field, int bank, + int record, uint32_t *value, int get) +{ + if ((unsigned int)bank >= be->km.nb_cam_banks) { + INDEX_TOO_LARGE_LOG; + return INDEX_TOO_LARGE; + } + + if ((unsigned int)record >= be->km.nb_cam_records) { + INDEX_TOO_LARGE_LOG; + return INDEX_TOO_LARGE; + } + + unsigned int index = bank * be->km.nb_cam_records + record; + + switch (_VER_) { + case 7: + switch (field) { + case HW_KM_CAM_PRESET_ALL: + if (get) { + UNSUP_FIELD_LOG; + return UNSUP_FIELD; + } + + memset(&be->km.v7.cam[index], (uint8_t)*value, sizeof(struct km_v7_cam_s)); + break; + + case HW_KM_CAM_W0: + GET_SET(be->km.v7.cam[index].w0, value); + break; + + case HW_KM_CAM_W1: + GET_SET(be->km.v7.cam[index].w1, value); + break; + + case HW_KM_CAM_W2: + GET_SET(be->km.v7.cam[index].w2, value); + break; + + case HW_KM_CAM_W3: + GET_SET(be->km.v7.cam[index].w3, value); + break; + + case HW_KM_CAM_W4: + GET_SET(be->km.v7.cam[index].w4, value); + break; + + case HW_KM_CAM_W5: + GET_SET(be->km.v7.cam[index].w5, value); + break; + + case HW_KM_CAM_FT0: + GET_SET(be->km.v7.cam[index].ft0, value); + break; + + case HW_KM_CAM_FT1: + GET_SET(be->km.v7.cam[index].ft1, value); + break; + + case HW_KM_CAM_FT2: + GET_SET(be->km.v7.cam[index].ft2, value); + break; + + case HW_KM_CAM_FT3: + GET_SET(be->km.v7.cam[index].ft3, value); + break; + + case HW_KM_CAM_FT4: + GET_SET(be->km.v7.cam[index].ft4, value); + break; + + case HW_KM_CAM_FT5: + GET_SET(be->km.v7.cam[index].ft5, value); + break; + + default: + UNSUP_FIELD_LOG; + return UNSUP_FIELD; + } + + break; + + /* end case 7 */ + default: + UNSUP_VER_LOG; + return UNSUP_VER; + } + + return 0; +} + +int hw_mod_km_cam_set(struct flow_api_backend_s *be, enum hw_km_e field, int bank, int record, + uint32_t value) +{ + return hw_mod_km_cam_mod(be, field, bank, record, &value, 0); +} + int hw_mod_km_tcam_flush(struct flow_api_backend_s *be, int start_bank, int count) { if (count == ALL_ENTRIES) @@ -273,6 +604,12 @@ int hw_mod_km_tcam_set(struct flow_api_backend_s *be, enum hw_km_e field, int ba return hw_mod_km_tcam_mod(be, field, bank, byte, byte_val, value_set, 0); } +int hw_mod_km_tcam_get(struct flow_api_backend_s *be, enum hw_km_e field, int bank, int byte, + int byte_val, uint32_t *value_set) +{ + return hw_mod_km_tcam_mod(be, field, bank, byte, byte_val, value_set, 1); +} + int hw_mod_km_tci_flush(struct flow_api_backend_s *be, int start_bank, int start_record, int count) { if (count == ALL_ENTRIES) @@ -288,6 +625,49 @@ int hw_mod_km_tci_flush(struct flow_api_backend_s *be, int start_bank, int start return be->iface->km_tci_flush(be->be_dev, &be->km, start_bank, start_record, count); } +static int hw_mod_km_tci_mod(struct flow_api_backend_s *be, enum hw_km_e field, int bank, + int record, uint32_t *value, int get) +{ + unsigned int index = bank * be->km.nb_tcam_bank_width + record; + + if (index >= (be->km.nb_tcam_banks * be->km.nb_tcam_bank_width)) { + INDEX_TOO_LARGE_LOG; + return INDEX_TOO_LARGE; + } + + switch (_VER_) { + case 7: + switch (field) { + case HW_KM_TCI_COLOR: + GET_SET(be->km.v7.tci[index].color, value); + break; + + case HW_KM_TCI_FT: + GET_SET(be->km.v7.tci[index].ft, value); + break; + + default: + UNSUP_FIELD_LOG; + return UNSUP_FIELD; + } + + break; + + /* end case 7 */ + default: + UNSUP_VER_LOG; + return UNSUP_VER; + } + + return 0; +} + +int hw_mod_km_tci_set(struct flow_api_backend_s *be, enum hw_km_e field, int bank, int record, + uint32_t value) +{ + return hw_mod_km_tci_mod(be, field, bank, record, &value, 0); +} + int hw_mod_km_tcq_flush(struct flow_api_backend_s *be, int start_bank, int start_record, int count) { if (count == ALL_ENTRIES) diff --git a/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_hw_db_inline.c b/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_hw_db_inline.c index 464c2fa81c..feac15cd9f 100644 --- a/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_hw_db_inline.c +++ b/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_hw_db_inline.c @@ -39,7 +39,19 @@ struct hw_db_inline_resource_db { int ref; } *cat; + struct hw_db_inline_resource_db_km_rcp { + struct hw_db_inline_km_rcp_data data; + int ref; + + struct hw_db_inline_resource_db_km_ft { + struct hw_db_inline_km_ft_data data; + int ref; + } *ft; + } *km; + uint32_t nb_cat; + uint32_t nb_km_ft; + uint32_t nb_km_rcp; /* Hardware */ @@ -90,6 +102,25 @@ int hw_db_inline_create(struct flow_nic_dev *ndev, void **db_handle) return -1; } + db->nb_km_ft = ndev->be.cat.nb_flow_types; + db->nb_km_rcp = ndev->be.km.nb_categories; + db->km = calloc(db->nb_km_rcp, sizeof(struct hw_db_inline_resource_db_km_rcp)); + + if (db->km == NULL) { + hw_db_inline_destroy(db); + return -1; + } + + for (uint32_t i = 0; i < db->nb_km_rcp; ++i) { + db->km[i].ft = calloc(db->nb_km_ft * db->nb_cat, + sizeof(struct hw_db_inline_resource_db_km_ft)); + + if (db->km[i].ft == NULL) { + hw_db_inline_destroy(db); + return -1; + } + } + *db_handle = db; return 0; } @@ -103,6 +134,13 @@ void hw_db_inline_destroy(void *db_handle) free(db->slc_lr); free(db->cat); + if (db->km) { + for (uint32_t i = 0; i < db->nb_km_rcp; ++i) + free(db->km[i].ft); + + free(db->km); + } + free(db->cfn); free(db); @@ -133,12 +171,61 @@ void hw_db_inline_deref_idxs(struct flow_nic_dev *ndev, void *db_handle, struct *(struct hw_db_slc_lr_idx *)&idxs[i]); break; + case HW_DB_IDX_TYPE_KM_RCP: + hw_db_inline_km_deref(ndev, db_handle, *(struct hw_db_km_idx *)&idxs[i]); + break; + + case HW_DB_IDX_TYPE_KM_FT: + hw_db_inline_km_ft_deref(ndev, db_handle, *(struct hw_db_km_ft *)&idxs[i]); + break; + default: break; } } } + +const void *hw_db_inline_find_data(struct flow_nic_dev *ndev, void *db_handle, + enum hw_db_idx_type type, struct hw_db_idx *idxs, uint32_t size) +{ + (void)ndev; + struct hw_db_inline_resource_db *db = (struct hw_db_inline_resource_db *)db_handle; + + for (uint32_t i = 0; i < size; ++i) { + if (idxs[i].type != type) + continue; + + switch (type) { + case HW_DB_IDX_TYPE_NONE: + return NULL; + + case HW_DB_IDX_TYPE_CAT: + return &db->cat[idxs[i].ids].data; + + case HW_DB_IDX_TYPE_QSL: + return &db->qsl[idxs[i].ids].data; + + case HW_DB_IDX_TYPE_COT: + return &db->cot[idxs[i].ids].data; + + case HW_DB_IDX_TYPE_SLC_LR: + return &db->slc_lr[idxs[i].ids].data; + + case HW_DB_IDX_TYPE_KM_RCP: + return &db->km[idxs[i].id1].data; + + case HW_DB_IDX_TYPE_KM_FT: + return NULL; /* FTs can't be easily looked up */ + + default: + return NULL; + } + } + + return NULL; +} + /******************************************************************************/ /* Filter */ /******************************************************************************/ @@ -613,3 +700,150 @@ void hw_db_inline_cat_deref(struct flow_nic_dev *ndev, void *db_handle, struct h db->cat[idx.ids].ref = 0; } } + +/******************************************************************************/ +/* KM RCP */ +/******************************************************************************/ + +static int hw_db_inline_km_compare(const struct hw_db_inline_km_rcp_data *data1, + const struct hw_db_inline_km_rcp_data *data2) +{ + return data1->rcp == data2->rcp; +} + +struct hw_db_km_idx hw_db_inline_km_add(struct flow_nic_dev *ndev, void *db_handle, + const struct hw_db_inline_km_rcp_data *data) +{ + struct hw_db_inline_resource_db *db = (struct hw_db_inline_resource_db *)db_handle; + struct hw_db_km_idx idx = { .raw = 0 }; + int found = 0; + + idx.type = HW_DB_IDX_TYPE_KM_RCP; + + for (uint32_t i = 0; i < db->nb_km_rcp; ++i) { + if (!found && db->km[i].ref <= 0) { + found = 1; + idx.id1 = i; + } + + if (db->km[i].ref > 0 && hw_db_inline_km_compare(data, &db->km[i].data)) { + idx.id1 = i; + hw_db_inline_km_ref(ndev, db, idx); + return idx; + } + } + + if (!found) { + idx.error = 1; + return idx; + } + + memcpy(&db->km[idx.id1].data, data, sizeof(struct hw_db_inline_km_rcp_data)); + db->km[idx.id1].ref = 1; + + return idx; +} + +void hw_db_inline_km_ref(struct flow_nic_dev *ndev, void *db_handle, struct hw_db_km_idx idx) +{ + (void)ndev; + struct hw_db_inline_resource_db *db = (struct hw_db_inline_resource_db *)db_handle; + + if (!idx.error) + db->km[idx.id1].ref += 1; +} + +void hw_db_inline_km_deref(struct flow_nic_dev *ndev, void *db_handle, struct hw_db_km_idx idx) +{ + (void)ndev; + (void)db_handle; + + if (idx.error) + return; +} + +/******************************************************************************/ +/* KM FT */ +/******************************************************************************/ + +static int hw_db_inline_km_ft_compare(const struct hw_db_inline_km_ft_data *data1, + const struct hw_db_inline_km_ft_data *data2) +{ + return data1->cat.raw == data2->cat.raw && data1->km.raw == data2->km.raw && + data1->action_set.raw == data2->action_set.raw; +} + +struct hw_db_km_ft hw_db_inline_km_ft_add(struct flow_nic_dev *ndev, void *db_handle, + const struct hw_db_inline_km_ft_data *data) +{ + struct hw_db_inline_resource_db *db = (struct hw_db_inline_resource_db *)db_handle; + struct hw_db_inline_resource_db_km_rcp *km_rcp = &db->km[data->km.id1]; + struct hw_db_km_ft idx = { .raw = 0 }; + uint32_t cat_offset = data->cat.ids * db->nb_cat; + int found = 0; + + idx.type = HW_DB_IDX_TYPE_KM_FT; + idx.id2 = data->km.id1; + idx.id3 = data->cat.ids; + + if (km_rcp->data.rcp == 0) { + idx.id1 = 0; + return idx; + } + + for (uint32_t i = 1; i < db->nb_km_ft; ++i) { + const struct hw_db_inline_resource_db_km_ft *km_ft = &km_rcp->ft[cat_offset + i]; + + if (!found && km_ft->ref <= 0) { + found = 1; + idx.id1 = i; + } + + if (km_ft->ref > 0 && hw_db_inline_km_ft_compare(data, &km_ft->data)) { + idx.id1 = i; + hw_db_inline_km_ft_ref(ndev, db, idx); + return idx; + } + } + + if (!found) { + idx.error = 1; + return idx; + } + + memcpy(&km_rcp->ft[cat_offset + idx.id1].data, data, + sizeof(struct hw_db_inline_km_ft_data)); + km_rcp->ft[cat_offset + idx.id1].ref = 1; + + return idx; +} + +void hw_db_inline_km_ft_ref(struct flow_nic_dev *ndev, void *db_handle, struct hw_db_km_ft idx) +{ + (void)ndev; + struct hw_db_inline_resource_db *db = (struct hw_db_inline_resource_db *)db_handle; + + if (!idx.error) { + uint32_t cat_offset = idx.id3 * db->nb_cat; + db->km[idx.id2].ft[cat_offset + idx.id1].ref += 1; + } +} + +void hw_db_inline_km_ft_deref(struct flow_nic_dev *ndev, void *db_handle, struct hw_db_km_ft idx) +{ + (void)ndev; + struct hw_db_inline_resource_db *db = (struct hw_db_inline_resource_db *)db_handle; + struct hw_db_inline_resource_db_km_rcp *km_rcp = &db->km[idx.id2]; + uint32_t cat_offset = idx.id3 * db->nb_cat; + + if (idx.error) + return; + + km_rcp->ft[cat_offset + idx.id1].ref -= 1; + + if (km_rcp->ft[cat_offset + idx.id1].ref <= 0) { + memset(&km_rcp->ft[cat_offset + idx.id1].data, 0x0, + sizeof(struct hw_db_inline_km_ft_data)); + km_rcp->ft[cat_offset + idx.id1].ref = 0; + } +} diff --git a/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_hw_db_inline.h b/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_hw_db_inline.h index d0435acaef..e104ba7327 100644 --- a/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_hw_db_inline.h +++ b/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_hw_db_inline.h @@ -32,6 +32,10 @@ struct hw_db_idx { HW_DB_IDX; }; +struct hw_db_action_set_idx { + HW_DB_IDX; +}; + struct hw_db_cot_idx { HW_DB_IDX; }; @@ -48,12 +52,22 @@ struct hw_db_slc_lr_idx { HW_DB_IDX; }; +struct hw_db_km_idx { + HW_DB_IDX; +}; + +struct hw_db_km_ft { + HW_DB_IDX; +}; + enum hw_db_idx_type { HW_DB_IDX_TYPE_NONE = 0, HW_DB_IDX_TYPE_COT, HW_DB_IDX_TYPE_CAT, HW_DB_IDX_TYPE_QSL, HW_DB_IDX_TYPE_SLC_LR, + HW_DB_IDX_TYPE_KM_RCP, + HW_DB_IDX_TYPE_KM_FT, }; /* Functionality data types */ @@ -123,6 +137,16 @@ struct hw_db_inline_action_set_data { }; }; +struct hw_db_inline_km_rcp_data { + uint32_t rcp; +}; + +struct hw_db_inline_km_ft_data { + struct hw_db_cat_idx cat; + struct hw_db_km_idx km; + struct hw_db_action_set_idx action_set; +}; + /**/ int hw_db_inline_create(struct flow_nic_dev *ndev, void **db_handle); @@ -130,6 +154,8 @@ void hw_db_inline_destroy(void *db_handle); void hw_db_inline_deref_idxs(struct flow_nic_dev *ndev, void *db_handle, struct hw_db_idx *idxs, uint32_t size); +const void *hw_db_inline_find_data(struct flow_nic_dev *ndev, void *db_handle, + enum hw_db_idx_type type, struct hw_db_idx *idxs, uint32_t size); /**/ struct hw_db_cot_idx hw_db_inline_cot_add(struct flow_nic_dev *ndev, void *db_handle, @@ -158,6 +184,18 @@ void hw_db_inline_cat_deref(struct flow_nic_dev *ndev, void *db_handle, struct h /**/ +struct hw_db_km_idx hw_db_inline_km_add(struct flow_nic_dev *ndev, void *db_handle, + const struct hw_db_inline_km_rcp_data *data); +void hw_db_inline_km_ref(struct flow_nic_dev *ndev, void *db_handle, struct hw_db_km_idx idx); +void hw_db_inline_km_deref(struct flow_nic_dev *ndev, void *db_handle, struct hw_db_km_idx idx); + +struct hw_db_km_ft hw_db_inline_km_ft_add(struct flow_nic_dev *ndev, void *db_handle, + const struct hw_db_inline_km_ft_data *data); +void hw_db_inline_km_ft_ref(struct flow_nic_dev *ndev, void *db_handle, struct hw_db_km_ft idx); +void hw_db_inline_km_ft_deref(struct flow_nic_dev *ndev, void *db_handle, struct hw_db_km_ft idx); + +/**/ + int hw_db_inline_setup_mbr_filter(struct flow_nic_dev *ndev, uint32_t cat_hw_id, uint32_t ft, uint32_t qsl_hw_id); diff --git a/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_profile_inline.c b/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_profile_inline.c index 9b504217d2..811659118d 100644 --- a/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_profile_inline.c +++ b/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_profile_inline.c @@ -2336,17 +2336,28 @@ static int setup_flow_flm_actions(struct flow_eth_dev *dev, uint32_t *flm_scrub, struct rte_flow_error *error) { - (void)dev; - (void)fd; (void)group; - (void)local_idxs; - (void)local_idx_counter; (void)flm_rpl_ext_ptr; (void)flm_ft; (void)flm_scrub; - (void)qsl_data; (void)hsh_data; - (void)error; + + const bool empty_pattern = fd_has_empty_pattern(fd); + + /* Setup COT */ + struct hw_db_inline_cot_data cot_data = { + .matcher_color_contrib = empty_pattern ? 0x0 : 0x4, /* FT key C */ + .frag_rcp = 0, + }; + struct hw_db_cot_idx cot_idx = + hw_db_inline_cot_add(dev->ndev, dev->ndev->hw_db_handle, &cot_data); + local_idxs[(*local_idx_counter)++] = cot_idx.raw; + + if (cot_idx.error) { + NT_LOG(ERR, FILTER, "Could not reference COT resource"); + flow_nic_set_error(ERR_MATCH_RESOURCE_EXHAUSTION, error); + return -1; + } /* Finalize QSL */ struct hw_db_qsl_idx qsl_idx = @@ -2448,6 +2459,8 @@ static struct flow_handle *create_flow_filter(struct flow_eth_dev *dev, struct n /* * Flow for group 0 */ + int identical_km_entry_ft = -1; + struct hw_db_inline_action_set_data action_set_data = { 0 }; (void)action_set_data; @@ -2522,6 +2535,130 @@ static struct flow_handle *create_flow_filter(struct flow_eth_dev *dev, struct n goto error_out; } + /* Setup KM RCP */ + struct hw_db_inline_km_rcp_data km_rcp_data = { .rcp = 0 }; + + if (fd->km.num_ftype_elem) { + struct flow_handle *flow = dev->ndev->flow_base, *found_flow = NULL; + + if (km_key_create(&fd->km, fh->port_id)) { + NT_LOG(ERR, FILTER, "KM creation failed"); + flow_nic_set_error(ERR_MATCH_FAILED_BY_HW_LIMITS, error); + goto error_out; + } + + fd->km.be = &dev->ndev->be; + + /* Look for existing KM RCPs */ + while (flow) { + if (flow->type == FLOW_HANDLE_TYPE_FLOW && + flow->fd->km.flow_type) { + int res = km_key_compare(&fd->km, &flow->fd->km); + + if (res < 0) { + /* Flow rcp and match data is identical */ + identical_km_entry_ft = flow->fd->km.flow_type; + found_flow = flow; + break; + } + + if (res > 0) { + /* Flow rcp found and match data is different */ + found_flow = flow; + } + } + + flow = flow->next; + } + + km_attach_ndev_resource_management(&fd->km, &dev->ndev->km_res_handle); + + if (found_flow != NULL) { + /* Reuse existing KM RCP */ + const struct hw_db_inline_km_rcp_data *other_km_rcp_data = + hw_db_inline_find_data(dev->ndev, dev->ndev->hw_db_handle, + HW_DB_IDX_TYPE_KM_RCP, + (struct hw_db_idx *) + found_flow->flm_db_idxs, + found_flow->flm_db_idx_counter); + + if (other_km_rcp_data == NULL || + flow_nic_ref_resource(dev->ndev, RES_KM_CATEGORY, + other_km_rcp_data->rcp)) { + NT_LOG(ERR, FILTER, + "Could not reference existing KM RCP resource"); + flow_nic_set_error(ERR_MATCH_RESOURCE_EXHAUSTION, error); + goto error_out; + } + + km_rcp_data.rcp = other_km_rcp_data->rcp; + } else { + /* Alloc new KM RCP */ + int rcp = flow_nic_alloc_resource(dev->ndev, RES_KM_CATEGORY, 1); + + if (rcp < 0) { + NT_LOG(ERR, FILTER, + "Could not reference KM RCP resource (flow_nic_alloc)"); + flow_nic_set_error(ERR_MATCH_RESOURCE_EXHAUSTION, error); + goto error_out; + } + + km_rcp_set(&fd->km, rcp); + km_rcp_data.rcp = (uint32_t)rcp; + } + } + + struct hw_db_km_idx km_idx = + hw_db_inline_km_add(dev->ndev, dev->ndev->hw_db_handle, &km_rcp_data); + + fh->db_idxs[fh->db_idx_counter++] = km_idx.raw; + + if (km_idx.error) { + NT_LOG(ERR, FILTER, "Could not reference KM RCP resource (db_inline)"); + flow_nic_set_error(ERR_MATCH_RESOURCE_EXHAUSTION, error); + goto error_out; + } + + /* Setup KM FT */ + struct hw_db_inline_km_ft_data km_ft_data = { + .cat = cat_idx, + .km = km_idx, + }; + struct hw_db_km_ft km_ft_idx = + hw_db_inline_km_ft_add(dev->ndev, dev->ndev->hw_db_handle, &km_ft_data); + fh->db_idxs[fh->db_idx_counter++] = km_ft_idx.raw; + + if (km_ft_idx.error) { + NT_LOG(ERR, FILTER, "Could not reference KM FT resource"); + flow_nic_set_error(ERR_MATCH_RESOURCE_EXHAUSTION, error); + goto error_out; + } + + /* Finalize KM RCP */ + if (fd->km.num_ftype_elem) { + if (identical_km_entry_ft >= 0 && identical_km_entry_ft != km_ft_idx.id1) { + NT_LOG(ERR, FILTER, + "Identical KM matches cannot have different KM FTs"); + flow_nic_set_error(ERR_MATCH_FAILED_BY_HW_LIMITS, error); + goto error_out; + } + + fd->km.flow_type = km_ft_idx.id1; + + if (fd->km.target == KM_CAM) { + uint32_t ft_a_mask = 0; + hw_mod_km_rcp_get(&dev->ndev->be, HW_KM_RCP_FTM_A, + (int)km_rcp_data.rcp, 0, &ft_a_mask); + hw_mod_km_rcp_set(&dev->ndev->be, HW_KM_RCP_FTM_A, + (int)km_rcp_data.rcp, 0, + ft_a_mask | (1 << fd->km.flow_type)); + } + + hw_mod_km_rcp_flush(&dev->ndev->be, (int)km_rcp_data.rcp, 1); + + km_write_data_match_entry(&fd->km, 0); + } + nic_insert_flow(dev->ndev, fh); } @@ -2802,6 +2939,25 @@ int flow_destroy_locked_profile_inline(struct flow_eth_dev *dev, } else { NT_LOG(DBG, FILTER, "removing flow :%p", fh); + if (fh->fd->km.num_ftype_elem) { + km_clear_data_match_entry(&fh->fd->km); + + const struct hw_db_inline_km_rcp_data *other_km_rcp_data = + hw_db_inline_find_data(dev->ndev, dev->ndev->hw_db_handle, + HW_DB_IDX_TYPE_KM_RCP, + (struct hw_db_idx *)fh->flm_db_idxs, + fh->flm_db_idx_counter); + + if (other_km_rcp_data != NULL && + flow_nic_deref_resource(dev->ndev, RES_KM_CATEGORY, + (int)other_km_rcp_data->rcp) == 0) { + hw_mod_km_rcp_set(&dev->ndev->be, HW_KM_RCP_PRESET_ALL, + (int)other_km_rcp_data->rcp, 0, 0); + hw_mod_km_rcp_flush(&dev->ndev->be, (int)other_km_rcp_data->rcp, + 1); + } + } + hw_db_inline_deref_idxs(dev->ndev, dev->ndev->hw_db_handle, (struct hw_db_idx *)fh->db_idxs, fh->db_idx_counter); free(fh->fd); -- 2.45.0