From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BE41C4411D; Fri, 31 May 2024 17:49:35 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 96AB142DE3; Fri, 31 May 2024 17:48:03 +0200 (CEST) Received: from egress-ip11a.ess.de.barracuda.com (egress-ip11a.ess.de.barracuda.com [18.184.203.234]) by mails.dpdk.org (Postfix) with ESMTP id D6D5142D99 for ; Fri, 31 May 2024 17:47:50 +0200 (CEST) Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-db3eur04lp2050.outbound.protection.outlook.com [104.47.12.50]) by mx-outbound41-244.eu-central-1c.ess.aws.cudaops.com (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 31 May 2024 15:47:47 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jgGYA5Xx366sFu6qgsjm/Hh7Td9cEUzGoBk1xcBaM8xW4s42QJIXwuMW+3DOycVLA/+IFlCpM7JkKyW5dPHUyCYIKNSXwc00CBnAMsHymVmg042/UAEoa7CGwy7zUYz5p85ph5SpXeg7hOO43WGuEGBPhmNEh2XzU0m5w/1WrjK6JHwiTfqMNsZXrApz5IZvBKadz8QPHXo+sFLU0dU4Fdr7W3eUzXXm8+4TuClrRjGy3e35p5YF83bM2SrNnS0amJ2WSnHSoJtozx/HWm00GScclOiWRbWhUWfxFN60w7edWEK9kroHzwWFlOcB+ZS8WcSPolUhxd5MjON8xjRdTw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=3r4qjUXO7OksVJeQGVkcBsNjuwSlNwbGRUI/MXWcujw=; b=GVNlAhTS/x13Y9z78HVL4AMEEyZvvnTsUlKrMLDB7hwGRr8mlop5vpfrR9mH/h0Lwu/mSN8SiSNVwQWPZXHl0/KVv9e0jEZaTWzfgzdh6X3Q5z0c5sp2ByLIa7fiTMvnYwsrL7I4wunvVbHo0TPLNcQ9ExqmRarfZMbHE9YMO21yK/uTpkLacKMNfUVwJPQg6O5ymz53o2PS1AeQOzCcn3IfZgHAqnOwMD6xA9B+cXxSfWvFXMLWw73zQFVSFoCBQjLwbUncTgYi/DBuoZdGoXqrjC1gWNaUhnuByRg11jUqn6SzqkTKs1DsvTfOZYsJXWxFjy4IJ1eyPIQUdF2YNA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=fail (sender ip is 178.72.21.4) smtp.rcpttodomain=dpdk.org smtp.mailfrom=napatech.com; dmarc=fail (p=reject sp=reject pct=100) action=oreject header.from=napatech.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=napatech.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3r4qjUXO7OksVJeQGVkcBsNjuwSlNwbGRUI/MXWcujw=; b=hyXC+UheLVTfoRqe+NCeXbNqz2pX+4m3zvu+twk1zz8cnohAfBfJyF1n/BZLJyq2z9jIhdFd7Hdjc7mLvY12KqpzoCyXYnR0n+XrCZtOa9iMUwolTM7yF/Z2bAkb/per3j/yLzvFV9wlv5gLwaDN64zdr0PEdzJFMalIREyg7+w= Received: from AS4P251CA0022.EURP251.PROD.OUTLOOK.COM (2603:10a6:20b:5d3::14) by PR3P190MB0860.EURP190.PROD.OUTLOOK.COM (2603:10a6:102:8c::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.24; Fri, 31 May 2024 15:47:38 +0000 Received: from AMS1EPF00000047.eurprd04.prod.outlook.com (2603:10a6:20b:5d3:cafe::70) by AS4P251CA0022.outlook.office365.com (2603:10a6:20b:5d3::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7452.36 via Frontend Transport; Fri, 31 May 2024 15:47:38 +0000 X-MS-Exchange-Authentication-Results: spf=fail (sender IP is 178.72.21.4) smtp.mailfrom=napatech.com; dkim=none (message not signed) header.d=none;dmarc=fail action=oreject header.from=napatech.com; Received-SPF: Fail (protection.outlook.com: domain of napatech.com does not designate 178.72.21.4 as permitted sender) receiver=protection.outlook.com; client-ip=178.72.21.4; helo=localhost.localdomain; Received: from localhost.localdomain (178.72.21.4) by AMS1EPF00000047.mail.protection.outlook.com (10.167.16.135) with Microsoft SMTP Server id 15.20.7633.15 via Frontend Transport; Fri, 31 May 2024 15:47:38 +0000 From: Serhii Iliushyk To: dev@dpdk.org Cc: mko-plv@napatech.com, ckm@napatech.com, andrew.rybchenko@oktetlabs.ru, ferruh.yigit@amd.com Subject: [PATCH v2 02/17] net/ntnic: add core platform functionality Date: Fri, 31 May 2024 17:47:06 +0200 Message-ID: <20240531154731.1856874-2-sil-plv@napatech.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240531154731.1856874-1-sil-plv@napatech.com> References: <20240530144929.4127931-1-sil-plv@napatech.com> <20240531154731.1856874-1-sil-plv@napatech.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AMS1EPF00000047:EE_|PR3P190MB0860:EE_ Content-Type: text/plain X-MS-Office365-Filtering-Correlation-Id: c418d191-5fe9-4630-106f-08dc8188ffb0 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230031|1800799015|376005|36860700004|82310400017; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?QhwMJ8hgBI0l9bzGs7GKrsxD7cFDzeAegboJW0KPUDHY+0rbTkbR979rggFg?= =?us-ascii?Q?txJnBn2xGBKK00LtpTFMvh8ZM8PDaytrwC92u86xwQpEa/GJo6HrdRKHWb67?= =?us-ascii?Q?GDjcysuEisin4XSAhYAMWg305aJjxokUsZFeflvuBfdqfum9PF7BBXnX0/gv?= =?us-ascii?Q?zrthd2VsiWl0uayGpxcLqJ2OBRD6Oop6OYKcyp7v57C5DCSWskTyxuAvPbk0?= =?us-ascii?Q?vSmso/pURyxtjX8aYvlmDLJ90DrFdT6fGChNyDUQF8YnILThjKnzfcXrj4oC?= =?us-ascii?Q?wzkfRruZaIpO+7Ds9F1D0Q499Vh6UutOdxl1Ag6vAqqomgYgIWD03ZAKJAFY?= =?us-ascii?Q?0mLcXKpU6ltS6j2vWRVdeK0Beg7svxwLjNJOzH6HpXz5a9Y4vdg/JRvj86BX?= =?us-ascii?Q?/4Mbb9DDWHfUIYcHrP+Uh5O3wiNpIhm0Dx9IBJ0VCJxQjvFl5EFhEuqZbgzB?= =?us-ascii?Q?HMTaEqkai4NcL6+KliT1k08js/uhKPlvNWoie1khyfa74yyF8o+hkaaWHOsp?= =?us-ascii?Q?BhAGSkGxDNU4dUZPSOt6Hm69LY0+e5kD3EdEMvJ1t2lvW9w5TabZ7F/H2zn2?= =?us-ascii?Q?zOR5zDWujUp/DE4dbjhiLrpGSb8tu0CiDr0p+OO1TB0YH/tDF5MtWomvJdDC?= =?us-ascii?Q?zQ6sgUx/roKyyMI8IX1/mRmQAiqfcfT0FXDrp7yqJRX0AAD9dFmwE/h/xDA4?= =?us-ascii?Q?ocjQAf3RA8HquNtKiz5dLRJm3lSS6y/0LZTH7uia2UxL6COttTgahGwtwIji?= =?us-ascii?Q?rmI3b5hkwZ1lnPs4xxAV3PKolgyfdh5v+ex/w6L9HSfzfs+7XTgPOpVJvH1q?= =?us-ascii?Q?eirwICaHOjwlJ1YtCjna8S16KOS2FXVtuNMZF+h/Mpr3uqJJYnqIa+sxj+z3?= =?us-ascii?Q?mi0qFdqRC2F7awUNWLZmrQCIU37nExaWFelhZd02z3pJFGsIlEFJhbK0s7qA?= =?us-ascii?Q?IO1JArHZ8SoGL3Dhizj1HlTUyZXeDTFXNBOCpSAklYUgy5hL/q7yK6Tvlx/h?= =?us-ascii?Q?OiV9+0cUMvnFehfDAIMvo+7UoTiU8Ej3Mj13GyT6hjmFCw8T100hP6CnfC5z?= =?us-ascii?Q?P4Oq1W/TosHWSYoFdTpXt/hYWshO0q1xibkd8GitD3fQq/+YQsEaT1Fj/cZD?= =?us-ascii?Q?YW8+29EHcg50/IKbHcEFtNgwnvndJMhgzlSFuLpHuRRXpzzKe4evueyYKCzM?= =?us-ascii?Q?yMKdXu5nCgv8ZAexZZvAhilEALdO6oHLfbpLgXKUtAeSmlgHoge7dRUyU3t5?= =?us-ascii?Q?TYkfFP72LfKIw3/NT30rKwXFdYbecwSrtmsJ0zblxLSJAuqIxdOl0u0xbacx?= =?us-ascii?Q?aw9D58znru3HUyWg4GeHn80R6KT8FonZNoci5kj/LBZsMmUV4Wk1WzE0E7zc?= =?us-ascii?Q?+IDOAMSRwD5ADnsrsmcD9fauo/zM?= X-Forefront-Antispam-Report: CIP:178.72.21.4; CTRY:DK; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:localhost.localdomain; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230031)(1800799015)(376005)(36860700004)(82310400017); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 9fhXxAF/hWItHRGZa9o8Igi+eSx+idIwM4s7rb/5XiVmef9amlgVkM2J5NK0uLYFQzImNxp08v4Pa88FfTxtvyVZer8/E3SDMdTUskBQPDNamA8qCchhYJjP7KFDA5Tq3fDG61GryyGA5Yrvp795emtEQLS801mnB19QX7LEjknecptmOMWdLQ9c9RM1AWBJrT86NfDL//07KgdiBtEYh80MI/U0QeFNsJ9bm/QYHjtcf+RWRVHwM9QxtKmlU6Qbk6cZtOsgUgEc1Qs9ABIcRugFYJ6nAkHhceBS1sfvjXlQF2tpGEHQcrFvjgX9KTjwBtfjQGFp39eJ7106rFSw3S7xkktq7VmVHxeXtnjC4Q01KzN12LfkXTDcY8v/8oC3mWeZ7mi7UQ5tgZfXoymmHkcSzuU2O53t+brdt1Dwz00x++wA/tvhm4Jz8ya03GfzqJTq68O/0fV67c2QoJNMpui7ISDvvhjecW0qJeJ4KEzM7KeXrUX7eOkFVI8ZOWRxXUujI7EMqXnWp3/bmkM8oxV/OqS2/Cy8MopzuHJZwMV4HrNFACjNUlEcZpDY+XCS2GWuftvrOH7AF3cWHhnSJjBgrUxTKEdd9IV09qBZaM8vcmCc0KAd5c2eggCehrdSlm5+LFgd+acTKXBxuXkb8w== X-OriginatorOrg: napatech.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 May 2024 15:47:38.3938 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c418d191-5fe9-4630-106f-08dc8188ffb0 X-MS-Exchange-CrossTenant-Id: c4540d0b-728a-4233-9da5-9ea30c7ec3ed X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=c4540d0b-728a-4233-9da5-9ea30c7ec3ed; Ip=[178.72.21.4]; Helo=[localhost.localdomain] X-MS-Exchange-CrossTenant-AuthSource: AMS1EPF00000047.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR3P190MB0860 X-BESS-ID: 1717170460-310740-12729-27671-2 X-BESS-VER: 2019.1_20240530.1612 X-BESS-Apparent-Source-IP: 104.47.12.50 X-BESS-Parts: H4sIAAAAAAACA4uuVkqtKFGyUioBkjpK+cVKVqamBhYmQGYGUDQxMSnFIM0k2d TE1CDRwNDY0DLFLNHS1DjJLNXCxMjYTKk2FgDqlV2WQgAAAA== X-BESS-Outbound-Spam-Score: 0.50 X-BESS-Outbound-Spam-Report: Code version 3.2, rules version 3.2.2.256627 [from cloudscan10-145.eu-central-1a.ess.aws.cudaops.com] Rule breakdown below pts rule name description ---- ---------------------- -------------------------------- 0.50 BSF_RULE7568M META: Custom Rule 7568M 0.00 BSF_BESS_OUTBOUND META: BESS Outbound X-BESS-Outbound-Spam-Status: SCORE=0.50 using account:ESS113687 scores of KILL_LEVEL=7.0 tests=BSF_RULE7568M, BSF_BESS_OUTBOUND X-BESS-BRTS-Status: 1 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add ntnic platform interfaces for FPGA registers Signed-off-by: Serhii Iliushyk --- drivers/net/ntnic/nthw/nthw_drv.h | 94 +++ drivers/net/ntnic/nthw/nthw_epp.c | 226 ++++++ drivers/net/ntnic/nthw/nthw_epp.h | 92 +++ drivers/net/ntnic/nthw/nthw_helper.h | 30 + drivers/net/ntnic/nthw/nthw_platform.c | 52 ++ drivers/net/ntnic/nthw/nthw_platform_drv.h | 49 ++ drivers/net/ntnic/nthw/nthw_profile.h | 16 + drivers/net/ntnic/nthw/nthw_rac.c | 801 +++++++++++++++++++++ drivers/net/ntnic/nthw/nthw_rac.h | 154 ++++ drivers/net/ntnic/nthw/nthw_register.h | 22 + drivers/net/ntnic/nthw/nthw_utils.c | 53 ++ drivers/net/ntnic/nthw/nthw_utils.h | 11 + 12 files changed, 1600 insertions(+) create mode 100644 drivers/net/ntnic/nthw/nthw_drv.h create mode 100644 drivers/net/ntnic/nthw/nthw_epp.c create mode 100644 drivers/net/ntnic/nthw/nthw_epp.h create mode 100644 drivers/net/ntnic/nthw/nthw_helper.h create mode 100644 drivers/net/ntnic/nthw/nthw_platform.c create mode 100644 drivers/net/ntnic/nthw/nthw_platform_drv.h create mode 100644 drivers/net/ntnic/nthw/nthw_profile.h create mode 100644 drivers/net/ntnic/nthw/nthw_rac.c create mode 100644 drivers/net/ntnic/nthw/nthw_rac.h create mode 100644 drivers/net/ntnic/nthw/nthw_register.h create mode 100644 drivers/net/ntnic/nthw/nthw_utils.c create mode 100644 drivers/net/ntnic/nthw/nthw_utils.h diff --git a/drivers/net/ntnic/nthw/nthw_drv.h b/drivers/net/ntnic/nthw/nthw_drv.h new file mode 100644 index 0000000000..e9e25bbc29 --- /dev/null +++ b/drivers/net/ntnic/nthw/nthw_drv.h @@ -0,0 +1,94 @@ +/* + * SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2023 Napatech A/S + */ + +#ifndef __NTHW_DRV_H__ +#define __NTHW_DRV_H__ + +#include "nthw_profile.h" + +typedef enum nt_meta_port_type_e { + PORT_TYPE_PHYSICAL, + PORT_TYPE_VIRTUAL, + PORT_TYPE_OVERRIDE, +} nt_meta_port_type_t; + +#include "nthw_helper.h" +#include "nthw_utils.h" +#include "nthw_platform_drv.h" +#include "nthw_fpga_model.h" +#include "ntnic_stat.h" +#include "ntnic_dbs.h" +#include "nthw_epp.h" +#include "nthw_core.h" + +typedef struct mcu_info_s { + bool mb_has_mcu; + int mn_mcu_type; + int mn_mcu_dram_size; +} mcu_info_t; + +typedef struct nthw_hw_info_s { + /* From FW */ + int hw_id; + int hw_id_emulated; + char hw_plat_id_str[32]; + + struct vpd_info_s { + int mn_mac_addr_count; + uint64_t mn_mac_addr_value; + uint8_t ma_mac_addr_octets[6]; + } vpd_info; +} nthw_hw_info_t; + +typedef struct fpga_info_s { + uint64_t n_fpga_ident; + + int n_fpga_type_id; + int n_fpga_prod_id; + int n_fpga_ver_id; + int n_fpga_rev_id; + + int n_fpga_build_time; + + int n_fpga_debug_mode; + + int n_nims; + int n_phy_ports; + int n_phy_quads; + int n_rx_ports; + int n_tx_ports; + int n_vf_offset; + + enum fpga_info_profile profile; + + struct nthw_fpga_s *mp_fpga; + + struct nthw_rac *mp_nthw_rac; + struct nthw_hif *mp_nthw_hif; + struct nthw_pcie3 *mp_nthw_pcie3; + struct nthw_tsm *mp_nthw_tsm; + + nthw_dbs_t *mp_nthw_dbs; + nthw_epp_t *mp_nthw_epp; + + uint8_t *bar0_addr; /* Needed for register read/write */ + size_t bar0_size; + + int adapter_no; /* Needed for nthw_rac DMA array indexing */ + uint32_t pciident; /* Needed for nthw_rac DMA memzone_reserve */ + int numa_node; /* Needed for nthw_rac DMA memzone_reserve */ + + char *mp_adapter_id_str;/* Pointer to string literal used in nthw log messages */ + + struct mcu_info_s mcu_info; + + struct nthw_hw_info_s nthw_hw_info; + + nthw_adapter_id_t n_nthw_adapter_id; + +} fpga_info_t; + + +#endif /* __NTHW_DRV_H__ */ diff --git a/drivers/net/ntnic/nthw/nthw_epp.c b/drivers/net/ntnic/nthw/nthw_epp.c new file mode 100644 index 0000000000..fe1c562394 --- /dev/null +++ b/drivers/net/ntnic/nthw/nthw_epp.c @@ -0,0 +1,226 @@ +/* + * SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2023 Napatech A/S + */ + +#include "ntlog.h" + +#include "nthw_drv.h" +#include "nthw_register.h" + +#include "nthw_epp.h" + +#include /* ENOTSUP */ + +nthw_epp_t *nthw_epp_new(void) +{ + nthw_epp_t *p = malloc(sizeof(nthw_epp_t)); + + if (p) + memset(p, 0, sizeof(nthw_epp_t)); + + return p; +} + +int nthw_epp_present(nthw_fpga_t *p_fpga, int n_instance) +{ + return nthw_epp_init(NULL, p_fpga, n_instance) == 0; +} + +int nthw_epp_init(nthw_epp_t *p, nthw_fpga_t *p_fpga, int n_instance) +{ + nthw_module_t *mod = nthw_fpga_query_module(p_fpga, MOD_EPP, n_instance); + + if (p == NULL) + return mod == NULL ? -1 : 0; + + if (mod == NULL) { + NT_LOG(ERR, NTHW, "%s: EPP %d: no such instance\n", + p_fpga->p_fpga_info->mp_adapter_id_str, n_instance); + return -1; + } + + p->mp_fpga = p_fpga; + p->mn_instance = n_instance; + p->mp_mod_epp = mod; + + p->mn_epp_categories = nthw_fpga_get_product_param(p_fpga, NT_EPP_CATEGORIES, 0); + + p->mp_reg_reciepe_memory_control = nthw_module_get_register(p->mp_mod_epp, EPP_RCP_CTRL); + p->mp_fld_reciepe_memory_control_adr = + nthw_register_get_field(p->mp_reg_reciepe_memory_control, EPP_RCP_CTRL_ADR); + p->mp_fld_reciepe_memory_control_cnt = + nthw_register_get_field(p->mp_reg_reciepe_memory_control, EPP_RCP_CTRL_CNT); + + p->mp_reg_reciepe_memory_data = nthw_module_get_register(p->mp_mod_epp, EPP_RCP_DATA); + p->mp_fld_reciepe_memory_data_tx_mtu_epp_enable = + nthw_register_get_field(p->mp_reg_reciepe_memory_data, EPP_RCP_DATA_TX_MTU_EPP_EN); + p->mp_fld_reciepe_memory_data_queue_mtu_epp_enable = + nthw_register_get_field(p->mp_reg_reciepe_memory_data, + EPP_RCP_DATA_QUEUE_MTU_EPP_EN); + p->mp_fld_reciepe_memory_data_size_adjust_tx_port = + nthw_register_get_field(p->mp_reg_reciepe_memory_data, + EPP_RCP_DATA_SIZE_ADJUST_TXP); + p->mp_fld_reciepe_memory_data_size_adjust_virtual_port = + nthw_register_get_field(p->mp_reg_reciepe_memory_data, + EPP_RCP_DATA_SIZE_ADJUST_VPORT); + p->mp_fld_reciepe_memory_data_fixed18b_l2_mtu = + nthw_register_get_field(p->mp_reg_reciepe_memory_data, + EPP_RCP_DATA_FIXED_18B_L2_MTU); + p->mp_fld_reciepe_memory_data_txp_qos_epp_enable = + nthw_register_get_field(p->mp_reg_reciepe_memory_data, EPP_RCP_DATA_TX_QOS_EPP_EN); + p->mp_fld_reciepe_memory_data_queue_qos_epp_enable = + nthw_register_get_field(p->mp_reg_reciepe_memory_data, + EPP_RCP_DATA_QUEUE_QOS_EPP_EN); + + p->mp_reg_txp_port_mtu_control = nthw_module_get_register(p->mp_mod_epp, EPP_TXP_MTU_CTRL); + p->mp_fld_txp_port_mtu_control_adr = + nthw_register_get_field(p->mp_reg_txp_port_mtu_control, EPP_TXP_MTU_CTRL_ADR); + p->mp_fld_txp_port_mtu_control_cnt = + nthw_register_get_field(p->mp_reg_txp_port_mtu_control, EPP_TXP_MTU_CTRL_CNT); + + p->mp_reg_txp_port_mtu_data = nthw_module_get_register(p->mp_mod_epp, EPP_TXP_MTU_DATA); + p->mp_fld_txp_port_mtu_data_max_mtu = + nthw_register_get_field(p->mp_reg_txp_port_mtu_data, EPP_TXP_MTU_DATA_MAX_MTU); + + p->mp_reg_queue_mtu_control = nthw_module_get_register(p->mp_mod_epp, EPP_QUEUE_MTU_CTRL); + p->mp_fld_queue_mtu_control_adr = + nthw_register_get_field(p->mp_reg_queue_mtu_control, EPP_QUEUE_MTU_CTRL_ADR); + p->mp_fld_queue_mtu_control_cnt = + nthw_register_get_field(p->mp_reg_queue_mtu_control, EPP_QUEUE_MTU_CTRL_CNT); + + p->mp_reg_queue_mtu_data = nthw_module_get_register(p->mp_mod_epp, EPP_QUEUE_MTU_DATA); + p->mp_fld_queue_mtu_data_max_mtu = + nthw_register_get_field(p->mp_reg_queue_mtu_data, EPP_QUEUE_MTU_DATA_MAX_MTU); + + p->mp_reg_txp_qos_control = nthw_module_get_register(p->mp_mod_epp, EPP_TXP_QOS_CTRL); + p->mp_fld_txp_qos_control_adr = + nthw_register_get_field(p->mp_reg_txp_qos_control, EPP_TXP_QOS_CTRL_ADR); + p->mp_fld_txp_qos_control_cnt = + nthw_register_get_field(p->mp_reg_txp_qos_control, EPP_TXP_QOS_CTRL_CNT); + + p->mp_reg_txp_qos_data = nthw_module_get_register(p->mp_mod_epp, EPP_TXP_QOS_DATA); + p->mp_fld_txp_qos_data_enable = + nthw_register_get_field(p->mp_reg_txp_qos_data, EPP_TXP_QOS_DATA_EN); + p->mp_fld_txp_qos_data_information_rate = + nthw_register_get_field(p->mp_reg_txp_qos_data, EPP_TXP_QOS_DATA_IR); + p->mp_fld_txp_qos_data_information_rate_fractional = + nthw_register_get_field(p->mp_reg_txp_qos_data, EPP_TXP_QOS_DATA_IR_FRACTION); + p->mp_fld_txp_qos_data_burst_size = + nthw_register_get_field(p->mp_reg_txp_qos_data, EPP_TXP_QOS_DATA_BS); + + p->mp_reg_vport_qos_control = nthw_module_get_register(p->mp_mod_epp, EPP_VPORT_QOS_CTRL); + p->mp_fld_vport_qos_control_adr = + nthw_register_get_field(p->mp_reg_vport_qos_control, EPP_VPORT_QOS_CTRL_ADR); + p->mp_fld_vport_qos_control_cnt = + nthw_register_get_field(p->mp_reg_vport_qos_control, EPP_VPORT_QOS_CTRL_CNT); + + p->mp_reg_vport_qos_data = nthw_module_get_register(p->mp_mod_epp, EPP_VPORT_QOS_DATA); + p->mp_fld_vport_qos_data_enable = + nthw_register_get_field(p->mp_reg_vport_qos_data, EPP_VPORT_QOS_DATA_EN); + p->mp_fld_vport_qos_data_information_rate = + nthw_register_get_field(p->mp_reg_vport_qos_data, EPP_VPORT_QOS_DATA_IR); + p->mp_fld_vport_qos_data_information_rate_fractional = + nthw_register_get_field(p->mp_reg_vport_qos_data, EPP_VPORT_QOS_DATA_IR_FRACTION); + p->mp_fld_vport_qos_data_burst_size = + nthw_register_get_field(p->mp_reg_vport_qos_data, EPP_VPORT_QOS_DATA_BS); + + p->mp_reg_queue_vport_control = + nthw_module_get_register(p->mp_mod_epp, EPP_QUEUE_VPORT_CTRL); + p->mp_fld_queue_vport_control_adr = + nthw_register_get_field(p->mp_reg_queue_vport_control, EPP_QUEUE_VPORT_CTRL_ADR); + p->mp_fld_queue_vport_control_cnt = + nthw_register_get_field(p->mp_reg_queue_vport_control, EPP_QUEUE_VPORT_CTRL_CNT); + + p->mp_reg_queue_vport_data = nthw_module_get_register(p->mp_mod_epp, EPP_QUEUE_VPORT_DATA); + p->mp_fld_queue_vport_data_vport = + nthw_register_get_field(p->mp_reg_queue_vport_data, EPP_QUEUE_VPORT_DATA_VPORT); + + return 0; +} + +int nthw_epp_setup(nthw_epp_t *p) +{ + if (p == NULL) + return 0; + + /* Set recieps for 2 first records */ + nthw_field_set_val32(p->mp_fld_reciepe_memory_control_cnt, 1); + + /* Zero all categories */ + for (int i = 0; i < p->mn_epp_categories; ++i) { + nthw_field_set_val32(p->mp_fld_reciepe_memory_control_adr, i); + nthw_register_flush(p->mp_reg_reciepe_memory_control, 1); + + nthw_field_set_val32(p->mp_fld_reciepe_memory_data_tx_mtu_epp_enable, 0); + nthw_field_set_val32(p->mp_fld_reciepe_memory_data_queue_mtu_epp_enable, 0); + nthw_field_set_val32(p->mp_fld_reciepe_memory_data_size_adjust_tx_port, 0); + nthw_field_set_val32(p->mp_fld_reciepe_memory_data_size_adjust_virtual_port, 0); + nthw_field_set_val32(p->mp_fld_reciepe_memory_data_fixed18b_l2_mtu, 0); + nthw_field_set_val32(p->mp_fld_reciepe_memory_data_txp_qos_epp_enable, 0); + nthw_field_set_val32(p->mp_fld_reciepe_memory_data_queue_qos_epp_enable, 0); + nthw_register_flush(p->mp_reg_reciepe_memory_data, 1); + } + + for (int i = 0; i < NRECIPE; ++i) { + nthw_field_set_val32(p->mp_fld_reciepe_memory_control_adr, i); + nthw_register_flush(p->mp_reg_reciepe_memory_control, 1); + + nthw_field_set_val32(p->mp_fld_reciepe_memory_data_tx_mtu_epp_enable, 1); + nthw_field_set_val32(p->mp_fld_reciepe_memory_data_queue_mtu_epp_enable, 1); + nthw_field_set_val32(p->mp_fld_reciepe_memory_data_size_adjust_tx_port, + rcp_data_size_adjust_txp[i]); + nthw_field_set_val32(p->mp_fld_reciepe_memory_data_size_adjust_virtual_port, + rcp_data_size_adjust_vport[i]); + nthw_field_set_val32(p->mp_fld_reciepe_memory_data_fixed18b_l2_mtu, 1); + nthw_field_set_val32(p->mp_fld_reciepe_memory_data_txp_qos_epp_enable, 1); + nthw_field_set_val32(p->mp_fld_reciepe_memory_data_queue_qos_epp_enable, 1); + nthw_register_flush(p->mp_reg_reciepe_memory_data, 1); + } + + /* phy mtu setup */ + nthw_field_set_val32(p->mp_fld_txp_port_mtu_control_cnt, 1); + + for (int i = 0; i < 2; ++i) { + nthw_field_set_val32(p->mp_fld_txp_port_mtu_control_adr, i); + nthw_register_flush(p->mp_reg_txp_port_mtu_control, 1); + + nthw_field_set_val32(p->mp_fld_txp_port_mtu_data_max_mtu, MTUINITVAL); + nthw_register_flush(p->mp_reg_txp_port_mtu_data, 1); + } + + /* phy QoS setup */ + nthw_field_set_val32(p->mp_fld_txp_qos_control_cnt, 1); + + for (int i = 0; i < 2; ++i) { + nthw_field_set_val32(p->mp_fld_txp_qos_control_adr, i); + nthw_register_flush(p->mp_reg_txp_qos_control, 1); + + nthw_field_set_val32(p->mp_fld_txp_qos_data_enable, 0); + nthw_register_flush(p->mp_reg_txp_qos_data, 1); + } + + /* virt mtu setup */ + nthw_field_set_val32(p->mp_fld_queue_mtu_control_cnt, 1); + + for (int i = 0; i < 128; ++i) { + nthw_field_set_val32(p->mp_fld_queue_mtu_control_adr, i); + nthw_register_flush(p->mp_reg_queue_mtu_control, 1); + + nthw_field_set_val32(p->mp_fld_queue_mtu_data_max_mtu, MTUINITVAL); + nthw_register_flush(p->mp_reg_queue_mtu_data, 1); + } + + /* virt QoS setup */ + nthw_field_set_val32(p->mp_fld_vport_qos_control_cnt, 1); + + for (int i = 0; i < 128; ++i) { + nthw_field_set_val32(p->mp_fld_vport_qos_control_adr, i); + nthw_register_flush(p->mp_reg_vport_qos_control, 1); + + nthw_field_set_val32(p->mp_fld_vport_qos_data_enable, 0); + nthw_register_flush(p->mp_reg_vport_qos_data, 1); + } + + return 0; +} diff --git a/drivers/net/ntnic/nthw/nthw_epp.h b/drivers/net/ntnic/nthw/nthw_epp.h new file mode 100644 index 0000000000..413a812273 --- /dev/null +++ b/drivers/net/ntnic/nthw/nthw_epp.h @@ -0,0 +1,92 @@ +/* + * SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2023 Napatech A/S + */ + +#ifndef NTHW_EPP_HPP_ +#define NTHW_EPP_HPP_ + +/* VXLAN adds extra 50 bytes */ +#define VXLANDATASIZEADJUST 50 +#define VXLANDATASIZEADJUSTIPV6 70 +#define MTUINITVAL 1500 +#define NRECIPE 3 + +/* List of size adjust values to put in the recipe memory data register at startup */ +static const int rcp_data_size_adjust_txp[NRECIPE] = { 0, VXLANDATASIZEADJUST, + VXLANDATASIZEADJUSTIPV6 + }; +static const int rcp_data_size_adjust_vport[NRECIPE] = { 0, VXLANDATASIZEADJUST, + VXLANDATASIZEADJUSTIPV6 + }; + +struct nthw_epp_s { + nthw_fpga_t *mp_fpga; + nthw_module_t *mp_mod_epp; + int mn_instance; + int mn_epp_categories; + + nthw_register_t *mp_reg_reciepe_memory_control; + nthw_field_t *mp_fld_reciepe_memory_control_adr; + nthw_field_t *mp_fld_reciepe_memory_control_cnt; + + nthw_register_t *mp_reg_reciepe_memory_data; + nthw_field_t *mp_fld_reciepe_memory_data_tx_mtu_epp_enable; + nthw_field_t *mp_fld_reciepe_memory_data_queue_mtu_epp_enable; + nthw_field_t *mp_fld_reciepe_memory_data_size_adjust_tx_port; + nthw_field_t *mp_fld_reciepe_memory_data_size_adjust_virtual_port; + nthw_field_t *mp_fld_reciepe_memory_data_fixed18b_l2_mtu; + nthw_field_t *mp_fld_reciepe_memory_data_txp_qos_epp_enable; + nthw_field_t *mp_fld_reciepe_memory_data_queue_qos_epp_enable; + + nthw_register_t *mp_reg_txp_port_mtu_control; + nthw_field_t *mp_fld_txp_port_mtu_control_adr; + nthw_field_t *mp_fld_txp_port_mtu_control_cnt; + + nthw_register_t *mp_reg_txp_port_mtu_data; + nthw_field_t *mp_fld_txp_port_mtu_data_max_mtu; + + nthw_register_t *mp_reg_queue_mtu_control; + nthw_field_t *mp_fld_queue_mtu_control_adr; + nthw_field_t *mp_fld_queue_mtu_control_cnt; + + nthw_register_t *mp_reg_queue_mtu_data; + nthw_field_t *mp_fld_queue_mtu_data_max_mtu; + + nthw_register_t *mp_reg_txp_qos_control; + nthw_field_t *mp_fld_txp_qos_control_adr; + nthw_field_t *mp_fld_txp_qos_control_cnt; + + nthw_register_t *mp_reg_txp_qos_data; + nthw_field_t *mp_fld_txp_qos_data_enable; + nthw_field_t *mp_fld_txp_qos_data_information_rate; + nthw_field_t *mp_fld_txp_qos_data_information_rate_fractional; + nthw_field_t *mp_fld_txp_qos_data_burst_size; + + nthw_register_t *mp_reg_vport_qos_control; + nthw_field_t *mp_fld_vport_qos_control_adr; + nthw_field_t *mp_fld_vport_qos_control_cnt; + + nthw_register_t *mp_reg_vport_qos_data; + nthw_field_t *mp_fld_vport_qos_data_enable; + nthw_field_t *mp_fld_vport_qos_data_information_rate; + nthw_field_t *mp_fld_vport_qos_data_information_rate_fractional; + nthw_field_t *mp_fld_vport_qos_data_burst_size; + + nthw_register_t *mp_reg_queue_vport_control; + nthw_field_t *mp_fld_queue_vport_control_adr; + nthw_field_t *mp_fld_queue_vport_control_cnt; + + nthw_register_t *mp_reg_queue_vport_data; + nthw_field_t *mp_fld_queue_vport_data_vport; +}; + +typedef struct nthw_epp_s nthw_epp_t; + +nthw_epp_t *nthw_epp_new(void); + +int nthw_epp_present(nthw_fpga_t *p_fpga, int n_instance); +int nthw_epp_init(nthw_epp_t *p, nthw_fpga_t *p_fpga, int n_instance); +int nthw_epp_setup(nthw_epp_t *p); + +#endif /* NTHW_EPP_HPP_ */ diff --git a/drivers/net/ntnic/nthw/nthw_helper.h b/drivers/net/ntnic/nthw/nthw_helper.h new file mode 100644 index 0000000000..d1bd5cec79 --- /dev/null +++ b/drivers/net/ntnic/nthw/nthw_helper.h @@ -0,0 +1,30 @@ +/* + * SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2023 Napatech A/S + */ + +#ifndef __NTHW_HELPER_H__ +#define __NTHW_HELPER_H__ + +#include +#include +#include +#include +#include +#include +#include +#include + +#ifndef PRIXPTR +#define PRIXPTR "llX" +#endif + +#ifndef UINT8_MAX +#define UINT8_MAX (U8_MAX) +#endif + +#ifndef ARRAY_SIZE +#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0])) +#endif + +#endif /* __NTHW_HELPER_H__ */ diff --git a/drivers/net/ntnic/nthw/nthw_platform.c b/drivers/net/ntnic/nthw/nthw_platform.c new file mode 100644 index 0000000000..510841f2af --- /dev/null +++ b/drivers/net/ntnic/nthw/nthw_platform.c @@ -0,0 +1,52 @@ +/* + * SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2023 Napatech A/S + */ + +#include "nthw_platform_drv.h" + +nthw_adapter_id_t nthw_platform_get_nthw_adapter_id(const uint16_t n_pci_device_id) +{ + switch (n_pci_device_id) { + case NT_HW_PCI_DEVICE_ID_NT40E3: + return NT_HW_ADAPTER_ID_NT40E3; + + case NT_HW_PCI_DEVICE_ID_NT100E3: + return NT_HW_ADAPTER_ID_NT100E3; + + case NT_HW_PCI_DEVICE_ID_NT80E3: + return NT_HW_ADAPTER_ID_NT80E3; + + case NT_HW_PCI_DEVICE_ID_NT40A00: + return NT_HW_ADAPTER_ID_NT40E3; + + case NT_HW_PCI_DEVICE_ID_NT40A01: + return NT_HW_ADAPTER_ID_NT40E3; + + case NT_HW_PCI_DEVICE_ID_NT200E3: + return NT_HW_ADAPTER_ID_NT200E3; + + case NT_HW_PCI_DEVICE_ID_NT200A01: + return NT_HW_ADAPTER_ID_NT200A01; + + case NT_HW_PCI_DEVICE_ID_NT200D01: + return NT_HW_ADAPTER_ID_NT200D01; + + case NT_HW_PCI_DEVICE_ID_NT200A02_LENOVO: + case NT_HW_PCI_DEVICE_ID_NT200A02: + return NT_HW_ADAPTER_ID_NT200A02; + + case NT_HW_PCI_DEVICE_ID_NT50B01_LENOVO: + case NT_HW_PCI_DEVICE_ID_NT50B01: + return NT_HW_ADAPTER_ID_NT50B01; + + case NT_HW_PCI_DEVICE_ID_NT100A01: + return NT_HW_ADAPTER_ID_NT100A01; + + case NT_HW_PCI_DEVICE_ID_NT400D11: + return NT_HW_ADAPTER_ID_NT400D11; + + default: + return NT_HW_ADAPTER_ID_UNKNOWN; + } +} diff --git a/drivers/net/ntnic/nthw/nthw_platform_drv.h b/drivers/net/ntnic/nthw/nthw_platform_drv.h new file mode 100644 index 0000000000..96245ffb3e --- /dev/null +++ b/drivers/net/ntnic/nthw/nthw_platform_drv.h @@ -0,0 +1,49 @@ +/* + * SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2023 Napatech A/S + */ + +#ifndef __NTHW_PLATFORM_DRV_H__ +#define __NTHW_PLATFORM_DRV_H__ + +#include "nthw_helper.h" + +#define NT_HW_PCI_VENDOR_ID (0x18f4) +#define NT_HW_PCI_VENDOR_ID_LENOVO (0x17aa) + +#define NT_HW_PCI_DEVICE_ID_NT40E3 (0x145) +#define NT_HW_PCI_DEVICE_ID_NT100E3 (0x155) +#define NT_HW_PCI_DEVICE_ID_NT80E3 (0x165) +#define NT_HW_PCI_DEVICE_ID_NT40A00 (0x175) /* ehrmmm bummer */ +#define NT_HW_PCI_DEVICE_ID_NT40A01 (0x185) +#define NT_HW_PCI_DEVICE_ID_NT200E3 (0x195) +#define NT_HW_PCI_DEVICE_ID_NT200A01 (0x1A5) +#define NT_HW_PCI_DEVICE_ID_NT200D01 (0x1B5) +#define NT_HW_PCI_DEVICE_ID_NT200A02 (0x1C5) +#define NT_HW_PCI_DEVICE_ID_NT50B01 (0x1D5) +#define NT_HW_PCI_DEVICE_ID_NT100A01 (0x1E5) +#define NT_HW_PCI_DEVICE_ID_NT400D11 (0x215) + +#define NT_HW_PCI_DEVICE_ID_NT200A02_LENOVO (0x05a1) +#define NT_HW_PCI_DEVICE_ID_NT50B01_LENOVO (0x05b1) + +enum nthw_adapter_id_e { + NT_HW_ADAPTER_ID_UNKNOWN = 0, + NT_HW_ADAPTER_ID_NT40E3, + NT_HW_ADAPTER_ID_NT40A01 = NT_HW_ADAPTER_ID_NT40E3, + NT_HW_ADAPTER_ID_NT50B01, + NT_HW_ADAPTER_ID_NT80E3, + NT_HW_ADAPTER_ID_NT100E3, + NT_HW_ADAPTER_ID_NT100A01, + NT_HW_ADAPTER_ID_NT200E3, + NT_HW_ADAPTER_ID_NT200A01, + NT_HW_ADAPTER_ID_NT200D01, + NT_HW_ADAPTER_ID_NT200A02, + NT_HW_ADAPTER_ID_NT400D11, +}; + +typedef enum nthw_adapter_id_e nthw_adapter_id_t; + +nthw_adapter_id_t nthw_platform_get_nthw_adapter_id(const uint16_t n_pci_device_id); + +#endif /* __NTHW_PLATFORM_DRV_H__ */ diff --git a/drivers/net/ntnic/nthw/nthw_profile.h b/drivers/net/ntnic/nthw/nthw_profile.h new file mode 100644 index 0000000000..d7ac30b20d --- /dev/null +++ b/drivers/net/ntnic/nthw/nthw_profile.h @@ -0,0 +1,16 @@ +/* + * SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2023 Napatech A/S + */ + +#ifndef __NTHW_PROFILE_H__ +#define __NTHW_PROFILE_H__ + +enum fpga_info_profile { + FPGA_INFO_PROFILE_UNKNOWN = 0, + FPGA_INFO_PROFILE_VSWITCH = 1, + FPGA_INFO_PROFILE_INLINE = 2, + FPGA_INFO_PROFILE_CAPTURE = 3, +}; + +#endif /* __NTHW_PROFILE_H__ */ diff --git a/drivers/net/ntnic/nthw/nthw_rac.c b/drivers/net/ntnic/nthw/nthw_rac.c new file mode 100644 index 0000000000..e0a8847d08 --- /dev/null +++ b/drivers/net/ntnic/nthw/nthw_rac.c @@ -0,0 +1,801 @@ +/* + * SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2023 Napatech A/S + */ + +#include "nt_util.h" +#include "ntlog.h" + +#include "nthw_drv.h" +#include "nthw_register.h" +#include "nthw_rac.h" + +#include + +/* + * Prevent that RAB echo debug trace ever gets into a release build + */ +#if defined(DEBUG) +#undef RAB_DEBUG_ECHO +#else +#undef RAB_DEBUG_ECHO +#endif /* DEBUG */ + +#define RAB_DMA_WAIT (1000000) + +#define RAB_READ (0x01) +#define RAB_WRITE (0x02) +#define RAB_ECHO (0x08) +#define RAB_COMPLETION (0x0F) + +#define RAB_OPR_LO (28) +#define RAB_OPR_HI (31) +#define RAB_OPR_BW (4) + +#define RAB_CNT_LO (20) +#define RAB_CNT_HI (27) +#define RAB_CNT_BW (8) + +#define RAB_BUSID_LO (16) +#define RAB_BUSID_HI (19) +#define RAB_BUSID_BW (4) + +#define RAB_ADDR_LO (0) +#define RAB_ADDR_HI (15) +#define RAB_ADDR_BW (16) + +nthw_rac_t *nthw_rac_new(void) +{ + nthw_rac_t *p = malloc(sizeof(nthw_rac_t)); + memset(p, 0, sizeof(nthw_rac_t)); + return p; +} + +int nthw_rac_init(nthw_rac_t *p, nthw_fpga_t *p_fpga, struct fpga_info_s *p_fpga_info) +{ + assert(p_fpga_info); + + const char *const p_adapter_id_str = p_fpga_info->mp_adapter_id_str; + nthw_module_t *p_mod = nthw_fpga_query_module(p_fpga, MOD_RAC, 0); + + if (p == NULL) + return p_mod == NULL ? -1 : 0; + + if (p_mod == NULL) { + NT_LOG(ERR, NTHW, "%s: RAC %d: no such instance\n", p_adapter_id_str, 0); + return -1; + } + + p->mp_fpga = p_fpga; + p->mp_mod_rac = p_mod; + + /* Params */ + p->mn_param_rac_rab_interfaces = + nthw_fpga_get_product_param(p->mp_fpga, NT_RAC_RAB_INTERFACES, 3); + NT_LOG(DBG, NTHW, "%s: NT_RAC_RAB_INTERFACES=%d\n", p_adapter_id_str, + p->mn_param_rac_rab_interfaces); + + p->mn_param_rac_rab_ob_update = + nthw_fpga_get_product_param(p->mp_fpga, NT_RAC_RAB_OB_UPDATE, 0); + NT_LOG(DBG, NTHW, "%s: NT_RAC_RAB_OB_UPDATE=%d\n", p_adapter_id_str, + p->mn_param_rac_rab_ob_update); + + /* Optional dummy test registers */ + p->mp_reg_dummy0 = nthw_module_query_register(p->mp_mod_rac, RAC_DUMMY0); + p->mp_reg_dummy1 = nthw_module_query_register(p->mp_mod_rac, RAC_DUMMY1); + p->mp_reg_dummy2 = nthw_module_query_register(p->mp_mod_rac, RAC_DUMMY2); + + p->mp_reg_rab_init = nthw_module_get_register(p->mp_mod_rac, RAC_RAB_INIT); + p->mp_fld_rab_init = nthw_register_get_field(p->mp_reg_rab_init, RAC_RAB_INIT_RAB); + p->mn_fld_rab_init_bw = nthw_field_get_bit_width(p->mp_fld_rab_init); + p->mn_fld_rab_init_mask = nthw_field_get_mask(p->mp_fld_rab_init); + + /* RAC_RAB_INIT_RAB reg/field sanity checks: */ + assert(p->mn_fld_rab_init_mask == ((1UL << p->mn_fld_rab_init_bw) - 1)); + assert(p->mn_fld_rab_init_bw == p->mn_param_rac_rab_interfaces); + + p->mp_reg_dbg_ctrl = nthw_module_query_register(p->mp_mod_rac, RAC_DBG_CTRL); + + if (p->mp_reg_dbg_ctrl) + p->mp_fld_dbg_ctrl = nthw_register_query_field(p->mp_reg_dbg_ctrl, RAC_DBG_CTRL_C); + + else + p->mp_fld_dbg_ctrl = NULL; + + p->mp_reg_dbg_data = nthw_module_query_register(p->mp_mod_rac, RAC_DBG_DATA); + + if (p->mp_reg_dbg_data) + p->mp_fld_dbg_data = nthw_register_query_field(p->mp_reg_dbg_data, RAC_DBG_DATA_D); + + else + p->mp_reg_dbg_data = NULL; + + p->mp_reg_rab_ib_data = nthw_module_get_register(p->mp_mod_rac, RAC_RAB_IB_DATA); + p->mp_fld_rab_ib_data = nthw_register_get_field(p->mp_reg_rab_ib_data, RAC_RAB_IB_DATA_D); + + p->mp_reg_rab_ob_data = nthw_module_get_register(p->mp_mod_rac, RAC_RAB_OB_DATA); + p->mp_fld_rab_ob_data = nthw_register_get_field(p->mp_reg_rab_ob_data, RAC_RAB_OB_DATA_D); + + p->mp_reg_rab_buf_free = nthw_module_get_register(p->mp_mod_rac, RAC_RAB_BUF_FREE); + p->mp_fld_rab_buf_free_ib_free = + nthw_register_get_field(p->mp_reg_rab_buf_free, RAC_RAB_BUF_FREE_IB_FREE); + p->mp_fld_rab_buf_free_ib_ovf = + nthw_register_get_field(p->mp_reg_rab_buf_free, RAC_RAB_BUF_FREE_IB_OVF); + p->mp_fld_rab_buf_free_ob_free = + nthw_register_get_field(p->mp_reg_rab_buf_free, RAC_RAB_BUF_FREE_OB_FREE); + p->mp_fld_rab_buf_free_ob_ovf = + nthw_register_get_field(p->mp_reg_rab_buf_free, RAC_RAB_BUF_FREE_OB_OVF); + p->mp_fld_rab_buf_free_timeout = + nthw_register_get_field(p->mp_reg_rab_buf_free, RAC_RAB_BUF_FREE_TIMEOUT); + + p->mp_reg_rab_buf_used = nthw_module_get_register(p->mp_mod_rac, RAC_RAB_BUF_USED); + p->mp_fld_rab_buf_used_ib_used = + nthw_register_get_field(p->mp_reg_rab_buf_used, RAC_RAB_BUF_USED_IB_USED); + p->mp_fld_rab_buf_used_ob_used = + nthw_register_get_field(p->mp_reg_rab_buf_used, RAC_RAB_BUF_USED_OB_USED); + p->mp_fld_rab_buf_used_flush = + nthw_register_get_field(p->mp_reg_rab_buf_used, RAC_RAB_BUF_USED_FLUSH); + + /* + * RAC_RAB_DMA regs are optional - only found in real + * NT4GA - not found in 9231/9232 and earlier + */ + p->mp_reg_rab_dma_ib_lo = nthw_module_get_register(p->mp_mod_rac, RAC_RAB_DMA_IB_LO); + p->mp_fld_rab_dma_ib_lo_phy_addr = + nthw_register_get_field(p->mp_reg_rab_dma_ib_lo, RAC_RAB_DMA_IB_LO_PHYADDR); + + p->mp_reg_rab_dma_ib_hi = nthw_module_get_register(p->mp_mod_rac, RAC_RAB_DMA_IB_HI); + p->mp_fld_rab_dma_ib_hi_phy_addr = + nthw_register_get_field(p->mp_reg_rab_dma_ib_hi, RAC_RAB_DMA_IB_HI_PHYADDR); + + p->mp_reg_rab_dma_ob_lo = nthw_module_get_register(p->mp_mod_rac, RAC_RAB_DMA_OB_LO); + p->mp_fld_rab_dma_ob_lo_phy_addr = + nthw_register_get_field(p->mp_reg_rab_dma_ob_lo, RAC_RAB_DMA_OB_LO_PHYADDR); + + p->mp_reg_rab_dma_ob_hi = nthw_module_get_register(p->mp_mod_rac, RAC_RAB_DMA_OB_HI); + p->mp_fld_rab_dma_ob_hi_phy_addr = + nthw_register_get_field(p->mp_reg_rab_dma_ob_hi, RAC_RAB_DMA_OB_HI_PHYADDR); + + p->mp_reg_rab_dma_ib_wr = nthw_module_get_register(p->mp_mod_rac, RAC_RAB_DMA_IB_WR); + p->mp_fld_rab_dma_ib_wr_ptr = + nthw_register_get_field(p->mp_reg_rab_dma_ib_wr, RAC_RAB_DMA_IB_WR_PTR); + + p->mp_reg_rab_dma_ib_rd = nthw_module_get_register(p->mp_mod_rac, RAC_RAB_DMA_IB_RD); + p->mp_fld_rab_dma_ib_rd_ptr = + nthw_register_get_field(p->mp_reg_rab_dma_ib_rd, RAC_RAB_DMA_IB_RD_PTR); + + p->mp_reg_rab_dma_ob_wr = nthw_module_get_register(p->mp_mod_rac, RAC_RAB_DMA_OB_WR); + p->mp_fld_rab_dma_ob_wr_ptr = + nthw_register_get_field(p->mp_reg_rab_dma_ob_wr, RAC_RAB_DMA_OB_WR_PTR); + + p->RAC_RAB_INIT_ADDR = nthw_register_get_address(p->mp_reg_rab_init); + p->RAC_RAB_IB_DATA_ADDR = nthw_register_get_address(p->mp_reg_rab_ib_data); + p->RAC_RAB_OB_DATA_ADDR = nthw_register_get_address(p->mp_reg_rab_ob_data); + p->RAC_RAB_BUF_FREE_ADDR = nthw_register_get_address(p->mp_reg_rab_buf_free); + p->RAC_RAB_BUF_USED_ADDR = nthw_register_get_address(p->mp_reg_rab_buf_used); + + /* + * RAC_RAB_DMA regs are optional - only found in real NT4GA - not found in 9231/9232 and + * earlier + */ + + p->RAC_RAB_DMA_IB_LO_ADDR = nthw_register_get_address(p->mp_reg_rab_dma_ib_lo); + p->RAC_RAB_DMA_IB_HI_ADDR = nthw_register_get_address(p->mp_reg_rab_dma_ib_hi); + p->RAC_RAB_DMA_OB_LO_ADDR = nthw_register_get_address(p->mp_reg_rab_dma_ob_lo); + p->RAC_RAB_DMA_OB_HI_ADDR = nthw_register_get_address(p->mp_reg_rab_dma_ob_hi); + p->RAC_RAB_DMA_IB_RD_ADDR = nthw_register_get_address(p->mp_reg_rab_dma_ib_rd); + p->RAC_RAB_DMA_OB_WR_ADDR = nthw_register_get_address(p->mp_reg_rab_dma_ob_wr); + p->RAC_RAB_DMA_IB_WR_ADDR = nthw_register_get_address(p->mp_reg_rab_dma_ib_wr); + + p->RAC_RAB_BUF_FREE_IB_FREE_MASK = nthw_field_get_mask(p->mp_fld_rab_buf_free_ib_free); + p->RAC_RAB_BUF_FREE_OB_FREE_MASK = nthw_field_get_mask(p->mp_fld_rab_buf_free_ob_free); + p->RAC_RAB_BUF_USED_IB_USED_MASK = nthw_field_get_mask(p->mp_fld_rab_buf_used_ib_used); + p->RAC_RAB_BUF_USED_OB_USED_MASK = nthw_field_get_mask(p->mp_fld_rab_buf_used_ob_used); + + p->RAC_RAB_BUF_USED_FLUSH_MASK = nthw_field_get_mask(p->mp_fld_rab_buf_used_flush); + + p->RAC_RAB_BUF_USED_OB_USED_LOW = + nthw_field_get_bit_pos_low(p->mp_fld_rab_buf_used_ob_used); + + p->mp_reg_rab_nmb_rd = nthw_module_query_register(p->mp_mod_rac, RAC_NMB_RD_ADR); + + if (p->mp_reg_rab_nmb_rd) + p->RAC_NMB_RD_ADR_ADDR = nthw_register_get_address(p->mp_reg_rab_nmb_rd); + + p->mp_reg_rab_nmb_data = nthw_module_query_register(p->mp_mod_rac, RAC_NMB_DATA); + + if (p->mp_reg_rab_nmb_data) + p->RAC_NMB_DATA_ADDR = nthw_register_get_address(p->mp_reg_rab_nmb_data); + + p->mp_reg_rab_nmb_wr = nthw_module_query_register(p->mp_mod_rac, RAC_NMB_WR_ADR); + + if (p->mp_reg_rab_nmb_wr) + p->RAC_NMB_WR_ADR_ADDR = nthw_register_get_address(p->mp_reg_rab_nmb_wr); + + p->mp_reg_rab_nmb_status = nthw_module_query_register(p->mp_mod_rac, RAC_NMB_STATUS); + + if (p->mp_reg_rab_nmb_status) + p->RAC_NMB_STATUS_ADDR = nthw_register_get_address(p->mp_reg_rab_nmb_status); + + p->m_dma = NULL; + + { + /* + * RAC is a primary communication channel - debug will be messy + * turn off debug by default - except for rac_rab_init + * NOTE: currently debug will not work - due to optimizations + */ + const int n_debug_mode = nthw_module_get_debug_mode(p->mp_mod_rac); + + if (n_debug_mode && n_debug_mode <= 0xff) { + nthw_module_set_debug_mode(p->mp_mod_rac, 0); + nthw_register_set_debug_mode(p->mp_reg_rab_init, n_debug_mode); + } + } + + pthread_mutex_init(&p->m_mutex, NULL); + + return 0; +} + +int nthw_rac_get_rab_interface_count(const nthw_rac_t *p) +{ + return p->mn_param_rac_rab_interfaces; +} + +/* private function for internal RAC operations - + * improves log flexibility and prevents log flooding + */ +static void nthw_rac_reg_read32(const struct fpga_info_s *p_fpga_info, uint32_t reg_addr, + uint32_t *p_data) +{ + *p_data = *(volatile uint32_t *)((uint8_t *)p_fpga_info->bar0_addr + reg_addr); +} + +/* private function for internal RAC operations - + * improves log flexibility and prevents log flooding + */ +static void nthw_rac_reg_write32(const struct fpga_info_s *p_fpga_info, uint32_t reg_addr, + uint32_t n_data) +{ + *(volatile uint32_t *)((uint8_t *)p_fpga_info->bar0_addr + reg_addr) = n_data; +} + +static inline int _nthw_rac_wait_for_rab_done(const nthw_rac_t *p, uint32_t address, + uint32_t word_cnt) +{ + const struct fpga_info_s *const p_fpga_info = p->mp_fpga->p_fpga_info; + const char *const p_adapter_id_str = p_fpga_info->mp_adapter_id_str; + uint32_t used = 0; + uint32_t retry; + + for (retry = 0; retry < 100000; retry++) { + nthw_rac_reg_read32(p_fpga_info, p->RAC_RAB_BUF_USED_ADDR, &used); + used = (used & p->RAC_RAB_BUF_USED_OB_USED_MASK) >> + p->RAC_RAB_BUF_USED_OB_USED_LOW; + + if (used >= word_cnt) + break; + } + + if (used < word_cnt) { + NT_LOG(ERR, NTHW, "%s: Fail rab bus r/w addr=0x%08X used=%x wordcount=%d\n", + p_adapter_id_str, address, used, word_cnt); + return -1; + } + + return 0; +} + +/* + * NT_PCI_REG_P9xyz_RAC_RAB_INIT + * + * Initializes (resets) the programmable registers on the Register Access Busses (RAB). + * This initialization must be performed by software as part of the driver load procedure. + * + * Bit n of this field initializes the programmable registers on RAB interface n. + * Software must write one to the bit and then clear the bit again. + * + * All RAB module registers will be reset to their defaults. + * This includes the product specific RESET module (eg RST9xyz) + * As a consequence of this behavior the official reset sequence + * must be excersised - as all RAB modules will be held in reset. + */ +int nthw_rac_rab_init(nthw_rac_t *p, uint32_t n_rab_intf_mask) +{ + /* + * Write rac_rab_init + * Perform operation twice - first to get trace of operation - + * second to get things done... + */ + const struct fpga_info_s *const p_fpga_info = p->mp_fpga->p_fpga_info; + nthw_field_set_val_flush32(p->mp_fld_rab_init, n_rab_intf_mask); + nthw_rac_reg_write32(p_fpga_info, p->RAC_RAB_INIT_ADDR, n_rab_intf_mask); + return 0; +} + +int nthw_rac_rab_reset(nthw_rac_t *p) +{ + const struct fpga_info_s *const p_fpga_info = p->mp_fpga->p_fpga_info; + const char *const p_adapter_id_str = p_fpga_info->mp_adapter_id_str; + (void)p_adapter_id_str; + + /* RAC RAB bus "flip/flip" reset */ + const int n_rac_rab_bus_count = nthw_rac_get_rab_interface_count(p); + const int n_rac_rab_bus_mask = (1 << n_rac_rab_bus_count) - 1; + + NT_LOG(DBG, NTHW, "%s: NT_RAC_RAB_INTERFACES=%d (0x%02X)\n", p_adapter_id_str, + n_rac_rab_bus_count, n_rac_rab_bus_mask); + assert(n_rac_rab_bus_count); + assert(n_rac_rab_bus_mask); + + /* RAC RAB bus "flip/flip" reset first stage - new impl (ref RMT#37020) */ + nthw_rac_rab_init(p, 0); + nthw_rac_rab_init(p, n_rac_rab_bus_mask); + nthw_rac_rab_init(p, n_rac_rab_bus_mask & ~0x01); + + return 0; +} + +int nthw_rac_rab_setup(nthw_rac_t *p) +{ + int rc = 0; + + const struct fpga_info_s *const p_fpga_info = p->mp_fpga->p_fpga_info; + uint32_t n_dma_buf_size = 2L * RAB_DMA_BUF_CNT * sizeof(uint32_t); + const size_t align_size = nt_util_align_size(n_dma_buf_size); + int numa_node = p_fpga_info->numa_node; + uint64_t dma_addr; + uint32_t buf; + + if (!p->m_dma) { + struct nt_dma_s *vfio_dma; + /* FPGA needs Page alignment (4K) */ + vfio_dma = nt_dma_alloc(align_size, 0x1000, numa_node); + + if (vfio_dma == NULL) { + NT_LOG(ERR, ETHDEV, "%s: nt_dma_alloc failed\n", __func__); + return -1; + } + + p->m_dma_in_buf = (uint32_t *)vfio_dma->addr; + p->m_dma_out_buf = p->m_dma_in_buf + RAB_DMA_BUF_CNT; + p->m_dma = vfio_dma; + } + + /* Setup DMA on the adapter */ + dma_addr = p->m_dma->iova; + nthw_rac_reg_write32(p_fpga_info, p->RAC_RAB_DMA_IB_LO_ADDR, dma_addr & 0xffffffff); + nthw_rac_reg_write32(p_fpga_info, p->RAC_RAB_DMA_IB_HI_ADDR, + (uint32_t)(dma_addr >> 32) & 0xffffffff); + dma_addr += RAB_DMA_BUF_CNT * sizeof(uint32_t); + nthw_rac_reg_write32(p_fpga_info, p->RAC_RAB_DMA_OB_LO_ADDR, dma_addr & 0xffffffff); + nthw_rac_reg_write32(p_fpga_info, p->RAC_RAB_DMA_OB_HI_ADDR, + (uint32_t)(dma_addr >> 32) & 0xffffffff); + + /* Set initial value of internal pointers */ + nthw_rac_reg_read32(p_fpga_info, p->RAC_RAB_DMA_IB_RD_ADDR, &buf); + p->m_dma_in_ptr_wr = (uint16_t)(buf / sizeof(uint32_t)); + nthw_rac_reg_read32(p_fpga_info, p->RAC_RAB_DMA_OB_WR_ADDR, &buf); + p->m_dma_out_ptr_rd = (uint16_t)(buf / sizeof(uint32_t)); + p->m_in_free = RAB_DMA_BUF_CNT; + + return rc; +} + +void nthw_rac_bar0_read32(const struct fpga_info_s *p_fpga_info, uint32_t reg_addr, + uint32_t word_cnt, uint32_t *p_data) +{ + volatile const uint32_t *const src_addr = + (uint32_t *)((uint8_t *)p_fpga_info->bar0_addr + reg_addr); + + for (uint32_t i = 0; i < word_cnt; i++) + p_data[i] = src_addr[i]; +} + +void nthw_rac_bar0_write32(const struct fpga_info_s *p_fpga_info, uint32_t reg_addr, + uint32_t word_cnt, const uint32_t *p_data) +{ + volatile uint32_t *const dst_addr = + (uint32_t *)((uint8_t *)p_fpga_info->bar0_addr + reg_addr); + + for (uint32_t i = 0; i < word_cnt; i++) + dst_addr[i] = p_data[i]; +} + +int nthw_rac_rab_write32(nthw_rac_t *p, bool trc, nthw_rab_bus_id_t bus_id, uint32_t address, + uint32_t word_cnt, const uint32_t *p_data) +{ + const struct fpga_info_s *const p_fpga_info = p->mp_fpga->p_fpga_info; + const char *const p_adapter_id_str = p_fpga_info->mp_adapter_id_str; + uint32_t buf_used; + uint32_t buf_free; + uint32_t in_buf_free; + uint32_t out_buf_free; + int res = 0; + + if (address > (1 << RAB_ADDR_BW)) { + NT_LOG(ERR, NTHW, "%s: RAB: Illegal address: value too large %d - max %d\n", + p_adapter_id_str, address, (1 << RAB_ADDR_BW)); + return -1; + } + + if (bus_id > (1 << RAB_BUSID_BW)) { + NT_LOG(ERR, NTHW, "%s: RAB: Illegal bus id: value too large %d - max %d\n", + p_adapter_id_str, bus_id, (1 << RAB_BUSID_BW)); + return -1; + } + + if (word_cnt == 0) { + NT_LOG(ERR, NTHW, "%s: RAB: Illegal word count: value is zero (%d)\n", + p_adapter_id_str, word_cnt); + return -1; + } + + if (word_cnt > (1 << RAB_CNT_BW)) { + NT_LOG(ERR, NTHW, "%s: RAB: Illegal word count: value too large %d - max %d\n", + p_adapter_id_str, word_cnt, (1 << RAB_CNT_BW)); + return -1; + } + + pthread_mutex_lock(&p->m_mutex); + + if (p->m_dma_active) { + NT_LOG(ERR, NTHW, "%s: RAB: Illegal operation: DMA enabled\n", p_adapter_id_str); + res = -1; + goto exit_unlock_res; + } + + /* Read buffer free register */ + nthw_rac_reg_read32(p_fpga_info, p->RAC_RAB_BUF_FREE_ADDR, &buf_free); + + in_buf_free = buf_free & p->RAC_RAB_BUF_FREE_IB_FREE_MASK; + out_buf_free = (buf_free & p->RAC_RAB_BUF_FREE_OB_FREE_MASK) >> 16; + + /* Read buffer used register */ + nthw_rac_reg_read32(p_fpga_info, p->RAC_RAB_BUF_USED_ADDR, &buf_used); + + buf_used = + buf_used & (p->RAC_RAB_BUF_USED_IB_USED_MASK | p->RAC_RAB_BUF_USED_OB_USED_MASK); + + /* + * Verify that output buffer can hold one completion word, + * input buffer can hold the number of words to be written + + * one write and one completion command + * and that the input and output "used" buffer is 0 + */ + if (out_buf_free >= 1 && in_buf_free >= word_cnt + 2 && buf_used == 0) { + const uint32_t rab_oper_cmpl = (RAB_COMPLETION << RAB_OPR_LO); + uint32_t rab_echo_oper_cmpl; + uint32_t word_cnt_expected = 1; + uint32_t rab_oper_wr; + uint32_t i; + + rab_oper_wr = (RAB_WRITE << RAB_OPR_LO) | + ((word_cnt & ((1 << RAB_CNT_BW) - 1)) << RAB_CNT_LO) | + (bus_id << RAB_BUSID_LO) | address; + + if (trc) { + rab_oper_wr |= (RAB_ECHO << RAB_OPR_LO); + word_cnt_expected += word_cnt + 1; + } + + /* Write command */ + nthw_rac_reg_write32(p_fpga_info, p->RAC_RAB_IB_DATA_ADDR, rab_oper_wr); + + /* Write data to input buffer */ + for (i = 0; i < word_cnt; i++) + nthw_rac_reg_write32(p_fpga_info, p->RAC_RAB_IB_DATA_ADDR, p_data[i]); + + /* Write completion command */ + nthw_rac_reg_write32(p_fpga_info, p->RAC_RAB_IB_DATA_ADDR, rab_oper_cmpl); + + /* Wait until done */ + if (_nthw_rac_wait_for_rab_done(p, address, word_cnt_expected)) { + res = -1; + goto exit_unlock_res; + } + + if (trc) { + uint32_t rab_echo_oper_wr; + nthw_rac_reg_read32(p_fpga_info, p->RAC_RAB_OB_DATA_ADDR, + &rab_echo_oper_wr); + + if (p->mn_param_rac_rab_ob_update) + nthw_rac_reg_write32(p_fpga_info, p->RAC_RAB_OB_DATA_ADDR, 0); + + if (rab_oper_wr != rab_echo_oper_wr) { + NT_LOG(ERR, NTHW, + "%s: expected rab read echo oper (0x%08X) - read (0x%08X)\n", + p_adapter_id_str, rab_oper_wr, rab_echo_oper_wr); + } + } + + { + /* Read data from output buffer */ + uint32_t data; + char *tmp_string; + + if (trc) { + tmp_string = ntlog_helper_str_alloc("Register::write"); + ntlog_helper_str_add(tmp_string, + "(Dev: NA, Bus: RAB%u, Addr: 0x%08X, Cnt: %d, Data:", + bus_id, address, word_cnt); + } + + for (i = 0; i < word_cnt; i++) { + nthw_rac_reg_read32(p_fpga_info, p->RAC_RAB_OB_DATA_ADDR, &data); + + if (p->mn_param_rac_rab_ob_update) { + nthw_rac_reg_write32(p_fpga_info, p->RAC_RAB_OB_DATA_ADDR, + 0); + } + + if (trc) + ntlog_helper_str_add(tmp_string, " 0x%08X", data); + } + + if (trc) { + ntlog_helper_str_add(tmp_string, ")"); + NT_LOG(DBG, NTHW, "%s", tmp_string); + ntlog_helper_str_free(tmp_string); + } + } + + /* Read completion from out buffer */ + nthw_rac_reg_read32(p_fpga_info, p->RAC_RAB_OB_DATA_ADDR, &rab_echo_oper_cmpl); + + if (p->mn_param_rac_rab_ob_update) + nthw_rac_reg_write32(p_fpga_info, p->RAC_RAB_OB_DATA_ADDR, 0); + + if (rab_echo_oper_cmpl != rab_oper_cmpl) { + NT_LOG(ERR, NTHW, + "%s: RAB: Unexpected value of completion (0x%08X)- inBufFree: 0x%08X, outBufFree: 0x%08X, bufUsed: 0x%08X\n", + p_adapter_id_str, rab_echo_oper_cmpl, in_buf_free, out_buf_free, + buf_used); + res = -1; + goto exit_unlock_res; + } + + /* Read buffer free register */ + nthw_rac_reg_read32(p_fpga_info, p->RAC_RAB_BUF_FREE_ADDR, &buf_free); + + if (buf_free & 0x80000000) { + /* Clear Timeout and overflow bits */ + nthw_rac_reg_write32(p_fpga_info, p->RAC_RAB_BUF_FREE_ADDR, 0x0); + NT_LOG(ERR, NTHW, + "%s: RAB: timeout - Access outside register - bus: %d addr: 0x%08X - inBufFree: 0x%08X, outBufFree: 0x%08X, bufUsed: 0x%08X\n", + p_adapter_id_str, bus_id, address, in_buf_free, out_buf_free, + buf_used); + res = -1; + goto exit_unlock_res; + } + + res = 0; + goto exit_unlock_res; + + } else { + NT_LOG(ERR, NTHW, + "%s: RAB: Fail rab bus buffer check - bus: %d addr: 0x%08X wordcount: %d - inBufFree: 0x%08X, outBufFree: 0x%08X, bufUsed: 0x%08X\n", + p_adapter_id_str, bus_id, address, word_cnt, in_buf_free, out_buf_free, + buf_used); + res = -1; + goto exit_unlock_res; + } + +exit_unlock_res: + pthread_mutex_unlock(&p->m_mutex); + return res; +} + +int nthw_rac_rab_read32(nthw_rac_t *p, bool trc, nthw_rab_bus_id_t bus_id, uint32_t address, + uint32_t word_cnt, uint32_t *p_data) +{ + const struct fpga_info_s *const p_fpga_info = p->mp_fpga->p_fpga_info; + const char *const p_adapter_id_str = p_fpga_info->mp_adapter_id_str; + uint32_t buf_used; + uint32_t buf_free; + uint32_t in_buf_free; + uint32_t out_buf_free; + int res = 0; + + pthread_mutex_lock(&p->m_mutex); + + if (address > (1 << RAB_ADDR_BW)) { + NT_LOG(ERR, NTHW, "%s: RAB: Illegal address: value too large %d - max %d\n", + p_adapter_id_str, address, (1 << RAB_ADDR_BW)); + res = -1; + goto exit_unlock_res; + } + + if (bus_id > (1 << RAB_BUSID_BW)) { + NT_LOG(ERR, NTHW, "%s: RAB: Illegal bus id: value too large %d - max %d\n", + p_adapter_id_str, bus_id, (1 << RAB_BUSID_BW)); + res = -1; + goto exit_unlock_res; + } + + if (word_cnt == 0) { + NT_LOG(ERR, NTHW, "%s: RAB: Illegal word count: value is zero (%d)\n", + p_adapter_id_str, word_cnt); + res = -1; + goto exit_unlock_res; + } + + if (word_cnt > (1 << RAB_CNT_BW)) { + NT_LOG(ERR, NTHW, "%s: RAB: Illegal word count: value too large %d - max %d\n", + p_adapter_id_str, word_cnt, (1 << RAB_CNT_BW)); + res = -1; + goto exit_unlock_res; + } + + /* Read buffer free register */ + nthw_rac_reg_read32(p_fpga_info, p->RAC_RAB_BUF_FREE_ADDR, &buf_free); + + in_buf_free = buf_free & p->RAC_RAB_BUF_FREE_IB_FREE_MASK; + out_buf_free = (buf_free & p->RAC_RAB_BUF_FREE_OB_FREE_MASK) >> 16; + + /* Read buffer used register */ + nthw_rac_reg_read32(p_fpga_info, p->RAC_RAB_BUF_USED_ADDR, &buf_used); + + buf_used = + buf_used & (p->RAC_RAB_BUF_USED_IB_USED_MASK | p->RAC_RAB_BUF_USED_OB_USED_MASK); + + /* + * Verify that output buffer can hold the number of words to be read, + * input buffer can hold one read command + * and that the input and output "used" buffer is 0 + */ + if (out_buf_free >= word_cnt && in_buf_free >= 1 && buf_used == 0) { + const uint32_t rab_oper_cmpl = (RAB_COMPLETION << RAB_OPR_LO); + uint32_t rab_read_oper_cmpl; + uint32_t word_cnt_expected = word_cnt + 1; + uint32_t rab_oper_rd; + + rab_oper_rd = (RAB_READ << RAB_OPR_LO) | + ((word_cnt & ((1 << RAB_CNT_BW) - 1)) << RAB_CNT_LO) | + (bus_id << RAB_BUSID_LO) | address; + + if (trc) { + rab_oper_rd |= (RAB_ECHO << RAB_OPR_LO); + word_cnt_expected++; + } + + /* Write command */ + nthw_rac_reg_write32(p_fpga_info, p->RAC_RAB_IB_DATA_ADDR, rab_oper_rd); + + /* Write completion command */ + nthw_rac_reg_write32(p_fpga_info, p->RAC_RAB_IB_DATA_ADDR, rab_oper_cmpl); + + /* Wait until done */ + if (_nthw_rac_wait_for_rab_done(p, address, word_cnt_expected)) { + res = -1; + goto exit_unlock_res; + } + + if (trc) { + uint32_t rab_echo_oper_rd; + nthw_rac_reg_read32(p_fpga_info, p->RAC_RAB_OB_DATA_ADDR, + &rab_echo_oper_rd); + + if (p->mn_param_rac_rab_ob_update) + nthw_rac_reg_write32(p_fpga_info, p->RAC_RAB_OB_DATA_ADDR, 0); + + if (rab_oper_rd != rab_echo_oper_rd) { + NT_LOG(ERR, NTHW, + "%s: RAB: expected rab read echo oper (0x%08X) - read (0x%08X)\n", + p_adapter_id_str, rab_oper_rd, rab_echo_oper_rd); + } + } + + { + /* Read data from output buffer */ + uint32_t i; + + for (i = 0; i < word_cnt; i++) { + nthw_rac_reg_read32(p_fpga_info, p->RAC_RAB_OB_DATA_ADDR, + &p_data[i]); + + if (p->mn_param_rac_rab_ob_update) { + nthw_rac_reg_write32(p_fpga_info, p->RAC_RAB_OB_DATA_ADDR, + 0); + } + } + + if (trc) { + char *tmp_string = ntlog_helper_str_alloc("Register::read"); + ntlog_helper_str_add(tmp_string, + "(Dev: NA, Bus: RAB%u, Addr: 0x%08X, Cnt: %d, Data:", + bus_id, address, word_cnt); + + for (i = 0; i < word_cnt; i++) + ntlog_helper_str_add(tmp_string, " 0x%08X", p_data[i]); + + ntlog_helper_str_add(tmp_string, ")"); + NT_LOG(DBG, NTHW, "%s", tmp_string); + ntlog_helper_str_free(tmp_string); + } + } + + /* Read completion from out buffer */ + nthw_rac_reg_read32(p_fpga_info, p->RAC_RAB_OB_DATA_ADDR, &rab_read_oper_cmpl); + + if (p->mn_param_rac_rab_ob_update) + nthw_rac_reg_write32(p_fpga_info, p->RAC_RAB_OB_DATA_ADDR, 0); + + if (rab_read_oper_cmpl != rab_oper_cmpl) { + NT_LOG(ERR, NTHW, + "%s: RAB: Unexpected value of completion (0x%08X)- inBufFree: 0x%08X, outBufFree: 0x%08X, bufUsed: 0x%08X\n", + p_adapter_id_str, rab_read_oper_cmpl, in_buf_free, out_buf_free, + buf_used); + res = -1; + goto exit_unlock_res; + } + + /* Read buffer free register */ + nthw_rac_reg_read32(p_fpga_info, p->RAC_RAB_BUF_FREE_ADDR, &buf_free); + + if (buf_free & 0x80000000) { + /* Clear Timeout and overflow bits */ + nthw_rac_reg_write32(p_fpga_info, p->RAC_RAB_BUF_FREE_ADDR, 0x0); + NT_LOG(ERR, NTHW, + "%s: RAB: timeout - Access outside register - bus: %d addr: 0x%08X - inBufFree: 0x%08X, outBufFree: 0x%08X, bufUsed: 0x%08X\n", + p_adapter_id_str, bus_id, address, in_buf_free, out_buf_free, + buf_used); + res = -1; + goto exit_unlock_res; + } + + res = 0; + goto exit_unlock_res; + + } else { + NT_LOG(ERR, NTHW, + "%s: RAB: Fail rab bus buffer check - bus: %d addr: 0x%08X wordcount: %d - inBufFree: 0x%08X, outBufFree: 0x%08X, bufUsed: 0x%08X\n", + p_adapter_id_str, bus_id, address, word_cnt, in_buf_free, out_buf_free, + buf_used); + res = -1; + goto exit_unlock_res; + } + +exit_unlock_res: + pthread_mutex_unlock(&p->m_mutex); + return res; +} + +int nthw_rac_rab_flush(nthw_rac_t *p) +{ + const struct fpga_info_s *const p_fpga_info = p->mp_fpga->p_fpga_info; + const char *const p_adapter_id_str = p_fpga_info->mp_adapter_id_str; + uint32_t data = 0; + uint32_t retry; + int res = 0; + + pthread_mutex_lock(&p->m_mutex); + + /* Set the flush bit */ + nthw_rac_reg_write32(p_fpga_info, p->RAC_RAB_BUF_USED_ADDR, + p->RAC_RAB_BUF_USED_FLUSH_MASK); + + /* Reset BUF FREE register */ + nthw_rac_reg_write32(p_fpga_info, p->RAC_RAB_BUF_FREE_ADDR, 0x0); + + /* Wait until OB_USED and IB_USED are 0 */ + for (retry = 0; retry < 100000; retry++) { + nthw_rac_reg_read32(p_fpga_info, p->RAC_RAB_BUF_USED_ADDR, &data); + + if ((data & 0xFFFFFFFF) == p->RAC_RAB_BUF_USED_FLUSH_MASK) + break; + } + + if (data != p->RAC_RAB_BUF_USED_FLUSH_MASK) { + NT_LOG(ERR, NTHW, "%s: RAB: Rab bus flush error.\n", p_adapter_id_str); + res = -1; + } + + /* Clear flush bit when done */ + nthw_rac_reg_write32(p_fpga_info, p->RAC_RAB_BUF_USED_ADDR, 0x0); + + pthread_mutex_unlock(&p->m_mutex); + return res; +} diff --git a/drivers/net/ntnic/nthw/nthw_rac.h b/drivers/net/ntnic/nthw/nthw_rac.h new file mode 100644 index 0000000000..44426cb608 --- /dev/null +++ b/drivers/net/ntnic/nthw/nthw_rac.h @@ -0,0 +1,154 @@ +/* + * SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2023 Napatech A/S + */ + +#ifndef __NTHW_RAC_H__ +#define __NTHW_RAC_H__ + +#include "nt_util.h" +#include "nthw_bus.h" + +#define RAB_DMA_BUF_CNT (0x4000) + +struct nthw_rac { + nthw_fpga_t *mp_fpga; + nthw_module_t *mp_mod_rac; + + pthread_mutex_t m_mutex; + + int mn_param_rac_rab_interfaces; + int mn_param_rac_rab_ob_update; + + nthw_register_t *mp_reg_dummy0; + nthw_register_t *mp_reg_dummy1; + nthw_register_t *mp_reg_dummy2; + + nthw_register_t *mp_reg_rab_init; + nthw_field_t *mp_fld_rab_init; + + int mn_fld_rab_init_bw; + uint32_t mn_fld_rab_init_mask; + + nthw_register_t *mp_reg_dbg_ctrl; + nthw_field_t *mp_fld_dbg_ctrl; + + nthw_register_t *mp_reg_dbg_data; + nthw_field_t *mp_fld_dbg_data; + + nthw_register_t *mp_reg_rab_ib_data; + nthw_field_t *mp_fld_rab_ib_data; + + nthw_register_t *mp_reg_rab_ob_data; + nthw_field_t *mp_fld_rab_ob_data; + + nthw_register_t *mp_reg_rab_buf_free; + nthw_field_t *mp_fld_rab_buf_free_ib_free; + nthw_field_t *mp_fld_rab_buf_free_ib_ovf; + nthw_field_t *mp_fld_rab_buf_free_ob_free; + nthw_field_t *mp_fld_rab_buf_free_ob_ovf; + nthw_field_t *mp_fld_rab_buf_free_timeout; + + nthw_register_t *mp_reg_rab_buf_used; + nthw_field_t *mp_fld_rab_buf_used_ib_used; + nthw_field_t *mp_fld_rab_buf_used_ob_used; + nthw_field_t *mp_fld_rab_buf_used_flush; + + nthw_register_t *mp_reg_rab_dma_ib_lo; + nthw_field_t *mp_fld_rab_dma_ib_lo_phy_addr; + + nthw_register_t *mp_reg_rab_dma_ib_hi; + nthw_field_t *mp_fld_rab_dma_ib_hi_phy_addr; + + nthw_register_t *mp_reg_rab_dma_ob_hi; + nthw_field_t *mp_fld_rab_dma_ob_hi_phy_addr; + + nthw_register_t *mp_reg_rab_dma_ob_lo; + nthw_field_t *mp_fld_rab_dma_ob_lo_phy_addr; + + nthw_register_t *mp_reg_rab_dma_ib_wr; + nthw_field_t *mp_fld_rab_dma_ib_wr_ptr; + + nthw_register_t *mp_reg_rab_dma_ib_rd; + nthw_field_t *mp_fld_rab_dma_ib_rd_ptr; + + nthw_register_t *mp_reg_rab_dma_ob_wr; + nthw_field_t *mp_fld_rab_dma_ob_wr_ptr; + + nthw_register_t *mp_reg_rab_nmb_rd; + nthw_register_t *mp_reg_rab_nmb_data; + nthw_register_t *mp_reg_rab_nmb_wr; + nthw_register_t *mp_reg_rab_nmb_status; + + uint32_t RAC_RAB_INIT_ADDR; + uint32_t RAC_RAB_IB_DATA_ADDR; + uint32_t RAC_RAB_OB_DATA_ADDR; + uint32_t RAC_RAB_BUF_FREE_ADDR; + uint32_t RAC_RAB_BUF_USED_ADDR; + + uint32_t RAC_RAB_DMA_IB_LO_ADDR; + uint32_t RAC_RAB_DMA_IB_HI_ADDR; + uint32_t RAC_RAB_DMA_OB_LO_ADDR; + uint32_t RAC_RAB_DMA_OB_HI_ADDR; + uint32_t RAC_RAB_DMA_IB_RD_ADDR; + uint32_t RAC_RAB_DMA_OB_WR_ADDR; + uint32_t RAC_RAB_DMA_IB_WR_ADDR; + + uint32_t RAC_RAB_BUF_FREE_IB_FREE_MASK; + uint32_t RAC_RAB_BUF_FREE_OB_FREE_MASK; + uint32_t RAC_RAB_BUF_USED_IB_USED_MASK; + uint32_t RAC_RAB_BUF_USED_OB_USED_MASK; + uint32_t RAC_RAB_BUF_USED_FLUSH_MASK; + + uint32_t RAC_RAB_BUF_USED_OB_USED_LOW; + + uint32_t RAC_NMB_RD_ADR_ADDR; + uint32_t RAC_NMB_DATA_ADDR; + uint32_t RAC_NMB_WR_ADR_ADDR; + uint32_t RAC_NMB_STATUS_ADDR; + + bool m_dma_active; + + struct nt_dma_s *m_dma; + + volatile uint32_t *m_dma_in_buf; + volatile uint32_t *m_dma_out_buf; + + uint16_t m_dma_out_ptr_rd; + uint16_t m_dma_in_ptr_wr; + uint32_t m_in_free; +}; + +typedef struct nthw_rac nthw_rac_t; +typedef struct nthw_rac nthw_rac; + +struct dma_buf_ptr { + uint32_t size; + uint32_t index; + volatile uint32_t *base; +}; + +nthw_rac_t *nthw_rac_new(void); +int nthw_rac_init(nthw_rac_t *p, nthw_fpga_t *p_fpga, struct fpga_info_s *p_fpga_info); + +int nthw_rac_get_rab_interface_count(const nthw_rac_t *p); + +int nthw_rac_rab_init(nthw_rac_t *p, uint32_t n_rab_intf_mask); + +int nthw_rac_rab_setup(nthw_rac_t *p); + +int nthw_rac_rab_reset(nthw_rac_t *p); + +int nthw_rac_rab_write32(nthw_rac_t *p, bool trc, nthw_rab_bus_id_t bus_id, uint32_t address, + uint32_t word_cnt, const uint32_t *p_data); +int nthw_rac_rab_read32(nthw_rac_t *p, bool trc, nthw_rab_bus_id_t bus_id, uint32_t address, + uint32_t word_cnt, uint32_t *p_data); + +int nthw_rac_rab_flush(nthw_rac_t *p); + +void nthw_rac_bar0_read32(const struct fpga_info_s *p_fpga_info, uint32_t reg_addr, + uint32_t word_cnt, uint32_t *p_data); +void nthw_rac_bar0_write32(const struct fpga_info_s *p_fpga_info, uint32_t reg_addr, + uint32_t word_cnt, const uint32_t *p_data); + +#endif /* __NTHW_RAC_H__ */ diff --git a/drivers/net/ntnic/nthw/nthw_register.h b/drivers/net/ntnic/nthw/nthw_register.h new file mode 100644 index 0000000000..ecc661a656 --- /dev/null +++ b/drivers/net/ntnic/nthw/nthw_register.h @@ -0,0 +1,22 @@ +/* + * SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2023 Napatech A/S + */ + +#ifndef NTHW_REGISTER_H_ +#define NTHW_REGISTER_H_ + +#include +#include +#include +#include + +#include "nthw_fpga_model.h" + +#include "fpga_model.h" + +#include "nthw_fpga_mod_defs.h" +#include "nthw_fpga_param_defs.h" +#include "nthw_fpga_reg_defs.h" + +#endif /* NTHW_REGISTER_H_ */ diff --git a/drivers/net/ntnic/nthw/nthw_utils.c b/drivers/net/ntnic/nthw/nthw_utils.c new file mode 100644 index 0000000000..d06460b67c --- /dev/null +++ b/drivers/net/ntnic/nthw/nthw_utils.c @@ -0,0 +1,53 @@ +/* + * SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2023 Napatech A/S + */ + +#include "ntlog.h" + +#include +#include +#include /* isprint */ +#include +#include +#include /* inet_addr */ +#include /* memset */ + +#include "nthw_utils.h" +#include "nthw_helper.h" + +int socket_loopback_setup(uint16_t port) +{ + int res = 0; + struct sockaddr_in serv_addr; + int sockfd; + int sockval; + + /* socket create and verification */ + sockfd = socket(AF_INET, SOCK_STREAM, 0); + + if (sockfd == -1) { + NT_LOG(ERR, NTHW, "socket creation failed...\n"); + res = -1; + } + + setsockopt(sockfd, SOL_SOCKET, SO_REUSEPORT, &sockval, sizeof(sockval)); + + memset(&serv_addr, 0, sizeof(serv_addr)); + serv_addr.sin_family = AF_INET; + serv_addr.sin_addr.s_addr = htonl(INADDR_LOOPBACK); + serv_addr.sin_port = htons(port); + + if ((bind(sockfd, (struct sockaddr *)&serv_addr, sizeof(serv_addr))) != 0) { + NT_LOG(ERR, NTHW, "socket bind failed...\n"); + res = -1; + } + + /* Now server is ready to listen and verification */ + if ((listen(sockfd, 5)) != 0) { + NT_LOG(ERR, NTHW, "Listen failed...\n"); + res = -1; + } + + return res == 0 ? sockfd : res; +} diff --git a/drivers/net/ntnic/nthw/nthw_utils.h b/drivers/net/ntnic/nthw/nthw_utils.h new file mode 100644 index 0000000000..b1ac4977b3 --- /dev/null +++ b/drivers/net/ntnic/nthw/nthw_utils.h @@ -0,0 +1,11 @@ +/* + * SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2023 Napatech A/S + */ + +#ifndef __NTHW_UTILS_H__ +#define __NTHW_UTILS_H__ + +int socket_loopback_setup(uint16_t port); + +#endif /* __NTHW_UTILS_H__ */ -- 2.44.0