From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1B24145509; Thu, 27 Jun 2024 09:40:57 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id BD71940E2B; Thu, 27 Jun 2024 09:39:46 +0200 (CEST) Received: from egress-ip11b.ess.de.barracuda.com (egress-ip11b.ess.de.barracuda.com [18.185.115.215]) by mails.dpdk.org (Postfix) with ESMTP id F3F6740DDE for ; Thu, 27 Jun 2024 09:39:39 +0200 (CEST) Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05lp2169.outbound.protection.outlook.com [104.47.17.169]) by mx-outbound43-83.eu-central-1c.ess.aws.cudaops.com (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 27 Jun 2024 07:39:37 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=n6kh0mkKTXpLUUHtslCvO1+HuHXFyQypeB+ISiJ4JIsEFiw0MmtbzyDMYsvnJprK52t3FBeCIrYpBbaXhIfaGGhN4J8oooNMq2flixU7k7ao9BvXQ5Qf81F1JVcEhLebeyApWw4bPdfmnsmuorys8dkL/2kW3+MGmHZt6fZjmXo0LWCgtfj/KKCtB8R1qSoVcWVzCetsctBUABT6F9Xbc0zchznM8QHsjYqaX0nZ1fD3gOSgOXX50iYFJH+zF8rPvoi3Mtw0YvW9/9gN111Tsn6s5gfluj+QA0vFyhVbR++4sPDhDAVeJFoTkrbG5x8DieIlfC7rncBtsOuSCz24iw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=3R5BFEYij5G8jaVMm/2QSxsjnagOMn2TjnU9pQxXFhM=; b=Fh+BNud3nrVxoTkFU56zoXxjwTZohPawIcrtucwAA/a+/gUAiE1JjnIzBG1zCLdgcYN1HUqpJ2VRqHARK3Vc5W0biY7677CQU/Zy+opYa4ClQ+NsOGdb/gToe29XNbD8YVJrYIPS3orb4rumBZ/0+f+jIbYfICG1Zmr8++eD0yJJKbsWwCdim2Q0e5B+DDWQ5/sJ5T6vFc7HaJYqgxOaya/Xp4aUiD3u/GlZ261mjDrRSANUj5GBile+OrgvEmn0ZjLxrscGb+FLtTHgUTCpGVC3ZCEOiBEEQ+uzKSv9xFMcRFiIu3UpjvWSuCfDZT6oOV5IyeX8nR6FZRQt89rNCA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=fail (sender ip is 178.72.21.4) smtp.rcpttodomain=dpdk.org smtp.mailfrom=napatech.com; dmarc=fail (p=reject sp=reject pct=100) action=oreject header.from=napatech.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=napatech.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3R5BFEYij5G8jaVMm/2QSxsjnagOMn2TjnU9pQxXFhM=; b=knZR7APkPWaNf4FWkjVoRTgjsz5y4XRNlEk4SoJGLXAefSETUGt/YXShKqHXUrfa+w//NJe4qLrJOTSnNJlUBb/pqmUQNnIAl6k7hdyJP7HyulqLakz/Vqpc1MjCHE36gE3/TxTnYkf4/YgvHK6pLizAQWe82Hh1Fo9NuRbK4Uk= Received: from DUZPR01CA0337.eurprd01.prod.exchangelabs.com (2603:10a6:10:4b8::10) by AS8P190MB1128.EURP190.PROD.OUTLOOK.COM (2603:10a6:20b:2e0::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7719.25; Thu, 27 Jun 2024 07:39:27 +0000 Received: from DB1PEPF000509E2.eurprd03.prod.outlook.com (2603:10a6:10:4b8:cafe::1d) by DUZPR01CA0337.outlook.office365.com (2603:10a6:10:4b8::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7719.25 via Frontend Transport; Thu, 27 Jun 2024 07:39:27 +0000 X-MS-Exchange-Authentication-Results: spf=fail (sender IP is 178.72.21.4) smtp.mailfrom=napatech.com; dkim=none (message not signed) header.d=none;dmarc=fail action=oreject header.from=napatech.com; Received-SPF: Fail (protection.outlook.com: domain of napatech.com does not designate 178.72.21.4 as permitted sender) receiver=protection.outlook.com; client-ip=178.72.21.4; helo=localhost.localdomain; Received: from localhost.localdomain (178.72.21.4) by DB1PEPF000509E2.mail.protection.outlook.com (10.167.242.52) with Microsoft SMTP Server id 15.20.7677.15 via Frontend Transport; Thu, 27 Jun 2024 07:39:26 +0000 From: Serhii Iliushyk To: dev@dpdk.org Cc: mko-plv@napatech.com, sil-plv@napatech.com, ckm@napatech.com, andrew.rybchenko@oktetlabs.ru, ferruh.yigit@amd.com Subject: [PATCH v5 06/23] net/ntnic: add NT NIC driver dependencies Date: Thu, 27 Jun 2024 09:38:47 +0200 Message-ID: <20240627073918.2039719-6-sil-plv@napatech.com> X-Mailer: git-send-email 2.45.0 In-Reply-To: <20240627073918.2039719-1-sil-plv@napatech.com> References: <20240530144929.4127931-1-sil-plv@napatech.com> <20240627073918.2039719-1-sil-plv@napatech.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DB1PEPF000509E2:EE_|AS8P190MB1128:EE_ Content-Type: text/plain X-MS-Office365-Filtering-Correlation-Id: 59fee880-f531-4a2b-15c1-08dc967c45c9 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|376014|82310400026|1800799024|36860700013; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?lOYRptCuRu4VGQoKcDEIQ9J0RKtx2As7PWQaofKsnD4d1Jkir6IlXqYC2vkJ?= =?us-ascii?Q?wOLSMOUuqQ7zes0Gkg8xKFxvS6G9t1jjN4NQ042jdoIPK8WqyQbfdajjjVW8?= =?us-ascii?Q?wOFAtedVRJC07zmulM7Ydi7pFvo4cvKOi1v/hMn8lvZWof3Au8Pwe1zwH6fR?= =?us-ascii?Q?davmJLfODOvVrJz5VEIpfZijrAnkBvUQyr9WCEktMpn3bsn8piKtUjWepBxM?= =?us-ascii?Q?Hqqblwsa0V3kHqVmQP+tND5wV0jryms0XLIuuCDPhBMi0tgue9xexI4WoRTh?= =?us-ascii?Q?fgOxkjWXLM9/mK8j3Gq6yzdqp3QYAuVUuMuWQrs0PGUily2FASJ7quIWPMrt?= =?us-ascii?Q?VKIF7vU2kWZSMvqJgzmWfeV5UQVN1s8MHShRyTxmMme7eG6ZQq29GELFRxZD?= =?us-ascii?Q?e6JwAIVnHTlPe9fY8BiBjVOTybyxOjpTziHp4V+N7q1xStKqz8LWeAWIe8Nx?= =?us-ascii?Q?Xe9pQyDueRBgjgB1rK2+ba53ikAihcS4MetsjJ0m6D+AY/WHClsdh2pGnObs?= =?us-ascii?Q?Xbviy7YOXOqLwxjRZeBTYHsXxTAYSglrwst+I0WMr3jPleheP4xc+l1p9Pao?= =?us-ascii?Q?4tiZUzBUJFU6Q2ulW/Q6kj+wvmsRH8Nuzb6SblCzJMFtCvscVC5TFhePT9w4?= =?us-ascii?Q?0KHwQBW3uHAB1cfo4iTNXJ1XRyd43wFQpGLEZi+o4ck+10O55CGcODLqzCAW?= =?us-ascii?Q?SZnUQZFp20/njF+c4C3hg11wclgpnBxLiswWBCqrE3G+aWN4wMmdkBGucs2M?= =?us-ascii?Q?B/kX6BK1ypG4U+lBr/UVIEJmvgDTpCoiImdQZAfwPONNgP/FRKRmJgY3y6Xe?= =?us-ascii?Q?8z9jUSdKrNnsDYm5L+VGzV4ZisJsQRrHufbtlyrCTg/r3WERT+LfMCZbgcVN?= =?us-ascii?Q?qesO8mmfdNSmbvPQc2p9KIKkWA1mXLKx+ZYRjf43QbEJSsBDRQucP94t6rod?= =?us-ascii?Q?dJsaCIMnTycje+82xaCE3cB4zQfrLxQw6aWakAh/4+lYUZnbP8MW6dRTS+MJ?= =?us-ascii?Q?Cwqg+4nOLllCLEF8GdKoy6Yj7Sv91t3Bq9p7mDOE/v29Bo3UP3p0PAIY63hy?= =?us-ascii?Q?lEJ9mLg85qKVTY4UiQ8QufnyUlwnecVXsDFK+ON9wwXbBTdg72S6fa69dwO4?= =?us-ascii?Q?JYBmHQuzYYAHQtmilZJEazc8tKBrcvE/arJtppZMGGcU5rAg05s2K191jMjx?= =?us-ascii?Q?cfeijrT/thr6aSG6YS0hTsc/4+rtBiahzWp0HcSzRKRefXCsF3ysoM6Tp3Jc?= =?us-ascii?Q?MJOD1sDcSFhuAnbaOq0vqVyz1jbH7Mq7WYxfDG4stiMUmRV2eOQwMPKpvS8l?= =?us-ascii?Q?Rr0/v1OUk/Ap59Pj3kI3RKTXkDHxqmnUCDGgBbodwMvxQXPZby430nuRvGwh?= =?us-ascii?Q?0ZYH9VnLTBQVi9CYRYbR5tolj1PTgmObCU1YOvcY47juVjcECJSpBRRgniZn?= =?us-ascii?Q?VnyuVTSTwVp2EtFuSo6ssc9TQkc+QPKH?= X-Forefront-Antispam-Report: CIP:178.72.21.4; CTRY:DK; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:localhost.localdomain; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(376014)(82310400026)(1800799024)(36860700013); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: qJqCYW3IS7W9eYNpOyhgYFqIR4OwFxQkudzyoXNo9b57QMnVoxoXzUvaLktGrpE7BAEMzucNxSJFF4BBy6sHmb1TgK2cBLdikjr6bSEfaqg59p1Kh3ITqelg8nFZ7+3yjEXH16TGyB1/HQ44gIcoQ7599BYyHK7iYT0rNipm7ILqVIsIBF+6K6PgoQX91GyJc6rESlXcEBDFurlJSUyon2U0ZH76kxcYupQ3QQb+382wVu6ExPYvSHGwkW4uYiMZyaUiot1PBrQzUqYDr/Oash7zNZV9OAsl7X8qrP3ahOIXlbbHA1U0DRniv6m5gEwGcpBiMxBQQFK6/b0F9+zquKA5Ycw5b/O2J3259R44ht+PXMiVhA7tQSnL1jg71kmkQhDU+LzwbD8hhYP0YxnhvX4NGwKEn7CAQCtemobG58M+sIrCtsdmTqOnC+UHTSVEnOB2nUTNTKK4rgGNi1sNomHp/wFWJLaTVJ8A5keMMsvVBZl29XwaAU3tNgN3ng3+jz945JSxCl9ZeSvcWB5IUCSOwLH96lGVyaIjUlfNf0q9zvvWqlr9gagXnNKrG73H23mfHLuody7NgEM25ZZ5Zl5Tw8xSYWXBnKFiLy+OIsDVKkwszW81yva2WgHN7Hg+K/1zbOUWTULgRM5PnhVBrQ== X-OriginatorOrg: napatech.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Jun 2024 07:39:26.7872 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 59fee880-f531-4a2b-15c1-08dc967c45c9 X-MS-Exchange-CrossTenant-Id: c4540d0b-728a-4233-9da5-9ea30c7ec3ed X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=c4540d0b-728a-4233-9da5-9ea30c7ec3ed; Ip=[178.72.21.4]; Helo=[localhost.localdomain] X-MS-Exchange-CrossTenant-AuthSource: DB1PEPF000509E2.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8P190MB1128 X-BESS-ID: 1719473977-311091-12763-2353-1 X-BESS-VER: 2019.1_20240620.2317 X-BESS-Apparent-Source-IP: 104.47.17.169 X-BESS-Parts: H4sIAAAAAAACA4uuVkqtKFGyUioBkjpK+cVKVoaGJpbGQGYGUNQ4yTAlNdnMzM LCxNjY0iDZ0DTJ0tDYyDQpySjFOCXJUqk2FgDLX1krQgAAAA== X-BESS-Outbound-Spam-Score: 0.00 X-BESS-Outbound-Spam-Report: Code version 3.2, rules version 3.2.2.257231 [from cloudscan8-116.eu-central-1a.ess.aws.cudaops.com] Rule breakdown below pts rule name description ---- ---------------------- -------------------------------- 0.00 BSF_BESS_OUTBOUND META: BESS Outbound X-BESS-Outbound-Spam-Status: SCORE=0.00 using account:ESS113687 scores of KILL_LEVEL=7.0 tests=BSF_BESS_OUTBOUND X-BESS-BRTS-Status: 1 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add structures and interfaces specific for NT smartNiC Signed-off-by: Serhii Iliushyk --- drivers/net/ntnic/include/ntdrv_4ga.h | 17 ++ drivers/net/ntnic/include/ntos_drv.h | 33 ++++ drivers/net/ntnic/include/ntos_system.h | 21 ++ drivers/net/ntnic/meson.build | 1 + drivers/net/ntnic/ntnic_ethdev.c | 243 +++++++++++++++++++++++- 5 files changed, 314 insertions(+), 1 deletion(-) create mode 100644 drivers/net/ntnic/include/ntdrv_4ga.h create mode 100644 drivers/net/ntnic/include/ntos_drv.h create mode 100644 drivers/net/ntnic/include/ntos_system.h diff --git a/drivers/net/ntnic/include/ntdrv_4ga.h b/drivers/net/ntnic/include/ntdrv_4ga.h new file mode 100644 index 0000000000..bcb7ddc242 --- /dev/null +++ b/drivers/net/ntnic/include/ntdrv_4ga.h @@ -0,0 +1,17 @@ +/* + * SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2023 Napatech A/S + */ + +#ifndef __NTDRV_4GA_H__ +#define __NTDRV_4GA_H__ + + +typedef struct ntdrv_4ga_s { + uint32_t pciident; + char *p_drv_name; + + volatile bool b_shutdown; +} ntdrv_4ga_t; + +#endif /* __NTDRV_4GA_H__ */ diff --git a/drivers/net/ntnic/include/ntos_drv.h b/drivers/net/ntnic/include/ntos_drv.h new file mode 100644 index 0000000000..aed9e83c8d --- /dev/null +++ b/drivers/net/ntnic/include/ntos_drv.h @@ -0,0 +1,33 @@ +/* + * SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2023 Napatech A/S + */ + +#ifndef __NTOS_DRV_H__ +#define __NTOS_DRV_H__ + +#include +#include +#include +#include + +#include + +#define NUM_MAC_ADDRS_PER_PORT (16U) +#define NUM_MULTICAST_ADDRS_PER_PORT (16U) + +#define NUM_ADAPTER_MAX (8) +#define NUM_ADAPTER_PORTS_MAX (128) + +struct pmd_internals { + const struct rte_pci_device *pci_dev; + char name[20]; + int n_intf_no; + int if_index; + uint32_t port; + uint32_t port_id; + struct drv_s *p_drv; + struct pmd_internals *next; +}; + +#endif /* __NTOS_DRV_H__ */ diff --git a/drivers/net/ntnic/include/ntos_system.h b/drivers/net/ntnic/include/ntos_system.h new file mode 100644 index 0000000000..74d7edf313 --- /dev/null +++ b/drivers/net/ntnic/include/ntos_system.h @@ -0,0 +1,21 @@ +/* + * SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2023 Napatech A/S + */ + +#ifndef __NTOS_SYSTEM_H__ +#define __NTOS_SYSTEM_H__ + +#include "ntdrv_4ga.h" + +struct drv_s { + int adapter_no; + struct rte_pci_device *p_dev; + struct ntdrv_4ga_s ntdrv; + + int n_eth_dev_init_count; + int probe_finished; + int setup_finished; +}; + +#endif /* __NTOS_SYSTEM_H__ */ diff --git a/drivers/net/ntnic/meson.build b/drivers/net/ntnic/meson.build index 145f586a92..ff5dd81a37 100644 --- a/drivers/net/ntnic/meson.build +++ b/drivers/net/ntnic/meson.build @@ -15,6 +15,7 @@ cflags += [ # includes includes = [ include_directories('.'), + include_directories('include'), include_directories('ntlog/include'), include_directories('ntutil/include'), ] diff --git a/drivers/net/ntnic/ntnic_ethdev.c b/drivers/net/ntnic/ntnic_ethdev.c index a4e32c30fa..7571fce12e 100644 --- a/drivers/net/ntnic/ntnic_ethdev.c +++ b/drivers/net/ntnic/ntnic_ethdev.c @@ -11,37 +11,240 @@ #include "ntlog.h" +#include "ntdrv_4ga.h" +#include "ntos_drv.h" +#include "ntos_system.h" #include "ntnic_vfio.h" #include "nt_util.h" #define EXCEPTION_PATH_HID 0 /* Global static variables: */ +static rte_spinlock_t hwlock = RTE_SPINLOCK_INITIALIZER; + +/* + * Store and get adapter info + */ + +static struct drv_s *_g_p_drv[NUM_ADAPTER_MAX] = { NULL }; + +static void +store_pdrv(struct drv_s *p_drv) +{ + if (p_drv->adapter_no > NUM_ADAPTER_MAX) { + NT_LOG(ERR, NTNIC, + "Internal error adapter number %u out of range. Max number of adapters: %u\n", + p_drv->adapter_no, NUM_ADAPTER_MAX); + return; + } + + if (_g_p_drv[p_drv->adapter_no] != 0) { + NT_LOG(WRN, NTNIC, + "Overwriting adapter structure for PCI " PCIIDENT_PRINT_STR + " with adapter structure for PCI " PCIIDENT_PRINT_STR "\n", + PCIIDENT_TO_DOMAIN(_g_p_drv[p_drv->adapter_no]->ntdrv.pciident), + PCIIDENT_TO_BUSNR(_g_p_drv[p_drv->adapter_no]->ntdrv.pciident), + PCIIDENT_TO_DEVNR(_g_p_drv[p_drv->adapter_no]->ntdrv.pciident), + PCIIDENT_TO_FUNCNR(_g_p_drv[p_drv->adapter_no]->ntdrv.pciident), + PCIIDENT_TO_DOMAIN(p_drv->ntdrv.pciident), + PCIIDENT_TO_BUSNR(p_drv->ntdrv.pciident), + PCIIDENT_TO_DEVNR(p_drv->ntdrv.pciident), + PCIIDENT_TO_FUNCNR(p_drv->ntdrv.pciident)); + } + + rte_spinlock_lock(&hwlock); + _g_p_drv[p_drv->adapter_no] = p_drv; + rte_spinlock_unlock(&hwlock); +} + +static void +clear_pdrv(struct drv_s *p_drv) +{ + if (p_drv->adapter_no > NUM_ADAPTER_MAX) + return; + + rte_spinlock_lock(&hwlock); + _g_p_drv[p_drv->adapter_no] = NULL; + rte_spinlock_unlock(&hwlock); +} + +static struct drv_s * +get_pdrv_from_pci(struct rte_pci_addr addr) +{ + int i; + struct drv_s *p_drv = NULL; + rte_spinlock_lock(&hwlock); + + for (i = 0; i < NUM_ADAPTER_MAX; i++) { + if (_g_p_drv[i]) { + if (PCIIDENT_TO_DOMAIN(_g_p_drv[i]->ntdrv.pciident) == addr.domain && + PCIIDENT_TO_BUSNR(_g_p_drv[i]->ntdrv.pciident) == addr.bus) { + p_drv = _g_p_drv[i]; + break; + } + } + } + + rte_spinlock_unlock(&hwlock); + return p_drv; +} + +static int +eth_dev_infos_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *dev_info) +{ + struct pmd_internals *internals = (struct pmd_internals *)eth_dev->data->dev_private; + + dev_info->if_index = internals->if_index; + dev_info->driver_name = internals->name; + + return 0; +} + +static int +eth_dev_configure(struct rte_eth_dev *eth_dev) +{ + struct pmd_internals *internals = (struct pmd_internals *)eth_dev->data->dev_private; + struct drv_s *p_drv = internals->p_drv; + + NT_LOG_DBGX(DEBUG, NTNIC, "Called for eth_dev %p\n", eth_dev); + + p_drv->probe_finished = 1; + + /* The device is ALWAYS running promiscuous mode. */ + eth_dev->data->promiscuous ^= ~eth_dev->data->promiscuous; + return 0; +} + +static int +eth_dev_start(struct rte_eth_dev *eth_dev) +{ + struct pmd_internals *internals = (struct pmd_internals *)eth_dev->data->dev_private; + + NT_LOG_DBGX(DEBUG, NTNIC, "Port %u, %u\n", internals->n_intf_no, internals->if_index); + + return 0; +} + +static int +eth_dev_stop(struct rte_eth_dev *eth_dev) +{ + struct pmd_internals *internals = (struct pmd_internals *)eth_dev->data->dev_private; + + NT_LOG_DBGX(DEBUG, NTNIC, "Port %u, %u\n", + internals->n_intf_no, internals->if_index); + + eth_dev->data->dev_link.link_status = RTE_ETH_LINK_DOWN; + return 0; +} + +static void +drv_deinit(struct drv_s *p_drv) +{ + if (p_drv == NULL) + return; + /* + * Mark the global pdrv for cleared. Used by some threads to terminate. + * 1 second to give the threads a chance to see the termonation. + */ + clear_pdrv(p_drv); + nt_os_wait_usec(1000000); + + /* clean memory */ + rte_free(p_drv); + p_drv = NULL; +} + +static int +eth_dev_close(struct rte_eth_dev *eth_dev) +{ + struct pmd_internals *internals = (struct pmd_internals *)eth_dev->data->dev_private; + struct drv_s *p_drv = internals->p_drv; + + internals->p_drv = NULL; + + rte_free(internals); + internals = NULL; + eth_dev->data->dev_private = NULL; + eth_dev->data->mac_addrs = NULL; + + rte_eth_dev_release_port(eth_dev); + + /* decrease initialized ethernet devices */ + p_drv->n_eth_dev_init_count--; + + /* + * rte_pci_dev has no private member for p_drv + * wait until all rte_eth_dev's are closed - then close adapters via p_drv + */ + if (!p_drv->n_eth_dev_init_count && p_drv) + drv_deinit(p_drv); + + return 0; +} + +static struct eth_dev_ops nthw_eth_dev_ops = { + .dev_configure = eth_dev_configure, + .dev_start = eth_dev_start, + .dev_stop = eth_dev_stop, + .dev_close = eth_dev_close, + .dev_infos_get = eth_dev_infos_get, +}; static int nthw_pci_dev_init(struct rte_pci_device *pci_dev) { nt_vfio_init(); + struct drv_s *p_drv; + ntdrv_4ga_t *p_nt_drv; uint32_t n_port_mask = -1; /* All ports enabled by default */ + uint32_t nb_rx_queues = 1; + uint32_t nb_tx_queues = 1; int n_phy_ports; NT_LOG_DBGX(DEBUG, NTNIC, "Dev %s PF #%i Init : %02x:%02x:%i\n", pci_dev->name, pci_dev->addr.function, pci_dev->addr.bus, pci_dev->addr.devid, pci_dev->addr.function); + /* alloc */ + p_drv = rte_zmalloc_socket(pci_dev->name, sizeof(struct drv_s), RTE_CACHE_LINE_SIZE, + pci_dev->device.numa_node); + + if (!p_drv) { + NT_LOG_DBGX(ERR, NTNIC, "%s: error %d\n", + (pci_dev->name[0] ? pci_dev->name : "NA"), -1); + return -1; + } + /* Setup VFIO context */ int vfio = nt_vfio_setup(pci_dev); if (vfio < 0) { NT_LOG_DBGX(ERR, TNIC, "%s: vfio_setup error %d\n", (pci_dev->name[0] ? pci_dev->name : "NA"), -1); + rte_free(p_drv); return -1; } + /* context */ + p_nt_drv = &p_drv->ntdrv; + + p_drv->p_dev = pci_dev; + + /* Set context for NtDrv */ + p_nt_drv->pciident = BDF_TO_PCIIDENT(pci_dev->addr.domain, pci_dev->addr.bus, + pci_dev->addr.devid, pci_dev->addr.function); + + + p_nt_drv->b_shutdown = false; + + /* store context */ + store_pdrv(p_drv); + n_phy_ports = 0; for (int n_intf_no = 0; n_intf_no < n_phy_ports; n_intf_no++) { + struct pmd_internals *internals = NULL; struct rte_eth_dev *eth_dev = NULL; char name[32]; @@ -50,6 +253,33 @@ nthw_pci_dev_init(struct rte_pci_device *pci_dev) snprintf(name, sizeof(name), "ntnic%d", n_intf_no); + internals = rte_zmalloc_socket(name, sizeof(struct pmd_internals), + RTE_CACHE_LINE_SIZE, pci_dev->device.numa_node); + + if (!internals) { + NT_LOG_DBGX(ERR, NTNIC, "%s: %s: error=%d\n", + (pci_dev->name[0] ? pci_dev->name : "NA"), name, -1); + return -1; + } + + internals->pci_dev = pci_dev; + internals->n_intf_no = n_intf_no; + internals->if_index = n_intf_no; + + + /* Setup queue_ids */ + if (nb_rx_queues > 1) { + NT_LOG(DBG, NTNIC, + "(%i) NTNIC configured with Rx multi queues. %i queues\n", + 0 /*port*/, nb_rx_queues); + } + + if (nb_tx_queues > 1) { + NT_LOG(DBG, NTNIC, + "(%i) NTNIC configured with Tx multi queues. %i queues\n", + 0 /*port*/, nb_tx_queues); + } + eth_dev = rte_eth_dev_allocate(name); /* TODO: name */ if (!eth_dev) { @@ -61,6 +291,11 @@ nthw_pci_dev_init(struct rte_pci_device *pci_dev) NT_LOG_DBGX(DEBUG, NTNIC, "eth_dev %p, port_id %u, if_index %u\n", eth_dev, eth_dev->data->port_id, n_intf_no); + /* connect structs */ + internals->p_drv = p_drv; + eth_dev->data->dev_private = internals; + + internals->port_id = eth_dev->data->port_id; struct rte_eth_link pmd_link; pmd_link.link_speed = RTE_ETH_SPEED_NUM_NONE; @@ -71,7 +306,7 @@ nthw_pci_dev_init(struct rte_pci_device *pci_dev) eth_dev->device = &pci_dev->device; eth_dev->data->dev_link = pmd_link; eth_dev->data->numa_node = pci_dev->device.numa_node; - eth_dev->dev_ops = NULL; + eth_dev->dev_ops = &nthw_eth_dev_ops; eth_dev->state = RTE_ETH_DEV_ATTACHED; rte_eth_copy_pci_info(eth_dev, pci_dev); @@ -79,8 +314,11 @@ nthw_pci_dev_init(struct rte_pci_device *pci_dev) eth_dev_pci_specific_init(eth_dev, pci_dev); /* increase initialized ethernet devices - PF */ + p_drv->n_eth_dev_init_count++; } + p_drv->setup_finished = 1; + return 0; } @@ -156,6 +394,9 @@ nthw_pci_remove(struct rte_pci_device *pci_dev) { NT_LOG_DBGX(DEBUG, NTNIC); + struct drv_s *p_drv = get_pdrv_from_pci(pci_dev->addr); + drv_deinit(p_drv); + return rte_eth_dev_pci_generic_remove(pci_dev, nthw_pci_dev_deinit); } -- 2.45.0