From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4765D431EB; Tue, 24 Oct 2023 04:32:28 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 35BFA42D83; Tue, 24 Oct 2023 04:30:19 +0200 (CEST) Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2111.outbound.protection.outlook.com [40.107.223.111]) by mails.dpdk.org (Postfix) with ESMTP id 728B642DEA for ; Tue, 24 Oct 2023 04:30:14 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=F4k/9x1Ir93zlvneOp7+olPN4Vjv5D9DKDNxEW0vLRJkGNd4klDdymcX69hVIQiZO86AHeYS89iqxgLPqdJG3+burgm/QwrEH6MSqfC/d0GZ1K0ACbuYt6dVbPXkoGHUqVQWavb7cbK+hjpoTknGkKWrQ15k1raowGdnHQUG2qD5DGfVXHUrnO0Fooj/4YAGwxWT2cp2p3fLGUHscuVF9yQnnQtH/3rp0Unvyj/HKysH2MXzuarz3exzY/z3PL+xjCh+QDwQ2+rUWe36bcsIbhaKMdU1H3dgbXJ4cl2ecLeZ2v7ZR+wAI3PgDpzrA+MKT8ezxnJd4fZQnHS57SnmGw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=7o2cFy4rIypdWxZm+ueiKs8obohKXirAMUwS2FczWOg=; b=dyfgJAq8t9pLtZFFlrqcYxtubNx4St/1UsiFqVGe2V4EgAUF61kWH5+OfUHTnDaAw2+hRolu0n5Y4t19je5YY+AKt/JFiAZVedaMgeLm1mM02ETwGXxoegFsGzQkyRc8m1oimeh7RZt0LpFLQX+8NmhM0LsCD/6ppC/gubAdB24puFoVTzMa3Qki80gPtwyUsm5tGakcP4nsG1dWN9DBwN50QRew8vsghC6sp9btupI28/KHjUGm+bhtLXaIRp9ifU7cYcaOZiP6yY1dBDXYM7fGDGRYpqq2ScDW6FrFEbyqcxMjbUAY5cTu349PpxPe7G/sxRfLMa61BBv4TEtb3A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=corigine.com; dmarc=pass action=none header.from=corigine.com; dkim=pass header.d=corigine.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=corigine.onmicrosoft.com; s=selector2-corigine-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=7o2cFy4rIypdWxZm+ueiKs8obohKXirAMUwS2FczWOg=; b=pITKA1bgMEgrCb3NAytwhP7Ly/XSx+lTnRjSQy1hA5yfanSfy3e3G4GizhGPmQqyfirS/QfOnFLTNPOTmOYa/fw8XjTn5ZJ4fEtvIwGU2wF/nYOXZv0nok5suzc+rKp0SW+UV0ySg/glMycsX66zF+BI2J/Ee3vY733tldQI75w= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=corigine.com; Received: from SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) by PH8PR13MB6182.namprd13.prod.outlook.com (2603:10b6:510:259::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6907.21; Tue, 24 Oct 2023 02:30:13 +0000 Received: from SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::142e:d47c:7320:8a9d]) by SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::142e:d47c:7320:8a9d%4]) with mapi id 15.20.6907.021; Tue, 24 Oct 2023 02:30:13 +0000 From: Chaoyong He To: dev@dpdk.org Cc: oss-drivers@corigine.com, Chaoyong He , Shujing Dong , Long Wu , Peng Zhang Subject: [PATCH v2 19/25] drivers: add the basic framework of vDPA PMD Date: Tue, 24 Oct 2023 10:28:43 +0800 Message-Id: <20231024022849.3092506-20-chaoyong.he@corigine.com> X-Mailer: git-send-email 2.39.1 In-Reply-To: <20231024022849.3092506-1-chaoyong.he@corigine.com> References: <20231017054545.1692509-1-chaoyong.he@corigine.com> <20231024022849.3092506-1-chaoyong.he@corigine.com> Content-Transfer-Encoding: 8bit Content-Type: text/plain X-ClientProxiedBy: BYAPR04CA0015.namprd04.prod.outlook.com (2603:10b6:a03:40::28) To SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ0PR13MB5545:EE_|PH8PR13MB6182:EE_ X-MS-Office365-Filtering-Correlation-Id: 9f467df0-6bda-4389-e5a0-08dbd439267c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: LlP0q1XaA/5nkNnavmpxR3tYGiq2tPTOT25DA5cS6HbjCTlnKTQMuYEa0e3aUIJUoS4LR+X+VRiX3SzZQuPF6laG7IzE72GhaSNMR4579mOh1ntIhtf8TgwJb+Nbi8aJb8WfsUJAHBD9sx/LuQa/6psoNQd1C0tJvKSctTS0Fq7uEGWhoytysW+4xH47yCPzAKjtTmTxvwp9QEtTu1OCFe1mwhiF8nyvy+qTrenhqAHWYjUpYIAFeyIj1+tNxhgCZzl4N/qm1axKqjzIc6xFihwoCQrBKE9fMAsKKaXHv6MlfWHE1sGMp2xxktyc6XQGNuTW1GSlXYh7fcMPLP/IBP6cuYOal5Lx+8wXA1bVw2NBqjl4nHruMhHFY9GOb505a8nwuVDQ23ECDM8dmymAmESJJmbHPRDfjGV6gLU9NurC3k4+y7fxDkSHqeqNUKFWi21c2n4E2a5i3Jk0AGIHm5dpBNbLXLFyg10vQuzxplAkqUyWQ1jcQR2HjgMNnGAJ/hORMy2hi7d0edqfuE0AjuoFMju9LsOq84ViwLfPThEVJXpEbLD6SDTFuFVVvhaSscC9dLE8MCYgAoCjVPVwYsJ42WRIQ5CJO7tLx+LY0pRC4kySlrUxJc9Frx3xSNbSH598hK1z+v6lTPVz8CfNdwZ1BMK7kKu/fPWhFOfltFw= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ0PR13MB5545.namprd13.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(366004)(396003)(376002)(39830400003)(346002)(136003)(230922051799003)(1800799009)(64100799003)(186009)(451199024)(86362001)(66946007)(316002)(2906002)(54906003)(66476007)(6916009)(66556008)(5660300002)(8936002)(44832011)(8676002)(4326008)(36756003)(41300700001)(2616005)(38100700002)(478600001)(1076003)(52116002)(6506007)(107886003)(6666004)(38350700005)(6512007)(26005)(83380400001)(6486002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?anVCx9pFlDZJ2nOANO/Ux7aDz+XjHRhEl8zzOEN1tC8w8GErS1kot+io0ti6?= =?us-ascii?Q?P6KXmTEoOXBLFvFICv7h3AQDDqlEtuzdyEH2FLohhBXiZImgwMAXCV540NjT?= =?us-ascii?Q?rsid3tAwgZcKhDO/2ngMkfoeUwLXkvpmNDx3KY/57wI3gR0UunFTZMuIue5R?= =?us-ascii?Q?ChrqdD1XvD0/dIKp4yr62hmuQ4cuzP9tT2g5aFsd9kL/FKMZcSaxWKCHLzuk?= =?us-ascii?Q?8V5BWwAo09cUi07y/I/8dd5AtQB6VZN8ZkAABfW/luZnds1osVN4hx7BNee9?= =?us-ascii?Q?z/3QrO2Ik6uzD4t5mhSZtu36TY1ivlqZQWO3DXccYkS3hW+8qzVx335aVedO?= =?us-ascii?Q?vn29BSD9dUM51luO2wuDT5SnigWWifjzgYJ6ev9BmI+IM5U9HmIFnaUzMxAk?= =?us-ascii?Q?4zzqHIKSd5iF1qrZ68u6EG1VdPMxajXtNMPU57zUmY9du8jpSXMyrsDNV5sH?= =?us-ascii?Q?fBsYzsUsEY+jbaaCoKBjw/IpgIHbBa7jpk6gZnt0jah8dROA1sqMtwh4x2xx?= =?us-ascii?Q?dmhgbfr1BdT60Im5URkVhfW5FMFqBTg35yEsNaUTNbZ8fqfrFc/Sb3OHob8r?= =?us-ascii?Q?vCmyrUBPplXVCW3LyjC/zfczikWtpHg8YkSNQe80LFOVU1R6ebBOtM4C1hlV?= =?us-ascii?Q?aq2ls5VR7D9mftQzZVzAsn6MN06+UJITIUKSn43oKCqxv6IHfrL09cHtriUL?= =?us-ascii?Q?rhykmFy8mrRfX94h75GGW3LofT+28an2MJjTVOSmPJn9+vmLKIgpFHn92u90?= =?us-ascii?Q?QqUzklS1vRQFfEkjR1N88NAz68sP8PjtbhNYbjqcCYgkjh2g0fGFslSdxVGt?= =?us-ascii?Q?mGyYqA5yuJIQaRhyInMcGY0xjCfEw1NWyON27CNmHe+STP87X+0/+bybIO5E?= =?us-ascii?Q?nAOxtFQp/YiEooBa6rf72S1w2K/Bgs+JZuvwHk66Lknm/Uu90V5iKKnkMqwv?= =?us-ascii?Q?rv3SKmszpF/cmwRDbAZGy+Aj6ZEvXxR0q91nYZWgJclRNmSM83uG2/9RV710?= =?us-ascii?Q?Nj4jDsHgYh7PW9QAAZfplrzxC86rZhJ0djdDHPIZzySHaWRKsFvxdhjTbDk8?= =?us-ascii?Q?xSHgcrnWpF5j7qzMD08NA/breNE4w188Y1SFHZ8PzKSh29X6hxycwEEBDiyR?= =?us-ascii?Q?r9VVOkNwp6NromXCIP4ba5Q8x0SBCMotgslXeX19cOcuy23E3+AdpBFruYrb?= =?us-ascii?Q?VV7ukd3FDngbcK/B0k5VNMChOJYJRvuofUdr7g8/3lin5N6KqT/JlZutvSOl?= =?us-ascii?Q?TTuvzIR9LzcDNL+OWA3yCa9V3XOifO7aP1BQJfBlu6t/DNubgoonQvkjaK0V?= =?us-ascii?Q?+oeTk8E5QH1DCx1OqJVdf9Y+SH8q3CbYcLlD/qMkSHFyr6+Y1DvwA8QcIQoE?= =?us-ascii?Q?xLOMz9wmLFGPJaRzy4Fg0GrUUDECjtxpk3+CUancWS8obIJIHdhqniezfoIH?= =?us-ascii?Q?3+XRBOldM2KEXAKhFYQsnUqgH1D0EujFTNf5JEz3vVKmognheFMGxJr3UvT6?= =?us-ascii?Q?VdzF+Xmx+Vz/4V9SS3rWibOx6yI1koG6LDAIF3WsmeAvV+f59VRvjU+1ulxV?= =?us-ascii?Q?V+PazG4SMfzoc/6HjUzUrZM8SHfEndoivqH87sTM5dUoj6ABudQHiZMLxFwb?= =?us-ascii?Q?sA=3D=3D?= X-OriginatorOrg: corigine.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9f467df0-6bda-4389-e5a0-08dbd439267c X-MS-Exchange-CrossTenant-AuthSource: SJ0PR13MB5545.namprd13.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Oct 2023 02:30:12.9844 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: NcZuNHsiSdHlqUZa3j60D31NJzxY0euypSm739nBUZ1G9AbV3j1Fnm+bgUiZaAPCL4P8gaPbiMttV8kytiaSl43k9bDPNVZcxj9NbevnA1c= X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH8PR13MB6182 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add the basic framework of vDPA PMD. Signed-off-by: Chaoyong He Signed-off-by: Shujing Dong Reviewed-by: Long Wu Reviewed-by: Peng Zhang --- drivers/common/nfp/nfp_common_pci.c | 1 + drivers/common/nfp/nfp_common_pci.h | 1 + drivers/vdpa/nfp/meson.build | 7 ++ drivers/vdpa/nfp/nfp_vdpa.c | 153 ++++++++++++++++++++++++++++ 4 files changed, 162 insertions(+) create mode 100644 drivers/vdpa/nfp/nfp_vdpa.c diff --git a/drivers/common/nfp/nfp_common_pci.c b/drivers/common/nfp/nfp_common_pci.c index 1aa8dd4bbd..c73f4888a9 100644 --- a/drivers/common/nfp/nfp_common_pci.c +++ b/drivers/common/nfp/nfp_common_pci.c @@ -32,6 +32,7 @@ static const struct { enum nfp_class drv_class; } nfp_classes[] = { { .name = "eth", .drv_class = NFP_CLASS_ETH }, + { .name = "vdpa", .drv_class = NFP_CLASS_VDPA }, }; static enum nfp_class diff --git a/drivers/common/nfp/nfp_common_pci.h b/drivers/common/nfp/nfp_common_pci.h index 21465fca68..80c027c3a5 100644 --- a/drivers/common/nfp/nfp_common_pci.h +++ b/drivers/common/nfp/nfp_common_pci.h @@ -16,6 +16,7 @@ typedef int (nfp_class_driver_remove_t)(struct rte_pci_device *dev); enum nfp_class { NFP_CLASS_ETH, + NFP_CLASS_VDPA, NFP_CLASS_INVALID, }; diff --git a/drivers/vdpa/nfp/meson.build b/drivers/vdpa/nfp/meson.build index b17866e157..e1cc244578 100644 --- a/drivers/vdpa/nfp/meson.build +++ b/drivers/vdpa/nfp/meson.build @@ -12,5 +12,12 @@ if not dpdk_conf.has('RTE_LIB_VHOST') endif sources = files( + 'nfp_vdpa.c', 'nfp_vdpa_log.c', ) + +deps += ['common_nfp'] + +if not dpdk_conf.has('RTE_COMMON_NFP') + error('Missing internal dependency "common/nfp"') +endif diff --git a/drivers/vdpa/nfp/nfp_vdpa.c b/drivers/vdpa/nfp/nfp_vdpa.c new file mode 100644 index 0000000000..ef4ae4946a --- /dev/null +++ b/drivers/vdpa/nfp/nfp_vdpa.c @@ -0,0 +1,153 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2023 Corigine, Inc. + * All rights reserved. + */ + +#include + +#include +#include +#include + +#include "nfp_vdpa_log.h" + +#define NFP_VDPA_DRIVER_NAME nfp_vdpa + +struct nfp_vdpa_dev { + struct rte_pci_device *pci_dev; + struct rte_vdpa_device *vdev; +}; + +struct nfp_vdpa_dev_node { + TAILQ_ENTRY(nfp_vdpa_dev_node) next; + struct nfp_vdpa_dev *device; +}; + +TAILQ_HEAD(vdpa_dev_list_head, nfp_vdpa_dev_node); + +static struct vdpa_dev_list_head vdpa_dev_list = + TAILQ_HEAD_INITIALIZER(vdpa_dev_list); + +static pthread_mutex_t vdpa_list_lock = PTHREAD_MUTEX_INITIALIZER; + +static struct nfp_vdpa_dev_node * +nfp_vdpa_find_node_by_pdev(struct rte_pci_device *pdev) +{ + bool found = false; + struct nfp_vdpa_dev_node *node; + + pthread_mutex_lock(&vdpa_list_lock); + + TAILQ_FOREACH(node, &vdpa_dev_list, next) { + if (pdev == node->device->pci_dev) { + found = true; + break; + } + } + + pthread_mutex_unlock(&vdpa_list_lock); + + if (found) + return node; + + return NULL; +} + +struct rte_vdpa_dev_ops nfp_vdpa_ops = { +}; + +static int +nfp_vdpa_pci_probe(struct rte_pci_device *pci_dev) +{ + struct nfp_vdpa_dev *device; + struct nfp_vdpa_dev_node *node; + + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return 0; + + node = calloc(1, sizeof(*node)); + if (node == NULL) + return -ENOMEM; + + device = calloc(1, sizeof(*device)); + if (device == NULL) + goto free_node; + + device->pci_dev = pci_dev; + + device->vdev = rte_vdpa_register_device(&pci_dev->device, &nfp_vdpa_ops); + if (device->vdev == NULL) { + DRV_VDPA_LOG(ERR, "Failed to register device %s", pci_dev->name); + goto free_device; + } + + node->device = device; + pthread_mutex_lock(&vdpa_list_lock); + TAILQ_INSERT_TAIL(&vdpa_dev_list, node, next); + pthread_mutex_unlock(&vdpa_list_lock); + + return 0; + +free_device: + free(device); +free_node: + free(node); + + return -1; +} + +static int +nfp_vdpa_pci_remove(struct rte_pci_device *pci_dev) +{ + struct nfp_vdpa_dev *device; + struct nfp_vdpa_dev_node *node; + + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return 0; + + node = nfp_vdpa_find_node_by_pdev(pci_dev); + if (node == NULL) { + DRV_VDPA_LOG(ERR, "Invalid device: %s", pci_dev->name); + return -ENODEV; + } + + device = node->device; + + pthread_mutex_lock(&vdpa_list_lock); + TAILQ_REMOVE(&vdpa_dev_list, node, next); + pthread_mutex_unlock(&vdpa_list_lock); + + rte_vdpa_unregister_device(device->vdev); + + free(device); + free(node); + + return 0; +} + +static const struct rte_pci_id pci_id_nfp_vdpa_map[] = { + { + RTE_PCI_DEVICE(PCI_VENDOR_ID_NETRONOME, + PCI_DEVICE_ID_NFP6000_VF_NIC) + }, + { + .vendor_id = 0, + }, +}; + +static struct nfp_class_driver nfp_vdpa = { + .drv_class = NFP_CLASS_VDPA, + .name = RTE_STR(NFP_VDPA_DRIVER_NAME), + .id_table = pci_id_nfp_vdpa_map, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC, + .probe = nfp_vdpa_pci_probe, + .remove = nfp_vdpa_pci_remove, +}; + +RTE_INIT(nfp_vdpa_init) +{ + nfp_class_driver_register(&nfp_vdpa); +} + +RTE_PMD_REGISTER_PCI_TABLE(NFP_VDPA_DRIVER_NAME, pci_id_nfp_vdpa_map); +RTE_PMD_REGISTER_KMOD_DEP(NFP_VDPA_DRIVER_NAME, "* vfio-pci"); -- 2.39.1