From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 38CC2A0548; Thu, 27 May 2021 15:38:59 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A34B941101; Thu, 27 May 2021 15:38:57 +0200 (CEST) Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2061.outbound.protection.outlook.com [40.107.237.61]) by mails.dpdk.org (Postfix) with ESMTP id 29991410F9 for ; Thu, 27 May 2021 15:38:56 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=AZjXFEtZD3W8vvBIVjcdFCNDlqA41UQMQ/MixLzt2iU2oVMCWUIajTbIvbXmh5P63z8BbyYs+seTJaXg2VKHsdeXze3YtNC3z6byae6Mx6jIvLhyKzKq1E5JSMFk20nSUoHMWLV1KoKY4dXz7nZ55qBtC26CWBJwiK9kK35Xj9teH/ZafJglLt9SWwGDDvjc/uBrgNsqY780X4MXOX1TwIdIRrU0fR6uyDksKEjV08FA/8cYgz24cMoEerPLAjbjUROia2MjWv6g+d2IQz8BvpMUxxox5cpFmZTZNOUDRwsikuWjZpv3czdpW8WvYYdesVXsAzWQPKNsHACwPav7+Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Oqt8AtawMf/fTmSFD8JvOCYXPzxC0OxAQf+t6iFkAfo=; b=IFX2eq7TeCO6jRYcOkdKUzWzDz7xWth5sYeQTYUNCH/Wl0DdZoGP/rQFmv8IxBdv7LUGEpcakkDOAXJkUULdKb+2W7cZ5tFj6+AtTh8KrSQK///uHbtBgJ/3ZiRfdhskqBxku42AeYFKnfYH6GNSEEKWyH8TXU9uurNliqXZnEahvbG9Y1yO4tDkckS0CiiDluJWfqOXMbMJEJqnbV5HCOEq2VegcT4qyGigGSov3uZld1tWTk6+33Nr8PisGac/Lu+5+D4/NLzQ5SyCQQnK8lES7UuHqV9cwhNOuMwvi1yF/xr2CvEntRlV9itw0OwrPK5RSe1GrmrhncHPPC6uAQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=tuxdriver.com smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Oqt8AtawMf/fTmSFD8JvOCYXPzxC0OxAQf+t6iFkAfo=; b=nMYT5bGl7DotGFFaHEs/KgzaYEP4zrHo20VIcJAIievAcSck/+K9h/awUerH41lrWbeKELFoayiR0psHTskWW3EBfAZnYYd9/S3V9QUvPgs9O+gewg0U7gisxGTaiF8UfZKiZsAp2BGIpbpBK5FCtkagJapSuy3DNb1yA9xY25v3GwXsJxfkUVY6Bw541cD/4atCNzO1nuKsASp9r2D5eijil7h5dUkK/wnTDLi4JqUX1weWx4HuzHtTIe1jn4xZ+5OLbBR+qCSuVSHYV6kshONlFnNRc+ui9R9WgEtuxvaHc771UPPFt+XnTGdhg++d0lq9PQwG7k5USz9eKrQfBw== Received: from CO2PR05CA0055.namprd05.prod.outlook.com (2603:10b6:102:2::23) by CH2PR12MB4278.namprd12.prod.outlook.com (2603:10b6:610:ab::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4173.20; Thu, 27 May 2021 13:38:54 +0000 Received: from CO1NAM11FT063.eop-nam11.prod.protection.outlook.com (2603:10b6:102:2:cafe::ab) by CO2PR05CA0055.outlook.office365.com (2603:10b6:102:2::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4195.10 via Frontend Transport; Thu, 27 May 2021 13:38:54 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; tuxdriver.com; dkim=none (message not signed) header.d=none;tuxdriver.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by CO1NAM11FT063.mail.protection.outlook.com (10.13.175.37) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4150.30 via Frontend Transport; Thu, 27 May 2021 13:38:53 +0000 Received: from nvidia.com (172.20.145.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 27 May 2021 13:38:50 +0000 From: Xueming Li To: Viacheslav Ovsiienko CC: , , Matan Azrad , Shahaf Shuler , Ray Kinsella , Neil Horman Date: Thu, 27 May 2021 16:37:46 +0300 Message-ID: <20210527133759.17401-2-xuemingl@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210527133759.17401-1-xuemingl@nvidia.com> References: <20210527133759.17401-1-xuemingl@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [172.20.145.6] X-ClientProxiedBy: HQMAIL111.nvidia.com (172.20.187.18) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 96ea216a-faeb-4b0d-bc57-08d92114c4fc X-MS-TrafficTypeDiagnostic: CH2PR12MB4278: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6108; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: cpFSOi+Bi2T0kkqVu9b4aWpLGo/sNYyF/NRLkxKPucbcD7eSkiwfxbGTpiVct6QzejJkLobQ6O0LYIQQ6UKnewhvqq+41RRMRjiI8DcsLw7XxH+YYDu0QgZD5gay53HnaxdmO/XtJ5n3El/YK6tWMIDF4Wy6aStxkcCF1BKpuDydnlq5uaU5Z3gede9+pqlg8gSlcmpyejlzUt3eDNF8c0D9+hZlaCSuOIWuNeS3iqBqWol3lgEXu4ehZLwShVu/GOGimYmwZx9gEPNc15LDMKPgmlM4+t2cUexnnXLrz4JkHvhvyKHS0G7bpep9sHibzkfTiOYMdEN2AcimMvCf2eZQNJv/Vnk9Ygr3ATngMQ/HBl8Is42kHX2waEnjdAMn+Ua2w4cj6SraY+gIQDr9/IdPIZsDOQ4GHrOFXE11xhhNpvGuKmWSA86KLDTbI4nAk/rvzf1GNhOk2KYYDnPfO2F0biOCddzIcORfGYHh+T155rDpBsPe1cfgkhtoHHevftKh15gFlTtzA0e0gGpohkZxt9Kb3Vnk8YpuJ85M+CyIJojc24c/BsHtVXsGuHS3zbktdB7XhJmXe86ZSQC2I0EFFvrDzSFKWBETuLtdgmzbJV8MS8z1LnL51uWbGJUuOUCL3DQ/Rraiqiq7gccapT4BryE4mzTGXnyQlB0G4Eo= X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(346002)(136003)(376002)(39860400002)(396003)(36840700001)(46966006)(83380400001)(6636002)(8676002)(6286002)(478600001)(30864003)(6862004)(8936002)(82310400003)(36906005)(55016002)(336012)(426003)(5660300002)(36860700001)(7696005)(86362001)(82740400003)(7636003)(1076003)(37006003)(356005)(70586007)(316002)(26005)(36756003)(6666004)(54906003)(2906002)(16526019)(186003)(2616005)(4326008)(47076005)(70206006); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 May 2021 13:38:53.6546 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 96ea216a-faeb-4b0d-bc57-08d92114c4fc X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT063.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR12MB4278 Subject: [dpdk-dev] [RFC 01/14] common/mlx5: add common device driver X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" To support auxiliary bus, introduces common device driver and callbacks, suppose to replace current mlx5 common PCI bus driver. mlx5 common PCI bus driver still used by mlx5 eth, vDPA and regex PMD, will remove once all PMD drivers adapt to new common driver. Signed-off-by: Xueming Li --- drivers/common/mlx5/linux/mlx5_common_verbs.c | 21 +- drivers/common/mlx5/mlx5_common.c | 322 +++++++++++++++++- drivers/common/mlx5/mlx5_common.h | 133 ++++++++ drivers/common/mlx5/mlx5_common_pci.c | 165 ++++++++- drivers/common/mlx5/mlx5_common_private.h | 42 +++ drivers/common/mlx5/version.map | 5 + 6 files changed, 672 insertions(+), 16 deletions(-) create mode 100644 drivers/common/mlx5/mlx5_common_private.h diff --git a/drivers/common/mlx5/linux/mlx5_common_verbs.c b/drivers/common/mlx5/linux/mlx5_common_verbs.c index aa560f05f2..a49440ef72 100644 --- a/drivers/common/mlx5/linux/mlx5_common_verbs.c +++ b/drivers/common/mlx5/linux/mlx5_common_verbs.c @@ -10,11 +10,31 @@ #include #include +#include +#include + +#include "mlx5_common_log.h" +#include "mlx5_common_utils.h" +#include "mlx5_common_private.h" #include "mlx5_autoconf.h" #include #include #include +struct ibv_device * +mlx5_get_ibv_device(const struct rte_device *dev) +{ + struct ibv_device *ibv = NULL; + + if (mlx5_dev_is_pci(dev)) + ibv = mlx5_get_pci_ibv_device(&RTE_DEV_TO_PCI_CONST(dev)->addr); + if (ibv == NULL) { + rte_errno = ENODEV; + DRV_LOG(ERR, "Verbs device not found: %s", dev->name); + } + return ibv; +} + /** * Register mr. Given protection domain pointer, pointer to addr and length * register the memory region. @@ -68,4 +88,3 @@ mlx5_common_verbs_dereg_mr(struct mlx5_pmd_mr *pmd_mr) memset(pmd_mr, 0, sizeof(*pmd_mr)); } } - diff --git a/drivers/common/mlx5/mlx5_common.c b/drivers/common/mlx5/mlx5_common.c index 25e9f09108..f2e2a95ae0 100644 --- a/drivers/common/mlx5/mlx5_common.c +++ b/drivers/common/mlx5/mlx5_common.c @@ -8,11 +8,14 @@ #include #include +#include +#include #include "mlx5_common.h" #include "mlx5_common_os.h" #include "mlx5_common_log.h" #include "mlx5_common_pci.h" +#include "mlx5_common_private.h" uint8_t haswell_broadwell_cpu; @@ -41,6 +44,321 @@ static inline void mlx5_cpu_id(unsigned int level, RTE_LOG_REGISTER_DEFAULT(mlx5_common_logtype, NOTICE) +/* Head of list of drivers. */ +static TAILQ_HEAD(mlx5_drivers, mlx5_class_driver) drivers_list = + TAILQ_HEAD_INITIALIZER(drivers_list); + +/* Head of devices. */ +static TAILQ_HEAD(mlx5_devices, mlx5_common_device) devices_list = + TAILQ_HEAD_INITIALIZER(devices_list); + +static const struct { + const char *name; + unsigned int drv_class; +} mlx5_classes[] = { + { .name = "vdpa", .drv_class = MLX5_CLASS_VDPA }, + { .name = "net", .drv_class = MLX5_CLASS_NET }, + { .name = "regex", .drv_class = MLX5_CLASS_REGEX }, + { .name = "compress", .drv_class = MLX5_CLASS_COMPRESS }, +}; + +static int +class_name_to_value(const char *class_name) +{ + unsigned int i; + + for (i = 0; i < RTE_DIM(mlx5_classes); i++) { + if (strcmp(class_name, mlx5_classes[i].name) == 0) + return mlx5_classes[i].drv_class; + } + return -EINVAL; +} + +static struct mlx5_class_driver * +driver_get(uint32_t class) +{ + struct mlx5_class_driver *driver; + + TAILQ_FOREACH(driver, &drivers_list, next) { + if ((uint32_t)driver->drv_class == class) + return driver; + } + return NULL; +} + +static int +parse_class_option(const struct rte_devargs *devargs) +{ + const char *cls; + struct rte_kvargs *kvlist = NULL; + int ret = 0; + + if (devargs == NULL) + return 0; + if (devargs->cls != NULL) { + /* Global syntax. */ + cls = devargs->cls->name; + } else { + /* Legacy syntax. */ + kvlist = rte_kvargs_parse(devargs->args, NULL); + if (kvlist == NULL) + return -EINVAL; + cls = rte_kvargs_get(kvlist, RTE_DEVARGS_KEY_CLASS); + } + if (cls != NULL) + ret = class_name_to_value(cls); + if (kvlist != NULL) + rte_kvargs_free(kvlist); + return ret; +} + +static const unsigned int mlx5_class_invalid_combinations[] = { + MLX5_CLASS_NET | MLX5_CLASS_VDPA, + /* New class combination should be added here. */ +}; + +static int +is_valid_class_combination(uint32_t user_classes) +{ + unsigned int i; + + /* Verify if user specified unsupported combination. */ + for (i = 0; i < RTE_DIM(mlx5_class_invalid_combinations); i++) { + if ((mlx5_class_invalid_combinations[i] & user_classes) == + mlx5_class_invalid_combinations[i]) + return -EINVAL; + } + /* Not found any invalid class combination. */ + return 0; +} + +static bool +device_class_enabled(const struct mlx5_common_device *device, uint32_t class) +{ + return (device->classes_loaded & class) > 0; +} + +static bool +mlx5_bus_match(const struct mlx5_class_driver *drv, + const struct rte_device *dev) +{ + if (mlx5_dev_is_pci(dev)) + return mlx5_dev_pci_match(drv, dev); + return true; +} + +static struct mlx5_common_device * +to_mlx5_device(const struct rte_device *rte_dev) +{ + struct mlx5_common_device *dev; + + TAILQ_FOREACH(dev, &devices_list, next) { + if (rte_dev == dev->dev) + return dev; + } + return NULL; +} + +static void +dev_release(struct mlx5_common_device *dev) +{ + TAILQ_REMOVE(&devices_list, dev, next); + rte_free(dev); +} + +static int +drivers_remove(struct mlx5_common_device *dev, uint32_t enabled_classes) +{ + struct mlx5_class_driver *driver; + int local_ret = -ENODEV; + unsigned int i = 0; + int ret = 0; + + enabled_classes &= dev->classes_loaded; + while (enabled_classes) { + driver = driver_get(RTE_BIT64(i)); + if (driver != NULL) { + local_ret = driver->remove(dev->dev); + if (local_ret == 0) + dev->classes_loaded &= ~RTE_BIT64(i); + else if (ret == 0) + ret = local_ret; + } + enabled_classes &= ~RTE_BIT64(i); + i++; + } + if (local_ret != 0 && ret == 0) + ret = local_ret; + return ret; +} + +static int +drivers_probe(struct mlx5_common_device *dev, uint32_t user_classes) +{ + struct mlx5_class_driver *driver; + uint32_t enabled_classes = 0; + bool already_loaded; + int ret; + + TAILQ_FOREACH(driver, &drivers_list, next) { + if ((driver->drv_class & user_classes) == 0) + continue; + if (!mlx5_bus_match(driver, dev->dev)) + continue; + already_loaded = dev->classes_loaded & driver->drv_class; + if (already_loaded && driver->probe_again == 0) { + DRV_LOG(ERR, "Device %s is already probed", + dev->dev->name); + ret = -EEXIST; + goto probe_err; + } + ret = driver->probe(dev->dev); + if (ret < 0) { + DRV_LOG(ERR, "Failed to load driver %s", + driver->name); + goto probe_err; + } + enabled_classes |= driver->drv_class; + } + dev->classes_loaded |= enabled_classes; + return 0; +probe_err: + /* Only unload drivers which are enabled which were enabled + * in this probe instance. + */ + drivers_remove(dev, enabled_classes); + return ret; +} + +int +mlx5_common_dev_probe(struct rte_device *eal_dev) +{ + struct mlx5_common_device *dev; + uint32_t user_class = 0; + bool new_device = false; + int ret; + + DRV_LOG(INFO, "probe device \"%s\".", eal_dev->name); + ret = parse_class_option(eal_dev->devargs); + if (ret < 0) + return ret; + user_class = ret; + if (user_class == 0) + /* Default to net class. */ + user_class = MLX5_CLASS_NET; + dev = to_mlx5_device(eal_dev); + if (!dev) { + dev = rte_zmalloc("mlx5_common_device", sizeof(*dev), 0); + if (!dev) + return -ENOMEM; + dev->dev = eal_dev; + TAILQ_INSERT_HEAD(&devices_list, dev, next); + new_device = true; + } else { + /* Validate combination here. */ + ret = is_valid_class_combination(user_class | + dev->classes_loaded); + if (ret != 0) { + DRV_LOG(ERR, "Unsupported mlx5 classes supplied."); + return ret; + } + } + ret = drivers_probe(dev, user_class); + if (ret) + goto class_err; + return 0; +class_err: + if (new_device) + dev_release(dev); + return ret; +} + +int +mlx5_common_dev_remove(struct rte_device *eal_dev) +{ + struct mlx5_common_device *dev; + int ret; + + dev = to_mlx5_device(eal_dev); + if (!dev) + return -ENODEV; + /* Matching device found, cleanup and unload drivers. */ + ret = drivers_remove(dev, dev->classes_loaded); + if (ret != 0) + dev_release(dev); + return ret; +} + +int +mlx5_common_dev_dma_map(struct rte_device *dev, void *addr, uint64_t iova, + size_t len) +{ + struct mlx5_class_driver *driver = NULL; + struct mlx5_class_driver *temp; + struct mlx5_common_device *mdev; + int ret = -EINVAL; + + mdev = to_mlx5_device(dev); + if (!mdev) + return -ENODEV; + TAILQ_FOREACH(driver, &drivers_list, next) { + if (!device_class_enabled(mdev, driver->drv_class) || + driver->dma_map == NULL) + continue; + ret = driver->dma_map(dev, addr, iova, len); + if (ret) + goto map_err; + } + return ret; +map_err: + TAILQ_FOREACH(temp, &drivers_list, next) { + if (temp == driver) + break; + if (device_class_enabled(mdev, temp->drv_class) && + temp->dma_map && temp->dma_unmap) + temp->dma_unmap(dev, addr, iova, len); + } + return ret; +} + +int +mlx5_common_dev_dma_unmap(struct rte_device *dev, void *addr, uint64_t iova, + size_t len) +{ + struct mlx5_class_driver *driver; + struct mlx5_common_device *mdev; + int local_ret = -EINVAL; + int ret = 0; + + mdev = to_mlx5_device(dev); + if (!mdev) + return -ENODEV; + /* There is no unmap error recovery in current implementation. */ + TAILQ_FOREACH_REVERSE(driver, &drivers_list, mlx5_drivers, next) { + if (!device_class_enabled(mdev, driver->drv_class) || + driver->dma_unmap == NULL) + continue; + local_ret = driver->dma_unmap(dev, addr, iova, len); + if (local_ret && (ret == 0)) + ret = local_ret; + } + if (local_ret) + ret = local_ret; + return ret; +} + +void +mlx5_class_driver_register(struct mlx5_class_driver *driver) +{ + mlx5_common_driver_on_register_pci(driver); + TAILQ_INSERT_TAIL(&drivers_list, driver, next); +} + +static void mlx5_common_driver_init(void) +{ + mlx5_common_pci_init(); +} + static bool mlx5_common_initialized; /** @@ -55,7 +373,7 @@ mlx5_common_init(void) return; mlx5_glue_constructor(); - mlx5_common_pci_init(); + mlx5_common_driver_init(); mlx5_common_initialized = true; } @@ -214,3 +532,5 @@ mlx5_devx_alloc_uar(void *ctx, int mapping) exit: return uar; } + +RTE_PMD_EXPORT_NAME(mlx5_class_driver, __COUNTER__); diff --git a/drivers/common/mlx5/mlx5_common.h b/drivers/common/mlx5/mlx5_common.h index 306f2f1ab7..0e67a36d0c 100644 --- a/drivers/common/mlx5/mlx5_common.h +++ b/drivers/common/mlx5/mlx5_common.h @@ -10,6 +10,7 @@ #include #include #include +#include #include #include #include @@ -134,6 +135,11 @@ enum { PCI_DEVICE_ID_MELLANOX_CONNECTX7BF = 0Xa2dc, }; +struct ibv_device; + +__rte_internal +struct ibv_device *mlx5_get_ibv_device(const struct rte_device *dev); + /* Maximum number of simultaneous unicast MAC addresses. */ #define MLX5_MAX_UC_MAC_ADDRESSES 128 /* Maximum number of simultaneous Multicast MAC addresses. */ @@ -242,4 +248,131 @@ extern uint8_t haswell_broadwell_cpu; __rte_internal void mlx5_common_init(void); +/** + * Common Driver Interface + * ConnectX common driver supports multiple classes: net,vdpa,regex,crypto and + * compress devices. This layer enables creating such multiple class of devices + * on a single device by allowing to bind multiple class specific device + * driver to attach to common driver. + * + * ---------------- + * | mlx5 class | + * | drivers | + * ---------------- + * || + * ----------------- + * | mlx5 | + * | common driver | + * ----------------- + * | | + * ----------- ----------------- + * | mlx5 | | mlx5 | + * | pci dev | | auxiliary dev | + * ----------- ----------------- + * + * - mlx5 pci bus driver binds to mlx5 PCI devices defined by PCI + * ID table of all related mlx5 PCI devices. + * - mlx5 class driver such as net, vdpa, regex PMD defines its + * specific PCI ID table and mlx5 bus driver probes matching + * class drivers. + * - mlx5 common driver is central place that validates supported + * class combinations. + * - mlx5 common driver hide bus difference by resolving device address + * from devargs, locating target RDMA device and probing with it. + */ + +/** + * Initialization function for the driver called during device probing. + */ +typedef int (mlx5_class_driver_probe_t)(struct rte_device *dev); + +/** + * Uninitialization function for the driver called during hot-unplugging. + */ +typedef int (mlx5_class_driver_remove_t)(struct rte_device *dev); + +/** + * Driver-specific DMA mapping. After a successful call the device + * will be able to read/write from/to this segment. + * + * @param dev + * Pointer to the device. + * @param addr + * Starting virtual address of memory to be mapped. + * @param iova + * Starting IOVA address of memory to be mapped. + * @param len + * Length of memory segment being mapped. + * @return + * - 0 On success. + * - Negative value and rte_errno is set otherwise. + */ +typedef int (mlx5_class_driver_dma_map_t)(struct rte_device *dev, void *addr, + uint64_t iova, size_t len); + +/** + * Driver-specific DMA un-mapping. After a successful call the device + * will not be able to read/write from/to this segment. + * + * @param dev + * Pointer to the device. + * @param addr + * Starting virtual address of memory to be unmapped. + * @param iova + * Starting IOVA address of memory to be unmapped. + * @param len + * Length of memory segment being unmapped. + * @return + * - 0 On success. + * - Negative value and rte_errno is set otherwise. + */ +typedef int (mlx5_class_driver_dma_unmap_t)(struct rte_device *dev, void *addr, + uint64_t iova, size_t len); + +/** Device already probed can be probed again to check for new ports. */ +#define MLX5_DRV_PROBE_AGAIN 0x0004 + +/** + * A structure describing a mlx5 common class driver. + */ +struct mlx5_class_driver { + TAILQ_ENTRY(mlx5_class_driver) next; + enum mlx5_class drv_class; /**< Class of this driver. */ + const char *name; /**< Driver name. */ + mlx5_class_driver_probe_t *probe; /**< Device Probe function. */ + mlx5_class_driver_remove_t *remove; /**< Device Remove function. */ + mlx5_class_driver_dma_map_t *dma_map; /**< device dma map function. */ + mlx5_class_driver_dma_unmap_t *dma_unmap; /**< device dma unmap function. */ + const struct rte_pci_id *id_table; /**< ID table, NULL terminated. */ + uint32_t probe_again:1; + /**< Device already probed can be probed again to check new device. */ + uint32_t intr_lsc:1; /**< Supports link state interrupt. */ + uint32_t intr_rmv:1; /**< Supports device remove interrupt. */ +}; + +/** + * Register a mlx5 device driver. + * + * @param driver + * A pointer to a mlx5_driver structure describing the driver + * to be registered. + */ +__rte_internal +void +mlx5_class_driver_register(struct mlx5_class_driver *driver); + +/** + * Test a device is PCI bus device. + * + * @param dev + * Pointer to device. + * + * @return + * - True on device devargs is a PCI bus device. + * - False otherwise. + */ +__rte_internal +bool +mlx5_dev_is_pci(const struct rte_device *dev); + #endif /* RTE_PMD_MLX5_COMMON_H_ */ diff --git a/drivers/common/mlx5/mlx5_common_pci.c b/drivers/common/mlx5/mlx5_common_pci.c index 34747c4e07..53090173a2 100644 --- a/drivers/common/mlx5/mlx5_common_pci.c +++ b/drivers/common/mlx5/mlx5_common_pci.c @@ -3,11 +3,19 @@ */ #include + #include +#include +#include #include #include "mlx5_common_log.h" #include "mlx5_common_pci.h" +#include "mlx5_common_private.h" + +static struct rte_pci_driver mlx5_common_pci_driver; + +/********** Legacy PCI bus driver, to be removed ********/ struct mlx5_pci_device { struct rte_pci_device *pci_dev; @@ -282,8 +290,8 @@ drivers_probe(struct mlx5_pci_device *dev, struct rte_pci_driver *pci_drv, * 0 on success, a negative errno value otherwise and rte_errno is set. */ static int -mlx5_common_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, - struct rte_pci_device *pci_dev) +mlx5_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, + struct rte_pci_device *pci_dev) { struct mlx5_pci_device *dev; uint32_t user_classes = 0; @@ -336,7 +344,7 @@ mlx5_common_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, * 0 on success, the function cannot fail. */ static int -mlx5_common_pci_remove(struct rte_pci_device *pci_dev) +mlx5_pci_remove(struct rte_pci_device *pci_dev) { struct mlx5_pci_device *dev; int ret; @@ -352,8 +360,8 @@ mlx5_common_pci_remove(struct rte_pci_device *pci_dev) } static int -mlx5_common_pci_dma_map(struct rte_pci_device *pci_dev, void *addr, - uint64_t iova, size_t len) +mlx5_pci_dma_map(struct rte_pci_device *pci_dev, void *addr, + uint64_t iova, size_t len) { struct mlx5_pci_driver *driver = NULL; struct mlx5_pci_driver *temp; @@ -385,8 +393,8 @@ mlx5_common_pci_dma_map(struct rte_pci_device *pci_dev, void *addr, } static int -mlx5_common_pci_dma_unmap(struct rte_pci_device *pci_dev, void *addr, - uint64_t iova, size_t len) +mlx5_pci_dma_unmap(struct rte_pci_device *pci_dev, void *addr, + uint64_t iova, size_t len) { struct mlx5_pci_driver *driver; struct mlx5_pci_device *dev; @@ -419,10 +427,10 @@ static struct rte_pci_driver mlx5_pci_driver = { .driver = { .name = MLX5_PCI_DRIVER_NAME, }, - .probe = mlx5_common_pci_probe, - .remove = mlx5_common_pci_remove, - .dma_map = mlx5_common_pci_dma_map, - .dma_unmap = mlx5_common_pci_dma_unmap, + .probe = mlx5_pci_probe, + .remove = mlx5_pci_remove, + .dma_map = mlx5_pci_dma_map, + .dma_unmap = mlx5_pci_dma_unmap, }; static int @@ -486,7 +494,7 @@ pci_ids_table_update(const struct rte_pci_id *driver_id_table) updated_table = calloc(num_ids, sizeof(*updated_table)); if (!updated_table) return -ENOMEM; - if (TAILQ_EMPTY(&drv_list)) { + if (old_table == NULL) { /* Copy the first driver's ID table. */ for (id_iter = driver_id_table; id_iter->vendor_id != 0; id_iter++, i++) @@ -502,6 +510,7 @@ pci_ids_table_update(const struct rte_pci_id *driver_id_table) /* Terminate table with empty entry. */ updated_table[i].vendor_id = 0; mlx5_pci_driver.id_table = updated_table; + mlx5_common_pci_driver.id_table = updated_table; mlx5_pci_id_table = updated_table; if (old_table) free(old_table); @@ -520,6 +529,133 @@ mlx5_pci_driver_register(struct mlx5_pci_driver *driver) TAILQ_INSERT_TAIL(&drv_list, driver, next); } +/********** New common PCI bus driver ********/ + +bool +mlx5_dev_is_pci(const struct rte_device *dev) +{ + struct rte_devargs *da = dev->devargs; + + if (da == NULL || da->bus == NULL) + return false; + return strcmp(da->bus->name, "pci") == 0; +} + +struct ibv_device * +mlx5_get_pci_ibv_device(const struct rte_pci_addr *addr) +{ + int n; + struct ibv_device **ibv_list = mlx5_glue->get_device_list(&n); + struct ibv_device *ibv_match = NULL; + + if (!ibv_list) { + rte_errno = ENOSYS; + return NULL; + } + while (n-- > 0) { + struct rte_pci_addr pci_addr; + + DRV_LOG(DEBUG, "Checking device \"%s\"..", ibv_list[n]->name); + if (mlx5_get_pci_addr(ibv_list[n]->ibdev_path, &pci_addr)) + continue; + if (rte_pci_addr_cmp(addr, &pci_addr)) + continue; + ibv_match = ibv_list[n]; + break; + } + if (!ibv_match) + rte_errno = ENOENT; + mlx5_glue->free_device_list(ibv_list); + return ibv_match; +} + +bool +mlx5_dev_pci_match(const struct mlx5_class_driver *drv, + const struct rte_device *dev) +{ + const struct rte_pci_device *pci_dev; + const struct rte_pci_id *id_table; + + if (!mlx5_dev_is_pci(dev)) + return false; + pci_dev = RTE_DEV_TO_PCI_CONST(dev); + for (id_table = drv->id_table; id_table->vendor_id != 0; + id_table++) { + /* Check if device's ids match the class driver's ids. */ + if (id_table->vendor_id != pci_dev->id.vendor_id && + id_table->vendor_id != RTE_PCI_ANY_ID) + continue; + if (id_table->device_id != pci_dev->id.device_id && + id_table->device_id != RTE_PCI_ANY_ID) + continue; + if (id_table->subsystem_vendor_id != + pci_dev->id.subsystem_vendor_id && + id_table->subsystem_vendor_id != RTE_PCI_ANY_ID) + continue; + if (id_table->subsystem_device_id != + pci_dev->id.subsystem_device_id && + id_table->subsystem_device_id != RTE_PCI_ANY_ID) + continue; + if (id_table->class_id != pci_dev->id.class_id && + id_table->class_id != RTE_CLASS_ANY_ID) + continue; + return true; + } + return false; +} + +static int +mlx5_common_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, + struct rte_pci_device *pci_dev) +{ + return mlx5_common_dev_probe(&pci_dev->device); +} + +static int +mlx5_common_pci_remove(struct rte_pci_device *pci_dev) +{ + return mlx5_common_dev_remove(&pci_dev->device); +} + +static int +mlx5_common_pci_dma_map(struct rte_pci_device *pci_dev, void *addr, + uint64_t iova, size_t len) +{ + return mlx5_common_dev_dma_map(&pci_dev->device, addr, iova, len); +} + +static int +mlx5_common_pci_dma_unmap(struct rte_pci_device *pci_dev, void *addr, + uint64_t iova, size_t len) +{ + return mlx5_common_dev_dma_unmap(&pci_dev->device, addr, iova, len); +} + +void +mlx5_common_driver_on_register_pci(struct mlx5_class_driver *driver) +{ + if (driver->id_table != NULL) { + if (pci_ids_table_update(driver->id_table) != 0) + return; + } + if (driver->probe_again) + mlx5_common_pci_driver.drv_flags |= RTE_PCI_DRV_PROBE_AGAIN; + if (driver->intr_lsc) + mlx5_common_pci_driver.drv_flags |= RTE_PCI_DRV_INTR_LSC; + if (driver->intr_rmv) + mlx5_common_pci_driver.drv_flags |= RTE_PCI_DRV_INTR_RMV; +} + +static struct rte_pci_driver mlx5_common_pci_driver = { + .driver = { + .name = MLX5_PCI_DRIVER_NAME, + }, + .probe = mlx5_common_pci_probe, + .remove = mlx5_common_pci_remove, + .dma_map = mlx5_common_pci_dma_map, + .dma_unmap = mlx5_common_pci_dma_unmap, +}; + void mlx5_common_pci_init(void) { const struct rte_pci_id empty_table[] = { @@ -535,7 +671,7 @@ void mlx5_common_pci_init(void) */ if (mlx5_pci_id_table == NULL && pci_ids_table_update(empty_table)) return; - rte_pci_register(&mlx5_pci_driver); + rte_pci_register(&mlx5_common_pci_driver); } RTE_FINI(mlx5_common_pci_finish) @@ -544,8 +680,9 @@ RTE_FINI(mlx5_common_pci_finish) /* Constructor doesn't register with PCI bus if it failed * to build the table. */ - rte_pci_unregister(&mlx5_pci_driver); + rte_pci_unregister(&mlx5_common_pci_driver); free(mlx5_pci_id_table); } } + RTE_PMD_EXPORT_NAME(mlx5_common_pci, __COUNTER__); diff --git a/drivers/common/mlx5/mlx5_common_private.h b/drivers/common/mlx5/mlx5_common_private.h new file mode 100644 index 0000000000..72df9aef35 --- /dev/null +++ b/drivers/common/mlx5/mlx5_common_private.h @@ -0,0 +1,42 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2021 Mellanox Technologies, Ltd + */ + +#ifndef _MLX5_COMMON_PRIVATE_H_ +#define _MLX5_COMMON_PRIVATE_H_ + +#include + +#include "mlx5_common.h" + +#ifdef __cplusplus +extern "C" { +#endif /* __cplusplus */ + +/* Common bus driver: */ + +struct mlx5_common_device { + struct rte_device *dev; + TAILQ_ENTRY(mlx5_common_device) next; + uint32_t classes_loaded; +}; + +int mlx5_common_dev_probe(struct rte_device *eal_dev); +int mlx5_common_dev_remove(struct rte_device *eal_dev); +int mlx5_common_dev_dma_map(struct rte_device *dev, void *addr, uint64_t iova, + size_t len); +int mlx5_common_dev_dma_unmap(struct rte_device *dev, void *addr, uint64_t iova, + size_t len); + +/* Common PCI bus driver: */ + +void mlx5_common_driver_on_register_pci(struct mlx5_class_driver *driver); +bool mlx5_dev_pci_match(const struct mlx5_class_driver *drv, + const struct rte_device *dev); +struct ibv_device *mlx5_get_pci_ibv_device(const struct rte_pci_addr *); + +#ifdef __cplusplus +} +#endif /* __cplusplus */ + +#endif /* _MLX5_COMMON_PRIVATE_H_ */ diff --git a/drivers/common/mlx5/version.map b/drivers/common/mlx5/version.map index db4f13f1f7..3c21719975 100644 --- a/drivers/common/mlx5/version.map +++ b/drivers/common/mlx5/version.map @@ -3,6 +3,8 @@ INTERNAL { haswell_broadwell_cpu; + mlx5_class_driver_register; + mlx5_common_init; mlx5_common_verbs_reg_mr; # WINDOWS_NO_EXPORT @@ -10,6 +12,7 @@ INTERNAL { mlx5_create_mr_ext; + mlx5_dev_is_pci; mlx5_dev_to_pci_addr; # WINDOWS_NO_EXPORT mlx5_devx_alloc_uar; # WINDOWS_NO_EXPORT @@ -69,6 +72,8 @@ INTERNAL { mlx5_free; + mlx5_get_ibv_device; # WINDOWS_NO_EXPORT + mlx5_get_ifname_sysfs; # WINDOWS_NO_EXPORT mlx5_glue; -- 2.25.1