From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id EA7E8A051E; Wed, 10 Jun 2020 19:18:57 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id D01FE1C00F; Wed, 10 Jun 2020 19:18:18 +0200 (CEST) Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05on2085.outbound.protection.outlook.com [40.107.22.85]) by dpdk.org (Postfix) with ESMTP id CEA2E1C001 for ; Wed, 10 Jun 2020 19:18:16 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SDv3L91T0FCSfsZFCSywcPzxBl9L1UABnXzBPkwGmBHYkpu5gOgosU6KoyW3UH/NV0EqGIxdl6kS0tNbFarXxveSLRiy/Xhm+riwgq75u4LbLInx85Q+kS+6tIbLCBjA5FoIOAUnC4naNxruTrfTHwIFI89hC0ndXr6DTRftEhfo3SP1ibxdCdtNyXfv1X9OCtzwp/tZCJ4Ovzj7SPJkkO+gMxSfBxNDry2gqEyEd4ybKyGeRRfas5ctFXDhSEoIUh09MN0cJ72Wmm+/toTel7I+R9GBcb4C/BQ3hlVX26eZve8TeqcCC6Ae3kmQqmN5wl4On29osRoVa79ZLfjU0g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=K+klCllpONd47z76Tr5dhpDF2C+vsILEa+yIQSRBKms=; b=NJTaJqaNgOMxStBDRaSKbk0mFwsl8j/GJ8kHNjd/UbUH/aBEzFdFTMXoD4PDQBFbAwfU9dAf1r0CpwhMO6kdtw7KYaOt2H4jnzOP3OPg0WSDmxKOeHmwGoJHEr4kcXgZB6BZNlYUUD/Rh6iqAvY6Y0jbpQRdXW4zOKDY+7atFnwgLEgFYdY05B8xJ3T0urc1cRQgWAlYVsPiDn4aXiLuefizdISnEaYk/IDW/xJgMdo34zKfTIUQvMXWext48SJ5LDVorwpkrjiH26JpLqGKcTZzVG9/TzKJj0D7+Wv6u+YLXEJZrslG1b5Wc7q+J6j5e7rYLF5ZJq9g+RBHGoVYcQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=K+klCllpONd47z76Tr5dhpDF2C+vsILEa+yIQSRBKms=; b=bsgHG+VmViFEy2ZgWWM88II86V0v50oBW2IhbwnRSBMW7XjNZ+1vpMN0cxiDOz4CW/oTeIS2fpklgjxdkjNvdbTFZD3TGrbvRd7zmUXSqiNZuN8a006iiZ8R9DMNM0qQNxVRnbJ3OhrNlG3qQSa0gwaFEX32cF6mWjY423zXmgQ= Authentication-Results: dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=none action=none header.from=mellanox.com; Received: from AM0PR05MB4866.eurprd05.prod.outlook.com (2603:10a6:208:c0::32) by AM0PR05MB5346.eurprd05.prod.outlook.com (2603:10a6:208:f5::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3066.18; Wed, 10 Jun 2020 17:18:15 +0000 Received: from AM0PR05MB4866.eurprd05.prod.outlook.com ([fe80::d44d:a804:c730:d2b7]) by AM0PR05MB4866.eurprd05.prod.outlook.com ([fe80::d44d:a804:c730:d2b7%2]) with mapi id 15.20.3066.023; Wed, 10 Jun 2020 17:18:15 +0000 From: Parav Pandit To: dev@dpdk.org Cc: ferruh.yigit@intel.com, thomasm@mellanox.com, orika@mellanox.com, matan@mellanox.com, Parav Pandit Date: Wed, 10 Jun 2020 17:17:27 +0000 Message-Id: <20200610171728.89-6-parav@mellanox.com> X-Mailer: git-send-email 2.25.4 In-Reply-To: <20200610171728.89-1-parav@mellanox.com> References: <20200610171728.89-1-parav@mellanox.com> Content-Transfer-Encoding: 8bit Content-Type: text/plain X-ClientProxiedBy: SN6PR16CA0060.namprd16.prod.outlook.com (2603:10b6:805:ca::37) To AM0PR05MB4866.eurprd05.prod.outlook.com (2603:10a6:208:c0::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from sw-mtx-036.mtx.labs.mlnx (208.176.44.194) by SN6PR16CA0060.namprd16.prod.outlook.com (2603:10b6:805:ca::37) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3088.21 via Frontend Transport; Wed, 10 Jun 2020 17:18:14 +0000 X-Mailer: git-send-email 2.25.4 X-Originating-IP: [208.176.44.194] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: b04cccab-6d63-470b-b8cf-08d80d6242cf X-MS-TrafficTypeDiagnostic: AM0PR05MB5346: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4303; X-Forefront-PRVS: 0430FA5CB7 X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: JtHaJ2Md8hNDgu+Ai0Ktn9W+lKCl7iOab84x+GdIliiSj1vLoEuJ4QBtqlNbD42OAcUbLi+4ViSk2yiPDjKMWIeoHSIpdy3/UAEAmMYM7uzBIZWO0fH0TXooNv0t9YVT03rHNKyGn3ZSV3+pnQjQhZ+RC80x1k+O35iaVXdczEwNGD1UDjBumzuu0Gd5wpnyrk3DjQLzKK7EvedLZFLDqnErSXlK+ROY5MHtgOi5bpRk1U4FzjWt0i9ljewXlrh+pbTVm3QAzDjsMjm/G39qzhPdREvfOUiretkKzik1/1lvC7l4KIb29P+fJq1y/mUY X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM0PR05MB4866.eurprd05.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(366004)(346002)(396003)(136003)(39860400002)(376002)(66476007)(6506007)(36756003)(16526019)(6486002)(6916009)(2906002)(316002)(26005)(5660300002)(186003)(66556008)(66946007)(6512007)(6666004)(8936002)(8676002)(4326008)(2616005)(83380400001)(956004)(478600001)(1076003)(86362001)(52116002)(107886003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: uKFZl2Q+fwXJYZAWvLrIO9TsrnFLGO+X5eZcJUxUiR6b8sFMeP9uX3XsLgiXj+GshLq5RUD913hSsF4Dq7DG26BGvX8rmNKdtB66T/+hkaRY+4VQVxDx/rFkCRVp3XAvsCHfQlExwYVx4xqLgc5Zi/KViuioemSKEje5o2ayiT/MxCYG5WJ3SRXXIj1Sm8e7tTmuLJAq4lKBkZKkxk43kw29mQpL56/PBOt7h+kHM/n7gGIskt7K/zYhIJq2rPoLnlPf+Bn6xNLjd9YOpVrSa91yyQz8ebz/Rl4QdQzn5BRThYNPs/VggBdFVr7FDnJEIFWuer9ntJULUNV1xzSYNVI3o04a/c81n3ezlV/IS93M36xSjc5caAoMbAFYxDLDi0ic/wZnPLRw4NPYHTbIk6xTOCCZXPji0i6gnww3aa/ZuT/uSb6Tv2sgvG4wO8UuppzvcejLnfBQIUr+K3C+8jXXvSqCoQ4G8J4Mqv7WrBHntSVW20+sSqgaXmurbnPX X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: b04cccab-6d63-470b-b8cf-08d80d6242cf X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Jun 2020 17:18:15.5238 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: HzpHqCJFUIKnrgR5f5U/qmO86m7vwl71eogSN0BSG6HJGVc9wmfvh6elQZmDejg7P+O3xVhNr8R1r55dlpIVBQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR05MB5346 Subject: [dpdk-dev] [RFC PATCH 5/6] bus/mlx5_pci: register a PCI driver X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Create a mlx5 bus driver framework for invoking drivers of multiple classes who have registered with the mlx5_pci bus driver. Validate user class arguments for supported class combinations. Signed-off-by: Parav Pandit --- drivers/bus/mlx5_pci/Makefile | 1 + drivers/bus/mlx5_pci/meson.build | 2 +- drivers/bus/mlx5_pci/mlx5_pci_bus.c | 253 ++++++++++++++++++++++++ drivers/bus/mlx5_pci/rte_bus_mlx5_pci.h | 1 + 4 files changed, 256 insertions(+), 1 deletion(-) diff --git a/drivers/bus/mlx5_pci/Makefile b/drivers/bus/mlx5_pci/Makefile index b36916e52..327076fe4 100644 --- a/drivers/bus/mlx5_pci/Makefile +++ b/drivers/bus/mlx5_pci/Makefile @@ -15,6 +15,7 @@ CFLAGS += -I$(BUILDDIR)/drivers/common/mlx5 CFLAGS += -I$(RTE_SDK)/drivers/bus/pci CFLAGS += -Wno-strict-prototypes LDLIBS += -lrte_eal +LDLIBS += -lrte_kvargs LDLIBS += -lrte_common_mlx5 LDLIBS += -lrte_pci -lrte_bus_pci diff --git a/drivers/bus/mlx5_pci/meson.build b/drivers/bus/mlx5_pci/meson.build index cc4a84e23..5111baa4e 100644 --- a/drivers/bus/mlx5_pci/meson.build +++ b/drivers/bus/mlx5_pci/meson.build @@ -1,6 +1,6 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2020 Mellanox Technologies Ltd -deps += ['pci', 'bus_pci', 'common_mlx5'] +deps += ['pci', 'bus_pci', 'common_mlx5', 'kvargs'] install_headers('rte_bus_mlx5_pci.h') sources = files('mlx5_pci_bus.c') diff --git a/drivers/bus/mlx5_pci/mlx5_pci_bus.c b/drivers/bus/mlx5_pci/mlx5_pci_bus.c index 66db3c7b0..8108e35ea 100644 --- a/drivers/bus/mlx5_pci/mlx5_pci_bus.c +++ b/drivers/bus/mlx5_pci/mlx5_pci_bus.c @@ -3,12 +3,265 @@ */ #include "rte_bus_mlx5_pci.h" +#include static TAILQ_HEAD(mlx5_pci_bus_drv_head, rte_mlx5_pci_driver) drv_list = TAILQ_HEAD_INITIALIZER(drv_list); +struct class_map { + const char *name; + enum mlx5_class dev_class; +}; + +const struct class_map mlx5_classes[] = { + { .name = "vdpa", .dev_class = MLX5_CLASS_VDPA }, + { .name = "net", .dev_class = MLX5_CLASS_NET }, +}; + +static const enum mlx5_class mlx5_valid_class_combo[] = { + MLX5_CLASS_NET, + MLX5_CLASS_VDPA, + /* New class combination should be added here */ +}; + +static const struct class_map *is_valid_class(const char *class_name) +{ + unsigned int i; + + for (i = 0; i < sizeof(mlx5_classes) / sizeof(struct class_map); + i++) { + if (strcmp(class_name, mlx5_classes[i].name) == 0) + return &mlx5_classes[i]; + + } + return NULL; +} + +static int is_valid_class_combo(const char *class_names) +{ + enum mlx5_class user_classes = 0; + char *nstr = strdup(class_names); + const struct class_map *entry; + char *copy = nstr; + int invalid = 0; + unsigned int i; + char *found; + + while (nstr) { + /* Extract each individual class name */ + found = strsep(&nstr, ":"); + if (!found) + continue; + + /* Check if its a valid class */ + entry = is_valid_class(found); + if (!entry) { + invalid = EINVAL; + break; + } + user_classes |= entry->dev_class; + } + if (copy) + free(copy); + if (invalid) + return invalid; + + /* Verify if user specified valid supported combination */ + for (i = 0; + i < sizeof(mlx5_valid_class_combo) / sizeof(enum mlx5_class); + i++) { + if (mlx5_valid_class_combo[i] == user_classes) + break; + } + /* Not found any valid class combination */ + if (i == sizeof(mlx5_valid_class_combo) / sizeof(enum mlx5_class)) + return -EINVAL; + else + return 0; +} + +static int +mlx5_bus_opt_handler(__rte_unused const char *key, const char *value, + void *opaque) +{ + int *ret = opaque; + + *ret = is_valid_class_combo(value); + if (*ret) + DRV_LOG(ERR, "Invalid mlx5 classes %s. Maybe typo in device" + " class argument setting?", value); + return 0; +} + +static int +mlx5_bus_options_valid(const struct rte_devargs *devargs) +{ + struct rte_kvargs *kvlist; + const char *key = MLX5_CLASS_ARG_NAME; + int ret = 0; + + if (devargs == NULL) + return 0; + kvlist = rte_kvargs_parse(devargs->args, NULL); + if (kvlist == NULL) + return 0; + if (rte_kvargs_count(kvlist, key)) + rte_kvargs_process(kvlist, key, mlx5_bus_opt_handler, &ret); + rte_kvargs_free(kvlist); + return ret; +} + void rte_mlx5_pci_driver_register(struct rte_mlx5_pci_driver *driver) { TAILQ_INSERT_TAIL(&drv_list, driver, next); } + +static bool +mlx5_bus_match(const struct rte_mlx5_pci_driver *drv, + const struct rte_pci_device *pci_dev) +{ + const struct rte_pci_id *id_table; + + for (id_table = drv->id_table; id_table->vendor_id != 0; id_table++) { + /* check if device's ids match the class driver's ones */ + if (id_table->vendor_id != pci_dev->id.vendor_id && + id_table->vendor_id != PCI_ANY_ID) + continue; + if (id_table->device_id != pci_dev->id.device_id && + id_table->device_id != PCI_ANY_ID) + continue; + if (id_table->subsystem_vendor_id != + pci_dev->id.subsystem_vendor_id && + id_table->subsystem_vendor_id != PCI_ANY_ID) + continue; + if (id_table->subsystem_device_id != + pci_dev->id.subsystem_device_id && + id_table->subsystem_device_id != PCI_ANY_ID) + continue; + if (id_table->class_id != pci_dev->id.class_id && + id_table->class_id != RTE_CLASS_ANY_ID) + continue; + + return true; + } + return false; +} + +/** + * DPDK callback to register to probe multiple PCI class devices. + * + * @param[in] pci_drv + * PCI driver structure. + * @param[in] dev + * PCI device information. + * + * @return + * 0 on success, 1 to skip this driver, a negative errno value otherwise + * and rte_errno is set. + */ +static int +mlx5_bus_pci_probe(struct rte_pci_driver *drv __rte_unused, + struct rte_pci_device *dev) +{ + struct rte_mlx5_pci_driver *class; + int ret = 0; + + ret = mlx5_bus_options_valid(dev->device.devargs); + if (ret) + return ret; + + TAILQ_FOREACH(class, &drv_list, next) { + if (!mlx5_bus_match(class, dev)) + continue; + + if (!mlx5_class_enabled(dev->device.devargs, class->dev_class)) + continue; + + ret = class->probe(drv, dev); + if (!ret) + class->loaded = true; + } + return 0; +} + +/** + * DPDK callback to remove one or more class devices for a PCI device. + * + * This function removes all class devices belong to a given PCI device. + * + * @param[in] pci_dev + * Pointer to the PCI device. + * + * @return + * 0 on success, the function cannot fail. + */ +static int +mlx5_bus_pci_remove(struct rte_pci_device *dev) +{ + struct rte_mlx5_pci_driver *class; + + /* Remove each class driver in reverse order */ + TAILQ_FOREACH_REVERSE(class, &drv_list, mlx5_pci_bus_drv_head, next) { + if (!mlx5_class_enabled(dev->device.devargs, class->dev_class)) + continue; + + if (class->loaded) + class->remove(dev); + } + return 0; +} + +static int +mlx5_bus_pci_dma_map(struct rte_pci_device *dev, void *addr, + uint64_t iova, size_t len) +{ + struct rte_mlx5_pci_driver *class; + int ret = -EINVAL; + + TAILQ_FOREACH(class, &drv_list, next) { + if (!class->dma_map) + continue; + + return class->dma_map(dev, addr, iova, len); + } + return ret; +} + +static int +mlx5_bus_pci_dma_unmap(struct rte_pci_device *dev, void *addr, + uint64_t iova, size_t len) +{ + struct rte_mlx5_pci_driver *class; + int ret = -EINVAL; + + TAILQ_FOREACH_REVERSE(class, &drv_list, mlx5_pci_bus_drv_head, next) { + if (!class->dma_unmap) + continue; + + return class->dma_unmap(dev, addr, iova, len); + } + return ret; +} + +static const struct rte_pci_id mlx5_bus_pci_id_map[] = { + { + .vendor_id = 0 + } +}; + +static struct rte_pci_driver mlx5_bus_driver = { + .driver = { + .name = "mlx5_bus_pci", + }, + .id_table = mlx5_bus_pci_id_map, + .probe = mlx5_bus_pci_probe, + .remove = mlx5_bus_pci_remove, + .dma_map = mlx5_bus_pci_dma_map, + .dma_unmap = mlx5_bus_pci_dma_unmap, + .drv_flags = RTE_PCI_DRV_INTR_LSC | RTE_PCI_DRV_INTR_RMV | + RTE_PCI_DRV_PROBE_AGAIN, +}; + +RTE_PMD_REGISTER_PCI(mlx5_bus, mlx5_bus_driver); +RTE_PMD_REGISTER_PCI_TABLE(mlx5_bus, mlx5_bus_pci_id_map); diff --git a/drivers/bus/mlx5_pci/rte_bus_mlx5_pci.h b/drivers/bus/mlx5_pci/rte_bus_mlx5_pci.h index b0423f99e..7be9a15cd 100644 --- a/drivers/bus/mlx5_pci/rte_bus_mlx5_pci.h +++ b/drivers/bus/mlx5_pci/rte_bus_mlx5_pci.h @@ -36,6 +36,7 @@ struct rte_mlx5_pci_driver { pci_dma_unmap_t *dma_unmap; /**< Class device dma unmap function. */ TAILQ_ENTRY(rte_mlx5_pci_driver) next; const struct rte_pci_id *id_table; /**< ID table, NULL terminated. */ + bool loaded; }; /** -- 2.25.4