From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id 69FFC5B1C for ; Wed, 3 Apr 2019 09:19:27 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 03 Apr 2019 00:19:26 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.60,303,1549958400"; d="scan'208";a="128206258" Received: from dpdk-tbie.sh.intel.com ([10.67.104.173]) by orsmga007.jf.intel.com with ESMTP; 03 Apr 2019 00:19:25 -0700 From: Tiwei Bie To: dev@dpdk.org Cc: cunming.liang@intel.com, bruce.richardson@intel.com, alejandro.lucero@netronome.com Date: Wed, 3 Apr 2019 15:18:43 +0800 Message-Id: <20190403071844.21126-3-tiwei.bie@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190403071844.21126-1-tiwei.bie@intel.com> References: <20190403071844.21126-1-tiwei.bie@intel.com> Subject: [dpdk-dev] [RFC 2/3] bus/mdev: add mdev bus support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Apr 2019 07:19:28 -0000 This patch adds the mdev (Mediated device) bus support in DPDK. This bus driver will scan all the mdev devices in the system, and do the probe based on device API (mdev_type/device_api). Signed-off-by: Cunming Liang Signed-off-by: Tiwei Bie --- config/common_base | 5 + config/common_linux | 1 + drivers/bus/Makefile | 1 + drivers/bus/mdev/Makefile | 41 +++ drivers/bus/mdev/linux/Makefile | 6 + drivers/bus/mdev/linux/mdev.c | 117 ++++++++ drivers/bus/mdev/mdev.c | 310 ++++++++++++++++++++++ drivers/bus/mdev/meson.build | 15 ++ drivers/bus/mdev/private.h | 90 +++++++ drivers/bus/mdev/rte_bus_mdev.h | 141 ++++++++++ drivers/bus/mdev/rte_bus_mdev_version.map | 12 + drivers/bus/meson.build | 2 +- mk/rte.app.mk | 1 + 13 files changed, 741 insertions(+), 1 deletion(-) create mode 100644 drivers/bus/mdev/Makefile create mode 100644 drivers/bus/mdev/linux/Makefile create mode 100644 drivers/bus/mdev/linux/mdev.c create mode 100644 drivers/bus/mdev/mdev.c create mode 100644 drivers/bus/mdev/meson.build create mode 100644 drivers/bus/mdev/private.h create mode 100644 drivers/bus/mdev/rte_bus_mdev.h create mode 100644 drivers/bus/mdev/rte_bus_mdev_version.map diff --git a/config/common_base b/config/common_base index 6292bc4af..d29e9a089 100644 --- a/config/common_base +++ b/config/common_base @@ -168,6 +168,11 @@ CONFIG_RTE_LIBRTE_COMMON_DPAAX=n # CONFIG_RTE_LIBRTE_IFPGA_BUS=y +# +# Compile the mdev bus +# +CONFIG_RTE_LIBRTE_MDEV_BUS=n + # # Compile PCI bus driver # diff --git a/config/common_linux b/config/common_linux index 75334273d..7de9624c0 100644 --- a/config/common_linux +++ b/config/common_linux @@ -25,6 +25,7 @@ CONFIG_RTE_LIBRTE_AVP_PMD=y CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD=y CONFIG_RTE_LIBRTE_NFP_PMD=y CONFIG_RTE_LIBRTE_POWER=y +CONFIG_RTE_LIBRTE_MDEV_BUS=y CONFIG_RTE_VIRTIO_USER=y CONFIG_RTE_PROC_INFO=y diff --git a/drivers/bus/Makefile b/drivers/bus/Makefile index cea3b55e6..b2144ee63 100644 --- a/drivers/bus/Makefile +++ b/drivers/bus/Makefile @@ -8,6 +8,7 @@ ifeq ($(CONFIG_RTE_EAL_VFIO),y) DIRS-$(CONFIG_RTE_LIBRTE_FSLMC_BUS) += fslmc endif DIRS-$(CONFIG_RTE_LIBRTE_IFPGA_BUS) += ifpga +DIRS-$(CONFIG_RTE_LIBRTE_MDEV_BUS) += mdev DIRS-$(CONFIG_RTE_LIBRTE_PCI_BUS) += pci DIRS-$(CONFIG_RTE_LIBRTE_VDEV_BUS) += vdev DIRS-$(CONFIG_RTE_LIBRTE_VMBUS) += vmbus diff --git a/drivers/bus/mdev/Makefile b/drivers/bus/mdev/Makefile new file mode 100644 index 000000000..b2faee395 --- /dev/null +++ b/drivers/bus/mdev/Makefile @@ -0,0 +1,41 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2019 Intel Corporation + +include $(RTE_SDK)/mk/rte.vars.mk + +# +# library name +# +LIB = librte_bus_mdev.a + +CFLAGS += -O3 +CFLAGS += $(WERROR_FLAGS) +CFLAGS += -DALLOW_EXPERIMENTAL_API +CFLAGS += -I$(SRCDIR) + +# versioning export map +EXPORT_MAP := rte_bus_mdev_version.map + +# library version +LIBABIVER := 1 + +ifneq ($(CONFIG_RTE_EXEC_ENV_LINUX),) +SYSTEM := linux +endif +ifneq ($(CONFIG_RTE_EXEC_ENV_FREEBSD),) +$(error "Mdev bus not implemented for BSD yet") +endif + +CFLAGS += -I$(RTE_SDK)/drivers/bus/mdev/$(SYSTEM) +CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common +CFLAGS += -I$(RTE_SDK)/lib/librte_eal/$(SYSTEM)/eal + +LDLIBS += -lrte_eal + +include $(RTE_SDK)/drivers/bus/mdev/$(SYSTEM)/Makefile +SRCS-$(CONFIG_RTE_LIBRTE_MDEV_BUS) := $(addprefix $(SYSTEM)/,$(SRCS)) +SRCS-$(CONFIG_RTE_LIBRTE_MDEV_BUS) += mdev.c + +SYMLINK-$(CONFIG_RTE_LIBRTE_MDEV_BUS)-include += rte_bus_mdev.h + +include $(RTE_SDK)/mk/rte.lib.mk diff --git a/drivers/bus/mdev/linux/Makefile b/drivers/bus/mdev/linux/Makefile new file mode 100644 index 000000000..a777ad3d4 --- /dev/null +++ b/drivers/bus/mdev/linux/Makefile @@ -0,0 +1,6 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2019 Intel Corporation + +SRCS += mdev.c + +CFLAGS += -D_GNU_SOURCE diff --git a/drivers/bus/mdev/linux/mdev.c b/drivers/bus/mdev/linux/mdev.c new file mode 100644 index 000000000..ecfe0eba6 --- /dev/null +++ b/drivers/bus/mdev/linux/mdev.c @@ -0,0 +1,117 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2019 Intel Corporation + */ + +#include +#include + +#include +#include + +#include "eal_filesystem.h" + +#include "private.h" + +static int +mdev_scan_one(const char *dirname, const rte_uuid_t addr) +{ + struct rte_mdev_device *mdev; + char device_api[PATH_MAX]; + char filename[PATH_MAX]; + char *ptr; + + mdev = malloc(sizeof(*mdev)); + if (mdev == NULL) + return -1; + + memset(mdev, 0, sizeof(*mdev)); + mdev->device.bus = &rte_mdev_bus.bus; + rte_uuid_copy(mdev->addr, addr); + + /* get device_api */ + snprintf(filename, sizeof(filename), "%s/mdev_type/device_api", + dirname); + if (rte_eal_parse_sysfs_str(filename, device_api, + sizeof(device_api)) < 0) { + free(mdev); + return -1; + } + + ptr = strchr(device_api, '\n'); + if (ptr != NULL) + *ptr = '\0'; + + mdev_name_set(mdev); + + if (strcmp(device_api, "vfio-pci") == 0) { + /* device api */ + mdev->dev_api = RTE_MDEV_DEV_API_VFIO_PCI; + + if (TAILQ_EMPTY(&rte_mdev_bus.device_list)) + rte_mdev_add_device(mdev); + else { + struct rte_mdev_device *dev; + int ret; + + TAILQ_FOREACH(dev, &rte_mdev_bus.device_list, next) { + ret = rte_uuid_compare(mdev->addr, dev->addr); + if (ret > 0) + continue; + + if (ret < 0) + rte_mdev_insert_device(dev, mdev); + else /* already registered */ + free(mdev); + + return 0; + } + + rte_mdev_add_device(mdev); + } + } else { + RTE_LOG(DEBUG, EAL, "%s(): mdev device_api %s is not supported\n", + __func__, device_api); + } + + return 0; +} + +/* + * Scan the content of the mdev bus, and the devices in the devices + * list + */ +int +rte_mdev_scan(void) +{ + struct dirent *e; + DIR *dir; + char dirname[PATH_MAX]; + rte_uuid_t addr; + + dir = opendir(rte_mdev_get_sysfs_path()); + if (dir == NULL) { + RTE_LOG(ERR, EAL, "%s(): opendir failed: %s\n", + __func__, strerror(errno)); + return -1; + } + + while ((e = readdir(dir)) != NULL) { + if (e->d_name[0] == '.') + continue; + + if (rte_uuid_parse(e->d_name, addr) != 0) + continue; + + snprintf(dirname, sizeof(dirname), "%s/%s", + rte_mdev_get_sysfs_path(), e->d_name); + + if (mdev_scan_one(dirname, addr) < 0) + goto error; + } + closedir(dir); + return 0; + +error: + closedir(dir); + return -1; +} diff --git a/drivers/bus/mdev/mdev.c b/drivers/bus/mdev/mdev.c new file mode 100644 index 000000000..2f9209cca --- /dev/null +++ b/drivers/bus/mdev/mdev.c @@ -0,0 +1,310 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2019 Intel Corporation + */ + +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "private.h" + +#define SYSFS_MDEV_DEVICES "/sys/bus/mdev/devices" + +const char *rte_mdev_get_sysfs_path(void) +{ + const char *path = NULL; + + path = getenv("SYSFS_MDEV_DEVICES"); + if (path == NULL) + return SYSFS_MDEV_DEVICES; + + return path; +} + +static void +rte_mdev_device_name(const rte_uuid_t addr, char *output, size_t size) +{ + RTE_VERIFY(size >= RTE_UUID_STRLEN); + rte_uuid_unparse(addr, output, size); +} + +static struct rte_devargs * +mdev_devargs_lookup(struct rte_mdev_device *dev) +{ + struct rte_devargs *devargs; + rte_uuid_t addr; + + RTE_EAL_DEVARGS_FOREACH("mdev", devargs) { + devargs->bus->parse(devargs->name, addr); + if (!rte_uuid_compare(dev->addr, addr)) + return devargs; + } + return NULL; +} + +void +mdev_name_set(struct rte_mdev_device *dev) +{ + struct rte_devargs *devargs; + + /* Each device has its internal, canonical name set. */ + rte_mdev_device_name(dev->addr, dev->name, sizeof(dev->name)); + devargs = mdev_devargs_lookup(dev); + dev->device.devargs = devargs; + /* In blacklist mode, if the device is not blacklisted, no + * rte_devargs exists for it. + */ + if (devargs != NULL) + /* If an rte_devargs exists, the generic rte_device uses the + * given name as its name. + */ + dev->device.name = dev->device.devargs->name; + else + /* Otherwise, it uses the internal, canonical form. */ + dev->device.name = dev->name; +} + +void +rte_mdev_register(struct rte_mdev_driver *driver) +{ + TAILQ_INSERT_TAIL(&rte_mdev_bus.driver_list, driver, next); + driver->bus = &rte_mdev_bus; +} + +void +rte_mdev_unregister(struct rte_mdev_driver *driver) +{ + TAILQ_REMOVE(&rte_mdev_bus.driver_list, driver, next); + driver->bus = NULL; +} + +void +rte_mdev_add_device(struct rte_mdev_device *mdev) +{ + TAILQ_INSERT_TAIL(&rte_mdev_bus.device_list, mdev, next); +} + +void +rte_mdev_insert_device(struct rte_mdev_device *exist_mdev, + struct rte_mdev_device *new_mdev) +{ + TAILQ_INSERT_BEFORE(exist_mdev, new_mdev, next); +} + +void +rte_mdev_remove_device(struct rte_mdev_device *mdev) +{ + TAILQ_REMOVE(&rte_mdev_bus.device_list, mdev, next); +} + +static struct rte_device * +mdev_find_device(const struct rte_device *start, rte_dev_cmp_t cmp, + const void *data) +{ + const struct rte_mdev_device *pstart; + struct rte_mdev_device *pdev; + + if (start != NULL) { + pstart = RTE_DEV_TO_MDEV_CONST(start); + pdev = TAILQ_NEXT(pstart, next); + } else { + pdev = TAILQ_FIRST(&rte_mdev_bus.device_list); + } + while (pdev != NULL) { + if (cmp(&pdev->device, data) == 0) + return &pdev->device; + pdev = TAILQ_NEXT(pdev, next); + } + return NULL; +} + +int +rte_mdev_match(const struct rte_mdev_driver *mdev_drv, + const struct rte_mdev_device *mdev_dev) +{ + if (mdev_drv->dev_api == mdev_dev->dev_api) + return 1; + + return 0; +} + +static int +rte_mdev_probe_one_driver(struct rte_mdev_driver *dr, + struct rte_mdev_device *dev) +{ + int ret; + + if (dr == NULL || dev == NULL) + return -EINVAL; + + /* no initialization when blacklisted, return without error */ + if (dev->device.devargs != NULL && + dev->device.devargs->policy == RTE_DEV_BLACKLISTED) { + RTE_LOG(INFO, EAL, "Device is blacklisted, not initializing\n"); + return 1; + } + + /* The device is not blacklisted; Check if driver supports it */ + if (!rte_mdev_match(dr, dev)) { + /* Match of device and driver failed */ + return 1; + } + + /* reference driver structure */ + dev->driver = dr; + + /* call the driver probe() function */ + ret = dr->probe(dr, dev); + if (ret != 0) + dev->driver = NULL; + + return ret; +} + +static int +mdev_probe_all_drivers(struct rte_mdev_device *dev) +{ + struct rte_mdev_driver *dr = NULL; + int rc = 0; + + if (dev == NULL) + return -1; + + /* Check if a driver is already loaded */ + if (dev->driver != NULL) + return 0; + + FOREACH_DRIVER_ON_MDEV_BUS(dr) { + rc = rte_mdev_probe_one_driver(dr, dev); + if (rc < 0) + /* negative value is an error */ + return -1; + if (rc > 0) + /* positive value means driver doesn't support it */ + continue; + return 0; + } + return 1; +} + +int +rte_mdev_probe(void) +{ + struct rte_mdev_device *mdev = NULL; + size_t probed = 0, failed = 0; + struct rte_devargs *devargs; + int probe_all = 0; + int ret = 0; + + if (rte_mdev_bus.bus.conf.scan_mode != RTE_BUS_SCAN_WHITELIST) + probe_all = 1; + + FOREACH_DEVICE_ON_MDEV_BUS(mdev) { + probed++; + + devargs = mdev->device.devargs; + /* probe all or only whitelisted devices */ + if (probe_all) + ret = mdev_probe_all_drivers(mdev); + else if (devargs != NULL && + devargs->policy == RTE_DEV_WHITELISTED) + ret = mdev_probe_all_drivers(mdev); + if (ret < 0) { + char name[RTE_UUID_STRLEN]; + rte_uuid_unparse(mdev->addr, name, sizeof(name)); + RTE_LOG(ERR, EAL, "Requested device %s cannot be used\n", + name); + rte_errno = errno; + failed++; + ret = 0; + } + } + + return (probed && probed == failed) ? -1 : 0; +} + +static int +mdev_plug(struct rte_device *dev) +{ + return mdev_probe_all_drivers(RTE_DEV_TO_MDEV(dev)); +} + +static int +rte_mdev_detach_dev(struct rte_mdev_device *dev) +{ + struct rte_mdev_driver *dr; + int ret = 0; + + if (dev == NULL) + return -EINVAL; + + dr = dev->driver; + + if (dr->remove) { + ret = dr->remove(dev); + if (ret != 0) + return ret; + } + + /* clear driver structure */ + dev->driver = NULL; + + return 0; +} + +static int +mdev_unplug(struct rte_device *dev) +{ + struct rte_mdev_device *pmdev; + int ret; + + pmdev = RTE_DEV_TO_MDEV(dev); + ret = rte_mdev_detach_dev(pmdev); + if (ret == 0) { + rte_mdev_remove_device(pmdev); + free(pmdev); + } + return ret; +} + +static int +mdev_parse(const char *name, void *addr) +{ + rte_uuid_t uuid; + int parse; + + parse = (rte_uuid_parse(name, uuid) == 0); + if (parse && addr != NULL) + rte_uuid_copy(addr, uuid); + return parse == false; +} + +struct rte_mdev_bus rte_mdev_bus = { + .bus = { + .scan = rte_mdev_scan, + .probe = rte_mdev_probe, + .find_device = mdev_find_device, + .plug = mdev_plug, + .unplug = mdev_unplug, + .parse = mdev_parse, + }, + .device_list = TAILQ_HEAD_INITIALIZER(rte_mdev_bus.device_list), + .driver_list = TAILQ_HEAD_INITIALIZER(rte_mdev_bus.driver_list), +}; + +RTE_REGISTER_BUS(mdev, rte_mdev_bus.bus); diff --git a/drivers/bus/mdev/meson.build b/drivers/bus/mdev/meson.build new file mode 100644 index 000000000..33c701cb9 --- /dev/null +++ b/drivers/bus/mdev/meson.build @@ -0,0 +1,15 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2019 Intel Corporation + +version = 1 +allow_experimental_apis = true +install_headers('rte_bus_mdev.h') +sources = files('mdev.c') + +if host_machine.system() == 'linux' + sources += files('linux/mdev.c') + includes += include_directories('linux') + cflags += ['-D_GNU_SOURCE'] +else + build = false +endif diff --git a/drivers/bus/mdev/private.h b/drivers/bus/mdev/private.h new file mode 100644 index 000000000..81cfe3045 --- /dev/null +++ b/drivers/bus/mdev/private.h @@ -0,0 +1,90 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2019 Intel Corporation + */ + +#ifndef _MDEV_PRIVATE_H_ +#define _MDEV_PRIVATE_H_ + +#include +#include +#include + +struct rte_mdev_driver; +struct rte_mdev_device; + +extern struct rte_mdev_bus rte_mdev_bus; + +/** + * Probe the mdev bus. + * + * @return + * - 0 on success. + * - !0 on error. + */ +int rte_mdev_probe(void); + +/** + * Scan the content of the mdev bus, and the devices in the devices + * list. + * + * @return + * 0 on success, negative on error + */ +int rte_mdev_scan(void); + +/** + * Set the name of a mdev device. + */ +void mdev_name_set(struct rte_mdev_device *dev); + +/** + * Add a mdev device to the mdev bus (append to mdev device list). This function + * also updates the bus references of the mdev device (and the generic device + * object embedded within. + * + * @param mdev + * mdev device to add + * @return void + */ +void rte_mdev_add_device(struct rte_mdev_device *mdev); + +/** + * Insert a mdev device in the mdev bus at a particular location in the device + * list. It also updates the mdev bus reference of the new devices to be + * inserted. + * + * @param exist_mdev + * existing mdev device in mdev bus + * @param new_mdev + * mdev device to be added before exist_mdev + * @return void + */ +void rte_mdev_insert_device(struct rte_mdev_device *exist_mdev, + struct rte_mdev_device *new_mdev); + +/** + * Remove a mdev device from the mdev bus. This sets to NULL the bus references + * in the mdev device object as well as the generic device object. + * + * @param mdev_device + * mdev device to be removed from mdev bus + * @return void + */ +void rte_mdev_remove_device(struct rte_mdev_device *mdev_device); + +/** + * Match the mdev driver and device using mdev device_api. + * + * @param mdev_drv + * mdev driver from which device_api would be extracted + * @param mdev_dev + * mdev device to match against the driver + * @return + * 1 for successful match + * 0 for unsuccessful match + */ +int +rte_mdev_match(const struct rte_mdev_driver *mdev_drv, + const struct rte_mdev_device *mdev_dev); + +#endif /* _MDEV_PRIVATE_H_ */ diff --git a/drivers/bus/mdev/rte_bus_mdev.h b/drivers/bus/mdev/rte_bus_mdev.h new file mode 100644 index 000000000..913521ace --- /dev/null +++ b/drivers/bus/mdev/rte_bus_mdev.h @@ -0,0 +1,141 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2019 Intel Corporation + */ + +#ifndef _RTE_BUS_MDEV_H_ +#define _RTE_BUS_MDEV_H_ + +/** + * @file + * + * RTE Mdev Bus Interface + */ + +#ifdef __cplusplus +extern "C" { +#endif + +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include + +struct rte_devargs; + +enum rte_mdev_device_api { + RTE_MDEV_DEV_API_VFIO_PCI = 0, + RTE_MDEV_DEV_API_MAX, +}; + +struct rte_mdev_bus; +struct rte_mdev_driver; +struct rte_mdev_device; + +/** Pathname of mdev devices directory. */ +const char * __rte_experimental rte_mdev_get_sysfs_path(void); + +/** + * Register a mdev driver. + * + * @param driver + * A pointer to a rte_mdev_driver structure describing the driver + * to be registered. + */ +void __rte_experimental rte_mdev_register(struct rte_mdev_driver *driver); + +#define RTE_MDEV_REGISTER_DRIVER(nm, mdev_drv) \ +RTE_INIT(mdevinitfn_ ##nm) \ +{ \ + (mdev_drv).driver.name = RTE_STR(nm); \ + rte_mdev_register(&mdev_drv); \ +} \ +RTE_PMD_EXPORT_NAME(nm, __COUNTER__) + +/** + * Unregister a mdev driver. + * + * @param driver + * A pointer to a rte_mdev_driver structure describing the driver + * to be unregistered. + */ +void __rte_experimental rte_mdev_unregister(struct rte_mdev_driver *driver); + +/** + * Initialisation function for the driver called during mdev probing. + */ +typedef int (mdev_probe_t)(struct rte_mdev_driver *, struct rte_mdev_device *); + +/** + * Uninitialisation function for the driver called during hotplugging. + */ +typedef int (mdev_remove_t)(struct rte_mdev_device *); + +/** + * A structure describing a mdev driver. + */ +struct rte_mdev_driver { + TAILQ_ENTRY(rte_mdev_driver) next; /**< Next in list. */ + struct rte_driver driver; /**< Inherit core driver. */ + struct rte_mdev_bus *bus; /**< Mdev bus reference. */ + mdev_probe_t *probe; /**< Device probe function. */ + mdev_remove_t *remove; /**< Device remove function. */ + enum rte_mdev_device_api dev_api; /**< Device API. */ +}; + +/** + * A structure describing a mdev device. + */ +struct rte_mdev_device { + TAILQ_ENTRY(rte_mdev_device) next; /**< Next mdev device. */ + struct rte_device device; /**< Inherit core device. */ + enum rte_mdev_device_api dev_api; /**< Device API. */ + struct rte_mdev_driver *driver; /**< Associated driver. */ + rte_uuid_t addr; /**< Location. */ + char name[RTE_UUID_STRLEN]; /**< Location (ASCII). */ + void *private; /**< Driver-specific data. */ +}; + +/** + * @internal + * Helper macro for drivers that need to convert to struct rte_mdev_device. + */ +#define RTE_DEV_TO_MDEV(ptr) container_of(ptr, struct rte_mdev_device, device) + +#define RTE_DEV_TO_MDEV_CONST(ptr) \ + container_of(ptr, const struct rte_mdev_device, device) + +/** List of mdev devices */ +TAILQ_HEAD(rte_mdev_device_list, rte_mdev_device); +/** List of mdev drivers */ +TAILQ_HEAD(rte_mdev_driver_list, rte_mdev_driver); + +/** + * Structure describing the mdev bus + */ +struct rte_mdev_bus { + struct rte_bus bus; /**< Inherit the generic class */ + struct rte_mdev_device_list device_list; /**< List of mdev devices */ + struct rte_mdev_driver_list driver_list; /**< List of mdev drivers */ +}; + +/* Mdev Bus iterators */ +#define FOREACH_DEVICE_ON_MDEV_BUS(p) \ + TAILQ_FOREACH(p, &(rte_mdev_bus.device_list), next) + +#define FOREACH_DRIVER_ON_MDEV_BUS(p) \ + TAILQ_FOREACH(p, &(rte_mdev_bus.driver_list), next) + +#ifdef __cplusplus +} +#endif + +#endif /* _RTE_BUS_MDEV_H_ */ diff --git a/drivers/bus/mdev/rte_bus_mdev_version.map b/drivers/bus/mdev/rte_bus_mdev_version.map new file mode 100644 index 000000000..7f73bf96b --- /dev/null +++ b/drivers/bus/mdev/rte_bus_mdev_version.map @@ -0,0 +1,12 @@ +DPDK_19.05 { + + local: *; +}; + +EXPERIMENTAL { + global: + + rte_mdev_get_sysfs_path; + rte_mdev_register; + rte_mdev_unregister; +}; diff --git a/drivers/bus/meson.build b/drivers/bus/meson.build index 80de2d91d..f0ab19a03 100644 --- a/drivers/bus/meson.build +++ b/drivers/bus/meson.build @@ -1,7 +1,7 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2017 Intel Corporation -drivers = ['dpaa', 'fslmc', 'ifpga', 'pci', 'vdev', 'vmbus'] +drivers = ['dpaa', 'fslmc', 'ifpga', 'mdev', 'pci', 'vdev', 'vmbus'] std_deps = ['eal'] config_flag_fmt = 'RTE_LIBRTE_@0@_BUS' driver_name_fmt = 'rte_bus_@0@' diff --git a/mk/rte.app.mk b/mk/rte.app.mk index 262132fc6..f8abe8237 100644 --- a/mk/rte.app.mk +++ b/mk/rte.app.mk @@ -123,6 +123,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_FSLMC_BUS),y) _LDLIBS-$(CONFIG_RTE_LIBRTE_COMMON_DPAAX) += -lrte_common_dpaax endif +_LDLIBS-$(CONFIG_RTE_LIBRTE_MDEV_BUS) += -lrte_bus_mdev _LDLIBS-$(CONFIG_RTE_LIBRTE_PCI_BUS) += -lrte_bus_pci _LDLIBS-$(CONFIG_RTE_LIBRTE_VDEV_BUS) += -lrte_bus_vdev _LDLIBS-$(CONFIG_RTE_LIBRTE_DPAA_BUS) += -lrte_bus_dpaa -- 2.17.1 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id CD03DA0679 for ; Wed, 3 Apr 2019 09:19:44 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 07FE05F11; Wed, 3 Apr 2019 09:19:34 +0200 (CEST) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id 69FFC5B1C for ; Wed, 3 Apr 2019 09:19:27 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 03 Apr 2019 00:19:26 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.60,303,1549958400"; d="scan'208";a="128206258" Received: from dpdk-tbie.sh.intel.com ([10.67.104.173]) by orsmga007.jf.intel.com with ESMTP; 03 Apr 2019 00:19:25 -0700 From: Tiwei Bie To: dev@dpdk.org Cc: cunming.liang@intel.com, bruce.richardson@intel.com, alejandro.lucero@netronome.com Date: Wed, 3 Apr 2019 15:18:43 +0800 Message-Id: <20190403071844.21126-3-tiwei.bie@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190403071844.21126-1-tiwei.bie@intel.com> References: <20190403071844.21126-1-tiwei.bie@intel.com> Subject: [dpdk-dev] [RFC 2/3] bus/mdev: add mdev bus support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Content-Type: text/plain; charset="UTF-8" Message-ID: <20190403071843.Uj-8nsIEbWdFgmhx0h51gqHrGkb96YMuQnv733vlV0k@z> This patch adds the mdev (Mediated device) bus support in DPDK. This bus driver will scan all the mdev devices in the system, and do the probe based on device API (mdev_type/device_api). Signed-off-by: Cunming Liang Signed-off-by: Tiwei Bie --- config/common_base | 5 + config/common_linux | 1 + drivers/bus/Makefile | 1 + drivers/bus/mdev/Makefile | 41 +++ drivers/bus/mdev/linux/Makefile | 6 + drivers/bus/mdev/linux/mdev.c | 117 ++++++++ drivers/bus/mdev/mdev.c | 310 ++++++++++++++++++++++ drivers/bus/mdev/meson.build | 15 ++ drivers/bus/mdev/private.h | 90 +++++++ drivers/bus/mdev/rte_bus_mdev.h | 141 ++++++++++ drivers/bus/mdev/rte_bus_mdev_version.map | 12 + drivers/bus/meson.build | 2 +- mk/rte.app.mk | 1 + 13 files changed, 741 insertions(+), 1 deletion(-) create mode 100644 drivers/bus/mdev/Makefile create mode 100644 drivers/bus/mdev/linux/Makefile create mode 100644 drivers/bus/mdev/linux/mdev.c create mode 100644 drivers/bus/mdev/mdev.c create mode 100644 drivers/bus/mdev/meson.build create mode 100644 drivers/bus/mdev/private.h create mode 100644 drivers/bus/mdev/rte_bus_mdev.h create mode 100644 drivers/bus/mdev/rte_bus_mdev_version.map diff --git a/config/common_base b/config/common_base index 6292bc4af..d29e9a089 100644 --- a/config/common_base +++ b/config/common_base @@ -168,6 +168,11 @@ CONFIG_RTE_LIBRTE_COMMON_DPAAX=n # CONFIG_RTE_LIBRTE_IFPGA_BUS=y +# +# Compile the mdev bus +# +CONFIG_RTE_LIBRTE_MDEV_BUS=n + # # Compile PCI bus driver # diff --git a/config/common_linux b/config/common_linux index 75334273d..7de9624c0 100644 --- a/config/common_linux +++ b/config/common_linux @@ -25,6 +25,7 @@ CONFIG_RTE_LIBRTE_AVP_PMD=y CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD=y CONFIG_RTE_LIBRTE_NFP_PMD=y CONFIG_RTE_LIBRTE_POWER=y +CONFIG_RTE_LIBRTE_MDEV_BUS=y CONFIG_RTE_VIRTIO_USER=y CONFIG_RTE_PROC_INFO=y diff --git a/drivers/bus/Makefile b/drivers/bus/Makefile index cea3b55e6..b2144ee63 100644 --- a/drivers/bus/Makefile +++ b/drivers/bus/Makefile @@ -8,6 +8,7 @@ ifeq ($(CONFIG_RTE_EAL_VFIO),y) DIRS-$(CONFIG_RTE_LIBRTE_FSLMC_BUS) += fslmc endif DIRS-$(CONFIG_RTE_LIBRTE_IFPGA_BUS) += ifpga +DIRS-$(CONFIG_RTE_LIBRTE_MDEV_BUS) += mdev DIRS-$(CONFIG_RTE_LIBRTE_PCI_BUS) += pci DIRS-$(CONFIG_RTE_LIBRTE_VDEV_BUS) += vdev DIRS-$(CONFIG_RTE_LIBRTE_VMBUS) += vmbus diff --git a/drivers/bus/mdev/Makefile b/drivers/bus/mdev/Makefile new file mode 100644 index 000000000..b2faee395 --- /dev/null +++ b/drivers/bus/mdev/Makefile @@ -0,0 +1,41 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2019 Intel Corporation + +include $(RTE_SDK)/mk/rte.vars.mk + +# +# library name +# +LIB = librte_bus_mdev.a + +CFLAGS += -O3 +CFLAGS += $(WERROR_FLAGS) +CFLAGS += -DALLOW_EXPERIMENTAL_API +CFLAGS += -I$(SRCDIR) + +# versioning export map +EXPORT_MAP := rte_bus_mdev_version.map + +# library version +LIBABIVER := 1 + +ifneq ($(CONFIG_RTE_EXEC_ENV_LINUX),) +SYSTEM := linux +endif +ifneq ($(CONFIG_RTE_EXEC_ENV_FREEBSD),) +$(error "Mdev bus not implemented for BSD yet") +endif + +CFLAGS += -I$(RTE_SDK)/drivers/bus/mdev/$(SYSTEM) +CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common +CFLAGS += -I$(RTE_SDK)/lib/librte_eal/$(SYSTEM)/eal + +LDLIBS += -lrte_eal + +include $(RTE_SDK)/drivers/bus/mdev/$(SYSTEM)/Makefile +SRCS-$(CONFIG_RTE_LIBRTE_MDEV_BUS) := $(addprefix $(SYSTEM)/,$(SRCS)) +SRCS-$(CONFIG_RTE_LIBRTE_MDEV_BUS) += mdev.c + +SYMLINK-$(CONFIG_RTE_LIBRTE_MDEV_BUS)-include += rte_bus_mdev.h + +include $(RTE_SDK)/mk/rte.lib.mk diff --git a/drivers/bus/mdev/linux/Makefile b/drivers/bus/mdev/linux/Makefile new file mode 100644 index 000000000..a777ad3d4 --- /dev/null +++ b/drivers/bus/mdev/linux/Makefile @@ -0,0 +1,6 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2019 Intel Corporation + +SRCS += mdev.c + +CFLAGS += -D_GNU_SOURCE diff --git a/drivers/bus/mdev/linux/mdev.c b/drivers/bus/mdev/linux/mdev.c new file mode 100644 index 000000000..ecfe0eba6 --- /dev/null +++ b/drivers/bus/mdev/linux/mdev.c @@ -0,0 +1,117 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2019 Intel Corporation + */ + +#include +#include + +#include +#include + +#include "eal_filesystem.h" + +#include "private.h" + +static int +mdev_scan_one(const char *dirname, const rte_uuid_t addr) +{ + struct rte_mdev_device *mdev; + char device_api[PATH_MAX]; + char filename[PATH_MAX]; + char *ptr; + + mdev = malloc(sizeof(*mdev)); + if (mdev == NULL) + return -1; + + memset(mdev, 0, sizeof(*mdev)); + mdev->device.bus = &rte_mdev_bus.bus; + rte_uuid_copy(mdev->addr, addr); + + /* get device_api */ + snprintf(filename, sizeof(filename), "%s/mdev_type/device_api", + dirname); + if (rte_eal_parse_sysfs_str(filename, device_api, + sizeof(device_api)) < 0) { + free(mdev); + return -1; + } + + ptr = strchr(device_api, '\n'); + if (ptr != NULL) + *ptr = '\0'; + + mdev_name_set(mdev); + + if (strcmp(device_api, "vfio-pci") == 0) { + /* device api */ + mdev->dev_api = RTE_MDEV_DEV_API_VFIO_PCI; + + if (TAILQ_EMPTY(&rte_mdev_bus.device_list)) + rte_mdev_add_device(mdev); + else { + struct rte_mdev_device *dev; + int ret; + + TAILQ_FOREACH(dev, &rte_mdev_bus.device_list, next) { + ret = rte_uuid_compare(mdev->addr, dev->addr); + if (ret > 0) + continue; + + if (ret < 0) + rte_mdev_insert_device(dev, mdev); + else /* already registered */ + free(mdev); + + return 0; + } + + rte_mdev_add_device(mdev); + } + } else { + RTE_LOG(DEBUG, EAL, "%s(): mdev device_api %s is not supported\n", + __func__, device_api); + } + + return 0; +} + +/* + * Scan the content of the mdev bus, and the devices in the devices + * list + */ +int +rte_mdev_scan(void) +{ + struct dirent *e; + DIR *dir; + char dirname[PATH_MAX]; + rte_uuid_t addr; + + dir = opendir(rte_mdev_get_sysfs_path()); + if (dir == NULL) { + RTE_LOG(ERR, EAL, "%s(): opendir failed: %s\n", + __func__, strerror(errno)); + return -1; + } + + while ((e = readdir(dir)) != NULL) { + if (e->d_name[0] == '.') + continue; + + if (rte_uuid_parse(e->d_name, addr) != 0) + continue; + + snprintf(dirname, sizeof(dirname), "%s/%s", + rte_mdev_get_sysfs_path(), e->d_name); + + if (mdev_scan_one(dirname, addr) < 0) + goto error; + } + closedir(dir); + return 0; + +error: + closedir(dir); + return -1; +} diff --git a/drivers/bus/mdev/mdev.c b/drivers/bus/mdev/mdev.c new file mode 100644 index 000000000..2f9209cca --- /dev/null +++ b/drivers/bus/mdev/mdev.c @@ -0,0 +1,310 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2019 Intel Corporation + */ + +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "private.h" + +#define SYSFS_MDEV_DEVICES "/sys/bus/mdev/devices" + +const char *rte_mdev_get_sysfs_path(void) +{ + const char *path = NULL; + + path = getenv("SYSFS_MDEV_DEVICES"); + if (path == NULL) + return SYSFS_MDEV_DEVICES; + + return path; +} + +static void +rte_mdev_device_name(const rte_uuid_t addr, char *output, size_t size) +{ + RTE_VERIFY(size >= RTE_UUID_STRLEN); + rte_uuid_unparse(addr, output, size); +} + +static struct rte_devargs * +mdev_devargs_lookup(struct rte_mdev_device *dev) +{ + struct rte_devargs *devargs; + rte_uuid_t addr; + + RTE_EAL_DEVARGS_FOREACH("mdev", devargs) { + devargs->bus->parse(devargs->name, addr); + if (!rte_uuid_compare(dev->addr, addr)) + return devargs; + } + return NULL; +} + +void +mdev_name_set(struct rte_mdev_device *dev) +{ + struct rte_devargs *devargs; + + /* Each device has its internal, canonical name set. */ + rte_mdev_device_name(dev->addr, dev->name, sizeof(dev->name)); + devargs = mdev_devargs_lookup(dev); + dev->device.devargs = devargs; + /* In blacklist mode, if the device is not blacklisted, no + * rte_devargs exists for it. + */ + if (devargs != NULL) + /* If an rte_devargs exists, the generic rte_device uses the + * given name as its name. + */ + dev->device.name = dev->device.devargs->name; + else + /* Otherwise, it uses the internal, canonical form. */ + dev->device.name = dev->name; +} + +void +rte_mdev_register(struct rte_mdev_driver *driver) +{ + TAILQ_INSERT_TAIL(&rte_mdev_bus.driver_list, driver, next); + driver->bus = &rte_mdev_bus; +} + +void +rte_mdev_unregister(struct rte_mdev_driver *driver) +{ + TAILQ_REMOVE(&rte_mdev_bus.driver_list, driver, next); + driver->bus = NULL; +} + +void +rte_mdev_add_device(struct rte_mdev_device *mdev) +{ + TAILQ_INSERT_TAIL(&rte_mdev_bus.device_list, mdev, next); +} + +void +rte_mdev_insert_device(struct rte_mdev_device *exist_mdev, + struct rte_mdev_device *new_mdev) +{ + TAILQ_INSERT_BEFORE(exist_mdev, new_mdev, next); +} + +void +rte_mdev_remove_device(struct rte_mdev_device *mdev) +{ + TAILQ_REMOVE(&rte_mdev_bus.device_list, mdev, next); +} + +static struct rte_device * +mdev_find_device(const struct rte_device *start, rte_dev_cmp_t cmp, + const void *data) +{ + const struct rte_mdev_device *pstart; + struct rte_mdev_device *pdev; + + if (start != NULL) { + pstart = RTE_DEV_TO_MDEV_CONST(start); + pdev = TAILQ_NEXT(pstart, next); + } else { + pdev = TAILQ_FIRST(&rte_mdev_bus.device_list); + } + while (pdev != NULL) { + if (cmp(&pdev->device, data) == 0) + return &pdev->device; + pdev = TAILQ_NEXT(pdev, next); + } + return NULL; +} + +int +rte_mdev_match(const struct rte_mdev_driver *mdev_drv, + const struct rte_mdev_device *mdev_dev) +{ + if (mdev_drv->dev_api == mdev_dev->dev_api) + return 1; + + return 0; +} + +static int +rte_mdev_probe_one_driver(struct rte_mdev_driver *dr, + struct rte_mdev_device *dev) +{ + int ret; + + if (dr == NULL || dev == NULL) + return -EINVAL; + + /* no initialization when blacklisted, return without error */ + if (dev->device.devargs != NULL && + dev->device.devargs->policy == RTE_DEV_BLACKLISTED) { + RTE_LOG(INFO, EAL, "Device is blacklisted, not initializing\n"); + return 1; + } + + /* The device is not blacklisted; Check if driver supports it */ + if (!rte_mdev_match(dr, dev)) { + /* Match of device and driver failed */ + return 1; + } + + /* reference driver structure */ + dev->driver = dr; + + /* call the driver probe() function */ + ret = dr->probe(dr, dev); + if (ret != 0) + dev->driver = NULL; + + return ret; +} + +static int +mdev_probe_all_drivers(struct rte_mdev_device *dev) +{ + struct rte_mdev_driver *dr = NULL; + int rc = 0; + + if (dev == NULL) + return -1; + + /* Check if a driver is already loaded */ + if (dev->driver != NULL) + return 0; + + FOREACH_DRIVER_ON_MDEV_BUS(dr) { + rc = rte_mdev_probe_one_driver(dr, dev); + if (rc < 0) + /* negative value is an error */ + return -1; + if (rc > 0) + /* positive value means driver doesn't support it */ + continue; + return 0; + } + return 1; +} + +int +rte_mdev_probe(void) +{ + struct rte_mdev_device *mdev = NULL; + size_t probed = 0, failed = 0; + struct rte_devargs *devargs; + int probe_all = 0; + int ret = 0; + + if (rte_mdev_bus.bus.conf.scan_mode != RTE_BUS_SCAN_WHITELIST) + probe_all = 1; + + FOREACH_DEVICE_ON_MDEV_BUS(mdev) { + probed++; + + devargs = mdev->device.devargs; + /* probe all or only whitelisted devices */ + if (probe_all) + ret = mdev_probe_all_drivers(mdev); + else if (devargs != NULL && + devargs->policy == RTE_DEV_WHITELISTED) + ret = mdev_probe_all_drivers(mdev); + if (ret < 0) { + char name[RTE_UUID_STRLEN]; + rte_uuid_unparse(mdev->addr, name, sizeof(name)); + RTE_LOG(ERR, EAL, "Requested device %s cannot be used\n", + name); + rte_errno = errno; + failed++; + ret = 0; + } + } + + return (probed && probed == failed) ? -1 : 0; +} + +static int +mdev_plug(struct rte_device *dev) +{ + return mdev_probe_all_drivers(RTE_DEV_TO_MDEV(dev)); +} + +static int +rte_mdev_detach_dev(struct rte_mdev_device *dev) +{ + struct rte_mdev_driver *dr; + int ret = 0; + + if (dev == NULL) + return -EINVAL; + + dr = dev->driver; + + if (dr->remove) { + ret = dr->remove(dev); + if (ret != 0) + return ret; + } + + /* clear driver structure */ + dev->driver = NULL; + + return 0; +} + +static int +mdev_unplug(struct rte_device *dev) +{ + struct rte_mdev_device *pmdev; + int ret; + + pmdev = RTE_DEV_TO_MDEV(dev); + ret = rte_mdev_detach_dev(pmdev); + if (ret == 0) { + rte_mdev_remove_device(pmdev); + free(pmdev); + } + return ret; +} + +static int +mdev_parse(const char *name, void *addr) +{ + rte_uuid_t uuid; + int parse; + + parse = (rte_uuid_parse(name, uuid) == 0); + if (parse && addr != NULL) + rte_uuid_copy(addr, uuid); + return parse == false; +} + +struct rte_mdev_bus rte_mdev_bus = { + .bus = { + .scan = rte_mdev_scan, + .probe = rte_mdev_probe, + .find_device = mdev_find_device, + .plug = mdev_plug, + .unplug = mdev_unplug, + .parse = mdev_parse, + }, + .device_list = TAILQ_HEAD_INITIALIZER(rte_mdev_bus.device_list), + .driver_list = TAILQ_HEAD_INITIALIZER(rte_mdev_bus.driver_list), +}; + +RTE_REGISTER_BUS(mdev, rte_mdev_bus.bus); diff --git a/drivers/bus/mdev/meson.build b/drivers/bus/mdev/meson.build new file mode 100644 index 000000000..33c701cb9 --- /dev/null +++ b/drivers/bus/mdev/meson.build @@ -0,0 +1,15 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2019 Intel Corporation + +version = 1 +allow_experimental_apis = true +install_headers('rte_bus_mdev.h') +sources = files('mdev.c') + +if host_machine.system() == 'linux' + sources += files('linux/mdev.c') + includes += include_directories('linux') + cflags += ['-D_GNU_SOURCE'] +else + build = false +endif diff --git a/drivers/bus/mdev/private.h b/drivers/bus/mdev/private.h new file mode 100644 index 000000000..81cfe3045 --- /dev/null +++ b/drivers/bus/mdev/private.h @@ -0,0 +1,90 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2019 Intel Corporation + */ + +#ifndef _MDEV_PRIVATE_H_ +#define _MDEV_PRIVATE_H_ + +#include +#include +#include + +struct rte_mdev_driver; +struct rte_mdev_device; + +extern struct rte_mdev_bus rte_mdev_bus; + +/** + * Probe the mdev bus. + * + * @return + * - 0 on success. + * - !0 on error. + */ +int rte_mdev_probe(void); + +/** + * Scan the content of the mdev bus, and the devices in the devices + * list. + * + * @return + * 0 on success, negative on error + */ +int rte_mdev_scan(void); + +/** + * Set the name of a mdev device. + */ +void mdev_name_set(struct rte_mdev_device *dev); + +/** + * Add a mdev device to the mdev bus (append to mdev device list). This function + * also updates the bus references of the mdev device (and the generic device + * object embedded within. + * + * @param mdev + * mdev device to add + * @return void + */ +void rte_mdev_add_device(struct rte_mdev_device *mdev); + +/** + * Insert a mdev device in the mdev bus at a particular location in the device + * list. It also updates the mdev bus reference of the new devices to be + * inserted. + * + * @param exist_mdev + * existing mdev device in mdev bus + * @param new_mdev + * mdev device to be added before exist_mdev + * @return void + */ +void rte_mdev_insert_device(struct rte_mdev_device *exist_mdev, + struct rte_mdev_device *new_mdev); + +/** + * Remove a mdev device from the mdev bus. This sets to NULL the bus references + * in the mdev device object as well as the generic device object. + * + * @param mdev_device + * mdev device to be removed from mdev bus + * @return void + */ +void rte_mdev_remove_device(struct rte_mdev_device *mdev_device); + +/** + * Match the mdev driver and device using mdev device_api. + * + * @param mdev_drv + * mdev driver from which device_api would be extracted + * @param mdev_dev + * mdev device to match against the driver + * @return + * 1 for successful match + * 0 for unsuccessful match + */ +int +rte_mdev_match(const struct rte_mdev_driver *mdev_drv, + const struct rte_mdev_device *mdev_dev); + +#endif /* _MDEV_PRIVATE_H_ */ diff --git a/drivers/bus/mdev/rte_bus_mdev.h b/drivers/bus/mdev/rte_bus_mdev.h new file mode 100644 index 000000000..913521ace --- /dev/null +++ b/drivers/bus/mdev/rte_bus_mdev.h @@ -0,0 +1,141 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2019 Intel Corporation + */ + +#ifndef _RTE_BUS_MDEV_H_ +#define _RTE_BUS_MDEV_H_ + +/** + * @file + * + * RTE Mdev Bus Interface + */ + +#ifdef __cplusplus +extern "C" { +#endif + +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include + +struct rte_devargs; + +enum rte_mdev_device_api { + RTE_MDEV_DEV_API_VFIO_PCI = 0, + RTE_MDEV_DEV_API_MAX, +}; + +struct rte_mdev_bus; +struct rte_mdev_driver; +struct rte_mdev_device; + +/** Pathname of mdev devices directory. */ +const char * __rte_experimental rte_mdev_get_sysfs_path(void); + +/** + * Register a mdev driver. + * + * @param driver + * A pointer to a rte_mdev_driver structure describing the driver + * to be registered. + */ +void __rte_experimental rte_mdev_register(struct rte_mdev_driver *driver); + +#define RTE_MDEV_REGISTER_DRIVER(nm, mdev_drv) \ +RTE_INIT(mdevinitfn_ ##nm) \ +{ \ + (mdev_drv).driver.name = RTE_STR(nm); \ + rte_mdev_register(&mdev_drv); \ +} \ +RTE_PMD_EXPORT_NAME(nm, __COUNTER__) + +/** + * Unregister a mdev driver. + * + * @param driver + * A pointer to a rte_mdev_driver structure describing the driver + * to be unregistered. + */ +void __rte_experimental rte_mdev_unregister(struct rte_mdev_driver *driver); + +/** + * Initialisation function for the driver called during mdev probing. + */ +typedef int (mdev_probe_t)(struct rte_mdev_driver *, struct rte_mdev_device *); + +/** + * Uninitialisation function for the driver called during hotplugging. + */ +typedef int (mdev_remove_t)(struct rte_mdev_device *); + +/** + * A structure describing a mdev driver. + */ +struct rte_mdev_driver { + TAILQ_ENTRY(rte_mdev_driver) next; /**< Next in list. */ + struct rte_driver driver; /**< Inherit core driver. */ + struct rte_mdev_bus *bus; /**< Mdev bus reference. */ + mdev_probe_t *probe; /**< Device probe function. */ + mdev_remove_t *remove; /**< Device remove function. */ + enum rte_mdev_device_api dev_api; /**< Device API. */ +}; + +/** + * A structure describing a mdev device. + */ +struct rte_mdev_device { + TAILQ_ENTRY(rte_mdev_device) next; /**< Next mdev device. */ + struct rte_device device; /**< Inherit core device. */ + enum rte_mdev_device_api dev_api; /**< Device API. */ + struct rte_mdev_driver *driver; /**< Associated driver. */ + rte_uuid_t addr; /**< Location. */ + char name[RTE_UUID_STRLEN]; /**< Location (ASCII). */ + void *private; /**< Driver-specific data. */ +}; + +/** + * @internal + * Helper macro for drivers that need to convert to struct rte_mdev_device. + */ +#define RTE_DEV_TO_MDEV(ptr) container_of(ptr, struct rte_mdev_device, device) + +#define RTE_DEV_TO_MDEV_CONST(ptr) \ + container_of(ptr, const struct rte_mdev_device, device) + +/** List of mdev devices */ +TAILQ_HEAD(rte_mdev_device_list, rte_mdev_device); +/** List of mdev drivers */ +TAILQ_HEAD(rte_mdev_driver_list, rte_mdev_driver); + +/** + * Structure describing the mdev bus + */ +struct rte_mdev_bus { + struct rte_bus bus; /**< Inherit the generic class */ + struct rte_mdev_device_list device_list; /**< List of mdev devices */ + struct rte_mdev_driver_list driver_list; /**< List of mdev drivers */ +}; + +/* Mdev Bus iterators */ +#define FOREACH_DEVICE_ON_MDEV_BUS(p) \ + TAILQ_FOREACH(p, &(rte_mdev_bus.device_list), next) + +#define FOREACH_DRIVER_ON_MDEV_BUS(p) \ + TAILQ_FOREACH(p, &(rte_mdev_bus.driver_list), next) + +#ifdef __cplusplus +} +#endif + +#endif /* _RTE_BUS_MDEV_H_ */ diff --git a/drivers/bus/mdev/rte_bus_mdev_version.map b/drivers/bus/mdev/rte_bus_mdev_version.map new file mode 100644 index 000000000..7f73bf96b --- /dev/null +++ b/drivers/bus/mdev/rte_bus_mdev_version.map @@ -0,0 +1,12 @@ +DPDK_19.05 { + + local: *; +}; + +EXPERIMENTAL { + global: + + rte_mdev_get_sysfs_path; + rte_mdev_register; + rte_mdev_unregister; +}; diff --git a/drivers/bus/meson.build b/drivers/bus/meson.build index 80de2d91d..f0ab19a03 100644 --- a/drivers/bus/meson.build +++ b/drivers/bus/meson.build @@ -1,7 +1,7 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2017 Intel Corporation -drivers = ['dpaa', 'fslmc', 'ifpga', 'pci', 'vdev', 'vmbus'] +drivers = ['dpaa', 'fslmc', 'ifpga', 'mdev', 'pci', 'vdev', 'vmbus'] std_deps = ['eal'] config_flag_fmt = 'RTE_LIBRTE_@0@_BUS' driver_name_fmt = 'rte_bus_@0@' diff --git a/mk/rte.app.mk b/mk/rte.app.mk index 262132fc6..f8abe8237 100644 --- a/mk/rte.app.mk +++ b/mk/rte.app.mk @@ -123,6 +123,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_FSLMC_BUS),y) _LDLIBS-$(CONFIG_RTE_LIBRTE_COMMON_DPAAX) += -lrte_common_dpaax endif +_LDLIBS-$(CONFIG_RTE_LIBRTE_MDEV_BUS) += -lrte_bus_mdev _LDLIBS-$(CONFIG_RTE_LIBRTE_PCI_BUS) += -lrte_bus_pci _LDLIBS-$(CONFIG_RTE_LIBRTE_VDEV_BUS) += -lrte_bus_vdev _LDLIBS-$(CONFIG_RTE_LIBRTE_DPAA_BUS) += -lrte_bus_dpaa -- 2.17.1