From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by dpdk.org (Postfix) with ESMTP id 04A411B345 for ; Fri, 22 Dec 2017 15:52:24 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga105.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 22 Dec 2017 06:52:24 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.45,441,1508828400"; d="scan'208";a="161014052" Received: from rhorton-mobl1.ger.corp.intel.com (HELO FC23.ir.intel.com) ([163.33.230.236]) by orsmga004.jf.intel.com with ESMTP; 22 Dec 2017 06:52:22 -0800 From: Remy Horton To: dev@dpdk.org Cc: Wenzhuo Lu , Jingjing Wu , Declan Doherty , Mohammad Abdul Awal Date: Fri, 22 Dec 2017 14:52:17 +0000 Message-Id: <20171222145221.10474-2-remy.horton@intel.com> X-Mailer: git-send-email 2.9.5 In-Reply-To: <20171222145221.10474-1-remy.horton@intel.com> References: <20171222145221.10474-1-remy.horton@intel.com> Subject: [dpdk-dev] [PATCH v3 1/5] lib: add Port Representor library X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Dec 2017 14:52:25 -0000 Port Representors provide a logical presentation in DPDK of VF (virtual function) ports for the purposes of control and monitoring. Each port representor device represents a single VF and is associated with it's parent physical function (PF) PMD which provides the back-end hooks for the representor device ops and defines the control domain to which that port belongs. This allows to use existing DPDK APIs to monitor and control the port without the need to create and maintain VF specific APIs. The library provides the broker infrastructure to be instantiated by base driver and corresponding methods to manage the broker infrastructure. The broker keeps records of list of representor PMDs. The library also provides methods to manage the representor PMDs by the broker. Signed-off-by: Declan Doherty Signed-off-by: Mohammad Abdul Awal Signed-off-by: Remy Horton --- config/common_base | 5 + lib/Makefile | 3 + lib/librte_representor/Makefile | 26 ++ lib/librte_representor/rte_port_representor.c | 326 +++++++++++++++++++++ lib/librte_representor/rte_port_representor.h | 60 ++++ .../rte_port_representor_driver.h | 138 +++++++++ .../rte_port_representor_version.map | 8 + mk/rte.app.mk | 1 + 8 files changed, 567 insertions(+) create mode 100644 lib/librte_representor/Makefile create mode 100644 lib/librte_representor/rte_port_representor.c create mode 100644 lib/librte_representor/rte_port_representor.h create mode 100644 lib/librte_representor/rte_port_representor_driver.h create mode 100644 lib/librte_representor/rte_port_representor_version.map diff --git a/config/common_base b/config/common_base index e74febe..febb80a 100644 --- a/config/common_base +++ b/config/common_base @@ -820,3 +820,8 @@ CONFIG_RTE_APP_CRYPTO_PERF=y # Compile the eventdev application # CONFIG_RTE_APP_EVENTDEV=y + +# +# Compile representor PMD +# +CONFIG_RTE_LIBRTE_REPRESENTOR=y diff --git a/lib/Makefile b/lib/Makefile index dc4e8df..b9202ff 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -101,6 +101,9 @@ DEPDIRS-librte_distributor := librte_eal librte_mbuf librte_ether DIRS-$(CONFIG_RTE_LIBRTE_PORT) += librte_port DEPDIRS-librte_port := librte_eal librte_mempool librte_mbuf librte_ether DEPDIRS-librte_port += librte_ip_frag librte_sched +DIRS-$(CONFIG_RTE_LIBRTE_REPRESENTOR) += librte_representor +DEPDIRS-librte_representor += librte_ether + ifeq ($(CONFIG_RTE_LIBRTE_KNI),y) DEPDIRS-librte_port += librte_kni endif diff --git a/lib/librte_representor/Makefile b/lib/librte_representor/Makefile new file mode 100644 index 0000000..4060cc6 --- /dev/null +++ b/lib/librte_representor/Makefile @@ -0,0 +1,26 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2017 Intel Corporation. All rights reserved. + +include $(RTE_SDK)/mk/rte.vars.mk + +# +# library name +# +LIB = librte_representor.a + +CFLAGS += -O3 +CFLAGS += $(WERROR_FLAGS) + +EXPORT_MAP := rte_port_representor_version.map + +LIBABIVER := 1 + +SRCS-$(CONFIG_RTE_LIBRTE_REPRESENTOR) += rte_port_representor.c + +# +# Export include files +# +SYMLINK-$(CONFIG_RTE_LIBRTE_REPRESENTOR)-include += rte_port_representor.h +SYMLINK-$(CONFIG_RTE_LIBRTE_REPRESENTOR)-include += rte_port_representor_driver.h + +include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/librte_representor/rte_port_representor.c b/lib/librte_representor/rte_port_representor.c new file mode 100644 index 0000000..69a4bfc --- /dev/null +++ b/lib/librte_representor/rte_port_representor.c @@ -0,0 +1,326 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2017 Intel Corporation. All rights reserved. + */ + +#include +#include + +#include + +TAILQ_HEAD(rte_broker_list, rte_representor_broker); + +struct rte_broker_list broker_list = + TAILQ_HEAD_INITIALIZER(broker_list); + +struct port_rep_parameters { + uint64_t vport_mask; + struct { + char bus[RTE_DEV_NAME_MAX_LEN]; + char device[RTE_DEV_NAME_MAX_LEN]; + } parent; +}; + +/* Macros to check for valid id */ +#define RTE_VERIFY_OR_ERR_RET(val, retval) do { \ + if (!(val)) { \ + RTE_PMD_DEBUG_TRACE("verify failed, ret= %d", (retval)); \ + return retval; \ + } \ +} while (0) + +#define RTE_VERIFY_OR_RET(val) do { \ + if (!(val)) { \ + RTE_PMD_DEBUG_TRACE("verify failed"); \ + return; \ + } \ +} while (0) + +int +rte_representor_broker_init(struct rte_representor_broker *broker) +{ + RTE_VERIFY_OR_ERR_RET(broker, -ENODEV); + + RTE_VERIFY_OR_ERR_RET(broker->bus && strlen(broker->bus), -ENXIO); + RTE_VERIFY_OR_ERR_RET(broker->device && strlen(broker->device), -ENXIO); + + RTE_VERIFY_OR_ERR_RET(broker->nb_virtual_ports > 0, -EINVAL); + + broker->vports = rte_malloc("rte_representor_ports", + sizeof(*(broker->vports)) * broker->nb_virtual_ports, 0); + if (broker->vports == NULL) + return -ENOMEM; + + RTE_VERIFY_OR_ERR_RET(broker->ops, -EINVAL); + + TAILQ_INSERT_TAIL(&broker_list, broker, next); + RTE_LOG(INFO, EAL, "Registered [%s_%s] broker.\n", broker->bus, + broker->device); + + return 0; +} + +int +rte_representor_broker_uninit(struct rte_representor_broker *broker) +{ + int i; + struct rte_eth_dev *ethdev; + char name[RTE_DEV_NAME_MAX_LEN]; + + for (i = 0; i < broker->nb_virtual_ports; i++) { + if (broker->vports[i].state == RTE_REPRESENTOR_PORT_VALID) { + ethdev = broker->vports[i].ethdev; + rte_eth_dev_detach(ethdev->data->port_id, + name); + } + } + + rte_free(broker->vports); + + TAILQ_REMOVE(&broker_list, broker, next); + RTE_LOG(DEBUG, EAL, "Unregistered [%s_%s] broker.\n", broker->bus, + broker->device); + + return 0; +} + +struct rte_representor_broker * +rte_representor_broker_find(const char *bus, const char *device) +{ + struct rte_representor_broker *broker = NULL; + + TAILQ_FOREACH(broker, &broker_list, next) { + if ((strcmp(broker->bus, bus) == 0) && + (strcmp(broker->device, device) == 0)) + break; + } + + return broker; +} + +#define RTE_REPRESENTOR_PORT_VALID_ETHDEV_OR_RET_ERR(ethdev, retval) do { \ + if (!strstr(ethdev->data->name, RTE_PORT_REPRESENTOR_DEVICE_NAME)) { \ + RTE_PMD_DEBUG_TRACE("port %d is not a representor port", \ + ethdev->data.port_id); \ + return retval; \ + } \ +} while (0) + +#define RTE_REPRESENTOR_PORT_VALID_OR_RET_ERR(vport, retval) do { \ + if (!(vport->state == RTE_REPRESENTOR_PORT_VALID)) { \ + RTE_PMD_DEBUG_TRACE("Vport is not a representor port"); \ + return retval; \ + } \ +} while (0) + +static struct rte_representor_port * +representor_port_find(struct rte_representor_broker *broker, + uint16_t vport_id) +{ + if (vport_id >= broker->nb_virtual_ports) + return NULL; + + return &broker->vports[vport_id]; +} + +int +rte_representor_port_register(char *pf_addr_str, + uint32_t vport_id, uint16_t *port_id) +{ + struct rte_eth_dev *rep_ethdev; + struct rte_representor_port *rep_port; + struct rte_representor_broker *broker; + char str_device_name[RTE_ETH_NAME_MAX_LEN]; + int len_name; + int retval; + char *addr_delim; + int addr_len; + char addr_bus[RTE_ETH_NAME_MAX_LEN]; + char addr_dev[RTE_ETH_NAME_MAX_LEN]; + + RTE_LOG(INFO, PMD, "Creating representor for VF %i from %s\n", + vport_id, pf_addr_str); + + /* the parent param has to be provided with type_name formatted */ + memset(addr_bus, 0, sizeof(addr_bus)); + memset(addr_dev, 0, sizeof(addr_dev)); + addr_delim = strchr(pf_addr_str, '_'); + if (addr_delim) { + addr_len = addr_delim - pf_addr_str; + strncpy(addr_bus, pf_addr_str, addr_len); + strncpy(addr_dev, addr_delim + 1, + strlen(pf_addr_str) - addr_len); + } else { + RTE_LOG(ERR, PMD, "parent address '%s' has invalid format\n", + pf_addr_str); + return -EINVAL; + } + + /* Fetch broker for PF */ + broker = rte_representor_broker_find(addr_bus, addr_dev); + if (broker == NULL) { + RTE_LOG(ERR, PMD, "Unable to find broker for %s on bus %s\n", + addr_dev, addr_bus); + return -EINVAL; + } + + /* Fetch representor slot for VF */ + rep_port = representor_port_find(broker, vport_id); + if (rep_port == NULL) { + RTE_LOG(ERR, PMD, "vport_id %i exceeds number of VF ports %i", + vport_id, broker->nb_virtual_ports); + return -EINVAL; + } + if (rep_port->state == RTE_REPRESENTOR_PORT_VALID) + return -EEXIST; + + /* Create representor ethdev */ + len_name = snprintf(str_device_name, RTE_ETH_NAME_MAX_LEN, "%s.%s.%i", + RTE_PORT_REPRESENTOR_DEVICE_NAME, addr_dev, vport_id); + if (len_name == RTE_ETH_NAME_MAX_LEN) { + RTE_LOG(ERR, PMD, "Port Representor name" + " exceeds RTE_ETH_NAME_MAX_LEN\n"); + return -EIO; + } + RTE_LOG(INFO, PMD, "Port Representor ethdev name: %s\n", + str_device_name); + rep_ethdev = rte_eth_dev_allocate(str_device_name); + if (rep_ethdev == NULL) { + RTE_LOG(ERR, PMD, "Unable to allocate Port Representor" + " ethdev\n"); + return -ENOMEM; + } + rep_ethdev->data->mac_addrs = rte_malloc("representor_port_mac_addrs", + sizeof(struct ether_addr), 0); + if (rep_ethdev->data->mac_addrs == NULL) { + RTE_LOG(ERR, PMD, "Unable to allocate Port Representor" + " mac_addrs\n"); + rte_eth_dev_release_port(rep_ethdev); + return -ENOMEM; + } + + /* Allocate private ethdev data for Port Representor info */ + rep_ethdev->data->dev_private = rte_malloc("rte_representor_port", + sizeof(struct rte_representor_port), 0); + if (rep_ethdev == NULL) { + RTE_LOG(ERR, PMD, "Unable to allocate Port Representor" + " private ethdev data\n"); + rte_eth_dev_release_port(rep_ethdev); + return -ENOMEM; + } + + /* Allocate an rte_device & rte_driver for the ethdev. */ + rep_port->rep_pcidev = rte_zmalloc_socket("rte_device", + sizeof(struct rte_device), 0, rte_socket_id()); + if (rep_port->rep_pcidev == NULL) { + RTE_LOG(ERR, PMD, "Unable to allocate Port Representor" + " devive\n"); + return -ENOMEM; + } + rep_port->rep_pcidrv = rte_zmalloc_socket("rte_driver", + sizeof(struct rte_driver), 0, rte_socket_id()); + if (rep_port->rep_pcidrv == NULL) { + RTE_LOG(ERR, PMD, "Unable to allocate Port Representor" + " driver\n"); + return -ENOMEM; + } + rep_port->rep_pcidrv->name = RTE_PORT_REPRESENTOR_DEVICE_NAME; + rep_port->rep_pcidev->driver = rep_port->rep_pcidrv; + rep_ethdev->device = rep_port->rep_pcidev; + + /* Register representor with broker */ + RTE_REPRESENTOR_PORT_VALID_ETHDEV_OR_RET_ERR(rep_ethdev, -EINVAL); + rep_port->vport_id = vport_id; + rep_port->ethdev = rep_ethdev; + rep_port->broker = broker; + rep_port->state = RTE_REPRESENTOR_PORT_VALID; + rep_ethdev->data->dev_private = rep_port; + + RTE_FUNC_PTR_OR_ERR_RET(broker->ops->port_init, -ENOTSUP); + retval = broker->ops->port_init(broker, rep_ethdev); + if (retval) { + rte_eth_dev_release_port(rep_ethdev); + return retval; + } + RTE_LOG(INFO, EAL, "Registered port representor " + "\n", + broker->bus, broker->device, vport_id); + + *port_id = rep_ethdev->data->port_id; + return 0; +} + +int +rte_representor_port_unregister(char *pf_addr_str, uint32_t vport_id) +{ + struct rte_representor_port *rep_port; + struct rte_representor_broker *broker; + char *addr_delim; + char addr_bus[RTE_ETH_NAME_MAX_LEN]; + char addr_dev[RTE_ETH_NAME_MAX_LEN]; + int addr_len; + + RTE_LOG(INFO, PMD, "Deleting representor for VF %i from %s\n", + vport_id, pf_addr_str); + + /* The parent param has to be provided with type_name formatted */ + memset(addr_bus, 0, sizeof(addr_bus)); + memset(addr_dev, 0, sizeof(addr_dev)); + addr_delim = strchr(pf_addr_str, '_'); + if (addr_delim) { + addr_len = addr_delim - pf_addr_str; + strncpy(addr_bus, pf_addr_str, addr_len); + strncpy(addr_dev, addr_delim + 1, + strlen(pf_addr_str) - addr_len); + } else { + RTE_LOG(ERR, PMD, "Parent address '%s' has invalid format\n", + pf_addr_str); + return -EINVAL; + } + + /* Fetch broker for PF */ + broker = rte_representor_broker_find(addr_bus, addr_dev); + if (broker == NULL) { + RTE_LOG(ERR, PMD, "Unable to find broker for %s on bus %s\n", + addr_dev, addr_bus); + return -EINVAL; + } + + /* Fetch representor slot for VF */ + rep_port = representor_port_find(broker, vport_id); + if (rep_port == NULL) { + RTE_LOG(ERR, PMD, "Unable to fetch Port Representor handle\n"); + return -EINVAL; + } + if (rep_port->state != RTE_REPRESENTOR_PORT_VALID) { + RTE_LOG(ERR, PMD, "Representor vport %i is not active\n", + vport_id); + return -EINVAL; + } + + /* Free up representor */ + RTE_FUNC_PTR_OR_ERR_RET(rep_port->broker->ops->port_uninit, -ENOTSUP); + rep_port->broker->ops->port_uninit(rep_port->broker, rep_port->ethdev); + rep_port->state = RTE_REPRESENTOR_PORT_INVALID; + rte_eth_dev_release_port(rep_port->ethdev); + + /* Cannot pass the pointers ethdev->device & ethdev->device->driver to + * rte_free() so have to store the allocated pointers in separate + * non-const variables. + */ + rte_free(rep_port->rep_pcidrv); + rte_free(rep_port->rep_pcidev); + + return 0; +} + +int +rte_representor_port_get_vport_id(struct rte_eth_dev *ethdev) +{ + struct rte_representor_port *port; + + RTE_REPRESENTOR_PORT_VALID_ETHDEV_OR_RET_ERR(ethdev, -EINVAL); + + port = (struct rte_representor_port *)ethdev->data->dev_private; + + return port->vport_id; +} diff --git a/lib/librte_representor/rte_port_representor.h b/lib/librte_representor/rte_port_representor.h new file mode 100644 index 0000000..ff0c567 --- /dev/null +++ b/lib/librte_representor/rte_port_representor.h @@ -0,0 +1,60 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2017 Intel Corporation. All rights reserved. + */ + +#ifndef _RTE_PORT_REPRESENTOR_H_ +#define _RTE_PORT_REPRESENTOR_H_ + +/** + * @file + * RTE Port Representor API + * + * Driver APIs for management and support of port representor + */ + +#ifdef __cplusplus +extern "C" { +#endif + +#include +#include +#include + +struct rte_representor_port; +struct rte_representor_broker; + +/** + * Registers a representor ethdev with the broker, allocating a port representor + * context and calling the representor port initialization function. + * + * @param pf_addr_str Address of parent PF + * @param vport_id Virtual port ID to register ethdev against + * @param port_id Pointer to integer that receives port_id + * + * @return + * - 0 on successful registration and initialization of representor port + * - errno on failure + */ +int +rte_representor_port_register(char *pf_addr_str, + uint32_t vport_id, uint16_t *port_id); + +/** + * Unregister a representor port, called during destruction of representor + * ethdev context. Freeing any allocated memory for the representor port. + * + * @param pf_addr_str Address of parent PF + * @param vport_id Virtual port ID to deregister + * + * @return + * - 0 on success + * - errno on failure + */ +int +rte_representor_port_unregister(char *pf_addr_str, uint32_t vport_id); + +#ifdef __cplusplus +} +#endif + +#endif /* _RTE_PORT_REPRESENTOR_H_ */ diff --git a/lib/librte_representor/rte_port_representor_driver.h b/lib/librte_representor/rte_port_representor_driver.h new file mode 100644 index 0000000..8827ebb --- /dev/null +++ b/lib/librte_representor/rte_port_representor_driver.h @@ -0,0 +1,138 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2017 Intel Corporation. All rights reserved. + */ + +#ifndef _RTE_PORT_REPRESENTOR_DRIVER_H_ +#define _RTE_PORT_REPRESENTOR_DRIVER_H_ + +#include + +/** + * @file + * RTE Port Representor Driver API + * + * Driver APIs for management and support of port representor + */ + +#ifdef __cplusplus +extern "C" { +#endif + +#define RTE_PORT_REPRESENTOR_DEVICE_NAME "net_representor" + +struct rte_representor_port; + +struct rte_representor_broker { + TAILQ_ENTRY(rte_representor_broker) next; + /**< Next broker object in linked list */ + + const char *bus; /**< bus name */ + const char *device; /**< device name */ + + uint16_t nb_virtual_ports; + /**< number of virtual ports supported by device */ + + struct rte_representor_port *vports; + /**< Representor port contexts */ + + struct rte_representor_broker_port_ops *ops; + /**< broker port operations functions */ + void *private_data; + /**< broker private data */ +}; + +/** Port Representor */ +struct rte_representor_port { + uint16_t vport_id; + /**< Virtual Port Identifier */ + struct rte_eth_dev *ethdev; + /**< ethdev handle of representor port */ + struct rte_representor_broker *broker; + /**< Broker handle to allow reverse lookup */ + enum { + RTE_REPRESENTOR_PORT_INVALID, + /**< No ethdev instantiated for virtual port */ + RTE_REPRESENTOR_PORT_VALID + /**< ethdev active for virtual port */ + } state; + /**< port state */ + void *priv_data; + /**< port private data */ + /**< Representor ethdev device */ + struct rte_device *rep_pcidev; + /**< Representor ethdev driver */ + struct rte_driver *rep_pcidrv; +}; + +typedef int (*representor_broker_port_init_t)( + struct rte_representor_broker *broker, + struct rte_eth_dev *ethdev); + +typedef int (*representor_broker_port_uninit_t)( + struct rte_representor_broker *broker, + struct rte_eth_dev *ethdev); + +struct rte_representor_broker_port_ops { + representor_broker_port_init_t port_init; + representor_broker_port_init_t port_uninit; +}; + +/** + * Initialize port representor broker + * + * This function is called by the PMDs initialization routine if the port + * representor is enabled by EAL command line argument. + * + * Verifies broker context parameters and initialises required resources. + * + * @param broker port representor broker context + * + * @return + * - 0 on success + * - errno on failure + */ +int +rte_representor_broker_init(struct rte_representor_broker *broker); + +/** + * Un-initialize the port representor broker freeing all associated resources. + * + * @param broker port representor broker + * + * @return + * - 0 on success + * - errno on failure + */ +int +rte_representor_broker_uninit(struct rte_representor_broker *broker); + +/** + * Fetch a port representor broker + * + * @param bus Bus parent PF is on + * @param device Address of PF + * + * @return + * - Pointer to port representor broker on success + * - NULL on failure + */ +struct rte_representor_broker * +rte_representor_broker_find(const char *bus, const char *device); + +/** + * Get the virtual port ID of given representor port ethdev context + * + * @param ethdev Port representor ethdev handle + * + * @return + * - Virtual port ID + * - errno on failure + */ +int +rte_representor_port_get_vport_id(struct rte_eth_dev *ethdev); + +#ifdef __cplusplus +} +#endif + +#endif /* _RTE_PORT_REPRESENTOR_DRIVER_H_ */ diff --git a/lib/librte_representor/rte_port_representor_version.map b/lib/librte_representor/rte_port_representor_version.map new file mode 100644 index 0000000..b26600c --- /dev/null +++ b/lib/librte_representor/rte_port_representor_version.map @@ -0,0 +1,8 @@ +DPDK_18.02 { + global: + + rte_representor_port_register; + rte_representor_port_unregister; + + local: *; +}; diff --git a/mk/rte.app.mk b/mk/rte.app.mk index 6a6a745..67266bf 100644 --- a/mk/rte.app.mk +++ b/mk/rte.app.mk @@ -104,6 +104,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_EAL) += -lrte_eal _LDLIBS-$(CONFIG_RTE_LIBRTE_CMDLINE) += -lrte_cmdline _LDLIBS-$(CONFIG_RTE_LIBRTE_REORDER) += -lrte_reorder _LDLIBS-$(CONFIG_RTE_LIBRTE_SCHED) += -lrte_sched +_LDLIBS-$(CONFIG_RTE_LIBRTE_REPRESENTOR) += -lrte_representor ifeq ($(CONFIG_RTE_EXEC_ENV_LINUXAPP),y) _LDLIBS-$(CONFIG_RTE_LIBRTE_KNI) += -lrte_kni -- 2.9.5