* [spp] [PATCH 0/3] revive vhost
@ 2019-12-23 5:00 Itsuro Oda
2019-12-23 5:00 ` [spp] [PATCH 1/3] multi process supported vhost PMD for SPP Itsuro Oda
` (5 more replies)
0 siblings, 6 replies; 37+ messages in thread
From: Itsuro Oda @ 2019-12-23 5:00 UTC (permalink / raw)
To: spp, ferruh.yigit, yasufum.o
vhost PMD can not be used by secondary processes since DPDK 18.11.
SPP project decided to have own vhost PMD which can be used by
secondary processes at the moment. This vhost PMD is based on the
original vhost PMD but is simplified very much only to support
functions used by SPP. Thereby it becomes easy to fix the probrem.
Itsuro Oda (3):
multi process supported vhost PMD for SPP
make use of vhost PMD for SPP
make robust against process start and termination
src/Makefile | 1 +
src/drivers/Makefile | 15 +
src/drivers/vhost/Makefile | 28 +
.../vhost/rte_pmd_spp_vhost_version.map | 4 +
src/drivers/vhost/rte_spp_vhost.c | 588 ++++++++++++++++++
src/mirror/Makefile | 3 +
src/nfv/Makefile | 3 +
src/nfv/commands.h | 1 +
src/nfv/main.c | 7 +-
src/primary/Makefile | 3 +
src/primary/main.c | 1 +
src/shared/common.h | 2 +-
src/shared/secondary/add_port.c | 8 +
src/shared/secondary/add_port.h | 2 +-
.../secondary/spp_worker_th/cmd_utils.c | 8 +-
src/vf/Makefile | 3 +
16 files changed, 672 insertions(+), 5 deletions(-)
create mode 100644 src/drivers/Makefile
create mode 100644 src/drivers/vhost/Makefile
create mode 100644 src/drivers/vhost/rte_pmd_spp_vhost_version.map
create mode 100644 src/drivers/vhost/rte_spp_vhost.c
--
2.17.1
^ permalink raw reply [flat|nested] 37+ messages in thread
* [spp] [PATCH 1/3] multi process supported vhost PMD for SPP
2019-12-23 5:00 [spp] [PATCH 0/3] revive vhost Itsuro Oda
@ 2019-12-23 5:00 ` Itsuro Oda
2019-12-23 5:00 ` [spp] [PATCH 2/3] make use of " Itsuro Oda
` (4 subsequent siblings)
5 siblings, 0 replies; 37+ messages in thread
From: Itsuro Oda @ 2019-12-23 5:00 UTC (permalink / raw)
To: spp, ferruh.yigit, yasufum.o
vhost PMD can not be used by secondary processes since DPDK 18.11.
SPP project decided to have own vhost PMD which can be used by
secondary processes at the moment. This vhost PMD is based on the
original vhost PMD but is simplified very much only to support
functions used by SPP. Thereby it becomes easy to fix the probrem.
The main idea of the fix is that execution of vhost start/stop
is moved to eth_dev_start/stop from probe/remove.
Note that only process which executes eth_dev_start can use the
vhost device although the vhost device is shared among the primary
process and secondary processes. Once eth_dev_stop is executed by
the process which used the vhost device, it is available to be
used by any process. It is user responsibility that multipul
processes don't use the vhost device at the same time.
Signed-off-by: Itsuro Oda <oda@valinux.co.jp>
---
src/Makefile | 1 +
src/drivers/Makefile | 15 +
src/drivers/vhost/Makefile | 28 +
.../vhost/rte_pmd_spp_vhost_version.map | 4 +
src/drivers/vhost/rte_spp_vhost.c | 588 ++++++++++++++++++
5 files changed, 636 insertions(+)
create mode 100644 src/drivers/Makefile
create mode 100644 src/drivers/vhost/Makefile
create mode 100644 src/drivers/vhost/rte_pmd_spp_vhost_version.map
create mode 100644 src/drivers/vhost/rte_spp_vhost.c
diff --git a/src/Makefile b/src/Makefile
index cab80db..2f46606 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -38,6 +38,7 @@ RTE_TARGET ?= x86_64-native-linuxapp-gcc
include $(RTE_SDK)/mk/rte.vars.mk
+DIRS-y += drivers
DIRS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += primary
DIRS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += nfv
DIRS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += vf
diff --git a/src/drivers/Makefile b/src/drivers/Makefile
new file mode 100644
index 0000000..f2deba7
--- /dev/null
+++ b/src/drivers/Makefile
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+
+ifeq ($(RTE_SDK),)
+$(error "Please define RTE_SDK environment variable")
+endif
+
+# Default target, can be overriden by command line or environment
+RTE_TARGET ?= x86_64-native-linuxapp-gcc
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+DIRS-y += vhost
+
+include $(RTE_SDK)/mk/rte.extsubdir.mk
diff --git a/src/drivers/vhost/Makefile b/src/drivers/vhost/Makefile
new file mode 100644
index 0000000..e29c330
--- /dev/null
+++ b/src/drivers/vhost/Makefile
@@ -0,0 +1,28 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+#
+# library name
+#
+LIB = librte_pmd_spp_vhost.a
+
+LDLIBS += -lpthread
+LDLIBS += -lrte_eal -lrte_mbuf -lrte_mempool -lrte_ring
+LDLIBS += -lrte_ethdev -lrte_net -lrte_kvargs -lrte_vhost
+LDLIBS += -lrte_bus_vdev
+
+CFLAGS += -O3
+CFLAGS += $(WERROR_FLAGS)
+
+EXPORT_MAP := rte_pmd_spp_vhost_version.map
+
+LIBABIVER := 2
+
+#
+# all source are stored in SRCS-y
+#
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_VHOST) += rte_spp_vhost.c
+
+include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/src/drivers/vhost/rte_pmd_spp_vhost_version.map b/src/drivers/vhost/rte_pmd_spp_vhost_version.map
new file mode 100644
index 0000000..ef35398
--- /dev/null
+++ b/src/drivers/vhost/rte_pmd_spp_vhost_version.map
@@ -0,0 +1,4 @@
+DPDK_2.0 {
+
+ local: *;
+};
diff --git a/src/drivers/vhost/rte_spp_vhost.c b/src/drivers/vhost/rte_spp_vhost.c
new file mode 100644
index 0000000..8eb6503
--- /dev/null
+++ b/src/drivers/vhost/rte_spp_vhost.c
@@ -0,0 +1,588 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2016 IGEL Co., Ltd.
+ * Copyright(c) 2016-2018 Intel Corporation
+ * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+ */
+#include <unistd.h>
+#include <pthread.h>
+#include <stdbool.h>
+
+#include <rte_mbuf.h>
+#include <rte_ethdev_driver.h>
+#include <rte_ethdev_vdev.h>
+#include <rte_malloc.h>
+#include <rte_memcpy.h>
+#include <rte_bus_vdev.h>
+#include <rte_kvargs.h>
+#include <rte_vhost.h>
+#include <rte_spinlock.h>
+
+static int vhost_logtype;
+
+#define VHOST_LOG(level, ...) \
+ rte_log(RTE_LOG_ ## level, vhost_logtype, __VA_ARGS__)
+
+enum {VIRTIO_RXQ, VIRTIO_TXQ, VIRTIO_QNUM};
+
+#define ETH_VHOST_IFACE_ARG "iface"
+#define ETH_VHOST_QUEUES_ARG "queues"
+#define ETH_VHOST_CLIENT_ARG "client"
+
+static const char *valid_arguments[] = {
+ ETH_VHOST_IFACE_ARG,
+ ETH_VHOST_QUEUES_ARG,
+ ETH_VHOST_CLIENT_ARG,
+ NULL
+};
+
+struct vhost_queue {
+ struct pmd_internal *internal;
+ struct rte_mempool *mb_pool;
+ uint16_t virtqueue_id;
+ rte_spinlock_t queuing_lock;
+ uint64_t pkts;
+ uint64_t missed_pkts;
+};
+
+struct pmd_internal {
+ uint16_t max_queues;
+ uint64_t vhost_flags;
+ struct rte_eth_dev_data *eth_dev_data;
+ int vid;
+ char iface_name[PATH_MAX];
+};
+
+static struct rte_eth_link pmd_link = {
+ .link_speed = 10000,
+ .link_duplex = ETH_LINK_FULL_DUPLEX,
+ .link_status = ETH_LINK_DOWN
+};
+
+#define MZ_RTE_VHOST_PMD_INTERNAL "vhost_pmd_internal"
+static struct pmd_internal **pmd_internal_list;
+
+static uint16_t
+eth_vhost_rx(void *q, struct rte_mbuf **bufs, uint16_t nb_bufs)
+{
+ struct vhost_queue *r = q;
+ struct pmd_internal *internal;
+ uint16_t nb_rx = 0;
+
+ if (!q)
+ return 0;
+
+ internal = r->internal;
+ rte_spinlock_lock(&r->queuing_lock);
+ if (internal->vid == -1)
+ goto out;
+
+ /* Dequeue packets from guest TX queue */
+ nb_rx = rte_vhost_dequeue_burst(internal->vid, r->virtqueue_id,
+ r->mb_pool, bufs, nb_bufs);
+ r->pkts += nb_rx;
+
+out:
+ rte_spinlock_unlock(&r->queuing_lock);
+
+ return nb_rx;
+}
+
+static uint16_t
+eth_vhost_tx(void *q, struct rte_mbuf **bufs, uint16_t nb_bufs)
+{
+ struct vhost_queue *r = q;
+ struct pmd_internal *internal;
+ uint16_t i, nb_tx = 0;
+
+ if (!q)
+ return 0;
+
+ internal = r->internal;
+ rte_spinlock_lock(&r->queuing_lock);
+ if (internal->vid == -1)
+ goto out;
+
+ /* Enqueue packets to guest RX queue */
+ nb_tx = rte_vhost_enqueue_burst(internal->vid, r->virtqueue_id,
+ bufs, nb_bufs);
+ r->pkts += nb_tx;
+ r->missed_pkts += nb_bufs - nb_tx;
+
+ for (i = 0; i < nb_tx; i++)
+ rte_pktmbuf_free(bufs[i]);
+
+out:
+ rte_spinlock_unlock(&r->queuing_lock);
+
+ return nb_tx;
+}
+
+static inline struct pmd_internal *
+find_internal_resource(int vid)
+{
+ struct pmd_internal *internal;
+ int i;
+ char ifname[PATH_MAX];
+
+ if (rte_vhost_get_ifname(vid, ifname, sizeof(ifname)) == -1)
+ return NULL;
+
+ /* likely(found in a few loops) */
+ for (i = 0; i < RTE_MAX_ETHPORTS; i++) {
+ internal = pmd_internal_list[i];
+ if (internal != NULL && !strcmp(internal->iface_name, ifname)) {
+ return internal;
+ }
+ }
+
+ return NULL;
+}
+
+static int
+new_device(int vid)
+{
+ struct pmd_internal *internal;
+
+ internal = find_internal_resource(vid);
+ if (internal == NULL) {
+ VHOST_LOG(INFO, "Invalid device : %d\n", vid);
+ return -1;
+ }
+
+ internal->vid = vid;
+ internal->eth_dev_data->dev_link.link_status = ETH_LINK_UP;
+
+ VHOST_LOG(INFO, "Vhost device %d created\n", vid);
+
+ return 0;
+}
+
+static void
+destroy_device(int vid)
+{
+ struct pmd_internal *internal;
+ struct vhost_queue *vq;
+ struct rte_eth_dev_data *data;
+ int i;
+
+ internal = find_internal_resource(vid);
+ if (internal == NULL) {
+ VHOST_LOG(ERR, "Invalid device : %d\n", vid);
+ return;
+ }
+ data = internal->eth_dev_data;
+
+ /* wait inflight queuing done */
+ for (i = 0; i < data->nb_rx_queues; i++) {
+ if ((vq = data->rx_queues[i]) != NULL)
+ rte_spinlock_lock(&vq->queuing_lock);
+ }
+ for (i = 0; i < data->nb_tx_queues; i++) {
+ if ((vq = data->tx_queues[i]) != NULL)
+ rte_spinlock_lock(&vq->queuing_lock);
+ }
+
+ data->dev_link.link_status = ETH_LINK_DOWN;
+ internal->vid = -1;
+
+ for (i = 0; i < data->nb_rx_queues; i++) {
+ if ((vq = data->rx_queues[i]) != NULL)
+ rte_spinlock_unlock(&vq->queuing_lock);
+ }
+ for (i = 0; i < data->nb_tx_queues; i++) {
+ if ((vq = data->tx_queues[i]) != NULL)
+ rte_spinlock_unlock(&vq->queuing_lock);
+ }
+
+ VHOST_LOG(INFO, "Vhost device %d destroyed\n", vid);
+}
+
+static struct vhost_device_ops vhost_ops = {
+ .new_device = new_device,
+ .destroy_device = destroy_device,
+};
+
+static int
+eth_dev_start(struct rte_eth_dev *eth_dev)
+{
+ struct pmd_internal *internal = eth_dev->data->dev_private;
+
+ if (rte_vhost_driver_register(internal->iface_name, internal->vhost_flags))
+ return -1;
+
+ if (rte_vhost_driver_callback_register(internal->iface_name, &vhost_ops) < 0) {
+ VHOST_LOG(ERR, "Can't register callbacks\n");
+ return -1;
+ }
+
+ if (rte_vhost_driver_start(internal->iface_name) < 0) {
+ VHOST_LOG(ERR, "Failed to start driver for %s\n",
+ internal->iface_name);
+ return -1;
+ }
+
+ return 0;
+}
+
+static void
+eth_dev_stop(struct rte_eth_dev *dev)
+{
+ struct pmd_internal *internal = dev->data->dev_private;
+
+ rte_vhost_driver_unregister(internal->iface_name);
+}
+
+static int
+eth_dev_configure(struct rte_eth_dev *dev __rte_unused)
+{
+ return 0;
+}
+
+static void
+eth_dev_info(struct rte_eth_dev *dev,
+ struct rte_eth_dev_info *dev_info)
+{
+ struct pmd_internal *internal = dev->data->dev_private;
+
+ dev_info->max_mac_addrs = 1;
+ dev_info->max_rx_pktlen = (uint32_t)-1;
+ dev_info->max_rx_queues = internal->max_queues;
+ dev_info->max_tx_queues = internal->max_queues;
+ dev_info->min_rx_bufsize = 0;
+}
+
+static int
+eth_rx_queue_setup(struct rte_eth_dev *dev, uint16_t rx_queue_id,
+ uint16_t nb_rx_desc __rte_unused,
+ unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf __rte_unused,
+ struct rte_mempool *mb_pool)
+{
+ struct pmd_internal *internal = dev->data->dev_private;
+ struct vhost_queue *vq;
+
+ vq = rte_zmalloc_socket(NULL, sizeof(struct vhost_queue),
+ RTE_CACHE_LINE_SIZE, socket_id);
+ if (vq == NULL) {
+ VHOST_LOG(ERR, "Failed to allocate memory for rx queue\n");
+ return -ENOMEM;
+ }
+
+ vq->internal = internal;
+ vq->mb_pool = mb_pool;
+ vq->virtqueue_id = rx_queue_id * VIRTIO_QNUM + VIRTIO_TXQ;
+ rte_spinlock_init(&vq->queuing_lock);
+ dev->data->rx_queues[rx_queue_id] = vq;
+
+ return 0;
+}
+
+static int
+eth_tx_queue_setup(struct rte_eth_dev *dev, uint16_t tx_queue_id,
+ uint16_t nb_tx_desc __rte_unused,
+ unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf __rte_unused)
+{
+ struct pmd_internal *internal = dev->data->dev_private;
+ struct vhost_queue *vq;
+
+ vq = rte_zmalloc_socket(NULL, sizeof(struct vhost_queue),
+ RTE_CACHE_LINE_SIZE, socket_id);
+ if (vq == NULL) {
+ VHOST_LOG(ERR, "Failed to allocate memory for tx queue\n");
+ return -ENOMEM;
+ }
+
+ vq->internal = internal;
+ vq->virtqueue_id = tx_queue_id * VIRTIO_QNUM + VIRTIO_RXQ;
+ rte_spinlock_init(&vq->queuing_lock);
+ dev->data->tx_queues[tx_queue_id] = vq;
+
+ return 0;
+}
+
+static void
+eth_queue_release(void *q)
+{
+ rte_free(q);
+}
+
+static int
+eth_link_update(struct rte_eth_dev *dev __rte_unused,
+ int wait_to_complete __rte_unused)
+{
+ return 0;
+}
+
+static int
+eth_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats)
+{
+ unsigned i;
+ unsigned long rx_total = 0, tx_total = 0, tx_err_total = 0;
+ struct vhost_queue *vq;
+
+ for (i = 0; i < RTE_ETHDEV_QUEUE_STAT_CNTRS &&
+ i < dev->data->nb_rx_queues; i++) {
+ if ((vq = dev->data->rx_queues[i]) != NULL) {
+ stats->q_ipackets[i] = vq->pkts;
+ rx_total += vq->pkts;
+ }
+ }
+
+ for (i = 0; i < RTE_ETHDEV_QUEUE_STAT_CNTRS &&
+ i < dev->data->nb_tx_queues; i++) {
+ if ((vq = dev->data->rx_queues[i]) != NULL) {
+ stats->q_opackets[i] = vq->pkts;
+ tx_total += vq->pkts;
+ stats->q_errors[i] = vq->missed_pkts;
+ tx_err_total += vq->missed_pkts;
+ }
+ }
+
+ stats->ipackets = rx_total;
+ stats->opackets = tx_total;
+ stats->oerrors = tx_err_total;
+
+ return 0;
+}
+
+static void
+eth_stats_reset(struct rte_eth_dev *dev)
+{
+ struct vhost_queue *vq;
+ unsigned i;
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ if ((vq = dev->data->rx_queues[i]) != NULL)
+ vq->pkts = 0;
+ }
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ if ((vq = dev->data->rx_queues[i]) != NULL) {
+ vq->pkts = 0;
+ vq->missed_pkts = 0;
+ }
+ }
+}
+
+static const struct eth_dev_ops ops = {
+ .dev_start = eth_dev_start,
+ .dev_stop = eth_dev_stop,
+ .dev_configure = eth_dev_configure,
+ .dev_infos_get = eth_dev_info,
+ .rx_queue_setup = eth_rx_queue_setup,
+ .tx_queue_setup = eth_tx_queue_setup,
+ .rx_queue_release = eth_queue_release,
+ .tx_queue_release = eth_queue_release,
+ .link_update = eth_link_update,
+ .stats_get = eth_stats_get,
+ .stats_reset = eth_stats_reset,
+};
+
+static int
+init_shared_data(void)
+{
+ const struct rte_memzone *mz;
+
+ if (pmd_internal_list == NULL) {
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+ size_t len = sizeof(*pmd_internal_list) * RTE_MAX_ETHPORTS;
+ mz = rte_memzone_reserve(MZ_RTE_VHOST_PMD_INTERNAL,
+ len, rte_socket_id(), 0);
+ if (mz)
+ memset(mz->addr, 0, len);
+ } else
+ mz = rte_memzone_lookup(MZ_RTE_VHOST_PMD_INTERNAL);
+ if (mz == NULL) {
+ VHOST_LOG(ERR, "Cannot allocate vhost shared data\n");
+ return -1;
+ }
+ pmd_internal_list = mz->addr;
+ }
+
+ return 0;
+}
+
+static int
+eth_dev_vhost_create(struct rte_vdev_device *dev, char *iface_name,
+ int16_t queues, const unsigned int numa_node, uint64_t flags)
+{
+ struct rte_eth_dev_data *data;
+ struct pmd_internal *internal;
+ struct rte_eth_dev *eth_dev;
+ struct rte_ether_addr *eth_addr;
+
+ VHOST_LOG(INFO, "Creating VHOST-USER backend on numa socket %u\n",
+ numa_node);
+
+ eth_addr = rte_zmalloc_socket(NULL, sizeof(*eth_addr), 0, numa_node);
+ if (eth_addr == NULL)
+ return -ENOMEM;
+
+ eth_dev = rte_eth_vdev_allocate(dev, sizeof(*internal));
+ if (eth_dev == NULL)
+ return -ENOMEM;
+
+ data = eth_dev->data;
+
+ data->dev_link = pmd_link;
+ data->dev_flags = RTE_ETH_DEV_INTR_LSC;
+ data->mac_addrs = eth_addr;
+
+ eth_dev->dev_ops = &ops;
+ eth_dev->rx_pkt_burst = eth_vhost_rx;
+ eth_dev->tx_pkt_burst = eth_vhost_tx;
+
+ internal = data->dev_private;
+
+ internal->max_queues = queues;
+ internal->vid = -1;
+ internal->vhost_flags = flags;
+ internal->eth_dev_data = data;
+ strncpy(internal->iface_name, iface_name, sizeof(internal->iface_name));
+
+ pmd_internal_list[data->port_id] = internal;
+
+ rte_eth_dev_probing_finish(eth_dev);
+
+ return 0;
+}
+
+static inline int
+open_iface(const char *key __rte_unused, const char *value, void *extra_args)
+{
+ const char **iface_name = extra_args;
+
+ if (value == NULL)
+ return -1;
+
+ *iface_name = value;
+
+ return 0;
+}
+
+static inline int
+open_int(const char *key __rte_unused, const char *value, void *extra_args)
+{
+ uint16_t *n = extra_args;
+ char *endptr;
+
+ if (value == NULL)
+ return -1;
+
+ *n = (uint16_t)strtoul(value, &endptr, 0);
+ if (*endptr != '\0' || errno == ERANGE)
+ return -1;
+
+ return 0;
+}
+
+static int
+rte_pmd_vhost_probe(struct rte_vdev_device *dev)
+{
+ struct rte_kvargs *kvlist = NULL;
+ int ret = 0;
+ char *iface_name = NULL;
+ uint16_t queues = 1;
+ uint64_t flags = 0;
+ uint16_t client_mode = 0;
+ struct rte_eth_dev *eth_dev;
+ const char *name = rte_vdev_device_name(dev);
+
+ VHOST_LOG(INFO, "Initializing pmd_vhost for %s\n", name);
+
+ if (init_shared_data() == -1)
+ return -ENOMEM;
+
+ if (rte_eal_process_type() == RTE_PROC_SECONDARY) {
+ eth_dev = rte_eth_dev_attach_secondary(name);
+ if (!eth_dev) {
+ VHOST_LOG(ERR, "Failed to probe %s\n", name);
+ return -ENOENT;
+ }
+ eth_dev->rx_pkt_burst = eth_vhost_rx;
+ eth_dev->tx_pkt_burst = eth_vhost_tx;
+
+ eth_dev->dev_ops = &ops;
+ eth_dev->device = &dev->device;
+ rte_eth_dev_probing_finish(eth_dev);
+ return 0;
+ }
+
+ kvlist = rte_kvargs_parse(rte_vdev_device_args(dev), valid_arguments);
+ if (kvlist == NULL)
+ return -EINVAL;
+
+ if (rte_kvargs_process(kvlist, ETH_VHOST_IFACE_ARG,
+ &open_iface, &iface_name) == -1 ||
+ rte_kvargs_process(kvlist, ETH_VHOST_QUEUES_ARG,
+ &open_int, &queues) == -1 ||
+ rte_kvargs_process(kvlist, ETH_VHOST_CLIENT_ARG,
+ &open_int, &client_mode) == -1) {
+ rte_kvargs_free(kvlist);
+ return -EINVAL;
+ }
+
+ if (iface_name == NULL ||
+ queues == 0 || queues > RTE_MAX_QUEUES_PER_PORT ||
+ (client_mode != 0 && client_mode != 1)) {
+ rte_kvargs_free(kvlist);
+ return -EINVAL;
+ }
+
+ if (client_mode)
+ flags |= RTE_VHOST_USER_CLIENT;
+
+ ret = eth_dev_vhost_create(dev, iface_name, queues, rte_socket_id(), flags);
+
+ rte_kvargs_free(kvlist);
+ return ret;
+}
+
+static int
+rte_pmd_vhost_remove(struct rte_vdev_device *dev)
+{
+ const char *name = rte_vdev_device_name(dev);
+ struct rte_eth_dev *eth_dev;
+ int i;
+
+ VHOST_LOG(INFO, "Un-Initializing pmd_vhost for %s\n", name);
+
+ eth_dev = rte_eth_dev_allocated(name);
+ if (eth_dev == NULL)
+ return 0;
+
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+ return rte_eth_dev_release_port(eth_dev);
+
+ if (eth_dev->data->dev_started) {
+ VHOST_LOG(WARNING, "device must be stoped.\n");
+ }
+
+ for (i = 0; i < eth_dev->data->nb_rx_queues; i++)
+ rte_free(eth_dev->data->rx_queues[i]);
+
+ for (i = 0; i < eth_dev->data->nb_tx_queues; i++)
+ rte_free(eth_dev->data->tx_queues[i]);
+
+ pmd_internal_list[eth_dev->data->port_id] = NULL;
+
+ return rte_eth_dev_release_port(eth_dev);
+}
+
+static struct rte_vdev_driver pmd_vhost_drv = {
+ .probe = rte_pmd_vhost_probe,
+ .remove = rte_pmd_vhost_remove,
+};
+
+RTE_PMD_REGISTER_VDEV(spp_vhost, pmd_vhost_drv);
+RTE_PMD_REGISTER_PARAM_STRING(spp_vhost,
+ "iface=<ifc> "
+ "queues=<int> "
+ "client=<0|1> ");
+
+RTE_INIT(vhost_init_log)
+{
+ vhost_logtype = rte_log_register("pmd.spp.vhost");
+ if (vhost_logtype >= 0)
+ rte_log_set_level(vhost_logtype, RTE_LOG_NOTICE);
+}
--
2.17.1
^ permalink raw reply [flat|nested] 37+ messages in thread
* [spp] [PATCH 2/3] make use of vhost PMD for SPP
2019-12-23 5:00 [spp] [PATCH 0/3] revive vhost Itsuro Oda
2019-12-23 5:00 ` [spp] [PATCH 1/3] multi process supported vhost PMD for SPP Itsuro Oda
@ 2019-12-23 5:00 ` Itsuro Oda
2019-12-23 5:00 ` [spp] [PATCH 3/3] make robust against process start and termination Itsuro Oda
` (3 subsequent siblings)
5 siblings, 0 replies; 37+ messages in thread
From: Itsuro Oda @ 2019-12-23 5:00 UTC (permalink / raw)
To: spp, ferruh.yigit, yasufum.o
It is necessary to change only two defines to switch using
original vhost PMD to vhost PMD for SPP (spp_vhost).
Makefiles of each SPP processes are added to link spp_vhost.
Signed-off-by: Itsuro Oda <oda@valinux.co.jp>
---
src/mirror/Makefile | 3 +++
src/nfv/Makefile | 3 +++
src/primary/Makefile | 3 +++
src/shared/common.h | 2 +-
src/shared/secondary/add_port.h | 2 +-
src/vf/Makefile | 3 +++
6 files changed, 14 insertions(+), 2 deletions(-)
diff --git a/src/mirror/Makefile b/src/mirror/Makefile
index 0bd079a..f82b08f 100644
--- a/src/mirror/Makefile
+++ b/src/mirror/Makefile
@@ -53,4 +53,7 @@ LDLIBS += -lrte_pmd_ring
LDLIBS += -lrte_pmd_vhost
endif
+SPP_DRIVERS_DIR = $(BASE_OUTPUT)/src/drivers
+EXTRA_LDLIBS = -L$(SPP_DRIVERS_DIR)/vhost --whole-archive -lrte_pmd_spp_vhost --no-whole-archive
+
include $(RTE_SDK)/mk/rte.extapp.mk
diff --git a/src/nfv/Makefile b/src/nfv/Makefile
index c3d2806..a717616 100644
--- a/src/nfv/Makefile
+++ b/src/nfv/Makefile
@@ -27,4 +27,7 @@ LDLIBS += -lrte_pmd_ring
LDLIBS += -lrte_pmd_vhost
endif
+SPP_DRIVERS_DIR = $(BASE_OUTPUT)/src/drivers
+EXTRA_LDLIBS = -L$(SPP_DRIVERS_DIR)/vhost --whole-archive -lrte_pmd_spp_vhost --no-whole-archive
+
include $(RTE_SDK)/mk/rte.extapp.mk
diff --git a/src/primary/Makefile b/src/primary/Makefile
index 14d8f30..ace6228 100644
--- a/src/primary/Makefile
+++ b/src/primary/Makefile
@@ -41,4 +41,7 @@ endif
# and so the next line can be removed in those cases.
EXTRA_CFLAGS += -fno-strict-aliasing
+SPP_DRIVERS_DIR = $(BASE_OUTPUT)/src/drivers
+EXTRA_LDLIBS = -L$(SPP_DRIVERS_DIR)/vhost --whole-archive -lrte_pmd_spp_vhost --no-whole-archive
+
include $(RTE_SDK)/mk/rte.extapp.mk
diff --git a/src/shared/common.h b/src/shared/common.h
index 9c46a64..431ad3e 100644
--- a/src/shared/common.h
+++ b/src/shared/common.h
@@ -34,7 +34,7 @@
#define VDEV_ETH_RING "eth_ring"
#define VDEV_NET_RING "net_ring"
-#define VDEV_ETH_VHOST "eth_vhost"
+#define VDEV_ETH_VHOST "spp_vhost"
#define VDEV_NET_VHOST "net_vhost"
#define VDEV_PCAP "net_pcap"
#define VDEV_ETH_TAP "eth_tap"
diff --git a/src/shared/secondary/add_port.h b/src/shared/secondary/add_port.h
index cfae1af..a75b28b 100644
--- a/src/shared/secondary/add_port.h
+++ b/src/shared/secondary/add_port.h
@@ -10,7 +10,7 @@
#define NR_DESCS 128
#define VHOST_IFACE_NAME "/tmp/sock%u"
-#define VHOST_BACKEND_NAME "eth_vhost%u"
+#define VHOST_BACKEND_NAME "spp_vhost%u"
#define PCAP_PMD_DEV_NAME "eth_pcap%u"
#define NULL_PMD_DEV_NAME "eth_null%u"
diff --git a/src/vf/Makefile b/src/vf/Makefile
index 43e5e4d..e42f8b7 100644
--- a/src/vf/Makefile
+++ b/src/vf/Makefile
@@ -48,4 +48,7 @@ LDLIBS += -lrte_pmd_ring
LDLIBS += -lrte_pmd_vhost
endif
+SPP_DRIVERS_DIR = $(BASE_OUTPUT)/src/drivers
+EXTRA_LDLIBS = -L$(SPP_DRIVERS_DIR)/vhost --whole-archive -lrte_pmd_spp_vhost --no-whole-archive
+
include $(RTE_SDK)/mk/rte.extapp.mk
--
2.17.1
^ permalink raw reply [flat|nested] 37+ messages in thread
* [spp] [PATCH 3/3] make robust against process start and termination
2019-12-23 5:00 [spp] [PATCH 0/3] revive vhost Itsuro Oda
2019-12-23 5:00 ` [spp] [PATCH 1/3] multi process supported vhost PMD for SPP Itsuro Oda
2019-12-23 5:00 ` [spp] [PATCH 2/3] make use of " Itsuro Oda
@ 2019-12-23 5:00 ` Itsuro Oda
2019-12-24 5:57 ` [spp] [PATCH 0/3] revive vhost Yasufumi Ogawa
` (2 subsequent siblings)
5 siblings, 0 replies; 37+ messages in thread
From: Itsuro Oda @ 2019-12-23 5:00 UTC (permalink / raw)
To: spp, ferruh.yigit, yasufum.o
A vhost device is shared among the primary process and secondary
processes. When a secodary process starts it recognizes vhost
devices if they are used by processes already started. It is not
appropriate to include to port information as PHY devices.
There are some fixes to ensure eth_dev_stop too.
Signed-off-by: Itsuro Oda <oda@valinux.co.jp>
---
src/nfv/commands.h | 1 +
src/nfv/main.c | 7 ++++++-
src/primary/main.c | 1 +
src/shared/secondary/add_port.c | 8 ++++++++
src/shared/secondary/spp_worker_th/cmd_utils.c | 8 ++++++--
5 files changed, 22 insertions(+), 3 deletions(-)
diff --git a/src/nfv/commands.h b/src/nfv/commands.h
index f6c2305..7e50c8c 100644
--- a/src/nfv/commands.h
+++ b/src/nfv/commands.h
@@ -25,6 +25,7 @@ do_del(char *p_type, int p_id)
port_id = find_port_id(p_id, VHOST);
if (port_id == PORT_RESET)
return -1;
+ rte_eth_dev_stop(port_id);
dev_detach_by_port_id(port_id);
} else if (!strcmp(p_type, "ring")) {
diff --git a/src/nfv/main.c b/src/nfv/main.c
index 513a98d..f2c6bfc 100644
--- a/src/nfv/main.c
+++ b/src/nfv/main.c
@@ -211,7 +211,12 @@ main(int argc, char *argv[])
if (port_type == PHY) {
port_id = nof_phy_port;
nof_phy_port++;
- }
+ } else if (port_type == VHOST)
+ continue;
+ /* NOTE: vhost may be used another process. even if no
+ * process uses, it is necessary to "add vhost" explicitly.
+ * not display to avoid confusion.
+ */
/* Update ports_fwd_array with phy port. */
ports_fwd_array[i].in_port_id = i;
diff --git a/src/primary/main.c b/src/primary/main.c
index 26e9c42..a1ca791 100644
--- a/src/primary/main.c
+++ b/src/primary/main.c
@@ -874,6 +874,7 @@ del_port(char *p_type, int p_id)
dev_id = find_ethdev_id(p_id, VHOST);
if (dev_id == PORT_RESET)
return -1;
+ rte_eth_dev_stop(dev_id);
dev_detach_by_port_id(dev_id);
} else if (!strcmp(p_type, "ring")) {
diff --git a/src/shared/secondary/add_port.c b/src/shared/secondary/add_port.c
index b072140..d845250 100644
--- a/src/shared/secondary/add_port.c
+++ b/src/shared/secondary/add_port.c
@@ -170,6 +170,14 @@ add_vhost_pmd(int index)
return ret;
}
+ /* NOTE: make sure the eth_dev is stopped.
+ * it is for the case a secondary process which used the vhost
+ * was down without stopping the device.
+ * note that it is still user responsibility to prevent multipul
+ * processes use a vhost at the same time.
+ */
+ rte_eth_dev_stop(vhost_port_id);
+
ret = rte_eth_dev_configure(vhost_port_id, nr_queues, nr_queues,
&port_conf);
if (ret < 0) {
diff --git a/src/shared/secondary/spp_worker_th/cmd_utils.c b/src/shared/secondary/spp_worker_th/cmd_utils.c
index 010a4b6..69d7222 100644
--- a/src/shared/secondary/spp_worker_th/cmd_utils.c
+++ b/src/shared/secondary/spp_worker_th/cmd_utils.c
@@ -437,8 +437,12 @@ init_host_port_info(void)
p_iface_info->phy[port_id].ethdev_port_id = port_id;
break;
case VHOST:
- p_iface_info->vhost[port_id].iface_type = port_type;
- p_iface_info->vhost[port_id].ethdev_port_id = port_id;
+ /* NOTE: a vhost can be used by one process.
+ * even if it exists, it is necessary to do
+ * add_vhost_pmd to setup the device.
+ * note that it is user responsibility to prevent
+ * multipul processes use a vhost at the same time.
+ */
break;
case RING:
p_iface_info->ring[port_id].iface_type = port_type;
--
2.17.1
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [spp] [PATCH 0/3] revive vhost
2019-12-23 5:00 [spp] [PATCH 0/3] revive vhost Itsuro Oda
` (2 preceding siblings ...)
2019-12-23 5:00 ` [spp] [PATCH 3/3] make robust against process start and termination Itsuro Oda
@ 2019-12-24 5:57 ` Yasufumi Ogawa
2019-12-24 6:09 ` Itsuro ODA
2019-12-25 4:49 ` [spp] [PATCH v2 00/12] " Itsuro Oda
2020-01-09 23:10 ` [spp] [PATCH v3 00/12] revive vhost Itsuro Oda
5 siblings, 1 reply; 37+ messages in thread
From: Yasufumi Ogawa @ 2019-12-24 5:57 UTC (permalink / raw)
To: Itsuro Oda; +Cc: spp, ferruh.yigit
On 2019/12/23 14:00, Itsuro Oda wrote:
> vhost PMD can not be used by secondary processes since DPDK 18.11.
> SPP project decided to have own vhost PMD which can be used by
> secondary processes at the moment. This vhost PMD is based on the
> original vhost PMD but is simplified very much only to support
> functions used by SPP. Thereby it becomes easy to fix the probrem.
Hi,
Could you add tags, such as "docs", "spp_primary" or so, at the head of
titles? I'd like to ask another thing is to divide your patches into
more tiny ones to be easy to add the tags. Although some patches are
very tiny, but it is helpful for finding the changes in future.
Thanks,
Yasufumi
>
> Itsuro Oda (3):
> multi process supported vhost PMD for SPP
> make use of vhost PMD for SPP
> make robust against process start and termination
>
> src/Makefile | 1 +
> src/drivers/Makefile | 15 +
> src/drivers/vhost/Makefile | 28 +
> .../vhost/rte_pmd_spp_vhost_version.map | 4 +
> src/drivers/vhost/rte_spp_vhost.c | 588 ++++++++++++++++++
> src/mirror/Makefile | 3 +
> src/nfv/Makefile | 3 +
> src/nfv/commands.h | 1 +
> src/nfv/main.c | 7 +-
> src/primary/Makefile | 3 +
> src/primary/main.c | 1 +
> src/shared/common.h | 2 +-
> src/shared/secondary/add_port.c | 8 +
> src/shared/secondary/add_port.h | 2 +-
> .../secondary/spp_worker_th/cmd_utils.c | 8 +-
> src/vf/Makefile | 3 +
> 16 files changed, 672 insertions(+), 5 deletions(-)
> create mode 100644 src/drivers/Makefile
> create mode 100644 src/drivers/vhost/Makefile
> create mode 100644 src/drivers/vhost/rte_pmd_spp_vhost_version.map
> create mode 100644 src/drivers/vhost/rte_spp_vhost.c
>
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [spp] [PATCH 0/3] revive vhost
2019-12-24 5:57 ` [spp] [PATCH 0/3] revive vhost Yasufumi Ogawa
@ 2019-12-24 6:09 ` Itsuro ODA
2019-12-24 6:30 ` Yasufumi Ogawa
0 siblings, 1 reply; 37+ messages in thread
From: Itsuro ODA @ 2019-12-24 6:09 UTC (permalink / raw)
To: Yasufumi Ogawa; +Cc: spp, ferruh.yigit
Hi Yasyfumi,
Ok, I will divide patches and add tags.
By the way, what should the tag of the common header
(ex. src/shared/common.h) be ?
Thanks.
On Tue, 24 Dec 2019 14:57:08 +0900
Yasufumi Ogawa <yasufum.o@gmail.com> wrote:
> On 2019/12/23 14:00, Itsuro Oda wrote:
> > vhost PMD can not be used by secondary processes since DPDK 18.11.
> > SPP project decided to have own vhost PMD which can be used by
> > secondary processes at the moment. This vhost PMD is based on the
> > original vhost PMD but is simplified very much only to support
> > functions used by SPP. Thereby it becomes easy to fix the probrem.
> Hi,
>
> Could you add tags, such as "docs", "spp_primary" or so, at the head of titles? I'd like to ask another thing is to divide your patches into more tiny ones to be easy to add the tags. Although some patches are very tiny, but it is helpful for finding the changes in future.
>
> Thanks,
> Yasufumi
> >
> > Itsuro Oda (3):
> > multi process supported vhost PMD for SPP
> > make use of vhost PMD for SPP
> > make robust against process start and termination
> >
> > src/Makefile | 1 +
> > src/drivers/Makefile | 15 +
> > src/drivers/vhost/Makefile | 28 +
> > .../vhost/rte_pmd_spp_vhost_version.map | 4 +
> > src/drivers/vhost/rte_spp_vhost.c | 588 ++++++++++++++++++
> > src/mirror/Makefile | 3 +
> > src/nfv/Makefile | 3 +
> > src/nfv/commands.h | 1 +
> > src/nfv/main.c | 7 +-
> > src/primary/Makefile | 3 +
> > src/primary/main.c | 1 +
> > src/shared/common.h | 2 +-
> > src/shared/secondary/add_port.c | 8 +
> > src/shared/secondary/add_port.h | 2 +-
> > .../secondary/spp_worker_th/cmd_utils.c | 8 +-
> > src/vf/Makefile | 3 +
> > 16 files changed, 672 insertions(+), 5 deletions(-)
> > create mode 100644 src/drivers/Makefile
> > create mode 100644 src/drivers/vhost/Makefile
> > create mode 100644 src/drivers/vhost/rte_pmd_spp_vhost_version.map
> > create mode 100644 src/drivers/vhost/rte_spp_vhost.c
> >
--
Itsuro ODA <oda@valinux.co.jp>
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [spp] [PATCH 0/3] revive vhost
2019-12-24 6:09 ` Itsuro ODA
@ 2019-12-24 6:30 ` Yasufumi Ogawa
0 siblings, 0 replies; 37+ messages in thread
From: Yasufumi Ogawa @ 2019-12-24 6:30 UTC (permalink / raw)
To: Itsuro ODA; +Cc: spp, ferruh.yigit
> Hi Yasyfumi,
>
> Ok, I will divide patches and add tags.
> By the way, what should the tag of the common header
> (ex. src/shared/common.h) be ?
It should be 'shared'.
For a driver you introduce in this patchset, I think 'drivers/vhost' is
appropriate for the tag name simply, although I'm not sure any other
drivers is added later.
Thanks
>
> Thanks.
>
> On Tue, 24 Dec 2019 14:57:08 +0900
> Yasufumi Ogawa <yasufum.o@gmail.com> wrote:
>
>> On 2019/12/23 14:00, Itsuro Oda wrote:
>>> vhost PMD can not be used by secondary processes since DPDK 18.11.
>>> SPP project decided to have own vhost PMD which can be used by
>>> secondary processes at the moment. This vhost PMD is based on the
>>> original vhost PMD but is simplified very much only to support
>>> functions used by SPP. Thereby it becomes easy to fix the probrem.
>> Hi,
>>
>> Could you add tags, such as "docs", "spp_primary" or so, at the head of titles? I'd like to ask another thing is to divide your patches into more tiny ones to be easy to add the tags. Although some patches are very tiny, but it is helpful for finding the changes in future.
>>
>> Thanks,
>> Yasufumi
>>>
>>> Itsuro Oda (3):
>>> multi process supported vhost PMD for SPP
>>> make use of vhost PMD for SPP
>>> make robust against process start and termination
>>>
>>> src/Makefile | 1 +
>>> src/drivers/Makefile | 15 +
>>> src/drivers/vhost/Makefile | 28 +
>>> .../vhost/rte_pmd_spp_vhost_version.map | 4 +
>>> src/drivers/vhost/rte_spp_vhost.c | 588 ++++++++++++++++++
>>> src/mirror/Makefile | 3 +
>>> src/nfv/Makefile | 3 +
>>> src/nfv/commands.h | 1 +
>>> src/nfv/main.c | 7 +-
>>> src/primary/Makefile | 3 +
>>> src/primary/main.c | 1 +
>>> src/shared/common.h | 2 +-
>>> src/shared/secondary/add_port.c | 8 +
>>> src/shared/secondary/add_port.h | 2 +-
>>> .../secondary/spp_worker_th/cmd_utils.c | 8 +-
>>> src/vf/Makefile | 3 +
>>> 16 files changed, 672 insertions(+), 5 deletions(-)
>>> create mode 100644 src/drivers/Makefile
>>> create mode 100644 src/drivers/vhost/Makefile
>>> create mode 100644 src/drivers/vhost/rte_pmd_spp_vhost_version.map
>>> create mode 100644 src/drivers/vhost/rte_spp_vhost.c
>>>
>
^ permalink raw reply [flat|nested] 37+ messages in thread
* [spp] [PATCH v2 00/12] revive vhost
2019-12-23 5:00 [spp] [PATCH 0/3] revive vhost Itsuro Oda
` (3 preceding siblings ...)
2019-12-24 5:57 ` [spp] [PATCH 0/3] revive vhost Yasufumi Ogawa
@ 2019-12-25 4:49 ` Itsuro Oda
2019-12-25 4:49 ` [spp] [PATCH v2 01/12] drivers/vhost: add multi process supported vhost PMD for SPP Itsuro Oda
` (11 more replies)
2020-01-09 23:10 ` [spp] [PATCH v3 00/12] revive vhost Itsuro Oda
5 siblings, 12 replies; 37+ messages in thread
From: Itsuro Oda @ 2019-12-25 4:49 UTC (permalink / raw)
To: spp, ferruh.yigit, yasufum.o
vhost PMD can not be used by secondary processes since DPDK 18.11.
SPP project decided to have own vhost PMD which can be used by
secondary processes at the moment. This vhost PMD is based on the
original vhost PMD but is simplified very much only to support
functions used by SPP. Thereby it becomes easy to fix the probrem.
v2:
- devide patches and add tags of commit title.
Itsuro Oda (12):
drivers/vhost: add multi process supported vhost PMD for SPP
drivers: add to build vhost PMD for SPP
shared: switch to use vhost PMD for SPP
spp_primary: add link to vhost PMD for SPP
spp_nfv: add link to vhost PMD for SPP
spp_vf: add link to vhost PMD for SPP
spp_mirror: add link to vhost PMD for SPP
spp_primary: stop vhost before detach
spp_nfv: stop vhost before detach
shared: make sure vhost is stopped before (re)using the vhost
spp_nfv: exclude vhosts at process initialization
spp_vf,spp_mirror: exclude vhosts at process initialization
src/Makefile | 1 +
src/drivers/Makefile | 15 +
src/drivers/vhost/Makefile | 28 +
.../vhost/rte_pmd_spp_vhost_version.map | 4 +
src/drivers/vhost/rte_spp_vhost.c | 588 ++++++++++++++++++
src/mirror/Makefile | 3 +
src/nfv/Makefile | 3 +
src/nfv/commands.h | 1 +
src/nfv/main.c | 7 +-
src/primary/Makefile | 3 +
src/primary/main.c | 1 +
src/shared/common.h | 2 +-
src/shared/secondary/add_port.c | 8 +
src/shared/secondary/add_port.h | 2 +-
.../secondary/spp_worker_th/cmd_utils.c | 8 +-
src/vf/Makefile | 3 +
16 files changed, 672 insertions(+), 5 deletions(-)
create mode 100644 src/drivers/Makefile
create mode 100644 src/drivers/vhost/Makefile
create mode 100644 src/drivers/vhost/rte_pmd_spp_vhost_version.map
create mode 100644 src/drivers/vhost/rte_spp_vhost.c
--
2.17.1
^ permalink raw reply [flat|nested] 37+ messages in thread
* [spp] [PATCH v2 01/12] drivers/vhost: add multi process supported vhost PMD for SPP
2019-12-25 4:49 ` [spp] [PATCH v2 00/12] " Itsuro Oda
@ 2019-12-25 4:49 ` Itsuro Oda
2020-01-07 10:41 ` Yasufumi Ogawa
2019-12-25 4:49 ` [spp] [PATCH v2 02/12] drivers: add to build " Itsuro Oda
` (10 subsequent siblings)
11 siblings, 1 reply; 37+ messages in thread
From: Itsuro Oda @ 2019-12-25 4:49 UTC (permalink / raw)
To: spp, ferruh.yigit, yasufum.o
vhost PMD can not be used by secondary processes since DPDK 18.11.
SPP project decided to have own vhost PMD which can be used by
secondary processes at the moment. This vhost PMD is based on the
original vhost PMD but is simplified very much only to support
functions used by SPP. Thereby it becomes easy to fix the probrem.
The main idea of the fix is that execution of vhost start/stop
is moved to eth_dev_start/stop from probe/remove.
Note that only process which executes eth_dev_start can use the
vhost device although the vhost device is shared among the primary
process and secondary processes. Once eth_dev_stop is executed by
the process which used the vhost device, it is available to be
used by any process. It is user responsibility that multipul
processes don't use the vhost device at the same time.
Signed-off-by: Itsuro Oda <oda@valinux.co.jp>
---
src/drivers/vhost/Makefile | 28 +
.../vhost/rte_pmd_spp_vhost_version.map | 4 +
src/drivers/vhost/rte_spp_vhost.c | 588 ++++++++++++++++++
3 files changed, 620 insertions(+)
create mode 100644 src/drivers/vhost/Makefile
create mode 100644 src/drivers/vhost/rte_pmd_spp_vhost_version.map
create mode 100644 src/drivers/vhost/rte_spp_vhost.c
diff --git a/src/drivers/vhost/Makefile b/src/drivers/vhost/Makefile
new file mode 100644
index 0000000..e29c330
--- /dev/null
+++ b/src/drivers/vhost/Makefile
@@ -0,0 +1,28 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+#
+# library name
+#
+LIB = librte_pmd_spp_vhost.a
+
+LDLIBS += -lpthread
+LDLIBS += -lrte_eal -lrte_mbuf -lrte_mempool -lrte_ring
+LDLIBS += -lrte_ethdev -lrte_net -lrte_kvargs -lrte_vhost
+LDLIBS += -lrte_bus_vdev
+
+CFLAGS += -O3
+CFLAGS += $(WERROR_FLAGS)
+
+EXPORT_MAP := rte_pmd_spp_vhost_version.map
+
+LIBABIVER := 2
+
+#
+# all source are stored in SRCS-y
+#
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_VHOST) += rte_spp_vhost.c
+
+include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/src/drivers/vhost/rte_pmd_spp_vhost_version.map b/src/drivers/vhost/rte_pmd_spp_vhost_version.map
new file mode 100644
index 0000000..ef35398
--- /dev/null
+++ b/src/drivers/vhost/rte_pmd_spp_vhost_version.map
@@ -0,0 +1,4 @@
+DPDK_2.0 {
+
+ local: *;
+};
diff --git a/src/drivers/vhost/rte_spp_vhost.c b/src/drivers/vhost/rte_spp_vhost.c
new file mode 100644
index 0000000..8eb6503
--- /dev/null
+++ b/src/drivers/vhost/rte_spp_vhost.c
@@ -0,0 +1,588 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2016 IGEL Co., Ltd.
+ * Copyright(c) 2016-2018 Intel Corporation
+ * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+ */
+#include <unistd.h>
+#include <pthread.h>
+#include <stdbool.h>
+
+#include <rte_mbuf.h>
+#include <rte_ethdev_driver.h>
+#include <rte_ethdev_vdev.h>
+#include <rte_malloc.h>
+#include <rte_memcpy.h>
+#include <rte_bus_vdev.h>
+#include <rte_kvargs.h>
+#include <rte_vhost.h>
+#include <rte_spinlock.h>
+
+static int vhost_logtype;
+
+#define VHOST_LOG(level, ...) \
+ rte_log(RTE_LOG_ ## level, vhost_logtype, __VA_ARGS__)
+
+enum {VIRTIO_RXQ, VIRTIO_TXQ, VIRTIO_QNUM};
+
+#define ETH_VHOST_IFACE_ARG "iface"
+#define ETH_VHOST_QUEUES_ARG "queues"
+#define ETH_VHOST_CLIENT_ARG "client"
+
+static const char *valid_arguments[] = {
+ ETH_VHOST_IFACE_ARG,
+ ETH_VHOST_QUEUES_ARG,
+ ETH_VHOST_CLIENT_ARG,
+ NULL
+};
+
+struct vhost_queue {
+ struct pmd_internal *internal;
+ struct rte_mempool *mb_pool;
+ uint16_t virtqueue_id;
+ rte_spinlock_t queuing_lock;
+ uint64_t pkts;
+ uint64_t missed_pkts;
+};
+
+struct pmd_internal {
+ uint16_t max_queues;
+ uint64_t vhost_flags;
+ struct rte_eth_dev_data *eth_dev_data;
+ int vid;
+ char iface_name[PATH_MAX];
+};
+
+static struct rte_eth_link pmd_link = {
+ .link_speed = 10000,
+ .link_duplex = ETH_LINK_FULL_DUPLEX,
+ .link_status = ETH_LINK_DOWN
+};
+
+#define MZ_RTE_VHOST_PMD_INTERNAL "vhost_pmd_internal"
+static struct pmd_internal **pmd_internal_list;
+
+static uint16_t
+eth_vhost_rx(void *q, struct rte_mbuf **bufs, uint16_t nb_bufs)
+{
+ struct vhost_queue *r = q;
+ struct pmd_internal *internal;
+ uint16_t nb_rx = 0;
+
+ if (!q)
+ return 0;
+
+ internal = r->internal;
+ rte_spinlock_lock(&r->queuing_lock);
+ if (internal->vid == -1)
+ goto out;
+
+ /* Dequeue packets from guest TX queue */
+ nb_rx = rte_vhost_dequeue_burst(internal->vid, r->virtqueue_id,
+ r->mb_pool, bufs, nb_bufs);
+ r->pkts += nb_rx;
+
+out:
+ rte_spinlock_unlock(&r->queuing_lock);
+
+ return nb_rx;
+}
+
+static uint16_t
+eth_vhost_tx(void *q, struct rte_mbuf **bufs, uint16_t nb_bufs)
+{
+ struct vhost_queue *r = q;
+ struct pmd_internal *internal;
+ uint16_t i, nb_tx = 0;
+
+ if (!q)
+ return 0;
+
+ internal = r->internal;
+ rte_spinlock_lock(&r->queuing_lock);
+ if (internal->vid == -1)
+ goto out;
+
+ /* Enqueue packets to guest RX queue */
+ nb_tx = rte_vhost_enqueue_burst(internal->vid, r->virtqueue_id,
+ bufs, nb_bufs);
+ r->pkts += nb_tx;
+ r->missed_pkts += nb_bufs - nb_tx;
+
+ for (i = 0; i < nb_tx; i++)
+ rte_pktmbuf_free(bufs[i]);
+
+out:
+ rte_spinlock_unlock(&r->queuing_lock);
+
+ return nb_tx;
+}
+
+static inline struct pmd_internal *
+find_internal_resource(int vid)
+{
+ struct pmd_internal *internal;
+ int i;
+ char ifname[PATH_MAX];
+
+ if (rte_vhost_get_ifname(vid, ifname, sizeof(ifname)) == -1)
+ return NULL;
+
+ /* likely(found in a few loops) */
+ for (i = 0; i < RTE_MAX_ETHPORTS; i++) {
+ internal = pmd_internal_list[i];
+ if (internal != NULL && !strcmp(internal->iface_name, ifname)) {
+ return internal;
+ }
+ }
+
+ return NULL;
+}
+
+static int
+new_device(int vid)
+{
+ struct pmd_internal *internal;
+
+ internal = find_internal_resource(vid);
+ if (internal == NULL) {
+ VHOST_LOG(INFO, "Invalid device : %d\n", vid);
+ return -1;
+ }
+
+ internal->vid = vid;
+ internal->eth_dev_data->dev_link.link_status = ETH_LINK_UP;
+
+ VHOST_LOG(INFO, "Vhost device %d created\n", vid);
+
+ return 0;
+}
+
+static void
+destroy_device(int vid)
+{
+ struct pmd_internal *internal;
+ struct vhost_queue *vq;
+ struct rte_eth_dev_data *data;
+ int i;
+
+ internal = find_internal_resource(vid);
+ if (internal == NULL) {
+ VHOST_LOG(ERR, "Invalid device : %d\n", vid);
+ return;
+ }
+ data = internal->eth_dev_data;
+
+ /* wait inflight queuing done */
+ for (i = 0; i < data->nb_rx_queues; i++) {
+ if ((vq = data->rx_queues[i]) != NULL)
+ rte_spinlock_lock(&vq->queuing_lock);
+ }
+ for (i = 0; i < data->nb_tx_queues; i++) {
+ if ((vq = data->tx_queues[i]) != NULL)
+ rte_spinlock_lock(&vq->queuing_lock);
+ }
+
+ data->dev_link.link_status = ETH_LINK_DOWN;
+ internal->vid = -1;
+
+ for (i = 0; i < data->nb_rx_queues; i++) {
+ if ((vq = data->rx_queues[i]) != NULL)
+ rte_spinlock_unlock(&vq->queuing_lock);
+ }
+ for (i = 0; i < data->nb_tx_queues; i++) {
+ if ((vq = data->tx_queues[i]) != NULL)
+ rte_spinlock_unlock(&vq->queuing_lock);
+ }
+
+ VHOST_LOG(INFO, "Vhost device %d destroyed\n", vid);
+}
+
+static struct vhost_device_ops vhost_ops = {
+ .new_device = new_device,
+ .destroy_device = destroy_device,
+};
+
+static int
+eth_dev_start(struct rte_eth_dev *eth_dev)
+{
+ struct pmd_internal *internal = eth_dev->data->dev_private;
+
+ if (rte_vhost_driver_register(internal->iface_name, internal->vhost_flags))
+ return -1;
+
+ if (rte_vhost_driver_callback_register(internal->iface_name, &vhost_ops) < 0) {
+ VHOST_LOG(ERR, "Can't register callbacks\n");
+ return -1;
+ }
+
+ if (rte_vhost_driver_start(internal->iface_name) < 0) {
+ VHOST_LOG(ERR, "Failed to start driver for %s\n",
+ internal->iface_name);
+ return -1;
+ }
+
+ return 0;
+}
+
+static void
+eth_dev_stop(struct rte_eth_dev *dev)
+{
+ struct pmd_internal *internal = dev->data->dev_private;
+
+ rte_vhost_driver_unregister(internal->iface_name);
+}
+
+static int
+eth_dev_configure(struct rte_eth_dev *dev __rte_unused)
+{
+ return 0;
+}
+
+static void
+eth_dev_info(struct rte_eth_dev *dev,
+ struct rte_eth_dev_info *dev_info)
+{
+ struct pmd_internal *internal = dev->data->dev_private;
+
+ dev_info->max_mac_addrs = 1;
+ dev_info->max_rx_pktlen = (uint32_t)-1;
+ dev_info->max_rx_queues = internal->max_queues;
+ dev_info->max_tx_queues = internal->max_queues;
+ dev_info->min_rx_bufsize = 0;
+}
+
+static int
+eth_rx_queue_setup(struct rte_eth_dev *dev, uint16_t rx_queue_id,
+ uint16_t nb_rx_desc __rte_unused,
+ unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf __rte_unused,
+ struct rte_mempool *mb_pool)
+{
+ struct pmd_internal *internal = dev->data->dev_private;
+ struct vhost_queue *vq;
+
+ vq = rte_zmalloc_socket(NULL, sizeof(struct vhost_queue),
+ RTE_CACHE_LINE_SIZE, socket_id);
+ if (vq == NULL) {
+ VHOST_LOG(ERR, "Failed to allocate memory for rx queue\n");
+ return -ENOMEM;
+ }
+
+ vq->internal = internal;
+ vq->mb_pool = mb_pool;
+ vq->virtqueue_id = rx_queue_id * VIRTIO_QNUM + VIRTIO_TXQ;
+ rte_spinlock_init(&vq->queuing_lock);
+ dev->data->rx_queues[rx_queue_id] = vq;
+
+ return 0;
+}
+
+static int
+eth_tx_queue_setup(struct rte_eth_dev *dev, uint16_t tx_queue_id,
+ uint16_t nb_tx_desc __rte_unused,
+ unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf __rte_unused)
+{
+ struct pmd_internal *internal = dev->data->dev_private;
+ struct vhost_queue *vq;
+
+ vq = rte_zmalloc_socket(NULL, sizeof(struct vhost_queue),
+ RTE_CACHE_LINE_SIZE, socket_id);
+ if (vq == NULL) {
+ VHOST_LOG(ERR, "Failed to allocate memory for tx queue\n");
+ return -ENOMEM;
+ }
+
+ vq->internal = internal;
+ vq->virtqueue_id = tx_queue_id * VIRTIO_QNUM + VIRTIO_RXQ;
+ rte_spinlock_init(&vq->queuing_lock);
+ dev->data->tx_queues[tx_queue_id] = vq;
+
+ return 0;
+}
+
+static void
+eth_queue_release(void *q)
+{
+ rte_free(q);
+}
+
+static int
+eth_link_update(struct rte_eth_dev *dev __rte_unused,
+ int wait_to_complete __rte_unused)
+{
+ return 0;
+}
+
+static int
+eth_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats)
+{
+ unsigned i;
+ unsigned long rx_total = 0, tx_total = 0, tx_err_total = 0;
+ struct vhost_queue *vq;
+
+ for (i = 0; i < RTE_ETHDEV_QUEUE_STAT_CNTRS &&
+ i < dev->data->nb_rx_queues; i++) {
+ if ((vq = dev->data->rx_queues[i]) != NULL) {
+ stats->q_ipackets[i] = vq->pkts;
+ rx_total += vq->pkts;
+ }
+ }
+
+ for (i = 0; i < RTE_ETHDEV_QUEUE_STAT_CNTRS &&
+ i < dev->data->nb_tx_queues; i++) {
+ if ((vq = dev->data->rx_queues[i]) != NULL) {
+ stats->q_opackets[i] = vq->pkts;
+ tx_total += vq->pkts;
+ stats->q_errors[i] = vq->missed_pkts;
+ tx_err_total += vq->missed_pkts;
+ }
+ }
+
+ stats->ipackets = rx_total;
+ stats->opackets = tx_total;
+ stats->oerrors = tx_err_total;
+
+ return 0;
+}
+
+static void
+eth_stats_reset(struct rte_eth_dev *dev)
+{
+ struct vhost_queue *vq;
+ unsigned i;
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ if ((vq = dev->data->rx_queues[i]) != NULL)
+ vq->pkts = 0;
+ }
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ if ((vq = dev->data->rx_queues[i]) != NULL) {
+ vq->pkts = 0;
+ vq->missed_pkts = 0;
+ }
+ }
+}
+
+static const struct eth_dev_ops ops = {
+ .dev_start = eth_dev_start,
+ .dev_stop = eth_dev_stop,
+ .dev_configure = eth_dev_configure,
+ .dev_infos_get = eth_dev_info,
+ .rx_queue_setup = eth_rx_queue_setup,
+ .tx_queue_setup = eth_tx_queue_setup,
+ .rx_queue_release = eth_queue_release,
+ .tx_queue_release = eth_queue_release,
+ .link_update = eth_link_update,
+ .stats_get = eth_stats_get,
+ .stats_reset = eth_stats_reset,
+};
+
+static int
+init_shared_data(void)
+{
+ const struct rte_memzone *mz;
+
+ if (pmd_internal_list == NULL) {
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+ size_t len = sizeof(*pmd_internal_list) * RTE_MAX_ETHPORTS;
+ mz = rte_memzone_reserve(MZ_RTE_VHOST_PMD_INTERNAL,
+ len, rte_socket_id(), 0);
+ if (mz)
+ memset(mz->addr, 0, len);
+ } else
+ mz = rte_memzone_lookup(MZ_RTE_VHOST_PMD_INTERNAL);
+ if (mz == NULL) {
+ VHOST_LOG(ERR, "Cannot allocate vhost shared data\n");
+ return -1;
+ }
+ pmd_internal_list = mz->addr;
+ }
+
+ return 0;
+}
+
+static int
+eth_dev_vhost_create(struct rte_vdev_device *dev, char *iface_name,
+ int16_t queues, const unsigned int numa_node, uint64_t flags)
+{
+ struct rte_eth_dev_data *data;
+ struct pmd_internal *internal;
+ struct rte_eth_dev *eth_dev;
+ struct rte_ether_addr *eth_addr;
+
+ VHOST_LOG(INFO, "Creating VHOST-USER backend on numa socket %u\n",
+ numa_node);
+
+ eth_addr = rte_zmalloc_socket(NULL, sizeof(*eth_addr), 0, numa_node);
+ if (eth_addr == NULL)
+ return -ENOMEM;
+
+ eth_dev = rte_eth_vdev_allocate(dev, sizeof(*internal));
+ if (eth_dev == NULL)
+ return -ENOMEM;
+
+ data = eth_dev->data;
+
+ data->dev_link = pmd_link;
+ data->dev_flags = RTE_ETH_DEV_INTR_LSC;
+ data->mac_addrs = eth_addr;
+
+ eth_dev->dev_ops = &ops;
+ eth_dev->rx_pkt_burst = eth_vhost_rx;
+ eth_dev->tx_pkt_burst = eth_vhost_tx;
+
+ internal = data->dev_private;
+
+ internal->max_queues = queues;
+ internal->vid = -1;
+ internal->vhost_flags = flags;
+ internal->eth_dev_data = data;
+ strncpy(internal->iface_name, iface_name, sizeof(internal->iface_name));
+
+ pmd_internal_list[data->port_id] = internal;
+
+ rte_eth_dev_probing_finish(eth_dev);
+
+ return 0;
+}
+
+static inline int
+open_iface(const char *key __rte_unused, const char *value, void *extra_args)
+{
+ const char **iface_name = extra_args;
+
+ if (value == NULL)
+ return -1;
+
+ *iface_name = value;
+
+ return 0;
+}
+
+static inline int
+open_int(const char *key __rte_unused, const char *value, void *extra_args)
+{
+ uint16_t *n = extra_args;
+ char *endptr;
+
+ if (value == NULL)
+ return -1;
+
+ *n = (uint16_t)strtoul(value, &endptr, 0);
+ if (*endptr != '\0' || errno == ERANGE)
+ return -1;
+
+ return 0;
+}
+
+static int
+rte_pmd_vhost_probe(struct rte_vdev_device *dev)
+{
+ struct rte_kvargs *kvlist = NULL;
+ int ret = 0;
+ char *iface_name = NULL;
+ uint16_t queues = 1;
+ uint64_t flags = 0;
+ uint16_t client_mode = 0;
+ struct rte_eth_dev *eth_dev;
+ const char *name = rte_vdev_device_name(dev);
+
+ VHOST_LOG(INFO, "Initializing pmd_vhost for %s\n", name);
+
+ if (init_shared_data() == -1)
+ return -ENOMEM;
+
+ if (rte_eal_process_type() == RTE_PROC_SECONDARY) {
+ eth_dev = rte_eth_dev_attach_secondary(name);
+ if (!eth_dev) {
+ VHOST_LOG(ERR, "Failed to probe %s\n", name);
+ return -ENOENT;
+ }
+ eth_dev->rx_pkt_burst = eth_vhost_rx;
+ eth_dev->tx_pkt_burst = eth_vhost_tx;
+
+ eth_dev->dev_ops = &ops;
+ eth_dev->device = &dev->device;
+ rte_eth_dev_probing_finish(eth_dev);
+ return 0;
+ }
+
+ kvlist = rte_kvargs_parse(rte_vdev_device_args(dev), valid_arguments);
+ if (kvlist == NULL)
+ return -EINVAL;
+
+ if (rte_kvargs_process(kvlist, ETH_VHOST_IFACE_ARG,
+ &open_iface, &iface_name) == -1 ||
+ rte_kvargs_process(kvlist, ETH_VHOST_QUEUES_ARG,
+ &open_int, &queues) == -1 ||
+ rte_kvargs_process(kvlist, ETH_VHOST_CLIENT_ARG,
+ &open_int, &client_mode) == -1) {
+ rte_kvargs_free(kvlist);
+ return -EINVAL;
+ }
+
+ if (iface_name == NULL ||
+ queues == 0 || queues > RTE_MAX_QUEUES_PER_PORT ||
+ (client_mode != 0 && client_mode != 1)) {
+ rte_kvargs_free(kvlist);
+ return -EINVAL;
+ }
+
+ if (client_mode)
+ flags |= RTE_VHOST_USER_CLIENT;
+
+ ret = eth_dev_vhost_create(dev, iface_name, queues, rte_socket_id(), flags);
+
+ rte_kvargs_free(kvlist);
+ return ret;
+}
+
+static int
+rte_pmd_vhost_remove(struct rte_vdev_device *dev)
+{
+ const char *name = rte_vdev_device_name(dev);
+ struct rte_eth_dev *eth_dev;
+ int i;
+
+ VHOST_LOG(INFO, "Un-Initializing pmd_vhost for %s\n", name);
+
+ eth_dev = rte_eth_dev_allocated(name);
+ if (eth_dev == NULL)
+ return 0;
+
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+ return rte_eth_dev_release_port(eth_dev);
+
+ if (eth_dev->data->dev_started) {
+ VHOST_LOG(WARNING, "device must be stoped.\n");
+ }
+
+ for (i = 0; i < eth_dev->data->nb_rx_queues; i++)
+ rte_free(eth_dev->data->rx_queues[i]);
+
+ for (i = 0; i < eth_dev->data->nb_tx_queues; i++)
+ rte_free(eth_dev->data->tx_queues[i]);
+
+ pmd_internal_list[eth_dev->data->port_id] = NULL;
+
+ return rte_eth_dev_release_port(eth_dev);
+}
+
+static struct rte_vdev_driver pmd_vhost_drv = {
+ .probe = rte_pmd_vhost_probe,
+ .remove = rte_pmd_vhost_remove,
+};
+
+RTE_PMD_REGISTER_VDEV(spp_vhost, pmd_vhost_drv);
+RTE_PMD_REGISTER_PARAM_STRING(spp_vhost,
+ "iface=<ifc> "
+ "queues=<int> "
+ "client=<0|1> ");
+
+RTE_INIT(vhost_init_log)
+{
+ vhost_logtype = rte_log_register("pmd.spp.vhost");
+ if (vhost_logtype >= 0)
+ rte_log_set_level(vhost_logtype, RTE_LOG_NOTICE);
+}
--
2.17.1
^ permalink raw reply [flat|nested] 37+ messages in thread
* [spp] [PATCH v2 02/12] drivers: add to build vhost PMD for SPP
2019-12-25 4:49 ` [spp] [PATCH v2 00/12] " Itsuro Oda
2019-12-25 4:49 ` [spp] [PATCH v2 01/12] drivers/vhost: add multi process supported vhost PMD for SPP Itsuro Oda
@ 2019-12-25 4:49 ` Itsuro Oda
2019-12-25 4:49 ` [spp] [PATCH v2 03/12] shared: switch to use " Itsuro Oda
` (9 subsequent siblings)
11 siblings, 0 replies; 37+ messages in thread
From: Itsuro Oda @ 2019-12-25 4:49 UTC (permalink / raw)
To: spp, ferruh.yigit, yasufum.o
This patch adds to build vhost PMD for SPP.
Signed-off-by: Itsuro Oda <oda@valinux.co.jp>
---
src/Makefile | 1 +
src/drivers/Makefile | 15 +++++++++++++++
2 files changed, 16 insertions(+)
create mode 100644 src/drivers/Makefile
diff --git a/src/Makefile b/src/Makefile
index cab80db..2f46606 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -38,6 +38,7 @@ RTE_TARGET ?= x86_64-native-linuxapp-gcc
include $(RTE_SDK)/mk/rte.vars.mk
+DIRS-y += drivers
DIRS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += primary
DIRS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += nfv
DIRS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += vf
diff --git a/src/drivers/Makefile b/src/drivers/Makefile
new file mode 100644
index 0000000..f2deba7
--- /dev/null
+++ b/src/drivers/Makefile
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+
+ifeq ($(RTE_SDK),)
+$(error "Please define RTE_SDK environment variable")
+endif
+
+# Default target, can be overriden by command line or environment
+RTE_TARGET ?= x86_64-native-linuxapp-gcc
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+DIRS-y += vhost
+
+include $(RTE_SDK)/mk/rte.extsubdir.mk
--
2.17.1
^ permalink raw reply [flat|nested] 37+ messages in thread
* [spp] [PATCH v2 03/12] shared: switch to use vhost PMD for SPP
2019-12-25 4:49 ` [spp] [PATCH v2 00/12] " Itsuro Oda
2019-12-25 4:49 ` [spp] [PATCH v2 01/12] drivers/vhost: add multi process supported vhost PMD for SPP Itsuro Oda
2019-12-25 4:49 ` [spp] [PATCH v2 02/12] drivers: add to build " Itsuro Oda
@ 2019-12-25 4:49 ` Itsuro Oda
2019-12-25 4:49 ` [spp] [PATCH v2 04/12] spp_primary: add link to " Itsuro Oda
` (8 subsequent siblings)
11 siblings, 0 replies; 37+ messages in thread
From: Itsuro Oda @ 2019-12-25 4:49 UTC (permalink / raw)
To: spp, ferruh.yigit, yasufum.o
This patch switches to use vhost PMD for SPP instead of original
vhost PMD.
Signed-off-by: Itsuro Oda <oda@valinux.co.jp>
---
src/shared/common.h | 2 +-
src/shared/secondary/add_port.h | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/src/shared/common.h b/src/shared/common.h
index 9c46a64..431ad3e 100644
--- a/src/shared/common.h
+++ b/src/shared/common.h
@@ -34,7 +34,7 @@
#define VDEV_ETH_RING "eth_ring"
#define VDEV_NET_RING "net_ring"
-#define VDEV_ETH_VHOST "eth_vhost"
+#define VDEV_ETH_VHOST "spp_vhost"
#define VDEV_NET_VHOST "net_vhost"
#define VDEV_PCAP "net_pcap"
#define VDEV_ETH_TAP "eth_tap"
diff --git a/src/shared/secondary/add_port.h b/src/shared/secondary/add_port.h
index cfae1af..a75b28b 100644
--- a/src/shared/secondary/add_port.h
+++ b/src/shared/secondary/add_port.h
@@ -10,7 +10,7 @@
#define NR_DESCS 128
#define VHOST_IFACE_NAME "/tmp/sock%u"
-#define VHOST_BACKEND_NAME "eth_vhost%u"
+#define VHOST_BACKEND_NAME "spp_vhost%u"
#define PCAP_PMD_DEV_NAME "eth_pcap%u"
#define NULL_PMD_DEV_NAME "eth_null%u"
--
2.17.1
^ permalink raw reply [flat|nested] 37+ messages in thread
* [spp] [PATCH v2 04/12] spp_primary: add link to vhost PMD for SPP
2019-12-25 4:49 ` [spp] [PATCH v2 00/12] " Itsuro Oda
` (2 preceding siblings ...)
2019-12-25 4:49 ` [spp] [PATCH v2 03/12] shared: switch to use " Itsuro Oda
@ 2019-12-25 4:49 ` Itsuro Oda
2019-12-25 4:49 ` [spp] [PATCH v2 05/12] spp_nfv: " Itsuro Oda
` (7 subsequent siblings)
11 siblings, 0 replies; 37+ messages in thread
From: Itsuro Oda @ 2019-12-25 4:49 UTC (permalink / raw)
To: spp, ferruh.yigit, yasufum.o
This patch adds link to vhost PMD for SPP.
Signed-off-by: Itsuro Oda <oda@valinux.co.jp>
---
src/primary/Makefile | 3 +++
1 file changed, 3 insertions(+)
diff --git a/src/primary/Makefile b/src/primary/Makefile
index 14d8f30..ace6228 100644
--- a/src/primary/Makefile
+++ b/src/primary/Makefile
@@ -41,4 +41,7 @@ endif
# and so the next line can be removed in those cases.
EXTRA_CFLAGS += -fno-strict-aliasing
+SPP_DRIVERS_DIR = $(BASE_OUTPUT)/src/drivers
+EXTRA_LDLIBS = -L$(SPP_DRIVERS_DIR)/vhost --whole-archive -lrte_pmd_spp_vhost --no-whole-archive
+
include $(RTE_SDK)/mk/rte.extapp.mk
--
2.17.1
^ permalink raw reply [flat|nested] 37+ messages in thread
* [spp] [PATCH v2 05/12] spp_nfv: add link to vhost PMD for SPP
2019-12-25 4:49 ` [spp] [PATCH v2 00/12] " Itsuro Oda
` (3 preceding siblings ...)
2019-12-25 4:49 ` [spp] [PATCH v2 04/12] spp_primary: add link to " Itsuro Oda
@ 2019-12-25 4:49 ` Itsuro Oda
2019-12-25 4:49 ` [spp] [PATCH v2 06/12] spp_vf: " Itsuro Oda
` (6 subsequent siblings)
11 siblings, 0 replies; 37+ messages in thread
From: Itsuro Oda @ 2019-12-25 4:49 UTC (permalink / raw)
To: spp, ferruh.yigit, yasufum.o
This patch adds link to vhost PMD for SPP.
Signed-off-by: Itsuro Oda <oda@valinux.co.jp>
---
src/nfv/Makefile | 3 +++
1 file changed, 3 insertions(+)
diff --git a/src/nfv/Makefile b/src/nfv/Makefile
index c3d2806..a717616 100644
--- a/src/nfv/Makefile
+++ b/src/nfv/Makefile
@@ -27,4 +27,7 @@ LDLIBS += -lrte_pmd_ring
LDLIBS += -lrte_pmd_vhost
endif
+SPP_DRIVERS_DIR = $(BASE_OUTPUT)/src/drivers
+EXTRA_LDLIBS = -L$(SPP_DRIVERS_DIR)/vhost --whole-archive -lrte_pmd_spp_vhost --no-whole-archive
+
include $(RTE_SDK)/mk/rte.extapp.mk
--
2.17.1
^ permalink raw reply [flat|nested] 37+ messages in thread
* [spp] [PATCH v2 06/12] spp_vf: add link to vhost PMD for SPP
2019-12-25 4:49 ` [spp] [PATCH v2 00/12] " Itsuro Oda
` (4 preceding siblings ...)
2019-12-25 4:49 ` [spp] [PATCH v2 05/12] spp_nfv: " Itsuro Oda
@ 2019-12-25 4:49 ` Itsuro Oda
2019-12-25 4:49 ` [spp] [PATCH v2 07/12] spp_mirror: " Itsuro Oda
` (5 subsequent siblings)
11 siblings, 0 replies; 37+ messages in thread
From: Itsuro Oda @ 2019-12-25 4:49 UTC (permalink / raw)
To: spp, ferruh.yigit, yasufum.o
This patch adds link to vhost PMD for SPP.
Signed-off-by: Itsuro Oda <oda@valinux.co.jp>
---
src/vf/Makefile | 3 +++
1 file changed, 3 insertions(+)
diff --git a/src/vf/Makefile b/src/vf/Makefile
index 43e5e4d..e42f8b7 100644
--- a/src/vf/Makefile
+++ b/src/vf/Makefile
@@ -48,4 +48,7 @@ LDLIBS += -lrte_pmd_ring
LDLIBS += -lrte_pmd_vhost
endif
+SPP_DRIVERS_DIR = $(BASE_OUTPUT)/src/drivers
+EXTRA_LDLIBS = -L$(SPP_DRIVERS_DIR)/vhost --whole-archive -lrte_pmd_spp_vhost --no-whole-archive
+
include $(RTE_SDK)/mk/rte.extapp.mk
--
2.17.1
^ permalink raw reply [flat|nested] 37+ messages in thread
* [spp] [PATCH v2 07/12] spp_mirror: add link to vhost PMD for SPP
2019-12-25 4:49 ` [spp] [PATCH v2 00/12] " Itsuro Oda
` (5 preceding siblings ...)
2019-12-25 4:49 ` [spp] [PATCH v2 06/12] spp_vf: " Itsuro Oda
@ 2019-12-25 4:49 ` Itsuro Oda
2019-12-25 4:49 ` [spp] [PATCH v2 08/12] spp_primary: stop vhost before detach Itsuro Oda
` (4 subsequent siblings)
11 siblings, 0 replies; 37+ messages in thread
From: Itsuro Oda @ 2019-12-25 4:49 UTC (permalink / raw)
To: spp, ferruh.yigit, yasufum.o
This patch adds link to vhost PMD for SPP.
Signed-off-by: Itsuro Oda <oda@valinux.co.jp>
---
src/mirror/Makefile | 3 +++
1 file changed, 3 insertions(+)
diff --git a/src/mirror/Makefile b/src/mirror/Makefile
index 0bd079a..f82b08f 100644
--- a/src/mirror/Makefile
+++ b/src/mirror/Makefile
@@ -53,4 +53,7 @@ LDLIBS += -lrte_pmd_ring
LDLIBS += -lrte_pmd_vhost
endif
+SPP_DRIVERS_DIR = $(BASE_OUTPUT)/src/drivers
+EXTRA_LDLIBS = -L$(SPP_DRIVERS_DIR)/vhost --whole-archive -lrte_pmd_spp_vhost --no-whole-archive
+
include $(RTE_SDK)/mk/rte.extapp.mk
--
2.17.1
^ permalink raw reply [flat|nested] 37+ messages in thread
* [spp] [PATCH v2 08/12] spp_primary: stop vhost before detach
2019-12-25 4:49 ` [spp] [PATCH v2 00/12] " Itsuro Oda
` (6 preceding siblings ...)
2019-12-25 4:49 ` [spp] [PATCH v2 07/12] spp_mirror: " Itsuro Oda
@ 2019-12-25 4:49 ` Itsuro Oda
2019-12-25 4:49 ` [spp] [PATCH v2 09/12] spp_nfv: " Itsuro Oda
` (3 subsequent siblings)
11 siblings, 0 replies; 37+ messages in thread
From: Itsuro Oda @ 2019-12-25 4:49 UTC (permalink / raw)
To: spp, ferruh.yigit, yasufum.o
Make sure to free resources of vhost before detach.
Signed-off-by: Itsuro Oda <oda@valinux.co.jp>
---
src/primary/main.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/src/primary/main.c b/src/primary/main.c
index 26e9c42..a1ca791 100644
--- a/src/primary/main.c
+++ b/src/primary/main.c
@@ -874,6 +874,7 @@ del_port(char *p_type, int p_id)
dev_id = find_ethdev_id(p_id, VHOST);
if (dev_id == PORT_RESET)
return -1;
+ rte_eth_dev_stop(dev_id);
dev_detach_by_port_id(dev_id);
} else if (!strcmp(p_type, "ring")) {
--
2.17.1
^ permalink raw reply [flat|nested] 37+ messages in thread
* [spp] [PATCH v2 09/12] spp_nfv: stop vhost before detach
2019-12-25 4:49 ` [spp] [PATCH v2 00/12] " Itsuro Oda
` (7 preceding siblings ...)
2019-12-25 4:49 ` [spp] [PATCH v2 08/12] spp_primary: stop vhost before detach Itsuro Oda
@ 2019-12-25 4:49 ` Itsuro Oda
2019-12-25 4:49 ` [spp] [PATCH v2 10/12] shared: make sure vhost is stopped before (re)using the vhost Itsuro Oda
` (2 subsequent siblings)
11 siblings, 0 replies; 37+ messages in thread
From: Itsuro Oda @ 2019-12-25 4:49 UTC (permalink / raw)
To: spp, ferruh.yigit, yasufum.o
Make sure to free resources of vhost before detach.
Signed-off-by: Itsuro Oda <oda@valinux.co.jp>
---
src/nfv/commands.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/src/nfv/commands.h b/src/nfv/commands.h
index f6c2305..7e50c8c 100644
--- a/src/nfv/commands.h
+++ b/src/nfv/commands.h
@@ -25,6 +25,7 @@ do_del(char *p_type, int p_id)
port_id = find_port_id(p_id, VHOST);
if (port_id == PORT_RESET)
return -1;
+ rte_eth_dev_stop(port_id);
dev_detach_by_port_id(port_id);
} else if (!strcmp(p_type, "ring")) {
--
2.17.1
^ permalink raw reply [flat|nested] 37+ messages in thread
* [spp] [PATCH v2 10/12] shared: make sure vhost is stopped before (re)using the vhost
2019-12-25 4:49 ` [spp] [PATCH v2 00/12] " Itsuro Oda
` (8 preceding siblings ...)
2019-12-25 4:49 ` [spp] [PATCH v2 09/12] spp_nfv: " Itsuro Oda
@ 2019-12-25 4:49 ` Itsuro Oda
2019-12-25 4:49 ` [spp] [PATCH v2 11/12] spp_nfv: exclude vhosts at process initialization Itsuro Oda
2019-12-25 4:49 ` [spp] [PATCH v2 12/12] spp_vf, spp_mirror: " Itsuro Oda
11 siblings, 0 replies; 37+ messages in thread
From: Itsuro Oda @ 2019-12-25 4:49 UTC (permalink / raw)
To: spp, ferruh.yigit, yasufum.o
It is for the case a secondary process which used the vhost
was down without stopping the eth_dev.
It makes the eth_dev stopped and available to do following
operations(ex. rte_eth_dev_configure). It is no-op if the
eth_dev is stopped.
Signed-off-by: Itsuro Oda <oda@valinux.co.jp>
---
src/shared/secondary/add_port.c | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/src/shared/secondary/add_port.c b/src/shared/secondary/add_port.c
index b072140..d845250 100644
--- a/src/shared/secondary/add_port.c
+++ b/src/shared/secondary/add_port.c
@@ -170,6 +170,14 @@ add_vhost_pmd(int index)
return ret;
}
+ /* NOTE: make sure the eth_dev is stopped.
+ * it is for the case a secondary process which used the vhost
+ * was down without stopping the device.
+ * note that it is still user responsibility to prevent multipul
+ * processes use a vhost at the same time.
+ */
+ rte_eth_dev_stop(vhost_port_id);
+
ret = rte_eth_dev_configure(vhost_port_id, nr_queues, nr_queues,
&port_conf);
if (ret < 0) {
--
2.17.1
^ permalink raw reply [flat|nested] 37+ messages in thread
* [spp] [PATCH v2 11/12] spp_nfv: exclude vhosts at process initialization
2019-12-25 4:49 ` [spp] [PATCH v2 00/12] " Itsuro Oda
` (9 preceding siblings ...)
2019-12-25 4:49 ` [spp] [PATCH v2 10/12] shared: make sure vhost is stopped before (re)using the vhost Itsuro Oda
@ 2019-12-25 4:49 ` Itsuro Oda
2019-12-25 4:49 ` [spp] [PATCH v2 12/12] spp_vf, spp_mirror: " Itsuro Oda
11 siblings, 0 replies; 37+ messages in thread
From: Itsuro Oda @ 2019-12-25 4:49 UTC (permalink / raw)
To: spp, ferruh.yigit, yasufum.o
A vhost device is shared among the primary process and secondary
processes. When a secodary process starts it recognizes vhost
devices if they are used by processes already started. It is not
appropriate to include to port information as PHY devices.
Signed-off-by: Itsuro Oda <oda@valinux.co.jp>
---
src/nfv/main.c | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/src/nfv/main.c b/src/nfv/main.c
index 513a98d..f2c6bfc 100644
--- a/src/nfv/main.c
+++ b/src/nfv/main.c
@@ -211,7 +211,12 @@ main(int argc, char *argv[])
if (port_type == PHY) {
port_id = nof_phy_port;
nof_phy_port++;
- }
+ } else if (port_type == VHOST)
+ continue;
+ /* NOTE: vhost may be used another process. even if no
+ * process uses, it is necessary to "add vhost" explicitly.
+ * not display to avoid confusion.
+ */
/* Update ports_fwd_array with phy port. */
ports_fwd_array[i].in_port_id = i;
--
2.17.1
^ permalink raw reply [flat|nested] 37+ messages in thread
* [spp] [PATCH v2 12/12] spp_vf, spp_mirror: exclude vhosts at process initialization
2019-12-25 4:49 ` [spp] [PATCH v2 00/12] " Itsuro Oda
` (10 preceding siblings ...)
2019-12-25 4:49 ` [spp] [PATCH v2 11/12] spp_nfv: exclude vhosts at process initialization Itsuro Oda
@ 2019-12-25 4:49 ` Itsuro Oda
11 siblings, 0 replies; 37+ messages in thread
From: Itsuro Oda @ 2019-12-25 4:49 UTC (permalink / raw)
To: spp, ferruh.yigit, yasufum.o
A vhost device is shared among the primary process and secondary
processes. When a secodary process starts it recognizes vhost
devices if they are used by processes already started. It is not
appropriate to include to port information as PHY devices.
Signed-off-by: Itsuro Oda <oda@valinux.co.jp>
---
src/shared/secondary/spp_worker_th/cmd_utils.c | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/src/shared/secondary/spp_worker_th/cmd_utils.c b/src/shared/secondary/spp_worker_th/cmd_utils.c
index 010a4b6..69d7222 100644
--- a/src/shared/secondary/spp_worker_th/cmd_utils.c
+++ b/src/shared/secondary/spp_worker_th/cmd_utils.c
@@ -437,8 +437,12 @@ init_host_port_info(void)
p_iface_info->phy[port_id].ethdev_port_id = port_id;
break;
case VHOST:
- p_iface_info->vhost[port_id].iface_type = port_type;
- p_iface_info->vhost[port_id].ethdev_port_id = port_id;
+ /* NOTE: a vhost can be used by one process.
+ * even if it exists, it is necessary to do
+ * add_vhost_pmd to setup the device.
+ * note that it is user responsibility to prevent
+ * multipul processes use a vhost at the same time.
+ */
break;
case RING:
p_iface_info->ring[port_id].iface_type = port_type;
--
2.17.1
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [spp] [PATCH v2 01/12] drivers/vhost: add multi process supported vhost PMD for SPP
2019-12-25 4:49 ` [spp] [PATCH v2 01/12] drivers/vhost: add multi process supported vhost PMD for SPP Itsuro Oda
@ 2020-01-07 10:41 ` Yasufumi Ogawa
2020-01-08 1:17 ` Itsuro ODA
0 siblings, 1 reply; 37+ messages in thread
From: Yasufumi Ogawa @ 2020-01-07 10:41 UTC (permalink / raw)
To: Itsuro Oda; +Cc: spp, ferruh.yigit
Hi,
On 2019/12/25 13:49, Itsuro Oda wrote:
> vhost PMD can not be used by secondary processes since DPDK 18.11.
> SPP project decided to have own vhost PMD which can be used by
> secondary processes at the moment. This vhost PMD is based on the
> original vhost PMD but is simplified very much only to support
> functions used by SPP. Thereby it becomes easy to fix the probrem.
>
> The main idea of the fix is that execution of vhost start/stop
> is moved to eth_dev_start/stop from probe/remove.
>
> Note that only process which executes eth_dev_start can use the
> vhost device although the vhost device is shared among the primary
> process and secondary processes. Once eth_dev_stop is executed by
> the process which used the vhost device, it is available to be
> used by any process. It is user responsibility that multipul
> processes don't use the vhost device at the same time.
>
> Signed-off-by: Itsuro Oda <oda@valinux.co.jp>
> ---
> src/drivers/vhost/Makefile | 28 +
> .../vhost/rte_pmd_spp_vhost_version.map | 4 +
> src/drivers/vhost/rte_spp_vhost.c | 588 ++++++++++++++++++
> 3 files changed, 620 insertions(+)
> create mode 100644 src/drivers/vhost/Makefile
> create mode 100644 src/drivers/vhost/rte_pmd_spp_vhost_version.map
> create mode 100644 src/drivers/vhost/rte_spp_vhost.c
>
> diff --git a/src/drivers/vhost/Makefile b/src/drivers/vhost/Makefile
[...]
> +
> +static inline struct pmd_internal *
> +find_internal_resource(int vid)
> +{
> + struct pmd_internal *internal;
> + int i;
> + char ifname[PATH_MAX];
This variable is declared, but not used from anywhere.
> +
> + if (rte_vhost_get_ifname(vid, ifname, sizeof(ifname)) == -1)
> + return NULL;
[...]
> +}
> +
> +static const struct eth_dev_ops ops = {
> + .dev_start = eth_dev_start,
> + .dev_stop = eth_dev_stop,
> + .dev_configure = eth_dev_configure,
> + .dev_infos_get = eth_dev_info, > + .rx_queue_setup = eth_rx_queue_setup,
> + .tx_queue_setup = eth_tx_queue_setup,
> + .rx_queue_release = eth_queue_release,
> + .tx_queue_release = eth_queue_release,
> + .link_update = eth_link_update,
> + .stats_get = eth_stats_get,
> + .stats_reset = eth_stats_reset,
> +};
This struct causes compilation errors because of incompatible pointer types.
$ make
== src
== drivers
== vhost
CC rte_spp_vhost.o
rte_spp_vhost.c:371:19: error: initialization from incompatible pointer
type [-Werror=incompatible-pointer-types]
.dev_infos_get = eth_dev_info,
^~~~~~~~~~~~
rte_spp_vhost.c:371:19: note: (near initialization for ‘ops.dev_infos_get’)
rte_spp_vhost.c:378:17: error: initialization from incompatible pointer
type [-Werror=incompatible-pointer-types]
.stats_reset = eth_stats_reset,
^~~~~~~~~~~~~~~
rte_spp_vhost.c:378:17: note: (near initialization for ‘ops.stats_reset’)
cc1: error: unrecognized command line option
‘-Wno-address-of-packed-member’ [-Werror]
cc1: all warnings being treated as errors
From my understanding, eth_dev_info() and eth_stats_reset() are defined
as void in your patch, but DPDK expects int by referring the definition
of eth_dev_ops in lib/librte_ethdev/rte_ethdev_core.h:609.
Regards,
Yasufumi
> +
> +static int
[...]
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [spp] [PATCH v2 01/12] drivers/vhost: add multi process supported vhost PMD for SPP
2020-01-07 10:41 ` Yasufumi Ogawa
@ 2020-01-08 1:17 ` Itsuro ODA
2020-01-08 1:23 ` Itsuro ODA
0 siblings, 1 reply; 37+ messages in thread
From: Itsuro ODA @ 2020-01-08 1:17 UTC (permalink / raw)
To: Yasufumi Ogawa; +Cc: spp, ferruh.yigit
Hi Yasufumi,
On Tue, 7 Jan 2020 19:41:52 +0900
Yasufumi Ogawa <yasufum.o@gmail.com> wrote:
> Hi,
>
> On 2019/12/25 13:49, Itsuro Oda wrote:
> > vhost PMD can not be used by secondary processes since DPDK 18.11.
> > SPP project decided to have own vhost PMD which can be used by
> > secondary processes at the moment. This vhost PMD is based on the
> > original vhost PMD but is simplified very much only to support
> > functions used by SPP. Thereby it becomes easy to fix the probrem.
> >
> > The main idea of the fix is that execution of vhost start/stop
> > is moved to eth_dev_start/stop from probe/remove.
> >
> > Note that only process which executes eth_dev_start can use the
> > vhost device although the vhost device is shared among the primary
> > process and secondary processes. Once eth_dev_stop is executed by
> > the process which used the vhost device, it is available to be
> > used by any process. It is user responsibility that multipul
> > processes don't use the vhost device at the same time.
> >
> > Signed-off-by: Itsuro Oda <oda@valinux.co.jp>
> > ---
> > src/drivers/vhost/Makefile | 28 +
> > .../vhost/rte_pmd_spp_vhost_version.map | 4 +
> > src/drivers/vhost/rte_spp_vhost.c | 588 ++++++++++++++++++
> > 3 files changed, 620 insertions(+)
> > create mode 100644 src/drivers/vhost/Makefile
> > create mode 100644 src/drivers/vhost/rte_pmd_spp_vhost_version.map
> > create mode 100644 src/drivers/vhost/rte_spp_vhost.c
> >
> > diff --git a/src/drivers/vhost/Makefile b/src/drivers/vhost/Makefile
> [...]
> > +
> > +static inline struct pmd_internal *
> > +find_internal_resource(int vid)
> > +{
> > + struct pmd_internal *internal;
> > + int i;
> > + char ifname[PATH_MAX];
> This variable is declared, but not used from anywhere.
>
> > +
> > + if (rte_vhost_get_ifname(vid, ifname, sizeof(ifname)) == -1)
> > + return NULL;
> [...]
rte_vhost_get_ifname stores interface name to ifname.
ifname is compared with internal->ifname a few after lines.
"if (internal != NULL && !strcmp(internal->iface_name, ifname)) "
> > +}
> > +
> > +static const struct eth_dev_ops ops = {
> > + .dev_start = eth_dev_start,
> > + .dev_stop = eth_dev_stop,
> > + .dev_configure = eth_dev_configure,
> > + .dev_infos_get = eth_dev_info, > + .rx_queue_setup = eth_rx_queue_setup,
> > + .tx_queue_setup = eth_tx_queue_setup,
> > + .rx_queue_release = eth_queue_release,
> > + .tx_queue_release = eth_queue_release,
> > + .link_update = eth_link_update,
> > + .stats_get = eth_stats_get,
> > + .stats_reset = eth_stats_reset,
> > +};
> This struct causes compilation errors because of incompatible pointer types.
>
> $ make
> == src
> == drivers
> == vhost
> CC rte_spp_vhost.o
> rte_spp_vhost.c:371:19: error: initialization from incompatible pointer type [-Werror=incompatible-pointer-types]
> .dev_infos_get = eth_dev_info,
> ^~~~~~~~~~~~
> rte_spp_vhost.c:371:19: note: (near initialization for ‘ops.dev_infos_get’)
> rte_spp_vhost.c:378:17: error: initialization from incompatible pointer type [-Werror=incompatible-pointer-types]
> .stats_reset = eth_stats_reset,
> ^~~~~~~~~~~~~~~
> rte_spp_vhost.c:378:17: note: (near initialization for ‘ops.stats_reset’)
> cc1: error: unrecognized command line option ‘-Wno-address-of-packed-member’ [-Werror]
> cc1: all warnings being treated as errors
>
> From my understanding, eth_dev_info() and eth_stats_reset() are defined as void in your patch, but DPDK expects int by referring the definition of eth_dev_ops in lib/librte_ethdev/rte_ethdev_core.h:609.
The API of these methods are diffrent from v18.08 and v18.11 (or after).
You seem to make under v18.11 (or after). The patch is under v18.08.
Make sure your RTE_SDK.
> Regards,
> Yasufumi
Thanks.
> > +
> > +static int
> [...]
--
Itsuro ODA <oda@valinux.co.jp>
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [spp] [PATCH v2 01/12] drivers/vhost: add multi process supported vhost PMD for SPP
2020-01-08 1:17 ` Itsuro ODA
@ 2020-01-08 1:23 ` Itsuro ODA
0 siblings, 0 replies; 37+ messages in thread
From: Itsuro ODA @ 2020-01-08 1:23 UTC (permalink / raw)
To: Yasufumi Ogawa; +Cc: spp, ferruh.yigit
Hi,
Oops.
> The API of these methods are diffrent from v18.08 and v18.11 (or after).
> You seem to make under v18.11 (or after). The patch is under v18.08.
> Make sure your RTE_SDK.
v18.08 -> v19.08
v18.11 -> v19.11
Thanks.
On Wed, 08 Jan 2020 10:17:52 +0900
Itsuro ODA <oda@valinux.co.jp> wrote:
> Hi Yasufumi,
>
> On Tue, 7 Jan 2020 19:41:52 +0900
> Yasufumi Ogawa <yasufum.o@gmail.com> wrote:
>
> > Hi,
> >
> > On 2019/12/25 13:49, Itsuro Oda wrote:
> > > vhost PMD can not be used by secondary processes since DPDK 18.11.
> > > SPP project decided to have own vhost PMD which can be used by
> > > secondary processes at the moment. This vhost PMD is based on the
> > > original vhost PMD but is simplified very much only to support
> > > functions used by SPP. Thereby it becomes easy to fix the probrem.
> > >
> > > The main idea of the fix is that execution of vhost start/stop
> > > is moved to eth_dev_start/stop from probe/remove.
> > >
> > > Note that only process which executes eth_dev_start can use the
> > > vhost device although the vhost device is shared among the primary
> > > process and secondary processes. Once eth_dev_stop is executed by
> > > the process which used the vhost device, it is available to be
> > > used by any process. It is user responsibility that multipul
> > > processes don't use the vhost device at the same time.
> > >
> > > Signed-off-by: Itsuro Oda <oda@valinux.co.jp>
> > > ---
> > > src/drivers/vhost/Makefile | 28 +
> > > .../vhost/rte_pmd_spp_vhost_version.map | 4 +
> > > src/drivers/vhost/rte_spp_vhost.c | 588 ++++++++++++++++++
> > > 3 files changed, 620 insertions(+)
> > > create mode 100644 src/drivers/vhost/Makefile
> > > create mode 100644 src/drivers/vhost/rte_pmd_spp_vhost_version.map
> > > create mode 100644 src/drivers/vhost/rte_spp_vhost.c
> > >
> > > diff --git a/src/drivers/vhost/Makefile b/src/drivers/vhost/Makefile
> > [...]
> > > +
> > > +static inline struct pmd_internal *
> > > +find_internal_resource(int vid)
> > > +{
> > > + struct pmd_internal *internal;
> > > + int i;
> > > + char ifname[PATH_MAX];
> > This variable is declared, but not used from anywhere.
> >
> > > +
> > > + if (rte_vhost_get_ifname(vid, ifname, sizeof(ifname)) == -1)
> > > + return NULL;
> > [...]
>
> rte_vhost_get_ifname stores interface name to ifname.
> ifname is compared with internal->ifname a few after lines.
> "if (internal != NULL && !strcmp(internal->iface_name, ifname)) "
>
> > > +}
> > > +
> > > +static const struct eth_dev_ops ops = {
> > > + .dev_start = eth_dev_start,
> > > + .dev_stop = eth_dev_stop,
> > > + .dev_configure = eth_dev_configure,
> > > + .dev_infos_get = eth_dev_info, > + .rx_queue_setup = eth_rx_queue_setup,
> > > + .tx_queue_setup = eth_tx_queue_setup,
> > > + .rx_queue_release = eth_queue_release,
> > > + .tx_queue_release = eth_queue_release,
> > > + .link_update = eth_link_update,
> > > + .stats_get = eth_stats_get,
> > > + .stats_reset = eth_stats_reset,
> > > +};
> > This struct causes compilation errors because of incompatible pointer types.
> >
> > $ make
> > == src
> > == drivers
> > == vhost
> > CC rte_spp_vhost.o
> > rte_spp_vhost.c:371:19: error: initialization from incompatible pointer type [-Werror=incompatible-pointer-types]
> > .dev_infos_get = eth_dev_info,
> > ^~~~~~~~~~~~
> > rte_spp_vhost.c:371:19: note: (near initialization for ‘ops.dev_infos_get’)
> > rte_spp_vhost.c:378:17: error: initialization from incompatible pointer type [-Werror=incompatible-pointer-types]
> > .stats_reset = eth_stats_reset,
> > ^~~~~~~~~~~~~~~
> > rte_spp_vhost.c:378:17: note: (near initialization for ‘ops.stats_reset’)
> > cc1: error: unrecognized command line option ‘-Wno-address-of-packed-member’ [-Werror]
> > cc1: all warnings being treated as errors
> >
> > From my understanding, eth_dev_info() and eth_stats_reset() are defined as void in your patch, but DPDK expects int by referring the definition of eth_dev_ops in lib/librte_ethdev/rte_ethdev_core.h:609.
>
> The API of these methods are diffrent from v18.08 and v18.11 (or after).
> You seem to make under v18.11 (or after). The patch is under v18.08.
> Make sure your RTE_SDK.
>
> > Regards,
> > Yasufumi
>
> Thanks.
>
> > > +
> > > +static int
> > [...]
>
> --
> Itsuro ODA <oda@valinux.co.jp>
--
Itsuro ODA <oda@valinux.co.jp>
^ permalink raw reply [flat|nested] 37+ messages in thread
* [spp] [PATCH v3 00/12] revive vhost
2019-12-23 5:00 [spp] [PATCH 0/3] revive vhost Itsuro Oda
` (4 preceding siblings ...)
2019-12-25 4:49 ` [spp] [PATCH v2 00/12] " Itsuro Oda
@ 2020-01-09 23:10 ` Itsuro Oda
2020-01-09 23:10 ` [spp] [PATCH v3 01/12] drivers/vhost: add multi process supported vhost PMD for SPP Itsuro Oda
` (12 more replies)
5 siblings, 13 replies; 37+ messages in thread
From: Itsuro Oda @ 2020-01-09 23:10 UTC (permalink / raw)
To: spp, ferruh.yigit, yasufum.o
vhost PMD can not be used by secondary processes since DPDK 18.11.
SPP project decided to have own vhost PMD which can be used by
secondary processes at the moment. This vhost PMD is based on the
original vhost PMD but is simplified very much only to support
functions used by SPP. Thereby it becomes easy to fix the probrem.
v2:
- devide patches and add tags of commit title.
v3:
- rebase from DPDK v19.08 to v19.11
Itsuro Oda (12):
drivers/vhost: add multi process supported vhost PMD for SPP
drivers: add to build vhost PMD for SPP
shared: switch to use vhost PMD for SPP
spp_primary: add link to vhost PMD for SPP
spp_nfv: add link to vhost PMD for SPP
spp_vf: add link to vhost PMD for SPP
spp_mirror: add link to vhost PMD for SPP
spp_primary: stop vhost before detach
spp_nfv: stop vhost before detach
shared: make sure vhost is stopped before (re)using the vhost
spp_nfv: exclude vhosts at process initialization
spp_vf,spp_mirror: exclude vhosts at process initialization
src/Makefile | 1 +
src/drivers/Makefile | 15 +
src/drivers/vhost/Makefile | 28 +
.../vhost/rte_pmd_spp_vhost_version.map | 4 +
src/drivers/vhost/rte_spp_vhost.c | 592 ++++++++++++++++++
src/mirror/Makefile | 3 +
src/nfv/Makefile | 3 +
src/nfv/commands.h | 1 +
src/nfv/main.c | 7 +-
src/primary/Makefile | 3 +
src/primary/main.c | 1 +
src/shared/common.h | 2 +-
src/shared/secondary/add_port.c | 8 +
src/shared/secondary/add_port.h | 2 +-
.../secondary/spp_worker_th/cmd_utils.c | 8 +-
src/vf/Makefile | 3 +
16 files changed, 676 insertions(+), 5 deletions(-)
create mode 100644 src/drivers/Makefile
create mode 100644 src/drivers/vhost/Makefile
create mode 100644 src/drivers/vhost/rte_pmd_spp_vhost_version.map
create mode 100644 src/drivers/vhost/rte_spp_vhost.c
--
2.17.1
^ permalink raw reply [flat|nested] 37+ messages in thread
* [spp] [PATCH v3 01/12] drivers/vhost: add multi process supported vhost PMD for SPP
2020-01-09 23:10 ` [spp] [PATCH v3 00/12] revive vhost Itsuro Oda
@ 2020-01-09 23:10 ` Itsuro Oda
2020-01-09 23:10 ` [spp] [PATCH v3 02/12] drivers: add to build " Itsuro Oda
` (11 subsequent siblings)
12 siblings, 0 replies; 37+ messages in thread
From: Itsuro Oda @ 2020-01-09 23:10 UTC (permalink / raw)
To: spp, ferruh.yigit, yasufum.o
vhost PMD can not be used by secondary processes since DPDK 18.11.
SPP project decided to have own vhost PMD which can be used by
secondary processes at the moment. This vhost PMD is based on the
original vhost PMD but is simplified very much only to support
functions used by SPP. Thereby it becomes easy to fix the probrem.
The main idea of the fix is that execution of vhost start/stop
is moved to eth_dev_start/stop from probe/remove.
Note that only process which executes eth_dev_start can use the
vhost device although the vhost device is shared among the primary
process and secondary processes. Once eth_dev_stop is executed by
the process which used the vhost device, it is available to be
used by any process. It is user responsibility that multipul
processes don't use the vhost device at the same time.
Signed-off-by: Itsuro Oda <oda@valinux.co.jp>
---
src/drivers/vhost/Makefile | 28 +
.../vhost/rte_pmd_spp_vhost_version.map | 4 +
src/drivers/vhost/rte_spp_vhost.c | 592 ++++++++++++++++++
3 files changed, 624 insertions(+)
create mode 100644 src/drivers/vhost/Makefile
create mode 100644 src/drivers/vhost/rte_pmd_spp_vhost_version.map
create mode 100644 src/drivers/vhost/rte_spp_vhost.c
diff --git a/src/drivers/vhost/Makefile b/src/drivers/vhost/Makefile
new file mode 100644
index 0000000..e29c330
--- /dev/null
+++ b/src/drivers/vhost/Makefile
@@ -0,0 +1,28 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+#
+# library name
+#
+LIB = librte_pmd_spp_vhost.a
+
+LDLIBS += -lpthread
+LDLIBS += -lrte_eal -lrte_mbuf -lrte_mempool -lrte_ring
+LDLIBS += -lrte_ethdev -lrte_net -lrte_kvargs -lrte_vhost
+LDLIBS += -lrte_bus_vdev
+
+CFLAGS += -O3
+CFLAGS += $(WERROR_FLAGS)
+
+EXPORT_MAP := rte_pmd_spp_vhost_version.map
+
+LIBABIVER := 2
+
+#
+# all source are stored in SRCS-y
+#
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_VHOST) += rte_spp_vhost.c
+
+include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/src/drivers/vhost/rte_pmd_spp_vhost_version.map b/src/drivers/vhost/rte_pmd_spp_vhost_version.map
new file mode 100644
index 0000000..ef35398
--- /dev/null
+++ b/src/drivers/vhost/rte_pmd_spp_vhost_version.map
@@ -0,0 +1,4 @@
+DPDK_2.0 {
+
+ local: *;
+};
diff --git a/src/drivers/vhost/rte_spp_vhost.c b/src/drivers/vhost/rte_spp_vhost.c
new file mode 100644
index 0000000..fe63064
--- /dev/null
+++ b/src/drivers/vhost/rte_spp_vhost.c
@@ -0,0 +1,592 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2016 IGEL Co., Ltd.
+ * Copyright(c) 2016-2018 Intel Corporation
+ * Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+ */
+#include <unistd.h>
+#include <pthread.h>
+#include <stdbool.h>
+
+#include <rte_mbuf.h>
+#include <rte_ethdev_driver.h>
+#include <rte_ethdev_vdev.h>
+#include <rte_malloc.h>
+#include <rte_memcpy.h>
+#include <rte_bus_vdev.h>
+#include <rte_kvargs.h>
+#include <rte_vhost.h>
+#include <rte_spinlock.h>
+
+static int vhost_logtype;
+
+#define VHOST_LOG(level, ...) \
+ rte_log(RTE_LOG_ ## level, vhost_logtype, __VA_ARGS__)
+
+enum {VIRTIO_RXQ, VIRTIO_TXQ, VIRTIO_QNUM};
+
+#define ETH_VHOST_IFACE_ARG "iface"
+#define ETH_VHOST_QUEUES_ARG "queues"
+#define ETH_VHOST_CLIENT_ARG "client"
+
+static const char *valid_arguments[] = {
+ ETH_VHOST_IFACE_ARG,
+ ETH_VHOST_QUEUES_ARG,
+ ETH_VHOST_CLIENT_ARG,
+ NULL
+};
+
+struct vhost_queue {
+ struct pmd_internal *internal;
+ struct rte_mempool *mb_pool;
+ uint16_t virtqueue_id;
+ rte_spinlock_t queuing_lock;
+ uint64_t pkts;
+ uint64_t missed_pkts;
+};
+
+struct pmd_internal {
+ uint16_t max_queues;
+ uint64_t vhost_flags;
+ struct rte_eth_dev_data *eth_dev_data;
+ int vid;
+ char iface_name[PATH_MAX];
+};
+
+static struct rte_eth_link pmd_link = {
+ .link_speed = 10000,
+ .link_duplex = ETH_LINK_FULL_DUPLEX,
+ .link_status = ETH_LINK_DOWN
+};
+
+#define MZ_RTE_VHOST_PMD_INTERNAL "vhost_pmd_internal"
+static struct pmd_internal **pmd_internal_list;
+
+static uint16_t
+eth_vhost_rx(void *q, struct rte_mbuf **bufs, uint16_t nb_bufs)
+{
+ struct vhost_queue *r = q;
+ struct pmd_internal *internal;
+ uint16_t nb_rx = 0;
+
+ if (!q)
+ return 0;
+
+ internal = r->internal;
+ rte_spinlock_lock(&r->queuing_lock);
+ if (internal->vid == -1)
+ goto out;
+
+ /* Dequeue packets from guest TX queue */
+ nb_rx = rte_vhost_dequeue_burst(internal->vid, r->virtqueue_id,
+ r->mb_pool, bufs, nb_bufs);
+ r->pkts += nb_rx;
+
+out:
+ rte_spinlock_unlock(&r->queuing_lock);
+
+ return nb_rx;
+}
+
+static uint16_t
+eth_vhost_tx(void *q, struct rte_mbuf **bufs, uint16_t nb_bufs)
+{
+ struct vhost_queue *r = q;
+ struct pmd_internal *internal;
+ uint16_t i, nb_tx = 0;
+
+ if (!q)
+ return 0;
+
+ internal = r->internal;
+ rte_spinlock_lock(&r->queuing_lock);
+ if (internal->vid == -1)
+ goto out;
+
+ /* Enqueue packets to guest RX queue */
+ nb_tx = rte_vhost_enqueue_burst(internal->vid, r->virtqueue_id,
+ bufs, nb_bufs);
+ r->pkts += nb_tx;
+ r->missed_pkts += nb_bufs - nb_tx;
+
+ for (i = 0; i < nb_tx; i++)
+ rte_pktmbuf_free(bufs[i]);
+
+out:
+ rte_spinlock_unlock(&r->queuing_lock);
+
+ return nb_tx;
+}
+
+static inline struct pmd_internal *
+find_internal_resource(int vid)
+{
+ struct pmd_internal *internal;
+ int i;
+ char ifname[PATH_MAX];
+
+ if (rte_vhost_get_ifname(vid, ifname, sizeof(ifname)) == -1)
+ return NULL;
+
+ /* likely(found in a few loops) */
+ for (i = 0; i < RTE_MAX_ETHPORTS; i++) {
+ internal = pmd_internal_list[i];
+ if (internal != NULL && !strcmp(internal->iface_name, ifname)) {
+ return internal;
+ }
+ }
+
+ return NULL;
+}
+
+static int
+new_device(int vid)
+{
+ struct pmd_internal *internal;
+
+ internal = find_internal_resource(vid);
+ if (internal == NULL) {
+ VHOST_LOG(INFO, "Invalid device : %d\n", vid);
+ return -1;
+ }
+
+ internal->vid = vid;
+ internal->eth_dev_data->dev_link.link_status = ETH_LINK_UP;
+
+ VHOST_LOG(INFO, "Vhost device %d created\n", vid);
+
+ return 0;
+}
+
+static void
+destroy_device(int vid)
+{
+ struct pmd_internal *internal;
+ struct vhost_queue *vq;
+ struct rte_eth_dev_data *data;
+ int i;
+
+ internal = find_internal_resource(vid);
+ if (internal == NULL) {
+ VHOST_LOG(ERR, "Invalid device : %d\n", vid);
+ return;
+ }
+ data = internal->eth_dev_data;
+
+ /* wait inflight queuing done */
+ for (i = 0; i < data->nb_rx_queues; i++) {
+ if ((vq = data->rx_queues[i]) != NULL)
+ rte_spinlock_lock(&vq->queuing_lock);
+ }
+ for (i = 0; i < data->nb_tx_queues; i++) {
+ if ((vq = data->tx_queues[i]) != NULL)
+ rte_spinlock_lock(&vq->queuing_lock);
+ }
+
+ data->dev_link.link_status = ETH_LINK_DOWN;
+ internal->vid = -1;
+
+ for (i = 0; i < data->nb_rx_queues; i++) {
+ if ((vq = data->rx_queues[i]) != NULL)
+ rte_spinlock_unlock(&vq->queuing_lock);
+ }
+ for (i = 0; i < data->nb_tx_queues; i++) {
+ if ((vq = data->tx_queues[i]) != NULL)
+ rte_spinlock_unlock(&vq->queuing_lock);
+ }
+
+ VHOST_LOG(INFO, "Vhost device %d destroyed\n", vid);
+}
+
+static struct vhost_device_ops vhost_ops = {
+ .new_device = new_device,
+ .destroy_device = destroy_device,
+};
+
+static int
+eth_dev_start(struct rte_eth_dev *eth_dev)
+{
+ struct pmd_internal *internal = eth_dev->data->dev_private;
+
+ if (rte_vhost_driver_register(internal->iface_name, internal->vhost_flags))
+ return -1;
+
+ if (rte_vhost_driver_callback_register(internal->iface_name, &vhost_ops) < 0) {
+ VHOST_LOG(ERR, "Can't register callbacks\n");
+ return -1;
+ }
+
+ if (rte_vhost_driver_start(internal->iface_name) < 0) {
+ VHOST_LOG(ERR, "Failed to start driver for %s\n",
+ internal->iface_name);
+ return -1;
+ }
+
+ return 0;
+}
+
+static void
+eth_dev_stop(struct rte_eth_dev *dev)
+{
+ struct pmd_internal *internal = dev->data->dev_private;
+
+ rte_vhost_driver_unregister(internal->iface_name);
+}
+
+static int
+eth_dev_configure(struct rte_eth_dev *dev __rte_unused)
+{
+ return 0;
+}
+
+static int
+eth_dev_info(struct rte_eth_dev *dev,
+ struct rte_eth_dev_info *dev_info)
+{
+ struct pmd_internal *internal = dev->data->dev_private;
+
+ dev_info->max_mac_addrs = 1;
+ dev_info->max_rx_pktlen = (uint32_t)-1;
+ dev_info->max_rx_queues = internal->max_queues;
+ dev_info->max_tx_queues = internal->max_queues;
+ dev_info->min_rx_bufsize = 0;
+
+ return 0;
+}
+
+static int
+eth_rx_queue_setup(struct rte_eth_dev *dev, uint16_t rx_queue_id,
+ uint16_t nb_rx_desc __rte_unused,
+ unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf __rte_unused,
+ struct rte_mempool *mb_pool)
+{
+ struct pmd_internal *internal = dev->data->dev_private;
+ struct vhost_queue *vq;
+
+ vq = rte_zmalloc_socket(NULL, sizeof(struct vhost_queue),
+ RTE_CACHE_LINE_SIZE, socket_id);
+ if (vq == NULL) {
+ VHOST_LOG(ERR, "Failed to allocate memory for rx queue\n");
+ return -ENOMEM;
+ }
+
+ vq->internal = internal;
+ vq->mb_pool = mb_pool;
+ vq->virtqueue_id = rx_queue_id * VIRTIO_QNUM + VIRTIO_TXQ;
+ rte_spinlock_init(&vq->queuing_lock);
+ dev->data->rx_queues[rx_queue_id] = vq;
+
+ return 0;
+}
+
+static int
+eth_tx_queue_setup(struct rte_eth_dev *dev, uint16_t tx_queue_id,
+ uint16_t nb_tx_desc __rte_unused,
+ unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf __rte_unused)
+{
+ struct pmd_internal *internal = dev->data->dev_private;
+ struct vhost_queue *vq;
+
+ vq = rte_zmalloc_socket(NULL, sizeof(struct vhost_queue),
+ RTE_CACHE_LINE_SIZE, socket_id);
+ if (vq == NULL) {
+ VHOST_LOG(ERR, "Failed to allocate memory for tx queue\n");
+ return -ENOMEM;
+ }
+
+ vq->internal = internal;
+ vq->virtqueue_id = tx_queue_id * VIRTIO_QNUM + VIRTIO_RXQ;
+ rte_spinlock_init(&vq->queuing_lock);
+ dev->data->tx_queues[tx_queue_id] = vq;
+
+ return 0;
+}
+
+static void
+eth_queue_release(void *q)
+{
+ rte_free(q);
+}
+
+static int
+eth_link_update(struct rte_eth_dev *dev __rte_unused,
+ int wait_to_complete __rte_unused)
+{
+ return 0;
+}
+
+static int
+eth_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats)
+{
+ unsigned i;
+ unsigned long rx_total = 0, tx_total = 0, tx_err_total = 0;
+ struct vhost_queue *vq;
+
+ for (i = 0; i < RTE_ETHDEV_QUEUE_STAT_CNTRS &&
+ i < dev->data->nb_rx_queues; i++) {
+ if ((vq = dev->data->rx_queues[i]) != NULL) {
+ stats->q_ipackets[i] = vq->pkts;
+ rx_total += vq->pkts;
+ }
+ }
+
+ for (i = 0; i < RTE_ETHDEV_QUEUE_STAT_CNTRS &&
+ i < dev->data->nb_tx_queues; i++) {
+ if ((vq = dev->data->rx_queues[i]) != NULL) {
+ stats->q_opackets[i] = vq->pkts;
+ tx_total += vq->pkts;
+ stats->q_errors[i] = vq->missed_pkts;
+ tx_err_total += vq->missed_pkts;
+ }
+ }
+
+ stats->ipackets = rx_total;
+ stats->opackets = tx_total;
+ stats->oerrors = tx_err_total;
+
+ return 0;
+}
+
+static int
+eth_stats_reset(struct rte_eth_dev *dev)
+{
+ struct vhost_queue *vq;
+ unsigned i;
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ if ((vq = dev->data->rx_queues[i]) != NULL)
+ vq->pkts = 0;
+ }
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ if ((vq = dev->data->rx_queues[i]) != NULL) {
+ vq->pkts = 0;
+ vq->missed_pkts = 0;
+ }
+ }
+
+ return 0;
+}
+
+static const struct eth_dev_ops ops = {
+ .dev_start = eth_dev_start,
+ .dev_stop = eth_dev_stop,
+ .dev_configure = eth_dev_configure,
+ .dev_infos_get = eth_dev_info,
+ .rx_queue_setup = eth_rx_queue_setup,
+ .tx_queue_setup = eth_tx_queue_setup,
+ .rx_queue_release = eth_queue_release,
+ .tx_queue_release = eth_queue_release,
+ .link_update = eth_link_update,
+ .stats_get = eth_stats_get,
+ .stats_reset = eth_stats_reset,
+};
+
+static int
+init_shared_data(void)
+{
+ const struct rte_memzone *mz;
+
+ if (pmd_internal_list == NULL) {
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+ size_t len = sizeof(*pmd_internal_list) * RTE_MAX_ETHPORTS;
+ mz = rte_memzone_reserve(MZ_RTE_VHOST_PMD_INTERNAL,
+ len, rte_socket_id(), 0);
+ if (mz)
+ memset(mz->addr, 0, len);
+ } else
+ mz = rte_memzone_lookup(MZ_RTE_VHOST_PMD_INTERNAL);
+ if (mz == NULL) {
+ VHOST_LOG(ERR, "Cannot allocate vhost shared data\n");
+ return -1;
+ }
+ pmd_internal_list = mz->addr;
+ }
+
+ return 0;
+}
+
+static int
+eth_dev_vhost_create(struct rte_vdev_device *dev, char *iface_name,
+ int16_t queues, const unsigned int numa_node, uint64_t flags)
+{
+ struct rte_eth_dev_data *data;
+ struct pmd_internal *internal;
+ struct rte_eth_dev *eth_dev;
+ struct rte_ether_addr *eth_addr;
+
+ VHOST_LOG(INFO, "Creating VHOST-USER backend on numa socket %u\n",
+ numa_node);
+
+ eth_addr = rte_zmalloc_socket(NULL, sizeof(*eth_addr), 0, numa_node);
+ if (eth_addr == NULL)
+ return -ENOMEM;
+
+ eth_dev = rte_eth_vdev_allocate(dev, sizeof(*internal));
+ if (eth_dev == NULL)
+ return -ENOMEM;
+
+ data = eth_dev->data;
+
+ data->dev_link = pmd_link;
+ data->dev_flags = RTE_ETH_DEV_INTR_LSC;
+ data->mac_addrs = eth_addr;
+
+ eth_dev->dev_ops = &ops;
+ eth_dev->rx_pkt_burst = eth_vhost_rx;
+ eth_dev->tx_pkt_burst = eth_vhost_tx;
+
+ internal = data->dev_private;
+
+ internal->max_queues = queues;
+ internal->vid = -1;
+ internal->vhost_flags = flags;
+ internal->eth_dev_data = data;
+ strncpy(internal->iface_name, iface_name, sizeof(internal->iface_name));
+
+ pmd_internal_list[data->port_id] = internal;
+
+ rte_eth_dev_probing_finish(eth_dev);
+
+ return 0;
+}
+
+static inline int
+open_iface(const char *key __rte_unused, const char *value, void *extra_args)
+{
+ const char **iface_name = extra_args;
+
+ if (value == NULL)
+ return -1;
+
+ *iface_name = value;
+
+ return 0;
+}
+
+static inline int
+open_int(const char *key __rte_unused, const char *value, void *extra_args)
+{
+ uint16_t *n = extra_args;
+ char *endptr;
+
+ if (value == NULL)
+ return -1;
+
+ *n = (uint16_t)strtoul(value, &endptr, 0);
+ if (*endptr != '\0' || errno == ERANGE)
+ return -1;
+
+ return 0;
+}
+
+static int
+rte_pmd_vhost_probe(struct rte_vdev_device *dev)
+{
+ struct rte_kvargs *kvlist = NULL;
+ int ret = 0;
+ char *iface_name = NULL;
+ uint16_t queues = 1;
+ uint64_t flags = 0;
+ uint16_t client_mode = 0;
+ struct rte_eth_dev *eth_dev;
+ const char *name = rte_vdev_device_name(dev);
+
+ VHOST_LOG(INFO, "Initializing pmd_vhost for %s\n", name);
+
+ if (init_shared_data() == -1)
+ return -ENOMEM;
+
+ if (rte_eal_process_type() == RTE_PROC_SECONDARY) {
+ eth_dev = rte_eth_dev_attach_secondary(name);
+ if (!eth_dev) {
+ VHOST_LOG(ERR, "Failed to probe %s\n", name);
+ return -ENOENT;
+ }
+ eth_dev->rx_pkt_burst = eth_vhost_rx;
+ eth_dev->tx_pkt_burst = eth_vhost_tx;
+
+ eth_dev->dev_ops = &ops;
+ eth_dev->device = &dev->device;
+ rte_eth_dev_probing_finish(eth_dev);
+ return 0;
+ }
+
+ kvlist = rte_kvargs_parse(rte_vdev_device_args(dev), valid_arguments);
+ if (kvlist == NULL)
+ return -EINVAL;
+
+ if (rte_kvargs_process(kvlist, ETH_VHOST_IFACE_ARG,
+ &open_iface, &iface_name) == -1 ||
+ rte_kvargs_process(kvlist, ETH_VHOST_QUEUES_ARG,
+ &open_int, &queues) == -1 ||
+ rte_kvargs_process(kvlist, ETH_VHOST_CLIENT_ARG,
+ &open_int, &client_mode) == -1) {
+ rte_kvargs_free(kvlist);
+ return -EINVAL;
+ }
+
+ if (iface_name == NULL ||
+ queues == 0 || queues > RTE_MAX_QUEUES_PER_PORT ||
+ (client_mode != 0 && client_mode != 1)) {
+ rte_kvargs_free(kvlist);
+ return -EINVAL;
+ }
+
+ if (client_mode)
+ flags |= RTE_VHOST_USER_CLIENT;
+
+ ret = eth_dev_vhost_create(dev, iface_name, queues, rte_socket_id(), flags);
+
+ rte_kvargs_free(kvlist);
+ return ret;
+}
+
+static int
+rte_pmd_vhost_remove(struct rte_vdev_device *dev)
+{
+ const char *name = rte_vdev_device_name(dev);
+ struct rte_eth_dev *eth_dev;
+ int i;
+
+ VHOST_LOG(INFO, "Un-Initializing pmd_vhost for %s\n", name);
+
+ eth_dev = rte_eth_dev_allocated(name);
+ if (eth_dev == NULL)
+ return 0;
+
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+ return rte_eth_dev_release_port(eth_dev);
+
+ if (eth_dev->data->dev_started) {
+ VHOST_LOG(WARNING, "device must be stoped.\n");
+ }
+
+ for (i = 0; i < eth_dev->data->nb_rx_queues; i++)
+ rte_free(eth_dev->data->rx_queues[i]);
+
+ for (i = 0; i < eth_dev->data->nb_tx_queues; i++)
+ rte_free(eth_dev->data->tx_queues[i]);
+
+ pmd_internal_list[eth_dev->data->port_id] = NULL;
+
+ return rte_eth_dev_release_port(eth_dev);
+}
+
+static struct rte_vdev_driver pmd_vhost_drv = {
+ .probe = rte_pmd_vhost_probe,
+ .remove = rte_pmd_vhost_remove,
+};
+
+RTE_PMD_REGISTER_VDEV(spp_vhost, pmd_vhost_drv);
+RTE_PMD_REGISTER_PARAM_STRING(spp_vhost,
+ "iface=<ifc> "
+ "queues=<int> "
+ "client=<0|1> ");
+
+RTE_INIT(vhost_init_log)
+{
+ vhost_logtype = rte_log_register("pmd.spp.vhost");
+ if (vhost_logtype >= 0)
+ rte_log_set_level(vhost_logtype, RTE_LOG_NOTICE);
+}
--
2.17.1
^ permalink raw reply [flat|nested] 37+ messages in thread
* [spp] [PATCH v3 02/12] drivers: add to build vhost PMD for SPP
2020-01-09 23:10 ` [spp] [PATCH v3 00/12] revive vhost Itsuro Oda
2020-01-09 23:10 ` [spp] [PATCH v3 01/12] drivers/vhost: add multi process supported vhost PMD for SPP Itsuro Oda
@ 2020-01-09 23:10 ` Itsuro Oda
2020-01-09 23:10 ` [spp] [PATCH v3 03/12] shared: switch to use " Itsuro Oda
` (10 subsequent siblings)
12 siblings, 0 replies; 37+ messages in thread
From: Itsuro Oda @ 2020-01-09 23:10 UTC (permalink / raw)
To: spp, ferruh.yigit, yasufum.o
This patch adds to build vhost PMD for SPP.
Signed-off-by: Itsuro Oda <oda@valinux.co.jp>
---
src/Makefile | 1 +
src/drivers/Makefile | 15 +++++++++++++++
2 files changed, 16 insertions(+)
create mode 100644 src/drivers/Makefile
diff --git a/src/Makefile b/src/Makefile
index cab80db..2f46606 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -38,6 +38,7 @@ RTE_TARGET ?= x86_64-native-linuxapp-gcc
include $(RTE_SDK)/mk/rte.vars.mk
+DIRS-y += drivers
DIRS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += primary
DIRS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += nfv
DIRS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += vf
diff --git a/src/drivers/Makefile b/src/drivers/Makefile
new file mode 100644
index 0000000..f2deba7
--- /dev/null
+++ b/src/drivers/Makefile
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+
+ifeq ($(RTE_SDK),)
+$(error "Please define RTE_SDK environment variable")
+endif
+
+# Default target, can be overriden by command line or environment
+RTE_TARGET ?= x86_64-native-linuxapp-gcc
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+DIRS-y += vhost
+
+include $(RTE_SDK)/mk/rte.extsubdir.mk
--
2.17.1
^ permalink raw reply [flat|nested] 37+ messages in thread
* [spp] [PATCH v3 03/12] shared: switch to use vhost PMD for SPP
2020-01-09 23:10 ` [spp] [PATCH v3 00/12] revive vhost Itsuro Oda
2020-01-09 23:10 ` [spp] [PATCH v3 01/12] drivers/vhost: add multi process supported vhost PMD for SPP Itsuro Oda
2020-01-09 23:10 ` [spp] [PATCH v3 02/12] drivers: add to build " Itsuro Oda
@ 2020-01-09 23:10 ` Itsuro Oda
2020-01-09 23:10 ` [spp] [PATCH v3 04/12] spp_primary: add link to " Itsuro Oda
` (9 subsequent siblings)
12 siblings, 0 replies; 37+ messages in thread
From: Itsuro Oda @ 2020-01-09 23:10 UTC (permalink / raw)
To: spp, ferruh.yigit, yasufum.o
This patch switches to use vhost PMD for SPP instead of original
vhost PMD.
Signed-off-by: Itsuro Oda <oda@valinux.co.jp>
---
src/shared/common.h | 2 +-
src/shared/secondary/add_port.h | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/src/shared/common.h b/src/shared/common.h
index 9c46a64..431ad3e 100644
--- a/src/shared/common.h
+++ b/src/shared/common.h
@@ -34,7 +34,7 @@
#define VDEV_ETH_RING "eth_ring"
#define VDEV_NET_RING "net_ring"
-#define VDEV_ETH_VHOST "eth_vhost"
+#define VDEV_ETH_VHOST "spp_vhost"
#define VDEV_NET_VHOST "net_vhost"
#define VDEV_PCAP "net_pcap"
#define VDEV_ETH_TAP "eth_tap"
diff --git a/src/shared/secondary/add_port.h b/src/shared/secondary/add_port.h
index cfae1af..a75b28b 100644
--- a/src/shared/secondary/add_port.h
+++ b/src/shared/secondary/add_port.h
@@ -10,7 +10,7 @@
#define NR_DESCS 128
#define VHOST_IFACE_NAME "/tmp/sock%u"
-#define VHOST_BACKEND_NAME "eth_vhost%u"
+#define VHOST_BACKEND_NAME "spp_vhost%u"
#define PCAP_PMD_DEV_NAME "eth_pcap%u"
#define NULL_PMD_DEV_NAME "eth_null%u"
--
2.17.1
^ permalink raw reply [flat|nested] 37+ messages in thread
* [spp] [PATCH v3 04/12] spp_primary: add link to vhost PMD for SPP
2020-01-09 23:10 ` [spp] [PATCH v3 00/12] revive vhost Itsuro Oda
` (2 preceding siblings ...)
2020-01-09 23:10 ` [spp] [PATCH v3 03/12] shared: switch to use " Itsuro Oda
@ 2020-01-09 23:10 ` Itsuro Oda
2020-01-09 23:10 ` [spp] [PATCH v3 05/12] spp_nfv: " Itsuro Oda
` (8 subsequent siblings)
12 siblings, 0 replies; 37+ messages in thread
From: Itsuro Oda @ 2020-01-09 23:10 UTC (permalink / raw)
To: spp, ferruh.yigit, yasufum.o
This patch adds link to vhost PMD for SPP.
Signed-off-by: Itsuro Oda <oda@valinux.co.jp>
---
src/primary/Makefile | 3 +++
1 file changed, 3 insertions(+)
diff --git a/src/primary/Makefile b/src/primary/Makefile
index 14d8f30..ace6228 100644
--- a/src/primary/Makefile
+++ b/src/primary/Makefile
@@ -41,4 +41,7 @@ endif
# and so the next line can be removed in those cases.
EXTRA_CFLAGS += -fno-strict-aliasing
+SPP_DRIVERS_DIR = $(BASE_OUTPUT)/src/drivers
+EXTRA_LDLIBS = -L$(SPP_DRIVERS_DIR)/vhost --whole-archive -lrte_pmd_spp_vhost --no-whole-archive
+
include $(RTE_SDK)/mk/rte.extapp.mk
--
2.17.1
^ permalink raw reply [flat|nested] 37+ messages in thread
* [spp] [PATCH v3 05/12] spp_nfv: add link to vhost PMD for SPP
2020-01-09 23:10 ` [spp] [PATCH v3 00/12] revive vhost Itsuro Oda
` (3 preceding siblings ...)
2020-01-09 23:10 ` [spp] [PATCH v3 04/12] spp_primary: add link to " Itsuro Oda
@ 2020-01-09 23:10 ` Itsuro Oda
2020-01-09 23:10 ` [spp] [PATCH v3 06/12] spp_vf: " Itsuro Oda
` (7 subsequent siblings)
12 siblings, 0 replies; 37+ messages in thread
From: Itsuro Oda @ 2020-01-09 23:10 UTC (permalink / raw)
To: spp, ferruh.yigit, yasufum.o
This patch adds link to vhost PMD for SPP.
Signed-off-by: Itsuro Oda <oda@valinux.co.jp>
---
src/nfv/Makefile | 3 +++
1 file changed, 3 insertions(+)
diff --git a/src/nfv/Makefile b/src/nfv/Makefile
index c3d2806..a717616 100644
--- a/src/nfv/Makefile
+++ b/src/nfv/Makefile
@@ -27,4 +27,7 @@ LDLIBS += -lrte_pmd_ring
LDLIBS += -lrte_pmd_vhost
endif
+SPP_DRIVERS_DIR = $(BASE_OUTPUT)/src/drivers
+EXTRA_LDLIBS = -L$(SPP_DRIVERS_DIR)/vhost --whole-archive -lrte_pmd_spp_vhost --no-whole-archive
+
include $(RTE_SDK)/mk/rte.extapp.mk
--
2.17.1
^ permalink raw reply [flat|nested] 37+ messages in thread
* [spp] [PATCH v3 06/12] spp_vf: add link to vhost PMD for SPP
2020-01-09 23:10 ` [spp] [PATCH v3 00/12] revive vhost Itsuro Oda
` (4 preceding siblings ...)
2020-01-09 23:10 ` [spp] [PATCH v3 05/12] spp_nfv: " Itsuro Oda
@ 2020-01-09 23:10 ` Itsuro Oda
2020-01-09 23:10 ` [spp] [PATCH v3 07/12] spp_mirror: " Itsuro Oda
` (6 subsequent siblings)
12 siblings, 0 replies; 37+ messages in thread
From: Itsuro Oda @ 2020-01-09 23:10 UTC (permalink / raw)
To: spp, ferruh.yigit, yasufum.o
This patch adds link to vhost PMD for SPP.
Signed-off-by: Itsuro Oda <oda@valinux.co.jp>
---
src/vf/Makefile | 3 +++
1 file changed, 3 insertions(+)
diff --git a/src/vf/Makefile b/src/vf/Makefile
index 43e5e4d..e42f8b7 100644
--- a/src/vf/Makefile
+++ b/src/vf/Makefile
@@ -48,4 +48,7 @@ LDLIBS += -lrte_pmd_ring
LDLIBS += -lrte_pmd_vhost
endif
+SPP_DRIVERS_DIR = $(BASE_OUTPUT)/src/drivers
+EXTRA_LDLIBS = -L$(SPP_DRIVERS_DIR)/vhost --whole-archive -lrte_pmd_spp_vhost --no-whole-archive
+
include $(RTE_SDK)/mk/rte.extapp.mk
--
2.17.1
^ permalink raw reply [flat|nested] 37+ messages in thread
* [spp] [PATCH v3 07/12] spp_mirror: add link to vhost PMD for SPP
2020-01-09 23:10 ` [spp] [PATCH v3 00/12] revive vhost Itsuro Oda
` (5 preceding siblings ...)
2020-01-09 23:10 ` [spp] [PATCH v3 06/12] spp_vf: " Itsuro Oda
@ 2020-01-09 23:10 ` Itsuro Oda
2020-01-09 23:10 ` [spp] [PATCH v3 08/12] spp_primary: stop vhost before detach Itsuro Oda
` (5 subsequent siblings)
12 siblings, 0 replies; 37+ messages in thread
From: Itsuro Oda @ 2020-01-09 23:10 UTC (permalink / raw)
To: spp, ferruh.yigit, yasufum.o
This patch adds link to vhost PMD for SPP.
Signed-off-by: Itsuro Oda <oda@valinux.co.jp>
---
src/mirror/Makefile | 3 +++
1 file changed, 3 insertions(+)
diff --git a/src/mirror/Makefile b/src/mirror/Makefile
index 0bd079a..f82b08f 100644
--- a/src/mirror/Makefile
+++ b/src/mirror/Makefile
@@ -53,4 +53,7 @@ LDLIBS += -lrte_pmd_ring
LDLIBS += -lrte_pmd_vhost
endif
+SPP_DRIVERS_DIR = $(BASE_OUTPUT)/src/drivers
+EXTRA_LDLIBS = -L$(SPP_DRIVERS_DIR)/vhost --whole-archive -lrte_pmd_spp_vhost --no-whole-archive
+
include $(RTE_SDK)/mk/rte.extapp.mk
--
2.17.1
^ permalink raw reply [flat|nested] 37+ messages in thread
* [spp] [PATCH v3 08/12] spp_primary: stop vhost before detach
2020-01-09 23:10 ` [spp] [PATCH v3 00/12] revive vhost Itsuro Oda
` (6 preceding siblings ...)
2020-01-09 23:10 ` [spp] [PATCH v3 07/12] spp_mirror: " Itsuro Oda
@ 2020-01-09 23:10 ` Itsuro Oda
2020-01-09 23:10 ` [spp] [PATCH v3 09/12] spp_nfv: " Itsuro Oda
` (4 subsequent siblings)
12 siblings, 0 replies; 37+ messages in thread
From: Itsuro Oda @ 2020-01-09 23:10 UTC (permalink / raw)
To: spp, ferruh.yigit, yasufum.o
Make sure to free resources of vhost before detach.
Signed-off-by: Itsuro Oda <oda@valinux.co.jp>
---
src/primary/main.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/src/primary/main.c b/src/primary/main.c
index 26e9c42..a1ca791 100644
--- a/src/primary/main.c
+++ b/src/primary/main.c
@@ -874,6 +874,7 @@ del_port(char *p_type, int p_id)
dev_id = find_ethdev_id(p_id, VHOST);
if (dev_id == PORT_RESET)
return -1;
+ rte_eth_dev_stop(dev_id);
dev_detach_by_port_id(dev_id);
} else if (!strcmp(p_type, "ring")) {
--
2.17.1
^ permalink raw reply [flat|nested] 37+ messages in thread
* [spp] [PATCH v3 09/12] spp_nfv: stop vhost before detach
2020-01-09 23:10 ` [spp] [PATCH v3 00/12] revive vhost Itsuro Oda
` (7 preceding siblings ...)
2020-01-09 23:10 ` [spp] [PATCH v3 08/12] spp_primary: stop vhost before detach Itsuro Oda
@ 2020-01-09 23:10 ` Itsuro Oda
2020-01-09 23:10 ` [spp] [PATCH v3 10/12] shared: make sure vhost is stopped before (re)using the vhost Itsuro Oda
` (3 subsequent siblings)
12 siblings, 0 replies; 37+ messages in thread
From: Itsuro Oda @ 2020-01-09 23:10 UTC (permalink / raw)
To: spp, ferruh.yigit, yasufum.o
Make sure to free resources of vhost before detach.
Signed-off-by: Itsuro Oda <oda@valinux.co.jp>
---
src/nfv/commands.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/src/nfv/commands.h b/src/nfv/commands.h
index f6c2305..7e50c8c 100644
--- a/src/nfv/commands.h
+++ b/src/nfv/commands.h
@@ -25,6 +25,7 @@ do_del(char *p_type, int p_id)
port_id = find_port_id(p_id, VHOST);
if (port_id == PORT_RESET)
return -1;
+ rte_eth_dev_stop(port_id);
dev_detach_by_port_id(port_id);
} else if (!strcmp(p_type, "ring")) {
--
2.17.1
^ permalink raw reply [flat|nested] 37+ messages in thread
* [spp] [PATCH v3 10/12] shared: make sure vhost is stopped before (re)using the vhost
2020-01-09 23:10 ` [spp] [PATCH v3 00/12] revive vhost Itsuro Oda
` (8 preceding siblings ...)
2020-01-09 23:10 ` [spp] [PATCH v3 09/12] spp_nfv: " Itsuro Oda
@ 2020-01-09 23:10 ` Itsuro Oda
2020-01-09 23:10 ` [spp] [PATCH v3 11/12] spp_nfv: exclude vhosts at process initialization Itsuro Oda
` (2 subsequent siblings)
12 siblings, 0 replies; 37+ messages in thread
From: Itsuro Oda @ 2020-01-09 23:10 UTC (permalink / raw)
To: spp, ferruh.yigit, yasufum.o
It is for the case a secondary process which used the vhost
was down without stopping the eth_dev.
It makes the eth_dev stopped and available to do following
operations(ex. rte_eth_dev_configure). It is no-op if the
eth_dev is stopped.
Signed-off-by: Itsuro Oda <oda@valinux.co.jp>
---
src/shared/secondary/add_port.c | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/src/shared/secondary/add_port.c b/src/shared/secondary/add_port.c
index b072140..d845250 100644
--- a/src/shared/secondary/add_port.c
+++ b/src/shared/secondary/add_port.c
@@ -170,6 +170,14 @@ add_vhost_pmd(int index)
return ret;
}
+ /* NOTE: make sure the eth_dev is stopped.
+ * it is for the case a secondary process which used the vhost
+ * was down without stopping the device.
+ * note that it is still user responsibility to prevent multipul
+ * processes use a vhost at the same time.
+ */
+ rte_eth_dev_stop(vhost_port_id);
+
ret = rte_eth_dev_configure(vhost_port_id, nr_queues, nr_queues,
&port_conf);
if (ret < 0) {
--
2.17.1
^ permalink raw reply [flat|nested] 37+ messages in thread
* [spp] [PATCH v3 11/12] spp_nfv: exclude vhosts at process initialization
2020-01-09 23:10 ` [spp] [PATCH v3 00/12] revive vhost Itsuro Oda
` (9 preceding siblings ...)
2020-01-09 23:10 ` [spp] [PATCH v3 10/12] shared: make sure vhost is stopped before (re)using the vhost Itsuro Oda
@ 2020-01-09 23:10 ` Itsuro Oda
2020-01-09 23:10 ` [spp] [PATCH v3 12/12] spp_vf, spp_mirror: " Itsuro Oda
2020-01-17 2:27 ` [spp] [PATCH v3 00/12] revive vhost Yasufumi Ogawa
12 siblings, 0 replies; 37+ messages in thread
From: Itsuro Oda @ 2020-01-09 23:10 UTC (permalink / raw)
To: spp, ferruh.yigit, yasufum.o
A vhost device is shared among the primary process and secondary
processes. When a secodary process starts it recognizes vhost
devices if they are used by processes already started. It is not
appropriate to include to port information as PHY devices.
Signed-off-by: Itsuro Oda <oda@valinux.co.jp>
---
src/nfv/main.c | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/src/nfv/main.c b/src/nfv/main.c
index 513a98d..f2c6bfc 100644
--- a/src/nfv/main.c
+++ b/src/nfv/main.c
@@ -211,7 +211,12 @@ main(int argc, char *argv[])
if (port_type == PHY) {
port_id = nof_phy_port;
nof_phy_port++;
- }
+ } else if (port_type == VHOST)
+ continue;
+ /* NOTE: vhost may be used another process. even if no
+ * process uses, it is necessary to "add vhost" explicitly.
+ * not display to avoid confusion.
+ */
/* Update ports_fwd_array with phy port. */
ports_fwd_array[i].in_port_id = i;
--
2.17.1
^ permalink raw reply [flat|nested] 37+ messages in thread
* [spp] [PATCH v3 12/12] spp_vf, spp_mirror: exclude vhosts at process initialization
2020-01-09 23:10 ` [spp] [PATCH v3 00/12] revive vhost Itsuro Oda
` (10 preceding siblings ...)
2020-01-09 23:10 ` [spp] [PATCH v3 11/12] spp_nfv: exclude vhosts at process initialization Itsuro Oda
@ 2020-01-09 23:10 ` Itsuro Oda
2020-01-17 2:27 ` [spp] [PATCH v3 00/12] revive vhost Yasufumi Ogawa
12 siblings, 0 replies; 37+ messages in thread
From: Itsuro Oda @ 2020-01-09 23:10 UTC (permalink / raw)
To: spp, ferruh.yigit, yasufum.o
A vhost device is shared among the primary process and secondary
processes. When a secodary process starts it recognizes vhost
devices if they are used by processes already started. It is not
appropriate to include to port information as PHY devices.
Signed-off-by: Itsuro Oda <oda@valinux.co.jp>
---
src/shared/secondary/spp_worker_th/cmd_utils.c | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/src/shared/secondary/spp_worker_th/cmd_utils.c b/src/shared/secondary/spp_worker_th/cmd_utils.c
index 010a4b6..69d7222 100644
--- a/src/shared/secondary/spp_worker_th/cmd_utils.c
+++ b/src/shared/secondary/spp_worker_th/cmd_utils.c
@@ -437,8 +437,12 @@ init_host_port_info(void)
p_iface_info->phy[port_id].ethdev_port_id = port_id;
break;
case VHOST:
- p_iface_info->vhost[port_id].iface_type = port_type;
- p_iface_info->vhost[port_id].ethdev_port_id = port_id;
+ /* NOTE: a vhost can be used by one process.
+ * even if it exists, it is necessary to do
+ * add_vhost_pmd to setup the device.
+ * note that it is user responsibility to prevent
+ * multipul processes use a vhost at the same time.
+ */
break;
case RING:
p_iface_info->ring[port_id].iface_type = port_type;
--
2.17.1
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [spp] [PATCH v3 00/12] revive vhost
2020-01-09 23:10 ` [spp] [PATCH v3 00/12] revive vhost Itsuro Oda
` (11 preceding siblings ...)
2020-01-09 23:10 ` [spp] [PATCH v3 12/12] spp_vf, spp_mirror: " Itsuro Oda
@ 2020-01-17 2:27 ` Yasufumi Ogawa
12 siblings, 0 replies; 37+ messages in thread
From: Yasufumi Ogawa @ 2020-01-17 2:27 UTC (permalink / raw)
To: Itsuro Oda; +Cc: spp, ferruh.yigit
On 2020/01/10 8:10, Itsuro Oda wrote:
> vhost PMD can not be used by secondary processes since DPDK 18.11.
> SPP project decided to have own vhost PMD which can be used by
> secondary processes at the moment. This vhost PMD is based on the
> original vhost PMD but is simplified very much only to support
> functions used by SPP. Thereby it becomes easy to fix the probrem.
Applied, thanks!
Acked-by: Yasufumi Ogawa <yasufum.o@gmail.com>
>
> v2:
> - devide patches and add tags of commit title.
>
> v3:
> - rebase from DPDK v19.08 to v19.11
>
> Itsuro Oda (12):
> drivers/vhost: add multi process supported vhost PMD for SPP
> drivers: add to build vhost PMD for SPP
> shared: switch to use vhost PMD for SPP
> spp_primary: add link to vhost PMD for SPP
> spp_nfv: add link to vhost PMD for SPP
> spp_vf: add link to vhost PMD for SPP
> spp_mirror: add link to vhost PMD for SPP
> spp_primary: stop vhost before detach
> spp_nfv: stop vhost before detach
> shared: make sure vhost is stopped before (re)using the vhost
> spp_nfv: exclude vhosts at process initialization
> spp_vf,spp_mirror: exclude vhosts at process initialization
>
> src/Makefile | 1 +
> src/drivers/Makefile | 15 +
> src/drivers/vhost/Makefile | 28 +
> .../vhost/rte_pmd_spp_vhost_version.map | 4 +
> src/drivers/vhost/rte_spp_vhost.c | 592 ++++++++++++++++++
> src/mirror/Makefile | 3 +
> src/nfv/Makefile | 3 +
> src/nfv/commands.h | 1 +
> src/nfv/main.c | 7 +-
> src/primary/Makefile | 3 +
> src/primary/main.c | 1 +
> src/shared/common.h | 2 +-
> src/shared/secondary/add_port.c | 8 +
> src/shared/secondary/add_port.h | 2 +-
> .../secondary/spp_worker_th/cmd_utils.c | 8 +-
> src/vf/Makefile | 3 +
> 16 files changed, 676 insertions(+), 5 deletions(-)
> create mode 100644 src/drivers/Makefile
> create mode 100644 src/drivers/vhost/Makefile
> create mode 100644 src/drivers/vhost/rte_pmd_spp_vhost_version.map
> create mode 100644 src/drivers/vhost/rte_spp_vhost.c
>
^ permalink raw reply [flat|nested] 37+ messages in thread
end of thread, other threads:[~2020-01-17 2:26 UTC | newest]
Thread overview: 37+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-12-23 5:00 [spp] [PATCH 0/3] revive vhost Itsuro Oda
2019-12-23 5:00 ` [spp] [PATCH 1/3] multi process supported vhost PMD for SPP Itsuro Oda
2019-12-23 5:00 ` [spp] [PATCH 2/3] make use of " Itsuro Oda
2019-12-23 5:00 ` [spp] [PATCH 3/3] make robust against process start and termination Itsuro Oda
2019-12-24 5:57 ` [spp] [PATCH 0/3] revive vhost Yasufumi Ogawa
2019-12-24 6:09 ` Itsuro ODA
2019-12-24 6:30 ` Yasufumi Ogawa
2019-12-25 4:49 ` [spp] [PATCH v2 00/12] " Itsuro Oda
2019-12-25 4:49 ` [spp] [PATCH v2 01/12] drivers/vhost: add multi process supported vhost PMD for SPP Itsuro Oda
2020-01-07 10:41 ` Yasufumi Ogawa
2020-01-08 1:17 ` Itsuro ODA
2020-01-08 1:23 ` Itsuro ODA
2019-12-25 4:49 ` [spp] [PATCH v2 02/12] drivers: add to build " Itsuro Oda
2019-12-25 4:49 ` [spp] [PATCH v2 03/12] shared: switch to use " Itsuro Oda
2019-12-25 4:49 ` [spp] [PATCH v2 04/12] spp_primary: add link to " Itsuro Oda
2019-12-25 4:49 ` [spp] [PATCH v2 05/12] spp_nfv: " Itsuro Oda
2019-12-25 4:49 ` [spp] [PATCH v2 06/12] spp_vf: " Itsuro Oda
2019-12-25 4:49 ` [spp] [PATCH v2 07/12] spp_mirror: " Itsuro Oda
2019-12-25 4:49 ` [spp] [PATCH v2 08/12] spp_primary: stop vhost before detach Itsuro Oda
2019-12-25 4:49 ` [spp] [PATCH v2 09/12] spp_nfv: " Itsuro Oda
2019-12-25 4:49 ` [spp] [PATCH v2 10/12] shared: make sure vhost is stopped before (re)using the vhost Itsuro Oda
2019-12-25 4:49 ` [spp] [PATCH v2 11/12] spp_nfv: exclude vhosts at process initialization Itsuro Oda
2019-12-25 4:49 ` [spp] [PATCH v2 12/12] spp_vf, spp_mirror: " Itsuro Oda
2020-01-09 23:10 ` [spp] [PATCH v3 00/12] revive vhost Itsuro Oda
2020-01-09 23:10 ` [spp] [PATCH v3 01/12] drivers/vhost: add multi process supported vhost PMD for SPP Itsuro Oda
2020-01-09 23:10 ` [spp] [PATCH v3 02/12] drivers: add to build " Itsuro Oda
2020-01-09 23:10 ` [spp] [PATCH v3 03/12] shared: switch to use " Itsuro Oda
2020-01-09 23:10 ` [spp] [PATCH v3 04/12] spp_primary: add link to " Itsuro Oda
2020-01-09 23:10 ` [spp] [PATCH v3 05/12] spp_nfv: " Itsuro Oda
2020-01-09 23:10 ` [spp] [PATCH v3 06/12] spp_vf: " Itsuro Oda
2020-01-09 23:10 ` [spp] [PATCH v3 07/12] spp_mirror: " Itsuro Oda
2020-01-09 23:10 ` [spp] [PATCH v3 08/12] spp_primary: stop vhost before detach Itsuro Oda
2020-01-09 23:10 ` [spp] [PATCH v3 09/12] spp_nfv: " Itsuro Oda
2020-01-09 23:10 ` [spp] [PATCH v3 10/12] shared: make sure vhost is stopped before (re)using the vhost Itsuro Oda
2020-01-09 23:10 ` [spp] [PATCH v3 11/12] spp_nfv: exclude vhosts at process initialization Itsuro Oda
2020-01-09 23:10 ` [spp] [PATCH v3 12/12] spp_vf, spp_mirror: " Itsuro Oda
2020-01-17 2:27 ` [spp] [PATCH v3 00/12] revive vhost Yasufumi Ogawa
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).