* [dpdk-dev] [PATCH v3 0/2] introduce vdpa sample
@ 2018-09-20 22:28 Xiaolong Ye
2018-09-20 22:28 ` [dpdk-dev] [PATCH v3 1/2] vhost: introduce API to get vDPA device number Xiaolong Ye
` (2 more replies)
0 siblings, 3 replies; 41+ messages in thread
From: Xiaolong Ye @ 2018-09-20 22:28 UTC (permalink / raw)
To: dev, Maxime Coquelin, Tiwei Bie, Zhihong Wang
Cc: xiao.w.wang, Rami Rosen, Wang Haiyue, Xiaolong Ye
Hi,
This patchset introduces vdpa sample to demonstrate the vDPA use case.
v3 changes:
* list cmd would show queue number and supported features of vdpa devices.
* address Xiao's review comments
v2 changes:
* fix a compilation error reported by Rosen
* improve create cmd in interactive mode and add two new cmds: list,
* quit
* add application documentation
Xiaolong Ye (2):
vhost: introduce API to get vDPA device number
examples/vdpa: introduce a new sample for vDPA
MAINTAINERS | 2 +
doc/guides/sample_app_ug/index.rst | 1 +
doc/guides/sample_app_ug/vdpa.rst | 115 +++++++
examples/Makefile | 2 +-
examples/vdpa/Makefile | 32 ++
examples/vdpa/main.c | 458 +++++++++++++++++++++++++
examples/vdpa/meson.build | 16 +
lib/librte_vhost/rte_vdpa.h | 3 +
lib/librte_vhost/rte_vhost_version.map | 1 +
lib/librte_vhost/vdpa.c | 6 +
10 files changed, 635 insertions(+), 1 deletion(-)
create mode 100644 doc/guides/sample_app_ug/vdpa.rst
create mode 100644 examples/vdpa/Makefile
create mode 100644 examples/vdpa/main.c
create mode 100644 examples/vdpa/meson.build
--
2.17.1
^ permalink raw reply [flat|nested] 41+ messages in thread* [dpdk-dev] [PATCH v3 1/2] vhost: introduce API to get vDPA device number 2018-09-20 22:28 [dpdk-dev] [PATCH v3 0/2] introduce vdpa sample Xiaolong Ye @ 2018-09-20 22:28 ` Xiaolong Ye 2018-09-20 22:28 ` [dpdk-dev] [PATCH v3 2/2] examples/vdpa: introduce a new sample for vDPA Xiaolong Ye 2018-09-24 8:42 ` [dpdk-dev] [PATCH v4 0/2] introduce vdpa sample Xiaolong Ye 2 siblings, 0 replies; 41+ messages in thread From: Xiaolong Ye @ 2018-09-20 22:28 UTC (permalink / raw) To: dev, Maxime Coquelin, Tiwei Bie, Zhihong Wang Cc: xiao.w.wang, Rami Rosen, Wang Haiyue, Xiaolong Ye Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com> --- lib/librte_vhost/rte_vdpa.h | 3 +++ lib/librte_vhost/rte_vhost_version.map | 1 + lib/librte_vhost/vdpa.c | 6 ++++++ 3 files changed, 10 insertions(+) diff --git a/lib/librte_vhost/rte_vdpa.h b/lib/librte_vhost/rte_vdpa.h index 90465ca26..b8223e337 100644 --- a/lib/librte_vhost/rte_vdpa.h +++ b/lib/librte_vhost/rte_vdpa.h @@ -84,4 +84,7 @@ rte_vdpa_find_device_id(struct rte_vdpa_dev_addr *addr); struct rte_vdpa_device * __rte_experimental rte_vdpa_get_device(int did); +/* Get current available vdpa device number */ +int __rte_experimental +rte_vdpa_get_device_num(void); #endif /* _RTE_VDPA_H_ */ diff --git a/lib/librte_vhost/rte_vhost_version.map b/lib/librte_vhost/rte_vhost_version.map index da220dd02..ae39b6e21 100644 --- a/lib/librte_vhost/rte_vhost_version.map +++ b/lib/librte_vhost/rte_vhost_version.map @@ -67,6 +67,7 @@ EXPERIMENTAL { rte_vdpa_unregister_device; rte_vdpa_find_device_id; rte_vdpa_get_device; + rte_vdpa_get_device_num; rte_vhost_driver_attach_vdpa_device; rte_vhost_driver_detach_vdpa_device; rte_vhost_driver_get_vdpa_device_id; diff --git a/lib/librte_vhost/vdpa.c b/lib/librte_vhost/vdpa.c index c82fd4370..c2c5dff1d 100644 --- a/lib/librte_vhost/vdpa.c +++ b/lib/librte_vhost/vdpa.c @@ -113,3 +113,9 @@ rte_vdpa_get_device(int did) return vdpa_devices[did]; } + +int +rte_vdpa_get_device_num(void) +{ + return vdpa_device_num; +} -- 2.17.1 ^ permalink raw reply [flat|nested] 41+ messages in thread
* [dpdk-dev] [PATCH v3 2/2] examples/vdpa: introduce a new sample for vDPA 2018-09-20 22:28 [dpdk-dev] [PATCH v3 0/2] introduce vdpa sample Xiaolong Ye 2018-09-20 22:28 ` [dpdk-dev] [PATCH v3 1/2] vhost: introduce API to get vDPA device number Xiaolong Ye @ 2018-09-20 22:28 ` Xiaolong Ye 2018-09-21 5:07 ` Wang, Xiao W 2018-09-24 8:42 ` [dpdk-dev] [PATCH v4 0/2] introduce vdpa sample Xiaolong Ye 2 siblings, 1 reply; 41+ messages in thread From: Xiaolong Ye @ 2018-09-20 22:28 UTC (permalink / raw) To: dev, Maxime Coquelin, Tiwei Bie, Zhihong Wang Cc: xiao.w.wang, Rami Rosen, Wang Haiyue, Xiaolong Ye The vdpa sample application creates vhost-user sockets by using the vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio ring compatible devices to serve virtio driver directly to enable datapath acceleration. As vDPA driver can help to set up vhost datapath, this application doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations. Signed-off-by: Xiao Wang <xiao.w.wang@intel.com> Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com> --- MAINTAINERS | 2 + doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/vdpa.rst | 115 ++++++++ examples/Makefile | 2 +- examples/vdpa/Makefile | 32 ++ examples/vdpa/main.c | 458 +++++++++++++++++++++++++++++ examples/vdpa/meson.build | 16 + 7 files changed, 625 insertions(+), 1 deletion(-) create mode 100644 doc/guides/sample_app_ug/vdpa.rst create mode 100644 examples/vdpa/Makefile create mode 100644 examples/vdpa/main.c create mode 100644 examples/vdpa/meson.build diff --git a/MAINTAINERS b/MAINTAINERS index 5967c1dd3..5656f18e8 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -683,6 +683,8 @@ F: doc/guides/sample_app_ug/vhost.rst F: examples/vhost_scsi/ F: doc/guides/sample_app_ug/vhost_scsi.rst F: examples/vhost_crypto/ +F: examples/vdpa/ +F: doc/guides/sample_app_ug/vdpa.rst Vhost PMD M: Maxime Coquelin <maxime.coquelin@redhat.com> diff --git a/doc/guides/sample_app_ug/index.rst b/doc/guides/sample_app_ug/index.rst index 5bedf4f6f..74b12af85 100644 --- a/doc/guides/sample_app_ug/index.rst +++ b/doc/guides/sample_app_ug/index.rst @@ -45,6 +45,7 @@ Sample Applications User Guides vhost vhost_scsi vhost_crypto + vdpa netmap_compatibility ip_pipeline test_pipeline diff --git a/doc/guides/sample_app_ug/vdpa.rst b/doc/guides/sample_app_ug/vdpa.rst new file mode 100644 index 000000000..44fe6736d --- /dev/null +++ b/doc/guides/sample_app_ug/vdpa.rst @@ -0,0 +1,115 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2018 Intel Corporation. + +Vdpa Sample Application +======================= + +The vdpa sample application creates vhost-user sockets by using the +vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes +virtio ring compatible devices to serve virtio driver directly to enable +datapath acceleration. As vDPA driver can help to set up vhost datapath, +this application doesn't need to launch dedicated worker threads for vhost +enqueue/dequeue operations. + +Testing steps +------------- + +This section shows the steps of how to start VMs with vDPA vhost-user +backend and verify network connection & live migration. + +Build +~~~~~ + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``vdpa`` sub-directory. + +Start the vdpa example +~~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: console + + ./vdpa [EAL options] -- [--interactive|-i] or [--iface SOCKET_PATH] + +where + +* --iface specifies the path prefix of the UNIX domain socket file, e.g. + /tmp/vhost-user-, then the socket files will be named as /tmp/vhost-user-<n> + (n starts from 0). +* --interactive means run the vdpa sample in interactive mode, currently 4 + internal cmds are supported: + + 1. help: show help message + 2. list: list all available vdpa devices + 3. create: create a new vdpa port with socket file and vdpa device address + 4. quit: unregister vhost driver and exit the application + +Take IFCVF driver for example: + +.. code-block:: console + + ./vdpa --log-level=9 -c 0x6 -n 4 --socket-mem 1024,1024 \ + -w 0000:06:00.2,vdpa=1 -w 0000:06:00.3,vdpa=1 \ + -- --interactive + +.. note:: + We need to bind vfio-pci to VFs before running vdpa sample. + + * modprobe vfio-pci + * ./usertools/dpdk-devbind.py -b vfio-pci 06:00.2 06:00.3 + +Then we can create 2 vdpa ports in interactive cmdline. + +.. code-block:: console + + vdpa> list + device id device address + 0 0000:06:00.2 + 1 0000:06:00.3 + vdpa> create /tmp/vdpa-socket0 0000:06:00.2 + vdpa> create /tmp/vdpa-socket1 0000:06:00.3 + +.. _vdpa_app_run_vm: + +Start the VMs +~~~~~~~~~~~~~ + +.. code-block:: console + + qemu-system-x86_64 -cpu host -enable-kvm \ + <snip> + -mem-prealloc \ + -chardev socket,id=char0,path=<socket_file created in above steps> \ + -netdev type=vhost-user,id=vdpa,chardev=char0 \ + -device virtio-net-pci,netdev=vdpa,mac=00:aa:bb:cc:dd:ee \ + +After the VMs launches, we can login the VMs and configure the ip, verify the +network connection via ping or netperf. + +.. note:: + Suggest to use QEMU 3.0.0 which extends vhost-user for vDPA. + +Live Migration +~~~~~~~~~~~~~~ +vDPA supports cross-backend live migration, user can migrate SW vhost backend +VM to vDPA backend VM and vice versa. Here are the detailed steps. Assume A is +the source host with SW vhost VM and B is the destination host with vDPA. + +1. Start vdpa sample and launch a VM with exact same parameters as the VM on A, + in migration-listen mode: + +.. code-block:: console + + B: <qemu-command-line> -incoming tcp:0:4444 (or other PORT)) + +2. Start the migration (on source host): + +.. code-block:: console + + A: (qemu) migrate -d tcp:<B ip>:4444 (or other PORT) + +3. Check the status (on source host): + +.. code-block:: console + + A: (qemu) info migrate diff --git a/examples/Makefile b/examples/Makefile index 481720cb6..356fcb1cd 100644 --- a/examples/Makefile +++ b/examples/Makefile @@ -65,7 +65,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_HASH),y) DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += tep_termination endif DIRS-$(CONFIG_RTE_LIBRTE_TIMER) += timer -DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi +DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi vdpa ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y) DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_crypto endif diff --git a/examples/vdpa/Makefile b/examples/vdpa/Makefile new file mode 100644 index 000000000..42672a2bc --- /dev/null +++ b/examples/vdpa/Makefile @@ -0,0 +1,32 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +ifeq ($(RTE_SDK),) +$(error "Please define RTE_SDK environment variable") +endif + +# Default target, can be overridden by command line or environment +RTE_TARGET ?= x86_64-native-linuxapp-gcc + +include $(RTE_SDK)/mk/rte.vars.mk + +ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp") +$(info This application can only operate in a linuxapp environment, \ +please change the definition of the RTE_TARGET environment variable) +all: +else + +# binary name +APP = vdpa + +# all source are stored in SRCS-y +SRCS-y := main.c + +CFLAGS += -O2 -D_FILE_OFFSET_BITS=64 +CFLAGS += $(WERROR_FLAGS) +CFLAGS += -D_GNU_SOURCE +CFLAGS += -DALLOW_EXPERIMENTAL_API + +include $(RTE_SDK)/mk/rte.extapp.mk + +endif diff --git a/examples/vdpa/main.c b/examples/vdpa/main.c new file mode 100644 index 000000000..b190e67a4 --- /dev/null +++ b/examples/vdpa/main.c @@ -0,0 +1,458 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2018 Intel Corporation + */ + +#include <getopt.h> +#include <signal.h> +#include <stdint.h> +#include <string.h> +#include <unistd.h> + +#include <rte_ethdev.h> +#include <rte_malloc.h> +#include <rte_vhost.h> +#include <rte_vdpa.h> +#include <rte_pci.h> + +#include <cmdline_parse.h> +#include <cmdline_socket.h> +#include <cmdline_parse_string.h> +#include <cmdline.h> + +#define MAX_PATH_LEN 128 +#define MAX_VDPA_SAMPLE_PORTS 1024 +#define RTE_LOGTYPE_VDPA RTE_LOGTYPE_USER1 + +struct vdpa_port { + char ifname[MAX_PATH_LEN]; + int did; + int vid; + uint64_t flags; +}; + +struct vdpa_port vports[MAX_VDPA_SAMPLE_PORTS]; + +static char iface[MAX_PATH_LEN]; +static int dev_total; +static int devcnt; +static int interactive; + +/* display usage */ +static void +vdpa_usage(const char *prgname) +{ + printf("Usage: %s [EAL options] -- " + " --interactive|-i: run in interactive mode.\n" + " --iface <path>: specify the path prefix of the socket files, e.g. /tmp/vhost-user-.\n", + prgname); +} + +static int +parse_args(int argc, char **argv) +{ + static const char *short_option = "i"; + static struct option long_option[] = { + {"iface", required_argument, NULL, 0}, + {"interactive", no_argument, &interactive, 1}, + {NULL, 0, 0, 0}, + }; + int opt, idx; + char *prgname = argv[0]; + + while ((opt = getopt_long(argc, argv, short_option, long_option, &idx)) + != EOF) { + switch (opt) { + case 'i': + printf("Interactive-mode selected\n"); + interactive = 1; + break; + /* long options */ + case 0: + if (strncmp(long_option[idx].name, "iface", + MAX_PATH_LEN) == 0) { + strncpy(iface, optarg, MAX_PATH_LEN); + printf("iface %s\n", iface); + } + if (!strcmp(long_option[idx].name, "interactive")) { + printf("Interactive-mode selected\n"); + interactive = 1; + } + break; + + default: + vdpa_usage(prgname); + return -1; + } + } + + if (iface[0] == '\0' && interactive == 0) { + vdpa_usage(prgname); + return -1; + } + + return 0; +} + +static int +data_init(void) +{ + dev_total = rte_vdpa_get_device_num(); + if (dev_total <= 0) { + printf("No available vdpa device found\n"); + return -1; + } + devcnt = 0; + interactive = 0; + memset(iface, 0, sizeof(iface)); + memset(vports, 0, sizeof(vports)); + return 0; +} + +static int +new_device(int vid) +{ + char ifname[MAX_PATH_LEN]; + int i; + + rte_vhost_get_ifname(vid, ifname, sizeof(ifname)); + for (i = 0; i < MAX_VDPA_SAMPLE_PORTS; i++) { + if (strncmp(ifname, vports[i].ifname, MAX_PATH_LEN) == 0) { + printf("\nnew port %s, did: %d\n", + ifname, vports[i].did); + vports[i].vid = vid; + break; + } + } + + if (i >= MAX_VDPA_SAMPLE_PORTS) + return -1; + + return 0; +} + +static void +destroy_device(int vid) +{ + char ifname[MAX_PATH_LEN]; + int i; + + rte_vhost_get_ifname(vid, ifname, sizeof(ifname)); + for (i = 0; i < MAX_VDPA_SAMPLE_PORTS; i++) { + if (strcmp(ifname, vports[i].ifname) == 0) { + printf("\ndestroy port %s, did: %d\n", + ifname, vports[i].did); + break; + } + } +} + +static const struct vhost_device_ops vdpa_sample_devops = { + .new_device = new_device, + .destroy_device = destroy_device, +}; + +static int +start_vdpa(struct vdpa_port *vport) +{ + int ret; + char *socket_path = vport->ifname; + int did = vport->did; + + if (access(socket_path, F_OK) != -1) { + RTE_LOG(ERR, VDPA, + "%s exists, please remove it or specify another file and try again.\n", + socket_path); + return -1; + } + ret = rte_vhost_driver_register(socket_path, vport->flags); + if (ret != 0) + rte_exit(EXIT_FAILURE, + "register driver failed: %s\n", + socket_path); + + ret = rte_vhost_driver_callback_register(socket_path, + &vdpa_sample_devops); + if (ret != 0) + rte_exit(EXIT_FAILURE, + "register driver ops failed: %s\n", + socket_path); + + ret = rte_vhost_driver_attach_vdpa_device(socket_path, did); + if (ret != 0) + rte_exit(EXIT_FAILURE, + "attach vdpa device failed: %s\n", + socket_path); + + if (rte_vhost_driver_start(socket_path) < 0) + rte_exit(EXIT_FAILURE, + "start vhost driver failed: %s\n", + socket_path); + return 0; +} + +static void +close_vdpa(struct vdpa_port *vport) +{ + int ret; + char *socket_path = vport->ifname; + + ret = rte_vhost_driver_detach_vdpa_device(socket_path); + if (ret != 0) + RTE_LOG(ERR, VDPA, + "attach vdpa device failed: %s\n", + socket_path); + + ret = rte_vhost_driver_unregister(socket_path); + if (ret != 0) + RTE_LOG(ERR, VDPA, + "Fail to unregister vhost driver for %s.\n", + socket_path); +} + +static void +vdpa_sample_quit(void) +{ + int i; + for (i = 0; i < RTE_MIN(MAX_VDPA_SAMPLE_PORTS, dev_total); i++) { + if (vports[i].ifname[0] != '\0') + close_vdpa(&vports[i]); + } +} + +static void +signal_handler(int signum) +{ + if (signum == SIGINT || signum == SIGTERM) { + printf("\nSignal %d received, preparing to exit...\n", signum); + vdpa_sample_quit(); + exit(0); + } +} + +/* interactive cmds */ + +/* *** Help command with introduction. *** */ +struct cmd_help_result { + cmdline_fixed_string_t help; +}; + +static void cmd_help_parsed(__attribute__((unused)) void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + cmdline_printf( + cl, + "\n" + "The following commands are currently available:\n\n" + "Control:\n" + " help : Show interactive instructions.\n" + " list : list all available vdpa devices.\n" + " create <socket file> <vdev addr> : create a new vdpa port.\n" + " quit : exit vdpa sample app.\n" + ); +} + +cmdline_parse_token_string_t cmd_help_help = + TOKEN_STRING_INITIALIZER(struct cmd_help_result, help, "help"); + +cmdline_parse_inst_t cmd_help = { + .f = cmd_help_parsed, + .data = NULL, + .help_str = "show help", + .tokens = { + (void *)&cmd_help_help, + NULL, + }, +}; + +/* *** List all available vdpa devices *** */ +struct cmd_list_result { + cmdline_fixed_string_t action; +}; + +static void cmd_list_vdpa_devices_parsed( + __attribute__((unused)) void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + int did; + uint32_t queue_num; + uint64_t features; + struct rte_vdpa_device *vdev; + struct rte_pci_addr addr; + + cmdline_printf(cl, "device id\tdevice address\tqueue num\tsupported features\n"); + for (did = 0; did < dev_total; did++) { + vdev = rte_vdpa_get_device(did); + if (!vdev) + continue; + if (vdev->ops->get_queue_num(did, &queue_num) < 0) { + RTE_LOG(ERR, VDPA, + "failed to get vdpa queue number " + "for device id %d.\n", did); + continue; + } + if (vdev->ops->get_features(did, &features) < 0) { + RTE_LOG(ERR, VDPA, + "failed to get vdpa features " + "for device id %d.\n", did); + continue; + } + addr = vdev->addr.pci_addr; + cmdline_printf(cl, "%d\t\t"PCI_PRI_FMT"\t%d\t\t0x%lx\n", did, + addr.domain, addr.bus, addr.devid, + addr.function, queue_num, features); + } +} + +cmdline_parse_token_string_t cmd_action_list = + TOKEN_STRING_INITIALIZER(struct cmd_list_result, action, "list"); + +cmdline_parse_inst_t cmd_list_vdpa_devices = { + .f = cmd_list_vdpa_devices_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "list all available vdpa devices", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_action_list, + NULL, + }, +}; + +/* *** Create new vdpa port *** */ +struct cmd_create_result { + cmdline_fixed_string_t action; + cmdline_fixed_string_t socket_path; + cmdline_fixed_string_t bdf; +}; + +static void cmd_create_vdpa_port_parsed(void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + int did; + struct cmd_create_result *res = parsed_result; + struct rte_vdpa_dev_addr addr; + + strncpy(vports[devcnt].ifname, res->socket_path, MAX_PATH_LEN); + if (rte_pci_addr_parse(res->bdf, &addr.pci_addr) != 0) { + cmdline_printf(cl, "Unable to parse the given bdf.\n"); + return; + } + addr.type = PCI_ADDR; + did = rte_vdpa_find_device_id(&addr); + if (did < 0) { + cmdline_printf(cl, "Unable to find vdpa devide id.\n"); + return; + } + + vports[devcnt].did = did; + + if (start_vdpa(&vports[devcnt]) == 0) + devcnt++; +} + +cmdline_parse_token_string_t cmd_action_create = + TOKEN_STRING_INITIALIZER(struct cmd_create_result, action, "create"); +cmdline_parse_token_string_t cmd_socket_path = + TOKEN_STRING_INITIALIZER(struct cmd_create_result, socket_path, NULL); +cmdline_parse_token_string_t cmd_bdf = + TOKEN_STRING_INITIALIZER(struct cmd_create_result, bdf, NULL); + +cmdline_parse_inst_t cmd_create_vdpa_port = { + .f = cmd_create_vdpa_port_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "create a new vdpa port", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_action_create, + (void *)&cmd_socket_path, + (void *)&cmd_bdf, + NULL, + }, +}; + +/* *** QUIT *** */ +struct cmd_quit_result { + cmdline_fixed_string_t quit; +}; + +static void cmd_quit_parsed(__attribute__((unused)) void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + vdpa_sample_quit(); + cmdline_quit(cl); +} + +cmdline_parse_token_string_t cmd_quit_quit = + TOKEN_STRING_INITIALIZER(struct cmd_quit_result, quit, "quit"); + +cmdline_parse_inst_t cmd_quit = { + .f = cmd_quit_parsed, + .data = NULL, + .help_str = "quit: exit application", + .tokens = { + (void *)&cmd_quit_quit, + NULL, + }, +}; +cmdline_parse_ctx_t main_ctx[] = { + (cmdline_parse_inst_t *)&cmd_help, + (cmdline_parse_inst_t *)&cmd_list_vdpa_devices, + (cmdline_parse_inst_t *)&cmd_create_vdpa_port, + (cmdline_parse_inst_t *)&cmd_quit, + NULL, +}; + +int +main(int argc, char *argv[]) +{ + char ch; + int i; + int ret; + struct cmdline *cl; + + ret = rte_eal_init(argc, argv); + if (ret < 0) + rte_exit(EXIT_FAILURE, "eal init failed\n"); + argc -= ret; + argv += ret; + + data_init(); + + signal(SIGINT, signal_handler); + signal(SIGTERM, signal_handler); + + ret = parse_args(argc, argv); + if (ret < 0) + rte_exit(EXIT_FAILURE, "invalid argument\n"); + + if (interactive == 1) { + cl = cmdline_stdin_new(main_ctx, "vdpa> "); + if (cl == NULL) + rte_panic("Cannot create cmdline instance\n"); + cmdline_interact(cl); + cmdline_stdin_exit(cl); + } else { + for (i = 0; i < RTE_MIN(MAX_VDPA_SAMPLE_PORTS, dev_total); + i++) { + vports[i].did = i; + snprintf(vports[i].ifname, MAX_PATH_LEN, "%s%d", + iface, i); + + start_vdpa(&vports[i]); + } + + printf("enter \'q\' to quit\n"); + while (scanf("%c", &ch)) { + if (ch == 'q') + break; + while (ch != '\n') { + if (scanf("%c", &ch)) + printf("%c", ch); + } + printf("enter \'q\' to quit\n"); + } + } + + return 0; +} diff --git a/examples/vdpa/meson.build b/examples/vdpa/meson.build new file mode 100644 index 000000000..2e38a069f --- /dev/null +++ b/examples/vdpa/meson.build @@ -0,0 +1,16 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +# meson file, for building this example as part of a main DPDK build. +# +# To build this example as a standalone application with an already-installed +# DPDK instance, use 'make' + +if host_machine.system() != 'linux' + build = false +endif +deps += 'vhost' +allow_experimental_apis = true +sources = files( + 'main.c' +) \ No newline at end of file -- 2.17.1 ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [dpdk-dev] [PATCH v3 2/2] examples/vdpa: introduce a new sample for vDPA 2018-09-20 22:28 ` [dpdk-dev] [PATCH v3 2/2] examples/vdpa: introduce a new sample for vDPA Xiaolong Ye @ 2018-09-21 5:07 ` Wang, Xiao W 2018-09-22 11:04 ` Ye Xiaolong 0 siblings, 1 reply; 41+ messages in thread From: Wang, Xiao W @ 2018-09-21 5:07 UTC (permalink / raw) To: Ye, Xiaolong, dev, Maxime Coquelin, Bie, Tiwei, Wang, Zhihong Cc: Rami Rosen, Wang, Haiyue Hi Xiaolong, > -----Original Message----- > From: Ye, Xiaolong > Sent: Friday, September 21, 2018 6:28 AM > To: dev@dpdk.org; Maxime Coquelin <maxime.coquelin@redhat.com>; Bie, > Tiwei <tiwei.bie@intel.com>; Wang, Zhihong <zhihong.wang@intel.com> > Cc: Wang, Xiao W <xiao.w.wang@intel.com>; Rami Rosen > <roszenrami@gmail.com>; Wang, Haiyue <haiyue.wang@intel.com>; Ye, > Xiaolong <xiaolong.ye@intel.com> > Subject: [PATCH v3 2/2] examples/vdpa: introduce a new sample for vDPA > > The vdpa sample application creates vhost-user sockets by using the > vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes > virtio ring compatible devices to serve virtio driver directly to enable > datapath acceleration. As vDPA driver can help to set up vhost datapath, > this application doesn't need to launch dedicated worker threads for vhost > enqueue/dequeue operations. > > Signed-off-by: Xiao Wang <xiao.w.wang@intel.com> > Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com> > --- > MAINTAINERS | 2 + > doc/guides/sample_app_ug/index.rst | 1 + > doc/guides/sample_app_ug/vdpa.rst | 115 ++++++++ > examples/Makefile | 2 +- > examples/vdpa/Makefile | 32 ++ > examples/vdpa/main.c | 458 +++++++++++++++++++++++++++++ > examples/vdpa/meson.build | 16 + > 7 files changed, 625 insertions(+), 1 deletion(-) > create mode 100644 doc/guides/sample_app_ug/vdpa.rst > create mode 100644 examples/vdpa/Makefile > create mode 100644 examples/vdpa/main.c > create mode 100644 examples/vdpa/meson.build > > diff --git a/MAINTAINERS b/MAINTAINERS > index 5967c1dd3..5656f18e8 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -683,6 +683,8 @@ F: doc/guides/sample_app_ug/vhost.rst > F: examples/vhost_scsi/ > F: doc/guides/sample_app_ug/vhost_scsi.rst > F: examples/vhost_crypto/ > +F: examples/vdpa/ > +F: doc/guides/sample_app_ug/vdpa.rst > > Vhost PMD > M: Maxime Coquelin <maxime.coquelin@redhat.com> > diff --git a/doc/guides/sample_app_ug/index.rst > b/doc/guides/sample_app_ug/index.rst > index 5bedf4f6f..74b12af85 100644 > --- a/doc/guides/sample_app_ug/index.rst > +++ b/doc/guides/sample_app_ug/index.rst > @@ -45,6 +45,7 @@ Sample Applications User Guides > vhost > vhost_scsi > vhost_crypto > + vdpa > netmap_compatibility > ip_pipeline > test_pipeline > diff --git a/doc/guides/sample_app_ug/vdpa.rst > b/doc/guides/sample_app_ug/vdpa.rst > new file mode 100644 > index 000000000..44fe6736d > --- /dev/null > +++ b/doc/guides/sample_app_ug/vdpa.rst > @@ -0,0 +1,115 @@ > +.. SPDX-License-Identifier: BSD-3-Clause > + Copyright(c) 2018 Intel Corporation. > + > +Vdpa Sample Application > +======================= > + > +The vdpa sample application creates vhost-user sockets by using the > +vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes > +virtio ring compatible devices to serve virtio driver directly to enable > +datapath acceleration. As vDPA driver can help to set up vhost datapath, > +this application doesn't need to launch dedicated worker threads for vhost > +enqueue/dequeue operations. > + > +Testing steps [...] > + > +Then we can create 2 vdpa ports in interactive cmdline. > + > +.. code-block:: console > + > + vdpa> list > + device id device address > + 0 0000:06:00.2 > + 1 0000:06:00.3 The features and queue numbers of each vDPA device could also be shown now, the doc should reflect this. > + vdpa> create /tmp/vdpa-socket0 0000:06:00.2 > + vdpa> create /tmp/vdpa-socket1 0000:06:00.3 > + > +.. _vdpa_app_run_vm: > + > +Start the VMs > +~~~~~~~~~~~~~ > + > +.. code-block:: console > + > + qemu-system-x86_64 -cpu host -enable-kvm \ > + <snip> > + -mem-prealloc \ > + -chardev socket,id=char0,path=<socket_file created in above steps> \ > + -netdev type=vhost-user,id=vdpa,chardev=char0 \ > + -device virtio-net-pci,netdev=vdpa,mac=00:aa:bb:cc:dd:ee \ > + > +After the VMs launches, we can login the VMs and configure the ip, verify the > +network connection via ping or netperf. > + > +.. note:: > + Suggest to use QEMU 3.0.0 which extends vhost-user for vDPA. > + > +Live Migration > +~~~~~~~~~~~~~~ > +vDPA supports cross-backend live migration, user can migrate SW vhost > backend > +VM to vDPA backend VM and vice versa. Here are the detailed steps. Assume > A is > +the source host with SW vhost VM and B is the destination host with vDPA. > + > +1. Start vdpa sample and launch a VM with exact same parameters as the VM > on A, > + in migration-listen mode: > + > +.. code-block:: console > + > + B: <qemu-command-line> -incoming tcp:0:4444 (or other PORT)) > + > +2. Start the migration (on source host): > + > +.. code-block:: console > + > + A: (qemu) migrate -d tcp:<B ip>:4444 (or other PORT) > + > +3. Check the status (on source host): > + > +.. code-block:: console > + > + A: (qemu) info migrate > diff --git a/examples/Makefile b/examples/Makefile > index 481720cb6..356fcb1cd 100644 > --- a/examples/Makefile > +++ b/examples/Makefile > @@ -65,7 +65,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_HASH),y) > DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += tep_termination > endif > DIRS-$(CONFIG_RTE_LIBRTE_TIMER) += timer > -DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi > +DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi vdpa > ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y) > DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_crypto > endif > diff --git a/examples/vdpa/Makefile b/examples/vdpa/Makefile > new file mode 100644 > index 000000000..42672a2bc > --- /dev/null > +++ b/examples/vdpa/Makefile > @@ -0,0 +1,32 @@ > +# SPDX-License-Identifier: BSD-3-Clause > +# Copyright(c) 2018 Intel Corporation > + > +ifeq ($(RTE_SDK),) > +$(error "Please define RTE_SDK environment variable") > +endif > + > +# Default target, can be overridden by command line or environment > +RTE_TARGET ?= x86_64-native-linuxapp-gcc > + > +include $(RTE_SDK)/mk/rte.vars.mk > + > +ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp") > +$(info This application can only operate in a linuxapp environment, \ > +please change the definition of the RTE_TARGET environment variable) > +all: > +else > + > +# binary name > +APP = vdpa > + > +# all source are stored in SRCS-y > +SRCS-y := main.c > + > +CFLAGS += -O2 -D_FILE_OFFSET_BITS=64 > +CFLAGS += $(WERROR_FLAGS) > +CFLAGS += -D_GNU_SOURCE > +CFLAGS += -DALLOW_EXPERIMENTAL_API > + > +include $(RTE_SDK)/mk/rte.extapp.mk > + [...] > + > +static int > +new_device(int vid) > +{ > + char ifname[MAX_PATH_LEN]; > + int i; > + > + rte_vhost_get_ifname(vid, ifname, sizeof(ifname)); > + for (i = 0; i < MAX_VDPA_SAMPLE_PORTS; i++) { > + if (strncmp(ifname, vports[i].ifname, MAX_PATH_LEN) == 0) { > + printf("\nnew port %s, did: %d\n", > + ifname, vports[i].did); > + vports[i].vid = vid; > + break; > + } > + } > + > + if (i >= MAX_VDPA_SAMPLE_PORTS) > + return -1; > + > + return 0; > +} > + > +static void > +destroy_device(int vid) > +{ > + char ifname[MAX_PATH_LEN]; > + int i; > + > + rte_vhost_get_ifname(vid, ifname, sizeof(ifname)); > + for (i = 0; i < MAX_VDPA_SAMPLE_PORTS; i++) { > + if (strcmp(ifname, vports[i].ifname) == 0) { > + printf("\ndestroy port %s, did: %d\n", > + ifname, vports[i].did); > + break; > + } > + } > +} > + > +static const struct vhost_device_ops vdpa_sample_devops = { > + .new_device = new_device, > + .destroy_device = destroy_device, > +}; > + > +static int > +start_vdpa(struct vdpa_port *vport) > +{ > + int ret; > + char *socket_path = vport->ifname; > + int did = vport->did; > + > + if (access(socket_path, F_OK) != -1) { > + RTE_LOG(ERR, VDPA, > + "%s exists, please remove it or specify another file and > try again.\n", > + socket_path); > + return -1; > + } > + ret = rte_vhost_driver_register(socket_path, vport->flags); It would be great if we can provide "client mode" option for this sample to create a client mode vdpa. > + if (ret != 0) > + rte_exit(EXIT_FAILURE, > + "register driver failed: %s\n", > + socket_path); > + > + ret = rte_vhost_driver_callback_register(socket_path, > + &vdpa_sample_devops); > + if (ret != 0) > + rte_exit(EXIT_FAILURE, > + "register driver ops failed: %s\n", > + socket_path); > + > + ret = rte_vhost_driver_attach_vdpa_device(socket_path, did); > + if (ret != 0) > + rte_exit(EXIT_FAILURE, > + "attach vdpa device failed: %s\n", > + socket_path); > + > + if (rte_vhost_driver_start(socket_path) < 0) > + rte_exit(EXIT_FAILURE, > + "start vhost driver failed: %s\n", > + socket_path); > + return 0; > +} > + > +static void > +close_vdpa(struct vdpa_port *vport) > +{ > + int ret; > + char *socket_path = vport->ifname; > + > + ret = rte_vhost_driver_detach_vdpa_device(socket_path); > + if (ret != 0) > + RTE_LOG(ERR, VDPA, > + "attach vdpa device failed: %s\n", > + socket_path); > + > + ret = rte_vhost_driver_unregister(socket_path); > + if (ret != 0) > + RTE_LOG(ERR, VDPA, > + "Fail to unregister vhost driver for %s.\n", > + socket_path); > +} > + > +static void > +vdpa_sample_quit(void) > +{ > + int i; > + for (i = 0; i < RTE_MIN(MAX_VDPA_SAMPLE_PORTS, dev_total); i++) { > + if (vports[i].ifname[0] != '\0') > + close_vdpa(&vports[i]); > + } > +} > + > +static void > +signal_handler(int signum) > +{ > + if (signum == SIGINT || signum == SIGTERM) { > + printf("\nSignal %d received, preparing to exit...\n", signum); > + vdpa_sample_quit(); > + exit(0); > + } > +} > + > +/* interactive cmds */ > + > +/* *** Help command with introduction. *** */ > +struct cmd_help_result { > + cmdline_fixed_string_t help; > +}; > + > +static void cmd_help_parsed(__attribute__((unused)) void *parsed_result, > + struct cmdline *cl, > + __attribute__((unused)) void *data) > +{ > + cmdline_printf( > + cl, > + "\n" > + "The following commands are currently available:\n\n" > + "Control:\n" > + " help : Show interactive instructions.\n" > + " list : list all available vdpa devices.\n" > + " create <socket file> <vdev addr> : create a new vdpa > port.\n" > + " quit : exit vdpa sample app.\n" > + ); > +} > + > +cmdline_parse_token_string_t cmd_help_help = > + TOKEN_STRING_INITIALIZER(struct cmd_help_result, help, "help"); > + > +cmdline_parse_inst_t cmd_help = { > + .f = cmd_help_parsed, > + .data = NULL, > + .help_str = "show help", > + .tokens = { > + (void *)&cmd_help_help, > + NULL, > + }, > +}; > + > +/* *** List all available vdpa devices *** */ > +struct cmd_list_result { > + cmdline_fixed_string_t action; > +}; > + > +static void cmd_list_vdpa_devices_parsed( > + __attribute__((unused)) void *parsed_result, > + struct cmdline *cl, > + __attribute__((unused)) void *data) > +{ > + int did; > + uint32_t queue_num; > + uint64_t features; > + struct rte_vdpa_device *vdev; > + struct rte_pci_addr addr; > + > + cmdline_printf(cl, "device id\tdevice address\tqueue num\tsupported > features\n"); > + for (did = 0; did < dev_total; did++) { > + vdev = rte_vdpa_get_device(did); > + if (!vdev) > + continue; > + if (vdev->ops->get_queue_num(did, &queue_num) < 0) { > + RTE_LOG(ERR, VDPA, > + "failed to get vdpa queue number " > + "for device id %d.\n", did); > + continue; > + } > + if (vdev->ops->get_features(did, &features) < 0) { > + RTE_LOG(ERR, VDPA, > + "failed to get vdpa features " > + "for device id %d.\n", did); > + continue; > + } > + addr = vdev->addr.pci_addr; > + cmdline_printf(cl, "%d\t\t"PCI_PRI_FMT"\t%d\t\t0x%lx\n", did, > + addr.domain, addr.bus, addr.devid, > + addr.function, queue_num, features); Use "%PRIu32"for queue_num, and use "PRIx64" instead of "%lx", otherwise you may have compile error on 32-bit machine. > + } > +} > + > +cmdline_parse_token_string_t cmd_action_list = > + TOKEN_STRING_INITIALIZER(struct cmd_list_result, action, "list"); > + > +cmdline_parse_inst_t cmd_list_vdpa_devices = { > + .f = cmd_list_vdpa_devices_parsed, /* function to call */ > + .data = NULL, /* 2nd arg of func */ This comment doesn't provide much info. Remove the comment for all the cmdline_parse_inst_t instances to keep alignment. > + .help_str = "list all available vdpa devices", > + .tokens = { /* token list, NULL terminated */ > + (void *)&cmd_action_list, > + NULL, > + }, > +}; > + > +/* *** Create new vdpa port *** */ > +struct cmd_create_result { > + cmdline_fixed_string_t action; > + cmdline_fixed_string_t socket_path; > + cmdline_fixed_string_t bdf; > +}; > + > +static void cmd_create_vdpa_port_parsed(void *parsed_result, > + struct cmdline *cl, > + __attribute__((unused)) void *data) > +{ > + int did; > + struct cmd_create_result *res = parsed_result; > + struct rte_vdpa_dev_addr addr; > + > + strncpy(vports[devcnt].ifname, res->socket_path, MAX_PATH_LEN); > + if (rte_pci_addr_parse(res->bdf, &addr.pci_addr) != 0) { > + cmdline_printf(cl, "Unable to parse the given bdf.\n"); > + return; > + } > + addr.type = PCI_ADDR; > + did = rte_vdpa_find_device_id(&addr); > + if (did < 0) { > + cmdline_printf(cl, "Unable to find vdpa devide id.\n"); > + return; > + } > + > + vports[devcnt].did = did; > + > + if (start_vdpa(&vports[devcnt]) == 0) > + devcnt++; > +} > + > +cmdline_parse_token_string_t cmd_action_create = > + TOKEN_STRING_INITIALIZER(struct cmd_create_result, action, > "create"); > +cmdline_parse_token_string_t cmd_socket_path = > + TOKEN_STRING_INITIALIZER(struct cmd_create_result, socket_path, > NULL); > +cmdline_parse_token_string_t cmd_bdf = > + TOKEN_STRING_INITIALIZER(struct cmd_create_result, bdf, NULL); > + > +cmdline_parse_inst_t cmd_create_vdpa_port = { > + .f = cmd_create_vdpa_port_parsed, /* function to call */ > + .data = NULL, /* 2nd arg of func */ > + .help_str = "create a new vdpa port", > + .tokens = { /* token list, NULL terminated */ > + (void *)&cmd_action_create, > + (void *)&cmd_socket_path, > + (void *)&cmd_bdf, > + NULL, > + }, > +}; > + > +/* *** QUIT *** */ > +struct cmd_quit_result { > + cmdline_fixed_string_t quit; > +}; > + > +static void cmd_quit_parsed(__attribute__((unused)) void *parsed_result, > + struct cmdline *cl, > + __attribute__((unused)) void *data) > +{ > + vdpa_sample_quit(); > + cmdline_quit(cl); > +} > + > +cmdline_parse_token_string_t cmd_quit_quit = > + TOKEN_STRING_INITIALIZER(struct cmd_quit_result, quit, "quit"); > + > +cmdline_parse_inst_t cmd_quit = { > + .f = cmd_quit_parsed, > + .data = NULL, > + .help_str = "quit: exit application", > + .tokens = { > + (void *)&cmd_quit_quit, > + NULL, > + }, > +}; > +cmdline_parse_ctx_t main_ctx[] = { > + (cmdline_parse_inst_t *)&cmd_help, > + (cmdline_parse_inst_t *)&cmd_list_vdpa_devices, > + (cmdline_parse_inst_t *)&cmd_create_vdpa_port, > + (cmdline_parse_inst_t *)&cmd_quit, > + NULL, > +}; > + > +int > +main(int argc, char *argv[]) > +{ > + char ch; > + int i; > + int ret; > + struct cmdline *cl; > + > + ret = rte_eal_init(argc, argv); > + if (ret < 0) > + rte_exit(EXIT_FAILURE, "eal init failed\n"); > + argc -= ret; > + argv += ret; > + > + data_init(); > + > + signal(SIGINT, signal_handler); > + signal(SIGTERM, signal_handler); > + > + ret = parse_args(argc, argv); > + if (ret < 0) > + rte_exit(EXIT_FAILURE, "invalid argument\n"); > + > + if (interactive == 1) { > + cl = cmdline_stdin_new(main_ctx, "vdpa> "); > + if (cl == NULL) > + rte_panic("Cannot create cmdline instance\n"); > + cmdline_interact(cl); > + cmdline_stdin_exit(cl); > + } else { > + for (i = 0; i < RTE_MIN(MAX_VDPA_SAMPLE_PORTS, dev_total); > + i++) { > + vports[i].did = i; > + snprintf(vports[i].ifname, MAX_PATH_LEN, "%s%d", > + iface, i); > + > + start_vdpa(&vports[i]); > + } > + > + printf("enter \'q\' to quit\n"); > + while (scanf("%c", &ch)) { > + if (ch == 'q') > + break; > + while (ch != '\n') { > + if (scanf("%c", &ch)) > + printf("%c", ch); > + } > + printf("enter \'q\' to quit\n"); > + } Do vdpa_sample_quit here to stop the vhost session and delete the vhost sockets. > + } > + > + return 0; > +} > diff --git a/examples/vdpa/meson.build b/examples/vdpa/meson.build > new file mode 100644 > index 000000000..2e38a069f > --- /dev/null > +++ b/examples/vdpa/meson.build > @@ -0,0 +1,16 @@ > +# SPDX-License-Identifier: BSD-3-Clause > +# Copyright(c) 2018 Intel Corporation > + > +# meson file, for building this example as part of a main DPDK build. > +# > +# To build this example as a standalone application with an already-installed > +# DPDK instance, use 'make' > + > +if host_machine.system() != 'linux' > + build = false > +endif > +deps += 'vhost' > +allow_experimental_apis = true > +sources = files( > + 'main.c' > +) > \ No newline at end of file > -- > 2.17.1 Do not forget to add this into 18.11 release_note. BRs, Xiao ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [dpdk-dev] [PATCH v3 2/2] examples/vdpa: introduce a new sample for vDPA 2018-09-21 5:07 ` Wang, Xiao W @ 2018-09-22 11:04 ` Ye Xiaolong 0 siblings, 0 replies; 41+ messages in thread From: Ye Xiaolong @ 2018-09-22 11:04 UTC (permalink / raw) To: Wang, Xiao W Cc: dev, Maxime Coquelin, Bie, Tiwei, Wang, Zhihong, Rami Rosen, Wang, Haiyue On 09/21, Wang, Xiao W wrote: >Hi Xiaolong, > >> -----Original Message----- >> From: Ye, Xiaolong >> Sent: Friday, September 21, 2018 6:28 AM >> To: dev@dpdk.org; Maxime Coquelin <maxime.coquelin@redhat.com>; Bie, >> Tiwei <tiwei.bie@intel.com>; Wang, Zhihong <zhihong.wang@intel.com> >> Cc: Wang, Xiao W <xiao.w.wang@intel.com>; Rami Rosen >> <roszenrami@gmail.com>; Wang, Haiyue <haiyue.wang@intel.com>; Ye, >> Xiaolong <xiaolong.ye@intel.com> >> Subject: [PATCH v3 2/2] examples/vdpa: introduce a new sample for vDPA >> >> The vdpa sample application creates vhost-user sockets by using the >> vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes >> virtio ring compatible devices to serve virtio driver directly to enable >> datapath acceleration. As vDPA driver can help to set up vhost datapath, >> this application doesn't need to launch dedicated worker threads for vhost >> enqueue/dequeue operations. >> >> Signed-off-by: Xiao Wang <xiao.w.wang@intel.com> >> Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com> >> --- >> MAINTAINERS | 2 + >> doc/guides/sample_app_ug/index.rst | 1 + >> doc/guides/sample_app_ug/vdpa.rst | 115 ++++++++ >> examples/Makefile | 2 +- >> examples/vdpa/Makefile | 32 ++ >> examples/vdpa/main.c | 458 +++++++++++++++++++++++++++++ >> examples/vdpa/meson.build | 16 + >> 7 files changed, 625 insertions(+), 1 deletion(-) >> create mode 100644 doc/guides/sample_app_ug/vdpa.rst >> create mode 100644 examples/vdpa/Makefile >> create mode 100644 examples/vdpa/main.c >> create mode 100644 examples/vdpa/meson.build >> >> diff --git a/MAINTAINERS b/MAINTAINERS >> index 5967c1dd3..5656f18e8 100644 >> --- a/MAINTAINERS >> +++ b/MAINTAINERS >> @@ -683,6 +683,8 @@ F: doc/guides/sample_app_ug/vhost.rst >> F: examples/vhost_scsi/ >> F: doc/guides/sample_app_ug/vhost_scsi.rst >> F: examples/vhost_crypto/ >> +F: examples/vdpa/ >> +F: doc/guides/sample_app_ug/vdpa.rst >> >> Vhost PMD >> M: Maxime Coquelin <maxime.coquelin@redhat.com> >> diff --git a/doc/guides/sample_app_ug/index.rst >> b/doc/guides/sample_app_ug/index.rst >> index 5bedf4f6f..74b12af85 100644 >> --- a/doc/guides/sample_app_ug/index.rst >> +++ b/doc/guides/sample_app_ug/index.rst >> @@ -45,6 +45,7 @@ Sample Applications User Guides >> vhost >> vhost_scsi >> vhost_crypto >> + vdpa >> netmap_compatibility >> ip_pipeline >> test_pipeline >> diff --git a/doc/guides/sample_app_ug/vdpa.rst >> b/doc/guides/sample_app_ug/vdpa.rst >> new file mode 100644 >> index 000000000..44fe6736d >> --- /dev/null >> +++ b/doc/guides/sample_app_ug/vdpa.rst >> @@ -0,0 +1,115 @@ >> +.. SPDX-License-Identifier: BSD-3-Clause >> + Copyright(c) 2018 Intel Corporation. >> + >> +Vdpa Sample Application >> +======================= >> + >> +The vdpa sample application creates vhost-user sockets by using the >> +vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes >> +virtio ring compatible devices to serve virtio driver directly to enable >> +datapath acceleration. As vDPA driver can help to set up vhost datapath, >> +this application doesn't need to launch dedicated worker threads for vhost >> +enqueue/dequeue operations. >> + >> +Testing steps > >[...] > >> + >> +Then we can create 2 vdpa ports in interactive cmdline. >> + >> +.. code-block:: console >> + >> + vdpa> list >> + device id device address >> + 0 0000:06:00.2 >> + 1 0000:06:00.3 > >The features and queue numbers of each vDPA device could also be shown now, the doc should reflect this. Got it. > >> + vdpa> create /tmp/vdpa-socket0 0000:06:00.2 >> + vdpa> create /tmp/vdpa-socket1 0000:06:00.3 >> + >> +.. _vdpa_app_run_vm: >> + >> +Start the VMs >> +~~~~~~~~~~~~~ >> + >> +.. code-block:: console >> + >> + qemu-system-x86_64 -cpu host -enable-kvm \ >> + <snip> >> + -mem-prealloc \ >> + -chardev socket,id=char0,path=<socket_file created in above steps> \ >> + -netdev type=vhost-user,id=vdpa,chardev=char0 \ >> + -device virtio-net-pci,netdev=vdpa,mac=00:aa:bb:cc:dd:ee \ >> + >> +After the VMs launches, we can login the VMs and configure the ip, verify the >> +network connection via ping or netperf. >> + >> +.. note:: >> + Suggest to use QEMU 3.0.0 which extends vhost-user for vDPA. >> + >> +Live Migration >> +~~~~~~~~~~~~~~ >> +vDPA supports cross-backend live migration, user can migrate SW vhost >> backend >> +VM to vDPA backend VM and vice versa. Here are the detailed steps. Assume >> A is >> +the source host with SW vhost VM and B is the destination host with vDPA. >> + >> +1. Start vdpa sample and launch a VM with exact same parameters as the VM >> on A, >> + in migration-listen mode: >> + >> +.. code-block:: console >> + >> + B: <qemu-command-line> -incoming tcp:0:4444 (or other PORT)) >> + >> +2. Start the migration (on source host): >> + >> +.. code-block:: console >> + >> + A: (qemu) migrate -d tcp:<B ip>:4444 (or other PORT) >> + >> +3. Check the status (on source host): >> + >> +.. code-block:: console >> + >> + A: (qemu) info migrate >> diff --git a/examples/Makefile b/examples/Makefile >> index 481720cb6..356fcb1cd 100644 >> --- a/examples/Makefile >> +++ b/examples/Makefile >> @@ -65,7 +65,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_HASH),y) >> DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += tep_termination >> endif >> DIRS-$(CONFIG_RTE_LIBRTE_TIMER) += timer >> -DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi >> +DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi vdpa >> ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y) >> DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_crypto >> endif >> diff --git a/examples/vdpa/Makefile b/examples/vdpa/Makefile >> new file mode 100644 >> index 000000000..42672a2bc >> --- /dev/null >> +++ b/examples/vdpa/Makefile >> @@ -0,0 +1,32 @@ >> +# SPDX-License-Identifier: BSD-3-Clause >> +# Copyright(c) 2018 Intel Corporation >> + >> +ifeq ($(RTE_SDK),) >> +$(error "Please define RTE_SDK environment variable") >> +endif >> + >> +# Default target, can be overridden by command line or environment >> +RTE_TARGET ?= x86_64-native-linuxapp-gcc >> + >> +include $(RTE_SDK)/mk/rte.vars.mk >> + >> +ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp") >> +$(info This application can only operate in a linuxapp environment, \ >> +please change the definition of the RTE_TARGET environment variable) >> +all: >> +else >> + >> +# binary name >> +APP = vdpa >> + >> +# all source are stored in SRCS-y >> +SRCS-y := main.c >> + >> +CFLAGS += -O2 -D_FILE_OFFSET_BITS=64 >> +CFLAGS += $(WERROR_FLAGS) >> +CFLAGS += -D_GNU_SOURCE >> +CFLAGS += -DALLOW_EXPERIMENTAL_API >> + >> +include $(RTE_SDK)/mk/rte.extapp.mk >> + > >[...] > >> + >> +static int >> +new_device(int vid) >> +{ >> + char ifname[MAX_PATH_LEN]; >> + int i; >> + >> + rte_vhost_get_ifname(vid, ifname, sizeof(ifname)); >> + for (i = 0; i < MAX_VDPA_SAMPLE_PORTS; i++) { >> + if (strncmp(ifname, vports[i].ifname, MAX_PATH_LEN) == 0) { >> + printf("\nnew port %s, did: %d\n", >> + ifname, vports[i].did); >> + vports[i].vid = vid; >> + break; >> + } >> + } >> + >> + if (i >= MAX_VDPA_SAMPLE_PORTS) >> + return -1; >> + >> + return 0; >> +} >> + >> +static void >> +destroy_device(int vid) >> +{ >> + char ifname[MAX_PATH_LEN]; >> + int i; >> + >> + rte_vhost_get_ifname(vid, ifname, sizeof(ifname)); >> + for (i = 0; i < MAX_VDPA_SAMPLE_PORTS; i++) { >> + if (strcmp(ifname, vports[i].ifname) == 0) { >> + printf("\ndestroy port %s, did: %d\n", >> + ifname, vports[i].did); >> + break; >> + } >> + } >> +} >> + >> +static const struct vhost_device_ops vdpa_sample_devops = { >> + .new_device = new_device, >> + .destroy_device = destroy_device, >> +}; >> + >> +static int >> +start_vdpa(struct vdpa_port *vport) >> +{ >> + int ret; >> + char *socket_path = vport->ifname; >> + int did = vport->did; >> + >> + if (access(socket_path, F_OK) != -1) { >> + RTE_LOG(ERR, VDPA, >> + "%s exists, please remove it or specify another file and >> try again.\n", >> + socket_path); >> + return -1; >> + } >> + ret = rte_vhost_driver_register(socket_path, vport->flags); > >It would be great if we can provide "client mode" option for this sample to create a client mode vdpa. Sure, I'll add client mode support. > >> + if (ret != 0) >> + rte_exit(EXIT_FAILURE, >> + "register driver failed: %s\n", >> + socket_path); >> + >> + ret = rte_vhost_driver_callback_register(socket_path, >> + &vdpa_sample_devops); >> + if (ret != 0) >> + rte_exit(EXIT_FAILURE, >> + "register driver ops failed: %s\n", >> + socket_path); >> + >> + ret = rte_vhost_driver_attach_vdpa_device(socket_path, did); >> + if (ret != 0) >> + rte_exit(EXIT_FAILURE, >> + "attach vdpa device failed: %s\n", >> + socket_path); >> + >> + if (rte_vhost_driver_start(socket_path) < 0) >> + rte_exit(EXIT_FAILURE, >> + "start vhost driver failed: %s\n", >> + socket_path); >> + return 0; >> +} >> + >> +static void >> +close_vdpa(struct vdpa_port *vport) >> +{ >> + int ret; >> + char *socket_path = vport->ifname; >> + >> + ret = rte_vhost_driver_detach_vdpa_device(socket_path); >> + if (ret != 0) >> + RTE_LOG(ERR, VDPA, >> + "attach vdpa device failed: %s\n", >> + socket_path); >> + >> + ret = rte_vhost_driver_unregister(socket_path); >> + if (ret != 0) >> + RTE_LOG(ERR, VDPA, >> + "Fail to unregister vhost driver for %s.\n", >> + socket_path); >> +} >> + >> +static void >> +vdpa_sample_quit(void) >> +{ >> + int i; >> + for (i = 0; i < RTE_MIN(MAX_VDPA_SAMPLE_PORTS, dev_total); i++) { >> + if (vports[i].ifname[0] != '\0') >> + close_vdpa(&vports[i]); >> + } >> +} >> + >> +static void >> +signal_handler(int signum) >> +{ >> + if (signum == SIGINT || signum == SIGTERM) { >> + printf("\nSignal %d received, preparing to exit...\n", signum); >> + vdpa_sample_quit(); >> + exit(0); >> + } >> +} >> + >> +/* interactive cmds */ >> + >> +/* *** Help command with introduction. *** */ >> +struct cmd_help_result { >> + cmdline_fixed_string_t help; >> +}; >> + >> +static void cmd_help_parsed(__attribute__((unused)) void *parsed_result, >> + struct cmdline *cl, >> + __attribute__((unused)) void *data) >> +{ >> + cmdline_printf( >> + cl, >> + "\n" >> + "The following commands are currently available:\n\n" >> + "Control:\n" >> + " help : Show interactive instructions.\n" >> + " list : list all available vdpa devices.\n" >> + " create <socket file> <vdev addr> : create a new vdpa >> port.\n" >> + " quit : exit vdpa sample app.\n" >> + ); >> +} >> + >> +cmdline_parse_token_string_t cmd_help_help = >> + TOKEN_STRING_INITIALIZER(struct cmd_help_result, help, "help"); >> + >> +cmdline_parse_inst_t cmd_help = { >> + .f = cmd_help_parsed, >> + .data = NULL, >> + .help_str = "show help", >> + .tokens = { >> + (void *)&cmd_help_help, >> + NULL, >> + }, >> +}; >> + >> +/* *** List all available vdpa devices *** */ >> +struct cmd_list_result { >> + cmdline_fixed_string_t action; >> +}; >> + >> +static void cmd_list_vdpa_devices_parsed( >> + __attribute__((unused)) void *parsed_result, >> + struct cmdline *cl, >> + __attribute__((unused)) void *data) >> +{ >> + int did; >> + uint32_t queue_num; >> + uint64_t features; >> + struct rte_vdpa_device *vdev; >> + struct rte_pci_addr addr; >> + >> + cmdline_printf(cl, "device id\tdevice address\tqueue num\tsupported >> features\n"); >> + for (did = 0; did < dev_total; did++) { >> + vdev = rte_vdpa_get_device(did); >> + if (!vdev) >> + continue; >> + if (vdev->ops->get_queue_num(did, &queue_num) < 0) { >> + RTE_LOG(ERR, VDPA, >> + "failed to get vdpa queue number " >> + "for device id %d.\n", did); >> + continue; >> + } >> + if (vdev->ops->get_features(did, &features) < 0) { >> + RTE_LOG(ERR, VDPA, >> + "failed to get vdpa features " >> + "for device id %d.\n", did); >> + continue; >> + } >> + addr = vdev->addr.pci_addr; >> + cmdline_printf(cl, "%d\t\t"PCI_PRI_FMT"\t%d\t\t0x%lx\n", did, >> + addr.domain, addr.bus, addr.devid, >> + addr.function, queue_num, features); > >Use "%PRIu32"for queue_num, and use "PRIx64" instead of "%lx", otherwise you may have compile error on 32-bit machine. Got it. > >> + } >> +} >> + >> +cmdline_parse_token_string_t cmd_action_list = >> + TOKEN_STRING_INITIALIZER(struct cmd_list_result, action, "list"); >> + >> +cmdline_parse_inst_t cmd_list_vdpa_devices = { >> + .f = cmd_list_vdpa_devices_parsed, /* function to call */ >> + .data = NULL, /* 2nd arg of func */ > >This comment doesn't provide much info. >Remove the comment for all the cmdline_parse_inst_t instances to keep alignment. Ok, I'll remove all useless comments. > >> + .help_str = "list all available vdpa devices", >> + .tokens = { /* token list, NULL terminated */ >> + (void *)&cmd_action_list, >> + NULL, >> + }, >> +}; >> + >> +/* *** Create new vdpa port *** */ >> +struct cmd_create_result { >> + cmdline_fixed_string_t action; >> + cmdline_fixed_string_t socket_path; >> + cmdline_fixed_string_t bdf; >> +}; >> + >> +static void cmd_create_vdpa_port_parsed(void *parsed_result, >> + struct cmdline *cl, >> + __attribute__((unused)) void *data) >> +{ >> + int did; >> + struct cmd_create_result *res = parsed_result; >> + struct rte_vdpa_dev_addr addr; >> + >> + strncpy(vports[devcnt].ifname, res->socket_path, MAX_PATH_LEN); >> + if (rte_pci_addr_parse(res->bdf, &addr.pci_addr) != 0) { >> + cmdline_printf(cl, "Unable to parse the given bdf.\n"); >> + return; >> + } >> + addr.type = PCI_ADDR; >> + did = rte_vdpa_find_device_id(&addr); >> + if (did < 0) { >> + cmdline_printf(cl, "Unable to find vdpa devide id.\n"); >> + return; >> + } >> + >> + vports[devcnt].did = did; >> + >> + if (start_vdpa(&vports[devcnt]) == 0) >> + devcnt++; >> +} >> + >> +cmdline_parse_token_string_t cmd_action_create = >> + TOKEN_STRING_INITIALIZER(struct cmd_create_result, action, >> "create"); >> +cmdline_parse_token_string_t cmd_socket_path = >> + TOKEN_STRING_INITIALIZER(struct cmd_create_result, socket_path, >> NULL); >> +cmdline_parse_token_string_t cmd_bdf = >> + TOKEN_STRING_INITIALIZER(struct cmd_create_result, bdf, NULL); >> + >> +cmdline_parse_inst_t cmd_create_vdpa_port = { >> + .f = cmd_create_vdpa_port_parsed, /* function to call */ >> + .data = NULL, /* 2nd arg of func */ >> + .help_str = "create a new vdpa port", >> + .tokens = { /* token list, NULL terminated */ >> + (void *)&cmd_action_create, >> + (void *)&cmd_socket_path, >> + (void *)&cmd_bdf, >> + NULL, >> + }, >> +}; >> + >> +/* *** QUIT *** */ >> +struct cmd_quit_result { >> + cmdline_fixed_string_t quit; >> +}; >> + >> +static void cmd_quit_parsed(__attribute__((unused)) void *parsed_result, >> + struct cmdline *cl, >> + __attribute__((unused)) void *data) >> +{ >> + vdpa_sample_quit(); >> + cmdline_quit(cl); >> +} >> + >> +cmdline_parse_token_string_t cmd_quit_quit = >> + TOKEN_STRING_INITIALIZER(struct cmd_quit_result, quit, "quit"); >> + >> +cmdline_parse_inst_t cmd_quit = { >> + .f = cmd_quit_parsed, >> + .data = NULL, >> + .help_str = "quit: exit application", >> + .tokens = { >> + (void *)&cmd_quit_quit, >> + NULL, >> + }, >> +}; >> +cmdline_parse_ctx_t main_ctx[] = { >> + (cmdline_parse_inst_t *)&cmd_help, >> + (cmdline_parse_inst_t *)&cmd_list_vdpa_devices, >> + (cmdline_parse_inst_t *)&cmd_create_vdpa_port, >> + (cmdline_parse_inst_t *)&cmd_quit, >> + NULL, >> +}; >> + >> +int >> +main(int argc, char *argv[]) >> +{ >> + char ch; >> + int i; >> + int ret; >> + struct cmdline *cl; >> + >> + ret = rte_eal_init(argc, argv); >> + if (ret < 0) >> + rte_exit(EXIT_FAILURE, "eal init failed\n"); >> + argc -= ret; >> + argv += ret; >> + >> + data_init(); >> + >> + signal(SIGINT, signal_handler); >> + signal(SIGTERM, signal_handler); >> + >> + ret = parse_args(argc, argv); >> + if (ret < 0) >> + rte_exit(EXIT_FAILURE, "invalid argument\n"); >> + >> + if (interactive == 1) { >> + cl = cmdline_stdin_new(main_ctx, "vdpa> "); >> + if (cl == NULL) >> + rte_panic("Cannot create cmdline instance\n"); >> + cmdline_interact(cl); >> + cmdline_stdin_exit(cl); >> + } else { >> + for (i = 0; i < RTE_MIN(MAX_VDPA_SAMPLE_PORTS, dev_total); >> + i++) { >> + vports[i].did = i; >> + snprintf(vports[i].ifname, MAX_PATH_LEN, "%s%d", >> + iface, i); >> + >> + start_vdpa(&vports[i]); >> + } >> + >> + printf("enter \'q\' to quit\n"); >> + while (scanf("%c", &ch)) { >> + if (ch == 'q') >> + break; >> + while (ch != '\n') { >> + if (scanf("%c", &ch)) >> + printf("%c", ch); >> + } >> + printf("enter \'q\' to quit\n"); >> + } > >Do vdpa_sample_quit here to stop the vhost session and delete the vhost sockets. It needs to call vdpa_sample_quit here. > >> + } >> + >> + return 0; >> +} >> diff --git a/examples/vdpa/meson.build b/examples/vdpa/meson.build >> new file mode 100644 >> index 000000000..2e38a069f >> --- /dev/null >> +++ b/examples/vdpa/meson.build >> @@ -0,0 +1,16 @@ >> +# SPDX-License-Identifier: BSD-3-Clause >> +# Copyright(c) 2018 Intel Corporation >> + >> +# meson file, for building this example as part of a main DPDK build. >> +# >> +# To build this example as a standalone application with an already-installed >> +# DPDK instance, use 'make' >> + >> +if host_machine.system() != 'linux' >> + build = false >> +endif >> +deps += 'vhost' >> +allow_experimental_apis = true >> +sources = files( >> + 'main.c' >> +) >> \ No newline at end of file >> -- >> 2.17.1 > >Do not forget to add this into 18.11 release_note. Ok, does this need to make a new separate commit? Thanks, Xiaolong > >BRs, >Xiao > ^ permalink raw reply [flat|nested] 41+ messages in thread
* [dpdk-dev] [PATCH v4 0/2] introduce vdpa sample 2018-09-20 22:28 [dpdk-dev] [PATCH v3 0/2] introduce vdpa sample Xiaolong Ye 2018-09-20 22:28 ` [dpdk-dev] [PATCH v3 1/2] vhost: introduce API to get vDPA device number Xiaolong Ye 2018-09-20 22:28 ` [dpdk-dev] [PATCH v3 2/2] examples/vdpa: introduce a new sample for vDPA Xiaolong Ye @ 2018-09-24 8:42 ` Xiaolong Ye 2018-09-24 8:42 ` [dpdk-dev] [PATCH v4 1/2] vhost: introduce API to get vDPA device number Xiaolong Ye ` (2 more replies) 2 siblings, 3 replies; 41+ messages in thread From: Xiaolong Ye @ 2018-09-24 8:42 UTC (permalink / raw) To: dev, Maxime Coquelin, Tiwei Bie, Zhihong Wang Cc: xiao.w.wang, Rami Rosen, Wang Haiyue, Xiaolong Ye Hi, This patchset introduces vdpa sample to demonstrate the vDPA use case. v4 changes: * add client mode support * improve the format to list the vDPA device info and improve the vdpa.rst accordingly * remove some useless comments * add introduction in 18.11 release note. v3 changes: * list cmd would show queue number and supported features of vdpa devices. * address Xiao's review comments v2 changes: * fix a compilation error reported by Rosen * improve create cmd in interactive mode and add two new cmds: list, * quit * add application documentation Xiaolong Ye (2): vhost: introduce API to get vDPA device number examples/vdpa: introduce a new sample for vDPA MAINTAINERS | 2 + doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/vdpa.rst | 115 +++++++ examples/Makefile | 2 +- examples/vdpa/Makefile | 32 ++ examples/vdpa/main.c | 458 +++++++++++++++++++++++++ examples/vdpa/meson.build | 16 + lib/librte_vhost/rte_vdpa.h | 3 + lib/librte_vhost/rte_vhost_version.map | 1 + lib/librte_vhost/vdpa.c | 6 + 10 files changed, 635 insertions(+), 1 deletion(-) create mode 100644 doc/guides/sample_app_ug/vdpa.rst create mode 100644 examples/vdpa/Makefile create mode 100644 examples/vdpa/main.c create mode 100644 examples/vdpa/meson.build -- 2.17.1 ^ permalink raw reply [flat|nested] 41+ messages in thread
* [dpdk-dev] [PATCH v4 1/2] vhost: introduce API to get vDPA device number 2018-09-24 8:42 ` [dpdk-dev] [PATCH v4 0/2] introduce vdpa sample Xiaolong Ye @ 2018-09-24 8:42 ` Xiaolong Ye 2018-09-24 8:42 ` [dpdk-dev] [PATCH v4 2/2] examples/vdpa: introduce a new sample for vDPA Xiaolong Ye 2018-09-25 12:07 ` [dpdk-dev] [PATCH v5 0/2] introduce vdpa sample Xiaolong Ye 2 siblings, 0 replies; 41+ messages in thread From: Xiaolong Ye @ 2018-09-24 8:42 UTC (permalink / raw) To: dev, Maxime Coquelin, Tiwei Bie, Zhihong Wang Cc: xiao.w.wang, Rami Rosen, Wang Haiyue, Xiaolong Ye It's used to get number of available registered vDPA devices. Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com> --- lib/librte_vhost/rte_vdpa.h | 3 +++ lib/librte_vhost/rte_vhost_version.map | 1 + lib/librte_vhost/vdpa.c | 6 ++++++ 3 files changed, 10 insertions(+) diff --git a/lib/librte_vhost/rte_vdpa.h b/lib/librte_vhost/rte_vdpa.h index 90465ca26..b8223e337 100644 --- a/lib/librte_vhost/rte_vdpa.h +++ b/lib/librte_vhost/rte_vdpa.h @@ -84,4 +84,7 @@ rte_vdpa_find_device_id(struct rte_vdpa_dev_addr *addr); struct rte_vdpa_device * __rte_experimental rte_vdpa_get_device(int did); +/* Get current available vdpa device number */ +int __rte_experimental +rte_vdpa_get_device_num(void); #endif /* _RTE_VDPA_H_ */ diff --git a/lib/librte_vhost/rte_vhost_version.map b/lib/librte_vhost/rte_vhost_version.map index da220dd02..ae39b6e21 100644 --- a/lib/librte_vhost/rte_vhost_version.map +++ b/lib/librte_vhost/rte_vhost_version.map @@ -67,6 +67,7 @@ EXPERIMENTAL { rte_vdpa_unregister_device; rte_vdpa_find_device_id; rte_vdpa_get_device; + rte_vdpa_get_device_num; rte_vhost_driver_attach_vdpa_device; rte_vhost_driver_detach_vdpa_device; rte_vhost_driver_get_vdpa_device_id; diff --git a/lib/librte_vhost/vdpa.c b/lib/librte_vhost/vdpa.c index c82fd4370..c2c5dff1d 100644 --- a/lib/librte_vhost/vdpa.c +++ b/lib/librte_vhost/vdpa.c @@ -113,3 +113,9 @@ rte_vdpa_get_device(int did) return vdpa_devices[did]; } + +int +rte_vdpa_get_device_num(void) +{ + return vdpa_device_num; +} -- 2.17.1 ^ permalink raw reply [flat|nested] 41+ messages in thread
* [dpdk-dev] [PATCH v4 2/2] examples/vdpa: introduce a new sample for vDPA 2018-09-24 8:42 ` [dpdk-dev] [PATCH v4 0/2] introduce vdpa sample Xiaolong Ye 2018-09-24 8:42 ` [dpdk-dev] [PATCH v4 1/2] vhost: introduce API to get vDPA device number Xiaolong Ye @ 2018-09-24 8:42 ` Xiaolong Ye 2018-09-24 4:12 ` Wang, Xiao W 2018-09-25 12:07 ` [dpdk-dev] [PATCH v5 0/2] introduce vdpa sample Xiaolong Ye 2 siblings, 1 reply; 41+ messages in thread From: Xiaolong Ye @ 2018-09-24 8:42 UTC (permalink / raw) To: dev, Maxime Coquelin, Tiwei Bie, Zhihong Wang Cc: xiao.w.wang, Rami Rosen, Wang Haiyue, Xiaolong Ye The vdpa sample application creates vhost-user sockets by using the vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio ring compatible devices to serve virtio driver directly to enable datapath acceleration. As vDPA driver can help to set up vhost datapath, this application doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations. Signed-off-by: Xiao Wang <xiao.w.wang@intel.com> Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com> --- MAINTAINERS | 2 + doc/guides/rel_notes/release_18_11.rst | 8 + doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/vdpa.rst | 118 +++++++ examples/Makefile | 2 +- examples/vdpa/Makefile | 32 ++ examples/vdpa/main.c | 466 +++++++++++++++++++++++++ examples/vdpa/meson.build | 16 + 8 files changed, 644 insertions(+), 1 deletion(-) create mode 100644 doc/guides/sample_app_ug/vdpa.rst create mode 100644 examples/vdpa/Makefile create mode 100644 examples/vdpa/main.c create mode 100644 examples/vdpa/meson.build diff --git a/MAINTAINERS b/MAINTAINERS index 5967c1dd3..5656f18e8 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -683,6 +683,8 @@ F: doc/guides/sample_app_ug/vhost.rst F: examples/vhost_scsi/ F: doc/guides/sample_app_ug/vhost_scsi.rst F: examples/vhost_crypto/ +F: examples/vdpa/ +F: doc/guides/sample_app_ug/vdpa.rst Vhost PMD M: Maxime Coquelin <maxime.coquelin@redhat.com> diff --git a/doc/guides/rel_notes/release_18_11.rst b/doc/guides/rel_notes/release_18_11.rst index bc9b74ec4..dd53a9ecf 100644 --- a/doc/guides/rel_notes/release_18_11.rst +++ b/doc/guides/rel_notes/release_18_11.rst @@ -67,6 +67,14 @@ New Features SR-IOV option in Hyper-V and Azure. This is an alternative to the previous vdev_netvsc, tap, and failsafe drivers combination. +* **Add a new sample for vDPA** + + The vdpa sample application creates vhost-user sockets by using the + vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes + virtio ring compatible devices to serve virtio driver directly to enable + datapath acceleration. As vDPA driver can help to set up vhost datapath, + this application doesn't need to launch dedicated worker threads for vhost + enqueue/dequeue operations. API Changes ----------- diff --git a/doc/guides/sample_app_ug/index.rst b/doc/guides/sample_app_ug/index.rst index 5bedf4f6f..74b12af85 100644 --- a/doc/guides/sample_app_ug/index.rst +++ b/doc/guides/sample_app_ug/index.rst @@ -45,6 +45,7 @@ Sample Applications User Guides vhost vhost_scsi vhost_crypto + vdpa netmap_compatibility ip_pipeline test_pipeline diff --git a/doc/guides/sample_app_ug/vdpa.rst b/doc/guides/sample_app_ug/vdpa.rst new file mode 100644 index 000000000..d05728a37 --- /dev/null +++ b/doc/guides/sample_app_ug/vdpa.rst @@ -0,0 +1,118 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2018 Intel Corporation. + +Vdpa Sample Application +======================= + +The vdpa sample application creates vhost-user sockets by using the +vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes +virtio ring compatible devices to serve virtio driver directly to enable +datapath acceleration. As vDPA driver can help to set up vhost datapath, +this application doesn't need to launch dedicated worker threads for vhost +enqueue/dequeue operations. + +Testing steps +------------- + +This section shows the steps of how to start VMs with vDPA vhost-user +backend and verify network connection & live migration. + +Build +~~~~~ + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``vdpa`` sub-directory. + +Start the vdpa example +~~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: console + + ./vdpa [EAL options] -- [--client] [--interactive|-i] or [--iface SOCKET_PATH] + +where + +* --client means running vdpa app in client mode, in the client mode, QEMU needs + to run as the server mode and take charge of socket file creation. +* --iface specifies the path prefix of the UNIX domain socket file, e.g. + /tmp/vhost-user-, then the socket files will be named as /tmp/vhost-user-<n> + (n starts from 0). +* --interactive means run the vdpa sample in interactive mode, currently 4 + internal cmds are supported: + + 1. help: show help message + 2. list: list all available vdpa devices + 3. create: create a new vdpa port with socket file and vdpa device address + 4. quit: unregister vhost driver and exit the application + +Take IFCVF driver for example: + +.. code-block:: console + + ./vdpa --log-level=9 -c 0x6 -n 4 --socket-mem 1024,1024 \ + -w 0000:06:00.3,vdpa=1 -w 0000:06:00.4,vdpa=1 \ + -- --interactive + +.. note:: + We need to bind vfio-pci to VFs before running vdpa sample. + + * modprobe vfio-pci + * ./usertools/dpdk-devbind.py -b vfio-pci 06:00.3 06:00.4 + +Then we can create 2 vdpa ports in interactive cmdline. + +.. code-block:: console + + vdpa> list + device id device address queue num supported features + 0 0000:06:00.3 1 0x5572362272 + 1 0000:06:00.4 1 0x5572362272 + + vdpa> create /tmp/vdpa-socket0 0000:06:00.3 + vdpa> create /tmp/vdpa-socket1 0000:06:00.4 + +.. _vdpa_app_run_vm: + +Start the VMs +~~~~~~~~~~~~~ + +.. code-block:: console + + qemu-system-x86_64 -cpu host -enable-kvm \ + <snip> + -mem-prealloc \ + -chardev socket,id=char0,path=<socket_file created in above steps> \ + -netdev type=vhost-user,id=vdpa,chardev=char0 \ + -device virtio-net-pci,netdev=vdpa,mac=00:aa:bb:cc:dd:ee \ + +After the VMs launches, we can login the VMs and configure the ip, verify the +network connection via ping or netperf. + +.. note:: + Suggest to use QEMU 3.0.0 which extends vhost-user for vDPA. + +Live Migration +~~~~~~~~~~~~~~ +vDPA supports cross-backend live migration, user can migrate SW vhost backend +VM to vDPA backend VM and vice versa. Here are the detailed steps. Assume A is +the source host with SW vhost VM and B is the destination host with vDPA. + +1. Start vdpa sample and launch a VM with exact same parameters as the VM on A, + in migration-listen mode: + +.. code-block:: console + + B: <qemu-command-line> -incoming tcp:0:4444 (or other PORT)) + +2. Start the migration (on source host): + +.. code-block:: console + + A: (qemu) migrate -d tcp:<B ip>:4444 (or other PORT) + +3. Check the status (on source host): + +.. code-block:: console + + A: (qemu) info migrate diff --git a/examples/Makefile b/examples/Makefile index 481720cb6..356fcb1cd 100644 --- a/examples/Makefile +++ b/examples/Makefile @@ -65,7 +65,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_HASH),y) DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += tep_termination endif DIRS-$(CONFIG_RTE_LIBRTE_TIMER) += timer -DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi +DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi vdpa ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y) DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_crypto endif diff --git a/examples/vdpa/Makefile b/examples/vdpa/Makefile new file mode 100644 index 000000000..42672a2bc --- /dev/null +++ b/examples/vdpa/Makefile @@ -0,0 +1,32 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +ifeq ($(RTE_SDK),) +$(error "Please define RTE_SDK environment variable") +endif + +# Default target, can be overridden by command line or environment +RTE_TARGET ?= x86_64-native-linuxapp-gcc + +include $(RTE_SDK)/mk/rte.vars.mk + +ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp") +$(info This application can only operate in a linuxapp environment, \ +please change the definition of the RTE_TARGET environment variable) +all: +else + +# binary name +APP = vdpa + +# all source are stored in SRCS-y +SRCS-y := main.c + +CFLAGS += -O2 -D_FILE_OFFSET_BITS=64 +CFLAGS += $(WERROR_FLAGS) +CFLAGS += -D_GNU_SOURCE +CFLAGS += -DALLOW_EXPERIMENTAL_API + +include $(RTE_SDK)/mk/rte.extapp.mk + +endif diff --git a/examples/vdpa/main.c b/examples/vdpa/main.c new file mode 100644 index 000000000..62c5ac90b --- /dev/null +++ b/examples/vdpa/main.c @@ -0,0 +1,466 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2018 Intel Corporation + */ + +#include <getopt.h> +#include <signal.h> +#include <stdint.h> +#include <string.h> +#include <unistd.h> + +#include <rte_ethdev.h> +#include <rte_malloc.h> +#include <rte_vhost.h> +#include <rte_vdpa.h> +#include <rte_pci.h> + +#include <cmdline_parse.h> +#include <cmdline_socket.h> +#include <cmdline_parse_string.h> +#include <cmdline.h> + +#define MAX_PATH_LEN 128 +#define MAX_VDPA_SAMPLE_PORTS 1024 +#define RTE_LOGTYPE_VDPA RTE_LOGTYPE_USER1 + +struct vdpa_port { + char ifname[MAX_PATH_LEN]; + int did; + int vid; + uint64_t flags; +}; + +struct vdpa_port vports[MAX_VDPA_SAMPLE_PORTS]; + +static char iface[MAX_PATH_LEN]; +static int dev_total; +static int devcnt; +static int interactive; +static int client_mode; + +/* display usage */ +static void +vdpa_usage(const char *prgname) +{ + printf("Usage: %s [EAL options] -- " + " --interactive|-i: run in interactive mode.\n" + " --iface <path>: specify the path prefix of the socket files, e.g. /tmp/vhost-user-.\n" + " --client: register a vhost-user socket as client mode.\n", + prgname); +} + +static int +parse_args(int argc, char **argv) +{ + static const char *short_option = "i"; + static struct option long_option[] = { + {"iface", required_argument, NULL, 0}, + {"interactive", no_argument, &interactive, 1}, + {"client", no_argument, &client_mode, 1}, + {NULL, 0, 0, 0}, + }; + int opt, idx; + char *prgname = argv[0]; + + while ((opt = getopt_long(argc, argv, short_option, long_option, &idx)) + != EOF) { + switch (opt) { + case 'i': + printf("Interactive-mode selected\n"); + interactive = 1; + break; + /* long options */ + case 0: + if (strncmp(long_option[idx].name, "iface", + MAX_PATH_LEN) == 0) { + strncpy(iface, optarg, MAX_PATH_LEN); + printf("iface %s\n", iface); + } + if (!strcmp(long_option[idx].name, "interactive")) { + printf("Interactive-mode selected\n"); + interactive = 1; + } + break; + + default: + vdpa_usage(prgname); + return -1; + } + } + + if (iface[0] == '\0' && interactive == 0) { + vdpa_usage(prgname); + return -1; + } + + return 0; +} + +static int +data_init(void) +{ + dev_total = rte_vdpa_get_device_num(); + if (dev_total <= 0) { + printf("No available vdpa device found\n"); + return -1; + } + devcnt = 0; + interactive = 0; + memset(iface, 0, sizeof(iface)); + memset(vports, 0, sizeof(vports)); + return 0; +} + +static int +new_device(int vid) +{ + char ifname[MAX_PATH_LEN]; + int i; + + rte_vhost_get_ifname(vid, ifname, sizeof(ifname)); + for (i = 0; i < MAX_VDPA_SAMPLE_PORTS; i++) { + if (strncmp(ifname, vports[i].ifname, MAX_PATH_LEN) == 0) { + printf("\nnew port %s, did: %d\n", + ifname, vports[i].did); + vports[i].vid = vid; + break; + } + } + + if (i >= MAX_VDPA_SAMPLE_PORTS) + return -1; + + return 0; +} + +static void +destroy_device(int vid) +{ + char ifname[MAX_PATH_LEN]; + int i; + + rte_vhost_get_ifname(vid, ifname, sizeof(ifname)); + for (i = 0; i < MAX_VDPA_SAMPLE_PORTS; i++) { + if (strcmp(ifname, vports[i].ifname) == 0) { + printf("\ndestroy port %s, did: %d\n", + ifname, vports[i].did); + break; + } + } +} + +static const struct vhost_device_ops vdpa_sample_devops = { + .new_device = new_device, + .destroy_device = destroy_device, +}; + +static int +start_vdpa(struct vdpa_port *vport) +{ + int ret; + char *socket_path = vport->ifname; + int did = vport->did; + + if (client_mode) + vport->flags |= RTE_VHOST_USER_CLIENT; + + if (access(socket_path, F_OK) != -1 && !client_mode) { + RTE_LOG(ERR, VDPA, + "%s exists, please remove it or specify another file and try again.\n", + socket_path); + return -1; + } + ret = rte_vhost_driver_register(socket_path, vport->flags); + if (ret != 0) + rte_exit(EXIT_FAILURE, + "register driver failed: %s\n", + socket_path); + + ret = rte_vhost_driver_callback_register(socket_path, + &vdpa_sample_devops); + if (ret != 0) + rte_exit(EXIT_FAILURE, + "register driver ops failed: %s\n", + socket_path); + + ret = rte_vhost_driver_attach_vdpa_device(socket_path, did); + if (ret != 0) + rte_exit(EXIT_FAILURE, + "attach vdpa device failed: %s\n", + socket_path); + + if (rte_vhost_driver_start(socket_path) < 0) + rte_exit(EXIT_FAILURE, + "start vhost driver failed: %s\n", + socket_path); + return 0; +} + +static void +close_vdpa(struct vdpa_port *vport) +{ + int ret; + char *socket_path = vport->ifname; + + ret = rte_vhost_driver_detach_vdpa_device(socket_path); + if (ret != 0) + RTE_LOG(ERR, VDPA, + "detach vdpa device failed: %s\n", + socket_path); + + ret = rte_vhost_driver_unregister(socket_path); + if (ret != 0) + RTE_LOG(ERR, VDPA, + "Fail to unregister vhost driver for %s.\n", + socket_path); +} + +static void +vdpa_sample_quit(void) +{ + int i; + for (i = 0; i < RTE_MIN(MAX_VDPA_SAMPLE_PORTS, dev_total); i++) { + if (vports[i].ifname[0] != '\0') + close_vdpa(&vports[i]); + } +} + +static void +signal_handler(int signum) +{ + if (signum == SIGINT || signum == SIGTERM) { + printf("\nSignal %d received, preparing to exit...\n", signum); + vdpa_sample_quit(); + exit(0); + } +} + +/* interactive cmds */ + +/* *** Help command with introduction. *** */ +struct cmd_help_result { + cmdline_fixed_string_t help; +}; + +static void cmd_help_parsed(__attribute__((unused)) void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + cmdline_printf( + cl, + "\n" + "The following commands are currently available:\n\n" + "Control:\n" + " help : Show interactive instructions.\n" + " list : list all available vdpa devices.\n" + " create <socket file> <vdev addr> : create a new vdpa port.\n" + " quit : exit vdpa sample app.\n" + ); +} + +cmdline_parse_token_string_t cmd_help_help = + TOKEN_STRING_INITIALIZER(struct cmd_help_result, help, "help"); + +cmdline_parse_inst_t cmd_help = { + .f = cmd_help_parsed, + .data = NULL, + .help_str = "show help", + .tokens = { + (void *)&cmd_help_help, + NULL, + }, +}; + +/* *** List all available vdpa devices *** */ +struct cmd_list_result { + cmdline_fixed_string_t action; +}; + +static void cmd_list_vdpa_devices_parsed( + __attribute__((unused)) void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + int did; + uint32_t queue_num; + uint64_t features; + struct rte_vdpa_device *vdev; + struct rte_pci_addr addr; + + cmdline_printf(cl, "device id\tdevice address\tqueue num\tsupported features\n"); + for (did = 0; did < dev_total; did++) { + vdev = rte_vdpa_get_device(did); + if (!vdev) + continue; + if (vdev->ops->get_queue_num(did, &queue_num) < 0) { + RTE_LOG(ERR, VDPA, + "failed to get vdpa queue number " + "for device id %d.\n", did); + continue; + } + if (vdev->ops->get_features(did, &features) < 0) { + RTE_LOG(ERR, VDPA, + "failed to get vdpa features " + "for device id %d.\n", did); + continue; + } + addr = vdev->addr.pci_addr; + cmdline_printf(cl, + "%d\t\t"PCI_PRI_FMT"\t%"PRIu32"\t\t0x%"PRIu64"\n", did, + addr.domain, addr.bus, addr.devid, + addr.function, queue_num, features); + } +} + +cmdline_parse_token_string_t cmd_action_list = + TOKEN_STRING_INITIALIZER(struct cmd_list_result, action, "list"); + +cmdline_parse_inst_t cmd_list_vdpa_devices = { + .f = cmd_list_vdpa_devices_parsed, + .data = NULL, + .help_str = "list all available vdpa devices", + .tokens = { + (void *)&cmd_action_list, + NULL, + }, +}; + +/* *** Create new vdpa port *** */ +struct cmd_create_result { + cmdline_fixed_string_t action; + cmdline_fixed_string_t socket_path; + cmdline_fixed_string_t bdf; +}; + +static void cmd_create_vdpa_port_parsed(void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + int did; + struct cmd_create_result *res = parsed_result; + struct rte_vdpa_dev_addr addr; + + strncpy(vports[devcnt].ifname, res->socket_path, MAX_PATH_LEN); + if (rte_pci_addr_parse(res->bdf, &addr.pci_addr) != 0) { + cmdline_printf(cl, "Unable to parse the given bdf.\n"); + return; + } + addr.type = PCI_ADDR; + did = rte_vdpa_find_device_id(&addr); + if (did < 0) { + cmdline_printf(cl, "Unable to find vdpa devide id.\n"); + return; + } + + vports[devcnt].did = did; + + if (start_vdpa(&vports[devcnt]) == 0) + devcnt++; +} + +cmdline_parse_token_string_t cmd_action_create = + TOKEN_STRING_INITIALIZER(struct cmd_create_result, action, "create"); +cmdline_parse_token_string_t cmd_socket_path = + TOKEN_STRING_INITIALIZER(struct cmd_create_result, socket_path, NULL); +cmdline_parse_token_string_t cmd_bdf = + TOKEN_STRING_INITIALIZER(struct cmd_create_result, bdf, NULL); + +cmdline_parse_inst_t cmd_create_vdpa_port = { + .f = cmd_create_vdpa_port_parsed, + .data = NULL, + .help_str = "create a new vdpa port", + .tokens = { + (void *)&cmd_action_create, + (void *)&cmd_socket_path, + (void *)&cmd_bdf, + NULL, + }, +}; + +/* *** QUIT *** */ +struct cmd_quit_result { + cmdline_fixed_string_t quit; +}; + +static void cmd_quit_parsed(__attribute__((unused)) void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + vdpa_sample_quit(); + cmdline_quit(cl); +} + +cmdline_parse_token_string_t cmd_quit_quit = + TOKEN_STRING_INITIALIZER(struct cmd_quit_result, quit, "quit"); + +cmdline_parse_inst_t cmd_quit = { + .f = cmd_quit_parsed, + .data = NULL, + .help_str = "quit: exit application", + .tokens = { + (void *)&cmd_quit_quit, + NULL, + }, +}; +cmdline_parse_ctx_t main_ctx[] = { + (cmdline_parse_inst_t *)&cmd_help, + (cmdline_parse_inst_t *)&cmd_list_vdpa_devices, + (cmdline_parse_inst_t *)&cmd_create_vdpa_port, + (cmdline_parse_inst_t *)&cmd_quit, + NULL, +}; + +int +main(int argc, char *argv[]) +{ + char ch; + int i; + int ret; + struct cmdline *cl; + + ret = rte_eal_init(argc, argv); + if (ret < 0) + rte_exit(EXIT_FAILURE, "eal init failed\n"); + argc -= ret; + argv += ret; + + data_init(); + + signal(SIGINT, signal_handler); + signal(SIGTERM, signal_handler); + + ret = parse_args(argc, argv); + if (ret < 0) + rte_exit(EXIT_FAILURE, "invalid argument\n"); + + if (interactive == 1) { + cl = cmdline_stdin_new(main_ctx, "vdpa> "); + if (cl == NULL) + rte_panic("Cannot create cmdline instance\n"); + cmdline_interact(cl); + cmdline_stdin_exit(cl); + } else { + for (i = 0; i < RTE_MIN(MAX_VDPA_SAMPLE_PORTS, dev_total); + i++) { + vports[i].did = i; + snprintf(vports[i].ifname, MAX_PATH_LEN, "%s%d", + iface, i); + + start_vdpa(&vports[i]); + } + + printf("enter \'q\' to quit\n"); + while (scanf("%c", &ch)) { + if (ch == 'q') + break; + while (ch != '\n') { + if (scanf("%c", &ch)) + printf("%c", ch); + } + printf("enter \'q\' to quit\n"); + } + vdpa_sample_quit(); + } + + return 0; +} diff --git a/examples/vdpa/meson.build b/examples/vdpa/meson.build new file mode 100644 index 000000000..2e38a069f --- /dev/null +++ b/examples/vdpa/meson.build @@ -0,0 +1,16 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +# meson file, for building this example as part of a main DPDK build. +# +# To build this example as a standalone application with an already-installed +# DPDK instance, use 'make' + +if host_machine.system() != 'linux' + build = false +endif +deps += 'vhost' +allow_experimental_apis = true +sources = files( + 'main.c' +) \ No newline at end of file -- 2.17.1 ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [dpdk-dev] [PATCH v4 2/2] examples/vdpa: introduce a new sample for vDPA 2018-09-24 8:42 ` [dpdk-dev] [PATCH v4 2/2] examples/vdpa: introduce a new sample for vDPA Xiaolong Ye @ 2018-09-24 4:12 ` Wang, Xiao W 2018-09-24 21:30 ` Ye Xiaolong 0 siblings, 1 reply; 41+ messages in thread From: Wang, Xiao W @ 2018-09-24 4:12 UTC (permalink / raw) To: Ye, Xiaolong, dev, Maxime Coquelin, Bie, Tiwei, Wang, Zhihong Cc: Rami Rosen, Wang, Haiyue Hi Xiaolong, Thanks for the update, 2 small comments below. > -----Original Message----- > From: Ye, Xiaolong > Sent: Monday, September 24, 2018 4:43 PM > To: dev@dpdk.org; Maxime Coquelin <maxime.coquelin@redhat.com>; Bie, > Tiwei <tiwei.bie@intel.com>; Wang, Zhihong <zhihong.wang@intel.com> > Cc: Wang, Xiao W <xiao.w.wang@intel.com>; Rami Rosen > <roszenrami@gmail.com>; Wang, Haiyue <haiyue.wang@intel.com>; Ye, > Xiaolong <xiaolong.ye@intel.com> > Subject: [PATCH v4 2/2] examples/vdpa: introduce a new sample for vDPA > > The vdpa sample application creates vhost-user sockets by using the > vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes > virtio ring compatible devices to serve virtio driver directly to enable > datapath acceleration. As vDPA driver can help to set up vhost datapath, > this application doesn't need to launch dedicated worker threads for vhost > enqueue/dequeue operations. > > Signed-off-by: Xiao Wang <xiao.w.wang@intel.com> > Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com> > --- > MAINTAINERS | 2 + > doc/guides/rel_notes/release_18_11.rst | 8 + > doc/guides/sample_app_ug/index.rst | 1 + > doc/guides/sample_app_ug/vdpa.rst | 118 +++++++ > examples/Makefile | 2 +- > examples/vdpa/Makefile | 32 ++ > examples/vdpa/main.c | 466 +++++++++++++++++++++++++ > examples/vdpa/meson.build | 16 + > 8 files changed, 644 insertions(+), 1 deletion(-) > create mode 100644 doc/guides/sample_app_ug/vdpa.rst > create mode 100644 examples/vdpa/Makefile > create mode 100644 examples/vdpa/main.c > create mode 100644 examples/vdpa/meson.build > > diff --git a/MAINTAINERS b/MAINTAINERS > index 5967c1dd3..5656f18e8 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -683,6 +683,8 @@ F: doc/guides/sample_app_ug/vhost.rst > F: examples/vhost_scsi/ > F: doc/guides/sample_app_ug/vhost_scsi.rst > F: examples/vhost_crypto/ > +F: examples/vdpa/ > +F: doc/guides/sample_app_ug/vdpa.rst > > Vhost PMD > M: Maxime Coquelin <maxime.coquelin@redhat.com> > diff --git a/doc/guides/rel_notes/release_18_11.rst > b/doc/guides/rel_notes/release_18_11.rst > index bc9b74ec4..dd53a9ecf 100644 > --- a/doc/guides/rel_notes/release_18_11.rst > +++ b/doc/guides/rel_notes/release_18_11.rst > @@ -67,6 +67,14 @@ New Features > SR-IOV option in Hyper-V and Azure. This is an alternative to the previous > vdev_netvsc, tap, and failsafe drivers combination. > > +* **Add a new sample for vDPA** > + > + The vdpa sample application creates vhost-user sockets by using the > + vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes > + virtio ring compatible devices to serve virtio driver directly to enable > + datapath acceleration. As vDPA driver can help to set up vhost datapath, > + this application doesn't need to launch dedicated worker threads for vhost > + enqueue/dequeue operations. > > API Changes > ----------- > diff --git a/doc/guides/sample_app_ug/index.rst > b/doc/guides/sample_app_ug/index.rst > index 5bedf4f6f..74b12af85 100644 > --- a/doc/guides/sample_app_ug/index.rst > +++ b/doc/guides/sample_app_ug/index.rst > @@ -45,6 +45,7 @@ Sample Applications User Guides > vhost > vhost_scsi > vhost_crypto > + vdpa > netmap_compatibility > ip_pipeline > test_pipeline > diff --git a/doc/guides/sample_app_ug/vdpa.rst > b/doc/guides/sample_app_ug/vdpa.rst > new file mode 100644 > index 000000000..d05728a37 > --- /dev/null > +++ b/doc/guides/sample_app_ug/vdpa.rst > @@ -0,0 +1,118 @@ > +.. SPDX-License-Identifier: BSD-3-Clause > + Copyright(c) 2018 Intel Corporation. > + > +Vdpa Sample Application > +======================= > + > +The vdpa sample application creates vhost-user sockets by using the > +vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes > +virtio ring compatible devices to serve virtio driver directly to enable > +datapath acceleration. As vDPA driver can help to set up vhost datapath, > +this application doesn't need to launch dedicated worker threads for vhost > +enqueue/dequeue operations. > + > +Testing steps > +------------- > + > +This section shows the steps of how to start VMs with vDPA vhost-user > +backend and verify network connection & live migration. > + > +Build > +~~~~~ > + > +To compile the sample application see :doc:`compiling`. > + > +The application is located in the ``vdpa`` sub-directory. > + > +Start the vdpa example > +~~~~~~~~~~~~~~~~~~~~~~ > + > +.. code-block:: console > + > + ./vdpa [EAL options] -- [--client] [--interactive|-i] or [--iface SOCKET_PATH] > + > +where > + > +* --client means running vdpa app in client mode, in the client mode, QEMU > needs > + to run as the server mode and take charge of socket file creation. > +* --iface specifies the path prefix of the UNIX domain socket file, e.g. > + /tmp/vhost-user-, then the socket files will be named as /tmp/vhost-user-<n> > + (n starts from 0). > +* --interactive means run the vdpa sample in interactive mode, currently 4 > + internal cmds are supported: > + > + 1. help: show help message > + 2. list: list all available vdpa devices > + 3. create: create a new vdpa port with socket file and vdpa device address > + 4. quit: unregister vhost driver and exit the application > + > +Take IFCVF driver for example: > + > +.. code-block:: console > + > + ./vdpa --log-level=9 -c 0x6 -n 4 --socket-mem 1024,1024 \ > + -w 0000:06:00.3,vdpa=1 -w 0000:06:00.4,vdpa=1 \ > + -- --interactive To demonstrate app doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations, We can use "-c 0x2" to indicate that no need to allocate dedicated worker threads. > + > +.. note:: > + We need to bind vfio-pci to VFs before running vdpa sample. > + > + * modprobe vfio-pci > + * ./usertools/dpdk-devbind.py -b vfio-pci 06:00.3 06:00.4 > + > +Then we can create 2 vdpa ports in interactive cmdline. > + > +.. code-block:: console > + > + vdpa> list > + device id device address queue num supported features > + 0 0000:06:00.3 1 0x5572362272 > + 1 0000:06:00.4 1 0x5572362272 > + > + vdpa> create /tmp/vdpa-socket0 0000:06:00.3 > + vdpa> create /tmp/vdpa-socket1 0000:06:00.4 > + > +.. _vdpa_app_run_vm: > + > +Start the VMs > +~~~~~~~~~~~~~ > + > +.. code-block:: console > + > + qemu-system-x86_64 -cpu host -enable-kvm \ > + <snip> > + -mem-prealloc \ > + -chardev socket,id=char0,path=<socket_file created in above steps> \ > + -netdev type=vhost-user,id=vdpa,chardev=char0 \ > + -device virtio-net-pci,netdev=vdpa,mac=00:aa:bb:cc:dd:ee \ > + > +After the VMs launches, we can login the VMs and configure the ip, verify the > +network connection via ping or netperf. > + > +.. note:: > + Suggest to use QEMU 3.0.0 which extends vhost-user for vDPA. [...] > + > +/* *** List all available vdpa devices *** */ > +struct cmd_list_result { > + cmdline_fixed_string_t action; > +}; > + > +static void cmd_list_vdpa_devices_parsed( > + __attribute__((unused)) void *parsed_result, > + struct cmdline *cl, > + __attribute__((unused)) void *data) > +{ > + int did; > + uint32_t queue_num; > + uint64_t features; > + struct rte_vdpa_device *vdev; > + struct rte_pci_addr addr; > + > + cmdline_printf(cl, "device id\tdevice address\tqueue num\tsupported > features\n"); > + for (did = 0; did < dev_total; did++) { > + vdev = rte_vdpa_get_device(did); > + if (!vdev) > + continue; > + if (vdev->ops->get_queue_num(did, &queue_num) < 0) { > + RTE_LOG(ERR, VDPA, > + "failed to get vdpa queue number " > + "for device id %d.\n", did); > + continue; > + } > + if (vdev->ops->get_features(did, &features) < 0) { > + RTE_LOG(ERR, VDPA, > + "failed to get vdpa features " > + "for device id %d.\n", did); > + continue; > + } > + addr = vdev->addr.pci_addr; > + cmdline_printf(cl, > + > "%d\t\t"PCI_PRI_FMT"\t%"PRIu32"\t\t0x%"PRIu64"\n", did, > + addr.domain, addr.bus, addr.devid, > + addr.function, queue_num, features); Use PRIx64 instead of PRIu64 for features. You can add a blank space between "PRIx64" and the other section to make it more readable. Refer to: lib/librte_vhost/vhost_user.c: "guest memory region %u, size: 0x%" PRIx64 "\n" BRs, Xiao ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [dpdk-dev] [PATCH v4 2/2] examples/vdpa: introduce a new sample for vDPA 2018-09-24 4:12 ` Wang, Xiao W @ 2018-09-24 21:30 ` Ye Xiaolong 0 siblings, 0 replies; 41+ messages in thread From: Ye Xiaolong @ 2018-09-24 21:30 UTC (permalink / raw) To: Wang, Xiao W Cc: dev, Maxime Coquelin, Bie, Tiwei, Wang, Zhihong, Rami Rosen, Wang, Haiyue On 09/24, Wang, Xiao W wrote: >Hi Xiaolong, > >Thanks for the update, 2 small comments below. > [snip] >> + ./vdpa --log-level=9 -c 0x6 -n 4 --socket-mem 1024,1024 \ >> + -w 0000:06:00.3,vdpa=1 -w 0000:06:00.4,vdpa=1 \ >> + -- --interactive > >To demonstrate app doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations, >We can use "-c 0x2" to indicate that no need to allocate dedicated worker threads. > Got it, will do. >> + >> +.. note:: [snip] >> "%d\t\t"PCI_PRI_FMT"\t%"PRIu32"\t\t0x%"PRIu64"\n", did, >> + addr.domain, addr.bus, addr.devid, >> + addr.function, queue_num, features); > >Use PRIx64 instead of PRIu64 for features. >You can add a blank space between "PRIx64" and the other section to make it more readable. >Refer to: > lib/librte_vhost/vhost_user.c: "guest memory region %u, size: 0x%" PRIx64 "\n" Got it, will do. Thanks, Xiaolong > >BRs, >Xiao ^ permalink raw reply [flat|nested] 41+ messages in thread
* [dpdk-dev] [PATCH v5 0/2] introduce vdpa sample 2018-09-24 8:42 ` [dpdk-dev] [PATCH v4 0/2] introduce vdpa sample Xiaolong Ye 2018-09-24 8:42 ` [dpdk-dev] [PATCH v4 1/2] vhost: introduce API to get vDPA device number Xiaolong Ye 2018-09-24 8:42 ` [dpdk-dev] [PATCH v4 2/2] examples/vdpa: introduce a new sample for vDPA Xiaolong Ye @ 2018-09-25 12:07 ` Xiaolong Ye 2018-09-25 12:07 ` [dpdk-dev] [PATCH v5 1/2] vhost: introduce API to get vDPA device number Xiaolong Ye ` (2 more replies) 2 siblings, 3 replies; 41+ messages in thread From: Xiaolong Ye @ 2018-09-25 12:07 UTC (permalink / raw) To: dev, Maxime Coquelin, Tiwei Bie, Zhihong Wang Cc: xiao.w.wang, Rami Rosen, Wang Haiyue, Xiaolong Ye Hi, This patchset introduces vdpa sample to demonstrate the vDPA use case. v5 changes: * improve print format and correct from "PRIu64" to "PRIx64" * use "-c 0x2" to better demonstrate app doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations in vdpa.rst v4 changes: * add client mode support * improve the format to list the vDPA device info and improve the vdpa.rst accordingly * remove some useless comments * add introduction in 18.11 release note. v3 changes: * list cmd would show queue number and supported features of vdpa devices. * address Xiao's review comments v2 changes: * fix a compilation error reported by Rosen * improve create cmd in interactive mode and add two new cmds: list, * quit * add application documentation Xiaolong Ye (2): vhost: introduce API to get vDPA device number examples/vdpa: introduce a new sample for vDPA MAINTAINERS | 2 + doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/vdpa.rst | 115 +++++++ examples/Makefile | 2 +- examples/vdpa/Makefile | 32 ++ examples/vdpa/main.c | 458 +++++++++++++++++++++++++ examples/vdpa/meson.build | 16 + lib/librte_vhost/rte_vdpa.h | 3 + lib/librte_vhost/rte_vhost_version.map | 1 + lib/librte_vhost/vdpa.c | 6 + 10 files changed, 635 insertions(+), 1 deletion(-) create mode 100644 doc/guides/sample_app_ug/vdpa.rst create mode 100644 examples/vdpa/Makefile create mode 100644 examples/vdpa/main.c create mode 100644 examples/vdpa/meson.build -- 2.17.1 ^ permalink raw reply [flat|nested] 41+ messages in thread
* [dpdk-dev] [PATCH v5 1/2] vhost: introduce API to get vDPA device number 2018-09-25 12:07 ` [dpdk-dev] [PATCH v5 0/2] introduce vdpa sample Xiaolong Ye @ 2018-09-25 12:07 ` Xiaolong Ye 2018-09-25 12:07 ` [dpdk-dev] [PATCH v5 2/2] examples/vdpa: introduce a new sample for vDPA Xiaolong Ye 2018-09-26 9:06 ` [dpdk-dev] [PATCH v6 0/2] introduce vdpa sample Xiaolong Ye 2 siblings, 0 replies; 41+ messages in thread From: Xiaolong Ye @ 2018-09-25 12:07 UTC (permalink / raw) To: dev, Maxime Coquelin, Tiwei Bie, Zhihong Wang Cc: xiao.w.wang, Rami Rosen, Wang Haiyue, Xiaolong Ye It's used to get number of available registered vDPA devices. Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com> --- lib/librte_vhost/rte_vdpa.h | 3 +++ lib/librte_vhost/rte_vhost_version.map | 1 + lib/librte_vhost/vdpa.c | 6 ++++++ 3 files changed, 10 insertions(+) diff --git a/lib/librte_vhost/rte_vdpa.h b/lib/librte_vhost/rte_vdpa.h index 90465ca26..b8223e337 100644 --- a/lib/librte_vhost/rte_vdpa.h +++ b/lib/librte_vhost/rte_vdpa.h @@ -84,4 +84,7 @@ rte_vdpa_find_device_id(struct rte_vdpa_dev_addr *addr); struct rte_vdpa_device * __rte_experimental rte_vdpa_get_device(int did); +/* Get current available vdpa device number */ +int __rte_experimental +rte_vdpa_get_device_num(void); #endif /* _RTE_VDPA_H_ */ diff --git a/lib/librte_vhost/rte_vhost_version.map b/lib/librte_vhost/rte_vhost_version.map index da220dd02..ae39b6e21 100644 --- a/lib/librte_vhost/rte_vhost_version.map +++ b/lib/librte_vhost/rte_vhost_version.map @@ -67,6 +67,7 @@ EXPERIMENTAL { rte_vdpa_unregister_device; rte_vdpa_find_device_id; rte_vdpa_get_device; + rte_vdpa_get_device_num; rte_vhost_driver_attach_vdpa_device; rte_vhost_driver_detach_vdpa_device; rte_vhost_driver_get_vdpa_device_id; diff --git a/lib/librte_vhost/vdpa.c b/lib/librte_vhost/vdpa.c index c82fd4370..c2c5dff1d 100644 --- a/lib/librte_vhost/vdpa.c +++ b/lib/librte_vhost/vdpa.c @@ -113,3 +113,9 @@ rte_vdpa_get_device(int did) return vdpa_devices[did]; } + +int +rte_vdpa_get_device_num(void) +{ + return vdpa_device_num; +} -- 2.17.1 ^ permalink raw reply [flat|nested] 41+ messages in thread
* [dpdk-dev] [PATCH v5 2/2] examples/vdpa: introduce a new sample for vDPA 2018-09-25 12:07 ` [dpdk-dev] [PATCH v5 0/2] introduce vdpa sample Xiaolong Ye 2018-09-25 12:07 ` [dpdk-dev] [PATCH v5 1/2] vhost: introduce API to get vDPA device number Xiaolong Ye @ 2018-09-25 12:07 ` Xiaolong Ye 2018-09-25 8:32 ` Wang, Xiao W 2018-09-26 9:06 ` [dpdk-dev] [PATCH v6 0/2] introduce vdpa sample Xiaolong Ye 2 siblings, 1 reply; 41+ messages in thread From: Xiaolong Ye @ 2018-09-25 12:07 UTC (permalink / raw) To: dev, Maxime Coquelin, Tiwei Bie, Zhihong Wang Cc: xiao.w.wang, Rami Rosen, Wang Haiyue, Xiaolong Ye The vdpa sample application creates vhost-user sockets by using the vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio ring compatible devices to serve virtio driver directly to enable datapath acceleration. As vDPA driver can help to set up vhost datapath, this application doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations. Signed-off-by: Xiao Wang <xiao.w.wang@intel.com> Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com> --- MAINTAINERS | 2 + doc/guides/rel_notes/release_18_11.rst | 8 + doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/vdpa.rst | 118 +++++++ examples/Makefile | 2 +- examples/vdpa/Makefile | 32 ++ examples/vdpa/main.c | 466 +++++++++++++++++++++++++ examples/vdpa/meson.build | 16 + 8 files changed, 644 insertions(+), 1 deletion(-) create mode 100644 doc/guides/sample_app_ug/vdpa.rst create mode 100644 examples/vdpa/Makefile create mode 100644 examples/vdpa/main.c create mode 100644 examples/vdpa/meson.build diff --git a/MAINTAINERS b/MAINTAINERS index 5967c1dd3..5656f18e8 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -683,6 +683,8 @@ F: doc/guides/sample_app_ug/vhost.rst F: examples/vhost_scsi/ F: doc/guides/sample_app_ug/vhost_scsi.rst F: examples/vhost_crypto/ +F: examples/vdpa/ +F: doc/guides/sample_app_ug/vdpa.rst Vhost PMD M: Maxime Coquelin <maxime.coquelin@redhat.com> diff --git a/doc/guides/rel_notes/release_18_11.rst b/doc/guides/rel_notes/release_18_11.rst index bc9b74ec4..dd53a9ecf 100644 --- a/doc/guides/rel_notes/release_18_11.rst +++ b/doc/guides/rel_notes/release_18_11.rst @@ -67,6 +67,14 @@ New Features SR-IOV option in Hyper-V and Azure. This is an alternative to the previous vdev_netvsc, tap, and failsafe drivers combination. +* **Add a new sample for vDPA** + + The vdpa sample application creates vhost-user sockets by using the + vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes + virtio ring compatible devices to serve virtio driver directly to enable + datapath acceleration. As vDPA driver can help to set up vhost datapath, + this application doesn't need to launch dedicated worker threads for vhost + enqueue/dequeue operations. API Changes ----------- diff --git a/doc/guides/sample_app_ug/index.rst b/doc/guides/sample_app_ug/index.rst index 5bedf4f6f..74b12af85 100644 --- a/doc/guides/sample_app_ug/index.rst +++ b/doc/guides/sample_app_ug/index.rst @@ -45,6 +45,7 @@ Sample Applications User Guides vhost vhost_scsi vhost_crypto + vdpa netmap_compatibility ip_pipeline test_pipeline diff --git a/doc/guides/sample_app_ug/vdpa.rst b/doc/guides/sample_app_ug/vdpa.rst new file mode 100644 index 000000000..a089393c0 --- /dev/null +++ b/doc/guides/sample_app_ug/vdpa.rst @@ -0,0 +1,118 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2018 Intel Corporation. + +Vdpa Sample Application +======================= + +The vdpa sample application creates vhost-user sockets by using the +vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes +virtio ring compatible devices to serve virtio driver directly to enable +datapath acceleration. As vDPA driver can help to set up vhost datapath, +this application doesn't need to launch dedicated worker threads for vhost +enqueue/dequeue operations. + +Testing steps +------------- + +This section shows the steps of how to start VMs with vDPA vhost-user +backend and verify network connection & live migration. + +Build +~~~~~ + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``vdpa`` sub-directory. + +Start the vdpa example +~~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: console + + ./vdpa [EAL options] -- [--client] [--interactive|-i] or [--iface SOCKET_PATH] + +where + +* --client means running vdpa app in client mode, in the client mode, QEMU needs + to run as the server mode and take charge of socket file creation. +* --iface specifies the path prefix of the UNIX domain socket file, e.g. + /tmp/vhost-user-, then the socket files will be named as /tmp/vhost-user-<n> + (n starts from 0). +* --interactive means run the vdpa sample in interactive mode, currently 4 + internal cmds are supported: + + 1. help: show help message + 2. list: list all available vdpa devices + 3. create: create a new vdpa port with socket file and vdpa device address + 4. quit: unregister vhost driver and exit the application + +Take IFCVF driver for example: + +.. code-block:: console + + ./vdpa --log-level=9 -c 0x2 -n 4 --socket-mem 1024,1024 \ + -w 0000:06:00.3,vdpa=1 -w 0000:06:00.4,vdpa=1 \ + -- --interactive + +.. note:: + We need to bind vfio-pci to VFs before running vdpa sample. + + * modprobe vfio-pci + * ./usertools/dpdk-devbind.py -b vfio-pci 06:00.3 06:00.4 + +Then we can create 2 vdpa ports in interactive cmdline. + +.. code-block:: console + + vdpa> list + device id device address queue num supported features + 0 0000:06:00.3 1 0x5572362272 + 1 0000:06:00.4 1 0x5572362272 + + vdpa> create /tmp/vdpa-socket0 0000:06:00.3 + vdpa> create /tmp/vdpa-socket1 0000:06:00.4 + +.. _vdpa_app_run_vm: + +Start the VMs +~~~~~~~~~~~~~ + +.. code-block:: console + + qemu-system-x86_64 -cpu host -enable-kvm \ + <snip> + -mem-prealloc \ + -chardev socket,id=char0,path=<socket_file created in above steps> \ + -netdev type=vhost-user,id=vdpa,chardev=char0 \ + -device virtio-net-pci,netdev=vdpa,mac=00:aa:bb:cc:dd:ee \ + +After the VMs launches, we can login the VMs and configure the ip, verify the +network connection via ping or netperf. + +.. note:: + Suggest to use QEMU 3.0.0 which extends vhost-user for vDPA. + +Live Migration +~~~~~~~~~~~~~~ +vDPA supports cross-backend live migration, user can migrate SW vhost backend +VM to vDPA backend VM and vice versa. Here are the detailed steps. Assume A is +the source host with SW vhost VM and B is the destination host with vDPA. + +1. Start vdpa sample and launch a VM with exact same parameters as the VM on A, + in migration-listen mode: + +.. code-block:: console + + B: <qemu-command-line> -incoming tcp:0:4444 (or other PORT)) + +2. Start the migration (on source host): + +.. code-block:: console + + A: (qemu) migrate -d tcp:<B ip>:4444 (or other PORT) + +3. Check the status (on source host): + +.. code-block:: console + + A: (qemu) info migrate diff --git a/examples/Makefile b/examples/Makefile index 481720cb6..356fcb1cd 100644 --- a/examples/Makefile +++ b/examples/Makefile @@ -65,7 +65,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_HASH),y) DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += tep_termination endif DIRS-$(CONFIG_RTE_LIBRTE_TIMER) += timer -DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi +DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi vdpa ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y) DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_crypto endif diff --git a/examples/vdpa/Makefile b/examples/vdpa/Makefile new file mode 100644 index 000000000..42672a2bc --- /dev/null +++ b/examples/vdpa/Makefile @@ -0,0 +1,32 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +ifeq ($(RTE_SDK),) +$(error "Please define RTE_SDK environment variable") +endif + +# Default target, can be overridden by command line or environment +RTE_TARGET ?= x86_64-native-linuxapp-gcc + +include $(RTE_SDK)/mk/rte.vars.mk + +ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp") +$(info This application can only operate in a linuxapp environment, \ +please change the definition of the RTE_TARGET environment variable) +all: +else + +# binary name +APP = vdpa + +# all source are stored in SRCS-y +SRCS-y := main.c + +CFLAGS += -O2 -D_FILE_OFFSET_BITS=64 +CFLAGS += $(WERROR_FLAGS) +CFLAGS += -D_GNU_SOURCE +CFLAGS += -DALLOW_EXPERIMENTAL_API + +include $(RTE_SDK)/mk/rte.extapp.mk + +endif diff --git a/examples/vdpa/main.c b/examples/vdpa/main.c new file mode 100644 index 000000000..7acda67c4 --- /dev/null +++ b/examples/vdpa/main.c @@ -0,0 +1,466 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2018 Intel Corporation + */ + +#include <getopt.h> +#include <signal.h> +#include <stdint.h> +#include <string.h> +#include <unistd.h> + +#include <rte_ethdev.h> +#include <rte_malloc.h> +#include <rte_vhost.h> +#include <rte_vdpa.h> +#include <rte_pci.h> + +#include <cmdline_parse.h> +#include <cmdline_socket.h> +#include <cmdline_parse_string.h> +#include <cmdline.h> + +#define MAX_PATH_LEN 128 +#define MAX_VDPA_SAMPLE_PORTS 1024 +#define RTE_LOGTYPE_VDPA RTE_LOGTYPE_USER1 + +struct vdpa_port { + char ifname[MAX_PATH_LEN]; + int did; + int vid; + uint64_t flags; +}; + +struct vdpa_port vports[MAX_VDPA_SAMPLE_PORTS]; + +static char iface[MAX_PATH_LEN]; +static int dev_total; +static int devcnt; +static int interactive; +static int client_mode; + +/* display usage */ +static void +vdpa_usage(const char *prgname) +{ + printf("Usage: %s [EAL options] -- " + " --interactive|-i: run in interactive mode.\n" + " --iface <path>: specify the path prefix of the socket files, e.g. /tmp/vhost-user-.\n" + " --client: register a vhost-user socket as client mode.\n", + prgname); +} + +static int +parse_args(int argc, char **argv) +{ + static const char *short_option = "i"; + static struct option long_option[] = { + {"iface", required_argument, NULL, 0}, + {"interactive", no_argument, &interactive, 1}, + {"client", no_argument, &client_mode, 1}, + {NULL, 0, 0, 0}, + }; + int opt, idx; + char *prgname = argv[0]; + + while ((opt = getopt_long(argc, argv, short_option, long_option, &idx)) + != EOF) { + switch (opt) { + case 'i': + printf("Interactive-mode selected\n"); + interactive = 1; + break; + /* long options */ + case 0: + if (strncmp(long_option[idx].name, "iface", + MAX_PATH_LEN) == 0) { + strncpy(iface, optarg, MAX_PATH_LEN); + printf("iface %s\n", iface); + } + if (!strcmp(long_option[idx].name, "interactive")) { + printf("Interactive-mode selected\n"); + interactive = 1; + } + break; + + default: + vdpa_usage(prgname); + return -1; + } + } + + if (iface[0] == '\0' && interactive == 0) { + vdpa_usage(prgname); + return -1; + } + + return 0; +} + +static int +data_init(void) +{ + dev_total = rte_vdpa_get_device_num(); + if (dev_total <= 0) { + printf("No available vdpa device found\n"); + return -1; + } + devcnt = 0; + interactive = 0; + memset(iface, 0, sizeof(iface)); + memset(vports, 0, sizeof(vports)); + return 0; +} + +static int +new_device(int vid) +{ + char ifname[MAX_PATH_LEN]; + int i; + + rte_vhost_get_ifname(vid, ifname, sizeof(ifname)); + for (i = 0; i < MAX_VDPA_SAMPLE_PORTS; i++) { + if (strncmp(ifname, vports[i].ifname, MAX_PATH_LEN) == 0) { + printf("\nnew port %s, did: %d\n", + ifname, vports[i].did); + vports[i].vid = vid; + break; + } + } + + if (i >= MAX_VDPA_SAMPLE_PORTS) + return -1; + + return 0; +} + +static void +destroy_device(int vid) +{ + char ifname[MAX_PATH_LEN]; + int i; + + rte_vhost_get_ifname(vid, ifname, sizeof(ifname)); + for (i = 0; i < MAX_VDPA_SAMPLE_PORTS; i++) { + if (strcmp(ifname, vports[i].ifname) == 0) { + printf("\ndestroy port %s, did: %d\n", + ifname, vports[i].did); + break; + } + } +} + +static const struct vhost_device_ops vdpa_sample_devops = { + .new_device = new_device, + .destroy_device = destroy_device, +}; + +static int +start_vdpa(struct vdpa_port *vport) +{ + int ret; + char *socket_path = vport->ifname; + int did = vport->did; + + if (client_mode) + vport->flags |= RTE_VHOST_USER_CLIENT; + + if (access(socket_path, F_OK) != -1 && !client_mode) { + RTE_LOG(ERR, VDPA, + "%s exists, please remove it or specify another file and try again.\n", + socket_path); + return -1; + } + ret = rte_vhost_driver_register(socket_path, vport->flags); + if (ret != 0) + rte_exit(EXIT_FAILURE, + "register driver failed: %s\n", + socket_path); + + ret = rte_vhost_driver_callback_register(socket_path, + &vdpa_sample_devops); + if (ret != 0) + rte_exit(EXIT_FAILURE, + "register driver ops failed: %s\n", + socket_path); + + ret = rte_vhost_driver_attach_vdpa_device(socket_path, did); + if (ret != 0) + rte_exit(EXIT_FAILURE, + "attach vdpa device failed: %s\n", + socket_path); + + if (rte_vhost_driver_start(socket_path) < 0) + rte_exit(EXIT_FAILURE, + "start vhost driver failed: %s\n", + socket_path); + return 0; +} + +static void +close_vdpa(struct vdpa_port *vport) +{ + int ret; + char *socket_path = vport->ifname; + + ret = rte_vhost_driver_detach_vdpa_device(socket_path); + if (ret != 0) + RTE_LOG(ERR, VDPA, + "detach vdpa device failed: %s\n", + socket_path); + + ret = rte_vhost_driver_unregister(socket_path); + if (ret != 0) + RTE_LOG(ERR, VDPA, + "Fail to unregister vhost driver for %s.\n", + socket_path); +} + +static void +vdpa_sample_quit(void) +{ + int i; + for (i = 0; i < RTE_MIN(MAX_VDPA_SAMPLE_PORTS, dev_total); i++) { + if (vports[i].ifname[0] != '\0') + close_vdpa(&vports[i]); + } +} + +static void +signal_handler(int signum) +{ + if (signum == SIGINT || signum == SIGTERM) { + printf("\nSignal %d received, preparing to exit...\n", signum); + vdpa_sample_quit(); + exit(0); + } +} + +/* interactive cmds */ + +/* *** Help command with introduction. *** */ +struct cmd_help_result { + cmdline_fixed_string_t help; +}; + +static void cmd_help_parsed(__attribute__((unused)) void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + cmdline_printf( + cl, + "\n" + "The following commands are currently available:\n\n" + "Control:\n" + " help : Show interactive instructions.\n" + " list : list all available vdpa devices.\n" + " create <socket file> <vdev addr> : create a new vdpa port.\n" + " quit : exit vdpa sample app.\n" + ); +} + +cmdline_parse_token_string_t cmd_help_help = + TOKEN_STRING_INITIALIZER(struct cmd_help_result, help, "help"); + +cmdline_parse_inst_t cmd_help = { + .f = cmd_help_parsed, + .data = NULL, + .help_str = "show help", + .tokens = { + (void *)&cmd_help_help, + NULL, + }, +}; + +/* *** List all available vdpa devices *** */ +struct cmd_list_result { + cmdline_fixed_string_t action; +}; + +static void cmd_list_vdpa_devices_parsed( + __attribute__((unused)) void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + int did; + uint32_t queue_num; + uint64_t features; + struct rte_vdpa_device *vdev; + struct rte_pci_addr addr; + + cmdline_printf(cl, "device id\tdevice address\tqueue num\tsupported features\n"); + for (did = 0; did < dev_total; did++) { + vdev = rte_vdpa_get_device(did); + if (!vdev) + continue; + if (vdev->ops->get_queue_num(did, &queue_num) < 0) { + RTE_LOG(ERR, VDPA, + "failed to get vdpa queue number " + "for device id %d.\n", did); + continue; + } + if (vdev->ops->get_features(did, &features) < 0) { + RTE_LOG(ERR, VDPA, + "failed to get vdpa features " + "for device id %d.\n", did); + continue; + } + addr = vdev->addr.pci_addr; + cmdline_printf(cl, + "%d\t\t" PCI_PRI_FMT "\t%" PRIu32 "\t\t0x%" PRIu64 "\n", did, + addr.domain, addr.bus, addr.devid, + addr.function, queue_num, features); + } +} + +cmdline_parse_token_string_t cmd_action_list = + TOKEN_STRING_INITIALIZER(struct cmd_list_result, action, "list"); + +cmdline_parse_inst_t cmd_list_vdpa_devices = { + .f = cmd_list_vdpa_devices_parsed, + .data = NULL, + .help_str = "list all available vdpa devices", + .tokens = { + (void *)&cmd_action_list, + NULL, + }, +}; + +/* *** Create new vdpa port *** */ +struct cmd_create_result { + cmdline_fixed_string_t action; + cmdline_fixed_string_t socket_path; + cmdline_fixed_string_t bdf; +}; + +static void cmd_create_vdpa_port_parsed(void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + int did; + struct cmd_create_result *res = parsed_result; + struct rte_vdpa_dev_addr addr; + + strncpy(vports[devcnt].ifname, res->socket_path, MAX_PATH_LEN); + if (rte_pci_addr_parse(res->bdf, &addr.pci_addr) != 0) { + cmdline_printf(cl, "Unable to parse the given bdf.\n"); + return; + } + addr.type = PCI_ADDR; + did = rte_vdpa_find_device_id(&addr); + if (did < 0) { + cmdline_printf(cl, "Unable to find vdpa devide id.\n"); + return; + } + + vports[devcnt].did = did; + + if (start_vdpa(&vports[devcnt]) == 0) + devcnt++; +} + +cmdline_parse_token_string_t cmd_action_create = + TOKEN_STRING_INITIALIZER(struct cmd_create_result, action, "create"); +cmdline_parse_token_string_t cmd_socket_path = + TOKEN_STRING_INITIALIZER(struct cmd_create_result, socket_path, NULL); +cmdline_parse_token_string_t cmd_bdf = + TOKEN_STRING_INITIALIZER(struct cmd_create_result, bdf, NULL); + +cmdline_parse_inst_t cmd_create_vdpa_port = { + .f = cmd_create_vdpa_port_parsed, + .data = NULL, + .help_str = "create a new vdpa port", + .tokens = { + (void *)&cmd_action_create, + (void *)&cmd_socket_path, + (void *)&cmd_bdf, + NULL, + }, +}; + +/* *** QUIT *** */ +struct cmd_quit_result { + cmdline_fixed_string_t quit; +}; + +static void cmd_quit_parsed(__attribute__((unused)) void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + vdpa_sample_quit(); + cmdline_quit(cl); +} + +cmdline_parse_token_string_t cmd_quit_quit = + TOKEN_STRING_INITIALIZER(struct cmd_quit_result, quit, "quit"); + +cmdline_parse_inst_t cmd_quit = { + .f = cmd_quit_parsed, + .data = NULL, + .help_str = "quit: exit application", + .tokens = { + (void *)&cmd_quit_quit, + NULL, + }, +}; +cmdline_parse_ctx_t main_ctx[] = { + (cmdline_parse_inst_t *)&cmd_help, + (cmdline_parse_inst_t *)&cmd_list_vdpa_devices, + (cmdline_parse_inst_t *)&cmd_create_vdpa_port, + (cmdline_parse_inst_t *)&cmd_quit, + NULL, +}; + +int +main(int argc, char *argv[]) +{ + char ch; + int i; + int ret; + struct cmdline *cl; + + ret = rte_eal_init(argc, argv); + if (ret < 0) + rte_exit(EXIT_FAILURE, "eal init failed\n"); + argc -= ret; + argv += ret; + + data_init(); + + signal(SIGINT, signal_handler); + signal(SIGTERM, signal_handler); + + ret = parse_args(argc, argv); + if (ret < 0) + rte_exit(EXIT_FAILURE, "invalid argument\n"); + + if (interactive == 1) { + cl = cmdline_stdin_new(main_ctx, "vdpa> "); + if (cl == NULL) + rte_panic("Cannot create cmdline instance\n"); + cmdline_interact(cl); + cmdline_stdin_exit(cl); + } else { + for (i = 0; i < RTE_MIN(MAX_VDPA_SAMPLE_PORTS, dev_total); + i++) { + vports[i].did = i; + snprintf(vports[i].ifname, MAX_PATH_LEN, "%s%d", + iface, i); + + start_vdpa(&vports[i]); + } + + printf("enter \'q\' to quit\n"); + while (scanf("%c", &ch)) { + if (ch == 'q') + break; + while (ch != '\n') { + if (scanf("%c", &ch)) + printf("%c", ch); + } + printf("enter \'q\' to quit\n"); + } + vdpa_sample_quit(); + } + + return 0; +} diff --git a/examples/vdpa/meson.build b/examples/vdpa/meson.build new file mode 100644 index 000000000..2e38a069f --- /dev/null +++ b/examples/vdpa/meson.build @@ -0,0 +1,16 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +# meson file, for building this example as part of a main DPDK build. +# +# To build this example as a standalone application with an already-installed +# DPDK instance, use 'make' + +if host_machine.system() != 'linux' + build = false +endif +deps += 'vhost' +allow_experimental_apis = true +sources = files( + 'main.c' +) \ No newline at end of file -- 2.17.1 ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [dpdk-dev] [PATCH v5 2/2] examples/vdpa: introduce a new sample for vDPA 2018-09-25 12:07 ` [dpdk-dev] [PATCH v5 2/2] examples/vdpa: introduce a new sample for vDPA Xiaolong Ye @ 2018-09-25 8:32 ` Wang, Xiao W 0 siblings, 0 replies; 41+ messages in thread From: Wang, Xiao W @ 2018-09-25 8:32 UTC (permalink / raw) To: Ye, Xiaolong, dev, Maxime Coquelin, Bie, Tiwei, Wang, Zhihong Cc: Rami Rosen, Wang, Haiyue Hi, > -----Original Message----- > From: Ye, Xiaolong > Sent: Tuesday, September 25, 2018 8:07 PM > To: dev@dpdk.org; Maxime Coquelin <maxime.coquelin@redhat.com>; Bie, > Tiwei <tiwei.bie@intel.com>; Wang, Zhihong <zhihong.wang@intel.com> > Cc: Wang, Xiao W <xiao.w.wang@intel.com>; Rami Rosen > <roszenrami@gmail.com>; Wang, Haiyue <haiyue.wang@intel.com>; Ye, > Xiaolong <xiaolong.ye@intel.com> > Subject: [PATCH v5 2/2] examples/vdpa: introduce a new sample for vDPA > > The vdpa sample application creates vhost-user sockets by using the > vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes > virtio ring compatible devices to serve virtio driver directly to enable > datapath acceleration. As vDPA driver can help to set up vhost datapath, > this application doesn't need to launch dedicated worker threads for vhost > enqueue/dequeue operations. > > Signed-off-by: Xiao Wang <xiao.w.wang@intel.com> > Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com> > --- > MAINTAINERS | 2 + > doc/guides/rel_notes/release_18_11.rst | 8 + > doc/guides/sample_app_ug/index.rst | 1 + > doc/guides/sample_app_ug/vdpa.rst | 118 +++++++ > examples/Makefile | 2 +- > examples/vdpa/Makefile | 32 ++ > examples/vdpa/main.c | 466 +++++++++++++++++++++++++ > examples/vdpa/meson.build | 16 + > 8 files changed, 644 insertions(+), 1 deletion(-) > create mode 100644 doc/guides/sample_app_ug/vdpa.rst > create mode 100644 examples/vdpa/Makefile > create mode 100644 examples/vdpa/main.c > create mode 100644 examples/vdpa/meson.build > > diff --git a/MAINTAINERS b/MAINTAINERS > index 5967c1dd3..5656f18e8 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -683,6 +683,8 @@ F: doc/guides/sample_app_ug/vhost.rst > F: examples/vhost_scsi/ > F: doc/guides/sample_app_ug/vhost_scsi.rst > F: examples/vhost_crypto/ > +F: examples/vdpa/ > +F: doc/guides/sample_app_ug/vdpa.rst > > Vhost PMD > M: Maxime Coquelin <maxime.coquelin@redhat.com> > diff --git a/doc/guides/rel_notes/release_18_11.rst > b/doc/guides/rel_notes/release_18_11.rst > index bc9b74ec4..dd53a9ecf 100644 > --- a/doc/guides/rel_notes/release_18_11.rst > +++ b/doc/guides/rel_notes/release_18_11.rst > @@ -67,6 +67,14 @@ New Features > SR-IOV option in Hyper-V and Azure. This is an alternative to the previous > vdev_netvsc, tap, and failsafe drivers combination. > > +* **Add a new sample for vDPA** > + > + The vdpa sample application creates vhost-user sockets by using the > + vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes > + virtio ring compatible devices to serve virtio driver directly to enable > + datapath acceleration. As vDPA driver can help to set up vhost datapath, > + this application doesn't need to launch dedicated worker threads for vhost > + enqueue/dequeue operations. > > API Changes > ----------- > diff --git a/doc/guides/sample_app_ug/index.rst > b/doc/guides/sample_app_ug/index.rst > index 5bedf4f6f..74b12af85 100644 > --- a/doc/guides/sample_app_ug/index.rst > +++ b/doc/guides/sample_app_ug/index.rst > @@ -45,6 +45,7 @@ Sample Applications User Guides > vhost > vhost_scsi > vhost_crypto > + vdpa > netmap_compatibility > ip_pipeline > test_pipeline > diff --git a/doc/guides/sample_app_ug/vdpa.rst > b/doc/guides/sample_app_ug/vdpa.rst > new file mode 100644 > index 000000000..a089393c0 > --- /dev/null > +++ b/doc/guides/sample_app_ug/vdpa.rst > @@ -0,0 +1,118 @@ > +.. SPDX-License-Identifier: BSD-3-Clause > + Copyright(c) 2018 Intel Corporation. > + > +Vdpa Sample Application > +======================= > + > +The vdpa sample application creates vhost-user sockets by using the > +vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes > +virtio ring compatible devices to serve virtio driver directly to enable > +datapath acceleration. As vDPA driver can help to set up vhost datapath, > +this application doesn't need to launch dedicated worker threads for vhost > +enqueue/dequeue operations. > + > +Testing steps > +------------- > + > +This section shows the steps of how to start VMs with vDPA vhost-user > +backend and verify network connection & live migration. > + > +Build > +~~~~~ > + > +To compile the sample application see :doc:`compiling`. > + > +The application is located in the ``vdpa`` sub-directory. > + > +Start the vdpa example > +~~~~~~~~~~~~~~~~~~~~~~ > + > +.. code-block:: console > + > + ./vdpa [EAL options] -- [--client] [--interactive|-i] or [--iface > SOCKET_PATH] > + > +where > + > +* --client means running vdpa app in client mode, in the client mode, QEMU > needs > + to run as the server mode and take charge of socket file creation. > +* --iface specifies the path prefix of the UNIX domain socket file, e.g. > + /tmp/vhost-user-, then the socket files will be named as /tmp/vhost-user-<n> > + (n starts from 0). > +* --interactive means run the vdpa sample in interactive mode, currently 4 > + internal cmds are supported: > + > + 1. help: show help message > + 2. list: list all available vdpa devices > + 3. create: create a new vdpa port with socket file and vdpa device address > + 4. quit: unregister vhost driver and exit the application > + > +Take IFCVF driver for example: > + > +.. code-block:: console > + > + ./vdpa --log-level=9 -c 0x2 -n 4 --socket-mem 1024,1024 \ > + -w 0000:06:00.3,vdpa=1 -w 0000:06:00.4,vdpa=1 \ > + -- --interactive DPDK has log level 1-8, setting the log-level=9 looks strange. Here you can add description about the whitelisted device, since they are virtio ring compatible devices. > + > +.. note:: > + We need to bind vfio-pci to VFs before running vdpa sample. > + > + * modprobe vfio-pci > + * ./usertools/dpdk-devbind.py -b vfio-pci 06:00.3 06:00.4 > + > +Then we can create 2 vdpa ports in interactive cmdline. > + > +.. code-block:: console > + > + vdpa> list > + device id device address queue num supported features > + 0 0000:06:00.3 1 0x5572362272 > + 1 0000:06:00.4 1 0x5572362272 > + > + vdpa> create /tmp/vdpa-socket0 0000:06:00.3 > + vdpa> create /tmp/vdpa-socket1 0000:06:00.4 > + > +.. _vdpa_app_run_vm: > + > +Start the VMs > +~~~~~~~~~~~~~ > + > +.. code-block:: console > + > + qemu-system-x86_64 -cpu host -enable-kvm \ > + <snip> > + -mem-prealloc \ > + -chardev socket,id=char0,path=<socket_file created in above steps> \ > + -netdev type=vhost-user,id=vdpa,chardev=char0 \ > + -device virtio-net-pci,netdev=vdpa,mac=00:aa:bb:cc:dd:ee \ In order to leverage EPT, add parameter "page-per-vq=on" for the virtio-net-pci deivce. > + > +After the VMs launches, we can login the VMs and configure the ip, verify the > +network connection via ping or netperf. > + > +.. note:: > + Suggest to use QEMU 3.0.0 which extends vhost-user for vDPA. > + > +Live Migration > +~~~~~~~~~~~~~~ > +vDPA supports cross-backend live migration, user can migrate SW vhost > backend > +VM to vDPA backend VM and vice versa. Here are the detailed steps. Assume > A is > +the source host with SW vhost VM and B is the destination host with vDPA. > + > +1. Start vdpa sample and launch a VM with exact same parameters as the VM > on A, > + in migration-listen mode: > + > +.. code-block:: console > + > + B: <qemu-command-line> -incoming tcp:0:4444 (or other PORT)) > + > +2. Start the migration (on source host): > + > +.. code-block:: console > + > + A: (qemu) migrate -d tcp:<B ip>:4444 (or other PORT) > + > +3. Check the status (on source host): > + > +.. code-block:: console > + > + A: (qemu) info migrate > diff --git a/examples/Makefile b/examples/Makefile > index 481720cb6..356fcb1cd 100644 > --- a/examples/Makefile > +++ b/examples/Makefile > @@ -65,7 +65,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_HASH),y) > DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += tep_termination > endif > DIRS-$(CONFIG_RTE_LIBRTE_TIMER) += timer > -DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi > +DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi vdpa > ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y) > DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_crypto > endif > diff --git a/examples/vdpa/Makefile b/examples/vdpa/Makefile > new file mode 100644 > index 000000000..42672a2bc > --- /dev/null > +++ b/examples/vdpa/Makefile > @@ -0,0 +1,32 @@ > +# SPDX-License-Identifier: BSD-3-Clause > +# Copyright(c) 2018 Intel Corporation > + > +ifeq ($(RTE_SDK),) > +$(error "Please define RTE_SDK environment variable") > +endif > + > +# Default target, can be overridden by command line or environment > +RTE_TARGET ?= x86_64-native-linuxapp-gcc > + > +include $(RTE_SDK)/mk/rte.vars.mk > + > +ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp") > +$(info This application can only operate in a linuxapp environment, \ > +please change the definition of the RTE_TARGET environment variable) > +all: > +else > + > +# binary name > +APP = vdpa > + > +# all source are stored in SRCS-y > +SRCS-y := main.c > + > +CFLAGS += -O2 -D_FILE_OFFSET_BITS=64 > +CFLAGS += $(WERROR_FLAGS) > +CFLAGS += -D_GNU_SOURCE > +CFLAGS += -DALLOW_EXPERIMENTAL_API > + > +include $(RTE_SDK)/mk/rte.extapp.mk > + > +endif > diff --git a/examples/vdpa/main.c b/examples/vdpa/main.c > new file mode 100644 > index 000000000..7acda67c4 > --- /dev/null > +++ b/examples/vdpa/main.c > @@ -0,0 +1,466 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(c) 2018 Intel Corporation > + */ > + > +#include <getopt.h> > +#include <signal.h> > +#include <stdint.h> > +#include <string.h> > +#include <unistd.h> > + > +#include <rte_ethdev.h> > +#include <rte_malloc.h> > +#include <rte_vhost.h> > +#include <rte_vdpa.h> > +#include <rte_pci.h> > + > +#include <cmdline_parse.h> > +#include <cmdline_socket.h> > +#include <cmdline_parse_string.h> > +#include <cmdline.h> > + > +#define MAX_PATH_LEN 128 > +#define MAX_VDPA_SAMPLE_PORTS 1024 > +#define RTE_LOGTYPE_VDPA RTE_LOGTYPE_USER1 > + [...] > +}; > + > +/* *** List all available vdpa devices *** */ > +struct cmd_list_result { > + cmdline_fixed_string_t action; > +}; > + > +static void cmd_list_vdpa_devices_parsed( > + __attribute__((unused)) void *parsed_result, > + struct cmdline *cl, > + __attribute__((unused)) void *data) > +{ > + int did; > + uint32_t queue_num; > + uint64_t features; > + struct rte_vdpa_device *vdev; > + struct rte_pci_addr addr; > + > + cmdline_printf(cl, "device id\tdevice address\tqueue num\tsupported > features\n"); > + for (did = 0; did < dev_total; did++) { > + vdev = rte_vdpa_get_device(did); > + if (!vdev) > + continue; > + if (vdev->ops->get_queue_num(did, &queue_num) < 0) { > + RTE_LOG(ERR, VDPA, > + "failed to get vdpa queue number " > + "for device id %d.\n", did); > + continue; > + } > + if (vdev->ops->get_features(did, &features) < 0) { > + RTE_LOG(ERR, VDPA, > + "failed to get vdpa features " > + "for device id %d.\n", did); > + continue; > + } > + addr = vdev->addr.pci_addr; > + cmdline_printf(cl, > + "%d\t\t" PCI_PRI_FMT "\t%" PRIu32 "\t\t0x%" PRIu64 > "\n", did, s/PRIu64/PRIx64 > + addr.domain, addr.bus, addr.devid, > + addr.function, queue_num, features); > + } > +} BRs, Xiao ^ permalink raw reply [flat|nested] 41+ messages in thread
* [dpdk-dev] [PATCH v6 0/2] introduce vdpa sample 2018-09-25 12:07 ` [dpdk-dev] [PATCH v5 0/2] introduce vdpa sample Xiaolong Ye 2018-09-25 12:07 ` [dpdk-dev] [PATCH v5 1/2] vhost: introduce API to get vDPA device number Xiaolong Ye 2018-09-25 12:07 ` [dpdk-dev] [PATCH v5 2/2] examples/vdpa: introduce a new sample for vDPA Xiaolong Ye @ 2018-09-26 9:06 ` Xiaolong Ye 2018-09-26 3:15 ` Wang, Xiao W ` (3 more replies) 2 siblings, 4 replies; 41+ messages in thread From: Xiaolong Ye @ 2018-09-26 9:06 UTC (permalink / raw) To: dev, Maxime Coquelin, Tiwei Bie, Zhihong Wang Cc: xiao.w.wang, Rami Rosen, Wang Haiyue, Xiaolong Ye Hi, This patchset introduces vdpa sample to demonstrate the vDPA use case. v6 changes: * improve the document according to Xiao's comments * fix a typo, PRIu64 -> PRIx64 v5 changes: * improve print format and correct from "PRIu64" to "PRIx64" * use "-c 0x2" to better demonstrate app doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations in vdpa.rst v4 changes: * add client mode support * improve the format to list the vDPA device info and improve the vdpa.rst accordingly * remove some useless comments * add introduction in 18.11 release note. v3 changes: * list cmd would show queue number and supported features of vdpa devices. * address Xiao's review comments v2 changes: * fix a compilation error reported by Rosen * improve create cmd in interactive mode and add two new cmds: list, quit * add application documentation Xiaolong Ye (2): vhost: introduce API to get vDPA device number examples/vdpa: introduce a new sample for vDPA MAINTAINERS | 2 + doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/vdpa.rst | 115 +++++++ examples/Makefile | 2 +- examples/vdpa/Makefile | 32 ++ examples/vdpa/main.c | 458 +++++++++++++++++++++++++ examples/vdpa/meson.build | 16 + lib/librte_vhost/rte_vdpa.h | 3 + lib/librte_vhost/rte_vhost_version.map | 1 + lib/librte_vhost/vdpa.c | 6 + 10 files changed, 635 insertions(+), 1 deletion(-) create mode 100644 doc/guides/sample_app_ug/vdpa.rst create mode 100644 examples/vdpa/Makefile create mode 100644 examples/vdpa/main.c create mode 100644 examples/vdpa/meson.build -- 2.17.1 ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [dpdk-dev] [PATCH v6 0/2] introduce vdpa sample 2018-09-26 9:06 ` [dpdk-dev] [PATCH v6 0/2] introduce vdpa sample Xiaolong Ye @ 2018-09-26 3:15 ` Wang, Xiao W 2018-09-26 9:06 ` [dpdk-dev] [PATCH v6 1/2] vhost: introduce API to get vDPA device number Xiaolong Ye ` (2 subsequent siblings) 3 siblings, 0 replies; 41+ messages in thread From: Wang, Xiao W @ 2018-09-26 3:15 UTC (permalink / raw) To: Ye, Xiaolong, dev, Maxime Coquelin, Bie, Tiwei, Wang, Zhihong Cc: Rami Rosen, Wang, Haiyue Hi, > -----Original Message----- > From: Ye, Xiaolong > Sent: Wednesday, September 26, 2018 5:07 PM > To: dev@dpdk.org; Maxime Coquelin <maxime.coquelin@redhat.com>; Bie, > Tiwei <tiwei.bie@intel.com>; Wang, Zhihong <zhihong.wang@intel.com> > Cc: Wang, Xiao W <xiao.w.wang@intel.com>; Rami Rosen > <roszenrami@gmail.com>; Wang, Haiyue <haiyue.wang@intel.com>; Ye, > Xiaolong <xiaolong.ye@intel.com> > Subject: [PATCH v6 0/2] introduce vdpa sample > > Hi, > > This patchset introduces vdpa sample to demonstrate the vDPA use case. > > v6 changes: > * improve the document according to Xiao's comments > * fix a typo, PRIu64 -> PRIx64 > > v5 changes: > * improve print format and correct from "PRIu64" to "PRIx64" > * use "-c 0x2" to better demonstrate app doesn't need to launch dedicated > worker threads for vhost enqueue/dequeue operations in vdpa.rst > > v4 changes: > * add client mode support > * improve the format to list the vDPA device info and improve the vdpa.rst > accordingly > * remove some useless comments > * add introduction in 18.11 release note. > > > v3 changes: > * list cmd would show queue number and supported features of vdpa devices. > * address Xiao's review comments > > v2 changes: > > * fix a compilation error reported by Rosen > * improve create cmd in interactive mode and add two new cmds: list, quit > * add application documentation > > > Xiaolong Ye (2): > vhost: introduce API to get vDPA device number > examples/vdpa: introduce a new sample for vDPA > > MAINTAINERS | 2 + > doc/guides/sample_app_ug/index.rst | 1 + > doc/guides/sample_app_ug/vdpa.rst | 115 +++++++ > examples/Makefile | 2 +- > examples/vdpa/Makefile | 32 ++ > examples/vdpa/main.c | 458 +++++++++++++++++++++++++ > examples/vdpa/meson.build | 16 + > lib/librte_vhost/rte_vdpa.h | 3 + > lib/librte_vhost/rte_vhost_version.map | 1 + > lib/librte_vhost/vdpa.c | 6 + > 10 files changed, 635 insertions(+), 1 deletion(-) > create mode 100644 doc/guides/sample_app_ug/vdpa.rst > create mode 100644 examples/vdpa/Makefile > create mode 100644 examples/vdpa/main.c > create mode 100644 examples/vdpa/meson.build > > -- > 2.17.1 Series Acked-by: Xiao Wang <xiao.w.wang@intel.com> BRs, Xiao ^ permalink raw reply [flat|nested] 41+ messages in thread
* [dpdk-dev] [PATCH v6 1/2] vhost: introduce API to get vDPA device number 2018-09-26 9:06 ` [dpdk-dev] [PATCH v6 0/2] introduce vdpa sample Xiaolong Ye 2018-09-26 3:15 ` Wang, Xiao W @ 2018-09-26 9:06 ` Xiaolong Ye 2018-09-26 9:06 ` [dpdk-dev] [PATCH v6 2/2] examples/vdpa: introduce a new sample for vDPA Xiaolong Ye 2018-09-28 11:23 ` [dpdk-dev] [PATCH v7 0/2] introduce vdpa sample Xiaolong Ye 3 siblings, 0 replies; 41+ messages in thread From: Xiaolong Ye @ 2018-09-26 9:06 UTC (permalink / raw) To: dev, Maxime Coquelin, Tiwei Bie, Zhihong Wang Cc: xiao.w.wang, Rami Rosen, Wang Haiyue, Xiaolong Ye It's used to get number of available registered vDPA devices. Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com> --- lib/librte_vhost/rte_vdpa.h | 3 +++ lib/librte_vhost/rte_vhost_version.map | 1 + lib/librte_vhost/vdpa.c | 6 ++++++ 3 files changed, 10 insertions(+) diff --git a/lib/librte_vhost/rte_vdpa.h b/lib/librte_vhost/rte_vdpa.h index 90465ca26..b8223e337 100644 --- a/lib/librte_vhost/rte_vdpa.h +++ b/lib/librte_vhost/rte_vdpa.h @@ -84,4 +84,7 @@ rte_vdpa_find_device_id(struct rte_vdpa_dev_addr *addr); struct rte_vdpa_device * __rte_experimental rte_vdpa_get_device(int did); +/* Get current available vdpa device number */ +int __rte_experimental +rte_vdpa_get_device_num(void); #endif /* _RTE_VDPA_H_ */ diff --git a/lib/librte_vhost/rte_vhost_version.map b/lib/librte_vhost/rte_vhost_version.map index da220dd02..ae39b6e21 100644 --- a/lib/librte_vhost/rte_vhost_version.map +++ b/lib/librte_vhost/rte_vhost_version.map @@ -67,6 +67,7 @@ EXPERIMENTAL { rte_vdpa_unregister_device; rte_vdpa_find_device_id; rte_vdpa_get_device; + rte_vdpa_get_device_num; rte_vhost_driver_attach_vdpa_device; rte_vhost_driver_detach_vdpa_device; rte_vhost_driver_get_vdpa_device_id; diff --git a/lib/librte_vhost/vdpa.c b/lib/librte_vhost/vdpa.c index c82fd4370..c2c5dff1d 100644 --- a/lib/librte_vhost/vdpa.c +++ b/lib/librte_vhost/vdpa.c @@ -113,3 +113,9 @@ rte_vdpa_get_device(int did) return vdpa_devices[did]; } + +int +rte_vdpa_get_device_num(void) +{ + return vdpa_device_num; +} -- 2.17.1 ^ permalink raw reply [flat|nested] 41+ messages in thread
* [dpdk-dev] [PATCH v6 2/2] examples/vdpa: introduce a new sample for vDPA 2018-09-26 9:06 ` [dpdk-dev] [PATCH v6 0/2] introduce vdpa sample Xiaolong Ye 2018-09-26 3:15 ` Wang, Xiao W 2018-09-26 9:06 ` [dpdk-dev] [PATCH v6 1/2] vhost: introduce API to get vDPA device number Xiaolong Ye @ 2018-09-26 9:06 ` Xiaolong Ye 2018-09-27 16:37 ` Maxime Coquelin 2018-09-28 11:23 ` [dpdk-dev] [PATCH v7 0/2] introduce vdpa sample Xiaolong Ye 3 siblings, 1 reply; 41+ messages in thread From: Xiaolong Ye @ 2018-09-26 9:06 UTC (permalink / raw) To: dev, Maxime Coquelin, Tiwei Bie, Zhihong Wang Cc: xiao.w.wang, Rami Rosen, Wang Haiyue, Xiaolong Ye The vdpa sample application creates vhost-user sockets by using the vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio ring compatible devices to serve virtio driver directly to enable datapath acceleration. As vDPA driver can help to set up vhost datapath, this application doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations. Signed-off-by: Xiao Wang <xiao.w.wang@intel.com> Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com> --- MAINTAINERS | 2 + doc/guides/rel_notes/release_18_11.rst | 8 + doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/vdpa.rst | 120 +++++++ examples/Makefile | 2 +- examples/vdpa/Makefile | 32 ++ examples/vdpa/main.c | 466 +++++++++++++++++++++++++ examples/vdpa/meson.build | 16 + 8 files changed, 646 insertions(+), 1 deletion(-) create mode 100644 doc/guides/sample_app_ug/vdpa.rst create mode 100644 examples/vdpa/Makefile create mode 100644 examples/vdpa/main.c create mode 100644 examples/vdpa/meson.build diff --git a/MAINTAINERS b/MAINTAINERS index 5967c1dd3..5656f18e8 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -683,6 +683,8 @@ F: doc/guides/sample_app_ug/vhost.rst F: examples/vhost_scsi/ F: doc/guides/sample_app_ug/vhost_scsi.rst F: examples/vhost_crypto/ +F: examples/vdpa/ +F: doc/guides/sample_app_ug/vdpa.rst Vhost PMD M: Maxime Coquelin <maxime.coquelin@redhat.com> diff --git a/doc/guides/rel_notes/release_18_11.rst b/doc/guides/rel_notes/release_18_11.rst index bc9b74ec4..dd53a9ecf 100644 --- a/doc/guides/rel_notes/release_18_11.rst +++ b/doc/guides/rel_notes/release_18_11.rst @@ -67,6 +67,14 @@ New Features SR-IOV option in Hyper-V and Azure. This is an alternative to the previous vdev_netvsc, tap, and failsafe drivers combination. +* **Add a new sample for vDPA** + + The vdpa sample application creates vhost-user sockets by using the + vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes + virtio ring compatible devices to serve virtio driver directly to enable + datapath acceleration. As vDPA driver can help to set up vhost datapath, + this application doesn't need to launch dedicated worker threads for vhost + enqueue/dequeue operations. API Changes ----------- diff --git a/doc/guides/sample_app_ug/index.rst b/doc/guides/sample_app_ug/index.rst index 5bedf4f6f..74b12af85 100644 --- a/doc/guides/sample_app_ug/index.rst +++ b/doc/guides/sample_app_ug/index.rst @@ -45,6 +45,7 @@ Sample Applications User Guides vhost vhost_scsi vhost_crypto + vdpa netmap_compatibility ip_pipeline test_pipeline diff --git a/doc/guides/sample_app_ug/vdpa.rst b/doc/guides/sample_app_ug/vdpa.rst new file mode 100644 index 000000000..745f196ca --- /dev/null +++ b/doc/guides/sample_app_ug/vdpa.rst @@ -0,0 +1,120 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2018 Intel Corporation. + +Vdpa Sample Application +======================= + +The vdpa sample application creates vhost-user sockets by using the +vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes +virtio ring compatible devices to serve virtio driver directly to enable +datapath acceleration. As vDPA driver can help to set up vhost datapath, +this application doesn't need to launch dedicated worker threads for vhost +enqueue/dequeue operations. + +Testing steps +------------- + +This section shows the steps of how to start VMs with vDPA vhost-user +backend and verify network connection & live migration. + +Build +~~~~~ + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``vdpa`` sub-directory. + +Start the vdpa example +~~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: console + + ./vdpa [EAL options] -- [--client] [--interactive|-i] or [--iface SOCKET_PATH] + +where + +* --client means running vdpa app in client mode, in the client mode, QEMU needs + to run as the server mode and take charge of socket file creation. +* --iface specifies the path prefix of the UNIX domain socket file, e.g. + /tmp/vhost-user-, then the socket files will be named as /tmp/vhost-user-<n> + (n starts from 0). +* --interactive means run the vdpa sample in interactive mode, currently 4 + internal cmds are supported: + + 1. help: show help message + 2. list: list all available vdpa devices + 3. create: create a new vdpa port with socket file and vdpa device address + 4. quit: unregister vhost driver and exit the application + +Take IFCVF driver for example: + +.. code-block:: console + + ./vdpa -c 0x2 -n 4 --socket-mem 1024,1024 \ + -w 0000:06:00.3,vdpa=1 -w 0000:06:00.4,vdpa=1 \ + -- --interactive + +.. note:: + Here 0000:06:00.3 and 0000:06:00.4 refer to virtio ring compatible devices, + and we need to bind vfio-pci to them before running vdpa sample. + + * modprobe vfio-pci + * ./usertools/dpdk-devbind.py -b vfio-pci 06:00.3 06:00.4 + +Then we can create 2 vdpa ports in interactive cmdline. + +.. code-block:: console + + vdpa> list + device id device address queue num supported features + 0 0000:06:00.3 1 0x14c238020 + 1 0000:06:00.4 1 0x14c238020 + 2 0000:06:00.5 1 0x14c238020 + + vdpa> create /tmp/vdpa-socket0 0000:06:00.3 + vdpa> create /tmp/vdpa-socket1 0000:06:00.4 + +.. _vdpa_app_run_vm: + +Start the VMs +~~~~~~~~~~~~~ + +.. code-block:: console + + qemu-system-x86_64 -cpu host -enable-kvm \ + <snip> + -mem-prealloc \ + -chardev socket,id=char0,path=<socket_file created in above steps> \ + -netdev type=vhost-user,id=vdpa,chardev=char0 \ + -device virtio-net-pci,netdev=vdpa,mac=00:aa:bb:cc:dd:ee,page-per-vq=on \ + +After the VMs launches, we can login the VMs and configure the ip, verify the +network connection via ping or netperf. + +.. note:: + Suggest to use QEMU 3.0.0 which extends vhost-user for vDPA. + +Live Migration +~~~~~~~~~~~~~~ +vDPA supports cross-backend live migration, user can migrate SW vhost backend +VM to vDPA backend VM and vice versa. Here are the detailed steps. Assume A is +the source host with SW vhost VM and B is the destination host with vDPA. + +1. Start vdpa sample and launch a VM with exact same parameters as the VM on A, + in migration-listen mode: + +.. code-block:: console + + B: <qemu-command-line> -incoming tcp:0:4444 (or other PORT)) + +2. Start the migration (on source host): + +.. code-block:: console + + A: (qemu) migrate -d tcp:<B ip>:4444 (or other PORT) + +3. Check the status (on source host): + +.. code-block:: console + + A: (qemu) info migrate diff --git a/examples/Makefile b/examples/Makefile index 481720cb6..356fcb1cd 100644 --- a/examples/Makefile +++ b/examples/Makefile @@ -65,7 +65,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_HASH),y) DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += tep_termination endif DIRS-$(CONFIG_RTE_LIBRTE_TIMER) += timer -DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi +DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi vdpa ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y) DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_crypto endif diff --git a/examples/vdpa/Makefile b/examples/vdpa/Makefile new file mode 100644 index 000000000..42672a2bc --- /dev/null +++ b/examples/vdpa/Makefile @@ -0,0 +1,32 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +ifeq ($(RTE_SDK),) +$(error "Please define RTE_SDK environment variable") +endif + +# Default target, can be overridden by command line or environment +RTE_TARGET ?= x86_64-native-linuxapp-gcc + +include $(RTE_SDK)/mk/rte.vars.mk + +ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp") +$(info This application can only operate in a linuxapp environment, \ +please change the definition of the RTE_TARGET environment variable) +all: +else + +# binary name +APP = vdpa + +# all source are stored in SRCS-y +SRCS-y := main.c + +CFLAGS += -O2 -D_FILE_OFFSET_BITS=64 +CFLAGS += $(WERROR_FLAGS) +CFLAGS += -D_GNU_SOURCE +CFLAGS += -DALLOW_EXPERIMENTAL_API + +include $(RTE_SDK)/mk/rte.extapp.mk + +endif diff --git a/examples/vdpa/main.c b/examples/vdpa/main.c new file mode 100644 index 000000000..898286e30 --- /dev/null +++ b/examples/vdpa/main.c @@ -0,0 +1,466 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2018 Intel Corporation + */ + +#include <getopt.h> +#include <signal.h> +#include <stdint.h> +#include <string.h> +#include <unistd.h> + +#include <rte_ethdev.h> +#include <rte_malloc.h> +#include <rte_vhost.h> +#include <rte_vdpa.h> +#include <rte_pci.h> + +#include <cmdline_parse.h> +#include <cmdline_socket.h> +#include <cmdline_parse_string.h> +#include <cmdline.h> + +#define MAX_PATH_LEN 128 +#define MAX_VDPA_SAMPLE_PORTS 1024 +#define RTE_LOGTYPE_VDPA RTE_LOGTYPE_USER1 + +struct vdpa_port { + char ifname[MAX_PATH_LEN]; + int did; + int vid; + uint64_t flags; +}; + +struct vdpa_port vports[MAX_VDPA_SAMPLE_PORTS]; + +static char iface[MAX_PATH_LEN]; +static int dev_total; +static int devcnt; +static int interactive; +static int client_mode; + +/* display usage */ +static void +vdpa_usage(const char *prgname) +{ + printf("Usage: %s [EAL options] -- " + " --interactive|-i: run in interactive mode.\n" + " --iface <path>: specify the path prefix of the socket files, e.g. /tmp/vhost-user-.\n" + " --client: register a vhost-user socket as client mode.\n", + prgname); +} + +static int +parse_args(int argc, char **argv) +{ + static const char *short_option = "i"; + static struct option long_option[] = { + {"iface", required_argument, NULL, 0}, + {"interactive", no_argument, &interactive, 1}, + {"client", no_argument, &client_mode, 1}, + {NULL, 0, 0, 0}, + }; + int opt, idx; + char *prgname = argv[0]; + + while ((opt = getopt_long(argc, argv, short_option, long_option, &idx)) + != EOF) { + switch (opt) { + case 'i': + printf("Interactive-mode selected\n"); + interactive = 1; + break; + /* long options */ + case 0: + if (strncmp(long_option[idx].name, "iface", + MAX_PATH_LEN) == 0) { + strncpy(iface, optarg, MAX_PATH_LEN); + printf("iface %s\n", iface); + } + if (!strcmp(long_option[idx].name, "interactive")) { + printf("Interactive-mode selected\n"); + interactive = 1; + } + break; + + default: + vdpa_usage(prgname); + return -1; + } + } + + if (iface[0] == '\0' && interactive == 0) { + vdpa_usage(prgname); + return -1; + } + + return 0; +} + +static int +data_init(void) +{ + dev_total = rte_vdpa_get_device_num(); + if (dev_total <= 0) { + printf("No available vdpa device found\n"); + return -1; + } + devcnt = 0; + interactive = 0; + memset(iface, 0, sizeof(iface)); + memset(vports, 0, sizeof(vports)); + return 0; +} + +static int +new_device(int vid) +{ + char ifname[MAX_PATH_LEN]; + int i; + + rte_vhost_get_ifname(vid, ifname, sizeof(ifname)); + for (i = 0; i < MAX_VDPA_SAMPLE_PORTS; i++) { + if (strncmp(ifname, vports[i].ifname, MAX_PATH_LEN) == 0) { + printf("\nnew port %s, did: %d\n", + ifname, vports[i].did); + vports[i].vid = vid; + break; + } + } + + if (i >= MAX_VDPA_SAMPLE_PORTS) + return -1; + + return 0; +} + +static void +destroy_device(int vid) +{ + char ifname[MAX_PATH_LEN]; + int i; + + rte_vhost_get_ifname(vid, ifname, sizeof(ifname)); + for (i = 0; i < MAX_VDPA_SAMPLE_PORTS; i++) { + if (strcmp(ifname, vports[i].ifname) == 0) { + printf("\ndestroy port %s, did: %d\n", + ifname, vports[i].did); + break; + } + } +} + +static const struct vhost_device_ops vdpa_sample_devops = { + .new_device = new_device, + .destroy_device = destroy_device, +}; + +static int +start_vdpa(struct vdpa_port *vport) +{ + int ret; + char *socket_path = vport->ifname; + int did = vport->did; + + if (client_mode) + vport->flags |= RTE_VHOST_USER_CLIENT; + + if (access(socket_path, F_OK) != -1 && !client_mode) { + RTE_LOG(ERR, VDPA, + "%s exists, please remove it or specify another file and try again.\n", + socket_path); + return -1; + } + ret = rte_vhost_driver_register(socket_path, vport->flags); + if (ret != 0) + rte_exit(EXIT_FAILURE, + "register driver failed: %s\n", + socket_path); + + ret = rte_vhost_driver_callback_register(socket_path, + &vdpa_sample_devops); + if (ret != 0) + rte_exit(EXIT_FAILURE, + "register driver ops failed: %s\n", + socket_path); + + ret = rte_vhost_driver_attach_vdpa_device(socket_path, did); + if (ret != 0) + rte_exit(EXIT_FAILURE, + "attach vdpa device failed: %s\n", + socket_path); + + if (rte_vhost_driver_start(socket_path) < 0) + rte_exit(EXIT_FAILURE, + "start vhost driver failed: %s\n", + socket_path); + return 0; +} + +static void +close_vdpa(struct vdpa_port *vport) +{ + int ret; + char *socket_path = vport->ifname; + + ret = rte_vhost_driver_detach_vdpa_device(socket_path); + if (ret != 0) + RTE_LOG(ERR, VDPA, + "detach vdpa device failed: %s\n", + socket_path); + + ret = rte_vhost_driver_unregister(socket_path); + if (ret != 0) + RTE_LOG(ERR, VDPA, + "Fail to unregister vhost driver for %s.\n", + socket_path); +} + +static void +vdpa_sample_quit(void) +{ + int i; + for (i = 0; i < RTE_MIN(MAX_VDPA_SAMPLE_PORTS, dev_total); i++) { + if (vports[i].ifname[0] != '\0') + close_vdpa(&vports[i]); + } +} + +static void +signal_handler(int signum) +{ + if (signum == SIGINT || signum == SIGTERM) { + printf("\nSignal %d received, preparing to exit...\n", signum); + vdpa_sample_quit(); + exit(0); + } +} + +/* interactive cmds */ + +/* *** Help command with introduction. *** */ +struct cmd_help_result { + cmdline_fixed_string_t help; +}; + +static void cmd_help_parsed(__attribute__((unused)) void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + cmdline_printf( + cl, + "\n" + "The following commands are currently available:\n\n" + "Control:\n" + " help : Show interactive instructions.\n" + " list : list all available vdpa devices.\n" + " create <socket file> <vdev addr> : create a new vdpa port.\n" + " quit : exit vdpa sample app.\n" + ); +} + +cmdline_parse_token_string_t cmd_help_help = + TOKEN_STRING_INITIALIZER(struct cmd_help_result, help, "help"); + +cmdline_parse_inst_t cmd_help = { + .f = cmd_help_parsed, + .data = NULL, + .help_str = "show help", + .tokens = { + (void *)&cmd_help_help, + NULL, + }, +}; + +/* *** List all available vdpa devices *** */ +struct cmd_list_result { + cmdline_fixed_string_t action; +}; + +static void cmd_list_vdpa_devices_parsed( + __attribute__((unused)) void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + int did; + uint32_t queue_num; + uint64_t features; + struct rte_vdpa_device *vdev; + struct rte_pci_addr addr; + + cmdline_printf(cl, "device id\tdevice address\tqueue num\tsupported features\n"); + for (did = 0; did < dev_total; did++) { + vdev = rte_vdpa_get_device(did); + if (!vdev) + continue; + if (vdev->ops->get_queue_num(did, &queue_num) < 0) { + RTE_LOG(ERR, VDPA, + "failed to get vdpa queue number " + "for device id %d.\n", did); + continue; + } + if (vdev->ops->get_features(did, &features) < 0) { + RTE_LOG(ERR, VDPA, + "failed to get vdpa features " + "for device id %d.\n", did); + continue; + } + addr = vdev->addr.pci_addr; + cmdline_printf(cl, + "%d\t\t" PCI_PRI_FMT "\t%" PRIu32 "\t\t0x%" PRIx64 "\n", + did, addr.domain, addr.bus, addr.devid, + addr.function, queue_num, features); + } +} + +cmdline_parse_token_string_t cmd_action_list = + TOKEN_STRING_INITIALIZER(struct cmd_list_result, action, "list"); + +cmdline_parse_inst_t cmd_list_vdpa_devices = { + .f = cmd_list_vdpa_devices_parsed, + .data = NULL, + .help_str = "list all available vdpa devices", + .tokens = { + (void *)&cmd_action_list, + NULL, + }, +}; + +/* *** Create new vdpa port *** */ +struct cmd_create_result { + cmdline_fixed_string_t action; + cmdline_fixed_string_t socket_path; + cmdline_fixed_string_t bdf; +}; + +static void cmd_create_vdpa_port_parsed(void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + int did; + struct cmd_create_result *res = parsed_result; + struct rte_vdpa_dev_addr addr; + + strncpy(vports[devcnt].ifname, res->socket_path, MAX_PATH_LEN); + if (rte_pci_addr_parse(res->bdf, &addr.pci_addr) != 0) { + cmdline_printf(cl, "Unable to parse the given bdf.\n"); + return; + } + addr.type = PCI_ADDR; + did = rte_vdpa_find_device_id(&addr); + if (did < 0) { + cmdline_printf(cl, "Unable to find vdpa devide id.\n"); + return; + } + + vports[devcnt].did = did; + + if (start_vdpa(&vports[devcnt]) == 0) + devcnt++; +} + +cmdline_parse_token_string_t cmd_action_create = + TOKEN_STRING_INITIALIZER(struct cmd_create_result, action, "create"); +cmdline_parse_token_string_t cmd_socket_path = + TOKEN_STRING_INITIALIZER(struct cmd_create_result, socket_path, NULL); +cmdline_parse_token_string_t cmd_bdf = + TOKEN_STRING_INITIALIZER(struct cmd_create_result, bdf, NULL); + +cmdline_parse_inst_t cmd_create_vdpa_port = { + .f = cmd_create_vdpa_port_parsed, + .data = NULL, + .help_str = "create a new vdpa port", + .tokens = { + (void *)&cmd_action_create, + (void *)&cmd_socket_path, + (void *)&cmd_bdf, + NULL, + }, +}; + +/* *** QUIT *** */ +struct cmd_quit_result { + cmdline_fixed_string_t quit; +}; + +static void cmd_quit_parsed(__attribute__((unused)) void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + vdpa_sample_quit(); + cmdline_quit(cl); +} + +cmdline_parse_token_string_t cmd_quit_quit = + TOKEN_STRING_INITIALIZER(struct cmd_quit_result, quit, "quit"); + +cmdline_parse_inst_t cmd_quit = { + .f = cmd_quit_parsed, + .data = NULL, + .help_str = "quit: exit application", + .tokens = { + (void *)&cmd_quit_quit, + NULL, + }, +}; +cmdline_parse_ctx_t main_ctx[] = { + (cmdline_parse_inst_t *)&cmd_help, + (cmdline_parse_inst_t *)&cmd_list_vdpa_devices, + (cmdline_parse_inst_t *)&cmd_create_vdpa_port, + (cmdline_parse_inst_t *)&cmd_quit, + NULL, +}; + +int +main(int argc, char *argv[]) +{ + char ch; + int i; + int ret; + struct cmdline *cl; + + ret = rte_eal_init(argc, argv); + if (ret < 0) + rte_exit(EXIT_FAILURE, "eal init failed\n"); + argc -= ret; + argv += ret; + + data_init(); + + signal(SIGINT, signal_handler); + signal(SIGTERM, signal_handler); + + ret = parse_args(argc, argv); + if (ret < 0) + rte_exit(EXIT_FAILURE, "invalid argument\n"); + + if (interactive == 1) { + cl = cmdline_stdin_new(main_ctx, "vdpa> "); + if (cl == NULL) + rte_panic("Cannot create cmdline instance\n"); + cmdline_interact(cl); + cmdline_stdin_exit(cl); + } else { + for (i = 0; i < RTE_MIN(MAX_VDPA_SAMPLE_PORTS, dev_total); + i++) { + vports[i].did = i; + snprintf(vports[i].ifname, MAX_PATH_LEN, "%s%d", + iface, i); + + start_vdpa(&vports[i]); + } + + printf("enter \'q\' to quit\n"); + while (scanf("%c", &ch)) { + if (ch == 'q') + break; + while (ch != '\n') { + if (scanf("%c", &ch)) + printf("%c", ch); + } + printf("enter \'q\' to quit\n"); + } + vdpa_sample_quit(); + } + + return 0; +} diff --git a/examples/vdpa/meson.build b/examples/vdpa/meson.build new file mode 100644 index 000000000..2e38a069f --- /dev/null +++ b/examples/vdpa/meson.build @@ -0,0 +1,16 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +# meson file, for building this example as part of a main DPDK build. +# +# To build this example as a standalone application with an already-installed +# DPDK instance, use 'make' + +if host_machine.system() != 'linux' + build = false +endif +deps += 'vhost' +allow_experimental_apis = true +sources = files( + 'main.c' +) \ No newline at end of file -- 2.17.1 ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [dpdk-dev] [PATCH v6 2/2] examples/vdpa: introduce a new sample for vDPA 2018-09-26 9:06 ` [dpdk-dev] [PATCH v6 2/2] examples/vdpa: introduce a new sample for vDPA Xiaolong Ye @ 2018-09-27 16:37 ` Maxime Coquelin 2018-09-28 7:32 ` Ye Xiaolong 0 siblings, 1 reply; 41+ messages in thread From: Maxime Coquelin @ 2018-09-27 16:37 UTC (permalink / raw) To: Xiaolong Ye, dev, Tiwei Bie, Zhihong Wang Cc: xiao.w.wang, Rami Rosen, Wang Haiyue On 09/26/2018 11:06 AM, Xiaolong Ye wrote: > The vdpa sample application creates vhost-user sockets by using the > vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes > virtio ring compatible devices to serve virtio driver directly to enable > datapath acceleration. As vDPA driver can help to set up vhost datapath, > this application doesn't need to launch dedicated worker threads for vhost > enqueue/dequeue operations. > > Signed-off-by: Xiao Wang <xiao.w.wang@intel.com> > Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com> > --- > MAINTAINERS | 2 + > doc/guides/rel_notes/release_18_11.rst | 8 + > doc/guides/sample_app_ug/index.rst | 1 + > doc/guides/sample_app_ug/vdpa.rst | 120 +++++++ > examples/Makefile | 2 +- > examples/vdpa/Makefile | 32 ++ > examples/vdpa/main.c | 466 +++++++++++++++++++++++++ > examples/vdpa/meson.build | 16 + > 8 files changed, 646 insertions(+), 1 deletion(-) > create mode 100644 doc/guides/sample_app_ug/vdpa.rst > create mode 100644 examples/vdpa/Makefile > create mode 100644 examples/vdpa/main.c > create mode 100644 examples/vdpa/meson.build > > diff --git a/MAINTAINERS b/MAINTAINERS > index 5967c1dd3..5656f18e8 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -683,6 +683,8 @@ F: doc/guides/sample_app_ug/vhost.rst > F: examples/vhost_scsi/ > F: doc/guides/sample_app_ug/vhost_scsi.rst > F: examples/vhost_crypto/ > +F: examples/vdpa/ > +F: doc/guides/sample_app_ug/vdpa.rst > > Vhost PMD > M: Maxime Coquelin <maxime.coquelin@redhat.com> > diff --git a/doc/guides/rel_notes/release_18_11.rst b/doc/guides/rel_notes/release_18_11.rst > index bc9b74ec4..dd53a9ecf 100644 > --- a/doc/guides/rel_notes/release_18_11.rst > +++ b/doc/guides/rel_notes/release_18_11.rst > @@ -67,6 +67,14 @@ New Features > SR-IOV option in Hyper-V and Azure. This is an alternative to the previous > vdev_netvsc, tap, and failsafe drivers combination. > > +* **Add a new sample for vDPA** > + > + The vdpa sample application creates vhost-user sockets by using the > + vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes > + virtio ring compatible devices to serve virtio driver directly to enable > + datapath acceleration. As vDPA driver can help to set up vhost datapath, > + this application doesn't need to launch dedicated worker threads for vhost > + enqueue/dequeue operations. > > API Changes > ----------- > diff --git a/doc/guides/sample_app_ug/index.rst b/doc/guides/sample_app_ug/index.rst > index 5bedf4f6f..74b12af85 100644 > --- a/doc/guides/sample_app_ug/index.rst > +++ b/doc/guides/sample_app_ug/index.rst > @@ -45,6 +45,7 @@ Sample Applications User Guides > vhost > vhost_scsi > vhost_crypto > + vdpa > netmap_compatibility > ip_pipeline > test_pipeline > diff --git a/doc/guides/sample_app_ug/vdpa.rst b/doc/guides/sample_app_ug/vdpa.rst > new file mode 100644 > index 000000000..745f196ca > --- /dev/null > +++ b/doc/guides/sample_app_ug/vdpa.rst > @@ -0,0 +1,120 @@ > +.. SPDX-License-Identifier: BSD-3-Clause > + Copyright(c) 2018 Intel Corporation. > + > +Vdpa Sample Application > +======================= > + > +The vdpa sample application creates vhost-user sockets by using the > +vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes > +virtio ring compatible devices to serve virtio driver directly to enable > +datapath acceleration. As vDPA driver can help to set up vhost datapath, > +this application doesn't need to launch dedicated worker threads for vhost > +enqueue/dequeue operations. > + > +Testing steps > +------------- > + > +This section shows the steps of how to start VMs with vDPA vhost-user > +backend and verify network connection & live migration. > + > +Build > +~~~~~ > + > +To compile the sample application see :doc:`compiling`. > + > +The application is located in the ``vdpa`` sub-directory. > + > +Start the vdpa example > +~~~~~~~~~~~~~~~~~~~~~~ > + > +.. code-block:: console > + > + ./vdpa [EAL options] -- [--client] [--interactive|-i] or [--iface SOCKET_PATH] > + > +where > + > +* --client means running vdpa app in client mode, in the client mode, QEMU needs > + to run as the server mode and take charge of socket file creation. > +* --iface specifies the path prefix of the UNIX domain socket file, e.g. > + /tmp/vhost-user-, then the socket files will be named as /tmp/vhost-user-<n> > + (n starts from 0). > +* --interactive means run the vdpa sample in interactive mode, currently 4 > + internal cmds are supported: > + > + 1. help: show help message > + 2. list: list all available vdpa devices > + 3. create: create a new vdpa port with socket file and vdpa device address > + 4. quit: unregister vhost driver and exit the application > + > +Take IFCVF driver for example: > + > +.. code-block:: console > + > + ./vdpa -c 0x2 -n 4 --socket-mem 1024,1024 \ > + -w 0000:06:00.3,vdpa=1 -w 0000:06:00.4,vdpa=1 \ > + -- --interactive > + > +.. note:: > + Here 0000:06:00.3 and 0000:06:00.4 refer to virtio ring compatible devices, > + and we need to bind vfio-pci to them before running vdpa sample. > + > + * modprobe vfio-pci > + * ./usertools/dpdk-devbind.py -b vfio-pci 06:00.3 06:00.4 > + > +Then we can create 2 vdpa ports in interactive cmdline. > + > +.. code-block:: console > + > + vdpa> list > + device id device address queue num supported features > + 0 0000:06:00.3 1 0x14c238020 > + 1 0000:06:00.4 1 0x14c238020 > + 2 0000:06:00.5 1 0x14c238020 > + > + vdpa> create /tmp/vdpa-socket0 0000:06:00.3 > + vdpa> create /tmp/vdpa-socket1 0000:06:00.4 > + > +.. _vdpa_app_run_vm: > + > +Start the VMs > +~~~~~~~~~~~~~ > + > +.. code-block:: console > + > + qemu-system-x86_64 -cpu host -enable-kvm \ > + <snip> > + -mem-prealloc \ > + -chardev socket,id=char0,path=<socket_file created in above steps> \ > + -netdev type=vhost-user,id=vdpa,chardev=char0 \ > + -device virtio-net-pci,netdev=vdpa,mac=00:aa:bb:cc:dd:ee,page-per-vq=on \ > + > +After the VMs launches, we can login the VMs and configure the ip, verify the > +network connection via ping or netperf. > + > +.. note:: > + Suggest to use QEMU 3.0.0 which extends vhost-user for vDPA. > + > +Live Migration > +~~~~~~~~~~~~~~ > +vDPA supports cross-backend live migration, user can migrate SW vhost backend > +VM to vDPA backend VM and vice versa. Here are the detailed steps. Assume A is > +the source host with SW vhost VM and B is the destination host with vDPA. > + > +1. Start vdpa sample and launch a VM with exact same parameters as the VM on A, > + in migration-listen mode: > + > +.. code-block:: console > + > + B: <qemu-command-line> -incoming tcp:0:4444 (or other PORT)) > + > +2. Start the migration (on source host): > + > +.. code-block:: console > + > + A: (qemu) migrate -d tcp:<B ip>:4444 (or other PORT) > + > +3. Check the status (on source host): > + > +.. code-block:: console > + > + A: (qemu) info migrate > diff --git a/examples/Makefile b/examples/Makefile > index 481720cb6..356fcb1cd 100644 > --- a/examples/Makefile > +++ b/examples/Makefile > @@ -65,7 +65,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_HASH),y) > DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += tep_termination > endif > DIRS-$(CONFIG_RTE_LIBRTE_TIMER) += timer > -DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi > +DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi vdpa > ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y) > DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_crypto > endif > diff --git a/examples/vdpa/Makefile b/examples/vdpa/Makefile > new file mode 100644 > index 000000000..42672a2bc > --- /dev/null > +++ b/examples/vdpa/Makefile > @@ -0,0 +1,32 @@ > +# SPDX-License-Identifier: BSD-3-Clause > +# Copyright(c) 2018 Intel Corporation > + > +ifeq ($(RTE_SDK),) > +$(error "Please define RTE_SDK environment variable") > +endif > + > +# Default target, can be overridden by command line or environment > +RTE_TARGET ?= x86_64-native-linuxapp-gcc > + > +include $(RTE_SDK)/mk/rte.vars.mk > + > +ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp") > +$(info This application can only operate in a linuxapp environment, \ > +please change the definition of the RTE_TARGET environment variable) > +all: > +else > + > +# binary name > +APP = vdpa > + > +# all source are stored in SRCS-y > +SRCS-y := main.c > + > +CFLAGS += -O2 -D_FILE_OFFSET_BITS=64 > +CFLAGS += $(WERROR_FLAGS) > +CFLAGS += -D_GNU_SOURCE > +CFLAGS += -DALLOW_EXPERIMENTAL_API > + > +include $(RTE_SDK)/mk/rte.extapp.mk > + > +endif > diff --git a/examples/vdpa/main.c b/examples/vdpa/main.c > new file mode 100644 > index 000000000..898286e30 > --- /dev/null > +++ b/examples/vdpa/main.c > @@ -0,0 +1,466 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(c) 2018 Intel Corporation > + */ > + > +#include <getopt.h> > +#include <signal.h> > +#include <stdint.h> > +#include <string.h> > +#include <unistd.h> > + > +#include <rte_ethdev.h> > +#include <rte_malloc.h> > +#include <rte_vhost.h> > +#include <rte_vdpa.h> > +#include <rte_pci.h> > + > +#include <cmdline_parse.h> > +#include <cmdline_socket.h> > +#include <cmdline_parse_string.h> > +#include <cmdline.h> > + > +#define MAX_PATH_LEN 128 > +#define MAX_VDPA_SAMPLE_PORTS 1024 > +#define RTE_LOGTYPE_VDPA RTE_LOGTYPE_USER1 > + > +struct vdpa_port { > + char ifname[MAX_PATH_LEN]; > + int did; > + int vid; > + uint64_t flags; > +}; > + > +struct vdpa_port vports[MAX_VDPA_SAMPLE_PORTS]; I guess it can be static? > + > +static char iface[MAX_PATH_LEN]; > +static int dev_total; > +static int devcnt; > +static int interactive; > +static int client_mode; > + > +/* display usage */ > +static void > +vdpa_usage(const char *prgname) > +{ > + printf("Usage: %s [EAL options] -- " > + " --interactive|-i: run in interactive mode.\n" > + " --iface <path>: specify the path prefix of the socket files, e.g. /tmp/vhost-user-.\n" > + " --client: register a vhost-user socket as client mode.\n", > + prgname); > +} > + > +static int > +parse_args(int argc, char **argv) > +{ > + static const char *short_option = "i"; > + static struct option long_option[] = { > + {"iface", required_argument, NULL, 0}, > + {"interactive", no_argument, &interactive, 1}, > + {"client", no_argument, &client_mode, 1}, > + {NULL, 0, 0, 0}, > + }; > + int opt, idx; > + char *prgname = argv[0]; > + > + while ((opt = getopt_long(argc, argv, short_option, long_option, &idx)) > + != EOF) { > + switch (opt) { > + case 'i': > + printf("Interactive-mode selected\n"); > + interactive = 1; > + break; > + /* long options */ > + case 0: > + if (strncmp(long_option[idx].name, "iface", > + MAX_PATH_LEN) == 0) { > + strncpy(iface, optarg, MAX_PATH_LEN); > + printf("iface %s\n", iface); > + } > + if (!strcmp(long_option[idx].name, "interactive")) { > + printf("Interactive-mode selected\n"); > + interactive = 1; > + } > + break; > + > + default: > + vdpa_usage(prgname); > + return -1; > + } > + } > + > + if (iface[0] == '\0' && interactive == 0) { > + vdpa_usage(prgname); > + return -1; > + } > + > + return 0; > +} > + > +static int > +data_init(void) > +{ > + dev_total = rte_vdpa_get_device_num(); > + if (dev_total <= 0) { > + printf("No available vdpa device found\n"); > + return -1; > + } > + devcnt = 0; This isn't needed as it is static. > + interactive = 0; Ditto > + memset(iface, 0, sizeof(iface)); Ditto > + memset(vports, 0, sizeof(vports)); And this one too when it will be made static. Appart these small comments, it looks good to me. When fixed, please apply my: Reviewed-by: Maxime coquelin <maxime.coquelin@redhat.com> Thanks, Maxime ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [dpdk-dev] [PATCH v6 2/2] examples/vdpa: introduce a new sample for vDPA 2018-09-27 16:37 ` Maxime Coquelin @ 2018-09-28 7:32 ` Ye Xiaolong 0 siblings, 0 replies; 41+ messages in thread From: Ye Xiaolong @ 2018-09-28 7:32 UTC (permalink / raw) To: Maxime Coquelin Cc: dev, Tiwei Bie, Zhihong Wang, xiao.w.wang, Rami Rosen, Wang Haiyue On 09/27, Maxime Coquelin wrote: [snip] >> + devcnt = 0; >This isn't needed as it is static. > >> + interactive = 0; >Ditto > >> + memset(iface, 0, sizeof(iface)); >Ditto > >> + memset(vports, 0, sizeof(vports)); >And this one too when it will be made static. > > >Appart these small comments, it looks good to me. >When fixed, please apply my: > >Reviewed-by: Maxime coquelin <maxime.coquelin@redhat.com> > Thanks for your comments, I'll adpot all the static related suggetions. Thanks, Xiaolong >Thanks, >Maxime ^ permalink raw reply [flat|nested] 41+ messages in thread
* [dpdk-dev] [PATCH v7 0/2] introduce vdpa sample 2018-09-26 9:06 ` [dpdk-dev] [PATCH v6 0/2] introduce vdpa sample Xiaolong Ye ` (2 preceding siblings ...) 2018-09-26 9:06 ` [dpdk-dev] [PATCH v6 2/2] examples/vdpa: introduce a new sample for vDPA Xiaolong Ye @ 2018-09-28 11:23 ` Xiaolong Ye 2018-09-28 11:23 ` [dpdk-dev] [PATCH v7 1/2] vhost: introduce API to get vDPA device number Xiaolong Ye ` (2 more replies) 3 siblings, 3 replies; 41+ messages in thread From: Xiaolong Ye @ 2018-09-28 11:23 UTC (permalink / raw) To: dev, Maxime Coquelin, Tiwei Bie, Zhihong Wang Cc: xiao.w.wang, Rami Rosen, Wang Haiyue, Xiaolong Ye Hi, This patchset introduces vdpa sample to demonstrate the vDPA use case. v7 changes: * make vports static * avoid unnecessary static variable initialization v6 changes: * improve the document according to Xiao's comments * fix a typo, PRIu64 -> PRIx64 v5 changes: * improve print format and correct from "PRIu64" to "PRIx64" * use "-c 0x2" to better demonstrate app doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations in vdpa.rst v4 changes: * add client mode support * improve the format to list the vDPA device info and improve the vdpa.rst accordingly * remove some useless comments * add introduction in 18.11 release note. v3 changes: * list cmd would show queue number and supported features of vdpa devices. * address Xiao's review comments v2 changes: * fix a compilation error reported by Rosen * improve create cmd in interactive mode and add two new cmds: list, quit * add application documentation Xiaolong Ye (2): vhost: introduce API to get vDPA device number examples/vdpa: introduce a new sample for vDPA MAINTAINERS | 2 + doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/vdpa.rst | 115 +++++++ examples/Makefile | 2 +- examples/vdpa/Makefile | 32 ++ examples/vdpa/main.c | 458 +++++++++++++++++++++++++ examples/vdpa/meson.build | 16 + lib/librte_vhost/rte_vdpa.h | 3 + lib/librte_vhost/rte_vhost_version.map | 1 + lib/librte_vhost/vdpa.c | 6 + 10 files changed, 635 insertions(+), 1 deletion(-) create mode 100644 doc/guides/sample_app_ug/vdpa.rst create mode 100644 examples/vdpa/Makefile create mode 100644 examples/vdpa/main.c create mode 100644 examples/vdpa/meson.build -- 2.17.1 ^ permalink raw reply [flat|nested] 41+ messages in thread
* [dpdk-dev] [PATCH v7 1/2] vhost: introduce API to get vDPA device number 2018-09-28 11:23 ` [dpdk-dev] [PATCH v7 0/2] introduce vdpa sample Xiaolong Ye @ 2018-09-28 11:23 ` Xiaolong Ye 2018-09-28 11:23 ` [dpdk-dev] [PATCH v7 2/2] examples/vdpa: introduce a new sample for vDPA Xiaolong Ye 2018-09-28 21:47 ` [dpdk-dev] [PATCH v8 0/2] introduce vdpa sample Xiaolong Ye 2 siblings, 0 replies; 41+ messages in thread From: Xiaolong Ye @ 2018-09-28 11:23 UTC (permalink / raw) To: dev, Maxime Coquelin, Tiwei Bie, Zhihong Wang Cc: xiao.w.wang, Rami Rosen, Wang Haiyue, Xiaolong Ye It's used to get number of available registered vDPA devices. Reviewed-by: Maxime coquelin <maxime.coquelin@redhat.com> Acked-by: Xiao Wang <xiao.w.wang@intel.com> Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com> --- lib/librte_vhost/rte_vdpa.h | 3 +++ lib/librte_vhost/rte_vhost_version.map | 1 + lib/librte_vhost/vdpa.c | 6 ++++++ 3 files changed, 10 insertions(+) diff --git a/lib/librte_vhost/rte_vdpa.h b/lib/librte_vhost/rte_vdpa.h index 90465ca26..b8223e337 100644 --- a/lib/librte_vhost/rte_vdpa.h +++ b/lib/librte_vhost/rte_vdpa.h @@ -84,4 +84,7 @@ rte_vdpa_find_device_id(struct rte_vdpa_dev_addr *addr); struct rte_vdpa_device * __rte_experimental rte_vdpa_get_device(int did); +/* Get current available vdpa device number */ +int __rte_experimental +rte_vdpa_get_device_num(void); #endif /* _RTE_VDPA_H_ */ diff --git a/lib/librte_vhost/rte_vhost_version.map b/lib/librte_vhost/rte_vhost_version.map index da220dd02..ae39b6e21 100644 --- a/lib/librte_vhost/rte_vhost_version.map +++ b/lib/librte_vhost/rte_vhost_version.map @@ -67,6 +67,7 @@ EXPERIMENTAL { rte_vdpa_unregister_device; rte_vdpa_find_device_id; rte_vdpa_get_device; + rte_vdpa_get_device_num; rte_vhost_driver_attach_vdpa_device; rte_vhost_driver_detach_vdpa_device; rte_vhost_driver_get_vdpa_device_id; diff --git a/lib/librte_vhost/vdpa.c b/lib/librte_vhost/vdpa.c index c82fd4370..c2c5dff1d 100644 --- a/lib/librte_vhost/vdpa.c +++ b/lib/librte_vhost/vdpa.c @@ -113,3 +113,9 @@ rte_vdpa_get_device(int did) return vdpa_devices[did]; } + +int +rte_vdpa_get_device_num(void) +{ + return vdpa_device_num; +} -- 2.17.1 ^ permalink raw reply [flat|nested] 41+ messages in thread
* [dpdk-dev] [PATCH v7 2/2] examples/vdpa: introduce a new sample for vDPA 2018-09-28 11:23 ` [dpdk-dev] [PATCH v7 0/2] introduce vdpa sample Xiaolong Ye 2018-09-28 11:23 ` [dpdk-dev] [PATCH v7 1/2] vhost: introduce API to get vDPA device number Xiaolong Ye @ 2018-09-28 11:23 ` Xiaolong Ye 2018-09-28 7:31 ` Maxime Coquelin 2018-09-28 21:47 ` [dpdk-dev] [PATCH v8 0/2] introduce vdpa sample Xiaolong Ye 2 siblings, 1 reply; 41+ messages in thread From: Xiaolong Ye @ 2018-09-28 11:23 UTC (permalink / raw) To: dev, Maxime Coquelin, Tiwei Bie, Zhihong Wang Cc: xiao.w.wang, Rami Rosen, Wang Haiyue, Xiaolong Ye The vdpa sample application creates vhost-user sockets by using the vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio ring compatible devices to serve virtio driver directly to enable datapath acceleration. As vDPA driver can help to set up vhost datapath, this application doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations. Reviewed-by: Maxime coquelin <maxime.coquelin@redhat.com> Acked-by: Xiao Wang <xiao.w.wang@intel.com> Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com> --- MAINTAINERS | 2 + doc/guides/rel_notes/release_18_11.rst | 8 + doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/vdpa.rst | 120 +++++++ examples/Makefile | 2 +- examples/vdpa/Makefile | 32 ++ examples/vdpa/main.c | 462 +++++++++++++++++++++++++ examples/vdpa/meson.build | 16 + 8 files changed, 642 insertions(+), 1 deletion(-) create mode 100644 doc/guides/sample_app_ug/vdpa.rst create mode 100644 examples/vdpa/Makefile create mode 100644 examples/vdpa/main.c create mode 100644 examples/vdpa/meson.build diff --git a/MAINTAINERS b/MAINTAINERS index 5967c1dd3..5656f18e8 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -683,6 +683,8 @@ F: doc/guides/sample_app_ug/vhost.rst F: examples/vhost_scsi/ F: doc/guides/sample_app_ug/vhost_scsi.rst F: examples/vhost_crypto/ +F: examples/vdpa/ +F: doc/guides/sample_app_ug/vdpa.rst Vhost PMD M: Maxime Coquelin <maxime.coquelin@redhat.com> diff --git a/doc/guides/rel_notes/release_18_11.rst b/doc/guides/rel_notes/release_18_11.rst index bc9b74ec4..dd53a9ecf 100644 --- a/doc/guides/rel_notes/release_18_11.rst +++ b/doc/guides/rel_notes/release_18_11.rst @@ -67,6 +67,14 @@ New Features SR-IOV option in Hyper-V and Azure. This is an alternative to the previous vdev_netvsc, tap, and failsafe drivers combination. +* **Add a new sample for vDPA** + + The vdpa sample application creates vhost-user sockets by using the + vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes + virtio ring compatible devices to serve virtio driver directly to enable + datapath acceleration. As vDPA driver can help to set up vhost datapath, + this application doesn't need to launch dedicated worker threads for vhost + enqueue/dequeue operations. API Changes ----------- diff --git a/doc/guides/sample_app_ug/index.rst b/doc/guides/sample_app_ug/index.rst index 5bedf4f6f..74b12af85 100644 --- a/doc/guides/sample_app_ug/index.rst +++ b/doc/guides/sample_app_ug/index.rst @@ -45,6 +45,7 @@ Sample Applications User Guides vhost vhost_scsi vhost_crypto + vdpa netmap_compatibility ip_pipeline test_pipeline diff --git a/doc/guides/sample_app_ug/vdpa.rst b/doc/guides/sample_app_ug/vdpa.rst new file mode 100644 index 000000000..745f196ca --- /dev/null +++ b/doc/guides/sample_app_ug/vdpa.rst @@ -0,0 +1,120 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2018 Intel Corporation. + +Vdpa Sample Application +======================= + +The vdpa sample application creates vhost-user sockets by using the +vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes +virtio ring compatible devices to serve virtio driver directly to enable +datapath acceleration. As vDPA driver can help to set up vhost datapath, +this application doesn't need to launch dedicated worker threads for vhost +enqueue/dequeue operations. + +Testing steps +------------- + +This section shows the steps of how to start VMs with vDPA vhost-user +backend and verify network connection & live migration. + +Build +~~~~~ + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``vdpa`` sub-directory. + +Start the vdpa example +~~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: console + + ./vdpa [EAL options] -- [--client] [--interactive|-i] or [--iface SOCKET_PATH] + +where + +* --client means running vdpa app in client mode, in the client mode, QEMU needs + to run as the server mode and take charge of socket file creation. +* --iface specifies the path prefix of the UNIX domain socket file, e.g. + /tmp/vhost-user-, then the socket files will be named as /tmp/vhost-user-<n> + (n starts from 0). +* --interactive means run the vdpa sample in interactive mode, currently 4 + internal cmds are supported: + + 1. help: show help message + 2. list: list all available vdpa devices + 3. create: create a new vdpa port with socket file and vdpa device address + 4. quit: unregister vhost driver and exit the application + +Take IFCVF driver for example: + +.. code-block:: console + + ./vdpa -c 0x2 -n 4 --socket-mem 1024,1024 \ + -w 0000:06:00.3,vdpa=1 -w 0000:06:00.4,vdpa=1 \ + -- --interactive + +.. note:: + Here 0000:06:00.3 and 0000:06:00.4 refer to virtio ring compatible devices, + and we need to bind vfio-pci to them before running vdpa sample. + + * modprobe vfio-pci + * ./usertools/dpdk-devbind.py -b vfio-pci 06:00.3 06:00.4 + +Then we can create 2 vdpa ports in interactive cmdline. + +.. code-block:: console + + vdpa> list + device id device address queue num supported features + 0 0000:06:00.3 1 0x14c238020 + 1 0000:06:00.4 1 0x14c238020 + 2 0000:06:00.5 1 0x14c238020 + + vdpa> create /tmp/vdpa-socket0 0000:06:00.3 + vdpa> create /tmp/vdpa-socket1 0000:06:00.4 + +.. _vdpa_app_run_vm: + +Start the VMs +~~~~~~~~~~~~~ + +.. code-block:: console + + qemu-system-x86_64 -cpu host -enable-kvm \ + <snip> + -mem-prealloc \ + -chardev socket,id=char0,path=<socket_file created in above steps> \ + -netdev type=vhost-user,id=vdpa,chardev=char0 \ + -device virtio-net-pci,netdev=vdpa,mac=00:aa:bb:cc:dd:ee,page-per-vq=on \ + +After the VMs launches, we can login the VMs and configure the ip, verify the +network connection via ping or netperf. + +.. note:: + Suggest to use QEMU 3.0.0 which extends vhost-user for vDPA. + +Live Migration +~~~~~~~~~~~~~~ +vDPA supports cross-backend live migration, user can migrate SW vhost backend +VM to vDPA backend VM and vice versa. Here are the detailed steps. Assume A is +the source host with SW vhost VM and B is the destination host with vDPA. + +1. Start vdpa sample and launch a VM with exact same parameters as the VM on A, + in migration-listen mode: + +.. code-block:: console + + B: <qemu-command-line> -incoming tcp:0:4444 (or other PORT)) + +2. Start the migration (on source host): + +.. code-block:: console + + A: (qemu) migrate -d tcp:<B ip>:4444 (or other PORT) + +3. Check the status (on source host): + +.. code-block:: console + + A: (qemu) info migrate diff --git a/examples/Makefile b/examples/Makefile index 481720cb6..356fcb1cd 100644 --- a/examples/Makefile +++ b/examples/Makefile @@ -65,7 +65,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_HASH),y) DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += tep_termination endif DIRS-$(CONFIG_RTE_LIBRTE_TIMER) += timer -DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi +DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi vdpa ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y) DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_crypto endif diff --git a/examples/vdpa/Makefile b/examples/vdpa/Makefile new file mode 100644 index 000000000..42672a2bc --- /dev/null +++ b/examples/vdpa/Makefile @@ -0,0 +1,32 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +ifeq ($(RTE_SDK),) +$(error "Please define RTE_SDK environment variable") +endif + +# Default target, can be overridden by command line or environment +RTE_TARGET ?= x86_64-native-linuxapp-gcc + +include $(RTE_SDK)/mk/rte.vars.mk + +ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp") +$(info This application can only operate in a linuxapp environment, \ +please change the definition of the RTE_TARGET environment variable) +all: +else + +# binary name +APP = vdpa + +# all source are stored in SRCS-y +SRCS-y := main.c + +CFLAGS += -O2 -D_FILE_OFFSET_BITS=64 +CFLAGS += $(WERROR_FLAGS) +CFLAGS += -D_GNU_SOURCE +CFLAGS += -DALLOW_EXPERIMENTAL_API + +include $(RTE_SDK)/mk/rte.extapp.mk + +endif diff --git a/examples/vdpa/main.c b/examples/vdpa/main.c new file mode 100644 index 000000000..b8055287f --- /dev/null +++ b/examples/vdpa/main.c @@ -0,0 +1,462 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2018 Intel Corporation + */ + +#include <getopt.h> +#include <signal.h> +#include <stdint.h> +#include <string.h> +#include <unistd.h> + +#include <rte_ethdev.h> +#include <rte_malloc.h> +#include <rte_vhost.h> +#include <rte_vdpa.h> +#include <rte_pci.h> + +#include <cmdline_parse.h> +#include <cmdline_socket.h> +#include <cmdline_parse_string.h> +#include <cmdline.h> + +#define MAX_PATH_LEN 128 +#define MAX_VDPA_SAMPLE_PORTS 1024 +#define RTE_LOGTYPE_VDPA RTE_LOGTYPE_USER1 + +struct vdpa_port { + char ifname[MAX_PATH_LEN]; + int did; + int vid; + uint64_t flags; +}; + +static struct vdpa_port vports[MAX_VDPA_SAMPLE_PORTS]; + +static char iface[MAX_PATH_LEN]; +static int dev_total; +static int devcnt; +static int interactive; +static int client_mode; + +/* display usage */ +static void +vdpa_usage(const char *prgname) +{ + printf("Usage: %s [EAL options] -- " + " --interactive|-i: run in interactive mode.\n" + " --iface <path>: specify the path prefix of the socket files, e.g. /tmp/vhost-user-.\n" + " --client: register a vhost-user socket as client mode.\n", + prgname); +} + +static int +parse_args(int argc, char **argv) +{ + static const char *short_option = "i"; + static struct option long_option[] = { + {"iface", required_argument, NULL, 0}, + {"interactive", no_argument, &interactive, 1}, + {"client", no_argument, &client_mode, 1}, + {NULL, 0, 0, 0}, + }; + int opt, idx; + char *prgname = argv[0]; + + while ((opt = getopt_long(argc, argv, short_option, long_option, &idx)) + != EOF) { + switch (opt) { + case 'i': + printf("Interactive-mode selected\n"); + interactive = 1; + break; + /* long options */ + case 0: + if (strncmp(long_option[idx].name, "iface", + MAX_PATH_LEN) == 0) { + strncpy(iface, optarg, MAX_PATH_LEN); + printf("iface %s\n", iface); + } + if (!strcmp(long_option[idx].name, "interactive")) { + printf("Interactive-mode selected\n"); + interactive = 1; + } + break; + + default: + vdpa_usage(prgname); + return -1; + } + } + + if (iface[0] == '\0' && interactive == 0) { + vdpa_usage(prgname); + return -1; + } + + return 0; +} + +static int +data_init(void) +{ + dev_total = rte_vdpa_get_device_num(); + if (dev_total <= 0) { + printf("No available vdpa device found\n"); + return -1; + } + return 0; +} + +static int +new_device(int vid) +{ + char ifname[MAX_PATH_LEN]; + int i; + + rte_vhost_get_ifname(vid, ifname, sizeof(ifname)); + for (i = 0; i < MAX_VDPA_SAMPLE_PORTS; i++) { + if (strncmp(ifname, vports[i].ifname, MAX_PATH_LEN) == 0) { + printf("\nnew port %s, did: %d\n", + ifname, vports[i].did); + vports[i].vid = vid; + break; + } + } + + if (i >= MAX_VDPA_SAMPLE_PORTS) + return -1; + + return 0; +} + +static void +destroy_device(int vid) +{ + char ifname[MAX_PATH_LEN]; + int i; + + rte_vhost_get_ifname(vid, ifname, sizeof(ifname)); + for (i = 0; i < MAX_VDPA_SAMPLE_PORTS; i++) { + if (strcmp(ifname, vports[i].ifname) == 0) { + printf("\ndestroy port %s, did: %d\n", + ifname, vports[i].did); + break; + } + } +} + +static const struct vhost_device_ops vdpa_sample_devops = { + .new_device = new_device, + .destroy_device = destroy_device, +}; + +static int +start_vdpa(struct vdpa_port *vport) +{ + int ret; + char *socket_path = vport->ifname; + int did = vport->did; + + if (client_mode) + vport->flags |= RTE_VHOST_USER_CLIENT; + + if (access(socket_path, F_OK) != -1 && !client_mode) { + RTE_LOG(ERR, VDPA, + "%s exists, please remove it or specify another file and try again.\n", + socket_path); + return -1; + } + ret = rte_vhost_driver_register(socket_path, vport->flags); + if (ret != 0) + rte_exit(EXIT_FAILURE, + "register driver failed: %s\n", + socket_path); + + ret = rte_vhost_driver_callback_register(socket_path, + &vdpa_sample_devops); + if (ret != 0) + rte_exit(EXIT_FAILURE, + "register driver ops failed: %s\n", + socket_path); + + ret = rte_vhost_driver_attach_vdpa_device(socket_path, did); + if (ret != 0) + rte_exit(EXIT_FAILURE, + "attach vdpa device failed: %s\n", + socket_path); + + if (rte_vhost_driver_start(socket_path) < 0) + rte_exit(EXIT_FAILURE, + "start vhost driver failed: %s\n", + socket_path); + return 0; +} + +static void +close_vdpa(struct vdpa_port *vport) +{ + int ret; + char *socket_path = vport->ifname; + + ret = rte_vhost_driver_detach_vdpa_device(socket_path); + if (ret != 0) + RTE_LOG(ERR, VDPA, + "detach vdpa device failed: %s\n", + socket_path); + + ret = rte_vhost_driver_unregister(socket_path); + if (ret != 0) + RTE_LOG(ERR, VDPA, + "Fail to unregister vhost driver for %s.\n", + socket_path); +} + +static void +vdpa_sample_quit(void) +{ + int i; + for (i = 0; i < RTE_MIN(MAX_VDPA_SAMPLE_PORTS, dev_total); i++) { + if (vports[i].ifname[0] != '\0') + close_vdpa(&vports[i]); + } +} + +static void +signal_handler(int signum) +{ + if (signum == SIGINT || signum == SIGTERM) { + printf("\nSignal %d received, preparing to exit...\n", signum); + vdpa_sample_quit(); + exit(0); + } +} + +/* interactive cmds */ + +/* *** Help command with introduction. *** */ +struct cmd_help_result { + cmdline_fixed_string_t help; +}; + +static void cmd_help_parsed(__attribute__((unused)) void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + cmdline_printf( + cl, + "\n" + "The following commands are currently available:\n\n" + "Control:\n" + " help : Show interactive instructions.\n" + " list : list all available vdpa devices.\n" + " create <socket file> <vdev addr> : create a new vdpa port.\n" + " quit : exit vdpa sample app.\n" + ); +} + +cmdline_parse_token_string_t cmd_help_help = + TOKEN_STRING_INITIALIZER(struct cmd_help_result, help, "help"); + +cmdline_parse_inst_t cmd_help = { + .f = cmd_help_parsed, + .data = NULL, + .help_str = "show help", + .tokens = { + (void *)&cmd_help_help, + NULL, + }, +}; + +/* *** List all available vdpa devices *** */ +struct cmd_list_result { + cmdline_fixed_string_t action; +}; + +static void cmd_list_vdpa_devices_parsed( + __attribute__((unused)) void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + int did; + uint32_t queue_num; + uint64_t features; + struct rte_vdpa_device *vdev; + struct rte_pci_addr addr; + + cmdline_printf(cl, "device id\tdevice address\tqueue num\tsupported features\n"); + for (did = 0; did < dev_total; did++) { + vdev = rte_vdpa_get_device(did); + if (!vdev) + continue; + if (vdev->ops->get_queue_num(did, &queue_num) < 0) { + RTE_LOG(ERR, VDPA, + "failed to get vdpa queue number " + "for device id %d.\n", did); + continue; + } + if (vdev->ops->get_features(did, &features) < 0) { + RTE_LOG(ERR, VDPA, + "failed to get vdpa features " + "for device id %d.\n", did); + continue; + } + addr = vdev->addr.pci_addr; + cmdline_printf(cl, + "%d\t\t" PCI_PRI_FMT "\t%" PRIu32 "\t\t0x%" PRIx64 "\n", + did, addr.domain, addr.bus, addr.devid, + addr.function, queue_num, features); + } +} + +cmdline_parse_token_string_t cmd_action_list = + TOKEN_STRING_INITIALIZER(struct cmd_list_result, action, "list"); + +cmdline_parse_inst_t cmd_list_vdpa_devices = { + .f = cmd_list_vdpa_devices_parsed, + .data = NULL, + .help_str = "list all available vdpa devices", + .tokens = { + (void *)&cmd_action_list, + NULL, + }, +}; + +/* *** Create new vdpa port *** */ +struct cmd_create_result { + cmdline_fixed_string_t action; + cmdline_fixed_string_t socket_path; + cmdline_fixed_string_t bdf; +}; + +static void cmd_create_vdpa_port_parsed(void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + int did; + struct cmd_create_result *res = parsed_result; + struct rte_vdpa_dev_addr addr; + + strncpy(vports[devcnt].ifname, res->socket_path, MAX_PATH_LEN); + if (rte_pci_addr_parse(res->bdf, &addr.pci_addr) != 0) { + cmdline_printf(cl, "Unable to parse the given bdf.\n"); + return; + } + addr.type = PCI_ADDR; + did = rte_vdpa_find_device_id(&addr); + if (did < 0) { + cmdline_printf(cl, "Unable to find vdpa devide id.\n"); + return; + } + + vports[devcnt].did = did; + + if (start_vdpa(&vports[devcnt]) == 0) + devcnt++; +} + +cmdline_parse_token_string_t cmd_action_create = + TOKEN_STRING_INITIALIZER(struct cmd_create_result, action, "create"); +cmdline_parse_token_string_t cmd_socket_path = + TOKEN_STRING_INITIALIZER(struct cmd_create_result, socket_path, NULL); +cmdline_parse_token_string_t cmd_bdf = + TOKEN_STRING_INITIALIZER(struct cmd_create_result, bdf, NULL); + +cmdline_parse_inst_t cmd_create_vdpa_port = { + .f = cmd_create_vdpa_port_parsed, + .data = NULL, + .help_str = "create a new vdpa port", + .tokens = { + (void *)&cmd_action_create, + (void *)&cmd_socket_path, + (void *)&cmd_bdf, + NULL, + }, +}; + +/* *** QUIT *** */ +struct cmd_quit_result { + cmdline_fixed_string_t quit; +}; + +static void cmd_quit_parsed(__attribute__((unused)) void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + vdpa_sample_quit(); + cmdline_quit(cl); +} + +cmdline_parse_token_string_t cmd_quit_quit = + TOKEN_STRING_INITIALIZER(struct cmd_quit_result, quit, "quit"); + +cmdline_parse_inst_t cmd_quit = { + .f = cmd_quit_parsed, + .data = NULL, + .help_str = "quit: exit application", + .tokens = { + (void *)&cmd_quit_quit, + NULL, + }, +}; +cmdline_parse_ctx_t main_ctx[] = { + (cmdline_parse_inst_t *)&cmd_help, + (cmdline_parse_inst_t *)&cmd_list_vdpa_devices, + (cmdline_parse_inst_t *)&cmd_create_vdpa_port, + (cmdline_parse_inst_t *)&cmd_quit, + NULL, +}; + +int +main(int argc, char *argv[]) +{ + char ch; + int i; + int ret; + struct cmdline *cl; + + ret = rte_eal_init(argc, argv); + if (ret < 0) + rte_exit(EXIT_FAILURE, "eal init failed\n"); + argc -= ret; + argv += ret; + + data_init(); + + signal(SIGINT, signal_handler); + signal(SIGTERM, signal_handler); + + ret = parse_args(argc, argv); + if (ret < 0) + rte_exit(EXIT_FAILURE, "invalid argument\n"); + + if (interactive == 1) { + cl = cmdline_stdin_new(main_ctx, "vdpa> "); + if (cl == NULL) + rte_panic("Cannot create cmdline instance\n"); + cmdline_interact(cl); + cmdline_stdin_exit(cl); + } else { + for (i = 0; i < RTE_MIN(MAX_VDPA_SAMPLE_PORTS, dev_total); + i++) { + vports[i].did = i; + snprintf(vports[i].ifname, MAX_PATH_LEN, "%s%d", + iface, i); + + start_vdpa(&vports[i]); + } + + printf("enter \'q\' to quit\n"); + while (scanf("%c", &ch)) { + if (ch == 'q') + break; + while (ch != '\n') { + if (scanf("%c", &ch)) + printf("%c", ch); + } + printf("enter \'q\' to quit\n"); + } + vdpa_sample_quit(); + } + + return 0; +} diff --git a/examples/vdpa/meson.build b/examples/vdpa/meson.build new file mode 100644 index 000000000..2e38a069f --- /dev/null +++ b/examples/vdpa/meson.build @@ -0,0 +1,16 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +# meson file, for building this example as part of a main DPDK build. +# +# To build this example as a standalone application with an already-installed +# DPDK instance, use 'make' + +if host_machine.system() != 'linux' + build = false +endif +deps += 'vhost' +allow_experimental_apis = true +sources = files( + 'main.c' +) \ No newline at end of file -- 2.17.1 ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [dpdk-dev] [PATCH v7 2/2] examples/vdpa: introduce a new sample for vDPA 2018-09-28 11:23 ` [dpdk-dev] [PATCH v7 2/2] examples/vdpa: introduce a new sample for vDPA Xiaolong Ye @ 2018-09-28 7:31 ` Maxime Coquelin 2018-09-28 14:33 ` Ye Xiaolong 0 siblings, 1 reply; 41+ messages in thread From: Maxime Coquelin @ 2018-09-28 7:31 UTC (permalink / raw) To: Xiaolong Ye, dev, Tiwei Bie, Zhihong Wang Cc: xiao.w.wang, Rami Rosen, Wang Haiyue On 09/28/2018 01:23 PM, Xiaolong Ye wrote: > The vdpa sample application creates vhost-user sockets by using the > vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes > virtio ring compatible devices to serve virtio driver directly to enable > datapath acceleration. As vDPA driver can help to set up vhost datapath, > this application doesn't need to launch dedicated worker threads for vhost > enqueue/dequeue operations. > > Reviewed-by: Maxime coquelin <maxime.coquelin@redhat.com> > Acked-by: Xiao Wang <xiao.w.wang@intel.com> > Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com> > --- > MAINTAINERS | 2 + > doc/guides/rel_notes/release_18_11.rst | 8 + > doc/guides/sample_app_ug/index.rst | 1 + > doc/guides/sample_app_ug/vdpa.rst | 120 +++++++ > examples/Makefile | 2 +- > examples/vdpa/Makefile | 32 ++ > examples/vdpa/main.c | 462 +++++++++++++++++++++++++ > examples/vdpa/meson.build | 16 + > 8 files changed, 642 insertions(+), 1 deletion(-) > create mode 100644 doc/guides/sample_app_ug/vdpa.rst > create mode 100644 examples/vdpa/Makefile > create mode 100644 examples/vdpa/main.c > create mode 100644 examples/vdpa/meson.build > > diff --git a/MAINTAINERS b/MAINTAINERS > index 5967c1dd3..5656f18e8 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -683,6 +683,8 @@ F: doc/guides/sample_app_ug/vhost.rst > F: examples/vhost_scsi/ > F: doc/guides/sample_app_ug/vhost_scsi.rst > F: examples/vhost_crypto/ > +F: examples/vdpa/ > +F: doc/guides/sample_app_ug/vdpa.rst > > Vhost PMD > M: Maxime Coquelin <maxime.coquelin@redhat.com> > diff --git a/doc/guides/rel_notes/release_18_11.rst b/doc/guides/rel_notes/release_18_11.rst > index bc9b74ec4..dd53a9ecf 100644 > --- a/doc/guides/rel_notes/release_18_11.rst > +++ b/doc/guides/rel_notes/release_18_11.rst > @@ -67,6 +67,14 @@ New Features > SR-IOV option in Hyper-V and Azure. This is an alternative to the previous > vdev_netvsc, tap, and failsafe drivers combination. > > +* **Add a new sample for vDPA** > + > + The vdpa sample application creates vhost-user sockets by using the > + vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes > + virtio ring compatible devices to serve virtio driver directly to enable > + datapath acceleration. As vDPA driver can help to set up vhost datapath, > + this application doesn't need to launch dedicated worker threads for vhost > + enqueue/dequeue operations. > > API Changes > ----------- > diff --git a/doc/guides/sample_app_ug/index.rst b/doc/guides/sample_app_ug/index.rst > index 5bedf4f6f..74b12af85 100644 > --- a/doc/guides/sample_app_ug/index.rst > +++ b/doc/guides/sample_app_ug/index.rst > @@ -45,6 +45,7 @@ Sample Applications User Guides > vhost > vhost_scsi > vhost_crypto > + vdpa > netmap_compatibility > ip_pipeline > test_pipeline > diff --git a/doc/guides/sample_app_ug/vdpa.rst b/doc/guides/sample_app_ug/vdpa.rst > new file mode 100644 > index 000000000..745f196ca > --- /dev/null > +++ b/doc/guides/sample_app_ug/vdpa.rst > @@ -0,0 +1,120 @@ > +.. SPDX-License-Identifier: BSD-3-Clause > + Copyright(c) 2018 Intel Corporation. > + > +Vdpa Sample Application > +======================= > + > +The vdpa sample application creates vhost-user sockets by using the > +vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes > +virtio ring compatible devices to serve virtio driver directly to enable > +datapath acceleration. As vDPA driver can help to set up vhost datapath, > +this application doesn't need to launch dedicated worker threads for vhost > +enqueue/dequeue operations. > + > +Testing steps > +------------- > + > +This section shows the steps of how to start VMs with vDPA vhost-user > +backend and verify network connection & live migration. > + > +Build > +~~~~~ > + > +To compile the sample application see :doc:`compiling`. > + > +The application is located in the ``vdpa`` sub-directory. > + > +Start the vdpa example > +~~~~~~~~~~~~~~~~~~~~~~ > + > +.. code-block:: console > + > + ./vdpa [EAL options] -- [--client] [--interactive|-i] or [--iface SOCKET_PATH] > + > +where > + > +* --client means running vdpa app in client mode, in the client mode, QEMU needs > + to run as the server mode and take charge of socket file creation. > +* --iface specifies the path prefix of the UNIX domain socket file, e.g. > + /tmp/vhost-user-, then the socket files will be named as /tmp/vhost-user-<n> > + (n starts from 0). > +* --interactive means run the vdpa sample in interactive mode, currently 4 > + internal cmds are supported: > + > + 1. help: show help message > + 2. list: list all available vdpa devices > + 3. create: create a new vdpa port with socket file and vdpa device address > + 4. quit: unregister vhost driver and exit the application > + > +Take IFCVF driver for example: > + > +.. code-block:: console > + > + ./vdpa -c 0x2 -n 4 --socket-mem 1024,1024 \ > + -w 0000:06:00.3,vdpa=1 -w 0000:06:00.4,vdpa=1 \ > + -- --interactive > + > +.. note:: > + Here 0000:06:00.3 and 0000:06:00.4 refer to virtio ring compatible devices, > + and we need to bind vfio-pci to them before running vdpa sample. > + > + * modprobe vfio-pci > + * ./usertools/dpdk-devbind.py -b vfio-pci 06:00.3 06:00.4 > + > +Then we can create 2 vdpa ports in interactive cmdline. > + > +.. code-block:: console > + > + vdpa> list > + device id device address queue num supported features > + 0 0000:06:00.3 1 0x14c238020 > + 1 0000:06:00.4 1 0x14c238020 > + 2 0000:06:00.5 1 0x14c238020 > + > + vdpa> create /tmp/vdpa-socket0 0000:06:00.3 > + vdpa> create /tmp/vdpa-socket1 0000:06:00.4 > + > +.. _vdpa_app_run_vm: > + > +Start the VMs > +~~~~~~~~~~~~~ > + > +.. code-block:: console > + > + qemu-system-x86_64 -cpu host -enable-kvm \ > + <snip> > + -mem-prealloc \ > + -chardev socket,id=char0,path=<socket_file created in above steps> \ > + -netdev type=vhost-user,id=vdpa,chardev=char0 \ > + -device virtio-net-pci,netdev=vdpa,mac=00:aa:bb:cc:dd:ee,page-per-vq=on \ > + > +After the VMs launches, we can login the VMs and configure the ip, verify the > +network connection via ping or netperf. > + > +.. note:: > + Suggest to use QEMU 3.0.0 which extends vhost-user for vDPA. > + > +Live Migration > +~~~~~~~~~~~~~~ > +vDPA supports cross-backend live migration, user can migrate SW vhost backend > +VM to vDPA backend VM and vice versa. Here are the detailed steps. Assume A is > +the source host with SW vhost VM and B is the destination host with vDPA. > + > +1. Start vdpa sample and launch a VM with exact same parameters as the VM on A, > + in migration-listen mode: > + > +.. code-block:: console > + > + B: <qemu-command-line> -incoming tcp:0:4444 (or other PORT)) > + > +2. Start the migration (on source host): > + > +.. code-block:: console > + > + A: (qemu) migrate -d tcp:<B ip>:4444 (or other PORT) > + > +3. Check the status (on source host): > + > +.. code-block:: console > + > + A: (qemu) info migrate > diff --git a/examples/Makefile b/examples/Makefile > index 481720cb6..356fcb1cd 100644 > --- a/examples/Makefile > +++ b/examples/Makefile > @@ -65,7 +65,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_HASH),y) > DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += tep_termination > endif > DIRS-$(CONFIG_RTE_LIBRTE_TIMER) += timer > -DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi > +DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi vdpa > ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y) > DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_crypto > endif > diff --git a/examples/vdpa/Makefile b/examples/vdpa/Makefile > new file mode 100644 > index 000000000..42672a2bc > --- /dev/null > +++ b/examples/vdpa/Makefile > @@ -0,0 +1,32 @@ > +# SPDX-License-Identifier: BSD-3-Clause > +# Copyright(c) 2018 Intel Corporation > + > +ifeq ($(RTE_SDK),) > +$(error "Please define RTE_SDK environment variable") > +endif > + > +# Default target, can be overridden by command line or environment > +RTE_TARGET ?= x86_64-native-linuxapp-gcc > + > +include $(RTE_SDK)/mk/rte.vars.mk > + > +ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp") > +$(info This application can only operate in a linuxapp environment, \ > +please change the definition of the RTE_TARGET environment variable) > +all: > +else > + > +# binary name > +APP = vdpa > + > +# all source are stored in SRCS-y > +SRCS-y := main.c > + > +CFLAGS += -O2 -D_FILE_OFFSET_BITS=64 > +CFLAGS += $(WERROR_FLAGS) > +CFLAGS += -D_GNU_SOURCE > +CFLAGS += -DALLOW_EXPERIMENTAL_API > + > +include $(RTE_SDK)/mk/rte.extapp.mk > + > +endif > diff --git a/examples/vdpa/main.c b/examples/vdpa/main.c > new file mode 100644 > index 000000000..b8055287f > --- /dev/null > +++ b/examples/vdpa/main.c > @@ -0,0 +1,462 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(c) 2018 Intel Corporation > + */ > + > +#include <getopt.h> > +#include <signal.h> > +#include <stdint.h> > +#include <string.h> > +#include <unistd.h> > + > +#include <rte_ethdev.h> > +#include <rte_malloc.h> > +#include <rte_vhost.h> > +#include <rte_vdpa.h> > +#include <rte_pci.h> > + > +#include <cmdline_parse.h> > +#include <cmdline_socket.h> > +#include <cmdline_parse_string.h> > +#include <cmdline.h> > + > +#define MAX_PATH_LEN 128 > +#define MAX_VDPA_SAMPLE_PORTS 1024 > +#define RTE_LOGTYPE_VDPA RTE_LOGTYPE_USER1 > + > +struct vdpa_port { > + char ifname[MAX_PATH_LEN]; > + int did; > + int vid; > + uint64_t flags; > +}; > + > +static struct vdpa_port vports[MAX_VDPA_SAMPLE_PORTS]; > + > +static char iface[MAX_PATH_LEN]; > +static int dev_total; > +static int devcnt; > +static int interactive; > +static int client_mode; > + > +/* display usage */ > +static void > +vdpa_usage(const char *prgname) > +{ > + printf("Usage: %s [EAL options] -- " > + " --interactive|-i: run in interactive mode.\n" > + " --iface <path>: specify the path prefix of the socket files, e.g. /tmp/vhost-user-.\n" > + " --client: register a vhost-user socket as client mode.\n", > + prgname); > +} > + > +static int > +parse_args(int argc, char **argv) > +{ > + static const char *short_option = "i"; > + static struct option long_option[] = { > + {"iface", required_argument, NULL, 0}, > + {"interactive", no_argument, &interactive, 1}, > + {"client", no_argument, &client_mode, 1}, > + {NULL, 0, 0, 0}, > + }; > + int opt, idx; > + char *prgname = argv[0]; > + > + while ((opt = getopt_long(argc, argv, short_option, long_option, &idx)) > + != EOF) { > + switch (opt) { > + case 'i': > + printf("Interactive-mode selected\n"); > + interactive = 1; > + break; > + /* long options */ > + case 0: > + if (strncmp(long_option[idx].name, "iface", > + MAX_PATH_LEN) == 0) { > + strncpy(iface, optarg, MAX_PATH_LEN); > + printf("iface %s\n", iface); > + } > + if (!strcmp(long_option[idx].name, "interactive")) { > + printf("Interactive-mode selected\n"); > + interactive = 1; > + } > + break; > + > + default: > + vdpa_usage(prgname); > + return -1; > + } > + } > + > + if (iface[0] == '\0' && interactive == 0) { > + vdpa_usage(prgname); > + return -1; > + } > + > + return 0; > +} > + > +static int > +data_init(void) > +{ > + dev_total = rte_vdpa_get_device_num(); > + if (dev_total <= 0) { > + printf("No available vdpa device found\n"); > + return -1; > + } > + return 0; > +} > + > +static int > +new_device(int vid) > +{ > + char ifname[MAX_PATH_LEN]; > + int i; > + > + rte_vhost_get_ifname(vid, ifname, sizeof(ifname)); > + for (i = 0; i < MAX_VDPA_SAMPLE_PORTS; i++) { > + if (strncmp(ifname, vports[i].ifname, MAX_PATH_LEN) == 0) { > + printf("\nnew port %s, did: %d\n", > + ifname, vports[i].did); > + vports[i].vid = vid; > + break; > + } > + } > + > + if (i >= MAX_VDPA_SAMPLE_PORTS) > + return -1; > + > + return 0; > +} > + > +static void > +destroy_device(int vid) > +{ > + char ifname[MAX_PATH_LEN]; > + int i; > + > + rte_vhost_get_ifname(vid, ifname, sizeof(ifname)); > + for (i = 0; i < MAX_VDPA_SAMPLE_PORTS; i++) { > + if (strcmp(ifname, vports[i].ifname) == 0) { > + printf("\ndestroy port %s, did: %d\n", > + ifname, vports[i].did); > + break; > + } > + } > +} > + > +static const struct vhost_device_ops vdpa_sample_devops = { > + .new_device = new_device, > + .destroy_device = destroy_device, > +}; > + > +static int > +start_vdpa(struct vdpa_port *vport) > +{ > + int ret; > + char *socket_path = vport->ifname; > + int did = vport->did; > + > + if (client_mode) > + vport->flags |= RTE_VHOST_USER_CLIENT; > + > + if (access(socket_path, F_OK) != -1 && !client_mode) { > + RTE_LOG(ERR, VDPA, > + "%s exists, please remove it or specify another file and try again.\n", > + socket_path); > + return -1; > + } > + ret = rte_vhost_driver_register(socket_path, vport->flags); > + if (ret != 0) > + rte_exit(EXIT_FAILURE, > + "register driver failed: %s\n", > + socket_path); > + > + ret = rte_vhost_driver_callback_register(socket_path, > + &vdpa_sample_devops); > + if (ret != 0) > + rte_exit(EXIT_FAILURE, > + "register driver ops failed: %s\n", > + socket_path); > + > + ret = rte_vhost_driver_attach_vdpa_device(socket_path, did); > + if (ret != 0) > + rte_exit(EXIT_FAILURE, > + "attach vdpa device failed: %s\n", > + socket_path); > + > + if (rte_vhost_driver_start(socket_path) < 0) > + rte_exit(EXIT_FAILURE, > + "start vhost driver failed: %s\n", > + socket_path); > + return 0; > +} > + > +static void > +close_vdpa(struct vdpa_port *vport) > +{ > + int ret; > + char *socket_path = vport->ifname; > + > + ret = rte_vhost_driver_detach_vdpa_device(socket_path); > + if (ret != 0) > + RTE_LOG(ERR, VDPA, > + "detach vdpa device failed: %s\n", > + socket_path); > + > + ret = rte_vhost_driver_unregister(socket_path); > + if (ret != 0) > + RTE_LOG(ERR, VDPA, > + "Fail to unregister vhost driver for %s.\n", > + socket_path); > +} > + > +static void > +vdpa_sample_quit(void) > +{ > + int i; > + for (i = 0; i < RTE_MIN(MAX_VDPA_SAMPLE_PORTS, dev_total); i++) { > + if (vports[i].ifname[0] != '\0') > + close_vdpa(&vports[i]); > + } > +} > + > +static void > +signal_handler(int signum) > +{ > + if (signum == SIGINT || signum == SIGTERM) { > + printf("\nSignal %d received, preparing to exit...\n", signum); > + vdpa_sample_quit(); > + exit(0); > + } > +} > + > +/* interactive cmds */ > + > +/* *** Help command with introduction. *** */ > +struct cmd_help_result { > + cmdline_fixed_string_t help; > +}; > + > +static void cmd_help_parsed(__attribute__((unused)) void *parsed_result, > + struct cmdline *cl, > + __attribute__((unused)) void *data) > +{ > + cmdline_printf( > + cl, > + "\n" > + "The following commands are currently available:\n\n" > + "Control:\n" > + " help : Show interactive instructions.\n" > + " list : list all available vdpa devices.\n" > + " create <socket file> <vdev addr> : create a new vdpa port.\n" > + " quit : exit vdpa sample app.\n" > + ); > +} > + > +cmdline_parse_token_string_t cmd_help_help = > + TOKEN_STRING_INITIALIZER(struct cmd_help_result, help, "help"); > + > +cmdline_parse_inst_t cmd_help = { > + .f = cmd_help_parsed, > + .data = NULL, > + .help_str = "show help", > + .tokens = { > + (void *)&cmd_help_help, > + NULL, > + }, > +}; > + > +/* *** List all available vdpa devices *** */ > +struct cmd_list_result { > + cmdline_fixed_string_t action; > +}; > + > +static void cmd_list_vdpa_devices_parsed( > + __attribute__((unused)) void *parsed_result, > + struct cmdline *cl, > + __attribute__((unused)) void *data) > +{ > + int did; > + uint32_t queue_num; > + uint64_t features; > + struct rte_vdpa_device *vdev; > + struct rte_pci_addr addr; > + > + cmdline_printf(cl, "device id\tdevice address\tqueue num\tsupported features\n"); > + for (did = 0; did < dev_total; did++) { > + vdev = rte_vdpa_get_device(did); > + if (!vdev) > + continue; > + if (vdev->ops->get_queue_num(did, &queue_num) < 0) { > + RTE_LOG(ERR, VDPA, > + "failed to get vdpa queue number " > + "for device id %d.\n", did); > + continue; > + } > + if (vdev->ops->get_features(did, &features) < 0) { > + RTE_LOG(ERR, VDPA, > + "failed to get vdpa features " > + "for device id %d.\n", did); > + continue; > + } > + addr = vdev->addr.pci_addr; > + cmdline_printf(cl, > + "%d\t\t" PCI_PRI_FMT "\t%" PRIu32 "\t\t0x%" PRIx64 "\n", > + did, addr.domain, addr.bus, addr.devid, > + addr.function, queue_num, features); > + } > +} > + > +cmdline_parse_token_string_t cmd_action_list = > + TOKEN_STRING_INITIALIZER(struct cmd_list_result, action, "list"); > + > +cmdline_parse_inst_t cmd_list_vdpa_devices = { > + .f = cmd_list_vdpa_devices_parsed, > + .data = NULL, > + .help_str = "list all available vdpa devices", > + .tokens = { > + (void *)&cmd_action_list, > + NULL, > + }, > +}; > + > +/* *** Create new vdpa port *** */ > +struct cmd_create_result { > + cmdline_fixed_string_t action; > + cmdline_fixed_string_t socket_path; > + cmdline_fixed_string_t bdf; > +}; > + > +static void cmd_create_vdpa_port_parsed(void *parsed_result, > + struct cmdline *cl, > + __attribute__((unused)) void *data) > +{ > + int did; > + struct cmd_create_result *res = parsed_result; > + struct rte_vdpa_dev_addr addr; > + > + strncpy(vports[devcnt].ifname, res->socket_path, MAX_PATH_LEN); > + if (rte_pci_addr_parse(res->bdf, &addr.pci_addr) != 0) { > + cmdline_printf(cl, "Unable to parse the given bdf.\n"); > + return; > + } > + addr.type = PCI_ADDR; > + did = rte_vdpa_find_device_id(&addr); > + if (did < 0) { > + cmdline_printf(cl, "Unable to find vdpa devide id.\n"); > + return; > + } > + > + vports[devcnt].did = did; > + > + if (start_vdpa(&vports[devcnt]) == 0) > + devcnt++; > +} > + > +cmdline_parse_token_string_t cmd_action_create = > + TOKEN_STRING_INITIALIZER(struct cmd_create_result, action, "create"); > +cmdline_parse_token_string_t cmd_socket_path = > + TOKEN_STRING_INITIALIZER(struct cmd_create_result, socket_path, NULL); > +cmdline_parse_token_string_t cmd_bdf = > + TOKEN_STRING_INITIALIZER(struct cmd_create_result, bdf, NULL); > + > +cmdline_parse_inst_t cmd_create_vdpa_port = { > + .f = cmd_create_vdpa_port_parsed, > + .data = NULL, > + .help_str = "create a new vdpa port", > + .tokens = { > + (void *)&cmd_action_create, > + (void *)&cmd_socket_path, > + (void *)&cmd_bdf, > + NULL, > + }, > +}; > + > +/* *** QUIT *** */ > +struct cmd_quit_result { > + cmdline_fixed_string_t quit; > +}; > + > +static void cmd_quit_parsed(__attribute__((unused)) void *parsed_result, > + struct cmdline *cl, > + __attribute__((unused)) void *data) > +{ > + vdpa_sample_quit(); > + cmdline_quit(cl); > +} > + > +cmdline_parse_token_string_t cmd_quit_quit = > + TOKEN_STRING_INITIALIZER(struct cmd_quit_result, quit, "quit"); > + > +cmdline_parse_inst_t cmd_quit = { > + .f = cmd_quit_parsed, > + .data = NULL, > + .help_str = "quit: exit application", > + .tokens = { > + (void *)&cmd_quit_quit, > + NULL, > + }, > +}; > +cmdline_parse_ctx_t main_ctx[] = { > + (cmdline_parse_inst_t *)&cmd_help, > + (cmdline_parse_inst_t *)&cmd_list_vdpa_devices, > + (cmdline_parse_inst_t *)&cmd_create_vdpa_port, > + (cmdline_parse_inst_t *)&cmd_quit, > + NULL, > +}; > + > +int > +main(int argc, char *argv[]) > +{ > + char ch; > + int i; > + int ret; > + struct cmdline *cl; > + > + ret = rte_eal_init(argc, argv); > + if (ret < 0) > + rte_exit(EXIT_FAILURE, "eal init failed\n"); > + argc -= ret; > + argv += ret; > + > + data_init(); You need to check return from data_init(). If this is fine for you, I can change to below code when applying: ret = data_init(); if (ret < 0) rte_exit(EXIT_FAILURE, "data init failed\n"); > + > + signal(SIGINT, signal_handler); > + signal(SIGTERM, signal_handler); > + > + ret = parse_args(argc, argv); > + if (ret < 0) > + rte_exit(EXIT_FAILURE, "invalid argument\n"); > + > + if (interactive == 1) { > + cl = cmdline_stdin_new(main_ctx, "vdpa> "); > + if (cl == NULL) > + rte_panic("Cannot create cmdline instance\n"); > + cmdline_interact(cl); > + cmdline_stdin_exit(cl); > + } else { > + for (i = 0; i < RTE_MIN(MAX_VDPA_SAMPLE_PORTS, dev_total); > + i++) { > + vports[i].did = i; > + snprintf(vports[i].ifname, MAX_PATH_LEN, "%s%d", > + iface, i); > + > + start_vdpa(&vports[i]); > + } > + > + printf("enter \'q\' to quit\n"); > + while (scanf("%c", &ch)) { > + if (ch == 'q') > + break; > + while (ch != '\n') { > + if (scanf("%c", &ch)) > + printf("%c", ch); > + } > + printf("enter \'q\' to quit\n"); > + } > + vdpa_sample_quit(); > + } > + > + return 0; > +} > diff --git a/examples/vdpa/meson.build b/examples/vdpa/meson.build > new file mode 100644 > index 000000000..2e38a069f > --- /dev/null > +++ b/examples/vdpa/meson.build > @@ -0,0 +1,16 @@ > +# SPDX-License-Identifier: BSD-3-Clause > +# Copyright(c) 2018 Intel Corporation > + > +# meson file, for building this example as part of a main DPDK build. > +# > +# To build this example as a standalone application with an already-installed > +# DPDK instance, use 'make' > + > +if host_machine.system() != 'linux' > + build = false > +endif > +deps += 'vhost' > +allow_experimental_apis = true > +sources = files( > + 'main.c' > +) > \ No newline at end of file > ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [dpdk-dev] [PATCH v7 2/2] examples/vdpa: introduce a new sample for vDPA 2018-09-28 7:31 ` Maxime Coquelin @ 2018-09-28 14:33 ` Ye Xiaolong 2018-09-28 7:49 ` Maxime Coquelin 0 siblings, 1 reply; 41+ messages in thread From: Ye Xiaolong @ 2018-09-28 14:33 UTC (permalink / raw) To: Maxime Coquelin Cc: dev, Tiwei Bie, Zhihong Wang, xiao.w.wang, Rami Rosen, Wang Haiyue On 09/28, Maxime Coquelin wrote: >> + ret = rte_eal_init(argc, argv); >> + if (ret < 0) >> + rte_exit(EXIT_FAILURE, "eal init failed\n"); >> + argc -= ret; >> + argv += ret; >> + >> + data_init(); > >You need to check return from data_init(). >If this is fine for you, I can change to below code when applying: > >ret = data_init(); >if (ret < 0) > rte_exit(EXIT_FAILURE, "data init failed\n"); > > I'm fine with this change. Thanks, Xiaolong ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [dpdk-dev] [PATCH v7 2/2] examples/vdpa: introduce a new sample for vDPA 2018-09-28 14:33 ` Ye Xiaolong @ 2018-09-28 7:49 ` Maxime Coquelin 2018-09-28 15:17 ` Ye Xiaolong 0 siblings, 1 reply; 41+ messages in thread From: Maxime Coquelin @ 2018-09-28 7:49 UTC (permalink / raw) To: Ye Xiaolong Cc: dev, Tiwei Bie, Zhihong Wang, xiao.w.wang, Rami Rosen, Wang Haiyue On 09/28/2018 04:33 PM, Ye Xiaolong wrote: > On 09/28, Maxime Coquelin wrote: >>> + ret = rte_eal_init(argc, argv); >>> + if (ret < 0) >>> + rte_exit(EXIT_FAILURE, "eal init failed\n"); >>> + argc -= ret; >>> + argv += ret; >>> + >>> + data_init(); >> >> You need to check return from data_init(). >> If this is fine for you, I can change to below code when applying: >> >> ret = data_init(); >> if (ret < 0) >> rte_exit(EXIT_FAILURE, "data init failed\n"); >> >> > > I'm fine with this change. Or even better, get rid of data_init and call rte_vdpa_get_device_num(); in main directly. > Thanks, > Xiaolong > ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [dpdk-dev] [PATCH v7 2/2] examples/vdpa: introduce a new sample for vDPA 2018-09-28 7:49 ` Maxime Coquelin @ 2018-09-28 15:17 ` Ye Xiaolong 2018-09-28 8:31 ` Maxime Coquelin 0 siblings, 1 reply; 41+ messages in thread From: Ye Xiaolong @ 2018-09-28 15:17 UTC (permalink / raw) To: Maxime Coquelin Cc: dev, Tiwei Bie, Zhihong Wang, xiao.w.wang, Rami Rosen, Wang Haiyue On 09/28, Maxime Coquelin wrote: > > >On 09/28/2018 04:33 PM, Ye Xiaolong wrote: >> On 09/28, Maxime Coquelin wrote: >> > > + ret = rte_eal_init(argc, argv); >> > > + if (ret < 0) >> > > + rte_exit(EXIT_FAILURE, "eal init failed\n"); >> > > + argc -= ret; >> > > + argv += ret; >> > > + >> > > + data_init(); >> > >> > You need to check return from data_init(). >> > If this is fine for you, I can change to below code when applying: >> > >> > ret = data_init(); >> > if (ret < 0) >> > rte_exit(EXIT_FAILURE, "data init failed\n"); >> > >> > >> >> I'm fine with this change. > >Or even better, get rid of data_init and >call rte_vdpa_get_device_num(); in main directly. Sounds better, as we've removed unnecessary static variable initialization in data_init, it does nothing but call rte_vdpa_get_device_num, we can remove this wrap. Would you do this change when applying or you perfer I send a new revision? Thanks, Xiaolong > >> Thanks, >> Xiaolong >> ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [dpdk-dev] [PATCH v7 2/2] examples/vdpa: introduce a new sample for vDPA 2018-09-28 15:17 ` Ye Xiaolong @ 2018-09-28 8:31 ` Maxime Coquelin 2018-09-28 17:37 ` Ye Xiaolong 0 siblings, 1 reply; 41+ messages in thread From: Maxime Coquelin @ 2018-09-28 8:31 UTC (permalink / raw) To: Ye Xiaolong Cc: dev, Tiwei Bie, Zhihong Wang, xiao.w.wang, Rami Rosen, Wang Haiyue On 09/28/2018 05:17 PM, Ye Xiaolong wrote: > On 09/28, Maxime Coquelin wrote: >> >> >> On 09/28/2018 04:33 PM, Ye Xiaolong wrote: >>> On 09/28, Maxime Coquelin wrote: >>>>> + ret = rte_eal_init(argc, argv); >>>>> + if (ret < 0) >>>>> + rte_exit(EXIT_FAILURE, "eal init failed\n"); >>>>> + argc -= ret; >>>>> + argv += ret; >>>>> + >>>>> + data_init(); >>>> >>>> You need to check return from data_init(). >>>> If this is fine for you, I can change to below code when applying: >>>> >>>> ret = data_init(); >>>> if (ret < 0) >>>> rte_exit(EXIT_FAILURE, "data init failed\n"); >>>> >>>> >>> >>> I'm fine with this change. >> >> Or even better, get rid of data_init and >> call rte_vdpa_get_device_num(); in main directly. > > Sounds better, as we've removed unnecessary static variable initialization in > data_init, it does nothing but call rte_vdpa_get_device_num, we can remove this > wrap. Would you do this change when applying or you perfer I send a new > revision? Please send a new revision, and in the commit message, fix first letter of my family name with an upper case, and move my R-b below your sign-off. Thanks! Maxime > Thanks, > Xiaolong > > >> >>> Thanks, >>> Xiaolong >>> ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [dpdk-dev] [PATCH v7 2/2] examples/vdpa: introduce a new sample for vDPA 2018-09-28 8:31 ` Maxime Coquelin @ 2018-09-28 17:37 ` Ye Xiaolong 0 siblings, 0 replies; 41+ messages in thread From: Ye Xiaolong @ 2018-09-28 17:37 UTC (permalink / raw) To: Maxime Coquelin Cc: dev, Tiwei Bie, Zhihong Wang, xiao.w.wang, Rami Rosen, Wang Haiyue On 09/28, Maxime Coquelin wrote: > > >On 09/28/2018 05:17 PM, Ye Xiaolong wrote: >> On 09/28, Maxime Coquelin wrote: >> > >> > >> > On 09/28/2018 04:33 PM, Ye Xiaolong wrote: >> > > On 09/28, Maxime Coquelin wrote: >> > > > > + ret = rte_eal_init(argc, argv); >> > > > > + if (ret < 0) >> > > > > + rte_exit(EXIT_FAILURE, "eal init failed\n"); >> > > > > + argc -= ret; >> > > > > + argv += ret; >> > > > > + >> > > > > + data_init(); >> > > > >> > > > You need to check return from data_init(). >> > > > If this is fine for you, I can change to below code when applying: >> > > > >> > > > ret = data_init(); >> > > > if (ret < 0) >> > > > rte_exit(EXIT_FAILURE, "data init failed\n"); >> > > > >> > > > >> > > >> > > I'm fine with this change. >> > >> > Or even better, get rid of data_init and >> > call rte_vdpa_get_device_num(); in main directly. >> >> Sounds better, as we've removed unnecessary static variable initialization in >> data_init, it does nothing but call rte_vdpa_get_device_num, we can remove this >> wrap. Would you do this change when applying or you perfer I send a new >> revision? > >Please send a new revision, and in the commit message, >fix first letter of my family name with an upper case, >and move my R-b below your sign-off. Got it. Thanks, Xiaolong > >Thanks! >Maxime > >> Thanks, >> Xiaolong >> >> >> > >> > > Thanks, >> > > Xiaolong >> > > ^ permalink raw reply [flat|nested] 41+ messages in thread
* [dpdk-dev] [PATCH v8 0/2] introduce vdpa sample 2018-09-28 11:23 ` [dpdk-dev] [PATCH v7 0/2] introduce vdpa sample Xiaolong Ye 2018-09-28 11:23 ` [dpdk-dev] [PATCH v7 1/2] vhost: introduce API to get vDPA device number Xiaolong Ye 2018-09-28 11:23 ` [dpdk-dev] [PATCH v7 2/2] examples/vdpa: introduce a new sample for vDPA Xiaolong Ye @ 2018-09-28 21:47 ` Xiaolong Ye 2018-09-28 21:47 ` [dpdk-dev] [PATCH v8 1/2] vhost: introduce API to get vDPA device number Xiaolong Ye ` (2 more replies) 2 siblings, 3 replies; 41+ messages in thread From: Xiaolong Ye @ 2018-09-28 21:47 UTC (permalink / raw) To: dev, Maxime Coquelin, Tiwei Bie, Zhihong Wang Cc: xiao.w.wang, Rami Rosen, Wang Haiyue, Xiaolong Ye Hi, This patchset introduces vdpa sample to demonstrate the vDPA use case. v8 changes: * move body of data_init to the main function * adjust Reviewed-by/Acked-by tag postion v7 changes: * make vports static * avoid unnecessary static variable initialization v6 changes: * improve the document according to Xiao's comments * fix a typo, PRIu64 -> PRIx64 v5 changes: * improve print format and correct from "PRIu64" to "PRIx64" * use "-c 0x2" to better demonstrate app doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations in vdpa.rst v4 changes: * add client mode support * improve the format to list the vDPA device info and improve the vdpa.rst accordingly * remove some useless comments * add introduction in 18.11 release note. v3 changes: * list cmd would show queue number and supported features of vdpa devices. * address Xiao's review comments v2 changes: * fix a compilation error reported by Rosen * improve create cmd in interactive mode and add two new cmds: list, quit * add application documentation Xiaolong Ye (2): vhost: introduce API to get vDPA device number examples/vdpa: introduce a new sample for vDPA MAINTAINERS | 2 + doc/guides/rel_notes/release_18_11.rst | 8 + doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/vdpa.rst | 120 +++++++ examples/Makefile | 2 +- examples/vdpa/Makefile | 32 ++ examples/vdpa/main.c | 453 +++++++++++++++++++++++++ examples/vdpa/meson.build | 16 + lib/librte_vhost/rte_vdpa.h | 3 + lib/librte_vhost/rte_vhost_version.map | 1 + lib/librte_vhost/vdpa.c | 6 + 11 files changed, 643 insertions(+), 1 deletion(-) create mode 100644 doc/guides/sample_app_ug/vdpa.rst create mode 100644 examples/vdpa/Makefile create mode 100644 examples/vdpa/main.c create mode 100644 examples/vdpa/meson.build -- 2.17.1 ^ permalink raw reply [flat|nested] 41+ messages in thread
* [dpdk-dev] [PATCH v8 1/2] vhost: introduce API to get vDPA device number 2018-09-28 21:47 ` [dpdk-dev] [PATCH v8 0/2] introduce vdpa sample Xiaolong Ye @ 2018-09-28 21:47 ` Xiaolong Ye 2018-10-02 13:19 ` Ferruh Yigit 2018-09-28 21:47 ` [dpdk-dev] [PATCH v8 2/2] examples/vdpa: introduce a new sample for vDPA Xiaolong Ye 2018-10-02 12:51 ` [dpdk-dev] [PATCH v8 0/2] introduce vdpa sample Maxime Coquelin 2 siblings, 1 reply; 41+ messages in thread From: Xiaolong Ye @ 2018-09-28 21:47 UTC (permalink / raw) To: dev, Maxime Coquelin, Tiwei Bie, Zhihong Wang Cc: xiao.w.wang, Rami Rosen, Wang Haiyue, Xiaolong Ye It's used to get number of available registered vDPA devices. Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com> Acked-by: Xiao Wang <xiao.w.wang@intel.com> Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com> --- lib/librte_vhost/rte_vdpa.h | 3 +++ lib/librte_vhost/rte_vhost_version.map | 1 + lib/librte_vhost/vdpa.c | 6 ++++++ 3 files changed, 10 insertions(+) diff --git a/lib/librte_vhost/rte_vdpa.h b/lib/librte_vhost/rte_vdpa.h index 90465ca26..b8223e337 100644 --- a/lib/librte_vhost/rte_vdpa.h +++ b/lib/librte_vhost/rte_vdpa.h @@ -84,4 +84,7 @@ rte_vdpa_find_device_id(struct rte_vdpa_dev_addr *addr); struct rte_vdpa_device * __rte_experimental rte_vdpa_get_device(int did); +/* Get current available vdpa device number */ +int __rte_experimental +rte_vdpa_get_device_num(void); #endif /* _RTE_VDPA_H_ */ diff --git a/lib/librte_vhost/rte_vhost_version.map b/lib/librte_vhost/rte_vhost_version.map index da220dd02..ae39b6e21 100644 --- a/lib/librte_vhost/rte_vhost_version.map +++ b/lib/librte_vhost/rte_vhost_version.map @@ -67,6 +67,7 @@ EXPERIMENTAL { rte_vdpa_unregister_device; rte_vdpa_find_device_id; rte_vdpa_get_device; + rte_vdpa_get_device_num; rte_vhost_driver_attach_vdpa_device; rte_vhost_driver_detach_vdpa_device; rte_vhost_driver_get_vdpa_device_id; diff --git a/lib/librte_vhost/vdpa.c b/lib/librte_vhost/vdpa.c index c82fd4370..c2c5dff1d 100644 --- a/lib/librte_vhost/vdpa.c +++ b/lib/librte_vhost/vdpa.c @@ -113,3 +113,9 @@ rte_vdpa_get_device(int did) return vdpa_devices[did]; } + +int +rte_vdpa_get_device_num(void) +{ + return vdpa_device_num; +} -- 2.17.1 ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [dpdk-dev] [PATCH v8 1/2] vhost: introduce API to get vDPA device number 2018-09-28 21:47 ` [dpdk-dev] [PATCH v8 1/2] vhost: introduce API to get vDPA device number Xiaolong Ye @ 2018-10-02 13:19 ` Ferruh Yigit 2018-10-03 5:53 ` Ye Xiaolong 0 siblings, 1 reply; 41+ messages in thread From: Ferruh Yigit @ 2018-10-02 13:19 UTC (permalink / raw) To: Xiaolong Ye, dev, Maxime Coquelin, Tiwei Bie, Zhihong Wang Cc: xiao.w.wang, Rami Rosen, Wang Haiyue On 9/28/2018 10:47 PM, Xiaolong Ye wrote: > It's used to get number of available registered vDPA devices. > > Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com> > Acked-by: Xiao Wang <xiao.w.wang@intel.com> > Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com> > --- > lib/librte_vhost/rte_vdpa.h | 3 +++ > lib/librte_vhost/rte_vhost_version.map | 1 + > lib/librte_vhost/vdpa.c | 6 ++++++ > 3 files changed, 10 insertions(+) > > diff --git a/lib/librte_vhost/rte_vdpa.h b/lib/librte_vhost/rte_vdpa.h > index 90465ca26..b8223e337 100644 > --- a/lib/librte_vhost/rte_vdpa.h > +++ b/lib/librte_vhost/rte_vdpa.h > @@ -84,4 +84,7 @@ rte_vdpa_find_device_id(struct rte_vdpa_dev_addr *addr); > struct rte_vdpa_device * __rte_experimental > rte_vdpa_get_device(int did); > > +/* Get current available vdpa device number */ > +int __rte_experimental > +rte_vdpa_get_device_num(void); Hi Xiaolong, Is rte_vdpa_get_device_num() an public API? (It should be since it will be used in an application in next patch.) If so, header needs to be part of API documentation: 1- Update doc/api/doxy-api-index.md to include this header. 2- Update all APIs comments to be an doxygen comment. 3- Add doxygen comment to all data structures. I can see rte_vdpa.h not documented at all, it is not just this API, so it can be better to send a new patch to address above items. Thanks, ferruh ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [dpdk-dev] [PATCH v8 1/2] vhost: introduce API to get vDPA device number 2018-10-02 13:19 ` Ferruh Yigit @ 2018-10-03 5:53 ` Ye Xiaolong 0 siblings, 0 replies; 41+ messages in thread From: Ye Xiaolong @ 2018-10-03 5:53 UTC (permalink / raw) To: Ferruh Yigit Cc: dev, Maxime Coquelin, Tiwei Bie, Zhihong Wang, xiao.w.wang, Rami Rosen, Wang Haiyue Hi, Ferruh, On 10/02, Ferruh Yigit wrote: >On 9/28/2018 10:47 PM, Xiaolong Ye wrote: >> It's used to get number of available registered vDPA devices. >> >> Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com> >> Acked-by: Xiao Wang <xiao.w.wang@intel.com> >> Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com> >> --- >> lib/librte_vhost/rte_vdpa.h | 3 +++ >> lib/librte_vhost/rte_vhost_version.map | 1 + >> lib/librte_vhost/vdpa.c | 6 ++++++ >> 3 files changed, 10 insertions(+) >> >> diff --git a/lib/librte_vhost/rte_vdpa.h b/lib/librte_vhost/rte_vdpa.h >> index 90465ca26..b8223e337 100644 >> --- a/lib/librte_vhost/rte_vdpa.h >> +++ b/lib/librte_vhost/rte_vdpa.h >> @@ -84,4 +84,7 @@ rte_vdpa_find_device_id(struct rte_vdpa_dev_addr *addr); >> struct rte_vdpa_device * __rte_experimental >> rte_vdpa_get_device(int did); >> >> +/* Get current available vdpa device number */ >> +int __rte_experimental >> +rte_vdpa_get_device_num(void); > >Hi Xiaolong, > >Is rte_vdpa_get_device_num() an public API? (It should be since it will be used >in an application in next patch.) > >If so, header needs to be part of API documentation: >1- Update doc/api/doxy-api-index.md to include this header. >2- Update all APIs comments to be an doxygen comment. >3- Add doxygen comment to all data structures. > >I can see rte_vdpa.h not documented at all, it is not just this API, so it can >be better to send a new patch to address above items. Thanks for the reminder, I'll send a new patch for it. Thanks, Xiaolong > >Thanks, >ferruh ^ permalink raw reply [flat|nested] 41+ messages in thread
* [dpdk-dev] [PATCH v8 2/2] examples/vdpa: introduce a new sample for vDPA 2018-09-28 21:47 ` [dpdk-dev] [PATCH v8 0/2] introduce vdpa sample Xiaolong Ye 2018-09-28 21:47 ` [dpdk-dev] [PATCH v8 1/2] vhost: introduce API to get vDPA device number Xiaolong Ye @ 2018-09-28 21:47 ` Xiaolong Ye 2018-10-02 14:03 ` Ferruh Yigit 2019-01-17 14:37 ` Shahaf Shuler 2018-10-02 12:51 ` [dpdk-dev] [PATCH v8 0/2] introduce vdpa sample Maxime Coquelin 2 siblings, 2 replies; 41+ messages in thread From: Xiaolong Ye @ 2018-09-28 21:47 UTC (permalink / raw) To: dev, Maxime Coquelin, Tiwei Bie, Zhihong Wang Cc: xiao.w.wang, Rami Rosen, Wang Haiyue, Xiaolong Ye The vdpa sample application creates vhost-user sockets by using the vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio ring compatible devices to serve virtio driver directly to enable datapath acceleration. As vDPA driver can help to set up vhost datapath, this application doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations. Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com> Acked-by: Xiao Wang <xiao.w.wang@intel.com> Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com> --- MAINTAINERS | 2 + doc/guides/rel_notes/release_18_11.rst | 8 + doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/vdpa.rst | 120 +++++++ examples/Makefile | 2 +- examples/vdpa/Makefile | 32 ++ examples/vdpa/main.c | 453 +++++++++++++++++++++++++ examples/vdpa/meson.build | 16 + 8 files changed, 633 insertions(+), 1 deletion(-) create mode 100644 doc/guides/sample_app_ug/vdpa.rst create mode 100644 examples/vdpa/Makefile create mode 100644 examples/vdpa/main.c create mode 100644 examples/vdpa/meson.build diff --git a/MAINTAINERS b/MAINTAINERS index 5967c1dd3..5656f18e8 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -683,6 +683,8 @@ F: doc/guides/sample_app_ug/vhost.rst F: examples/vhost_scsi/ F: doc/guides/sample_app_ug/vhost_scsi.rst F: examples/vhost_crypto/ +F: examples/vdpa/ +F: doc/guides/sample_app_ug/vdpa.rst Vhost PMD M: Maxime Coquelin <maxime.coquelin@redhat.com> diff --git a/doc/guides/rel_notes/release_18_11.rst b/doc/guides/rel_notes/release_18_11.rst index bc9b74ec4..dd53a9ecf 100644 --- a/doc/guides/rel_notes/release_18_11.rst +++ b/doc/guides/rel_notes/release_18_11.rst @@ -67,6 +67,14 @@ New Features SR-IOV option in Hyper-V and Azure. This is an alternative to the previous vdev_netvsc, tap, and failsafe drivers combination. +* **Add a new sample for vDPA** + + The vdpa sample application creates vhost-user sockets by using the + vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes + virtio ring compatible devices to serve virtio driver directly to enable + datapath acceleration. As vDPA driver can help to set up vhost datapath, + this application doesn't need to launch dedicated worker threads for vhost + enqueue/dequeue operations. API Changes ----------- diff --git a/doc/guides/sample_app_ug/index.rst b/doc/guides/sample_app_ug/index.rst index 5bedf4f6f..74b12af85 100644 --- a/doc/guides/sample_app_ug/index.rst +++ b/doc/guides/sample_app_ug/index.rst @@ -45,6 +45,7 @@ Sample Applications User Guides vhost vhost_scsi vhost_crypto + vdpa netmap_compatibility ip_pipeline test_pipeline diff --git a/doc/guides/sample_app_ug/vdpa.rst b/doc/guides/sample_app_ug/vdpa.rst new file mode 100644 index 000000000..745f196ca --- /dev/null +++ b/doc/guides/sample_app_ug/vdpa.rst @@ -0,0 +1,120 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2018 Intel Corporation. + +Vdpa Sample Application +======================= + +The vdpa sample application creates vhost-user sockets by using the +vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes +virtio ring compatible devices to serve virtio driver directly to enable +datapath acceleration. As vDPA driver can help to set up vhost datapath, +this application doesn't need to launch dedicated worker threads for vhost +enqueue/dequeue operations. + +Testing steps +------------- + +This section shows the steps of how to start VMs with vDPA vhost-user +backend and verify network connection & live migration. + +Build +~~~~~ + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``vdpa`` sub-directory. + +Start the vdpa example +~~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: console + + ./vdpa [EAL options] -- [--client] [--interactive|-i] or [--iface SOCKET_PATH] + +where + +* --client means running vdpa app in client mode, in the client mode, QEMU needs + to run as the server mode and take charge of socket file creation. +* --iface specifies the path prefix of the UNIX domain socket file, e.g. + /tmp/vhost-user-, then the socket files will be named as /tmp/vhost-user-<n> + (n starts from 0). +* --interactive means run the vdpa sample in interactive mode, currently 4 + internal cmds are supported: + + 1. help: show help message + 2. list: list all available vdpa devices + 3. create: create a new vdpa port with socket file and vdpa device address + 4. quit: unregister vhost driver and exit the application + +Take IFCVF driver for example: + +.. code-block:: console + + ./vdpa -c 0x2 -n 4 --socket-mem 1024,1024 \ + -w 0000:06:00.3,vdpa=1 -w 0000:06:00.4,vdpa=1 \ + -- --interactive + +.. note:: + Here 0000:06:00.3 and 0000:06:00.4 refer to virtio ring compatible devices, + and we need to bind vfio-pci to them before running vdpa sample. + + * modprobe vfio-pci + * ./usertools/dpdk-devbind.py -b vfio-pci 06:00.3 06:00.4 + +Then we can create 2 vdpa ports in interactive cmdline. + +.. code-block:: console + + vdpa> list + device id device address queue num supported features + 0 0000:06:00.3 1 0x14c238020 + 1 0000:06:00.4 1 0x14c238020 + 2 0000:06:00.5 1 0x14c238020 + + vdpa> create /tmp/vdpa-socket0 0000:06:00.3 + vdpa> create /tmp/vdpa-socket1 0000:06:00.4 + +.. _vdpa_app_run_vm: + +Start the VMs +~~~~~~~~~~~~~ + +.. code-block:: console + + qemu-system-x86_64 -cpu host -enable-kvm \ + <snip> + -mem-prealloc \ + -chardev socket,id=char0,path=<socket_file created in above steps> \ + -netdev type=vhost-user,id=vdpa,chardev=char0 \ + -device virtio-net-pci,netdev=vdpa,mac=00:aa:bb:cc:dd:ee,page-per-vq=on \ + +After the VMs launches, we can login the VMs and configure the ip, verify the +network connection via ping or netperf. + +.. note:: + Suggest to use QEMU 3.0.0 which extends vhost-user for vDPA. + +Live Migration +~~~~~~~~~~~~~~ +vDPA supports cross-backend live migration, user can migrate SW vhost backend +VM to vDPA backend VM and vice versa. Here are the detailed steps. Assume A is +the source host with SW vhost VM and B is the destination host with vDPA. + +1. Start vdpa sample and launch a VM with exact same parameters as the VM on A, + in migration-listen mode: + +.. code-block:: console + + B: <qemu-command-line> -incoming tcp:0:4444 (or other PORT)) + +2. Start the migration (on source host): + +.. code-block:: console + + A: (qemu) migrate -d tcp:<B ip>:4444 (or other PORT) + +3. Check the status (on source host): + +.. code-block:: console + + A: (qemu) info migrate diff --git a/examples/Makefile b/examples/Makefile index 481720cb6..356fcb1cd 100644 --- a/examples/Makefile +++ b/examples/Makefile @@ -65,7 +65,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_HASH),y) DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += tep_termination endif DIRS-$(CONFIG_RTE_LIBRTE_TIMER) += timer -DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi +DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi vdpa ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y) DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_crypto endif diff --git a/examples/vdpa/Makefile b/examples/vdpa/Makefile new file mode 100644 index 000000000..42672a2bc --- /dev/null +++ b/examples/vdpa/Makefile @@ -0,0 +1,32 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +ifeq ($(RTE_SDK),) +$(error "Please define RTE_SDK environment variable") +endif + +# Default target, can be overridden by command line or environment +RTE_TARGET ?= x86_64-native-linuxapp-gcc + +include $(RTE_SDK)/mk/rte.vars.mk + +ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp") +$(info This application can only operate in a linuxapp environment, \ +please change the definition of the RTE_TARGET environment variable) +all: +else + +# binary name +APP = vdpa + +# all source are stored in SRCS-y +SRCS-y := main.c + +CFLAGS += -O2 -D_FILE_OFFSET_BITS=64 +CFLAGS += $(WERROR_FLAGS) +CFLAGS += -D_GNU_SOURCE +CFLAGS += -DALLOW_EXPERIMENTAL_API + +include $(RTE_SDK)/mk/rte.extapp.mk + +endif diff --git a/examples/vdpa/main.c b/examples/vdpa/main.c new file mode 100644 index 000000000..f96fa88b7 --- /dev/null +++ b/examples/vdpa/main.c @@ -0,0 +1,453 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2018 Intel Corporation + */ + +#include <getopt.h> +#include <signal.h> +#include <stdint.h> +#include <string.h> +#include <unistd.h> + +#include <rte_ethdev.h> +#include <rte_malloc.h> +#include <rte_vhost.h> +#include <rte_vdpa.h> +#include <rte_pci.h> + +#include <cmdline_parse.h> +#include <cmdline_socket.h> +#include <cmdline_parse_string.h> +#include <cmdline.h> + +#define MAX_PATH_LEN 128 +#define MAX_VDPA_SAMPLE_PORTS 1024 +#define RTE_LOGTYPE_VDPA RTE_LOGTYPE_USER1 + +struct vdpa_port { + char ifname[MAX_PATH_LEN]; + int did; + int vid; + uint64_t flags; +}; + +static struct vdpa_port vports[MAX_VDPA_SAMPLE_PORTS]; + +static char iface[MAX_PATH_LEN]; +static int dev_total; +static int devcnt; +static int interactive; +static int client_mode; + +/* display usage */ +static void +vdpa_usage(const char *prgname) +{ + printf("Usage: %s [EAL options] -- " + " --interactive|-i: run in interactive mode.\n" + " --iface <path>: specify the path prefix of the socket files, e.g. /tmp/vhost-user-.\n" + " --client: register a vhost-user socket as client mode.\n", + prgname); +} + +static int +parse_args(int argc, char **argv) +{ + static const char *short_option = "i"; + static struct option long_option[] = { + {"iface", required_argument, NULL, 0}, + {"interactive", no_argument, &interactive, 1}, + {"client", no_argument, &client_mode, 1}, + {NULL, 0, 0, 0}, + }; + int opt, idx; + char *prgname = argv[0]; + + while ((opt = getopt_long(argc, argv, short_option, long_option, &idx)) + != EOF) { + switch (opt) { + case 'i': + printf("Interactive-mode selected\n"); + interactive = 1; + break; + /* long options */ + case 0: + if (strncmp(long_option[idx].name, "iface", + MAX_PATH_LEN) == 0) { + strncpy(iface, optarg, MAX_PATH_LEN); + printf("iface %s\n", iface); + } + if (!strcmp(long_option[idx].name, "interactive")) { + printf("Interactive-mode selected\n"); + interactive = 1; + } + break; + + default: + vdpa_usage(prgname); + return -1; + } + } + + if (iface[0] == '\0' && interactive == 0) { + vdpa_usage(prgname); + return -1; + } + + return 0; +} + +static int +new_device(int vid) +{ + char ifname[MAX_PATH_LEN]; + int i; + + rte_vhost_get_ifname(vid, ifname, sizeof(ifname)); + for (i = 0; i < MAX_VDPA_SAMPLE_PORTS; i++) { + if (strncmp(ifname, vports[i].ifname, MAX_PATH_LEN) == 0) { + printf("\nnew port %s, did: %d\n", + ifname, vports[i].did); + vports[i].vid = vid; + break; + } + } + + if (i >= MAX_VDPA_SAMPLE_PORTS) + return -1; + + return 0; +} + +static void +destroy_device(int vid) +{ + char ifname[MAX_PATH_LEN]; + int i; + + rte_vhost_get_ifname(vid, ifname, sizeof(ifname)); + for (i = 0; i < MAX_VDPA_SAMPLE_PORTS; i++) { + if (strcmp(ifname, vports[i].ifname) == 0) { + printf("\ndestroy port %s, did: %d\n", + ifname, vports[i].did); + break; + } + } +} + +static const struct vhost_device_ops vdpa_sample_devops = { + .new_device = new_device, + .destroy_device = destroy_device, +}; + +static int +start_vdpa(struct vdpa_port *vport) +{ + int ret; + char *socket_path = vport->ifname; + int did = vport->did; + + if (client_mode) + vport->flags |= RTE_VHOST_USER_CLIENT; + + if (access(socket_path, F_OK) != -1 && !client_mode) { + RTE_LOG(ERR, VDPA, + "%s exists, please remove it or specify another file and try again.\n", + socket_path); + return -1; + } + ret = rte_vhost_driver_register(socket_path, vport->flags); + if (ret != 0) + rte_exit(EXIT_FAILURE, + "register driver failed: %s\n", + socket_path); + + ret = rte_vhost_driver_callback_register(socket_path, + &vdpa_sample_devops); + if (ret != 0) + rte_exit(EXIT_FAILURE, + "register driver ops failed: %s\n", + socket_path); + + ret = rte_vhost_driver_attach_vdpa_device(socket_path, did); + if (ret != 0) + rte_exit(EXIT_FAILURE, + "attach vdpa device failed: %s\n", + socket_path); + + if (rte_vhost_driver_start(socket_path) < 0) + rte_exit(EXIT_FAILURE, + "start vhost driver failed: %s\n", + socket_path); + return 0; +} + +static void +close_vdpa(struct vdpa_port *vport) +{ + int ret; + char *socket_path = vport->ifname; + + ret = rte_vhost_driver_detach_vdpa_device(socket_path); + if (ret != 0) + RTE_LOG(ERR, VDPA, + "detach vdpa device failed: %s\n", + socket_path); + + ret = rte_vhost_driver_unregister(socket_path); + if (ret != 0) + RTE_LOG(ERR, VDPA, + "Fail to unregister vhost driver for %s.\n", + socket_path); +} + +static void +vdpa_sample_quit(void) +{ + int i; + for (i = 0; i < RTE_MIN(MAX_VDPA_SAMPLE_PORTS, dev_total); i++) { + if (vports[i].ifname[0] != '\0') + close_vdpa(&vports[i]); + } +} + +static void +signal_handler(int signum) +{ + if (signum == SIGINT || signum == SIGTERM) { + printf("\nSignal %d received, preparing to exit...\n", signum); + vdpa_sample_quit(); + exit(0); + } +} + +/* interactive cmds */ + +/* *** Help command with introduction. *** */ +struct cmd_help_result { + cmdline_fixed_string_t help; +}; + +static void cmd_help_parsed(__attribute__((unused)) void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + cmdline_printf( + cl, + "\n" + "The following commands are currently available:\n\n" + "Control:\n" + " help : Show interactive instructions.\n" + " list : list all available vdpa devices.\n" + " create <socket file> <vdev addr> : create a new vdpa port.\n" + " quit : exit vdpa sample app.\n" + ); +} + +cmdline_parse_token_string_t cmd_help_help = + TOKEN_STRING_INITIALIZER(struct cmd_help_result, help, "help"); + +cmdline_parse_inst_t cmd_help = { + .f = cmd_help_parsed, + .data = NULL, + .help_str = "show help", + .tokens = { + (void *)&cmd_help_help, + NULL, + }, +}; + +/* *** List all available vdpa devices *** */ +struct cmd_list_result { + cmdline_fixed_string_t action; +}; + +static void cmd_list_vdpa_devices_parsed( + __attribute__((unused)) void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + int did; + uint32_t queue_num; + uint64_t features; + struct rte_vdpa_device *vdev; + struct rte_pci_addr addr; + + cmdline_printf(cl, "device id\tdevice address\tqueue num\tsupported features\n"); + for (did = 0; did < dev_total; did++) { + vdev = rte_vdpa_get_device(did); + if (!vdev) + continue; + if (vdev->ops->get_queue_num(did, &queue_num) < 0) { + RTE_LOG(ERR, VDPA, + "failed to get vdpa queue number " + "for device id %d.\n", did); + continue; + } + if (vdev->ops->get_features(did, &features) < 0) { + RTE_LOG(ERR, VDPA, + "failed to get vdpa features " + "for device id %d.\n", did); + continue; + } + addr = vdev->addr.pci_addr; + cmdline_printf(cl, + "%d\t\t" PCI_PRI_FMT "\t%" PRIu32 "\t\t0x%" PRIx64 "\n", + did, addr.domain, addr.bus, addr.devid, + addr.function, queue_num, features); + } +} + +cmdline_parse_token_string_t cmd_action_list = + TOKEN_STRING_INITIALIZER(struct cmd_list_result, action, "list"); + +cmdline_parse_inst_t cmd_list_vdpa_devices = { + .f = cmd_list_vdpa_devices_parsed, + .data = NULL, + .help_str = "list all available vdpa devices", + .tokens = { + (void *)&cmd_action_list, + NULL, + }, +}; + +/* *** Create new vdpa port *** */ +struct cmd_create_result { + cmdline_fixed_string_t action; + cmdline_fixed_string_t socket_path; + cmdline_fixed_string_t bdf; +}; + +static void cmd_create_vdpa_port_parsed(void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + int did; + struct cmd_create_result *res = parsed_result; + struct rte_vdpa_dev_addr addr; + + strncpy(vports[devcnt].ifname, res->socket_path, MAX_PATH_LEN); + if (rte_pci_addr_parse(res->bdf, &addr.pci_addr) != 0) { + cmdline_printf(cl, "Unable to parse the given bdf.\n"); + return; + } + addr.type = PCI_ADDR; + did = rte_vdpa_find_device_id(&addr); + if (did < 0) { + cmdline_printf(cl, "Unable to find vdpa devide id.\n"); + return; + } + + vports[devcnt].did = did; + + if (start_vdpa(&vports[devcnt]) == 0) + devcnt++; +} + +cmdline_parse_token_string_t cmd_action_create = + TOKEN_STRING_INITIALIZER(struct cmd_create_result, action, "create"); +cmdline_parse_token_string_t cmd_socket_path = + TOKEN_STRING_INITIALIZER(struct cmd_create_result, socket_path, NULL); +cmdline_parse_token_string_t cmd_bdf = + TOKEN_STRING_INITIALIZER(struct cmd_create_result, bdf, NULL); + +cmdline_parse_inst_t cmd_create_vdpa_port = { + .f = cmd_create_vdpa_port_parsed, + .data = NULL, + .help_str = "create a new vdpa port", + .tokens = { + (void *)&cmd_action_create, + (void *)&cmd_socket_path, + (void *)&cmd_bdf, + NULL, + }, +}; + +/* *** QUIT *** */ +struct cmd_quit_result { + cmdline_fixed_string_t quit; +}; + +static void cmd_quit_parsed(__attribute__((unused)) void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + vdpa_sample_quit(); + cmdline_quit(cl); +} + +cmdline_parse_token_string_t cmd_quit_quit = + TOKEN_STRING_INITIALIZER(struct cmd_quit_result, quit, "quit"); + +cmdline_parse_inst_t cmd_quit = { + .f = cmd_quit_parsed, + .data = NULL, + .help_str = "quit: exit application", + .tokens = { + (void *)&cmd_quit_quit, + NULL, + }, +}; +cmdline_parse_ctx_t main_ctx[] = { + (cmdline_parse_inst_t *)&cmd_help, + (cmdline_parse_inst_t *)&cmd_list_vdpa_devices, + (cmdline_parse_inst_t *)&cmd_create_vdpa_port, + (cmdline_parse_inst_t *)&cmd_quit, + NULL, +}; + +int +main(int argc, char *argv[]) +{ + char ch; + int i; + int ret; + struct cmdline *cl; + + ret = rte_eal_init(argc, argv); + if (ret < 0) + rte_exit(EXIT_FAILURE, "eal init failed\n"); + argc -= ret; + argv += ret; + + dev_total = rte_vdpa_get_device_num(); + if (dev_total <= 0) + rte_exit(EXIT_FAILURE, "No available vdpa device found\n"); + + signal(SIGINT, signal_handler); + signal(SIGTERM, signal_handler); + + ret = parse_args(argc, argv); + if (ret < 0) + rte_exit(EXIT_FAILURE, "invalid argument\n"); + + if (interactive == 1) { + cl = cmdline_stdin_new(main_ctx, "vdpa> "); + if (cl == NULL) + rte_panic("Cannot create cmdline instance\n"); + cmdline_interact(cl); + cmdline_stdin_exit(cl); + } else { + for (i = 0; i < RTE_MIN(MAX_VDPA_SAMPLE_PORTS, dev_total); + i++) { + vports[i].did = i; + snprintf(vports[i].ifname, MAX_PATH_LEN, "%s%d", + iface, i); + + start_vdpa(&vports[i]); + } + + printf("enter \'q\' to quit\n"); + while (scanf("%c", &ch)) { + if (ch == 'q') + break; + while (ch != '\n') { + if (scanf("%c", &ch)) + printf("%c", ch); + } + printf("enter \'q\' to quit\n"); + } + vdpa_sample_quit(); + } + + return 0; +} diff --git a/examples/vdpa/meson.build b/examples/vdpa/meson.build new file mode 100644 index 000000000..2e38a069f --- /dev/null +++ b/examples/vdpa/meson.build @@ -0,0 +1,16 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +# meson file, for building this example as part of a main DPDK build. +# +# To build this example as a standalone application with an already-installed +# DPDK instance, use 'make' + +if host_machine.system() != 'linux' + build = false +endif +deps += 'vhost' +allow_experimental_apis = true +sources = files( + 'main.c' +) \ No newline at end of file -- 2.17.1 ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [dpdk-dev] [PATCH v8 2/2] examples/vdpa: introduce a new sample for vDPA 2018-09-28 21:47 ` [dpdk-dev] [PATCH v8 2/2] examples/vdpa: introduce a new sample for vDPA Xiaolong Ye @ 2018-10-02 14:03 ` Ferruh Yigit 2018-10-03 6:04 ` Ye Xiaolong 2019-01-17 14:37 ` Shahaf Shuler 1 sibling, 1 reply; 41+ messages in thread From: Ferruh Yigit @ 2018-10-02 14:03 UTC (permalink / raw) To: Xiaolong Ye, dev, Maxime Coquelin, Tiwei Bie, Zhihong Wang Cc: xiao.w.wang, Rami Rosen, Wang Haiyue On 9/28/2018 10:47 PM, Xiaolong Ye wrote: > The vdpa sample application creates vhost-user sockets by using the > vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes > virtio ring compatible devices to serve virtio driver directly to enable > datapath acceleration. As vDPA driver can help to set up vhost datapath, > this application doesn't need to launch dedicated worker threads for vhost > enqueue/dequeue operations. > > Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com> > Acked-by: Xiao Wang <xiao.w.wang@intel.com> > Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com> <...> > +static int > +parse_args(int argc, char **argv) > +{ > + static const char *short_option = "i"; > + static struct option long_option[] = { > + {"iface", required_argument, NULL, 0}, > + {"interactive", no_argument, &interactive, 1}, > + {"client", no_argument, &client_mode, 1}, > + {NULL, 0, 0, 0}, > + }; > + int opt, idx; > + char *prgname = argv[0]; > + > + while ((opt = getopt_long(argc, argv, short_option, long_option, &idx)) > + != EOF) { > + switch (opt) { > + case 'i': > + printf("Interactive-mode selected\n"); > + interactive = 1; > + break; > + /* long options */ > + case 0: > + if (strncmp(long_option[idx].name, "iface", > + MAX_PATH_LEN) == 0) { > + strncpy(iface, optarg, MAX_PATH_LEN); Giving compiler warning [1], rte_strscpy() is safer to use. [1] In function ‘parse_args’, inlined from ‘main’ at .../examples/vdpa/main.c:419:8: .../examples/vdpa/main.c:76:5: error: ‘strncpy’ specified bound 128 equals destination size [-Werror=stringop-truncation] strncpy(iface, optarg, MAX_PATH_LEN); ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [dpdk-dev] [PATCH v8 2/2] examples/vdpa: introduce a new sample for vDPA 2018-10-02 14:03 ` Ferruh Yigit @ 2018-10-03 6:04 ` Ye Xiaolong 2018-10-03 7:48 ` Ferruh Yigit 0 siblings, 1 reply; 41+ messages in thread From: Ye Xiaolong @ 2018-10-03 6:04 UTC (permalink / raw) To: Ferruh Yigit Cc: dev, Maxime Coquelin, Tiwei Bie, Zhihong Wang, xiao.w.wang, Rami Rosen, Wang Haiyue Hi, Ferruh, On 10/02, Ferruh Yigit wrote: >On 9/28/2018 10:47 PM, Xiaolong Ye wrote: >> The vdpa sample application creates vhost-user sockets by using the >> vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes >> virtio ring compatible devices to serve virtio driver directly to enable >> datapath acceleration. As vDPA driver can help to set up vhost datapath, >> this application doesn't need to launch dedicated worker threads for vhost >> enqueue/dequeue operations. >> >> Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com> >> Acked-by: Xiao Wang <xiao.w.wang@intel.com> >> Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com> > ><...> > >> +static int >> +parse_args(int argc, char **argv) >> +{ >> + static const char *short_option = "i"; >> + static struct option long_option[] = { >> + {"iface", required_argument, NULL, 0}, >> + {"interactive", no_argument, &interactive, 1}, >> + {"client", no_argument, &client_mode, 1}, >> + {NULL, 0, 0, 0}, >> + }; >> + int opt, idx; >> + char *prgname = argv[0]; >> + >> + while ((opt = getopt_long(argc, argv, short_option, long_option, &idx)) >> + != EOF) { >> + switch (opt) { >> + case 'i': >> + printf("Interactive-mode selected\n"); >> + interactive = 1; >> + break; >> + /* long options */ >> + case 0: >> + if (strncmp(long_option[idx].name, "iface", >> + MAX_PATH_LEN) == 0) { >> + strncpy(iface, optarg, MAX_PATH_LEN); > >Giving compiler warning [1], rte_strscpy() is safer to use. > >[1] >In function ‘parse_args’, > > > > inlined from ‘main’ at .../examples/vdpa/main.c:419:8: > > > >.../examples/vdpa/main.c:76:5: error: ‘strncpy’ specified bound 128 equals >destination size [-Werror=stringop-truncation] > > > strncpy(iface, optarg, MAX_PATH_LEN); > > > > ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Thanks for the finding, by the way, how do you get this warning? I can't see it in my local build, what compiler you used? Thanks, Xiaolong ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [dpdk-dev] [PATCH v8 2/2] examples/vdpa: introduce a new sample for vDPA 2018-10-03 6:04 ` Ye Xiaolong @ 2018-10-03 7:48 ` Ferruh Yigit 0 siblings, 0 replies; 41+ messages in thread From: Ferruh Yigit @ 2018-10-03 7:48 UTC (permalink / raw) To: Ye Xiaolong Cc: dev, Maxime Coquelin, Tiwei Bie, Zhihong Wang, xiao.w.wang, Rami Rosen, Wang Haiyue On 10/3/2018 7:04 AM, Ye Xiaolong wrote: > Hi, Ferruh, > > On 10/02, Ferruh Yigit wrote: >> On 9/28/2018 10:47 PM, Xiaolong Ye wrote: >>> The vdpa sample application creates vhost-user sockets by using the >>> vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes >>> virtio ring compatible devices to serve virtio driver directly to enable >>> datapath acceleration. As vDPA driver can help to set up vhost datapath, >>> this application doesn't need to launch dedicated worker threads for vhost >>> enqueue/dequeue operations. >>> >>> Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com> >>> Acked-by: Xiao Wang <xiao.w.wang@intel.com> >>> Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com> >> >> <...> >> >>> +static int >>> +parse_args(int argc, char **argv) >>> +{ >>> + static const char *short_option = "i"; >>> + static struct option long_option[] = { >>> + {"iface", required_argument, NULL, 0}, >>> + {"interactive", no_argument, &interactive, 1}, >>> + {"client", no_argument, &client_mode, 1}, >>> + {NULL, 0, 0, 0}, >>> + }; >>> + int opt, idx; >>> + char *prgname = argv[0]; >>> + >>> + while ((opt = getopt_long(argc, argv, short_option, long_option, &idx)) >>> + != EOF) { >>> + switch (opt) { >>> + case 'i': >>> + printf("Interactive-mode selected\n"); >>> + interactive = 1; >>> + break; >>> + /* long options */ >>> + case 0: >>> + if (strncmp(long_option[idx].name, "iface", >>> + MAX_PATH_LEN) == 0) { >>> + strncpy(iface, optarg, MAX_PATH_LEN); >> >> Giving compiler warning [1], rte_strscpy() is safer to use. >> >> [1] >> In function ‘parse_args’, >> >> >> >> inlined from ‘main’ at .../examples/vdpa/main.c:419:8: >> >> >> >> .../examples/vdpa/main.c:76:5: error: ‘strncpy’ specified bound 128 equals >> destination size [-Werror=stringop-truncation] >> >> >> strncpy(iface, optarg, MAX_PATH_LEN); >> >> >> >> ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > Thanks for the finding, by the way, how do you get this warning? I can't see > it in my local build, what compiler you used? gcc, $ gcc --version gcc (GCC) 8.1.1 20180712 (Red Hat 8.1.1-5) ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [dpdk-dev] [PATCH v8 2/2] examples/vdpa: introduce a new sample for vDPA 2018-09-28 21:47 ` [dpdk-dev] [PATCH v8 2/2] examples/vdpa: introduce a new sample for vDPA Xiaolong Ye 2018-10-02 14:03 ` Ferruh Yigit @ 2019-01-17 14:37 ` Shahaf Shuler 2019-01-17 15:13 ` Ye Xiaolong 1 sibling, 1 reply; 41+ messages in thread From: Shahaf Shuler @ 2019-01-17 14:37 UTC (permalink / raw) To: Xiaolong Ye, dev, Maxime Coquelin, Tiwei Bie, Zhihong Wang Cc: xiao.w.wang, Rami Rosen, Wang Haiyue Hi, Going back to this merged commit, Saturday, September 29, 2018 12:48 AM, Xiaolong Ye: > Subject: [dpdk-dev] [PATCH v8 2/2] examples/vdpa: introduce a new sample > for vDPA > > The vdpa sample application creates vhost-user sockets by using the vDPA > backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio > ring compatible devices to serve virtio driver directly to enable datapath > acceleration. As vDPA driver can help to set up vhost datapath, this > application doesn't need to launch dedicated worker threads for vhost > enqueue/dequeue operations. > > Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com> > Acked-by: Xiao Wang <xiao.w.wang@intel.com> > Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com> > --- [...] > + > +Start the VMs > +~~~~~~~~~~~~~ > + > +.. code-block:: console > + > + qemu-system-x86_64 -cpu host -enable-kvm \ > + <snip> > + -mem-prealloc \ > + -chardev socket,id=char0,path=<socket_file created in above steps> \ > + -netdev type=vhost-user,id=vdpa,chardev=char0 \ > + -device > + virtio-net-pci,netdev=vdpa,mac=00:aa:bb:cc:dd:ee,page-per-vq=on \ It doesn't seems like qemu has the id=vdpa and netdev=vdpa options. Grepping vdpa on qemu returns empty. Am using QEMU v3.1.0-513-gfb177a0. Commit head is fb177a036f2e ("MAINTAINERS: Mark RISC-V as Supported"). Did I missed anything? ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [dpdk-dev] [PATCH v8 2/2] examples/vdpa: introduce a new sample for vDPA 2019-01-17 14:37 ` Shahaf Shuler @ 2019-01-17 15:13 ` Ye Xiaolong 2019-01-17 15:17 ` Shahaf Shuler 0 siblings, 1 reply; 41+ messages in thread From: Ye Xiaolong @ 2019-01-17 15:13 UTC (permalink / raw) To: Shahaf Shuler Cc: dev, Maxime Coquelin, Tiwei Bie, Zhihong Wang, xiao.w.wang, Rami Rosen, Wang Haiyue Hi, On 01/17, Shahaf Shuler wrote: >Hi, > >Going back to this merged commit, > >Saturday, September 29, 2018 12:48 AM, Xiaolong Ye: >> Subject: [dpdk-dev] [PATCH v8 2/2] examples/vdpa: introduce a new sample >> for vDPA >> >> The vdpa sample application creates vhost-user sockets by using the vDPA >> backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio >> ring compatible devices to serve virtio driver directly to enable datapath >> acceleration. As vDPA driver can help to set up vhost datapath, this >> application doesn't need to launch dedicated worker threads for vhost >> enqueue/dequeue operations. >> >> Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com> >> Acked-by: Xiao Wang <xiao.w.wang@intel.com> >> Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com> >> --- > >[...] > >> + >> +Start the VMs >> +~~~~~~~~~~~~~ >> + >> +.. code-block:: console >> + >> + qemu-system-x86_64 -cpu host -enable-kvm \ >> + <snip> >> + -mem-prealloc \ >> + -chardev socket,id=char0,path=<socket_file created in above steps> \ >> + -netdev type=vhost-user,id=vdpa,chardev=char0 \ >> + -device >> + virtio-net-pci,netdev=vdpa,mac=00:aa:bb:cc:dd:ee,page-per-vq=on \ > > >It doesn't seems like qemu has the id=vdpa and netdev=vdpa options. >Grepping vdpa on qemu returns empty. > >Am using QEMU v3.1.0-513-gfb177a0. Commit head is fb177a036f2e ("MAINTAINERS: Mark RISC-V as Supported"). > >Did I missed anything? > Not quite sure about your question here, the vdpa used in above qemu cmdline is just a normal string, you can replace it with other string like -netdev type=vhost-user,id=mynet,chardev=char0 \ -device virtio-net-pci,netdev=mynet,... The major support for vdpa is done in dpdk side as the vhost user backend, qemu will use normal vhost user protocol to communicate with it. Thanks, Xiaolong ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [dpdk-dev] [PATCH v8 2/2] examples/vdpa: introduce a new sample for vDPA 2019-01-17 15:13 ` Ye Xiaolong @ 2019-01-17 15:17 ` Shahaf Shuler 0 siblings, 0 replies; 41+ messages in thread From: Shahaf Shuler @ 2019-01-17 15:17 UTC (permalink / raw) To: Ye Xiaolong Cc: dev, Maxime Coquelin, Tiwei Bie, Zhihong Wang, xiao.w.wang, Rami Rosen, Wang Haiyue Thursday, January 17, 2019 5:13 PM, Ye Xiaolong: > Subject: Re: [dpdk-dev] [PATCH v8 2/2] examples/vdpa: introduce a new > sample for vDPA > > Hi, > > On 01/17, Shahaf Shuler wrote: > >Hi, > > > >Going back to this merged commit, > > > >Saturday, September 29, 2018 12:48 AM, Xiaolong Ye: > >> Subject: [dpdk-dev] [PATCH v8 2/2] examples/vdpa: introduce a new > >> sample for vDPA > >> > >> The vdpa sample application creates vhost-user sockets by using the > >> vDPA backend. vDPA stands for vhost Data Path Acceleration which > >> utilizes virtio ring compatible devices to serve virtio driver > >> directly to enable datapath acceleration. As vDPA driver can help to > >> set up vhost datapath, this application doesn't need to launch > >> dedicated worker threads for vhost enqueue/dequeue operations. > >> > >> Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com> > >> Acked-by: Xiao Wang <xiao.w.wang@intel.com> > >> Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com> > >> --- > > > >[...] > > > >> + > >> +Start the VMs > >> +~~~~~~~~~~~~~ > >> + > >> +.. code-block:: console > >> + > >> + qemu-system-x86_64 -cpu host -enable-kvm \ > >> + <snip> > >> + -mem-prealloc \ > >> + -chardev socket,id=char0,path=<socket_file created in above steps> > \ > >> + -netdev type=vhost-user,id=vdpa,chardev=char0 \ > >> + -device > >> + virtio-net-pci,netdev=vdpa,mac=00:aa:bb:cc:dd:ee,page-per-vq=on \ > > > > > >It doesn't seems like qemu has the id=vdpa and netdev=vdpa options. > >Grepping vdpa on qemu returns empty. > > > >Am using QEMU v3.1.0-513-gfb177a0. Commit head is fb177a036f2e > ("MAINTAINERS: Mark RISC-V as Supported"). > > > >Did I missed anything? > > > > Not quite sure about your question here, the vdpa used in above qemu > cmdline is just a normal string, you can replace it with other string like > > -netdev type=vhost-user,id=mynet,chardev=char0 \ -device virtio-net- > pci,netdev=mynet,... > > The major support for vdpa is done in dpdk side as the vhost user backend, > qemu will use normal vhost user protocol to communicate with it. Thanks. I mis-interpreted the option fields. > > Thanks, > Xiaolong ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [dpdk-dev] [PATCH v8 0/2] introduce vdpa sample 2018-09-28 21:47 ` [dpdk-dev] [PATCH v8 0/2] introduce vdpa sample Xiaolong Ye 2018-09-28 21:47 ` [dpdk-dev] [PATCH v8 1/2] vhost: introduce API to get vDPA device number Xiaolong Ye 2018-09-28 21:47 ` [dpdk-dev] [PATCH v8 2/2] examples/vdpa: introduce a new sample for vDPA Xiaolong Ye @ 2018-10-02 12:51 ` Maxime Coquelin 2 siblings, 0 replies; 41+ messages in thread From: Maxime Coquelin @ 2018-10-02 12:51 UTC (permalink / raw) To: Xiaolong Ye, dev, Tiwei Bie, Zhihong Wang Cc: xiao.w.wang, Rami Rosen, Wang Haiyue On 09/28/2018 11:47 PM, Xiaolong Ye wrote: > Hi, > > This patchset introduces vdpa sample to demonstrate the vDPA use case. > > v8 changes: > * move body of data_init to the main function > * adjust Reviewed-by/Acked-by tag postion > > v7 changes: > * make vports static > * avoid unnecessary static variable initialization > > v6 changes: > * improve the document according to Xiao's comments > * fix a typo, PRIu64 -> PRIx64 > > v5 changes: > * improve print format and correct from "PRIu64" to "PRIx64" > * use "-c 0x2" to better demonstrate app doesn't need to launch dedicated > worker threads for vhost enqueue/dequeue operations in vdpa.rst > > v4 changes: > * add client mode support > * improve the format to list the vDPA device info and improve the vdpa.rst > accordingly > * remove some useless comments > * add introduction in 18.11 release note. > > > v3 changes: > * list cmd would show queue number and supported features of vdpa devices. > * address Xiao's review comments > > v2 changes: > > * fix a compilation error reported by Rosen > * improve create cmd in interactive mode and add two new cmds: list, quit > * add application documentation > > > Xiaolong Ye (2): > vhost: introduce API to get vDPA device number > examples/vdpa: introduce a new sample for vDPA > > MAINTAINERS | 2 + > doc/guides/rel_notes/release_18_11.rst | 8 + > doc/guides/sample_app_ug/index.rst | 1 + > doc/guides/sample_app_ug/vdpa.rst | 120 +++++++ > examples/Makefile | 2 +- > examples/vdpa/Makefile | 32 ++ > examples/vdpa/main.c | 453 +++++++++++++++++++++++++ > examples/vdpa/meson.build | 16 + > lib/librte_vhost/rte_vdpa.h | 3 + > lib/librte_vhost/rte_vhost_version.map | 1 + > lib/librte_vhost/vdpa.c | 6 + > 11 files changed, 643 insertions(+), 1 deletion(-) > create mode 100644 doc/guides/sample_app_ug/vdpa.rst > create mode 100644 examples/vdpa/Makefile > create mode 100644 examples/vdpa/main.c > create mode 100644 examples/vdpa/meson.build > Applied to dpdk-next-virtio/master. Thanks, Maxime ^ permalink raw reply [flat|nested] 41+ messages in thread
end of thread, other threads:[~2019-01-17 15:17 UTC | newest] Thread overview: 41+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2018-09-20 22:28 [dpdk-dev] [PATCH v3 0/2] introduce vdpa sample Xiaolong Ye 2018-09-20 22:28 ` [dpdk-dev] [PATCH v3 1/2] vhost: introduce API to get vDPA device number Xiaolong Ye 2018-09-20 22:28 ` [dpdk-dev] [PATCH v3 2/2] examples/vdpa: introduce a new sample for vDPA Xiaolong Ye 2018-09-21 5:07 ` Wang, Xiao W 2018-09-22 11:04 ` Ye Xiaolong 2018-09-24 8:42 ` [dpdk-dev] [PATCH v4 0/2] introduce vdpa sample Xiaolong Ye 2018-09-24 8:42 ` [dpdk-dev] [PATCH v4 1/2] vhost: introduce API to get vDPA device number Xiaolong Ye 2018-09-24 8:42 ` [dpdk-dev] [PATCH v4 2/2] examples/vdpa: introduce a new sample for vDPA Xiaolong Ye 2018-09-24 4:12 ` Wang, Xiao W 2018-09-24 21:30 ` Ye Xiaolong 2018-09-25 12:07 ` [dpdk-dev] [PATCH v5 0/2] introduce vdpa sample Xiaolong Ye 2018-09-25 12:07 ` [dpdk-dev] [PATCH v5 1/2] vhost: introduce API to get vDPA device number Xiaolong Ye 2018-09-25 12:07 ` [dpdk-dev] [PATCH v5 2/2] examples/vdpa: introduce a new sample for vDPA Xiaolong Ye 2018-09-25 8:32 ` Wang, Xiao W 2018-09-26 9:06 ` [dpdk-dev] [PATCH v6 0/2] introduce vdpa sample Xiaolong Ye 2018-09-26 3:15 ` Wang, Xiao W 2018-09-26 9:06 ` [dpdk-dev] [PATCH v6 1/2] vhost: introduce API to get vDPA device number Xiaolong Ye 2018-09-26 9:06 ` [dpdk-dev] [PATCH v6 2/2] examples/vdpa: introduce a new sample for vDPA Xiaolong Ye 2018-09-27 16:37 ` Maxime Coquelin 2018-09-28 7:32 ` Ye Xiaolong 2018-09-28 11:23 ` [dpdk-dev] [PATCH v7 0/2] introduce vdpa sample Xiaolong Ye 2018-09-28 11:23 ` [dpdk-dev] [PATCH v7 1/2] vhost: introduce API to get vDPA device number Xiaolong Ye 2018-09-28 11:23 ` [dpdk-dev] [PATCH v7 2/2] examples/vdpa: introduce a new sample for vDPA Xiaolong Ye 2018-09-28 7:31 ` Maxime Coquelin 2018-09-28 14:33 ` Ye Xiaolong 2018-09-28 7:49 ` Maxime Coquelin 2018-09-28 15:17 ` Ye Xiaolong 2018-09-28 8:31 ` Maxime Coquelin 2018-09-28 17:37 ` Ye Xiaolong 2018-09-28 21:47 ` [dpdk-dev] [PATCH v8 0/2] introduce vdpa sample Xiaolong Ye 2018-09-28 21:47 ` [dpdk-dev] [PATCH v8 1/2] vhost: introduce API to get vDPA device number Xiaolong Ye 2018-10-02 13:19 ` Ferruh Yigit 2018-10-03 5:53 ` Ye Xiaolong 2018-09-28 21:47 ` [dpdk-dev] [PATCH v8 2/2] examples/vdpa: introduce a new sample for vDPA Xiaolong Ye 2018-10-02 14:03 ` Ferruh Yigit 2018-10-03 6:04 ` Ye Xiaolong 2018-10-03 7:48 ` Ferruh Yigit 2019-01-17 14:37 ` Shahaf Shuler 2019-01-17 15:13 ` Ye Xiaolong 2019-01-17 15:17 ` Shahaf Shuler 2018-10-02 12:51 ` [dpdk-dev] [PATCH v8 0/2] introduce vdpa sample Maxime Coquelin
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).