From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by dpdk.org (Postfix) with ESMTP id 904451AFF for ; Mon, 24 Sep 2018 06:12:50 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 23 Sep 2018 21:12:49 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.54,296,1534834800"; d="scan'208";a="85983708" Received: from fmsmsx103.amr.corp.intel.com ([10.18.124.201]) by orsmga003.jf.intel.com with ESMTP; 23 Sep 2018 21:12:44 -0700 Received: from fmsmsx155.amr.corp.intel.com (10.18.116.71) by FMSMSX103.amr.corp.intel.com (10.18.124.201) with Microsoft SMTP Server (TLS) id 14.3.319.2; Sun, 23 Sep 2018 21:12:44 -0700 Received: from shsmsx152.ccr.corp.intel.com (10.239.6.52) by FMSMSX155.amr.corp.intel.com (10.18.116.71) with Microsoft SMTP Server (TLS) id 14.3.319.2; Sun, 23 Sep 2018 21:12:44 -0700 Received: from shsmsx101.ccr.corp.intel.com ([169.254.1.220]) by SHSMSX152.ccr.corp.intel.com ([169.254.6.37]) with mapi id 14.03.0319.002; Mon, 24 Sep 2018 12:12:42 +0800 From: "Wang, Xiao W" To: "Ye, Xiaolong" , "dev@dpdk.org" , Maxime Coquelin , "Bie, Tiwei" , "Wang, Zhihong" CC: Rami Rosen , "Wang, Haiyue" Thread-Topic: [PATCH v4 2/2] examples/vdpa: introduce a new sample for vDPA Thread-Index: AQHUU6mlMUQPpN8WC0a3WEvKcQ+GB6T+ysNg Date: Mon, 24 Sep 2018 04:12:41 +0000 Message-ID: References: <20180920222817.45829-1-xiaolong.ye@intel.com> <20180924084248.91811-1-xiaolong.ye@intel.com> <20180924084248.91811-3-xiaolong.ye@intel.com> In-Reply-To: <20180924084248.91811-3-xiaolong.ye@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ctpclassification: CTP_NT x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiMmQ0MTc3MjgtOTMyMy00NWJmLTg0MjQtYzAwODE5Y2IzZDg4IiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoiaGNyUmxmM0JmUlwveEVEcGFiS3AyS25sXC9YRjdKb0ZMMkZaekVVRk83NTdTSFZJT1oySXRUTXVGRFQwekVSNktWIn0= dlp-product: dlpe-windows dlp-version: 11.0.400.15 dlp-reaction: no-action x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH v4 2/2] examples/vdpa: introduce a new sample for vDPA X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Sep 2018 04:12:51 -0000 Hi Xiaolong, Thanks for the update, 2 small comments below. > -----Original Message----- > From: Ye, Xiaolong > Sent: Monday, September 24, 2018 4:43 PM > To: dev@dpdk.org; Maxime Coquelin ; Bie, > Tiwei ; Wang, Zhihong > Cc: Wang, Xiao W ; Rami Rosen > ; Wang, Haiyue ; Ye, > Xiaolong > Subject: [PATCH v4 2/2] examples/vdpa: introduce a new sample for vDPA >=20 > The vdpa sample application creates vhost-user sockets by using the > vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes > virtio ring compatible devices to serve virtio driver directly to enable > datapath acceleration. As vDPA driver can help to set up vhost datapath, > this application doesn't need to launch dedicated worker threads for vhos= t > enqueue/dequeue operations. >=20 > Signed-off-by: Xiao Wang > Signed-off-by: Xiaolong Ye > --- > MAINTAINERS | 2 + > doc/guides/rel_notes/release_18_11.rst | 8 + > doc/guides/sample_app_ug/index.rst | 1 + > doc/guides/sample_app_ug/vdpa.rst | 118 +++++++ > examples/Makefile | 2 +- > examples/vdpa/Makefile | 32 ++ > examples/vdpa/main.c | 466 +++++++++++++++++++++++++ > examples/vdpa/meson.build | 16 + > 8 files changed, 644 insertions(+), 1 deletion(-) > create mode 100644 doc/guides/sample_app_ug/vdpa.rst > create mode 100644 examples/vdpa/Makefile > create mode 100644 examples/vdpa/main.c > create mode 100644 examples/vdpa/meson.build >=20 > diff --git a/MAINTAINERS b/MAINTAINERS > index 5967c1dd3..5656f18e8 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -683,6 +683,8 @@ F: doc/guides/sample_app_ug/vhost.rst > F: examples/vhost_scsi/ > F: doc/guides/sample_app_ug/vhost_scsi.rst > F: examples/vhost_crypto/ > +F: examples/vdpa/ > +F: doc/guides/sample_app_ug/vdpa.rst >=20 > Vhost PMD > M: Maxime Coquelin > diff --git a/doc/guides/rel_notes/release_18_11.rst > b/doc/guides/rel_notes/release_18_11.rst > index bc9b74ec4..dd53a9ecf 100644 > --- a/doc/guides/rel_notes/release_18_11.rst > +++ b/doc/guides/rel_notes/release_18_11.rst > @@ -67,6 +67,14 @@ New Features > SR-IOV option in Hyper-V and Azure. This is an alternative to the prev= ious > vdev_netvsc, tap, and failsafe drivers combination. >=20 > +* **Add a new sample for vDPA** > + > + The vdpa sample application creates vhost-user sockets by using the > + vDPA backend. vDPA stands for vhost Data Path Acceleration which utili= zes > + virtio ring compatible devices to serve virtio driver directly to enab= le > + datapath acceleration. As vDPA driver can help to set up vhost datapat= h, > + this application doesn't need to launch dedicated worker threads for v= host > + enqueue/dequeue operations. >=20 > API Changes > ----------- > diff --git a/doc/guides/sample_app_ug/index.rst > b/doc/guides/sample_app_ug/index.rst > index 5bedf4f6f..74b12af85 100644 > --- a/doc/guides/sample_app_ug/index.rst > +++ b/doc/guides/sample_app_ug/index.rst > @@ -45,6 +45,7 @@ Sample Applications User Guides > vhost > vhost_scsi > vhost_crypto > + vdpa > netmap_compatibility > ip_pipeline > test_pipeline > diff --git a/doc/guides/sample_app_ug/vdpa.rst > b/doc/guides/sample_app_ug/vdpa.rst > new file mode 100644 > index 000000000..d05728a37 > --- /dev/null > +++ b/doc/guides/sample_app_ug/vdpa.rst > @@ -0,0 +1,118 @@ > +.. SPDX-License-Identifier: BSD-3-Clause > + Copyright(c) 2018 Intel Corporation. > + > +Vdpa Sample Application > +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > + > +The vdpa sample application creates vhost-user sockets by using the > +vDPA backend. vDPA stands for vhost Data Path Acceleration which utilize= s > +virtio ring compatible devices to serve virtio driver directly to enable > +datapath acceleration. As vDPA driver can help to set up vhost datapath, > +this application doesn't need to launch dedicated worker threads for vho= st > +enqueue/dequeue operations. > + > +Testing steps > +------------- > + > +This section shows the steps of how to start VMs with vDPA vhost-user > +backend and verify network connection & live migration. > + > +Build > +~~~~~ > + > +To compile the sample application see :doc:`compiling`. > + > +The application is located in the ``vdpa`` sub-directory. > + > +Start the vdpa example > +~~~~~~~~~~~~~~~~~~~~~~ > + > +.. code-block:: console > + > + ./vdpa [EAL options] -- [--client] [--interactive|-i] or [--ifa= ce SOCKET_PATH] > + > +where > + > +* --client means running vdpa app in client mode, in the client mode, QE= MU > needs > + to run as the server mode and take charge of socket file creation. > +* --iface specifies the path prefix of the UNIX domain socket file, e.g. > + /tmp/vhost-user-, then the socket files will be named as /tmp/vhost-us= er- > + (n starts from 0). > +* --interactive means run the vdpa sample in interactive mode, currently= 4 > + internal cmds are supported: > + > + 1. help: show help message > + 2. list: list all available vdpa devices > + 3. create: create a new vdpa port with socket file and vdpa device add= ress > + 4. quit: unregister vhost driver and exit the application > + > +Take IFCVF driver for example: > + > +.. code-block:: console > + > + ./vdpa --log-level=3D9 -c 0x6 -n 4 --socket-mem 1024,1024 \ > + -w 0000:06:00.3,vdpa=3D1 -w 0000:06:00.4,vdpa=3D1 \ > + -- --interactive To demonstrate app doesn't need to launch dedicated worker threads for vhos= t enqueue/dequeue operations, We can use "-c 0x2" to indicate that no need to allocate dedicated worker t= hreads. > + > +.. note:: > + We need to bind vfio-pci to VFs before running vdpa sample. > + > + * modprobe vfio-pci > + * ./usertools/dpdk-devbind.py -b vfio-pci 06:00.3 06:00.4 > + > +Then we can create 2 vdpa ports in interactive cmdline. > + > +.. code-block:: console > + > + vdpa> list > + device id device address queue num supported featur= es > + 0 0000:06:00.3 1 0x5572362272 > + 1 0000:06:00.4 1 0x5572362272 > + > + vdpa> create /tmp/vdpa-socket0 0000:06:00.3 > + vdpa> create /tmp/vdpa-socket1 0000:06:00.4 > + > +.. _vdpa_app_run_vm: > + > +Start the VMs > +~~~~~~~~~~~~~ > + > +.. code-block:: console > + > + qemu-system-x86_64 -cpu host -enable-kvm \ > + > + -mem-prealloc \ > + -chardev socket,id=3Dchar0,path=3D \ > + -netdev type=3Dvhost-user,id=3Dvdpa,chardev=3Dchar0 \ > + -device virtio-net-pci,netdev=3Dvdpa,mac=3D00:aa:bb:cc:dd:ee \ > + > +After the VMs launches, we can login the VMs and configure the ip, verif= y the > +network connection via ping or netperf. > + > +.. note:: > + Suggest to use QEMU 3.0.0 which extends vhost-user for vDPA. [...] > + > +/* *** List all available vdpa devices *** */ > +struct cmd_list_result { > + cmdline_fixed_string_t action; > +}; > + > +static void cmd_list_vdpa_devices_parsed( > + __attribute__((unused)) void *parsed_result, > + struct cmdline *cl, > + __attribute__((unused)) void *data) > +{ > + int did; > + uint32_t queue_num; > + uint64_t features; > + struct rte_vdpa_device *vdev; > + struct rte_pci_addr addr; > + > + cmdline_printf(cl, "device id\tdevice address\tqueue num\tsupported > features\n"); > + for (did =3D 0; did < dev_total; did++) { > + vdev =3D rte_vdpa_get_device(did); > + if (!vdev) > + continue; > + if (vdev->ops->get_queue_num(did, &queue_num) < 0) { > + RTE_LOG(ERR, VDPA, > + "failed to get vdpa queue number " > + "for device id %d.\n", did); > + continue; > + } > + if (vdev->ops->get_features(did, &features) < 0) { > + RTE_LOG(ERR, VDPA, > + "failed to get vdpa features " > + "for device id %d.\n", did); > + continue; > + } > + addr =3D vdev->addr.pci_addr; > + cmdline_printf(cl, > + > "%d\t\t"PCI_PRI_FMT"\t%"PRIu32"\t\t0x%"PRIu64"\n", did, > + addr.domain, addr.bus, addr.devid, > + addr.function, queue_num, features); Use PRIx64 instead of PRIu64 for features. You can add a blank space between "PRIx64" and the other section to make it= more readable. Refer to: lib/librte_vhost/vhost_user.c: "guest memory region %u, s= ize: 0x%" PRIx64 "\n"=20 BRs, Xiao