From: Maxime Coquelin <maxime.coquelin@redhat.com>
To: Stefan Hajnoczi <stefanha@redhat.com>, dev@dpdk.org
Cc: Yuanhan Liu <yliu@fridaylinux.org>,
wei.w.wang@intel.com, mst@redhat.com, zhiyong.yang@intel.com,
jasowang@redhat.com
Subject: Re: [dpdk-dev] [RFC 00/24] vhost: add virtio-vhost-user transport
Date: Wed, 31 Jan 2018 11:02:58 +0100 [thread overview]
Message-ID: <1c34e10d-e59b-cae8-0bfe-fa511116ef47@redhat.com> (raw)
In-Reply-To: <20180119134444.24927-1-stefanha@redhat.com>
Hi Stefan,
I went through the series, and I really like how it fits with existing
vhost-user over af_unix socket.
I will post the change I suggested to fix the regression met with SPDK
and multiqueue.
Thanks,
Maxime
On 01/19/2018 02:44 PM, Stefan Hajnoczi wrote:
> This patch series implements the virtio-vhost-user device, which tunnels
> vhost-user protocol messages over virtio. This lets guests act as vhost device
> backends for other guests.
>
> The virtio-vhost-user device is the result of discussion about Wei Wang and
> Zhiyong Yang's vhost-pci device. This patch series demonstrates that vhost
> device backends, such as the vhost vdev driver, can work over both AF_UNIX and
> virtio-vhost-user without significant modifications. This allows a lot of code
> to be shared between traditional AF_UNIX vhost-user and virtio-vhost-user. The
> vhost-pci patches duplicated the vhost net device backend and didn't reuse
> librte_vhost:
>
> http://dpdk.org/ml/archives/dev/2017-November/082615.html
>
> User-visible changes
> --------------------
> The vhost vdev can now be used when DPDK runs inside a guest with a
> virtio-vhost-user PCI device:
>
> --vdev net_vhost0,iface="0000:00:04.0",virtio-transport=1
>
> The vhost-scsi example has also been extended to support virtio-vhost-user:
>
> ./vhost-scsi ... -- --virtio-vhost-user-pci "0000:00:04.0"
>
> For more information (including instructions for running the code), see
> https://wiki.qemu.org/Features/VirtioVhostUser
>
> Virtio device design
> --------------------
> The virtio-vhost-user device is a new virtio device type. It acts as a
> vhost-user transport and is an alternative for the traditional AF_UNIX
> transport.
>
> You can find the virtio-vhost-user VIRTIO device specification here:
> https://stefanha.github.io/virtio/vhost-user-slave.html#x1-2830007
>
> librte_vhost API changes
> ------------------------
> This patch series extends librte_vhost so that it now accepts:
>
> rte_vhost_driver_register("0000:00:04.0",
> RTE_VHOST_USER_VIRTIO_TRANSPORT);
>
> All other librte_vhost API usage remains unchanged, except that the file
> descriptors exposed in some <rte_vhost.h> structs will be -1 since there is no
> file descriptor passing involved.
>
> This makes it extremely easy to support virtio-vhost-user in existing vhost
> device backends! I have extended the vhost vdev driver and the vhost-scsi
> example application in this patch series.
>
> Patch series overview
> ---------------------
> This series is based on commit 814339ba7eea13d132508af2cccec2f73568e2d0 from
> dpdk-next-virtio/master. You can also get my git branch here:
>
> https://github.com/stefanha/dpdk/tree/virtio-vhost-user
>
> The initial patches refactor librte_vhost so that AF_UNIX-specific code is moved
> to a new trans_af_unix.c file. This also introduces a struct
> vhost_transport_ops interface that all transports will implement:
>
> adf412c3c vhost: move vring_call() into trans_af_unix.c
> 04079a077 vhost: move AF_UNIX code from socket.c to trans_af_unix.c
> 885642091 vhost: allocate per-socket transport state
> 9c48377df vhost: move socket_fd and un sockaddr into trans_af_unix.c
> c646e6292 vhost: move start_server/client() calls to trans_af_unix.c
> db07ef7a8 vhost: move vhost_user_connection to trans_af_unix.c
> d10b80163 vhost: move vhost_user_reconnect_init() into trans_af_unix.c
> 1258bcd68 vhost: move vhost_user.fdset to trans_af_unix.c
> bc7f6d7ab vhost: pass vhost_transport_ops through vhost_new_device()
> b82187a29 vhost: embed struct virtio_net inside struct vhost_user_connection
> abbd544f2 vhost: extract vhost_user.c socket I/O into transport
> 0658d711b vhost: move slave_req_fd field to AF_UNIX transport
> e2ecf78ed vhost: move mmap/munmap to AF_UNIX transport
>
> I was about to add the virtio-vhost-user PCI driver when I realized that
> lib/librte_vhost/ cannot have a dependency on rte_bus_pci.h (it lives in
> drivers/). The solution I chose is to move all of librte_vhost to
> drivers/librte_vhost/, but I'm open to suggestions if this is undesirable.
>
> 8615c2140 vhost: move librte_vhost to drivers/
>
> Since virtio-vhost-user is a virtio device it's necessary to perform a sequence
> of device initialization steps and set up virtqueues. I didn't see a virtio
> API in DPDK and didn't have time to create one myself. I copied the virtio
> code from drivers/net/virtio/ as a quick hack, but really there needs to be a
> librte_virtio that is shared.
>
> c26937c66 vhost: add virtio pci framework
>
> Next the virtio-vhost-user transport is added along with the
> RTE_VHOST_USER_VIRTIO_TRANSPORT flag:
>
> 7fad5adc4 vhost: remember a vhost_virtqueue's queue index
> d26922892 vhost: add virtio-vhost-user transport
> f562a70ab vhost: add RTE_VHOST_USER_VIRTIO_TRANSPORT flag
>
> Then I extended the vhost vdev and vhost-scsi example application to support
> virtio-vhost-user:
>
> 47434f2a2 net/vhost: add virtio-vhost-user support
> 22ca05d1b examples/vhost_scsi: add --socket-file argument
> db3b391dc examples/vhost_scsi: add virtio-vhost-user support
>
> It was also necessary to tweak dpdk-devbind.py to support virtio-vhost-user:
>
> 4aee6f653 usertools: add virtio-vhost-user devices to dpdk-devbind.py
>
> Finally, vhost-scsi seems broken to me so two workarounds were needed so it can
> be tested again:
>
> b9d17bfaf WORKAROUND revert virtio-net mq vring deletion
> cadb25e7d WORKAROUND examples/vhost_scsi: avoid broken EVENT_IDX
>
> Stefan Hajnoczi (24):
> vhost: move vring_call() into trans_af_unix.c
> vhost: move AF_UNIX code from socket.c to trans_af_unix.c
> vhost: allocate per-socket transport state
> vhost: move socket_fd and un sockaddr into trans_af_unix.c
> vhost: move start_server/client() calls to trans_af_unix.c
> vhost: move vhost_user_connection to trans_af_unix.c
> vhost: move vhost_user_reconnect_init() into trans_af_unix.c
> vhost: move vhost_user.fdset to trans_af_unix.c
> vhost: pass vhost_transport_ops through vhost_new_device()
> vhost: embed struct virtio_net inside struct vhost_user_connection
> vhost: extract vhost_user.c socket I/O into transport
> vhost: move slave_req_fd field to AF_UNIX transport
> vhost: move mmap/munmap to AF_UNIX transport
> vhost: move librte_vhost to drivers/
> vhost: add virtio pci framework
> vhost: remember a vhost_virtqueue's queue index
> vhost: add virtio-vhost-user transport
> vhost: add RTE_VHOST_USER_VIRTIO_TRANSPORT flag
> net/vhost: add virtio-vhost-user support
> examples/vhost_scsi: add --socket-file argument
> examples/vhost_scsi: add virtio-vhost-user support
> usertools: add virtio-vhost-user devices to dpdk-devbind.py
> WORKAROUND revert virtio-net mq vring deletion
> WORKAROUND examples/vhost_scsi: avoid broken EVENT_IDX
>
> drivers/Makefile | 2 +
> {lib => drivers}/librte_vhost/Makefile | 5 +-
> lib/Makefile | 3 -
> {lib => drivers}/librte_vhost/fd_man.h | 0
> {lib => drivers}/librte_vhost/iotlb.h | 0
> {lib => drivers}/librte_vhost/rte_vhost.h | 1 +
> {lib => drivers}/librte_vhost/vhost.h | 193 +++-
> {lib => drivers}/librte_vhost/vhost_user.h | 9 +-
> drivers/librte_vhost/virtio_pci.h | 267 +++++
> drivers/librte_vhost/virtio_vhost_user.h | 18 +
> drivers/librte_vhost/virtqueue.h | 181 ++++
> {lib => drivers}/librte_vhost/fd_man.c | 0
> {lib => drivers}/librte_vhost/iotlb.c | 0
> drivers/librte_vhost/socket.c | 282 ++++++
> drivers/librte_vhost/trans_af_unix.c | 795 +++++++++++++++
> drivers/librte_vhost/trans_virtio_vhost_user.c | 1050 ++++++++++++++++++++
> {lib => drivers}/librte_vhost/vhost.c | 18 +-
> {lib => drivers}/librte_vhost/vhost_user.c | 200 +---
> {lib => drivers}/librte_vhost/virtio_net.c | 0
> drivers/librte_vhost/virtio_pci.c | 504 ++++++++++
> drivers/net/vhost/rte_eth_vhost.c | 13 +
> examples/vhost_scsi/vhost_scsi.c | 104 +-
> lib/librte_vhost/socket.c | 828 ---------------
> .../librte_vhost/rte_vhost_version.map | 0
> usertools/dpdk-devbind.py | 8 +
> 25 files changed, 3455 insertions(+), 1026 deletions(-)
> rename {lib => drivers}/librte_vhost/Makefile (86%)
> rename {lib => drivers}/librte_vhost/fd_man.h (100%)
> rename {lib => drivers}/librte_vhost/iotlb.h (100%)
> rename {lib => drivers}/librte_vhost/rte_vhost.h (99%)
> rename {lib => drivers}/librte_vhost/vhost.h (68%)
> rename {lib => drivers}/librte_vhost/vhost_user.h (93%)
> create mode 100644 drivers/librte_vhost/virtio_pci.h
> create mode 100644 drivers/librte_vhost/virtio_vhost_user.h
> create mode 100644 drivers/librte_vhost/virtqueue.h
> rename {lib => drivers}/librte_vhost/fd_man.c (100%)
> rename {lib => drivers}/librte_vhost/iotlb.c (100%)
> create mode 100644 drivers/librte_vhost/socket.c
> create mode 100644 drivers/librte_vhost/trans_af_unix.c
> create mode 100644 drivers/librte_vhost/trans_virtio_vhost_user.c
> rename {lib => drivers}/librte_vhost/vhost.c (97%)
> rename {lib => drivers}/librte_vhost/vhost_user.c (89%)
> rename {lib => drivers}/librte_vhost/virtio_net.c (100%)
> create mode 100644 drivers/librte_vhost/virtio_pci.c
> delete mode 100644 lib/librte_vhost/socket.c
> rename {lib => drivers}/librte_vhost/rte_vhost_version.map (100%)
>
next prev parent reply other threads:[~2018-01-31 10:03 UTC|newest]
Thread overview: 29+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-01-19 13:44 Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 01/24] vhost: move vring_call() into trans_af_unix.c Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 02/24] vhost: move AF_UNIX code from socket.c to trans_af_unix.c Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 03/24] vhost: allocate per-socket transport state Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 04/24] vhost: move socket_fd and un sockaddr into trans_af_unix.c Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 05/24] vhost: move start_server/client() calls to trans_af_unix.c Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 06/24] vhost: move vhost_user_connection " Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 07/24] vhost: move vhost_user_reconnect_init() into trans_af_unix.c Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 08/24] vhost: move vhost_user.fdset to trans_af_unix.c Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 09/24] vhost: pass vhost_transport_ops through vhost_new_device() Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 10/24] vhost: embed struct virtio_net inside struct vhost_user_connection Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 11/24] vhost: extract vhost_user.c socket I/O into transport Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 12/24] vhost: move slave_req_fd field to AF_UNIX transport Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 13/24] vhost: move mmap/munmap " Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 14/24] vhost: move librte_vhost to drivers/ Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 15/24] vhost: add virtio pci framework Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 16/24] vhost: remember a vhost_virtqueue's queue index Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 17/24] vhost: add virtio-vhost-user transport Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 18/24] vhost: add RTE_VHOST_USER_VIRTIO_TRANSPORT flag Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 19/24] net/vhost: add virtio-vhost-user support Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 20/24] examples/vhost_scsi: add --socket-file argument Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 21/24] examples/vhost_scsi: add virtio-vhost-user support Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 22/24] usertools: add virtio-vhost-user devices to dpdk-devbind.py Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 23/24] WORKAROUND revert virtio-net mq vring deletion Stefan Hajnoczi
2018-01-30 17:52 ` Maxime Coquelin
2018-01-19 13:44 ` [dpdk-dev] [RFC 24/24] WORKAROUND examples/vhost_scsi: avoid broken EVENT_IDX Stefan Hajnoczi
2018-01-19 19:31 ` Michael S. Tsirkin
2018-01-31 10:02 ` Maxime Coquelin [this message]
[not found] ` <20180410093847.GA22081@stefanha-x1.localdomain>
2018-04-10 14:56 ` [dpdk-dev] [RFC 00/24] vhost: add virtio-vhost-user transport Wang, Wei W
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1c34e10d-e59b-cae8-0bfe-fa511116ef47@redhat.com \
--to=maxime.coquelin@redhat.com \
--cc=dev@dpdk.org \
--cc=jasowang@redhat.com \
--cc=mst@redhat.com \
--cc=stefanha@redhat.com \
--cc=wei.w.wang@intel.com \
--cc=yliu@fridaylinux.org \
--cc=zhiyong.yang@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).