DPDK patches and discussions
 help / color / mirror / Atom feed
From: Stefan Hajnoczi <stefanha@redhat.com>
To: dev@dpdk.org
Cc: maxime.coquelin@redhat.com, Yuanhan Liu <yliu@fridaylinux.org>,
	wei.w.wang@intel.com, mst@redhat.com, zhiyong.yang@intel.com,
	jasowang@redhat.com, Stefan Hajnoczi <stefanha@redhat.com>
Subject: [dpdk-dev] [RFC 00/24] vhost: add virtio-vhost-user transport
Date: Fri, 19 Jan 2018 13:44:20 +0000	[thread overview]
Message-ID: <20180119134444.24927-1-stefanha@redhat.com> (raw)

This patch series implements the virtio-vhost-user device, which tunnels
vhost-user protocol messages over virtio.  This lets guests act as vhost device
backends for other guests.

The virtio-vhost-user device is the result of discussion about Wei Wang and
Zhiyong Yang's vhost-pci device.  This patch series demonstrates that vhost
device backends, such as the vhost vdev driver, can work over both AF_UNIX and
virtio-vhost-user without significant modifications.  This allows a lot of code
to be shared between traditional AF_UNIX vhost-user and virtio-vhost-user.  The
vhost-pci patches duplicated the vhost net device backend and didn't reuse
librte_vhost:

  http://dpdk.org/ml/archives/dev/2017-November/082615.html

User-visible changes
--------------------
The vhost vdev can now be used when DPDK runs inside a guest with a
virtio-vhost-user PCI device:

  --vdev net_vhost0,iface="0000:00:04.0",virtio-transport=1

The vhost-scsi example has also been extended to support virtio-vhost-user:

  ./vhost-scsi ... -- --virtio-vhost-user-pci "0000:00:04.0"

For more information (including instructions for running the code), see
https://wiki.qemu.org/Features/VirtioVhostUser

Virtio device design
--------------------
The virtio-vhost-user device is a new virtio device type.  It acts as a
vhost-user transport and is an alternative for the traditional AF_UNIX
transport.

You can find the virtio-vhost-user VIRTIO device specification here:
https://stefanha.github.io/virtio/vhost-user-slave.html#x1-2830007

librte_vhost API changes
------------------------
This patch series extends librte_vhost so that it now accepts:

  rte_vhost_driver_register("0000:00:04.0",
                            RTE_VHOST_USER_VIRTIO_TRANSPORT);

All other librte_vhost API usage remains unchanged, except that the file
descriptors exposed in some <rte_vhost.h> structs will be -1 since there is no
file descriptor passing involved.

This makes it extremely easy to support virtio-vhost-user in existing vhost
device backends!  I have extended the vhost vdev driver and the vhost-scsi
example application in this patch series.

Patch series overview
---------------------
This series is based on commit 814339ba7eea13d132508af2cccec2f73568e2d0 from
dpdk-next-virtio/master.  You can also get my git branch here:

  https://github.com/stefanha/dpdk/tree/virtio-vhost-user

The initial patches refactor librte_vhost so that AF_UNIX-specific code is moved
to a new trans_af_unix.c file.  This also introduces a struct
vhost_transport_ops interface that all transports will implement:

  adf412c3c vhost: move vring_call() into trans_af_unix.c
  04079a077 vhost: move AF_UNIX code from socket.c to trans_af_unix.c
  885642091 vhost: allocate per-socket transport state
  9c48377df vhost: move socket_fd and un sockaddr into trans_af_unix.c
  c646e6292 vhost: move start_server/client() calls to trans_af_unix.c
  db07ef7a8 vhost: move vhost_user_connection to trans_af_unix.c
  d10b80163 vhost: move vhost_user_reconnect_init() into trans_af_unix.c
  1258bcd68 vhost: move vhost_user.fdset to trans_af_unix.c
  bc7f6d7ab vhost: pass vhost_transport_ops through vhost_new_device()
  b82187a29 vhost: embed struct virtio_net inside struct vhost_user_connection
  abbd544f2 vhost: extract vhost_user.c socket I/O into transport
  0658d711b vhost: move slave_req_fd field to AF_UNIX transport
  e2ecf78ed vhost: move mmap/munmap to AF_UNIX transport

I was about to add the virtio-vhost-user PCI driver when I realized that
lib/librte_vhost/ cannot have a dependency on rte_bus_pci.h (it lives in
drivers/).  The solution I chose is to move all of librte_vhost to
drivers/librte_vhost/, but I'm open to suggestions if this is undesirable.

  8615c2140 vhost: move librte_vhost to drivers/

Since virtio-vhost-user is a virtio device it's necessary to perform a sequence
of device initialization steps and set up virtqueues.  I didn't see a virtio
API in DPDK and didn't have time to create one myself.  I copied the virtio
code from drivers/net/virtio/ as a quick hack, but really there needs to be a
librte_virtio that is shared.

  c26937c66 vhost: add virtio pci framework

Next the virtio-vhost-user transport is added along with the
RTE_VHOST_USER_VIRTIO_TRANSPORT flag:

  7fad5adc4 vhost: remember a vhost_virtqueue's queue index
  d26922892 vhost: add virtio-vhost-user transport
  f562a70ab vhost: add RTE_VHOST_USER_VIRTIO_TRANSPORT flag

Then I extended the vhost vdev and vhost-scsi example application to support
virtio-vhost-user:

  47434f2a2 net/vhost: add virtio-vhost-user support
  22ca05d1b examples/vhost_scsi: add --socket-file argument
  db3b391dc examples/vhost_scsi: add virtio-vhost-user support

It was also necessary to tweak dpdk-devbind.py to support virtio-vhost-user:

  4aee6f653 usertools: add virtio-vhost-user devices to dpdk-devbind.py

Finally, vhost-scsi seems broken to me so two workarounds were needed so it can
be tested again:

  b9d17bfaf WORKAROUND revert virtio-net mq vring deletion
  cadb25e7d WORKAROUND examples/vhost_scsi: avoid broken EVENT_IDX

Stefan Hajnoczi (24):
  vhost: move vring_call() into trans_af_unix.c
  vhost: move AF_UNIX code from socket.c to trans_af_unix.c
  vhost: allocate per-socket transport state
  vhost: move socket_fd and un sockaddr into trans_af_unix.c
  vhost: move start_server/client() calls to trans_af_unix.c
  vhost: move vhost_user_connection to trans_af_unix.c
  vhost: move vhost_user_reconnect_init() into trans_af_unix.c
  vhost: move vhost_user.fdset to trans_af_unix.c
  vhost: pass vhost_transport_ops through vhost_new_device()
  vhost: embed struct virtio_net inside struct vhost_user_connection
  vhost: extract vhost_user.c socket I/O into transport
  vhost: move slave_req_fd field to AF_UNIX transport
  vhost: move mmap/munmap to AF_UNIX transport
  vhost: move librte_vhost to drivers/
  vhost: add virtio pci framework
  vhost: remember a vhost_virtqueue's queue index
  vhost: add virtio-vhost-user transport
  vhost: add RTE_VHOST_USER_VIRTIO_TRANSPORT flag
  net/vhost: add virtio-vhost-user support
  examples/vhost_scsi: add --socket-file argument
  examples/vhost_scsi: add virtio-vhost-user support
  usertools: add virtio-vhost-user devices to dpdk-devbind.py
  WORKAROUND revert virtio-net mq vring deletion
  WORKAROUND examples/vhost_scsi: avoid broken EVENT_IDX

 drivers/Makefile                                   |    2 +
 {lib => drivers}/librte_vhost/Makefile             |    5 +-
 lib/Makefile                                       |    3 -
 {lib => drivers}/librte_vhost/fd_man.h             |    0
 {lib => drivers}/librte_vhost/iotlb.h              |    0
 {lib => drivers}/librte_vhost/rte_vhost.h          |    1 +
 {lib => drivers}/librte_vhost/vhost.h              |  193 +++-
 {lib => drivers}/librte_vhost/vhost_user.h         |    9 +-
 drivers/librte_vhost/virtio_pci.h                  |  267 +++++
 drivers/librte_vhost/virtio_vhost_user.h           |   18 +
 drivers/librte_vhost/virtqueue.h                   |  181 ++++
 {lib => drivers}/librte_vhost/fd_man.c             |    0
 {lib => drivers}/librte_vhost/iotlb.c              |    0
 drivers/librte_vhost/socket.c                      |  282 ++++++
 drivers/librte_vhost/trans_af_unix.c               |  795 +++++++++++++++
 drivers/librte_vhost/trans_virtio_vhost_user.c     | 1050 ++++++++++++++++++++
 {lib => drivers}/librte_vhost/vhost.c              |   18 +-
 {lib => drivers}/librte_vhost/vhost_user.c         |  200 +---
 {lib => drivers}/librte_vhost/virtio_net.c         |    0
 drivers/librte_vhost/virtio_pci.c                  |  504 ++++++++++
 drivers/net/vhost/rte_eth_vhost.c                  |   13 +
 examples/vhost_scsi/vhost_scsi.c                   |  104 +-
 lib/librte_vhost/socket.c                          |  828 ---------------
 .../librte_vhost/rte_vhost_version.map             |    0
 usertools/dpdk-devbind.py                          |    8 +
 25 files changed, 3455 insertions(+), 1026 deletions(-)
 rename {lib => drivers}/librte_vhost/Makefile (86%)
 rename {lib => drivers}/librte_vhost/fd_man.h (100%)
 rename {lib => drivers}/librte_vhost/iotlb.h (100%)
 rename {lib => drivers}/librte_vhost/rte_vhost.h (99%)
 rename {lib => drivers}/librte_vhost/vhost.h (68%)
 rename {lib => drivers}/librte_vhost/vhost_user.h (93%)
 create mode 100644 drivers/librte_vhost/virtio_pci.h
 create mode 100644 drivers/librte_vhost/virtio_vhost_user.h
 create mode 100644 drivers/librte_vhost/virtqueue.h
 rename {lib => drivers}/librte_vhost/fd_man.c (100%)
 rename {lib => drivers}/librte_vhost/iotlb.c (100%)
 create mode 100644 drivers/librte_vhost/socket.c
 create mode 100644 drivers/librte_vhost/trans_af_unix.c
 create mode 100644 drivers/librte_vhost/trans_virtio_vhost_user.c
 rename {lib => drivers}/librte_vhost/vhost.c (97%)
 rename {lib => drivers}/librte_vhost/vhost_user.c (89%)
 rename {lib => drivers}/librte_vhost/virtio_net.c (100%)
 create mode 100644 drivers/librte_vhost/virtio_pci.c
 delete mode 100644 lib/librte_vhost/socket.c
 rename {lib => drivers}/librte_vhost/rte_vhost_version.map (100%)

-- 
2.14.3

             reply	other threads:[~2018-01-19 13:44 UTC|newest]

Thread overview: 29+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-01-19 13:44 Stefan Hajnoczi [this message]
2018-01-19 13:44 ` [dpdk-dev] [RFC 01/24] vhost: move vring_call() into trans_af_unix.c Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 02/24] vhost: move AF_UNIX code from socket.c to trans_af_unix.c Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 03/24] vhost: allocate per-socket transport state Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 04/24] vhost: move socket_fd and un sockaddr into trans_af_unix.c Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 05/24] vhost: move start_server/client() calls to trans_af_unix.c Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 06/24] vhost: move vhost_user_connection " Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 07/24] vhost: move vhost_user_reconnect_init() into trans_af_unix.c Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 08/24] vhost: move vhost_user.fdset to trans_af_unix.c Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 09/24] vhost: pass vhost_transport_ops through vhost_new_device() Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 10/24] vhost: embed struct virtio_net inside struct vhost_user_connection Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 11/24] vhost: extract vhost_user.c socket I/O into transport Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 12/24] vhost: move slave_req_fd field to AF_UNIX transport Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 13/24] vhost: move mmap/munmap " Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 14/24] vhost: move librte_vhost to drivers/ Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 15/24] vhost: add virtio pci framework Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 16/24] vhost: remember a vhost_virtqueue's queue index Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 17/24] vhost: add virtio-vhost-user transport Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 18/24] vhost: add RTE_VHOST_USER_VIRTIO_TRANSPORT flag Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 19/24] net/vhost: add virtio-vhost-user support Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 20/24] examples/vhost_scsi: add --socket-file argument Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 21/24] examples/vhost_scsi: add virtio-vhost-user support Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 22/24] usertools: add virtio-vhost-user devices to dpdk-devbind.py Stefan Hajnoczi
2018-01-19 13:44 ` [dpdk-dev] [RFC 23/24] WORKAROUND revert virtio-net mq vring deletion Stefan Hajnoczi
2018-01-30 17:52   ` Maxime Coquelin
2018-01-19 13:44 ` [dpdk-dev] [RFC 24/24] WORKAROUND examples/vhost_scsi: avoid broken EVENT_IDX Stefan Hajnoczi
2018-01-19 19:31   ` Michael S. Tsirkin
2018-01-31 10:02 ` [dpdk-dev] [RFC 00/24] vhost: add virtio-vhost-user transport Maxime Coquelin
     [not found] ` <20180410093847.GA22081@stefanha-x1.localdomain>
2018-04-10 14:56   ` Wang, Wei W

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20180119134444.24927-1-stefanha@redhat.com \
    --to=stefanha@redhat.com \
    --cc=dev@dpdk.org \
    --cc=jasowang@redhat.com \
    --cc=maxime.coquelin@redhat.com \
    --cc=mst@redhat.com \
    --cc=wei.w.wang@intel.com \
    --cc=yliu@fridaylinux.org \
    --cc=zhiyong.yang@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).