DPDK patches and discussions
 help / color / mirror / Atom feed
From: Nikos Dragazis <ndragazis@arrikto.com>
To: dev@dpdk.org
Cc: Maxime Coquelin <maxime.coquelin@redhat.com>,
	Tiwei Bie <tiwei.bie@intel.com>,
	Zhihong Wang <zhihong.wang@intel.com>,
	Stefan Hajnoczi <stefanha@redhat.com>,
	Wei Wang <wei.w.wang@intel.com>,
	Stojaczyk Dariusz <dariusz.stojaczyk@intel.com>,
	Vangelis Koukis <vkoukis@arrikto.com>
Subject: [dpdk-dev] [PATCH 00/28] vhost: add virtio-vhost-user transport
Date: Wed, 19 Jun 2019 18:14:25 +0300	[thread overview]
Message-ID: <1560957293-17294-1-git-send-email-ndragazis@arrikto.com> (raw)

Hi everyone,

this patch series introduces the concept of the virtio-vhost-user
transport. This is actually a revised version of an earlier RFC
implementation that has been proposed by Stefan Hajnoczi [1]. Though
this is a great feature, it seems to have been stalled, so I’d like to
restart the conversation on this and hopefully get it merged with your
help. Let me give you an overview.

The virtio-vhost-user transport is a vhost-user transport implementation
that is based on the virtio-vhost-user device. Its key difference with
the existing transport is that it allows deploying vhost-user targets
inside dedicated Storage Appliance VMs instead of host user space. In
other words, it allows having guests that act as vhost-user backends for
other guests.

The virtio-vhost-user device implements the vhost-user control plane
(master-slave communication) as follows:

1. it parses the vhost-user messages from the vhost-user unix domain
   socket and forwards them to the slave guest through virtqueues

2. it maps the vhost memory regions in QEMU’s process address space and
   exposes them to the slave guest as a RAM-backed PCI MMIO region

3. it hooks up doorbells to the callfds. The slave guest can use these
   doorbells to interrupt the master guest driver

The device code has not yet been merged into upstream QEMU, but this is
definitely the end goal. The current state is that we are awaiting for
the approval of the virtio spec.

I have Cced Darek from the SPDK community who has helped me a lot by
reviewing this series. Note that any device type could be implemented
over this new transport. So, adding the virtio-vhost-user transport in
DPDK would allow using it from SPDK as well.

Getting into the code internals, this patch series makes the following
changes:

1. introduce a generic interface for the transport-specific operations.
   Each of the two available transports, the pre-existing AF_UNIX
   transport and the virtio-vhost-user transport, is going to implement
   this interface. The AF_UNIX-specific code has been extracted from the
   core vhost-user code and is now part of the AF_UNIX transport
   implementation in trans_af_unix.c.

2. introduce the virtio-vhost-user transport. The virtio-vhost-user
   transport requires a driver for the virtio-vhost-user devices. The
   driver along with the transport implementation have been packed into
   a separate library in `drivers/virtio_vhost_user/`. The necessary
   virtio-pci code has been copied from `drivers/net/virtio/`. Some
   additional changes have been made so that the driver can utilize the
   additional resources of the virtio-vhost-user device.

3. update librte_vhost public API to enable choosing transport for each
   new vhost device. Extend the vhost net driver and vhost-scsi example
   application to export this new API to the end user.

The primary changes I did to Stefan’s RFC implementation are the
following:

1. moved postcopy live migration code into trans_af_unix.c. Postcopy
   live migration relies on the userfault fd mechanism, which cannot be
   supported by virtio-vhost-user.

2. moved setup of the log memory region into trans_af_unix.c. Setting up
   the log memory region involves mapping/unmapping guest memory. This
   is an AF_UNIX transport-specific operation.

3. introduced a vhost transport operation for
   process_slave_message_reply()

4. moved the virtio-vhost-user transport/driver into a separate library
   in `drivers/virtio_vhost_user/`. This required making vhost.h and
   vhost_user.h part of librte_vhost public API and exporting some
   private symbols via the version script. This looks better to me that
   just moving the entire librte_vhost into `drivers/`. I am not sure if
   this is the most appropriate solution. I am looking forward to your
   suggestions on this.

5. made use of the virtio PCI capabilities for the additional device
   resources (doorbells, shared memory). This required changes in
   virtio_pci.c and trans_virtio_vhost_user.c.

6. [minor] changed some commit headlines to comply with
   check-git-log.sh.

Please, have a look and let me know about your thoughts. Any
reviews/pointers/suggestions are welcome.

Best regards,
Nikos

[1] http://mails.dpdk.org/archives/dev/2018-January/088155.html


Nikos Dragazis (23):
  vhost: introduce vhost transport operations structure
  vhost: move socket management code
  vhost: move socket fd and un sockaddr
  vhost: move vhost-user connection
  vhost: move vhost-user reconnection
  vhost: move vhost-user fdset
  vhost: propagate vhost transport operations
  vhost: use a single structure for the device state
  vhost: extract socket I/O into transport
  vhost: move slave request fd and lock
  vhost: move mmap/munmap
  vhost: move setup of the log memory region
  vhost: remove main fd parameter from msg handlers
  vhost: move postcopy live migration code
  vhost: support registering additional vhost-user transports
  drivers/virtio_vhost_user: add virtio PCI framework
  drivers: add virtio-vhost-user transport
  drivers/virtio_vhost_user: use additional device resources
  vhost: add flag for choosing vhost-user transport
  net/vhost: add virtio-vhost-user support
  mk: link apps with virtio-vhost-user driver
  config: add option for the virtio-vhost-user transport
  usertools: add virtio-vhost-user devices to dpdk-devbind.py

Stefan Hajnoczi (5):
  vhost: allocate per-socket transport state
  vhost: move start server/client calls
  vhost: add index field in vhost virtqueues
  examples/vhost_scsi: add --socket-file argument
  examples/vhost_scsi: add virtio-vhost-user support

 config/common_base                                 |    6 +
 config/common_linux                                |    1 +
 drivers/Makefile                                   |    5 +
 drivers/net/vhost/rte_eth_vhost.c                  |   13 +
 drivers/virtio_vhost_user/Makefile                 |   27 +
 .../rte_virtio_vhost_user_version.map              |    4 +
 .../virtio_vhost_user/trans_virtio_vhost_user.c    | 1077 +++++++++++++++++++
 drivers/virtio_vhost_user/virtio_pci.c             |  520 ++++++++++
 drivers/virtio_vhost_user/virtio_pci.h             |  289 ++++++
 drivers/virtio_vhost_user/virtio_vhost_user.h      |   18 +
 drivers/virtio_vhost_user/virtqueue.h              |  181 ++++
 examples/vhost_scsi/vhost_scsi.c                   |  103 +-
 lib/librte_vhost/Makefile                          |    4 +-
 lib/librte_vhost/rte_vhost.h                       |    1 +
 lib/librte_vhost/rte_vhost_version.map             |   11 +
 lib/librte_vhost/socket.c                          |  685 +-----------
 lib/librte_vhost/trans_af_unix.c                   | 1094 ++++++++++++++++++++
 lib/librte_vhost/vhost.c                           |   22 +-
 lib/librte_vhost/vhost.h                           |  298 +++++-
 lib/librte_vhost/vhost_user.c                      |  474 ++-------
 lib/librte_vhost/vhost_user.h                      |   10 +-
 mk/rte.app.mk                                      |    6 +
 usertools/dpdk-devbind.py                          |    7 +
 23 files changed, 3764 insertions(+), 1092 deletions(-)
 create mode 100644 drivers/virtio_vhost_user/Makefile
 create mode 100644 drivers/virtio_vhost_user/rte_virtio_vhost_user_version.map
 create mode 100644 drivers/virtio_vhost_user/trans_virtio_vhost_user.c
 create mode 100644 drivers/virtio_vhost_user/virtio_pci.c
 create mode 100644 drivers/virtio_vhost_user/virtio_pci.h
 create mode 100644 drivers/virtio_vhost_user/virtio_vhost_user.h
 create mode 100644 drivers/virtio_vhost_user/virtqueue.h
 create mode 100644 lib/librte_vhost/trans_af_unix.c

-- 
2.7.4


             reply	other threads:[~2019-06-19 15:15 UTC|newest]

Thread overview: 40+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-06-19 15:14 Nikos Dragazis [this message]
2019-06-19 15:14 ` [dpdk-dev] [PATCH 01/28] vhost: introduce vhost transport operations structure Nikos Dragazis
2019-06-19 20:14   ` Aaron Conole
2019-06-20 10:30     ` Bruce Richardson
2019-06-20 18:24       ` Nikos Dragazis
2019-06-20 18:19     ` Nikos Dragazis
2019-06-19 15:14 ` [dpdk-dev] [PATCH 02/28] vhost: move socket management code Nikos Dragazis
2019-06-19 15:14 ` [dpdk-dev] [PATCH 03/28] vhost: allocate per-socket transport state Nikos Dragazis
2019-06-19 15:14 ` [dpdk-dev] [PATCH 04/28] vhost: move socket fd and un sockaddr Nikos Dragazis
2019-06-19 15:14 ` [dpdk-dev] [PATCH 05/28] vhost: move start server/client calls Nikos Dragazis
2019-06-19 15:14 ` [dpdk-dev] [PATCH 06/28] vhost: move vhost-user connection Nikos Dragazis
2019-06-19 15:14 ` [dpdk-dev] [PATCH 07/28] vhost: move vhost-user reconnection Nikos Dragazis
2019-06-19 15:14 ` [dpdk-dev] [PATCH 08/28] vhost: move vhost-user fdset Nikos Dragazis
2019-06-19 15:14 ` [dpdk-dev] [PATCH 09/28] vhost: propagate vhost transport operations Nikos Dragazis
2019-06-19 15:14 ` [dpdk-dev] [PATCH 10/28] vhost: use a single structure for the device state Nikos Dragazis
2019-06-19 15:14 ` [dpdk-dev] [PATCH 11/28] vhost: extract socket I/O into transport Nikos Dragazis
2019-06-19 15:14 ` [dpdk-dev] [PATCH 12/28] vhost: move slave request fd and lock Nikos Dragazis
2019-06-19 15:14 ` [dpdk-dev] [PATCH 13/28] vhost: move mmap/munmap Nikos Dragazis
2019-06-19 15:14 ` [dpdk-dev] [PATCH 14/28] vhost: move setup of the log memory region Nikos Dragazis
2019-06-19 15:14 ` [dpdk-dev] [PATCH 15/28] vhost: remove main fd parameter from msg handlers Nikos Dragazis
2019-06-19 15:14 ` [dpdk-dev] [PATCH 16/28] vhost: move postcopy live migration code Nikos Dragazis
2019-06-19 15:14 ` [dpdk-dev] [PATCH 17/28] vhost: support registering additional vhost-user transports Nikos Dragazis
2019-06-19 15:14 ` [dpdk-dev] [PATCH 18/28] drivers/virtio_vhost_user: add virtio PCI framework Nikos Dragazis
2019-09-05 16:34   ` Maxime Coquelin
2019-09-09  8:42     ` Nikos Dragazis
2019-09-09  8:44       ` Maxime Coquelin
2019-06-19 15:14 ` [dpdk-dev] [PATCH 19/28] vhost: add index field in vhost virtqueues Nikos Dragazis
2019-06-19 15:14 ` [dpdk-dev] [PATCH 20/28] drivers: add virtio-vhost-user transport Nikos Dragazis
2019-06-19 15:14 ` [dpdk-dev] [PATCH 21/28] drivers/virtio_vhost_user: use additional device resources Nikos Dragazis
2019-06-19 15:14 ` [dpdk-dev] [PATCH 22/28] vhost: add flag for choosing vhost-user transport Nikos Dragazis
2019-06-19 15:14 ` [dpdk-dev] [PATCH 23/28] net/vhost: add virtio-vhost-user support Nikos Dragazis
2019-06-19 15:14 ` [dpdk-dev] [PATCH 24/28] examples/vhost_scsi: add --socket-file argument Nikos Dragazis
2019-06-19 15:14 ` [dpdk-dev] [PATCH 25/28] examples/vhost_scsi: add virtio-vhost-user support Nikos Dragazis
2019-06-19 15:14 ` [dpdk-dev] [PATCH 26/28] mk: link apps with virtio-vhost-user driver Nikos Dragazis
2019-06-19 15:14 ` [dpdk-dev] [PATCH 27/28] config: add option for the virtio-vhost-user transport Nikos Dragazis
2019-06-19 15:14 ` [dpdk-dev] [PATCH 28/28] usertools: add virtio-vhost-user devices to dpdk-devbind.py Nikos Dragazis
     [not found] ` <CGME20190620113240eucas1p22ca4faa64a36bbb7aec38a81298ade56@eucas1p2.samsung.com>
2019-06-20 11:32   ` [dpdk-dev] [PATCH 00/28] vhost: add virtio-vhost-user transport Ilya Maximets
2019-06-20 23:44     ` Nikos Dragazis
2019-06-20 11:35 ` Maxime Coquelin
2019-06-22 20:26   ` Nikos Dragazis

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1560957293-17294-1-git-send-email-ndragazis@arrikto.com \
    --to=ndragazis@arrikto.com \
    --cc=dariusz.stojaczyk@intel.com \
    --cc=dev@dpdk.org \
    --cc=maxime.coquelin@redhat.com \
    --cc=stefanha@redhat.com \
    --cc=tiwei.bie@intel.com \
    --cc=vkoukis@arrikto.com \
    --cc=wei.w.wang@intel.com \
    --cc=zhihong.wang@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).