DPDK patches and discussions
 help / color / mirror / Atom feed
From: Ilya Maximets <i.maximets@ovn.org>
To: Maxime Coquelin <maxime.coquelin@redhat.com>
Cc: Chenbo Xia <chenbo.xia@intel.com>,
	dev@dpdk.org, Adrian Moreno <amorenoz@redhat.com>,
	Stefan Hajnoczi <stefanha@redhat.com>,
	Julia Suvorova <jusual@redhat.com>,
	Ilya Maximets <i.maximets@ovn.org>
Subject: [dpdk-dev] [RFC 0/4] SocketPair Broker support for vhost and virtio-user.
Date: Wed, 17 Mar 2021 21:25:26 +0100	[thread overview]
Message-ID: <20210317202530.4145673-1-i.maximets@ovn.org> (raw)

TL;DR;
  Managing socket files is too much fun. :)  And here is how this
  could be improved:
    https://github.com/igsilya/one-socket
    https://github.com/igsilya/one-socket/blob/main/doc/socketpair-broker.rst
  In particular for vhost-user case.

In modern virtualization setups there are tens or hundreds of different
socket files for different purposes.  Sockets to manage various
daemons, vhost-user sockets for various virtual devices, memif sockets
for memif network interfaces and so on.

In order to make things work in containerized environments software
systems has to share these sockets with containers.  In most cases
this sharing is implemented as a shared directory mounted inside the
container, because socket files could be re-created in runtime or even
not be available at the container startup.  For example, if they are
created by the application inside the container.

Even more configuration tricks required in order to share some sockets
between different containers and not only with the host, e.g. to
create service chains.
And some housekeeping usually required for applications in case the
socket server terminated abnormally and socket files left on a file
system:
 "failed to bind to vhu: Address already in use; remove it and try again"

Additionally, all applications (system and user's!) should follow
naming conventions and place socket files in particular location on a
file system to make things work.

In particular, this applies to vhost-user sockets.

This patch-set aims to eliminate most of the inconveniences by
leveraging an infrastructure service provided by a SocketPair Broker.

*SocketPair Broker* is a daemon that mediates establishment of direct
socket-based connections between clients.

*One Socket* is a reference implementation of a SocketPair Broker
Daemon, SocketPair Broker Protocol and a helper library for client
applications (libspbroker):

  https://github.com/igsilya/one-socket

It's fully functional, but not completely ready for production use
for now.  See 'todo' section in README.rst in one-socket repository.

Basically, it's a daemon that listens on a single unix socket
(broker socket) and accepts clients.  Client connects and provides a
'key'.  If two clients provided the same 'key', One Socket daemon
creates a pair of connected sockets with socketpair() and sends
sides of this pair to these two clients.  At this point two clients
have a direct communication channel between them.  They will disconnect
from the broker and continue to operate and communicate normally.

Workflow overview with pictures available here:

  https://github.com/igsilya/one-socket/blob/main/doc/socketpair-broker.rst

Communication with a broker based on a SocketPair Broker Protocol:

  https://github.com/igsilya/one-socket/blob/main/doc/socketpair-broker-proto-spec.rst


This patch-set extends vhost library, vhost pmd and virtio-user pmd to
support SocketPair Broker as one of the connection methods.
Usage example:

  # Starting a One Socket daemon with socket './one.socket':
  $ ONE_SOCKET_PATH=./one.socket ./one-socket

  # Starting testpmd #1 with virtio-user device in server mode:
  $ dpdk-testpmd --no-pci --in-memory --single-file-segments \
      --vdev="net_virtio_user,path=./one.socket,broker-key=MY-KEY,server=1"

  # Starting testpmd #2 with vhost pmd in client mode:
  $ dpdk-testpmd --no-pci --in-memory --single-file-segments \
      --vdev="eth_vhost0,iface=./one.socket,broker-key=MY-KEY,client=1"

Details how to build and install One Socket are in README.rst in
one-socket repository.

DPDK side is the first step of implementation.  Once available in DPDK,
support could be easily added to Open vSwith or VPP or any DPDK-based
application.  Same support could be added to QEMU (found a volunteer
for this part).

Since SocketPair Broker is completely independent from the purposes
connection will be used for, it has a potential to unify and replace
all one-to-one unix socket connections on a host.  This one persistent
broker socket could be passed to any containers and can be used by
any application greatly simplifying system management.

Any feedback or suggestions on any component of this solution including
this patch-set, One Socket Daemon, SocketPair Broker Protocol or
libspbroker library are very welcome.

*Note* about the patch set:

First patch in a series is a *bug* fix, so it should be considered even
outside of this series.  It basically fixes unregistering of a
listening socket that never happens in current code.

The virtio-user part of the series heavily depends on this bug fix
since broker connection unlike listening socket will not persist and
will generate lots of interrupts if not unregistered.

Ilya Maximets (4):
  net/virtio: fix interrupt unregistering for listening socket
  vhost: add support for SocketPair Broker
  net/vhost: add support for SocketPair Broker
  net/virtio: add support for SocketPair Broker

 doc/guides/nics/vhost.rst                     |   5 +
 doc/guides/nics/virtio.rst                    |   5 +
 doc/guides/prog_guide/vhost_lib.rst           |  10 +
 drivers/net/vhost/rte_eth_vhost.c             |  42 ++-
 drivers/net/virtio/meson.build                |   6 +
 drivers/net/virtio/virtio_user/vhost_user.c   | 122 ++++++++-
 .../net/virtio/virtio_user/virtio_user_dev.c  | 142 +++++++---
 .../net/virtio/virtio_user/virtio_user_dev.h  |   6 +-
 drivers/net/virtio/virtio_user_ethdev.c       |  30 ++-
 lib/librte_vhost/meson.build                  |   7 +
 lib/librte_vhost/rte_vhost.h                  |   1 +
 lib/librte_vhost/socket.c                     | 245 ++++++++++++++++--
 12 files changed, 550 insertions(+), 71 deletions(-)

-- 
2.26.2


             reply	other threads:[~2021-03-17 20:25 UTC|newest]

Thread overview: 38+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-03-17 20:25 Ilya Maximets [this message]
2021-03-17 20:25 ` [dpdk-dev] [PATCH 1/4] net/virtio: fix interrupt unregistering for listening socket Ilya Maximets
2021-03-25  8:32   ` Maxime Coquelin
2021-04-07  7:21     ` Xia, Chenbo
2021-03-17 20:25 ` [dpdk-dev] [RFC 2/4] vhost: add support for SocketPair Broker Ilya Maximets
2021-03-17 20:25 ` [dpdk-dev] [RFC 3/4] net/vhost: " Ilya Maximets
2021-03-17 20:25 ` [dpdk-dev] [RFC 4/4] net/virtio: " Ilya Maximets
2021-03-18 17:52 ` [dpdk-dev] [RFC 0/4] SocketPair Broker support for vhost and virtio-user Stefan Hajnoczi
2021-03-18 19:47   ` Ilya Maximets
2021-03-18 20:14     ` Ilya Maximets
2021-03-19 14:16       ` Stefan Hajnoczi
2021-03-19 15:37         ` Ilya Maximets
2021-03-19 16:01           ` Stefan Hajnoczi
2021-03-19 16:02           ` Marc-André Lureau
2021-03-19  8:51     ` Marc-André Lureau
2021-03-19 11:25       ` Ilya Maximets
2021-03-19 14:05     ` Stefan Hajnoczi
2021-03-19 15:29       ` Ilya Maximets
2021-03-19 17:21         ` Stefan Hajnoczi
2021-03-23 17:57           ` Adrian Moreno
2021-03-23 18:27             ` Ilya Maximets
2021-03-23 20:54               ` Billy McFall
2021-03-24 12:05                 ` Stefan Hajnoczi
2021-03-24 13:11                   ` Ilya Maximets
2021-03-24 15:07                     ` Stefan Hajnoczi
2021-03-25  9:35                     ` Stefan Hajnoczi
2021-03-25 11:00                       ` Ilya Maximets
2021-03-25 16:43                         ` Stefan Hajnoczi
2021-03-25 17:58                           ` Ilya Maximets
2021-03-30 15:01                             ` Stefan Hajnoczi
2021-03-19 14:39 ` Stefan Hajnoczi
2021-03-19 16:11   ` Ilya Maximets
2021-03-19 16:45     ` Ilya Maximets
2021-03-24 20:56       ` Maxime Coquelin
2021-03-24 21:39         ` Ilya Maximets
2021-03-24 21:51           ` Maxime Coquelin
2021-03-24 22:17             ` Ilya Maximets
2023-06-30  3:45 ` Stephen Hemminger

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210317202530.4145673-1-i.maximets@ovn.org \
    --to=i.maximets@ovn.org \
    --cc=amorenoz@redhat.com \
    --cc=chenbo.xia@intel.com \
    --cc=dev@dpdk.org \
    --cc=jusual@redhat.com \
    --cc=maxime.coquelin@redhat.com \
    --cc=stefanha@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).