From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 91650A0561; Wed, 17 Mar 2021 21:25:42 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 08E37140F4F; Wed, 17 Mar 2021 21:25:42 +0100 (CET) Received: from relay3-d.mail.gandi.net (relay3-d.mail.gandi.net [217.70.183.195]) by mails.dpdk.org (Postfix) with ESMTP id 9AEC6140F4D for ; Wed, 17 Mar 2021 21:25:40 +0100 (CET) X-Originating-IP: 78.45.89.65 Received: from im-t490s.redhat.com (ip-78-45-89-65.net.upcbroadband.cz [78.45.89.65]) (Authenticated sender: i.maximets@ovn.org) by relay3-d.mail.gandi.net (Postfix) with ESMTPSA id B4A9B60005; Wed, 17 Mar 2021 20:25:38 +0000 (UTC) From: Ilya Maximets To: Maxime Coquelin Cc: Chenbo Xia , dev@dpdk.org, Adrian Moreno , Stefan Hajnoczi , Julia Suvorova , Ilya Maximets Date: Wed, 17 Mar 2021 21:25:26 +0100 Message-Id: <20210317202530.4145673-1-i.maximets@ovn.org> X-Mailer: git-send-email 2.26.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [dpdk-dev] [RFC 0/4] SocketPair Broker support for vhost and virtio-user. X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" TL;DR; Managing socket files is too much fun. :) And here is how this could be improved: https://github.com/igsilya/one-socket https://github.com/igsilya/one-socket/blob/main/doc/socketpair-broker.rst In particular for vhost-user case. In modern virtualization setups there are tens or hundreds of different socket files for different purposes. Sockets to manage various daemons, vhost-user sockets for various virtual devices, memif sockets for memif network interfaces and so on. In order to make things work in containerized environments software systems has to share these sockets with containers. In most cases this sharing is implemented as a shared directory mounted inside the container, because socket files could be re-created in runtime or even not be available at the container startup. For example, if they are created by the application inside the container. Even more configuration tricks required in order to share some sockets between different containers and not only with the host, e.g. to create service chains. And some housekeeping usually required for applications in case the socket server terminated abnormally and socket files left on a file system: "failed to bind to vhu: Address already in use; remove it and try again" Additionally, all applications (system and user's!) should follow naming conventions and place socket files in particular location on a file system to make things work. In particular, this applies to vhost-user sockets. This patch-set aims to eliminate most of the inconveniences by leveraging an infrastructure service provided by a SocketPair Broker. *SocketPair Broker* is a daemon that mediates establishment of direct socket-based connections between clients. *One Socket* is a reference implementation of a SocketPair Broker Daemon, SocketPair Broker Protocol and a helper library for client applications (libspbroker): https://github.com/igsilya/one-socket It's fully functional, but not completely ready for production use for now. See 'todo' section in README.rst in one-socket repository. Basically, it's a daemon that listens on a single unix socket (broker socket) and accepts clients. Client connects and provides a 'key'. If two clients provided the same 'key', One Socket daemon creates a pair of connected sockets with socketpair() and sends sides of this pair to these two clients. At this point two clients have a direct communication channel between them. They will disconnect from the broker and continue to operate and communicate normally. Workflow overview with pictures available here: https://github.com/igsilya/one-socket/blob/main/doc/socketpair-broker.rst Communication with a broker based on a SocketPair Broker Protocol: https://github.com/igsilya/one-socket/blob/main/doc/socketpair-broker-proto-spec.rst This patch-set extends vhost library, vhost pmd and virtio-user pmd to support SocketPair Broker as one of the connection methods. Usage example: # Starting a One Socket daemon with socket './one.socket': $ ONE_SOCKET_PATH=./one.socket ./one-socket # Starting testpmd #1 with virtio-user device in server mode: $ dpdk-testpmd --no-pci --in-memory --single-file-segments \ --vdev="net_virtio_user,path=./one.socket,broker-key=MY-KEY,server=1" # Starting testpmd #2 with vhost pmd in client mode: $ dpdk-testpmd --no-pci --in-memory --single-file-segments \ --vdev="eth_vhost0,iface=./one.socket,broker-key=MY-KEY,client=1" Details how to build and install One Socket are in README.rst in one-socket repository. DPDK side is the first step of implementation. Once available in DPDK, support could be easily added to Open vSwith or VPP or any DPDK-based application. Same support could be added to QEMU (found a volunteer for this part). Since SocketPair Broker is completely independent from the purposes connection will be used for, it has a potential to unify and replace all one-to-one unix socket connections on a host. This one persistent broker socket could be passed to any containers and can be used by any application greatly simplifying system management. Any feedback or suggestions on any component of this solution including this patch-set, One Socket Daemon, SocketPair Broker Protocol or libspbroker library are very welcome. *Note* about the patch set: First patch in a series is a *bug* fix, so it should be considered even outside of this series. It basically fixes unregistering of a listening socket that never happens in current code. The virtio-user part of the series heavily depends on this bug fix since broker connection unlike listening socket will not persist and will generate lots of interrupts if not unregistered. Ilya Maximets (4): net/virtio: fix interrupt unregistering for listening socket vhost: add support for SocketPair Broker net/vhost: add support for SocketPair Broker net/virtio: add support for SocketPair Broker doc/guides/nics/vhost.rst | 5 + doc/guides/nics/virtio.rst | 5 + doc/guides/prog_guide/vhost_lib.rst | 10 + drivers/net/vhost/rte_eth_vhost.c | 42 ++- drivers/net/virtio/meson.build | 6 + drivers/net/virtio/virtio_user/vhost_user.c | 122 ++++++++- .../net/virtio/virtio_user/virtio_user_dev.c | 142 +++++++--- .../net/virtio/virtio_user/virtio_user_dev.h | 6 +- drivers/net/virtio/virtio_user_ethdev.c | 30 ++- lib/librte_vhost/meson.build | 7 + lib/librte_vhost/rte_vhost.h | 1 + lib/librte_vhost/socket.c | 245 ++++++++++++++++-- 12 files changed, 550 insertions(+), 71 deletions(-) -- 2.26.2