From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id AE91B42D65; Fri, 30 Jun 2023 05:45:45 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 3FE6A406B5; Fri, 30 Jun 2023 05:45:45 +0200 (CEST) Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) by mails.dpdk.org (Postfix) with ESMTP id 86E3B4021F for ; Fri, 30 Jun 2023 05:45:43 +0200 (CEST) Received: by mail-pl1-f175.google.com with SMTP id d9443c01a7336-1b7f2239bfdso11632895ad.1 for ; Thu, 29 Jun 2023 20:45:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20221208.gappssmtp.com; s=20221208; t=1688096742; x=1690688742; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=3I9RtELCGE63CMm2plA3tHPjVH69QInnABcf/18b5Pk=; b=dWXJqFFqCuAKq/yoa2V6tNQ5m+YM7Tf/xCeYqGVOvvRIpNqqtbIDeBbd4nKS+I+VCF GYD4Ei0TJgCh1/wVn69ooLlevGAu2CKDozgtu6+SJ7vrbPfG94qAgLiHFRR5+tZU0iSv P6/ehnpoN8yEmxlMIvFlw//k3ROn4u9VrZtu/OlikQLZSZ3NSZWfuJHbfE93KDpuHgZx c+gT1S7CkVT79ry45E54JxBvu/N0QW89jZS7dSRNLlStVJRiW1G6xPAw2iY5M0aYGp9d MJfGh5mezWeEGrXZPYQQGNhQmfnNTbrsk8maP/ds4oTfVX3+ldvZIkNvaCi4biM91wCF ZRSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688096742; x=1690688742; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3I9RtELCGE63CMm2plA3tHPjVH69QInnABcf/18b5Pk=; b=XnyJVgoEXAR26wBrpIdhq+sLpiiT5z1sHz9qwW5CuAX5R3uGX7HPHzRTF1GoY94W/s oaS6z6ZxRT+bESEj7XPUblsHTZ3SmolyYZChKk8N/pO7uWpMSRfgyAGQlvoEOoPUg1s/ 6zkut4LQPyjH0ky7/U4oPH2WqeqX/hxSX1X8j6zy0FJdrcZutJLXdcMt0i6HPLkLW96J 3wv0IeT8PrRkBlITVXdKmlyMFZg3hv83N1yICCWYsq9bvehue5ofZIx4XzJp7w6/S7Y8 v/QTO3tF7jn1jPdbvcPyIMPkgPJgycJwlCx63Ql7A6UCDSgu0mcm2Mt07v8T/PEquMPM pAlA== X-Gm-Message-State: ABy/qLb23mbY8NvLdBEUuNJTnhFi4FgFszj3i4UeIHV3PORZh9MBFejz pG/JIEMfTUSK+RCZuzVz4+lBQw== X-Google-Smtp-Source: APBJJlERGyT5f2je6gJMlbISKrC5D0PIni+cdj9aIZnSgGTjVPTmuBpJeBHXlrPZGrkssNycJu86mg== X-Received: by 2002:a17:902:da92:b0:1b6:9551:e2b8 with SMTP id j18-20020a170902da9200b001b69551e2b8mr1316438plx.34.1688096742178; Thu, 29 Jun 2023 20:45:42 -0700 (PDT) Received: from hermes.local (204-195-120-218.wavecable.com. [204.195.120.218]) by smtp.gmail.com with ESMTPSA id jn16-20020a170903051000b001b53472353csm9746099plb.211.2023.06.29.20.45.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Jun 2023 20:45:41 -0700 (PDT) Date: Thu, 29 Jun 2023 20:45:40 -0700 From: Stephen Hemminger To: Ilya Maximets Cc: Maxime Coquelin , Chenbo Xia , dev@dpdk.org, Adrian Moreno , Stefan Hajnoczi , Julia Suvorova Subject: Re: [dpdk-dev] [RFC 0/4] SocketPair Broker support for vhost and virtio-user. Message-ID: <20230629204540.3b87da13@hermes.local> In-Reply-To: <20210317202530.4145673-1-i.maximets@ovn.org> References: <20210317202530.4145673-1-i.maximets@ovn.org> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Wed, 17 Mar 2021 21:25:26 +0100 Ilya Maximets wrote: > TL;DR; > Managing socket files is too much fun. :) And here is how this > could be improved: > https://github.com/igsilya/one-socket > https://github.com/igsilya/one-socket/blob/main/doc/socketpair-broker.rst > In particular for vhost-user case. > > In modern virtualization setups there are tens or hundreds of different > socket files for different purposes. Sockets to manage various > daemons, vhost-user sockets for various virtual devices, memif sockets > for memif network interfaces and so on. > > In order to make things work in containerized environments software > systems has to share these sockets with containers. In most cases > this sharing is implemented as a shared directory mounted inside the > container, because socket files could be re-created in runtime or even > not be available at the container startup. For example, if they are > created by the application inside the container. > > Even more configuration tricks required in order to share some sockets > between different containers and not only with the host, e.g. to > create service chains. > And some housekeeping usually required for applications in case the > socket server terminated abnormally and socket files left on a file > system: > "failed to bind to vhu: Address already in use; remove it and try again" > > Additionally, all applications (system and user's!) should follow > naming conventions and place socket files in particular location on a > file system to make things work. > > In particular, this applies to vhost-user sockets. > > This patch-set aims to eliminate most of the inconveniences by > leveraging an infrastructure service provided by a SocketPair Broker. > > *SocketPair Broker* is a daemon that mediates establishment of direct > socket-based connections between clients. > > *One Socket* is a reference implementation of a SocketPair Broker > Daemon, SocketPair Broker Protocol and a helper library for client > applications (libspbroker): > > https://github.com/igsilya/one-socket > > It's fully functional, but not completely ready for production use > for now. See 'todo' section in README.rst in one-socket repository. > > Basically, it's a daemon that listens on a single unix socket > (broker socket) and accepts clients. Client connects and provides a > 'key'. If two clients provided the same 'key', One Socket daemon > creates a pair of connected sockets with socketpair() and sends > sides of this pair to these two clients. At this point two clients > have a direct communication channel between them. They will disconnect > from the broker and continue to operate and communicate normally. > > Workflow overview with pictures available here: > > https://github.com/igsilya/one-socket/blob/main/doc/socketpair-broker.rst > > Communication with a broker based on a SocketPair Broker Protocol: > > https://github.com/igsilya/one-socket/blob/main/doc/socketpair-broker-proto-spec.rst > > > This patch-set extends vhost library, vhost pmd and virtio-user pmd to > support SocketPair Broker as one of the connection methods. > Usage example: > > # Starting a One Socket daemon with socket './one.socket': > $ ONE_SOCKET_PATH=./one.socket ./one-socket > > # Starting testpmd #1 with virtio-user device in server mode: > $ dpdk-testpmd --no-pci --in-memory --single-file-segments \ > --vdev="net_virtio_user,path=./one.socket,broker-key=MY-KEY,server=1" > > # Starting testpmd #2 with vhost pmd in client mode: > $ dpdk-testpmd --no-pci --in-memory --single-file-segments \ > --vdev="eth_vhost0,iface=./one.socket,broker-key=MY-KEY,client=1" > > Details how to build and install One Socket are in README.rst in > one-socket repository. > > DPDK side is the first step of implementation. Once available in DPDK, > support could be easily added to Open vSwith or VPP or any DPDK-based > application. Same support could be added to QEMU (found a volunteer > for this part). > > Since SocketPair Broker is completely independent from the purposes > connection will be used for, it has a potential to unify and replace > all one-to-one unix socket connections on a host. This one persistent > broker socket could be passed to any containers and can be used by > any application greatly simplifying system management. > > Any feedback or suggestions on any component of this solution including > this patch-set, One Socket Daemon, SocketPair Broker Protocol or > libspbroker library are very welcome. > > *Note* about the patch set: > > First patch in a series is a *bug* fix, so it should be considered even > outside of this series. It basically fixes unregistering of a > listening socket that never happens in current code. > > The virtio-user part of the series heavily depends on this bug fix > since broker connection unlike listening socket will not persist and > will generate lots of interrupts if not unregistered. > > Ilya Maximets (4): > net/virtio: fix interrupt unregistering for listening socket > vhost: add support for SocketPair Broker > net/vhost: add support for SocketPair Broker > net/virtio: add support for SocketPair Broker > > doc/guides/nics/vhost.rst | 5 + > doc/guides/nics/virtio.rst | 5 + > doc/guides/prog_guide/vhost_lib.rst | 10 + > drivers/net/vhost/rte_eth_vhost.c | 42 ++- > drivers/net/virtio/meson.build | 6 + > drivers/net/virtio/virtio_user/vhost_user.c | 122 ++++++++- > .../net/virtio/virtio_user/virtio_user_dev.c | 142 +++++++--- > .../net/virtio/virtio_user/virtio_user_dev.h | 6 +- > drivers/net/virtio/virtio_user_ethdev.c | 30 ++- > lib/librte_vhost/meson.build | 7 + > lib/librte_vhost/rte_vhost.h | 1 + > lib/librte_vhost/socket.c | 245 ++++++++++++++++-- > 12 files changed, 550 insertions(+), 71 deletions(-) > Adding yet another dependency in DPDK for something that really should be part of the OS (systemd?) seems like something that is not going to be widely adopted or appreciated. If this is a real problem, this should be solved outside of DPDK with a common infrastructure available in almost all distros.