From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id EE382A0A02; Thu, 25 Mar 2021 17:43:44 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9A918140E9E; Thu, 25 Mar 2021 17:43:44 +0100 (CET) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id ED777140D19 for ; Thu, 25 Mar 2021 17:43:42 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1616690622; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=ob9o8pYg2wfQg3NmZBjbanJFufUZEonUCajCiBXXlDA=; b=LcGTTUoKTEEKxTzRNFhmEgD2xj7MPLBoP3olY3tNAueb+TMhadWlLw8uejw8uIqXDJCNSl D9bezKuDfTdinu7a38z+GnVpt1YoIJMle4sMrEdXyvjTj0eSI4WfZsenu26BaqMRoWX3Fp WHXWczkfu8trjoHUosS2wo7373wEpys= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-62-8qK-k5Q3NPytxRrVy5JXQg-1; Thu, 25 Mar 2021 12:43:37 -0400 X-MC-Unique: 8qK-k5Q3NPytxRrVy5JXQg-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id F2B2E874998; Thu, 25 Mar 2021 16:43:36 +0000 (UTC) Received: from localhost (ovpn-114-246.ams2.redhat.com [10.36.114.246]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7145861F49; Thu, 25 Mar 2021 16:43:02 +0000 (UTC) Date: Thu, 25 Mar 2021 16:43:01 +0000 From: Stefan Hajnoczi To: Ilya Maximets Cc: Billy McFall , Adrian Moreno , Maxime Coquelin , Chenbo Xia , dev@dpdk.org, Julia Suvorova , =?iso-8859-1?Q?Marc-Andr=E9?= Lureau , Daniel Berrange Message-ID: References: <53dd4b66-9e44-01c3-9f9a-b37dcadb14b7@ovn.org> <597d1ec7-d271-dc0d-522d-b900c9cb00ea@ovn.org> <2ba6ff01-fe2d-253f-cb36-303b63ba2133@ovn.org> MIME-Version: 1.0 In-Reply-To: <2ba6ff01-fe2d-253f-cb36-303b63ba2133@ovn.org> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=stefanha@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="i5z3nQcZi+A8mcJm" Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.29 Subject: Re: [dpdk-dev] [RFC 0/4] SocketPair Broker support for vhost and virtio-user. X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" --i5z3nQcZi+A8mcJm Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Mar 25, 2021 at 12:00:11PM +0100, Ilya Maximets wrote: > On 3/25/21 10:35 AM, Stefan Hajnoczi wrote: > > On Wed, Mar 24, 2021 at 02:11:31PM +0100, Ilya Maximets wrote: > >> On 3/24/21 1:05 PM, Stefan Hajnoczi wrote: > >>> On Tue, Mar 23, 2021 at 04:54:57PM -0400, Billy McFall wrote: > >>>> On Tue, Mar 23, 2021 at 3:52 PM Ilya Maximets w= rote: > >>>>> On 3/23/21 6:57 PM, Adrian Moreno wrote: > >>>>>> On 3/19/21 6:21 PM, Stefan Hajnoczi wrote: > >>>>>>> On Fri, Mar 19, 2021 at 04:29:21PM +0100, Ilya Maximets wrote: > >>>>>>>> On 3/19/21 3:05 PM, Stefan Hajnoczi wrote: > >>>>>>>>> On Thu, Mar 18, 2021 at 08:47:12PM +0100, Ilya Maximets wrote: > >>>>>>>>>> On 3/18/21 6:52 PM, Stefan Hajnoczi wrote: > >>>>>>>>>>> On Wed, Mar 17, 2021 at 09:25:26PM +0100, Ilya Maximets wrote= : > >> - How to get this fd again after the OVS restart? CNI will not be inv= oked > >> at this point to pass a new fd. > >> > >> - If application will close the connection for any reason (restart, so= me > >> reconfiguration internal to the application) and OVS will be re-star= ted > >> at the same time, abstract socket will be gone. Need a persistent d= aemon > >> to hold it. > >=20 > > I remembered that these two points can be solved by sd_notify(3) > > FDSTORE=3D1. This requires that OVS runs as a systemd service. Not sure= if > > this is the case (at least in the CNI use case)? > >=20 > > https://www.freedesktop.org/software/systemd/man/sd_notify.html >=20 > IIUC, these file descriptors only passed on the restart of the service, > so port-del + port-add scenario is not covered (and this is a very > common usecase, users are implementing some configuration changes this > way and also this is internally possible scenario, e.g. this sequence > will be triggered internally to change the OpenFlow port number). > port-del will release all the resources including the listening socket. > Keeping the fd for later use is not an option, because OVS will not know > if this port will be added back or not and fds is a limited resource. If users of the CNI plugin are reasonably expected to do this then it sounds like a blocker for the sd_notify(3) approach. Maybe it could be fixed by introducing an atomic port-rename (?) operation, but this is starting to sound too invasive. > It's also unclear how to map these file descriptors to particular ports > they belong to after restart. The first fd would be a memfd containing a description of the remaining fds plus any other crash recovery state that OVS wants. > OVS could run as a system pod or as a systemd service. It differs from > one setup to another. So it might not be controlled by systemd. Does the CNI plugin allow both configurations? It's impossible to come up with one approach that works for everyone in the general case (beyond the CNI plugin, beyond Kubernetes). I think we need to enumerate use cases and decide which ones are currently not addressed satisfactorily. > Also, it behaves as an old-style daemon, so it closes all the file > descriptors, forkes and so on. This might be adjusted, though, with > some rework of the deamonization procedure. Doesn't sound like fun but may be doable. > On the side note, it maybe interesting to allow user application to > create a socket and pass a pollable file descriptor directly to > rte_vhost_driver_register() instead of a socket path. This way > the user application may choose to use an abstract socket or a file > socket or any other future type of socket connections. This will > also allow user application to store these sockets somewhere, or > receive them from systemd/init/other management software. Yes, sounds useful. Stefan --i5z3nQcZi+A8mcJm--