From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6EC29A0A02; Wed, 24 Mar 2021 23:17:33 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D562D4067B; Wed, 24 Mar 2021 23:17:32 +0100 (CET) Received: from relay3-d.mail.gandi.net (relay3-d.mail.gandi.net [217.70.183.195]) by mails.dpdk.org (Postfix) with ESMTP id 8A6CF4014F for ; Wed, 24 Mar 2021 23:17:31 +0100 (CET) X-Originating-IP: 78.45.89.65 Received: from [192.168.1.23] (ip-78-45-89-65.net.upcbroadband.cz [78.45.89.65]) (Authenticated sender: i.maximets@ovn.org) by relay3-d.mail.gandi.net (Postfix) with ESMTPSA id E1BB960008; Wed, 24 Mar 2021 22:17:29 +0000 (UTC) To: Maxime Coquelin , Ilya Maximets , Stefan Hajnoczi Cc: Chenbo Xia , dev@dpdk.org, Adrian Moreno , Julia Suvorova References: <20210317202530.4145673-1-i.maximets@ovn.org> <4f26e9a9-8dd1-f619-4337-dcecf35d9f3b@ovn.org> From: Ilya Maximets Message-ID: <47df18f1-fe75-1d77-92db-411b6433478d@ovn.org> Date: Wed, 24 Mar 2021 23:17:29 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit Subject: Re: [dpdk-dev] [RFC 0/4] SocketPair Broker support for vhost and virtio-user. X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 3/24/21 10:51 PM, Maxime Coquelin wrote: > > > On 3/24/21 10:39 PM, Ilya Maximets wrote: >> On 3/24/21 9:56 PM, Maxime Coquelin wrote: >>> Hi Ilya, >>> >>> On 3/19/21 5:45 PM, Ilya Maximets wrote: >>>> On 3/19/21 5:11 PM, Ilya Maximets wrote: >>>>> On 3/19/21 3:39 PM, Stefan Hajnoczi wrote: >>>>>> Hi Ilya, >>>>>> By the way, it's not clear to me why dpdkvhostuser is deprecated. If OVS >>>>>> is restarted then existing vhost-user connections drop with an error but >>>>>> QEMU could attempt to reconnect to the UNIX domain socket which the new >>>>>> OVS instance will set up. >>>>>> >>>>>> Why is it impossible to reconnect when OVS owns the listen socket? >>>>> >>>>> Well, AFAIK, qemu reconnects client connections only: >>>>> >>>>> ``reconnect`` sets the timeout for reconnecting on non-server >>>>> sockets when the remote end goes away. qemu will delay this many >>>>> seconds and then attempt to reconnect. Zero disables reconnecting, >>>>> and is the default. >>>>> >>>>> I'm not sure about exact reason. It was historically this way. >>>>> For me it doesn't make much sense. I mean, your right that it's >>>>> just a socket, so it should not matter who listens and who connects. >>>>> If reconnection is possible in one direction, it should be possible >>>>> in the opposite direction too. >>>> >>>> Sorry, my thought slipped. :) Yes, QEMU supports re-connection >>>> for client sockets. So, in theory, dpdkvhostuser ports should work >>>> after re-connection. And that would be nice. I don't remember >>>> right now why this doesn't work... Maybe vhost-user parts in QEMU >>>> doesn't handle this case. Need to dig some more into that and refresh >>>> my memory. It was so long ago... >>>> >>>> Maxime, do you remember? >>> >>> Sorry for the delay. I didn't remember, so I wanted to have a try. >>> >>> I can confirm reconnect works with QEMU as client and with Vhost PMD as >>> server with: >>> >>> >>> >>> >>> >>> >>> >>> >>> >>>
>> function='0x0'/> >>> >> >> Cool, thanks for checking. :) >> If it works with vhost PMD, it should probably work with OVS too. >> >> There is still a couple of problems: >> >> 1. OpenStack Nova doesn't support configuration of a 'reconnect' >> in libvirt xml (it only adds the 'source'): >> https://github.com/openstack/nova/blob/master/nova/virt/libvirt/config.py#L1834 >> >> 2. 'reconnect' configuration supported only starting from libvirt 4.1.0. >> It's released in 2018, but still some systems are using older versions. >> e.g. Ubuntu 18.04 which will be supported until 2023 uses libvirt 4.0.0. > > Is it really a problem? We can keep using OVS as client for OSP, Sure, no problem here. I think, we scared enough users so they mostly switched to this configuration and it's a good thing. > and use OVS as server for containers. There is a problem here that OVS emits a deprecation warning each time you're trying to use dpdkvhostuser ports. There are no problems functionally, though, except that deprecation also implies that we're consciously not adding new features to these ports. Another thing is that current state of the art of k8s CNI requires mounting of the whole /var/run/openvswitch or managing directories by hands. So, IMO, Broker idea is still valid is some form in terms of hassle reduction for users. > >>> >>>> >>>>> >>>>> dpdkvhostuser was deprecated just to scare users and force them to >>>>> migrate to dpdkvhostuserclient and avoid constant bug reports like: >>>>> >>>>> "OVS service restarted and network is lost now". >>>>> >>>>> BTW, virtio-user ports in DPDK doesn't support re-connection in client >>>>> mode too. >>>> >>>> This is still true, though. virtio-user in client mode doesn't reconnect. >>> >>> That could be added, and it is maybe not as important for containers as >>> it is for VM to support it, given the ephemeral nature of containers? >> >> Well, restart of OVS should not require restarting of all the containers >> on the host even though they are "stateless". >> >> BTW, some infrastructure changes that I made in this series might be >> reused to implement client-side reconnection for virtio-user. > > Great, I'll look at the series when we'll work on adding reconnect for > clients. Please, take a look at the patch #1 in this set sooner. It's a bug fix. :) > > Thanks, > Maxime > >>> >>> Regards, >>> Maxime >>> >>>> >>>>> >>>>> BTW2, with SocketPair Broker it might be cheaper to implement server >>>>> reconnection in QEMU because all connections in these case are client >>>>> connections, i.e. both ends will connect() to a broker. >>>>> >>>>> Bets regards, Ilya Maximets. >>>>> >>>> >> >