From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by dpdk.org (Postfix) with ESMTP id 00F70684E for ; Wed, 11 May 2016 23:46:11 +0200 (CEST) Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 53CBD821C7; Wed, 11 May 2016 21:46:11 +0000 (UTC) Received: from redhat.com (vpn1-7-70.ams2.redhat.com [10.36.7.70]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with SMTP id u4BLk9ZS022326; Wed, 11 May 2016 17:46:09 -0400 Date: Thu, 12 May 2016 00:46:08 +0300 From: "Michael S. Tsirkin" To: "Loftus, Ciara" Cc: "Xie, Huawei" , Yuanhan Liu , "dev@dpdk.org" Message-ID: <20160512004240-mutt-send-email-mst@redhat.com> References: <1462603224-29510-4-git-send-email-yuanhan.liu@linux.intel.com> <20160509181254.GE5641@yliu-dev.sh.intel.com> <20160510105138-mutt-send-email-mst@redhat.com> <20160510114044-mutt-send-email-mst@redhat.com> <20160510121350-mutt-send-email-mst@redhat.com> <74F120C019F4A64C9B78E802F6AD4CC24F8A74B0@IRSMSX106.ger.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <74F120C019F4A64C9B78E802F6AD4CC24F8A74B0@IRSMSX106.ger.corp.intel.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Wed, 11 May 2016 21:46:11 +0000 (UTC) Subject: Re: [dpdk-dev] [PATCH 3/6] vhost: add reconnect ability X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 May 2016 21:46:12 -0000 On Tue, May 10, 2016 at 05:17:50PM +0000, Loftus, Ciara wrote: > > On Tue, May 10, 2016 at 09:00:45AM +0000, Xie, Huawei wrote: > > > On 5/10/2016 4:42 PM, Michael S. Tsirkin wrote: > > > > On Tue, May 10, 2016 at 08:07:00AM +0000, Xie, Huawei wrote: > > > >> On 5/10/2016 3:56 PM, Michael S. Tsirkin wrote: > > > >>> On Tue, May 10, 2016 at 07:24:10AM +0000, Xie, Huawei wrote: > > > >>>> On 5/10/2016 2:08 AM, Yuanhan Liu wrote: > > > >>>>> On Mon, May 09, 2016 at 04:47:02PM +0000, Xie, Huawei wrote: > > > >>>>>> On 5/7/2016 2:36 PM, Yuanhan Liu wrote: > > > >>>>>>> +static void * > > > >>>>>>> +vhost_user_client_reconnect(void *arg) > > > >>>>>>> +{ > > > >>>>>>> + struct reconnect_info *reconn = arg; > > > >>>>>>> + int ret; > > > >>>>>>> + > > > >>>>>>> + RTE_LOG(ERR, VHOST_CONFIG, "reconnecting...\n"); > > > >>>>>>> + while (1) { > > > >>>>>>> + ret = connect(reconn->fd, (struct sockaddr > > *)&reconn->un, > > > >>>>>>> + sizeof(reconn->un)); > > > >>>>>>> + if (ret == 0) > > > >>>>>>> + break; > > > >>>>>>> + sleep(1); > > > >>>>>>> + } > > > >>>>>>> + > > > >>>>>>> + vhost_user_add_connection(reconn->fd, reconn->vsocket); > > > >>>>>>> + free(reconn); > > > >>>>>>> + > > > >>>>>>> + return NULL; > > > >>>>>>> +} > > > >>>>>>> + > > > >>>>>> We could create hundreds of vhost-user ports in OVS. Wihout > > connections > > > >>>>>> with QEMU established, those ports are just inactive. This works > > fine in > > > >>>>>> server mode. > > > >>>>>> With client modes, do we need to create hundreds of vhost > > threads? This > > > >>>>>> would be too interruptible. > > > >>>>>> How about we create only one thread and do the reconnections > > for all the > > > >>>>>> unconnected socket? > > > >>>>> Yes, good point and good suggestion. Will do it in v2. > > > >>>> Hi Michael: > > > >>>> This reminds me another irrelevant issue. > > > >>>> In OVS, currently for each vhost port, we create an unix domain > > socket, > > > >>>> and QEMU vhost proxy connects to this socket, and we use this to > > > >>>> identify the connection. This works fine but is our workaround, > > > >>>> otherwise we have no way to identify the connection. > > > >>>> Do you think if this is an issue? > > > >> Let us say vhost creates one unix domain socket, with path as > > "sockpath", > > > >> and two virtio ports in two VMS both connect to the same socket with > > the > > > >> following command line > > > >> -chardev socket,id=char0,path=sockpath > > > >> How could vhost identify the connection? > > > > getpeername(2)? > > > > > > getpeer name returns host/port? then it isn't useful. > > > > Maybe but I'm still in the dark. Useful for what? > > > > > The typical scenario in my mind is: > > > We create a OVS port with the name "port1", and when we receive an > > > virtio connection with ID "port1", we attach this virtio interface to > > > the OVS port "port1". > > > > If you are going to listen on a socket, you can create ports > > and attach socket fds to it dynamically as long as you get connections. > > What is wrong with that? > > Hi Michael, > > I haven't reviewed the patchset fully, but to hopefully provide more clarify on how OVS uses vHost: > > OVS with DPDK needs some way to distinguish vHost connections from one another so it can switch traffic to the correct port depending on how the switch is programmed. > At the moment this is achieved by: > 1. user provides unique port name eg. 'vhost0' (this is normal behaviour in OVS - checks are put in place to avoid overlapping port names) > 2. DPDK vHost lib creates socket called 'vhost0' > 3. VM launched with vhost0 socket // -chardev socket,id=char0,path=/path/to/vhost0 > 4. OVS receives 'new_device' vhost callback, checks the name of the device (virtio_dev->ifname == vhost0?), if the name matches the name provided in step 1, OVS stores the virtio_net *dev pointer > 5. OVS uses *dev pointer to send and receive traffic to vhost0 via calls to vhost library functions eg. enqueue(*dev) / dequeue(*dev) > 6. Repeat for multiple vhost devices > > We need to make sure that there is still some way to distinguish devices from one another like in step 4. Let me know if you need any further clarification. > > Thanks, > Ciara I see. I think that the path approach is better personally - it is easier to secure, just give each QEMU access to the correct path only. > > > > > > > > > > > > > > > > > > >> Workarounds: > > > >> vhost creates two unix domain sockets, with path as "sockpath1" and > > > >> "sockpath2" respectively, > > > >> and the virtio ports in two VMS respectively connect to "sockpath1" and > > > >> "sockpath2". > > > >> > > > >> If we have some name string from QEMU over vhost, as you > > mentioned, we > > > >> could create only one socket with path "sockpath". > > > >> User ensure that the names are unique, just as how they do with > > multiple > > > >> sockets. > > > >> > > > > Seems rather fragile. > > > > > > >From the scenario above, it is enough. That is also how it works today > > > in DPDK OVS implementation with multiple sockets. > > > Any other idea? > > > > > > > > > > >>> I'm sorry, I have trouble understanding what you wrote above. > > > >>> What is the issue you are trying to work around? > > > >>> > > > >>>> Do we have plan to support identification in > > VHOST_USER_MESSAGE? With > > > >>>> the identification, if vhost as server, we only need to create one > > > >>>> socket which receives multiple connections, and use the ID in the > > > >>>> message to identify the connection. > > > >>>> > > > >>>> /huawei > > > >>> Sending e.g. -name string from qemu over vhost might be useful > > > >>> for debugging, but I'm not sure it's a good idea to > > > >>> rely on it being unique. > > > >>> > > > >>>>> Thanks. > > > >>>>> > > > >>>>> --yliu > > > >>>>> > > >