From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by dpdk.org (Postfix) with ESMTP id 348D01CE47 for ; Fri, 6 Apr 2018 09:14:18 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga105.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 06 Apr 2018 00:14:17 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.48,414,1517904000"; d="scan'208";a="218075771" Received: from fmsmsx107.amr.corp.intel.com ([10.18.124.205]) by fmsmga006.fm.intel.com with ESMTP; 06 Apr 2018 00:14:17 -0700 Received: from FMSMSX110.amr.corp.intel.com (10.18.116.10) by fmsmsx107.amr.corp.intel.com (10.18.124.205) with Microsoft SMTP Server (TLS) id 14.3.319.2; Fri, 6 Apr 2018 00:14:16 -0700 Received: from bgsmsx105.gar.corp.intel.com (10.223.43.197) by fmsmsx110.amr.corp.intel.com (10.18.116.10) with Microsoft SMTP Server (TLS) id 14.3.319.2; Fri, 6 Apr 2018 00:14:16 -0700 Received: from bgsmsx101.gar.corp.intel.com ([169.254.1.52]) by BGSMSX105.gar.corp.intel.com ([169.254.3.206]) with mapi id 14.03.0319.002; Fri, 6 Apr 2018 12:44:14 +0530 From: "Yang, Zhiyong" To: "Tan, Jianfeng" , "dev@dpdk.org" CC: "maxime.coquelin@redhat.com" , "Bie, Tiwei" , "Wang, Dong1" , "Wang, Zhihong" Thread-Topic: [PATCH v6] net/virtio-user: add support for server mode Thread-Index: AQHTzP0HKgR/0KeKZ0ahiljpjZSMeKPyHWGAgAE0hdA= Date: Fri, 6 Apr 2018 07:14:13 +0000 Message-ID: References: <20180404171753.43422-1-zhiyong.yang@intel.com> <20180406001855.54062-1-zhiyong.yang@intel.com> <61fda2c8-588d-7afe-bee5-44d73c04a470@intel.com> In-Reply-To: <61fda2c8-588d-7afe-bee5-44d73c04a470@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-version: 11.0.0.116 dlp-reaction: no-action x-originating-ip: [10.223.10.10] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH v6] net/virtio-user: add support for server mode X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Apr 2018 07:14:19 -0000 Hi Jianfeng, > -----Original Message----- > From: Tan, Jianfeng > Sent: Friday, April 6, 2018 2:13 AM > To: Yang, Zhiyong ; dev@dpdk.org > Cc: maxime.coquelin@redhat.com; Bie, Tiwei ; Wang, > Dong1 ; Wang, Zhihong > Subject: Re: [PATCH v6] net/virtio-user: add support for server mode >=20 >=20 >=20 > On 4/6/2018 8:18 AM, zhiyong.yang@intel.com wrote: > > In a container environment if the vhost-user backend restarts, there's > > no way for it to reconnect to virtio-user. To address this, support > > for server mode is added. In this mode the socket file is created by > > virtio- user, which the backend then connects to. This means that if > > the backend restarts, it can reconnect to virtio-user and continue > communications. > > > > With current implementation, LSC is enabled at virtio-user side to > > support to accept the coming connection. > > > > Release note is updated in this patch. > > > > Signed-off-by: Zhiyong Yang > > --- > > > > Changes in V6: > > 1. fix report wrong link stauts in server mode. > > 2. fix some code style issues. > > > > Changes in V5: > > 1. Support server mode virtio-user startup in non-blocking mode. > > 2. rebase on top of dpdk-next-virtio. > > > > Changes in V4: > > 1. Don't create new pthread any more and use librte_eal interrupt threa= d. > > 2. virtio-user doesn't work in blocking mode any more for the first > connection. > > Client mode vhost-user startups firstly, then server mode virtio-user > > creates socket file and startups. Keep consistency with usage of > > client mode virtio-user. > > > > Changes in V3: > > 1. use EAL epoll mechanism instead of vhost events. Cancel to export > > vhost event APIs. > > 2. rebase the code on top of dpdk-next-virtio > > > > Changes in V2: > > 1. split two patches 1/5 and 2/5 from v1 patchset to fix some existing > > issues which is not strongly related to support for server mode 2. > > move fdset related functions to librte_eal from librte_vhost exposed > > as new APIs. > > 3. release note is added in the patch 5/5. > > 4. squash data structure change patch into 4/5 according to Maxime's > suggestion. > > > > doc/guides/rel_notes/release_18_05.rst | 6 ++ > > drivers/net/virtio/virtio_user/vhost_user.c | 45 ++++++++-- > > drivers/net/virtio/virtio_user/virtio_user_dev.c | 40 +++++++-- > > drivers/net/virtio/virtio_user/virtio_user_dev.h | 3 + > > drivers/net/virtio/virtio_user_ethdev.c | 101 > ++++++++++++++++++++--- > > 5 files changed, 171 insertions(+), 24 deletions(-) > > > > diff --git a/doc/guides/rel_notes/release_18_05.rst > > b/doc/guides/rel_notes/release_18_05.rst > > index 9cc77f893..f8897b2e9 100644 > > --- a/doc/guides/rel_notes/release_18_05.rst > > +++ b/doc/guides/rel_notes/release_18_05.rst > > @@ -58,6 +58,12 @@ New Features > > * Added support for NVGRE, VXLAN and GENEVE filters in flow API. > > * Added support for DROP action in flow API. > > > > +* **Added support for virtio-user server mode.** > > + In a container environment if the vhost-user backend restarts, > > +there's no way > > + for it to reconnect to virtio-user. To address this, support for > > +server mode > > + is added. In this mode the socket file is created by virtio-user, > > +which the > > + backend connects to. This means that if the backend restarts, it > > +can reconnect > > + to virtio-user and continue communications. > > > > API Changes > > ----------- > > diff --git a/drivers/net/virtio/virtio_user/vhost_user.c > > b/drivers/net/virtio/virtio_user/vhost_user.c > > index 91c6449bb..a6df97a00 100644 > > --- a/drivers/net/virtio/virtio_user/vhost_user.c > > +++ b/drivers/net/virtio/virtio_user/vhost_user.c > > @@ -378,6 +378,30 @@ vhost_user_sock(struct virtio_user_dev *dev, > > return 0; > > } > > > > +#define MAX_VIRTIO_USER_BACKLOG 1 > > +static int > > +virtio_user_start_server(struct virtio_user_dev *dev, struct > > +sockaddr_un *un) { > > + int ret; > > + int flag; > > + int fd =3D dev->listenfd; > > + > > + ret =3D bind(fd, (struct sockaddr *)un, sizeof(*un)); > > + if (ret < 0) { > > + PMD_DRV_LOG(ERR, "failed to bind to %s: %s; remove it and > try again\n", > > + dev->path, strerror(errno)); > > + return -1; > > + } > > + ret =3D listen(fd, MAX_VIRTIO_USER_BACKLOG); > > + if (ret < 0) > > + return -1; > > + > > + flag =3D fcntl(fd, F_GETFL); > > + fcntl(fd, F_SETFL, flag | O_NONBLOCK); > > + > > + return 0; > > +} > > + > > /** > > * Set up environment to talk with a vhost user backend. > > * > > @@ -405,13 +429,24 @@ vhost_user_setup(struct virtio_user_dev *dev) > > memset(&un, 0, sizeof(un)); > > un.sun_family =3D AF_UNIX; > > snprintf(un.sun_path, sizeof(un.sun_path), "%s", dev->path); > > - if (connect(fd, (struct sockaddr *)&un, sizeof(un)) < 0) { > > - PMD_DRV_LOG(ERR, "connect error, %s", strerror(errno)); > > - close(fd); > > - return -1; > > + > > + if (dev->is_server) { > > + dev->listenfd =3D fd; > > + if (virtio_user_start_server(dev, &un) < 0) { > > + PMD_DRV_LOG(ERR, "virtio-user startup fails in > server mode"); > > + close(fd); > > + return -1; > > + } > > + dev->vhostfd =3D -1; > > + } else { > > + if (connect(fd, (struct sockaddr *)&un, sizeof(un)) < 0) { > > + PMD_DRV_LOG(ERR, "connect error, %s", > strerror(errno)); > > + close(fd); > > + return -1; > > + } > > + dev->vhostfd =3D fd; > > } > > > > - dev->vhostfd =3D fd; > > return 0; > > } > > > > diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c > > b/drivers/net/virtio/virtio_user/virtio_user_dev.c > > index f90fee9e5..3b776282c 100644 > > --- a/drivers/net/virtio/virtio_user/virtio_user_dev.c > > +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c > > @@ -99,6 +99,9 @@ virtio_user_start_device(struct virtio_user_dev *dev) > > uint64_t features; > > int ret; > > > > + if (dev->vhostfd < 0) > > + return -1; >=20 > This breaks virtio-user with vhost-kernel which does not need vhostfd, an= d > will be always -1. >=20 Ok, I will modify the code as followings if (is_vhost_user_by_type(dev->path) && dev->vhostfd < 0) return -1;=20 (is_vhost_user_by_type(dev->path)) > > + > > /* Do not check return as already done in init, or reset in stop */ > > dev->ops->send_request(dev, VHOST_USER_SET_OWNER, NULL); > > > > @@ -254,6 +257,8 @@ virtio_user_fill_intr_handle(struct virtio_user_dev > *dev) > > eth_dev->intr_handle->fd =3D -1; > > if (dev->vhostfd >=3D 0) > > eth_dev->intr_handle->fd =3D dev->vhostfd; > > + else if (dev->is_server) > > + eth_dev->intr_handle->fd =3D dev->listenfd; > > > > return 0; > > } > > @@ -267,7 +272,7 @@ virtio_user_dev_setup(struct virtio_user_dev *dev) > > dev->vhostfds =3D NULL; > > dev->tapfds =3D NULL; > > > > - if (is_vhost_user_by_type(dev->path)) { > > + if (dev->is_server || is_vhost_user_by_type(dev->path)) { >=20 > I think we still fail to pick out an invalidated case: specify "server" > parameter for a vhost-kernel path. I will add the checking at the starting of the function. If (dev->is_server && access(dev->path) =3D=3D 0) Return -1; How about the change ? >=20 > > dev->ops =3D &ops_user; > > } else { > > dev->ops =3D &ops_kernel; > > @@ -337,16 +342,25 @@ virtio_user_dev_init(struct virtio_user_dev *dev, > char *path, int queues, > > return -1; > > } > > > > - if (dev->ops->send_request(dev, VHOST_USER_SET_OWNER, NULL) > < 0) { > > - PMD_INIT_LOG(ERR, "set_owner fails: %s", strerror(errno)); > > - return -1; > > - } > > + if (dev->vhostfd >=3D 0) { > > + if (dev->ops->send_request(dev, > VHOST_USER_SET_OWNER, > > + NULL) < 0) { > > + PMD_INIT_LOG(ERR, "set_owner fails: %s", > > + strerror(errno)); > > + return -1; > > + } > > > > - if (dev->ops->send_request(dev, VHOST_USER_GET_FEATURES, > > - &dev->device_features) < 0) { > > - PMD_INIT_LOG(ERR, "get_features failed: %s", > strerror(errno)); > > - return -1; > > + if (dev->ops->send_request(dev, > VHOST_USER_GET_FEATURES, > > + &dev->device_features) < 0) { > > + PMD_INIT_LOG(ERR, "get_features failed: %s", > > + strerror(errno)); > > + return -1; > > + } > > + } else { > > + /* Just pretend vhost-user can support all these features */ >=20 > Although I don't think we need to fix this, please also note that this co= uld be > problematic that if some feature is negotiated but not supported by the > vhost-user which comes later. > Good description. I will add it in next version. =20 Thanks for your those comments Zhiyong