From: "Yang, Zhiyong" <zhiyong.yang@intel.com>
To: "Tan, Jianfeng" <jianfeng.tan@intel.com>, "dev@dpdk.org" <dev@dpdk.org>
Cc: "maxime.coquelin@redhat.com" <maxime.coquelin@redhat.com>,
"Bie, Tiwei" <tiwei.bie@intel.com>,
"Wang, Dong1" <dong1.wang@intel.com>,
"Wang, Zhihong" <zhihong.wang@intel.com>
Subject: Re: [dpdk-dev] [PATCH v6] net/virtio-user: add support for server mode
Date: Fri, 6 Apr 2018 07:14:13 +0000 [thread overview]
Message-ID: <E182254E98A5DA4EB1E657AC7CB9BD2A8B08874E@BGSMSX101.gar.corp.intel.com> (raw)
In-Reply-To: <61fda2c8-588d-7afe-bee5-44d73c04a470@intel.com>
Hi Jianfeng,
> -----Original Message-----
> From: Tan, Jianfeng
> Sent: Friday, April 6, 2018 2:13 AM
> To: Yang, Zhiyong <zhiyong.yang@intel.com>; dev@dpdk.org
> Cc: maxime.coquelin@redhat.com; Bie, Tiwei <tiwei.bie@intel.com>; Wang,
> Dong1 <dong1.wang@intel.com>; Wang, Zhihong <zhihong.wang@intel.com>
> Subject: Re: [PATCH v6] net/virtio-user: add support for server mode
>
>
>
> On 4/6/2018 8:18 AM, zhiyong.yang@intel.com wrote:
> > In a container environment if the vhost-user backend restarts, there's
> > no way for it to reconnect to virtio-user. To address this, support
> > for server mode is added. In this mode the socket file is created by
> > virtio- user, which the backend then connects to. This means that if
> > the backend restarts, it can reconnect to virtio-user and continue
> communications.
> >
> > With current implementation, LSC is enabled at virtio-user side to
> > support to accept the coming connection.
> >
> > Release note is updated in this patch.
> >
> > Signed-off-by: Zhiyong Yang <zhiyong.yang@intel.com>
> > ---
> >
> > Changes in V6:
> > 1. fix report wrong link stauts in server mode.
> > 2. fix some code style issues.
> >
> > Changes in V5:
> > 1. Support server mode virtio-user startup in non-blocking mode.
> > 2. rebase on top of dpdk-next-virtio.
> >
> > Changes in V4:
> > 1. Don't create new pthread any more and use librte_eal interrupt thread.
> > 2. virtio-user doesn't work in blocking mode any more for the first
> connection.
> > Client mode vhost-user startups firstly, then server mode virtio-user
> > creates socket file and startups. Keep consistency with usage of
> > client mode virtio-user.
> >
> > Changes in V3:
> > 1. use EAL epoll mechanism instead of vhost events. Cancel to export
> > vhost event APIs.
> > 2. rebase the code on top of dpdk-next-virtio
> >
> > Changes in V2:
> > 1. split two patches 1/5 and 2/5 from v1 patchset to fix some existing
> > issues which is not strongly related to support for server mode 2.
> > move fdset related functions to librte_eal from librte_vhost exposed
> > as new APIs.
> > 3. release note is added in the patch 5/5.
> > 4. squash data structure change patch into 4/5 according to Maxime's
> suggestion.
> >
> > doc/guides/rel_notes/release_18_05.rst | 6 ++
> > drivers/net/virtio/virtio_user/vhost_user.c | 45 ++++++++--
> > drivers/net/virtio/virtio_user/virtio_user_dev.c | 40 +++++++--
> > drivers/net/virtio/virtio_user/virtio_user_dev.h | 3 +
> > drivers/net/virtio/virtio_user_ethdev.c | 101
> ++++++++++++++++++++---
> > 5 files changed, 171 insertions(+), 24 deletions(-)
> >
> > diff --git a/doc/guides/rel_notes/release_18_05.rst
> > b/doc/guides/rel_notes/release_18_05.rst
> > index 9cc77f893..f8897b2e9 100644
> > --- a/doc/guides/rel_notes/release_18_05.rst
> > +++ b/doc/guides/rel_notes/release_18_05.rst
> > @@ -58,6 +58,12 @@ New Features
> > * Added support for NVGRE, VXLAN and GENEVE filters in flow API.
> > * Added support for DROP action in flow API.
> >
> > +* **Added support for virtio-user server mode.**
> > + In a container environment if the vhost-user backend restarts,
> > +there's no way
> > + for it to reconnect to virtio-user. To address this, support for
> > +server mode
> > + is added. In this mode the socket file is created by virtio-user,
> > +which the
> > + backend connects to. This means that if the backend restarts, it
> > +can reconnect
> > + to virtio-user and continue communications.
> >
> > API Changes
> > -----------
> > diff --git a/drivers/net/virtio/virtio_user/vhost_user.c
> > b/drivers/net/virtio/virtio_user/vhost_user.c
> > index 91c6449bb..a6df97a00 100644
> > --- a/drivers/net/virtio/virtio_user/vhost_user.c
> > +++ b/drivers/net/virtio/virtio_user/vhost_user.c
> > @@ -378,6 +378,30 @@ vhost_user_sock(struct virtio_user_dev *dev,
> > return 0;
> > }
> >
> > +#define MAX_VIRTIO_USER_BACKLOG 1
> > +static int
> > +virtio_user_start_server(struct virtio_user_dev *dev, struct
> > +sockaddr_un *un) {
> > + int ret;
> > + int flag;
> > + int fd = dev->listenfd;
> > +
> > + ret = bind(fd, (struct sockaddr *)un, sizeof(*un));
> > + if (ret < 0) {
> > + PMD_DRV_LOG(ERR, "failed to bind to %s: %s; remove it and
> try again\n",
> > + dev->path, strerror(errno));
> > + return -1;
> > + }
> > + ret = listen(fd, MAX_VIRTIO_USER_BACKLOG);
> > + if (ret < 0)
> > + return -1;
> > +
> > + flag = fcntl(fd, F_GETFL);
> > + fcntl(fd, F_SETFL, flag | O_NONBLOCK);
> > +
> > + return 0;
> > +}
> > +
> > /**
> > * Set up environment to talk with a vhost user backend.
> > *
> > @@ -405,13 +429,24 @@ vhost_user_setup(struct virtio_user_dev *dev)
> > memset(&un, 0, sizeof(un));
> > un.sun_family = AF_UNIX;
> > snprintf(un.sun_path, sizeof(un.sun_path), "%s", dev->path);
> > - if (connect(fd, (struct sockaddr *)&un, sizeof(un)) < 0) {
> > - PMD_DRV_LOG(ERR, "connect error, %s", strerror(errno));
> > - close(fd);
> > - return -1;
> > +
> > + if (dev->is_server) {
> > + dev->listenfd = fd;
> > + if (virtio_user_start_server(dev, &un) < 0) {
> > + PMD_DRV_LOG(ERR, "virtio-user startup fails in
> server mode");
> > + close(fd);
> > + return -1;
> > + }
> > + dev->vhostfd = -1;
> > + } else {
> > + if (connect(fd, (struct sockaddr *)&un, sizeof(un)) < 0) {
> > + PMD_DRV_LOG(ERR, "connect error, %s",
> strerror(errno));
> > + close(fd);
> > + return -1;
> > + }
> > + dev->vhostfd = fd;
> > }
> >
> > - dev->vhostfd = fd;
> > return 0;
> > }
> >
> > diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c
> > b/drivers/net/virtio/virtio_user/virtio_user_dev.c
> > index f90fee9e5..3b776282c 100644
> > --- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
> > +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
> > @@ -99,6 +99,9 @@ virtio_user_start_device(struct virtio_user_dev *dev)
> > uint64_t features;
> > int ret;
> >
> > + if (dev->vhostfd < 0)
> > + return -1;
>
> This breaks virtio-user with vhost-kernel which does not need vhostfd, and
> will be always -1.
>
Ok, I will modify the code as followings
if (is_vhost_user_by_type(dev->path) && dev->vhostfd < 0)
return -1;
(is_vhost_user_by_type(dev->path))
> > +
> > /* Do not check return as already done in init, or reset in stop */
> > dev->ops->send_request(dev, VHOST_USER_SET_OWNER, NULL);
> >
> > @@ -254,6 +257,8 @@ virtio_user_fill_intr_handle(struct virtio_user_dev
> *dev)
> > eth_dev->intr_handle->fd = -1;
> > if (dev->vhostfd >= 0)
> > eth_dev->intr_handle->fd = dev->vhostfd;
> > + else if (dev->is_server)
> > + eth_dev->intr_handle->fd = dev->listenfd;
> >
> > return 0;
> > }
> > @@ -267,7 +272,7 @@ virtio_user_dev_setup(struct virtio_user_dev *dev)
> > dev->vhostfds = NULL;
> > dev->tapfds = NULL;
> >
> > - if (is_vhost_user_by_type(dev->path)) {
> > + if (dev->is_server || is_vhost_user_by_type(dev->path)) {
>
> I think we still fail to pick out an invalidated case: specify "server"
> parameter for a vhost-kernel path.
I will add the checking at the starting of the function.
If (dev->is_server && access(dev->path) == 0)
Return -1;
How about the change ?
>
> > dev->ops = &ops_user;
> > } else {
> > dev->ops = &ops_kernel;
> > @@ -337,16 +342,25 @@ virtio_user_dev_init(struct virtio_user_dev *dev,
> char *path, int queues,
> > return -1;
> > }
> >
> > - if (dev->ops->send_request(dev, VHOST_USER_SET_OWNER, NULL)
> < 0) {
> > - PMD_INIT_LOG(ERR, "set_owner fails: %s", strerror(errno));
> > - return -1;
> > - }
> > + if (dev->vhostfd >= 0) {
> > + if (dev->ops->send_request(dev,
> VHOST_USER_SET_OWNER,
> > + NULL) < 0) {
> > + PMD_INIT_LOG(ERR, "set_owner fails: %s",
> > + strerror(errno));
> > + return -1;
> > + }
> >
> > - if (dev->ops->send_request(dev, VHOST_USER_GET_FEATURES,
> > - &dev->device_features) < 0) {
> > - PMD_INIT_LOG(ERR, "get_features failed: %s",
> strerror(errno));
> > - return -1;
> > + if (dev->ops->send_request(dev,
> VHOST_USER_GET_FEATURES,
> > + &dev->device_features) < 0) {
> > + PMD_INIT_LOG(ERR, "get_features failed: %s",
> > + strerror(errno));
> > + return -1;
> > + }
> > + } else {
> > + /* Just pretend vhost-user can support all these features */
>
> Although I don't think we need to fix this, please also note that this could be
> problematic that if some feature is negotiated but not supported by the
> vhost-user which comes later.
>
Good description. I will add it in next version.
Thanks for your those comments
Zhiyong
next prev parent reply other threads:[~2018-04-06 7:14 UTC|newest]
Thread overview: 64+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-02-14 14:53 [dpdk-dev] [PATCH 0/4] add to support for virtio-user " Zhiyong Yang
2018-02-14 14:53 ` [dpdk-dev] [PATCH 1/4] vhost: move fdset functions from fd_man.c to fd_man.h Zhiyong Yang
2018-02-27 17:51 ` Maxime Coquelin
2018-02-28 1:36 ` Yang, Zhiyong
2018-02-28 8:45 ` Maxime Coquelin
2018-03-01 6:02 ` Tan, Jianfeng
2018-03-01 14:13 ` Thomas Monjalon
2018-03-05 7:43 ` Yang, Zhiyong
2018-03-05 8:54 ` Thomas Monjalon
2018-03-13 8:46 ` Yang, Zhiyong
[not found] ` <9059097.ABoCBN0gVk@xps>
2018-03-13 9:50 ` Yang, Zhiyong
2018-03-15 9:32 ` Thomas Monjalon
2018-03-16 8:43 ` Yang, Zhiyong
2018-03-21 6:51 ` Yang, Zhiyong
2018-03-15 9:45 ` [dpdk-dev] [PATCH v2 0/5] add support for virtio-user server mode zhiyong.yang
2018-03-15 9:45 ` [dpdk-dev] [PATCH v2 1/5] net/virtio: fix add pointer checking zhiyong.yang
2018-03-15 9:45 ` [dpdk-dev] [PATCH v2 2/5] net/virtio: add checking for cvq zhiyong.yang
2018-03-15 9:45 ` [dpdk-dev] [PATCH v2 3/5] eal: expose fdset related APIs zhiyong.yang
2018-03-15 9:45 ` [dpdk-dev] [PATCH v2 4/5] net/virtio-user: add support for server mode zhiyong.yang
2018-03-15 9:45 ` [dpdk-dev] [PATCH v2 5/5] net/vhost: add memory checking zhiyong.yang
2018-02-14 14:53 ` [dpdk-dev] [PATCH 2/4] net/virtio-user: add data members to support server mode Zhiyong Yang
2018-02-27 17:53 ` Maxime Coquelin
2018-02-28 1:38 ` Yang, Zhiyong
2018-02-14 14:53 ` [dpdk-dev] [PATCH 3/4] net/virtio-user: " Zhiyong Yang
2018-02-27 18:01 ` Maxime Coquelin
2018-02-28 1:53 ` Yang, Zhiyong
2018-02-28 8:33 ` Maxime Coquelin
2018-02-14 14:53 ` [dpdk-dev] [PATCH 4/4] net/vhost: add memory checking to support client mode Zhiyong Yang
2018-03-21 3:03 ` [dpdk-dev] [PATCH v3 0/4] add support for virtio-user server mode zhiyong.yang
2018-03-21 3:03 ` [dpdk-dev] [PATCH v3 1/4] net/virtio: fix add pointer checking zhiyong.yang
2018-03-28 7:26 ` Tan, Jianfeng
2018-03-28 7:48 ` Yang, Zhiyong
2018-03-29 11:59 ` Maxime Coquelin
2018-03-29 12:01 ` Maxime Coquelin
2018-03-21 3:03 ` [dpdk-dev] [PATCH v3 2/4] net/virtio: add checking for cvq zhiyong.yang
2018-03-28 8:34 ` Tan, Jianfeng
2018-03-29 11:59 ` Maxime Coquelin
2018-03-29 12:06 ` Maxime Coquelin
2018-03-21 3:03 ` [dpdk-dev] [PATCH v3 3/4] net/virtio-user: add support for server mode zhiyong.yang
2018-03-28 15:14 ` Tan, Jianfeng
2018-03-30 2:08 ` Yang, Zhiyong
2018-03-21 3:03 ` [dpdk-dev] [PATCH v3 4/4] net/vhost: add NULL pointer checking zhiyong.yang
2018-03-29 13:19 ` Maxime Coquelin
2018-03-30 2:00 ` Yang, Zhiyong
2018-03-30 7:41 ` Yang, Zhiyong
2018-04-03 12:20 ` [dpdk-dev] [PATCH v4 0/1] server mode virtio-user zhiyong.yang
2018-04-03 12:20 ` [dpdk-dev] [PATCH v4 1/1] net/virtio-user: add support for server mode zhiyong.yang
2018-04-03 15:16 ` Tan, Jianfeng
2018-04-04 3:31 ` Yang, Zhiyong
2018-04-04 3:47 ` Tan, Jianfeng
2018-04-04 5:37 ` Tiwei Bie
2018-04-04 9:59 ` Yang, Zhiyong
2018-04-04 14:57 ` Yang, Zhiyong
2018-04-04 17:17 ` [dpdk-dev] [PATCH v5] " zhiyong.yang
2018-04-05 8:29 ` Tiwei Bie
2018-04-05 9:19 ` Yang, Zhiyong
2018-04-06 7:22 ` Yang, Zhiyong
2018-04-05 9:21 ` Yang, Zhiyong
2018-04-06 0:18 ` [dpdk-dev] [PATCH v6] " zhiyong.yang
2018-04-05 18:13 ` Tan, Jianfeng
2018-04-06 7:14 ` Yang, Zhiyong [this message]
2018-04-06 9:25 ` [dpdk-dev] [PATCH v7] " zhiyong.yang
2018-04-08 0:36 ` Tan, Jianfeng
2018-04-10 11:55 ` Maxime Coquelin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=E182254E98A5DA4EB1E657AC7CB9BD2A8B08874E@BGSMSX101.gar.corp.intel.com \
--to=zhiyong.yang@intel.com \
--cc=dev@dpdk.org \
--cc=dong1.wang@intel.com \
--cc=jianfeng.tan@intel.com \
--cc=maxime.coquelin@redhat.com \
--cc=tiwei.bie@intel.com \
--cc=zhihong.wang@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).