From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id E011891BF for ; Thu, 8 Sep 2016 10:53:24 +0200 (CEST) Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga102.fm.intel.com with ESMTP; 08 Sep 2016 01:53:24 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.30,298,1470726000"; d="scan'208";a="5936965" Received: from shwdeisgchi083.ccr.corp.intel.com (HELO [10.239.67.193]) ([10.239.67.193]) by fmsmga006.fm.intel.com with ESMTP; 08 Sep 2016 01:53:23 -0700 To: Yuanhan Liu References: <1470397003-5782-1-git-send-email-jianfeng.tan@intel.com> <1470397003-5782-3-git-send-email-jianfeng.tan@intel.com> <20160906064246.GQ30752@yliu-dev.sh.intel.com> <20160906082011.GA23158@yliu-dev.sh.intel.com> Cc: dev@dpdk.org, zhihong.wang@intel.com, lining18@jd.com From: "Tan, Jianfeng" Message-ID: <53e1d78c-e3b0-e16c-992d-e45ed1b31a4f@intel.com> Date: Thu, 8 Sep 2016 16:53:22 +0800 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.3.0 MIME-Version: 1.0 In-Reply-To: <20160906082011.GA23158@yliu-dev.sh.intel.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [PATCH 2/3] net/virtio_user: fix wrong sequence of messages X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Sep 2016 08:53:25 -0000 On 9/6/2016 4:20 PM, Yuanhan Liu wrote: > On Tue, Sep 06, 2016 at 03:54:30PM +0800, Tan, Jianfeng wrote: >> Hi Yuanhan, >> >> >> On 9/6/2016 2:42 PM, Yuanhan Liu wrote: >>> On Fri, Aug 05, 2016 at 11:36:42AM +0000, Jianfeng Tan wrote: >>>> When virtio_user is used with VPP's native vhost user, it cannot >>>> send/receive any packets. >>>> >>>> The root cause is that vpp-vhost-user translates the message >>>> VHOST_USER_SET_FEATURES as puting this device into init state, >>>> aka, zero all related structures. However, previous code >>>> puts this message at last in the whole initialization process, >>>> which leads to all previous information are zeroed. >>>> >>>> To fix this issue, we rearrange the sequence of those messages. >>>> - step 0, send VHOST_USER_SET_VRING_CALL so that vhost allocates >>>> virtqueue structures; >>> Yes, it is. However, it's not that right to do that (you see there is >>> a FIXME in vhost_user_set_vring_call()). >> I suppose you are specifying vhost_set_vring_call(). > Oh, I was talking about the new code: I have renamed it to > vhost_user_set_vring_call :) > >>> That means it need be fixed: we should not rely on fact that it's the >>> first per-vring message we will get in the current QEMU implementation >>> as the truth. >>> >>> That also means, naming a function like virtio_user_create_queue() based >>> on above behaviour is wrong. >> It's actually a good catch. After a light thought, I think in DPDK vhost, we >> may need to create those virtqueues once unix socket gets connected, just >> like in vhost-net, virtqueues are created on char file open. Right? > There is a difference: for vhost-net and tap mode, IIRC, it knows how > many queues before doing setup. No, from linux/drivers/vhost/net.c:vhost_net_open(), we can see that virtqueues are allocated according to VHOST_NET_VQ_MAX. How about reconsidering previous suggestion to allocate vq once connection is established? Never mind, above fix on the vhost side will not take effect on existing vpp-vhost implementations. We still need to fix it in the virtio side. > While for vhost-user, it doesn't. That > means, we have to allocate and setup virtqueues reactively: just like > what we have done in vhost_set_vring_call(). What doesn't look perfect > is it assume SET_VRING_CALL is the first per-vring message we will get. Yes, depending on the assumption that SET_VRING_CALL is the first per-vring message, looks like a bad implementation. As Stephen has suggested, it's more like a bug in vpp. If we treat it like that way, we will fix nothing here. >>>> - step 1, send VHOST_USER_SET_FEATURES to confirm the features; >>>> - step 2, send VHOST_USER_SET_MEM_TABLE to share mem regions; >>>> - step 3, send VHOST_USER_SET_VRING_NUM, VHOST_USER_SET_VRING_BASE, >>>> VHOST_USER_SET_VRING_ADDR, VHOST_USER_SET_VRING_KICK for each >>>> queue; >>>> - ... >>>> >>>> Fixes: 37a7eb2ae816 ("net/virtio-user: add device emulation layer") >>>> >>>> Reported-by: Zhihong Wang >>>> Signed-off-by: Jianfeng Tan >>>> --- >>>> drivers/net/virtio/virtio_user/virtio_user_dev.c | 120 ++++++++++++++--------- >>>> 1 file changed, 72 insertions(+), 48 deletions(-) >>> That's too much of code for a bug fix. I'm wondering how about just >>> moving VHOST_USER_GET_PROTOCOL_FEATURES ahead, to the begining of >>> virtio_user_start_device()? It should fix this issue. >> Why does VHOST_USER_GET_PROTOCOL_FEATURES care? Do you mean shifting >> VHOST_USER_SET_FEATURES earlier? > Oops, right, I meant SET_FEATURES. Sorry about confusion introduced by > the silly auto-completion. Still not working. VPP needs SET_VRING_CALL to create vq firstly. Thanks, Jianfeng > > --yliu