From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from szxga03-in.huawei.com (szxga03-in.huawei.com [119.145.14.66]) by dpdk.org (Postfix) with ESMTP id C24065A87 for ; Wed, 10 Jun 2015 04:37:06 +0200 (CEST) Received: from 172.24.2.119 (EHLO szxeml431-hub.china.huawei.com) ([172.24.2.119]) by szxrg03-dlp.huawei.com (MOS 4.4.3-GA FastPath queued) with ESMTP id BHL49098; Wed, 10 Jun 2015 10:36:53 +0800 (CST) Received: from [127.0.0.1] (10.177.19.115) by szxeml431-hub.china.huawei.com (10.82.67.208) with Microsoft SMTP Server id 14.3.158.1; Wed, 10 Jun 2015 10:36:50 +0800 Message-ID: <5577A2C0.3060609@huawei.com> Date: Wed, 10 Jun 2015 10:36:48 +0800 From: Linhaifeng User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:31.0) Gecko/20100101 Thunderbird/31.1.0 MIME-Version: 1.0 To: "Xie, Huawei" , "Michael S. Tsirkin" References: <1429720392-25345-1-git-send-email-huawei.xie@intel.com> <553995DB.4000801@huawei.com> <55768FE2.5060505@huawei.com> <20150609084613.GA18121@redhat.com> In-Reply-To: Content-Type: text/plain; charset="windows-1252" Content-Transfer-Encoding: 7bit X-Originating-IP: [10.177.19.115] X-CFilter-Loop: Reflected X-Mirapoint-Virus-RAPID-Raw: score=unknown(0), refid=str=0001.0A020203.5577A2C6.0035, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0, ip=0.0.0.0, so=2013-05-26 15:14:31, dmn=2013-03-21 17:37:32 X-Mirapoint-Loop-Id: c61f996eee8d2ada3ae1b664901a535e Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] [PATCH] vhost: flush used->idx update before reading avail->flags X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Jun 2015 02:37:08 -0000 On 2015/6/9 21:34, Xie, Huawei wrote: > On 6/9/2015 4:47 PM, Michael S. Tsirkin wrote: >> On Tue, Jun 09, 2015 at 03:04:02PM +0800, Linhaifeng wrote: >>> >>> On 2015/4/24 15:27, Luke Gorrie wrote: >>>> On 24 April 2015 at 03:01, Linhaifeng wrote: >>>> >>>>> If not add memory fence what would happen? Packets loss or interrupt >>>>> loss?How to test it ? >>>>> >>>> You should be able to test it like this: >>>> >>>> 1. Boot two Linux kernel (e.g. 3.13) guests. >>>> 2. Connect them via vhost switch. >>>> 3. Run continuous traffic between them (e.g. iperf). >>>> >>>> I would expect that within a reasonable timeframe (< 1 hour) one of the >>>> guests' network interfaces will hang indefinitely due to a missed interrupt. >>>> >>>> You won't be able to reproduce this using DPDK guests because they are not >>>> using the same interrupt suppression method. >>>> >>>> This is a serious real-world problem. I wouldn't deploy the vhost >>>> implementation without this fix. >>>> >>>> Cheers, >>>> -Luke >>>> >>> I think this patch can't resole this problem. On the other hand we still would miss interrupt. >>> >>> After add rte_mb() function the we want the case is : >>> 1.write used->idx. ring is full or empty. >>> 2.virtio_net open interrupt. >>> 3.read avail->flags. >>> >>> but this case(miss interrupt) would happen too: >>> 1.write used->idx. ring is full or empty. >>> 2.read avail->flags. >>> 3.virtio_net open interrupt. >>> >> That's why a correct guest, after detecting an empty used ring, must always >> re-check used idx at least once after writing avail->flags. >> >> By the way, similarly, host side must re-check avail idx after writing >> used flags. I don't see where snabbswitch does it - is that a bug >> in snabbswitch? >> > yes, both host and guest should recheck if there is more work added > after they toggle the flag. > For DPDK vHost, as it runs in polling mode, we will recheck avail idx > soon, so we don't need recheck. > > DPDK does check the avail idx but does nothing like this: if (vq->last_used_idx == avail_idx) { return; } If we miss an interrupt after calling rte_mb(), (!(vq->avail->flags & VRING_AVAIL_F_NO_INTERRUPT)) is False; while (vq->last_used_idx == avail_idx) is True, then the guest will miss the interrupt forever and virtio-net would stop working. Would this case happen?