From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mailout2.w1.samsung.com (mailout2.w1.samsung.com [210.118.77.12]) by dpdk.org (Postfix) with ESMTP id 3D1C21B137 for ; Wed, 5 Dec 2018 12:30:44 +0100 (CET) Received: from eucas1p1.samsung.com (unknown [182.198.249.206]) by mailout2.w1.samsung.com (KnoxPortal) with ESMTP id 20181205113043euoutp02c23dce57137f8d17b38168f704b7625f~tbGh1oL8X1435714357euoutp02b for ; Wed, 5 Dec 2018 11:30:43 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.w1.samsung.com 20181205113043euoutp02c23dce57137f8d17b38168f704b7625f~tbGh1oL8X1435714357euoutp02b DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1544009443; bh=GpdmQXl5BMbe4tq/BT7mPbjuliUx+2ohpIArtcdb+AU=; h=Subject:To:Cc:From:Date:In-Reply-To:References:From; b=SBpHrfA2pTT5zjj4qd9321mRxjjopXmSMXhFYYRHIY3EEf50LjifwHMj3TAD2ej+7 8jUSgujrwB+vZaEMPbm3/jR4DnlzCUMJzwOhb18eqiuYBn9uvhtoCPyDGF+TjhK6S4 e8Us0LULfu/mnhGsbHd8RMMvi212LigcOa+YUxg4= Received: from eusmges3new.samsung.com (unknown [203.254.199.245]) by eucas1p2.samsung.com (KnoxPortal) with ESMTP id 20181205113042eucas1p2130b486dbe0a1617923d2c6185ab32b1~tbGhX2yZg2027220272eucas1p2J; Wed, 5 Dec 2018 11:30:42 +0000 (GMT) Received: from eucas1p2.samsung.com ( [182.198.249.207]) by eusmges3new.samsung.com (EUCPMTA) with SMTP id FC.1F.04806.2E6B70C5; Wed, 5 Dec 2018 11:30:42 +0000 (GMT) Received: from eusmtrp1.samsung.com (unknown [182.198.249.138]) by eucas1p1.samsung.com (KnoxPortal) with ESMTPA id 20181205113041eucas1p1943b9c13af2fb5b736ba4906b59a9cd5~tbGgnGJWB0542605426eucas1p1T; Wed, 5 Dec 2018 11:30:41 +0000 (GMT) Received: from eusmgms1.samsung.com (unknown [182.198.249.179]) by eusmtrp1.samsung.com (KnoxPortal) with ESMTP id 20181205113041eusmtrp1dfabcfde90660b95c2e303c83aaffd46~tbGgYZQXB3057230572eusmtrp1Y; Wed, 5 Dec 2018 11:30:41 +0000 (GMT) X-AuditID: cbfec7f5-367ff700000012c6-0e-5c07b6e24f69 Received: from eusmtip1.samsung.com ( [203.254.199.221]) by eusmgms1.samsung.com (EUCPMTA) with SMTP id 7C.70.04284.1E6B70C5; Wed, 5 Dec 2018 11:30:41 +0000 (GMT) Received: from [106.109.129.180] (unknown [106.109.129.180]) by eusmtip1.samsung.com (KnoxPortal) with ESMTPA id 20181205113041eusmtip1734fc940525a364ad33407a75a96ce2f~tbGf24umn2896028960eusmtip1h; Wed, 5 Dec 2018 11:30:40 +0000 (GMT) To: Maxime Coquelin , dev@dpdk.org, jfreimann@redhat.com, tiwei.bie@intel.com, zhihong.wang@intel.com, jasowang@redhat.com Cc: stable@dpdk.org From: Ilya Maximets Message-ID: Date: Wed, 5 Dec 2018 14:30:36 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1 MIME-Version: 1.0 In-Reply-To: <20181205094957.1938-2-maxime.coquelin@redhat.com> Content-Language: en-GB Content-Transfer-Encoding: 7bit X-Brightmail-Tracker: H4sIAAAAAAAAA02Sa0hTYRjHe8852zmOjpxtDh80iJYURs0sg0OIl7AYRdCHIkkjT3pQ06lt aplQZqVrqXgJZKtEyXlbEHmZoeZlmUsHRuKlCFSi8oaGt8LSWfMY+e33PP/n9n95KVxmEXlR 8UmpvDaJS1SKJYS1Z+Xdgc9WMvJg2wMFO7fQjLGDuSskWzWwiLH9z8wE23O/jWCd+lWSbcpa x9iG98VYCKX+VW4WqZ+2TWHq7+1DYnVBYx06Q1yQBMbwifHpvNYvKEoS92naiFJW5dfz5h3i LJTDGJAbBUwAlOgNuAFJKBlTg8A5PIkJwRKCe5NjpBAsIiibX0AGRG20OL5t5qsR5BkfEUIw j6BvfRl3zZUzYTA6+nNjrgdjQpA9YxS7BJyRQ273MOFiMbMf+izdyMU0EwRVneMbTDA+MFPq WuFGKZhw0I9ZxEKNFHqNXzZ63ZhguPN2AgkzPSF7qVYk8E5onn2MC+asJEz8UAscBoWvLaTA cpi2N27yDnCU5BEC34Kxu1PIdTQwegSlNicmCMHQONNPuuzjjC88b/ET0qGwkvMEE17FHT7M SoUT3KHYWooLaRr0OTKh2gd+d1VvXuYFH+cWyUKkNG0xZtpixrTFjOn/3nJE1CFPPk2nieV1 h5P4ayodp9GlJcWqopM19ejvF3I47csvUfvqZRtiKKTcTsNDcaRMxKXrMjQ2BBSu9KBrAslI GR3DZdzgtcmXtGmJvM6GvClC6UlnbhuPkDGxXCqfwPMpvPafilFuXlkoOrWzqKDCRqdPRhny jzeUvQiM7pL2nj45cp52Jl91DAWcyCWaWusqzUW5Fa+4hN17rSGhV5Jba46cLeqvzG+oLTDc 9rh4KGZtj3HgXLA9fC1T5e59NEIurT21rdtfoapeP1Yf2R7WZskzKzriiu1vfB1dLSM3pRMd XbvGqwb9vioJXRznvw/X6rg/wKAXrT4DAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrLIsWRmVeSWpSXmKPExsVy+t/xu7oPt7HHGEybqGDx7tN2Josr7T/Z LZZd+sxkcW7NUhaLY517WCz+dfxht9ja8J/JYvPFSUwOHB6/Fixl9Vi85yWTx/t9V9k8+ras YgxgidKzKcovLUlVyMgvLrFVija0MNIztLTQMzKx1DM0No+1MjJV0rezSUnNySxLLdK3S9DL uP1qJmPBH+GKno+n2RoY2wS6GDk4JARMJE4/Y+9i5OIQEljKKLG/czpjFyMnUFxK4sevC6wQ trDEn2tdbBBF7xklZp+fwwySEBZwkbh37zszSEJEYBajxMoTIFWcHMxAHe1HrrGAbBASsJNo 3aEOEmYT0JE4tfoI2AJeoPCyAw/AbBYBFYnX00Gu4OQQFYiQOPtyHVSNoMTJmU9YQGxOAXuJ 5hPPGSHGq0v8mXeJGcIWl2j6spIVwpaX2P52DvMERqFZSNpnIWmZhaRlFpKWBYwsqxhFUkuL c9Nziw31ihNzi0vz0vWS83M3MQKjbtuxn5t3MF7aGHyIUYCDUYmHV3IKW4wQa2JZcWXuIUYJ DmYlEd4VNuwxQrwpiZVVqUX58UWlOanFhxhNgZ6byCwlmpwPTAh5JfGGpobmFpaG5sbmxmYW SuK85w0qo4QE0hNLUrNTUwtSi2D6mDg4pRoYTdjLv1UHBbvvyDCLig7V2bu37OoJ95+msg5n z280ONr6XHd+xeK648muP8RueE/qsI8/7M+TU3H/wb75Je1zbC7Pk9sWzvxWQ8Z1/xOmGbut Lk7oajmQ8lS778GcXe+0QqeemsF25t7k0j0pYSo2vWuXyzox79T4t0UsxMKtykVZUPrTbL8/ SizFGYmGWsxFxYkA2Njcd9ACAAA= X-CMS-MailID: 20181205113041eucas1p1943b9c13af2fb5b736ba4906b59a9cd5 X-Msg-Generator: CA Content-Type: text/plain; charset="utf-8" X-RootMTR: 20181205113041eucas1p1943b9c13af2fb5b736ba4906b59a9cd5 X-EPHeader: CA CMS-TYPE: 201P X-CMS-RootMailID: 20181205113041eucas1p1943b9c13af2fb5b736ba4906b59a9cd5 References: <20181205094957.1938-2-maxime.coquelin@redhat.com> Subject: Re: [dpdk-dev] [1/5] vhost: enforce avail index and desc read ordering X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Dec 2018 11:30:44 -0000 On 05.12.2018 12:49, Maxime Coquelin wrote: > A read barrier is required to ensure the ordering between > available index and the descriptor reads is enforced. > > Fixes: 4796ad63ba1f ("examples/vhost: import userspace vhost application") > Cc: stable@dpdk.org > > Reported-by: Jason Wang > Signed-off-by: Maxime Coquelin > --- > lib/librte_vhost/virtio_net.c | 12 ++++++++++++ > 1 file changed, 12 insertions(+) > > diff --git a/lib/librte_vhost/virtio_net.c b/lib/librte_vhost/virtio_net.c > index 5e1a1a727..f11ebb54f 100644 > --- a/lib/librte_vhost/virtio_net.c > +++ b/lib/librte_vhost/virtio_net.c > @@ -791,6 +791,12 @@ virtio_dev_rx_split(struct virtio_net *dev, struct vhost_virtqueue *vq, > rte_prefetch0(&vq->avail->ring[vq->last_avail_idx & (vq->size - 1)]); > avail_head = *((volatile uint16_t *)&vq->avail->idx); > > + /* > + * The ordering between avail index and > + * desc reads needs to be enforced. > + */ > + rte_smp_rmb(); > + Hmm. This looks weird to me. Could you please describe the bad scenario here? (It'll be good to have it in commit message too) As I understand, you're enforcing the read of avail->idx to happen before reading the avail->ring[avail_idx]. Is it correct? But we have following code sequence: 1. read avail->idx (avail_head). 2. check that last_avail_idx != avail_head. 3. read from the ring using last_avail_idx. So, there is a strict dependency between all 3 steps and the memory transaction will be finished at the step #2 in any case. There is no way to read the ring before reading the avail->idx. Am I missing something? > for (pkt_idx = 0; pkt_idx < count; pkt_idx++) { > uint32_t pkt_len = pkts[pkt_idx]->pkt_len + dev->vhost_hlen; > uint16_t nr_vec = 0; > @@ -1373,6 +1379,12 @@ virtio_dev_tx_split(struct virtio_net *dev, struct vhost_virtqueue *vq, > if (free_entries == 0) > return 0; > > + /* > + * The ordering between avail index and > + * desc reads needs to be enforced. > + */ > + rte_smp_rmb(); > + This one is strange too. free_entries = *((volatile uint16_t *)&vq->avail->idx) - vq->last_avail_idx; if (free_entries == 0) return 0; The code reads the value of avail->idx and uses the value on the next line even with any compiler optimizations. There is no way for CPU to postpone the actual read. > VHOST_LOG_DEBUG(VHOST_DATA, "(%d) %s\n", dev->vid, __func__); > > count = RTE_MIN(count, MAX_PKT_BURST); >