From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mailout1.w1.samsung.com (mailout1.w1.samsung.com [210.118.77.11]) by dpdk.org (Postfix) with ESMTP id 9FE621B122 for ; Wed, 5 Dec 2018 17:01:26 +0100 (CET) Received: from eucas1p1.samsung.com (unknown [182.198.249.206]) by mailout1.w1.samsung.com (KnoxPortal) with ESMTP id 20181205160125euoutp018f1d0cbb20184d4befc5fcda8084dd6d~tey400-341482014820euoutp01A for ; Wed, 5 Dec 2018 16:01:25 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout1.w1.samsung.com 20181205160125euoutp018f1d0cbb20184d4befc5fcda8084dd6d~tey400-341482014820euoutp01A DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1544025685; bh=phY+bLddNhRE7/O/Knt5EqJ9QAX0COx+1x+Ju/0t7Uw=; h=Subject:To:From:Cc:Date:In-Reply-To:References:From; b=tXp521U57JolSU7cX20IXdFGt9byIxuiAamPnmXo3yt2kIPdq1GljdQE0hsjxg/t7 IyJ2nWMiz05NtDTB95D0Z4he4ZiYnNodhd0pP8RWp1OdU3IShfQ0p79eDupLnShu2b /ytBAdWM64/CVb6RdVPUGJI+SKOXpqJDHdGSEB0w= Received: from eusmges3new.samsung.com (unknown [203.254.199.245]) by eucas1p2.samsung.com (KnoxPortal) with ESMTP id 20181205160125eucas1p224569f022813cd6d7aad63a701b9a649~tey4YplpH2605126051eucas1p23; Wed, 5 Dec 2018 16:01:25 +0000 (GMT) Received: from eucas1p2.samsung.com ( [182.198.249.207]) by eusmges3new.samsung.com (EUCPMTA) with SMTP id 7B.37.04806.456F70C5; Wed, 5 Dec 2018 16:01:24 +0000 (GMT) Received: from eusmtrp2.samsung.com (unknown [182.198.249.139]) by eucas1p1.samsung.com (KnoxPortal) with ESMTPA id 20181205160124eucas1p1470e3dc9afe8e59ceab54a58140cf400~tey3pvSfk2135821358eucas1p11; Wed, 5 Dec 2018 16:01:24 +0000 (GMT) Received: from eusmgms1.samsung.com (unknown [182.198.249.179]) by eusmtrp2.samsung.com (KnoxPortal) with ESMTP id 20181205160124eusmtrp25047b842c04f7c870fb1ecec5680efce~tey3bVGcI1036310363eusmtrp2f; Wed, 5 Dec 2018 16:01:24 +0000 (GMT) X-AuditID: cbfec7f5-367ff700000012c6-e8-5c07f654dee0 Received: from eusmtip2.samsung.com ( [203.254.199.222]) by eusmgms1.samsung.com (EUCPMTA) with SMTP id 0E.36.04284.456F70C5; Wed, 5 Dec 2018 16:01:24 +0000 (GMT) Received: from [106.109.129.180] (unknown [106.109.129.180]) by eusmtip2.samsung.com (KnoxPortal) with ESMTPA id 20181205160123eusmtip262ea23ea6a661e5668d460ed31b58835~tey246bOc0683506835eusmtip2v; Wed, 5 Dec 2018 16:01:23 +0000 (GMT) To: Maxime Coquelin , dev@dpdk.org, tiwei.bie@intel.com, zhihong.wang@intel.com, jfreimann@redhat.com From: Ilya Maximets Cc: "Michael S. Tsirkin" Message-ID: <7fbcfcea-3c81-d5d1-86bf-8fe8e63d4468@samsung.com> Date: Wed, 5 Dec 2018 19:01:23 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1 MIME-Version: 1.0 In-Reply-To: <20181128094700.14598-1-maxime.coquelin@redhat.com> Content-Language: en-GB Content-Transfer-Encoding: 7bit X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrMKsWRmVeSWpSXmKPExsWy7djP87oh39hjDJ7/M7d492k7k8WV9p/s FufWLGWxONa5h8Xi/69XrBZbG/4zWWy+OInJgd3j14KlrB6L97xk8ni/7yqbR9+WVYwBLFFc NimpOZllqUX6dglcGUtf+BRclqy48XYdawPjRZEuRk4OCQETiX/n57B0MXJxCAmsYJTY8+cz E4TzhVFi0vezUM5nRomVe6axwrQ0NSyAalnOKLHtzyywhJDAR0aJ25P4uhg5OIQF/CUufg0G CYsINDNK3NzAAmKzCehInFp9hBGkhFlAU+L+NlWQMK+AncSvtzeYQGwWARWJRb+ngZWLCkRI dNxfzQZRIyhxcuYTsDingIPE+f1djCA2s4C4RNOXlawQtrzE9rdzmEFOkxBYxy7x7eUONoib XSS+L37JDmELS7w6vgXKlpH4v3M+E4RdL3G/5SUjRHMHo8T0Q/+gEvYSW16fY4c5ev0ufYiw o8Sbq/tYQMISAnwSN94KQtzAJzFp23RmiDCvREebEES1isTvg8uZIWwpiZvvPrNPYFSaheSz WUi+mYXkm1kIexcwsqxiFE8tLc5NTy02zkst1ytOzC0uzUvXS87P3cQITDqn/x3/uoNx35+k Q4wCHIxKPLwvHrDHCLEmlhVX5h5ilOBgVhLhXWEDFOJNSaysSi3Kjy8qzUktPsQozcGiJM5b zfAgWkggPbEkNTs1tSC1CCbLxMEp1cB4pW+aw5+1ry/9emTpfeaVrmgqz6bwrwJFlZLdoW0N OSs4D3N1PFnZkZ2fuE2xhsd2n1y0G0/bhynxVnEZq6adZDD6Om3JBImu+2bHBdJnzOte9ev0 3jWVb+5krt3Z1FSn1n9tdSaXxHqFyjPzTbVnCssyNZp2isVU8AkyNOwsNCk89X3hy4VKLMUZ iYZazEXFiQCTGTqONgMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrBIsWRmVeSWpSXmKPExsVy+t/xe7oh39hjDL4tZbd492k7k8WV9p/s FufWLGWxONa5h8Xi/69XrBZbG/4zWWy+OInJgd3j14KlrB6L97xk8ni/7yqbR9+WVYwBLFF6 NkX5pSWpChn5xSW2StGGFkZ6hpYWekYmlnqGxuaxVkamSvp2NimpOZllqUX6dgl6GUtf+BRc lqy48XYdawPjRZEuRk4OCQETiaaGBSxdjFwcQgJLGSXeXzjADJGQkvjx6wIrhC0s8edaFxtE 0XtGiV9Pz7CAJIQFfCV6u5eAdYsINDNKXDs2ixEkISRgL3H7XANYN5uAjsSp1UeA4hwczAKa Eve3qYKEeQXsJH69vcEEYrMIqEgs+j0NbKaoQITE2ZfrGCFqBCVOznwCFucUcJA4v78LLM4s oC7xZ94lZghbXKLpy0pWCFteYvvbOcwTGIVmIWmfhaRlFpKWWUhaFjCyrGIUSS0tzk3PLTbU K07MLS7NS9dLzs/dxAiMtW3Hfm7ewXhpY/AhRgEORiUe3hcP2GOEWBPLiitzDzFKcDArifCu sAEK8aYkVlalFuXHF5XmpBYfYjQFem4is5Rocj4wDeSVxBuaGppbWBqaG5sbm1koifOeN6iM EhJITyxJzU5NLUgtgulj4uCUamCMjkmf2F2cvHviltprN/syX2f2bGbuNt70+6zFzr/PZGfk 8vsbLnpu8IPB4b5tm9onJga26Yt0W18yLDj0NWlH7QLR2zoyxb4Fv7XaJAR/xNpeP/PUY/fK w0/+PJ28tl+uiPe8qrLEySaNhZY2gQybpXXfHSh/xb/ikeLnDK0IlXW6D34fNM9XYinOSDTU Yi4qTgQADBjXLcsCAAA= X-CMS-MailID: 20181205160124eucas1p1470e3dc9afe8e59ceab54a58140cf400 X-Msg-Generator: CA Content-Type: text/plain; charset="utf-8" X-RootMTR: 20181205160124eucas1p1470e3dc9afe8e59ceab54a58140cf400 X-EPHeader: CA CMS-TYPE: 201P X-CMS-RootMailID: 20181205160124eucas1p1470e3dc9afe8e59ceab54a58140cf400 References: <20181128094700.14598-1-maxime.coquelin@redhat.com> Subject: Re: [dpdk-dev] vhost: batch used descriptors chains write-back with packed ring X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Dec 2018 16:01:26 -0000 On 28.11.2018 12:47, Maxime Coquelin wrote: > Instead of writing back descriptors chains in order, let's > write the first chain flags last in order to improve batching. I'm not sure if this fully compliant with virtio spec. It says that 'each side (driver and device) are only required to poll (or test) a single location in memory', but it does not forbid to test other descriptors. So, if the driver will try to check not only 'the next device descriptor after the one they processed previously, in circular order' but a few descriptors ahead, it could read an inconsistent memory because there are no more write barriers between updates for flags and id/len for them. What do you think ? > > With Kernel's pktgen benchmark, ~3% performance gain is measured. > > Signed-off-by: Maxime Coquelin > Tested-by: Jens Freimann > Reviewed-by: Jens Freimann > --- > lib/librte_vhost/virtio_net.c | 37 ++++++++++++++++++++++------------- > 1 file changed, 23 insertions(+), 14 deletions(-) > > diff --git a/lib/librte_vhost/virtio_net.c b/lib/librte_vhost/virtio_net.c > index 5e1a1a727..f54642c2d 100644 > --- a/lib/librte_vhost/virtio_net.c > +++ b/lib/librte_vhost/virtio_net.c > @@ -135,19 +135,10 @@ flush_shadow_used_ring_packed(struct virtio_net *dev, > struct vhost_virtqueue *vq) > { > int i; > - uint16_t used_idx = vq->last_used_idx; > + uint16_t head_flags, head_idx = vq->last_used_idx; > > - /* Split loop in two to save memory barriers */ > - for (i = 0; i < vq->shadow_used_idx; i++) { > - vq->desc_packed[used_idx].id = vq->shadow_used_packed[i].id; > - vq->desc_packed[used_idx].len = vq->shadow_used_packed[i].len; > - > - used_idx += vq->shadow_used_packed[i].count; > - if (used_idx >= vq->size) > - used_idx -= vq->size; > - } > - > - rte_smp_wmb(); > + if (unlikely(vq->shadow_used_idx == 0)) > + return; > > for (i = 0; i < vq->shadow_used_idx; i++) { > uint16_t flags; > @@ -165,12 +156,22 @@ flush_shadow_used_ring_packed(struct virtio_net *dev, > flags &= ~VRING_DESC_F_AVAIL; > } > > - vq->desc_packed[vq->last_used_idx].flags = flags; > + vq->desc_packed[vq->last_used_idx].id = > + vq->shadow_used_packed[i].id; > + vq->desc_packed[vq->last_used_idx].len = > + vq->shadow_used_packed[i].len; > + > + if (i > 0) { > + vq->desc_packed[vq->last_used_idx].flags = flags; > > - vhost_log_cache_used_vring(dev, vq, > + vhost_log_cache_used_vring(dev, vq, > vq->last_used_idx * > sizeof(struct vring_packed_desc), > sizeof(struct vring_packed_desc)); > + } else { > + head_idx = vq->last_used_idx; > + head_flags = flags; > + } > > vq->last_used_idx += vq->shadow_used_packed[i].count; > if (vq->last_used_idx >= vq->size) { > @@ -180,7 +181,15 @@ flush_shadow_used_ring_packed(struct virtio_net *dev, > } > > rte_smp_wmb(); > + > + vq->desc_packed[head_idx].flags = head_flags; > vq->shadow_used_idx = 0; > + > + vhost_log_cache_used_vring(dev, vq, > + head_idx * > + sizeof(struct vring_packed_desc), > + sizeof(struct vring_packed_desc)); > + > vhost_log_cache_sync(dev, vq); > } > >