From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mailout2.w1.samsung.com (mailout2.w1.samsung.com [210.118.77.12]) by dpdk.org (Postfix) with ESMTP id 43AF91B3D2 for ; Wed, 12 Dec 2018 16:23:30 +0100 (CET) Received: from eucas1p1.samsung.com (unknown [182.198.249.206]) by mailout2.w1.samsung.com (KnoxPortal) with ESMTP id 20181212152329euoutp028d9841a269b20634206d392dda7ef598~vnywlMLv31208112081euoutp02j for ; Wed, 12 Dec 2018 15:23:29 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.w1.samsung.com 20181212152329euoutp028d9841a269b20634206d392dda7ef598~vnywlMLv31208112081euoutp02j DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1544628209; bh=ZOyMgWy3lr5sza/bhDVF/Xn7GLA+FLxUeZiv5FgAG7Q=; h=Subject:To:From:Date:In-Reply-To:References:From; b=r7lcvhJvs2tzxFe2S5RB6rVNwa+EyHswJBMDYHm+X1qao9mUqbc3a5F2Vra7eeyV2 c2kGlzpuhJ/+voB9W86cFKLHps2he/wftzRzElYp8vsN1wyJC4kySaB5aRIQAVnpqi 0ARKs/VIJTXOmGYryDGQqeVzQdMA9g+Nfx36GMnw= Received: from eusmges1new.samsung.com (unknown [203.254.199.242]) by eucas1p2.samsung.com (KnoxPortal) with ESMTP id 20181212152328eucas1p2b7cf52679aa7029a070b72c5903e9533~vnywE33Ai2476024760eucas1p2u; Wed, 12 Dec 2018 15:23:28 +0000 (GMT) Received: from eucas1p2.samsung.com ( [182.198.249.207]) by eusmges1new.samsung.com (EUCPMTA) with SMTP id 94.38.04441.0F7211C5; Wed, 12 Dec 2018 15:23:28 +0000 (GMT) Received: from eusmtrp2.samsung.com (unknown [182.198.249.139]) by eucas1p2.samsung.com (KnoxPortal) with ESMTPA id 20181212152327eucas1p273a69af195ccd6836462890e4f4d3b18~vnyvOroLr2475324753eucas1p2o; Wed, 12 Dec 2018 15:23:27 +0000 (GMT) Received: from eusmgms1.samsung.com (unknown [182.198.249.179]) by eusmtrp2.samsung.com (KnoxPortal) with ESMTP id 20181212152327eusmtrp28283f9c0f8cf59aa106abc481522c0ba~vnyvAKuDH2317023170eusmtrp2R; Wed, 12 Dec 2018 15:23:27 +0000 (GMT) X-AuditID: cbfec7f2-5e3ff70000001159-27-5c1127f08fe3 Received: from eusmtip2.samsung.com ( [203.254.199.222]) by eusmgms1.samsung.com (EUCPMTA) with SMTP id 50.F6.04284.FE7211C5; Wed, 12 Dec 2018 15:23:27 +0000 (GMT) Received: from [106.109.129.180] (unknown [106.109.129.180]) by eusmtip2.samsung.com (KnoxPortal) with ESMTPA id 20181212152326eusmtip2ce739963b9783510c4eeb31e68226dd6~vnyujRVUZ0987209872eusmtip2b; Wed, 12 Dec 2018 15:23:26 +0000 (GMT) To: Maxime Coquelin , dev@dpdk.org, tiwei.bie@intel.com, zhihong.wang@intel.com, jfreimann@redhat.com, mst@redhat.com From: Ilya Maximets Message-ID: Date: Wed, 12 Dec 2018 18:23:22 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1 MIME-Version: 1.0 In-Reply-To: <20181212082403.12002-1-maxime.coquelin@redhat.com> Content-Language: en-GB Content-Transfer-Encoding: 7bit X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrKKsWRmVeSWpSXmKPExsWy7djP87of1AVjDDYsFrV492k7k8WV9p/s FufWLGWxONa5h8Xi/69XrBZbG/4zWWy+OInJgd3j14KlrB6L97xk8ni/7yqbR9+WVYwBLFFc NimpOZllqUX6dglcGQ8e97MVzJOoeHrYqoFxsXAXIyeHhICJxOEVh5m7GLk4hARWMEqcbVjK AuF8YZRoODaXHcL5zCixYNIjJpiWdTNOskIkljNKdK5ezAbhfGSUeHJ2JhtIlbBAuMTEO+cZ QRIiAlMZJbaeWskKkmAT0JE4tfoII4jNK2AncfLgBLAGFgFVievdl5lBbFGBCImO+6vZIGoE JU7OfAJ0FAcHp4CDxPUmfZAws4C4RNMXiJHMAvIS29/OAXtCQmARu8S0t5dZQeolBFwkupsU Ia4Wlnh1fAs7hC0j8X/nfKhv6iXut7xkhOjtYJSYfugfVMJeYsvrc+wgc5gFNCXW79KHCDtK TOvdzwwxnk/ixltBiBP4JCZtmw4V5pXoaBOCqFaR+H1wOTOELSVx891nqAs8JCYu6GOcwKg4 C8mPs5A8NgvJY7MQbljAyLKKUTy1tDg3PbXYMC+1XK84Mbe4NC9dLzk/dxMjMPGc/nf80w7G r5eSDjEKcDAq8fAekBWMEWJNLCuuzD3EKMHBrCTCK2MkECPEm5JYWZValB9fVJqTWnyIUZqD RUmct5rhQbSQQHpiSWp2ampBahFMlomDU6qBMWsRSyxz2n3+7qm+e9KfbvOf0d595UivxVxG i7fc/rs08tMmtSuyZ61/0HxS7cltriktIcmLXG8aR/Ko2UUqSbV9e2f0p2T/tUNNunZb56fz c937XhGuPdlrw9TimOwD9xfxrPAt8pzDysQ0PUv1S7vyvrBrryVl8++F3HQ6dMqnZELTyWNv lViKMxINtZiLihMBJO/LAzgDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrLIsWRmVeSWpSXmKPExsVy+t/xe7rv1QVjDJ78lbJ492k7k8WV9p/s FufWLGWxONa5h8Xi/69XrBZbG/4zWWy+OInJgd3j14KlrB6L97xk8ni/7yqbR9+WVYwBLFF6 NkX5pSWpChn5xSW2StGGFkZ6hpYWekYmlnqGxuaxVkamSvp2NimpOZllqUX6dgl6GQ8e97MV zJOoeHrYqoFxsXAXIyeHhICJxLoZJ1m7GLk4hASWMkqcf3SRBSIhJfHj1wVWCFtY4s+1LjaI oveMEh3zWplAEsIC4RIT75xnBEmICExllPhy/ys7RNVkRol/J2Yyg1SxCehInFp9hBHE5hWw kzh5cAIbiM0ioCpxvfsyWI2oQITE2ZfroGoEJU7OfAJ0BgcHp4CDxPUmfZAws4C6xJ95l5gh bHGJpi8rWSFseYntb+cwT2AUnIWkexaSlllIWmYhaVnAyLKKUSS1tDg3PbfYUK84Mbe4NC9d Lzk/dxMjMKq2Hfu5eQfjpY3BhxgFOBiVeHgzpAVjhFgTy4orcw8xSnAwK4nwyhgJxAjxpiRW VqUW5ccXleakFh9iNAX6bSKzlGhyPjDi80riDU0NzS0sDc2NzY3NLJTEec8bVEYJCaQnlqRm p6YWpBbB9DFxcEo1MOY6xKx5G2gk+qQp9+Cbn1JaVz01eJzV2ht2PZA3nrHke+P/y3eXsNZW piYcviJQdf2Bt9lEIb5YziO/jViXrdxQq/7sOPv92WXHvi/0Ofq35ErRjrvvPMS6HBJvzrZ+ 5REqJWDEUCHQ0W2avIBj/tm46jV3zJcfvFUpriD6apOIfI/U8kMOaUosxRmJhlrMRcWJALVV sfXAAgAA X-CMS-MailID: 20181212152327eucas1p273a69af195ccd6836462890e4f4d3b18 X-Msg-Generator: CA Content-Type: text/plain; charset="utf-8" X-RootMTR: 20181212082505epcas4p36ea3086cfd223aae70a925e946fced48 X-EPHeader: CA CMS-TYPE: 201P X-CMS-RootMailID: 20181212082505epcas4p36ea3086cfd223aae70a925e946fced48 References: <20181212082403.12002-1-maxime.coquelin@redhat.com> Subject: Re: [dpdk-dev] [PATCH] vhost: batch used descriptors chains write-back with packed ring X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Dec 2018 15:23:30 -0000 On 12.12.2018 11:24, Maxime Coquelin wrote: > Instead of writing back descriptors chains in order, let's > write the first chain flags last in order to improve batching. > > With Kernel's pktgen benchmark, ~3% performance gain is measured. > > Signed-off-by: Maxime Coquelin > --- > lib/librte_vhost/virtio_net.c | 39 +++++++++++++++++++++-------------- > 1 file changed, 24 insertions(+), 15 deletions(-) > Hi. I made some rough testing on my ARMv8 system with this patch and v1 of it. Here is the performance difference with current master: v1: +1.1 % v2: -3.6 % So, write barriers are quiet heavy in practice. My testcase is the three instances of testpmd on a same host (with v11 from Jens): txonly (virtio_user0) --> fwd mode io (vhost0, vhost1) --> rxonly (virtio_user1) Best regards, Ilya Maximets. > diff --git a/lib/librte_vhost/virtio_net.c b/lib/librte_vhost/virtio_net.c > index 5e1a1a727..c0b3d1137 100644 > --- a/lib/librte_vhost/virtio_net.c > +++ b/lib/librte_vhost/virtio_net.c > @@ -135,19 +135,10 @@ flush_shadow_used_ring_packed(struct virtio_net *dev, > struct vhost_virtqueue *vq) > { > int i; > - uint16_t used_idx = vq->last_used_idx; > + uint16_t head_flags, head_idx = vq->last_used_idx; > > - /* Split loop in two to save memory barriers */ > - for (i = 0; i < vq->shadow_used_idx; i++) { > - vq->desc_packed[used_idx].id = vq->shadow_used_packed[i].id; > - vq->desc_packed[used_idx].len = vq->shadow_used_packed[i].len; > - > - used_idx += vq->shadow_used_packed[i].count; > - if (used_idx >= vq->size) > - used_idx -= vq->size; > - } > - > - rte_smp_wmb(); > + if (unlikely(vq->shadow_used_idx == 0)) > + return; > > for (i = 0; i < vq->shadow_used_idx; i++) { > uint16_t flags; > @@ -165,12 +156,24 @@ flush_shadow_used_ring_packed(struct virtio_net *dev, > flags &= ~VRING_DESC_F_AVAIL; > } > > - vq->desc_packed[vq->last_used_idx].flags = flags; > + vq->desc_packed[vq->last_used_idx].id = > + vq->shadow_used_packed[i].id; > + vq->desc_packed[vq->last_used_idx].len = > + vq->shadow_used_packed[i].len; > + > + rte_smp_wmb(); > > - vhost_log_cache_used_vring(dev, vq, > + if (i > 0) { > + vq->desc_packed[vq->last_used_idx].flags = flags; > + > + vhost_log_cache_used_vring(dev, vq, > vq->last_used_idx * > sizeof(struct vring_packed_desc), > sizeof(struct vring_packed_desc)); > + } else { > + head_idx = vq->last_used_idx; > + head_flags = flags; > + } > > vq->last_used_idx += vq->shadow_used_packed[i].count; > if (vq->last_used_idx >= vq->size) { > @@ -179,8 +182,14 @@ flush_shadow_used_ring_packed(struct virtio_net *dev, > } > } > > - rte_smp_wmb(); > + vq->desc_packed[head_idx].flags = head_flags; > vq->shadow_used_idx = 0; > + > + vhost_log_cache_used_vring(dev, vq, > + head_idx * > + sizeof(struct vring_packed_desc), > + sizeof(struct vring_packed_desc)); > + > vhost_log_cache_sync(dev, vq); > } > >