From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 65107A2EFC for ; Thu, 19 Sep 2019 06:04:31 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 199E31E8AC; Thu, 19 Sep 2019 06:04:30 +0200 (CEST) Received: from EUR02-VE1-obe.outbound.protection.outlook.com (mail-eopbgr20056.outbound.protection.outlook.com [40.107.2.56]) by dpdk.org (Postfix) with ESMTP id E6F641E8AA for ; Thu, 19 Sep 2019 06:04:28 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=C5On7CNsKghtTB0360OY1gyvAtqgYJKV/VrxdsnDfZM=; b=GxL9GoaqGafZ4FXo0aDoKvCqsaLtj7eEWrfWSmTPa3ECcbrF2YQA55EhY7qZ32z+DWsMJU1BmBmnkcrK8ROqIZRgoGWvCnjo3vg97Q2T8+GRKxOGNwQcQkSJuic9DnbgZPygvmVbev+iFx4fqmgDli3FHAlFc7jpDEGxJgJyn7k= Received: from VI1PR0802CA0030.eurprd08.prod.outlook.com (2603:10a6:800:a9::16) by VI1PR08MB5502.eurprd08.prod.outlook.com (2603:10a6:803:138::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2284.20; Thu, 19 Sep 2019 04:04:26 +0000 Received: from AM5EUR03FT052.eop-EUR03.prod.protection.outlook.com (2a01:111:f400:7e08::209) by VI1PR0802CA0030.outlook.office365.com (2603:10a6:800:a9::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2284.18 via Frontend Transport; Thu, 19 Sep 2019 04:04:26 +0000 Authentication-Results: spf=temperror (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=none action=none header.from=arm.com; Received-SPF: TempError (protection.outlook.com: error in processing during lookup of arm.com: DNS Timeout) Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM5EUR03FT052.mail.protection.outlook.com (10.152.17.161) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2284.20 via Frontend Transport; Thu, 19 Sep 2019 04:04:24 +0000 Received: ("Tessian outbound d5a1f2820a4f:v31"); Thu, 19 Sep 2019 04:04:22 +0000 X-CR-MTA-TID: 64aa7808 Received: from 64091f23a0a6.1 (ip-172-16-0-2.eu-west-1.compute.internal [104.47.6.52]) by 64aa7808-outbound-1.mta.getcheckrecipient.com id 547CB36A-13FC-48F0-A4C0-38913A5EC50E.1; Thu, 19 Sep 2019 04:04:17 +0000 Received: from EUR02-VE1-obe.outbound.protection.outlook.com (mail-ve1eur02lp2052.outbound.protection.outlook.com [104.47.6.52]) by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 64091f23a0a6.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384); Thu, 19 Sep 2019 04:04:17 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LcCIVPYUxaFPhvkVuUgeiRd2WUS9r8cO3o9pXy5wUhKlaQlN34EoG0iNSmPh98DN9ChUyyJ29bw28nH0BVVwTgs52LqJX2RqNRAQ2nZaZA17PeSGoZXkx15dIW9LTyQ2n5X9xnFutGi7MHuHHNSSG6FBCJL4hHaZo5c821SybVzGT6aMsriFylm3JaJh5KQmQj9/jaVnySP2bUOfe9fM96YgAk/gMB01Kl4B4CCZLYWxFBGHaGWFxKlhaiPnOqpbl3DPKhphhPDfXLMoT7Ooo5npCRGrK5PKc1LU+DGnVE6mjaz1dZtlc4YLyAm7RX2/lQmp1TYAzBfBH/l72SEYvw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=C5On7CNsKghtTB0360OY1gyvAtqgYJKV/VrxdsnDfZM=; b=g/MK43knuGDya0Kcs7QmybXbnt0+JHZ6N2R1dhSVyf3BF9t7vDmCvYWn4aTWIuN90Mcs6EbcC9FX6tNLa20wzMHVxYSUAhyYBTvsra3h4EirBuIR1FTYB5rsMCdoNJXmnGJRs+o7X9LnHvV79Q54u1OUTQZ0N+N6UEbzcf4Cpa7KNEpfMkLPnhNxQi8ttANjC8wgOVRFzx+pZMaxsIP4Z7J120ZgoOozWQvOj5def8jnG2XPkTQZ1khrk2Fl70F3sMK/Ve1Z4PCuXgZfZFxnQRwWhio8xmxp1RmyzWN+7z6d5Rdwc3gL2MTX77TTYc8yJ/hNgrf1YquGnPhivcq+BQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=C5On7CNsKghtTB0360OY1gyvAtqgYJKV/VrxdsnDfZM=; b=GxL9GoaqGafZ4FXo0aDoKvCqsaLtj7eEWrfWSmTPa3ECcbrF2YQA55EhY7qZ32z+DWsMJU1BmBmnkcrK8ROqIZRgoGWvCnjo3vg97Q2T8+GRKxOGNwQcQkSJuic9DnbgZPygvmVbev+iFx4fqmgDli3FHAlFc7jpDEGxJgJyn7k= Received: from VI1PR08MB5376.eurprd08.prod.outlook.com (52.133.244.200) by VI1PR08MB4014.eurprd08.prod.outlook.com (20.178.126.91) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2284.18; Thu, 19 Sep 2019 04:04:15 +0000 Received: from VI1PR08MB5376.eurprd08.prod.outlook.com ([fe80::c09a:49d3:eab2:f707]) by VI1PR08MB5376.eurprd08.prod.outlook.com ([fe80::c09a:49d3:eab2:f707%3]) with mapi id 15.20.2284.009; Thu, 19 Sep 2019 04:04:15 +0000 From: "Gavin Hu (Arm Technology China)" To: "Wang, Yinan" , "Joyce Kong (Arm Technology China)" , "dev@dpdk.org" CC: nd , "maxime.coquelin@redhat.com" , "Richardson, Bruce" , "Bie, Tiwei" , "Wang, Zhihong" , "amorenoz@redhat.com" , "Wang, Xiao W" , "Liu, Yong" , "jfreimann@redhat.com" , Honnappa Nagarahalli Thread-Topic: [PATCH v4 2/2] virtio: one way barrier for packed vring desc used flags Thread-Index: AQHVbRjbnJQF4qm6zUePALMNP9uREKcw53IAgAF7lmA= Date: Thu, 19 Sep 2019 04:04:15 +0000 Message-ID: References: <1566893979-3290-1-git-send-email-joyce.kong@arm.com> <1568698107-36668-3-git-send-email-joyce.kong@arm.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: 3ee32d10-9041-4b91-a4c8-53d417d741d2.0 x-checkrecipientchecked: true Authentication-Results-Original: spf=none (sender IP is ) smtp.mailfrom=Gavin.Hu@arm.com; x-originating-ip: [113.29.88.7] x-ms-publictraffictype: Email X-MS-Office365-Filtering-Correlation-Id: cb46d520-f875-417b-7e94-08d73cb67556 X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam-Untrusted: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(7168020)(4627221)(201703031133081)(201702281549075)(8990200)(5600167)(711020)(4605104)(1401327)(4618075)(2017052603328)(7193020); SRVR:VI1PR08MB4014; X-MS-TrafficTypeDiagnostic: VI1PR08MB4014:|VI1PR08MB4014:|VI1PR08MB5502: x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true x-ms-oob-tlc-oobclassifiers: OLM:4502;OLM:4502; x-forefront-prvs: 016572D96D X-Forefront-Antispam-Report-Untrusted: SFV:NSPM; SFS:(10009020)(4636009)(376002)(366004)(396003)(39860400002)(346002)(136003)(199004)(189003)(13464003)(9686003)(8936002)(74316002)(486006)(55016002)(6506007)(476003)(256004)(52536014)(14454004)(25786009)(86362001)(229853002)(3846002)(6116002)(66556008)(6436002)(76116006)(81156014)(81166006)(8676002)(66476007)(66446008)(64756008)(446003)(66946007)(33656002)(6246003)(186003)(7736002)(7696005)(26005)(102836004)(55236004)(11346002)(14444005)(76176011)(4326008)(478600001)(53546011)(316002)(110136005)(54906003)(305945005)(71190400001)(71200400001)(7416002)(66066001)(99286004)(2906002)(2501003)(5660300002); DIR:OUT; SFP:1101; SCL:1; SRVR:VI1PR08MB4014; H:VI1PR08MB5376.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Message-Info-Original: XdPvBgWrCsDe2fY1SqWCuytjaMuxKilDwozWXjtXh5CueoJuSEK/vfcTRJbHmGvmoxtszAiE1HhP2OaRO3i6/30AnK3JkPrwN3Fs+7RJK2F7IA3CpmxJ/pJrvT+nQcTxuaVmHV6ZIuRGxG0KwijAQPbzX3JjMxuWsgB0dxBVDsqW9DdF3JuLATszrze0YoqoS14zf/SmnDMaq7/Ae/5y4Zezeuj2Gf/KV+76NGZrbGrv/DnmobuBwQIPktVvOWXWsXLd4PZMcAPL6ApeMgiazuRQgGrf8qaKEEeMOuzgfc87pXNCnlVLOTBnVQbPhaCKgcbZE2BETjrjNWkIGDh09cCI5DUzgtHL3oR1RF4UNBpqc4fbMHolkKU1hJfyrBrta96Mqlla38XbG8CSptRREtXqPNIcH1nsz2H3Ph1erow= Content-Type: text/plain; charset="iso-2022-jp" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB4014 Original-Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Gavin.Hu@arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM5EUR03FT052.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; IPV:NLI; CTRY:IE; EFV:NLI; SFV:NSPM; SFS:(10009020)(4636009)(136003)(39860400002)(396003)(376002)(346002)(13464003)(189003)(199004)(26005)(3846002)(99286004)(81156014)(102836004)(22756006)(446003)(76176011)(7736002)(336012)(356004)(70586007)(63350400001)(9686003)(126002)(74316002)(2501003)(476003)(55016002)(229853002)(7696005)(305945005)(47776003)(66066001)(54906003)(5660300002)(2906002)(6506007)(53546011)(52536014)(70206006)(186003)(316002)(6116002)(8676002)(81166006)(8936002)(25786009)(33656002)(478600001)(23736002)(4326008)(50466002)(11346002)(6246003)(486006)(14454004)(86362001)(110136005)(36906005)(8746002)(14444005); DIR:OUT; SFP:1101; SCL:1; SRVR:VI1PR08MB5502; H:64aa7808-outbound-1.mta.getcheckrecipient.com; FPR:; SPF:TempError; LANG:en; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; MX:1; A:1; X-MS-Office365-Filtering-Correlation-Id-Prvs: bcae1831-bc8a-4a31-1735-08d73cb66fce X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600167)(710020)(711020)(4605104)(1401327)(4618075)(2017052603328)(7193020); SRVR:VI1PR08MB5502; NoDisclaimer: True X-Forefront-PRVS: 016572D96D X-Microsoft-Antispam-Message-Info: QBoAulb32ceABzsmNd/GhYaTw3faDkRdHeqKWXZS38NQk/jjuVFWpP43Rz3kX05WkBBAFjGrI9VVU+euwgxYdFEwyE02IYJQeA1KaUSHLcXWeXo447tnjEvBoSAiPVEY5Mz5bbs7tw76lNSa73F56BHL+tC1efowWfr11nL0uWuYMG445EhkGnh2Mk1Ba1qjgehXnmok16MR9c57KPPvwcaD33MDlx4DhWoKhI9YcsRnPI62H4CoEILYvcTZL5miKI7eC76zw0ZTbf4Tm46PwxjyDAVoE8YNyznRtg28RWFVsjzOWGG4PeVO4P+cYFAj1KyrV8Matxnm4z1kKB2zSdPCR8NiWVVslN/n/MbibvaJOc1qsnbWKglP1/ec8fKI+1U+uMkoDYZ1eS3qF3iNGfoBvSR8AOBUkOcH8n0WUbY= X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Sep 2019 04:04:24.5811 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: cb46d520-f875-417b-7e94-08d73cb67556 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB5502 Subject: Re: [dpdk-dev] [PATCH v4 2/2] virtio: one way barrier for packed vring desc used flags X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Yinan, Thanks for verification and reporting this, I will dump the assembly code a= nd make an analysis after I am back from travel. >From my understanding the assembly code for x86 should be same, but I am no= t sure if anything we were missing.=20 Is this <1 perf drop coming from run to run variances, or system noise?=20 /Gavin > -----Original Message----- > From: Wang, Yinan > Sent: Wednesday, September 18, 2019 7:21 AM > To: Joyce Kong (Arm Technology China) ; > dev@dpdk.org > Cc: nd ; maxime.coquelin@redhat.com; Richardson, Bruce > ; Bie, Tiwei ; Wang, > Zhihong ; amorenoz@redhat.com; Wang, Xiao W > ; Liu, Yong ; > jfreimann@redhat.com; Honnappa Nagarahalli > ; Gavin Hu (Arm Technology China) > > Subject: RE: [PATCH v4 2/2] virtio: one way barrier for packed vring desc= used > flags >=20 >=20 > Hi Joyce, >=20 > I test performance impact of your patch set with code base commit id: > d03d8622db48918d14bfe805641b1766ecc40088, after applying your v4 patch > set , packed ring shows small performance drop as below: >=20 > PVP vhost/virtio 1c1q test commit:d03d8622db48918 apply v4 > patch set >=20 > pvp_virtio11_mergeable 7.218 7.147 > pvp_virtio11_normal 7.217 7.182 >=20 >=20 > Thanks, > Yinan >=20 >=20 >=20 >=20 > > -----Original Message----- > > From: Joyce Kong [mailto:joyce.kong@arm.com] > > Sent: 2019=1B$BG/=1B(B9=1B$B7n=1B(B17=1B$BF|=1B(B 13:28 > > To: dev@dpdk.org > > Cc: nd@arm.com; maxime.coquelin@redhat.com; Wang, Yinan > > ; Richardson, Bruce ; > Bie, > > Tiwei ; Wang, Zhihong ; > > amorenoz@redhat.com; Wang, Xiao W ; Liu, Yong > > ; jfreimann@redhat.com; > honnappa.nagarahalli@arm.com; > > gavin.hu@arm.com > > Subject: [PATCH v4 2/2] virtio: one way barrier for packed vring desc u= sed > flags > > > > In case VIRTIO_F_ORDER_PLATFORM(36) is not negotiated, then the > frontend > > and backend are assumed to be implemented in software, that is they can > run on > > identical CPUs in an SMP configuration. > > Thus a weak form of memory barriers like rte_smp_r/wmb, other than > > rte_cio_r/wmb, is sufficient for this case(vq->hw->weak_barriers =3D=3D= 1) and > > yields better performance. > > For the above case, this patch helps yielding even better performance b= y > > replacing the two-way barriers with C11 one-way barriers for used flags= in > packed > > ring. > > > > Signed-off-by: Joyce Kong > > Reviewed-by: Gavin Hu > > Reviewed-by: Phil Yang > > Reviewed-by: Maxime Coquelin > > --- > > drivers/net/virtio/virtio_rxtx.c | 12 +++++++--- > > drivers/net/virtio/virtio_user/virtio_user_dev.c | 4 ++-- > > drivers/net/virtio/virtqueue.h | 28 > > +++++++++++++++++++++++- > > lib/librte_vhost/virtio_net.c | 5 ++--- > > 4 files changed, 40 insertions(+), 9 deletions(-) > > > > diff --git a/drivers/net/virtio/virtio_rxtx.c b/drivers/net/virtio/virt= io_rxtx.c > > index a87ffe1..2f0879c 100644 > > --- a/drivers/net/virtio/virtio_rxtx.c > > +++ b/drivers/net/virtio/virtio_rxtx.c > > @@ -122,9 +122,11 @@ virtqueue_dequeue_burst_rx_packed(struct > virtqueue > > *vq, > > > > for (i =3D 0; i < num; i++) { > > used_idx =3D vq->vq_used_cons_idx; > > + /* desc_is_used has a load-acquire or rte_cio_rmb inside > > + * and wait for used desc in virtqueue. > > + */ > > if (!desc_is_used(&desc[used_idx], vq)) > > return i; > > - virtio_rmb(vq->hw->weak_barriers); > > len[i] =3D desc[used_idx].len; > > id =3D desc[used_idx].id; > > cookie =3D (struct rte_mbuf *)vq->vq_descx[id].cookie; @@ - > 233,8 > > +235,10 @@ virtio_xmit_cleanup_inorder_packed(struct virtqueue *vq, int > num) > > struct vq_desc_extra *dxp; > > > > used_idx =3D vq->vq_used_cons_idx; > > + /* desc_is_used has a load-acquire or rte_cio_rmb inside > > + * and wait for used desc in virtqueue. > > + */ > > while (num > 0 && desc_is_used(&desc[used_idx], vq)) { > > - virtio_rmb(vq->hw->weak_barriers); > > id =3D desc[used_idx].id; > > do { > > curr_id =3D used_idx; > > @@ -265,8 +269,10 @@ virtio_xmit_cleanup_normal_packed(struct > virtqueue > > *vq, int num) > > struct vq_desc_extra *dxp; > > > > used_idx =3D vq->vq_used_cons_idx; > > + /* desc_is_used has a load-acquire or rte_cio_rmb inside > > + * and wait for used desc in virtqueue. > > + */ > > while (num-- && desc_is_used(&desc[used_idx], vq)) { > > - virtio_rmb(vq->hw->weak_barriers); > > id =3D desc[used_idx].id; > > dxp =3D &vq->vq_descx[id]; > > vq->vq_used_cons_idx +=3D dxp->ndescs; > > diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c > > b/drivers/net/virtio/virtio_user/virtio_user_dev.c > > index 7911c39..1c575d0 100644 > > --- a/drivers/net/virtio/virtio_user/virtio_user_dev.c > > +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c > > @@ -698,8 +698,8 @@ virtio_user_handle_cq_packed(struct > virtio_user_dev > > *dev, uint16_t queue_idx) > > if (vq->used_wrap_counter) > > flags |=3D VRING_PACKED_DESC_F_AVAIL_USED; > > > > - rte_smp_wmb(); > > - vring->desc[vq->used_idx].flags =3D flags; > > + __atomic_store_n(&vring->desc[vq->used_idx].flags, flags, > > + __ATOMIC_RELEASE); > > > > vq->used_idx +=3D n_descs; > > if (vq->used_idx >=3D dev->queue_size) { diff --git > > a/drivers/net/virtio/virtqueue.h b/drivers/net/virtio/virtqueue.h index > > b728ff8..8d7f197 100644 > > --- a/drivers/net/virtio/virtqueue.h > > +++ b/drivers/net/virtio/virtqueue.h > > @@ -54,6 +54,32 @@ virtio_wmb(uint8_t weak_barriers) > > rte_cio_wmb(); > > } > > > > +static inline uint16_t > > +virtqueue_fetch_flags_packed(struct vring_packed_desc *dp, > > + uint8_t weak_barriers) > > +{ > > + uint16_t flags; > > + > > + if (weak_barriers) { > > +/* x86 prefers to using rte_smp_rmb over __atomic_load_n as it reports > > + * a better perf(~1.5%), which comes from the saved branch by the > compiler. > > + * The if and else branch are identical with the smp and cio barriers > > +both > > + * defined as compiler barriers on x86. > > + */ > > +#ifdef RTE_ARCH_X86_64 > > + flags =3D dp->flags; > > + rte_smp_rmb(); > > +#else > > + flags =3D __atomic_load_n(&dp->flags, __ATOMIC_ACQUIRE); > #endif > > + } else { > > + flags =3D dp->flags; > > + rte_cio_rmb(); > > + } > > + > > + return flags; > > +} > > + > > static inline void > > virtqueue_store_flags_packed(struct vring_packed_desc *dp, > > uint16_t flags, uint8_t weak_barriers) @@ -307,7 > +333,7 > > @@ desc_is_used(struct vring_packed_desc *desc, struct virtqueue *vq) = { > > uint16_t used, avail, flags; > > > > - flags =3D desc->flags; > > + flags =3D virtqueue_fetch_flags_packed(desc, vq->hw->weak_barriers); > > used =3D !!(flags & VRING_PACKED_DESC_F_USED); > > avail =3D !!(flags & VRING_PACKED_DESC_F_AVAIL); > > > > diff --git a/lib/librte_vhost/virtio_net.c b/lib/librte_vhost/virtio_ne= t.c index > > e7463ff..241d467 100644 > > --- a/lib/librte_vhost/virtio_net.c > > +++ b/lib/librte_vhost/virtio_net.c > > @@ -110,8 +110,6 @@ flush_shadow_used_ring_packed(struct virtio_net > *dev, > > used_idx -=3D vq->size; > > } > > > > - rte_smp_wmb(); > > - > > for (i =3D 0; i < vq->shadow_used_idx; i++) { > > uint16_t flags; > > > > @@ -147,7 +145,8 @@ flush_shadow_used_ring_packed(struct virtio_net > *dev, > > } > > } > > > > - vq->desc_packed[head_idx].flags =3D head_flags; > > + __atomic_store_n(&vq->desc_packed[head_idx].flags, head_flags, > > + __ATOMIC_RELEASE); > > > > vhost_log_cache_used_vring(dev, vq, > > head_idx * > > -- > > 2.7.4