From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9C849A058A; Fri, 17 Apr 2020 10:15:02 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 99FF81DE8E; Fri, 17 Apr 2020 10:15:01 +0200 (CEST) Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-eopbgr80048.outbound.protection.outlook.com [40.107.8.48]) by dpdk.org (Postfix) with ESMTP id 408461DE89 for ; Fri, 17 Apr 2020 10:15:00 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=owYEea4qFlyBPB+VUtqMN8l/tPNoXoj+ichHRyeHK+E=; b=CV+T2fEcpyD4ENZbVhI2/3zq/LLCO0LsI/b81OyswbJ4d/4uVcLPTg0cohLyC9+6Eka3WV3ylGehL2vsDtknF4u208UOU53jZS40jylBRz2tg0pk8D+Qp5x60WUPGDQhjptpaftxfkic/vLwYIAroCup4bodEnLJfFco43LZNck= Received: from DB6PR0501CA0018.eurprd05.prod.outlook.com (2603:10a6:4:8f::28) by DB7PR08MB3371.eurprd08.prod.outlook.com (2603:10a6:10:4c::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2921.26; Fri, 17 Apr 2020 08:14:57 +0000 Received: from DB5EUR03FT036.eop-EUR03.prod.protection.outlook.com (2603:10a6:4:8f:cafe::35) by DB6PR0501CA0018.outlook.office365.com (2603:10a6:4:8f::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2921.27 via Frontend Transport; Fri, 17 Apr 2020 08:14:57 +0000 Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=bestguesspass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DB5EUR03FT036.mail.protection.outlook.com (10.152.20.185) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2900.15 via Frontend Transport; Fri, 17 Apr 2020 08:14:57 +0000 Received: ("Tessian outbound a45624f5910b:v50"); Fri, 17 Apr 2020 08:14:57 +0000 X-CR-MTA-TID: 64aa7808 Received: from 6b7fa36ca1b2.2 by 64aa7808-outbound-1.mta.getcheckrecipient.com id BED3EE97-77E5-4EE8-8374-ACF4C642C2B9.1; Fri, 17 Apr 2020 08:14:52 +0000 Received: from EUR05-VI1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 6b7fa36ca1b2.2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Fri, 17 Apr 2020 08:14:52 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YqbDZhUDt3yBPMe4YKnhKoyPOM71k1XOjxXRO8oxcBEbdTmZmg6zwUvKMXemnrb7nGFF5SZupHrWBYMaIp+fwXPurxWX68ZKUXn/NCF8XNCPTOOinLU1O2k3lTFeZ6BHni+Ye9xUPF5pZplQgADR7cHtBQ6MfpNAdv0+DBYS6TPB/M4dEY+PJjag/SM3sN4+Dgbs2jLqPMXEJcq3Z4Xtjm4sC+5eKB+a6mDsNfs2BlDQIbO5PUHytL32EuuNbno3INWA+g1sCQNPSyTRfSbxJ/4JxIQEurAF3vzldS7l9JDA5j+RZ5CsYBuRmQGsAfbIm/dCQw8inLSEUOAzv+N9MA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=owYEea4qFlyBPB+VUtqMN8l/tPNoXoj+ichHRyeHK+E=; b=ICqQgPC4nlJQw/RuTlKvKVqS6dE6Zagg0kbubi7b59O+NHwz9KnLg4G/WZPL0rgBLGFy8tfxp0nOCB31C0zn4QP8qCseqgXVbvpBact4ktcOFpX6VV9z+SuyZYjpqW00KJWRY+qSzMccpBc5NuuHEjo7zwDvqrprWbjbgl2wqKa8HbvFtZr4CBJBFiaFObuS03focI16LXCYtVpqkNC+i4L7MPHQaOfv1aSyFTvVlx08RZ4XeTF0kxkfRn4InsTQXb7OK6GHEIY5nZfNWQiBdCWf2Zel+tH3iqy8Fl0V88i5ka3wGfbHtWCpanK5YdzmEiN2Ma+TzRuQ7gHz4Tm/fA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=owYEea4qFlyBPB+VUtqMN8l/tPNoXoj+ichHRyeHK+E=; b=CV+T2fEcpyD4ENZbVhI2/3zq/LLCO0LsI/b81OyswbJ4d/4uVcLPTg0cohLyC9+6Eka3WV3ylGehL2vsDtknF4u208UOU53jZS40jylBRz2tg0pk8D+Qp5x60WUPGDQhjptpaftxfkic/vLwYIAroCup4bodEnLJfFco43LZNck= Received: from DB7PR08MB3307.eurprd08.prod.outlook.com (2603:10a6:5:1b::32) by DB7PR08MB2971.eurprd08.prod.outlook.com (2603:10a6:5:1c::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2900.26; Fri, 17 Apr 2020 08:14:50 +0000 Received: from DB7PR08MB3307.eurprd08.prod.outlook.com ([fe80::dc88:50dc:97a2:69bd]) by DB7PR08MB3307.eurprd08.prod.outlook.com ([fe80::dc88:50dc:97a2:69bd%7]) with mapi id 15.20.2900.030; Fri, 17 Apr 2020 08:14:50 +0000 From: Joyce Kong To: Ye Xiaolong CC: "maxime.coquelin@redhat.com" , "stephen@networkplumber.org" , "tiwei.bie@intel.com" , "zhihong.wang@intel.com" , "thomas@monjalon.net" , "jerinj@marvell.com" , "yinan.wang@intel.com" , Honnappa Nagarahalli , Gavin Hu , nd , "dev@dpdk.org" Thread-Topic: [dpdk-dev] [PATCH v3 1/2] virtio: one way barrier for split vring used idx Thread-Index: AQHWFIVClLcolEw5q0+5S20lYMxnX6h89xEw Date: Fri, 17 Apr 2020 08:14:50 +0000 Message-ID: References: <20200212092456.29433-1-joyce.kong@arm.com> <20200406152634.606-2-joyce.kong@arm.com> <20200417065145.GA57965@intel.com> In-Reply-To: <20200417065145.GA57965@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: 3d5c06da-352f-4f52-a9f1-f1ff46146582.0 x-checkrecipientchecked: true Authentication-Results-Original: spf=none (sender IP is ) smtp.mailfrom=Joyce.Kong@arm.com; x-originating-ip: [113.29.88.7] x-ms-publictraffictype: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: c728c8e6-7c9c-4d70-0b8e-08d7e2a76aea x-ms-traffictypediagnostic: DB7PR08MB2971:|DB7PR08MB2971:|DB7PR08MB3371: x-ld-processed: f34e5979-57d9-4aaa-ad4d-b122a662184d,ExtAddr x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true nodisclaimer: true x-ms-oob-tlc-oobclassifiers: OLM:9508;OLM:9508; x-forefront-prvs: 0376ECF4DD X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DB7PR08MB3307.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(10009020)(4636009)(396003)(39860400002)(136003)(366004)(376002)(346002)(7696005)(8676002)(26005)(55016002)(71200400001)(6916009)(478600001)(8936002)(4326008)(2906002)(52536014)(33656002)(81156014)(54906003)(66476007)(66946007)(5660300002)(316002)(66446008)(53546011)(76116006)(186003)(66556008)(86362001)(64756008)(55236004)(9686003)(6506007); DIR:OUT; SFP:1101; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: 1qCnPNwh13iqVWD5kp4fFjWUBRpZu2PfyMNztAcidpH2HdodUxAqe94b5ro6f93kYX3i72/LaRNUtUYy24X8vMiVLOPWD2BqoyH/yzZqv7sIfCPdq5+Gt5xOj49x8kk2pVV1NTLR0jEldEgD9ejqUBuI3Pi2yZRc8h8OV4BahPJPrEGdMNnTmzFsU7JpdmVvkqO6iOWxE6LbYF7dQG7U9hGTi/UrtqTMYG99VN7bXDQu3VcXhzkM+MT7rl/OugZ16VbK8pRu7xaR/sSaOMbKJBFtfIGDQfTUwUuXh0XaLSmZey7eUe4rzptEwTpjY6nF7kRmIZ+n9lcl5TRmWEr5IH9P4vyDuG053ECIKq67WOjlVBFIhR1vdnssTUaqdIFX9O24e+hVJMRUGorpQx8J9yxw5xX0W1RJRrfM3z6Et23OamjRaEF7v40Bzdw754oa x-ms-exchange-antispam-messagedata: 2ZLTFWb1/Ey0i0Ay/NfKKkP48VTo+Q+Z/xiDVrulVSGGTUe5hMg9697ibjOdcDrJ8Ny5tTPGRzYCZmPZeVXF4olZ9j+IW4y6SeEjNUjXRewAIAri3UqWMdU2T0d0N1WcPRSL+2LCMVG4Rqb8IVs22w== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR08MB2971 Original-Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Joyce.Kong@arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DB5EUR03FT036.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFTY:; SFS:(10009020)(4636009)(376002)(39860400002)(136003)(396003)(346002)(46966005)(478600001)(81156014)(82740400003)(52536014)(86362001)(2906002)(55016002)(8676002)(81166007)(8936002)(186003)(9686003)(26005)(6862004)(53546011)(33656002)(336012)(7696005)(4326008)(6506007)(70586007)(356005)(70206006)(47076004)(26826003)(5660300002)(316002)(54906003); DIR:OUT; SFP:1101; X-MS-Office365-Filtering-Correlation-Id-Prvs: 466f2e8d-ef3d-4df2-71af-08d7e2a7666b X-Forefront-PRVS: 0376ECF4DD X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: FHTfc6iaC3ci5Re/oB0CU3019vXs/6xj9PxYGcuGEasFt/9J7p1jZ42KXDZ86RTpjxx0bTGAgnKSUHcRgjamf2Gy3Ov7RKGGoudyhE5GV37kHbo+x//LHpCrUy+mLBUZLYZ5/V9XByLu4ZpVkwuzZTxrHKjkuf6tqZrmqNmjhlq5HGrnRNVHoQT5t+29mfTjrqXygXhO9p/9XXsCWwDOCp2gj/+nDdQmprlEzpA8IN9m46lBoy9tQyeICnrxh4J4f4It3WZakONssB+kHwGlce+kr99ElRLlQchU4a18vYk8+MrX9mUu1PSuRq0RoFFmXskYyQ81IeN/NKsw783lW36b+k6bGaBiSGp0a7lhJZ85azB6B7ASeISto0u4qBIdHXWFoBV7zYQ8mQo1r5wC73c8MZJmIy1L4V7PReTtdCZyElq4V6X2DcICRisTfPyIuHTi8qPeS9NwKCuW1NEDTZ04ojn10H3VmibQlBPQmek6nUfU3Fl27CCA8XQuOEHQb6x2YjGFT/LV8vHtPK5Rlg== X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Apr 2020 08:14:57.7293 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c728c8e6-7c9c-4d70-0b8e-08d7e2a76aea X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR08MB3371 Subject: Re: [dpdk-dev] [PATCH v3 1/2] virtio: one way barrier for split vring used idx X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > -----Original Message----- > From: Ye Xiaolong > Sent: Friday, April 17, 2020 2:52 PM > To: Joyce Kong > Cc: maxime.coquelin@redhat.com; stephen@networkplumber.org; > tiwei.bie@intel.com; zhihong.wang@intel.com; thomas@monjalon.net; > jerinj@marvell.com; yinan.wang@intel.com; Honnappa Nagarahalli > ; Gavin Hu ; nd > ; dev@dpdk.org > Subject: Re: [dpdk-dev] [PATCH v3 1/2] virtio: one way barrier for split = vring > used idx >=20 > On 04/06, Joyce Kong wrote: > >In case VIRTIO_F_ORDER_PLATFORM(36) is not negotiated, then the > >frontend and backend are assumed to be implemented in software, that is > >they can run on identical CPUs in an SMP configuration. > >Thus a weak form of memory barriers like rte_smp_r/wmb, other than > >rte_cio_r/wmb, is sufficient for this case(vq->hw->weak_barriers =3D=3D = 1) > >and yields better performance. > >For the above case, this patch helps yielding even better performance > >by replacing the two-way barriers with C11 one-way barriers for used > >index in split ring. > > > >Signed-off-by: Joyce Kong > >Reviewed-by: Gavin Hu > >--- > > drivers/net/virtio/virtio_ethdev.c | 9 ++-- > > drivers/net/virtio/virtio_ring.h | 2 +- > > drivers/net/virtio/virtio_rxtx.c | 46 +++++++++---------- > > drivers/net/virtio/virtio_rxtx_simple_neon.c | 5 +- > > drivers/net/virtio/virtio_rxtx_simple_sse.c | 5 +- > > .../net/virtio/virtio_user/virtio_user_dev.c | 8 ++-- > > drivers/net/virtio/virtqueue.c | 2 +- > > drivers/net/virtio/virtqueue.h | 37 ++++++++++++--- > > lib/librte_vhost/virtio_net.c | 5 +- > > 9 files changed, 71 insertions(+), 48 deletions(-) > > > >diff --git a/drivers/net/virtio/virtio_ethdev.c > >b/drivers/net/virtio/virtio_ethdev.c > >index f9d0ea70d..a4a865bfa 100644 > >--- a/drivers/net/virtio/virtio_ethdev.c > >+++ b/drivers/net/virtio/virtio_ethdev.c > >@@ -285,13 +285,12 @@ virtio_send_command_split(struct virtnet_ctl > >*cvq, > > > > virtqueue_notify(vq); > > > >- rte_rmb(); > >- while (VIRTQUEUE_NUSED(vq) =3D=3D 0) { > >- rte_rmb(); > >+ /* virtqueue_nused has a load-acquire or rte_cio_rmb inside */ > >+ while (virtqueue_nused(vq) =3D=3D 0) > > usleep(100); > >- } > > > >- while (VIRTQUEUE_NUSED(vq)) { > >+ /* virtqueue_nused has a load-acquire or rte_cio_rmb inside */ > >+ while (virtqueue_nused(vq)) { > > uint32_t idx, desc_idx, used_idx; > > struct vring_used_elem *uep; > > > >diff --git a/drivers/net/virtio/virtio_ring.h > >b/drivers/net/virtio/virtio_ring.h > >index 7ba34662e..0f6574f68 100644 > >--- a/drivers/net/virtio/virtio_ring.h > >+++ b/drivers/net/virtio/virtio_ring.h > >@@ -59,7 +59,7 @@ struct vring_used_elem { > > > > struct vring_used { > > uint16_t flags; > >- volatile uint16_t idx; > >+ uint16_t idx; > > struct vring_used_elem ring[0]; > > }; > > > >diff --git a/drivers/net/virtio/virtio_rxtx.c > >b/drivers/net/virtio/virtio_rxtx.c > >index 752faa0f6..9ba26fd95 100644 > >--- a/drivers/net/virtio/virtio_rxtx.c > >+++ b/drivers/net/virtio/virtio_rxtx.c > >@@ -45,7 +45,7 @@ virtio_dev_rx_queue_done(void *rxq, uint16_t offset) > > struct virtnet_rx *rxvq =3D rxq; > > struct virtqueue *vq =3D rxvq->vq; > > > >- return VIRTQUEUE_NUSED(vq) >=3D offset; > >+ return virtqueue_nused(vq) >=3D offset; > > } > > > > void > >@@ -1243,9 +1243,8 @@ virtio_recv_pkts(void *rx_queue, struct rte_mbuf > **rx_pkts, uint16_t nb_pkts) > > if (unlikely(hw->started =3D=3D 0)) > > return nb_rx; > > > >- nb_used =3D VIRTQUEUE_NUSED(vq); > >- > >- virtio_rmb(hw->weak_barriers); > >+ /* virtqueue_nused has a load-acquire or rte_cio_rmb inside */ >=20 > Small nit, I don't think we need to add this comment to every occurrence = of > virtqueue_nused, what about moving it to the definition of this function? >=20 > Thanks, > Xiaolong Will modify as this in v4. Thanks, Joyce