From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 78585A329E for ; Wed, 23 Oct 2019 18:29:54 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id BAD971C25E; Wed, 23 Oct 2019 18:29:53 +0200 (CEST) Received: from EUR02-VE1-obe.outbound.protection.outlook.com (mail-eopbgr20070.outbound.protection.outlook.com [40.107.2.70]) by dpdk.org (Postfix) with ESMTP id 109831C214 for ; Wed, 23 Oct 2019 18:29:53 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yAqqf71rNqkeurtP6WMJdy8f+Ld0fsdNy0ozGg5vjO0=; b=8rB0gxWpFRMK8KyW3sJHhHS669/bEdonAtzxIjB07sQuXbBobGfIcHJmBu2+qLyxBWCjDX5tRmjqm7J9r0EYrmFacwEXl5zgTtzzzDSe8s7rAuUhxFNsZcHQj2s1fqkswh46AKbhjOms9tbA6dQ0iH+s6RnrxGvBXOFm884YVis= Received: from AM6PR08CA0027.eurprd08.prod.outlook.com (2603:10a6:20b:c0::15) by VE1PR08MB4800.eurprd08.prod.outlook.com (2603:10a6:802:a9::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2347.16; Wed, 23 Oct 2019 16:29:51 +0000 Received: from VE1EUR03FT026.eop-EUR03.prod.protection.outlook.com (2a01:111:f400:7e09::203) by AM6PR08CA0027.outlook.office365.com (2603:10a6:20b:c0::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2387.20 via Frontend Transport; Wed, 23 Oct 2019 16:29:50 +0000 Authentication-Results: spf=fail (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=none action=none header.from=arm.com; Received-SPF: Fail (protection.outlook.com: domain of arm.com does not designate 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by VE1EUR03FT026.mail.protection.outlook.com (10.152.18.148) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2367.23 via Frontend Transport; Wed, 23 Oct 2019 16:29:50 +0000 Received: ("Tessian outbound e4042aced47b:v33"); Wed, 23 Oct 2019 16:29:49 +0000 X-CR-MTA-TID: 64aa7808 Received: from 150a6bee44af.2 (ip-172-16-0-2.eu-west-1.compute.internal [104.47.13.55]) by 64aa7808-outbound-1.mta.getcheckrecipient.com id 146BD3A9-0BA2-43D1-A1C7-77DFB38E26DA.1; Wed, 23 Oct 2019 16:29:44 +0000 Received: from EUR04-HE1-obe.outbound.protection.outlook.com (mail-he1eur04lp2055.outbound.protection.outlook.com [104.47.13.55]) by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 150a6bee44af.2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Wed, 23 Oct 2019 16:29:44 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=n/DBZFtsnmIjcvx8ZYK3SZPrc6tHBFIFa8VY2/yweQ3eoil1h0UrMa2PEx1S30/MHdWqaYP9AnUq3xW/si+Inovhq2n33d74itZd2/4FnL4xvbRfSKG2DNj/tckfDe6lIbSVEhLKKHn7ezI47J5Z+Bby/o+14f/A5UqrZOTXmnAKd4X3RwQJNCHTkN6Fi7/xK8C5B6uWfXS357EX3nPR/Qgzd88aJKqJUW1U/0aUo6+P01SJC42wMf8Amgcuw7hTO09MS15uOeuAavMzTHgCcbOIspaBp+uOKhgqAIK1KFPB7dOLgZt2dHfCeBQyr+1LM3MnQqBYFvLgdvOfc67Q7A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yAqqf71rNqkeurtP6WMJdy8f+Ld0fsdNy0ozGg5vjO0=; b=Fi5G1+tA36XNciUOWA8WW5Y3RF7Ewcj3Lq/gdXDaBc9b8/gmIggZg5sUnD9KFOyjyQypplfCbV3+JsGSD8T9KO/osaHyp7T9J1iJ0LGGpKw7gLC7gvxA23n6C5Q6zVnC1zPiMFmgD4t0ow5qP693oUNcUPx80j7iIku/iL2/6xtWn9gk0oxAstZI+BuGvH6JwjCquWZlf3pweFpXCqqWl7AyDzioFehuVQX1hB+v10mRhVipSqa3cVrwKaZFBiHLAxPyf1z0er5U3pkkAxzxf+DVxBIVyKDs6gBH6S6v3sexP+LUI7KTk/iyLK/9hMI9mWr6DXJnwSN0J24ZPBfcLw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yAqqf71rNqkeurtP6WMJdy8f+Ld0fsdNy0ozGg5vjO0=; b=8rB0gxWpFRMK8KyW3sJHhHS669/bEdonAtzxIjB07sQuXbBobGfIcHJmBu2+qLyxBWCjDX5tRmjqm7J9r0EYrmFacwEXl5zgTtzzzDSe8s7rAuUhxFNsZcHQj2s1fqkswh46AKbhjOms9tbA6dQ0iH+s6RnrxGvBXOFm884YVis= Received: from AM0PR08MB5363.eurprd08.prod.outlook.com (52.132.214.213) by AM0PR08MB5139.eurprd08.prod.outlook.com (10.255.29.146) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2387.22; Wed, 23 Oct 2019 16:29:41 +0000 Received: from AM0PR08MB5363.eurprd08.prod.outlook.com ([fe80::b483:301f:e382:a94d]) by AM0PR08MB5363.eurprd08.prod.outlook.com ([fe80::b483:301f:e382:a94d%6]) with mapi id 15.20.2367.025; Wed, 23 Oct 2019 16:29:41 +0000 From: "Gavin Hu (Arm Technology China)" To: "Ananyev, Konstantin" , "dev@dpdk.org" CC: nd , "thomas@monjalon.net" , "stephen@networkplumber.org" , "hemant.agrawal@nxp.com" , "jerinj@marvell.com" , "pbhagavatula@marvell.com" , Honnappa Nagarahalli , "Ruifeng Wang (Arm Technology China)" , "Phil Yang (Arm Technology China)" , Steve Capper , nd , nd Thread-Topic: [dpdk-dev] [PATCH v7 2/7] eal: add the APIs to wait until equal Thread-Index: AQHVdPZmZDiMClkJakOV0igd5gNYTKde7MQggAA7nkCACWXHAIAABICA Date: Wed, 23 Oct 2019 16:29:41 +0000 Message-ID: References: <1561911676-37718-1-git-send-email-gavin.hu@arm.com> <1569562904-43950-3-git-send-email-gavin.hu@arm.com> <2601191342CEEE43887BDE71AB97725801A8C6AB30@IRSMSX104.ger.corp.intel.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: 9dec6245-f2d1-4c09-93fa-4a25b0feffe1.0 x-checkrecipientchecked: true Authentication-Results-Original: spf=none (sender IP is ) smtp.mailfrom=Gavin.Hu@arm.com; x-originating-ip: [113.29.88.7] x-ms-publictraffictype: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 102d5da2-fcf4-4eb6-8333-08d757d63a44 X-MS-TrafficTypeDiagnostic: AM0PR08MB5139:|AM0PR08MB5139:|VE1PR08MB4800: x-ld-processed: f34e5979-57d9-4aaa-ad4d-b122a662184d,ExtAddr x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true x-ms-oob-tlc-oobclassifiers: OLM:8273;OLM:8273; x-forefront-prvs: 019919A9E4 X-Forefront-Antispam-Report-Untrusted: SFV:NSPM; SFS:(10009020)(4636009)(366004)(136003)(39860400002)(376002)(346002)(396003)(189003)(199004)(13464003)(71200400001)(71190400001)(9686003)(74316002)(14454004)(55016002)(478600001)(6246003)(2501003)(86362001)(30864003)(52536014)(5660300002)(6436002)(2906002)(256004)(14444005)(305945005)(7736002)(2940100002)(66946007)(66476007)(66556008)(64756008)(66446008)(81166006)(81156014)(8676002)(8936002)(66066001)(11346002)(446003)(486006)(476003)(76116006)(110136005)(316002)(99286004)(33656002)(54906003)(7696005)(186003)(76176011)(6506007)(26005)(102836004)(53546011)(25786009)(229853002)(6116002)(3846002)(4326008)(55236004)(21314003); DIR:OUT; SFP:1101; SCL:1; SRVR:AM0PR08MB5139; H:AM0PR08MB5363.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: UazGDSEfqFfpZg0HNIN9WFeYafxJdm2uW318SpYm+HGqCpn7tJGRkJvPIMMlbvwwnHHRJ0ou2Al/BKw/HKXxAXG/lZ96wyMd9B5a6EPPtIPPRQepZEcgb4ZQPflWdBj5uY4IoOx/v0yUOfT0YJYGuQMJ0oeVa+8V08AdtcLagiO84XQK4xUMzGBlfcuSLVLw2luyTM+XtDMemfEHbVtTz5u5edVHKllAB4yIxdDqAS8TUyZEAUgcfNHl+VxjXEZr7O1W60AnyQq07EkAVvPD4VFBav0jwsoC0OOjepqgQOovOFcV1zJonJ7ZpiNp6uDAlTAhMVt5txeP1S0oHPwVdaoi6hgh1u1dTRtJer1UefaLuCaRrLL2LTnhNjRf1aE64h46SSJhymkROE1mtHktn9Rb9tG+8zv86gJ6qrQXMVoPUd671r0JPpxocyU8ObST Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB5139 Original-Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Gavin.Hu@arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: VE1EUR03FT026.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; IPV:CAL; SCL:-1; CTRY:IE; EFV:NLI; SFV:NSPM; SFS:(10009020)(4636009)(346002)(39860400002)(376002)(136003)(396003)(1110001)(339900001)(13464003)(199004)(189003)(81166006)(76130400001)(81156014)(70206006)(8676002)(229853002)(8936002)(53546011)(26826003)(6506007)(478600001)(7696005)(126002)(476003)(486006)(50466002)(2501003)(110136005)(7736002)(305945005)(99286004)(74316002)(52536014)(97756001)(8746002)(70586007)(55016002)(6116002)(22756006)(30864003)(9686003)(4326008)(86362001)(2940100002)(336012)(356004)(14444005)(47776003)(36906005)(33656002)(54906003)(316002)(26005)(76176011)(186003)(46406003)(2906002)(25786009)(102836004)(6246003)(5660300002)(105606002)(14454004)(66066001)(3846002)(23726003)(446003)(11346002)(21314003); DIR:OUT; SFP:1101; SCL:1; SRVR:VE1PR08MB4800; H:64aa7808-outbound-1.mta.getcheckrecipient.com; FPR:; SPF:Fail; LANG:en; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; A:1; MX:1; X-MS-Office365-Filtering-Correlation-Id-Prvs: 8fce714e-ff9e-4cf4-a5f6-08d757d634ae NoDisclaimer: True X-Forefront-PRVS: 019919A9E4 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: qR8wAttAwP+6jlHu7dwWH7SbnZEWdxU7hldBX02uchv1dBYtc6wI/stRRRQUhidIhi4Rv6/qSTyoAqeTH76S8dg2wcCA/xtgWIwVbHNrTRlQJiAWtSQqqq1g8gaseYtFR/QPH/84JziMW4J6s9N/5WHQ0UJmy1U0qbbvMQm+cJCClwYX0ZNfMaJKSqfYKDlKPlFlVcZn38QQvmxrntqjWt9etfBp27+lmkPNhmPI0Ec8opwUhT5PyIHDbvSzIFFMG1CMgE5EZWxL1TstbqZx87vcvTUkYZfojyKlrzR86oVUv90LEMMNAGdHScWV8iEenNIRRakF5VGOPS9PP2epICGRwCMVfKjM5m7JMk5cwOV/z151dgYKUtC3V83UoYH66z5XvdJw7o6fT6vTjGl0XWfr6bm9T8kIUhxODVblc5Jsgv6B4rqKyoYoB81SSzpW X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Oct 2019 16:29:50.7290 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 102d5da2-fcf4-4eb6-8333-08d757d63a44 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB4800 Subject: Re: [dpdk-dev] [PATCH v7 2/7] eal: add the APIs to wait until equal X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Konstantin, > -----Original Message----- > From: Gavin Hu (Arm Technology China) > Sent: Thursday, October 24, 2019 12:20 AM > To: Ananyev, Konstantin ; dev@dpdk.org > Cc: nd ; thomas@monjalon.net; > stephen@networkplumber.org; hemant.agrawal@nxp.com; > jerinj@marvell.com; pbhagavatula@marvell.com; Honnappa Nagarahalli > ; Ruifeng Wang (Arm Technology China) > ; Phil Yang (Arm Technology China) > ; Steve Capper ; nd > > Subject: RE: [dpdk-dev] [PATCH v7 2/7] eal: add the APIs to wait until eq= ual >=20 > Hi Konstantin, >=20 > > -----Original Message----- > > From: Ananyev, Konstantin > > Sent: Friday, October 18, 2019 12:44 AM > > To: Gavin Hu (Arm Technology China) ; > > dev@dpdk.org > > Cc: nd ; thomas@monjalon.net; > > stephen@networkplumber.org; hemant.agrawal@nxp.com; > > jerinj@marvell.com; pbhagavatula@marvell.com; Honnappa Nagarahalli > > ; Ruifeng Wang (Arm Technology > China) > > ; Phil Yang (Arm Technology China) > > ; Steve Capper > > Subject: RE: [dpdk-dev] [PATCH v7 2/7] eal: add the APIs to wait until = equal > > > > > > > > > > Hi Gavin, > > > > > > > > > > > The rte_wait_until_equal_xx APIs abstract the functionality of > > > > 'polling for a memory location to become equal to a given value'. > > > > > > > > Add the RTE_ARM_USE_WFE configuration entry for aarch64, disabled > > > > by default. When it is enabled, the above APIs will call WFE instru= ction > > > > to save CPU cycles and power. > > > > > > > > Signed-off-by: Gavin Hu > > > > Reviewed-by: Ruifeng Wang > > > > Reviewed-by: Steve Capper > > > > Reviewed-by: Ola Liljedahl > > > > Reviewed-by: Honnappa Nagarahalli > > > > > Reviewed-by: Phil Yang > > > > Acked-by: Pavan Nikhilesh > > > > --- > > > > config/arm/meson.build | 1 + > > > > config/common_base | 5 + > > > > .../common/include/arch/arm/rte_pause_64.h | 30 ++++++ > > > > lib/librte_eal/common/include/generic/rte_pause.h | 106 > > +++++++++++++++++++++ > > > > 4 files changed, 142 insertions(+) > > > > > > > > diff --git a/config/arm/meson.build b/config/arm/meson.build > > > > index 979018e..b4b4cac 100644 > > > > --- a/config/arm/meson.build > > > > +++ b/config/arm/meson.build > > > > @@ -26,6 +26,7 @@ flags_common_default =3D [ > > > > ['RTE_LIBRTE_AVP_PMD', false], > > > > > > > > ['RTE_SCHED_VECTOR', false], > > > > + ['RTE_ARM_USE_WFE', false], > > > > ] > > > > > > > > flags_generic =3D [ > > > > diff --git a/config/common_base b/config/common_base > > > > index 8ef75c2..8861713 100644 > > > > --- a/config/common_base > > > > +++ b/config/common_base > > > > @@ -111,6 +111,11 @@ CONFIG_RTE_MAX_VFIO_CONTAINERS=3D64 > > > > CONFIG_RTE_MALLOC_DEBUG=3Dn > > > > CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES=3Dn > > > > CONFIG_RTE_USE_LIBBSD=3Dn > > > > +# Use WFE instructions to implement the rte_wait_for_equal_xxx API= s, > > > > +# calling these APIs put the cores in low power state while waitin= g > > > > +# for the memory address to become equal to the expected value. > > > > +# This is supported only by aarch64. > > > > +CONFIG_RTE_ARM_USE_WFE=3Dn > > > > > > > > # > > > > # Recognize/ignore the AVX/AVX512 CPU flags for performance/power > > testing. > > > > diff --git a/lib/librte_eal/common/include/arch/arm/rte_pause_64.h > > b/lib/librte_eal/common/include/arch/arm/rte_pause_64.h > > > > index 93895d3..dabde17 100644 > > > > --- a/lib/librte_eal/common/include/arch/arm/rte_pause_64.h > > > > +++ b/lib/librte_eal/common/include/arch/arm/rte_pause_64.h > > > > @@ -1,5 +1,6 @@ > > > > /* SPDX-License-Identifier: BSD-3-Clause > > > > * Copyright(c) 2017 Cavium, Inc > > > > + * Copyright(c) 2019 Arm Limited > > > > */ > > > > > > > > #ifndef _RTE_PAUSE_ARM64_H_ > > > > @@ -17,6 +18,35 @@ static inline void rte_pause(void) > > > > asm volatile("yield" ::: "memory"); > > > > } > > > > > > > > +#ifdef RTE_ARM_USE_WFE > > > > +#define __WAIT_UNTIL_EQUAL(name, asm_op, wide, type) \ > > > > +static __rte_always_inline void \ > > > > +rte_wait_until_equal_##name(volatile type * addr, type expected) \ > > > > +{ \ > > > > + type tmp; \ > > > > + asm volatile( \ > > > > + #asm_op " %" #wide "[tmp], %[addr]\n" \ > > > > + "cmp %" #wide "[tmp], %" #wide "[expected]\n" \ > > > > + "b.eq 2f\n" \ > > > > + "sevl\n" \ > > > > + "1: wfe\n" \ > > > > + #asm_op " %" #wide "[tmp], %[addr]\n" \ > > > > + "cmp %" #wide "[tmp], %" #wide "[expected]\n" \ > > > > + "bne 1b\n" \ > > > > + "2:\n" \ > > > > + : [tmp] "=3D&r" (tmp) \ > > > > + : [addr] "Q"(*addr), [expected] "r"(expected) \ > > > > + : "cc", "memory"); \ > > > > +} > > > > One more thought: > > Why do you need to write asm code for the whole procedure? > > Why not to do like linux kernel: > > define wfe() and sev() macros and use them inside normal C code? > > > > #define sev() asm volatile("sev" : : : "memory") > > #define wfe() asm volatile("wfe" : : : "memory") > > > > Then: > > rte_wait_until_equal_32(volatile uint32_t *addr, uint32_t expected, int > > memorder) > > { > > if (__atomic_load_n(addr, memorder) !=3D expected) { > > sev(); > > do { > > wfe(); > > } while ((__atomic_load_n(addr, memorder) !=3D expected); > > } > > } > > > > ? > A really good suggestion, I made corresponding changes to v8 already, but= it > missed a armv8 specific feature after internal discussion. > We call wfe to wait/sleep on the 'monitored' address, it will be waken up > upon someone write to the monitor address, so before wfe, we have to call > load-exclusive instruction to 'monitor'. > __atomic_load_n - disassembled to "ldr" does not do so. We have to use > "ldxrh" for relaxed mem ordering and "ldaxrh" for acquire ordering, in > example of 16-bit. >=20 > Let me re-think coming back to the full assembly procedure or implementin= g > a 'load-exclusive' function. What do you think? > /Gavin Forgot to mention, kernel uses wfe() without preceding load-exclusive instr= uctions because: 1) it replies on the timer, to wake up, i.e. __delay() 2) explicit calling sev to send wake events, for all kinds of locks 3) IPI instructions. Our patches can't count on these events, due to of lack of these events or = performance impact.=20 /Gavin >=20 > > > > +/* Wait for *addr to be updated with expected value */ > > > > +__WAIT_UNTIL_EQUAL(relaxed_16, ldxrh, w, uint16_t) > > > > +__WAIT_UNTIL_EQUAL(acquire_16, ldaxrh, w, uint16_t) > > > > +__WAIT_UNTIL_EQUAL(relaxed_32, ldxr, w, uint32_t) > > > > +__WAIT_UNTIL_EQUAL(acquire_32, ldaxr, w, uint32_t) > > > > +__WAIT_UNTIL_EQUAL(relaxed_64, ldxr, x, uint64_t) > > > > +__WAIT_UNTIL_EQUAL(acquire_64, ldaxr, x, uint64_t) > > > > +#endif > > > > + > > > > #ifdef __cplusplus > > > > } > > > > #endif > > > > diff --git a/lib/librte_eal/common/include/generic/rte_pause.h > > b/lib/librte_eal/common/include/generic/rte_pause.h > > > > index 52bd4db..8906473 100644 > > > > --- a/lib/librte_eal/common/include/generic/rte_pause.h > > > > +++ b/lib/librte_eal/common/include/generic/rte_pause.h > > > > @@ -1,5 +1,6 @@ > > > > /* SPDX-License-Identifier: BSD-3-Clause > > > > * Copyright(c) 2017 Cavium, Inc > > > > + * Copyright(c) 2019 Arm Limited > > > > */ > > > > > > > > #ifndef _RTE_PAUSE_H_ > > > > @@ -12,6 +13,10 @@ > > > > * > > > > */ > > > > > > > > +#include > > > > +#include > > > > +#include > > > > + > > > > /** > > > > * Pause CPU execution for a short while > > > > * > > > > @@ -20,4 +25,105 @@ > > > > */ > > > > static inline void rte_pause(void); > > > > > > > > +/** > > > > + * Wait for *addr to be updated with a 16-bit expected value, with= a > > relaxed > > > > + * memory ordering model meaning the loads around this API can be > > reordered. > > > > + * > > > > + * @param addr > > > > + * A pointer to the memory location. > > > > + * @param expected > > > > + * A 16-bit expected value to be in the memory location. > > > > + */ > > > > +__rte_always_inline > > > > +static void > > > > +rte_wait_until_equal_relaxed_16(volatile uint16_t *addr, uint16_t > > expected); > > > > + > > > > +/** > > > > + * Wait for *addr to be updated with a 32-bit expected value, with= a > > relaxed > > > > + * memory ordering model meaning the loads around this API can be > > reordered. > > > > + * > > > > + * @param addr > > > > + * A pointer to the memory location. > > > > + * @param expected > > > > + * A 32-bit expected value to be in the memory location. > > > > + */ > > > > +__rte_always_inline > > > > +static void > > > > +rte_wait_until_equal_relaxed_32(volatile uint32_t *addr, uint32_t > > expected); > > > > + > > > > +/** > > > > + * Wait for *addr to be updated with a 64-bit expected value, with= a > > relaxed > > > > + * memory ordering model meaning the loads around this API can be > > reordered. > > > > + * > > > > + * @param addr > > > > + * A pointer to the memory location. > > > > + * @param expected > > > > + * A 64-bit expected value to be in the memory location. > > > > + */ > > > > +__rte_always_inline > > > > +static void > > > > +rte_wait_until_equal_relaxed_64(volatile uint64_t *addr, uint64_t > > expected); > > > > + > > > > +/** > > > > + * Wait for *addr to be updated with a 16-bit expected value, with= an > > acquire > > > > + * memory ordering model meaning the loads after this API can't be > > observed > > > > + * before this API. > > > > + * > > > > + * @param addr > > > > + * A pointer to the memory location. > > > > + * @param expected > > > > + * A 16-bit expected value to be in the memory location. > > > > + */ > > > > +__rte_always_inline > > > > +static void > > > > +rte_wait_until_equal_acquire_16(volatile uint16_t *addr, uint16_t > > expected); > > > > + > > > > +/** > > > > + * Wait for *addr to be updated with a 32-bit expected value, with= an > > acquire > > > > + * memory ordering model meaning the loads after this API can't be > > observed > > > > + * before this API. > > > > + * > > > > + * @param addr > > > > + * A pointer to the memory location. > > > > + * @param expected > > > > + * A 32-bit expected value to be in the memory location. > > > > + */ > > > > +__rte_always_inline > > > > +static void > > > > +rte_wait_until_equal_acquire_32(volatile uint32_t *addr, uint32_t > > expected); > > > > > > LGTM in general. > > > One stylish thing: wouldn't it be better to have an API like that: > > > rte_wait_until_equal_acquire_X(addr, expected, memory_order) > > > ? > > > > > > I.E. - pass memorder as parameter, not to incorporate it into functio= n > > name? > > > Less functions, plus user can specify order himself. > > > Plus looks similar to C11 atomic instrincts. > > > > > > > > > > + > > > > +/** > > > > + * Wait for *addr to be updated with a 64-bit expected value, with= an > > acquire > > > > + * memory ordering model meaning the loads after this API can't be > > observed > > > > + * before this API. > > > > + * > > > > + * @param addr > > > > + * A pointer to the memory location. > > > > + * @param expected > > > > + * A 64-bit expected value to be in the memory location. > > > > + */ > > > > +__rte_always_inline > > > > +static void > > > > +rte_wait_until_equal_acquire_64(volatile uint64_t *addr, uint64_t > > expected); > > > > + > > > > +#if !defined(RTE_ARM_USE_WFE) > > > > +#define __WAIT_UNTIL_EQUAL(op_name, size, type, memorder) \ > > > > +__rte_always_inline \ > > > > +static void \ > > > > +rte_wait_until_equal_##op_name##_##size(volatile type *addr, \ > > > > + type expected) \ > > > > +{ \ > > > > + while (__atomic_load_n(addr, memorder) !=3D expected) \ > > > > + rte_pause(); \ > > > > +} > > > > + > > > > +/* Wait for *addr to be updated with expected value */ > > > > +__WAIT_UNTIL_EQUAL(relaxed, 16, uint16_t, __ATOMIC_RELAXED) > > > > +__WAIT_UNTIL_EQUAL(acquire, 16, uint16_t, __ATOMIC_ACQUIRE) > > > > +__WAIT_UNTIL_EQUAL(relaxed, 32, uint32_t, __ATOMIC_RELAXED) > > > > +__WAIT_UNTIL_EQUAL(acquire, 32, uint32_t, __ATOMIC_ACQUIRE) > > > > +__WAIT_UNTIL_EQUAL(relaxed, 64, uint64_t, __ATOMIC_RELAXED) > > > > +__WAIT_UNTIL_EQUAL(acquire, 64, uint64_t, __ATOMIC_ACQUIRE) > > > > +#endif /* RTE_ARM_USE_WFE */ > > > > + > > > > #endif /* _RTE_PAUSE_H_ */ > > > > -- > > > > 2.7.4