From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0FB40A317C for ; Thu, 17 Oct 2019 18:44:18 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 5D2551E9C1; Thu, 17 Oct 2019 18:44:17 +0200 (CEST) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id BD20B1E915 for ; Thu, 17 Oct 2019 18:44:15 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 17 Oct 2019 09:44:14 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.67,308,1566889200"; d="scan'208";a="221438835" Received: from irsmsx109.ger.corp.intel.com ([163.33.3.23]) by fmsmga004.fm.intel.com with ESMTP; 17 Oct 2019 09:44:10 -0700 Received: from irsmsx104.ger.corp.intel.com ([169.254.5.252]) by IRSMSX109.ger.corp.intel.com ([169.254.13.52]) with mapi id 14.03.0439.000; Thu, 17 Oct 2019 17:44:11 +0100 From: "Ananyev, Konstantin" To: Gavin Hu , "dev@dpdk.org" CC: "nd@arm.com" , "thomas@monjalon.net" , "stephen@networkplumber.org" , "hemant.agrawal@nxp.com" , "jerinj@marvell.com" , "pbhagavatula@marvell.com" , "Honnappa.Nagarahalli@arm.com" , "ruifeng.wang@arm.com" , "phil.yang@arm.com" , "steve.capper@arm.com" Thread-Topic: [dpdk-dev] [PATCH v7 2/7] eal: add the APIs to wait until equal Thread-Index: AQHVdPZsIn7e7nyRGEuInrt1oftpBKde7MQggAA7nkA= Date: Thu, 17 Oct 2019 16:44:10 +0000 Message-ID: <2601191342CEEE43887BDE71AB97725801A8C6AB30@IRSMSX104.ger.corp.intel.com> References: <1561911676-37718-1-git-send-email-gavin.hu@arm.com> <1569562904-43950-3-git-send-email-gavin.hu@arm.com> Accept-Language: en-IE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiMTQxOTdhMjUtNDdlZi00MTIxLWJhZWQtZGY3YjlhNmRmYmQzIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoiQVdSbHlyaGJCMjlTWmIxMTdQbVZHbWRxWDViaUpIMnY3Ykw1em1qOUplR0UrZTVzZVozd1ZicGwzYWJSRFVIeiJ9 x-ctpclassification: CTP_NT dlp-product: dlpe-windows dlp-version: 11.2.0.6 dlp-reaction: no-action x-originating-ip: [163.33.239.180] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH v7 2/7] eal: add the APIs to wait until equal X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" >=20 > Hi Gavin, >=20 > > > > The rte_wait_until_equal_xx APIs abstract the functionality of > > 'polling for a memory location to become equal to a given value'. > > > > Add the RTE_ARM_USE_WFE configuration entry for aarch64, disabled > > by default. When it is enabled, the above APIs will call WFE instructio= n > > to save CPU cycles and power. > > > > Signed-off-by: Gavin Hu > > Reviewed-by: Ruifeng Wang > > Reviewed-by: Steve Capper > > Reviewed-by: Ola Liljedahl > > Reviewed-by: Honnappa Nagarahalli > > Reviewed-by: Phil Yang > > Acked-by: Pavan Nikhilesh > > --- > > config/arm/meson.build | 1 + > > config/common_base | 5 + > > .../common/include/arch/arm/rte_pause_64.h | 30 ++++++ > > lib/librte_eal/common/include/generic/rte_pause.h | 106 +++++++++++++= ++++++++ > > 4 files changed, 142 insertions(+) > > > > diff --git a/config/arm/meson.build b/config/arm/meson.build > > index 979018e..b4b4cac 100644 > > --- a/config/arm/meson.build > > +++ b/config/arm/meson.build > > @@ -26,6 +26,7 @@ flags_common_default =3D [ > > ['RTE_LIBRTE_AVP_PMD', false], > > > > ['RTE_SCHED_VECTOR', false], > > + ['RTE_ARM_USE_WFE', false], > > ] > > > > flags_generic =3D [ > > diff --git a/config/common_base b/config/common_base > > index 8ef75c2..8861713 100644 > > --- a/config/common_base > > +++ b/config/common_base > > @@ -111,6 +111,11 @@ CONFIG_RTE_MAX_VFIO_CONTAINERS=3D64 > > CONFIG_RTE_MALLOC_DEBUG=3Dn > > CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES=3Dn > > CONFIG_RTE_USE_LIBBSD=3Dn > > +# Use WFE instructions to implement the rte_wait_for_equal_xxx APIs, > > +# calling these APIs put the cores in low power state while waiting > > +# for the memory address to become equal to the expected value. > > +# This is supported only by aarch64. > > +CONFIG_RTE_ARM_USE_WFE=3Dn > > > > # > > # Recognize/ignore the AVX/AVX512 CPU flags for performance/power test= ing. > > diff --git a/lib/librte_eal/common/include/arch/arm/rte_pause_64.h b/li= b/librte_eal/common/include/arch/arm/rte_pause_64.h > > index 93895d3..dabde17 100644 > > --- a/lib/librte_eal/common/include/arch/arm/rte_pause_64.h > > +++ b/lib/librte_eal/common/include/arch/arm/rte_pause_64.h > > @@ -1,5 +1,6 @@ > > /* SPDX-License-Identifier: BSD-3-Clause > > * Copyright(c) 2017 Cavium, Inc > > + * Copyright(c) 2019 Arm Limited > > */ > > > > #ifndef _RTE_PAUSE_ARM64_H_ > > @@ -17,6 +18,35 @@ static inline void rte_pause(void) > > asm volatile("yield" ::: "memory"); > > } > > > > +#ifdef RTE_ARM_USE_WFE > > +#define __WAIT_UNTIL_EQUAL(name, asm_op, wide, type) \ > > +static __rte_always_inline void \ > > +rte_wait_until_equal_##name(volatile type * addr, type expected) \ > > +{ \ > > + type tmp; \ > > + asm volatile( \ > > + #asm_op " %" #wide "[tmp], %[addr]\n" \ > > + "cmp %" #wide "[tmp], %" #wide "[expected]\n" \ > > + "b.eq 2f\n" \ > > + "sevl\n" \ > > + "1: wfe\n" \ > > + #asm_op " %" #wide "[tmp], %[addr]\n" \ > > + "cmp %" #wide "[tmp], %" #wide "[expected]\n" \ > > + "bne 1b\n" \ > > + "2:\n" \ > > + : [tmp] "=3D&r" (tmp) \ > > + : [addr] "Q"(*addr), [expected] "r"(expected) \ > > + : "cc", "memory"); \ > > +} One more thought: Why do you need to write asm code for the whole procedure? Why not to do like linux kernel: define wfe() and sev() macros and use them inside normal C code? =20 #define sev() asm volatile("sev" : : : "memory") #define wfe() asm volatile("wfe" : : : "memory") Then: rte_wait_until_equal_32(volatile uint32_t *addr, uint32_t expected, int mem= order) { if (__atomic_load_n(addr, memorder) !=3D expected) { sev(); do { wfe(); } while ((__atomic_load_n(addr, memorder) !=3D expected); } } ? > > +/* Wait for *addr to be updated with expected value */ > > +__WAIT_UNTIL_EQUAL(relaxed_16, ldxrh, w, uint16_t) > > +__WAIT_UNTIL_EQUAL(acquire_16, ldaxrh, w, uint16_t) > > +__WAIT_UNTIL_EQUAL(relaxed_32, ldxr, w, uint32_t) > > +__WAIT_UNTIL_EQUAL(acquire_32, ldaxr, w, uint32_t) > > +__WAIT_UNTIL_EQUAL(relaxed_64, ldxr, x, uint64_t) > > +__WAIT_UNTIL_EQUAL(acquire_64, ldaxr, x, uint64_t) > > +#endif > > + > > #ifdef __cplusplus > > } > > #endif > > diff --git a/lib/librte_eal/common/include/generic/rte_pause.h b/lib/li= brte_eal/common/include/generic/rte_pause.h > > index 52bd4db..8906473 100644 > > --- a/lib/librte_eal/common/include/generic/rte_pause.h > > +++ b/lib/librte_eal/common/include/generic/rte_pause.h > > @@ -1,5 +1,6 @@ > > /* SPDX-License-Identifier: BSD-3-Clause > > * Copyright(c) 2017 Cavium, Inc > > + * Copyright(c) 2019 Arm Limited > > */ > > > > #ifndef _RTE_PAUSE_H_ > > @@ -12,6 +13,10 @@ > > * > > */ > > > > +#include > > +#include > > +#include > > + > > /** > > * Pause CPU execution for a short while > > * > > @@ -20,4 +25,105 @@ > > */ > > static inline void rte_pause(void); > > > > +/** > > + * Wait for *addr to be updated with a 16-bit expected value, with a r= elaxed > > + * memory ordering model meaning the loads around this API can be reor= dered. > > + * > > + * @param addr > > + * A pointer to the memory location. > > + * @param expected > > + * A 16-bit expected value to be in the memory location. > > + */ > > +__rte_always_inline > > +static void > > +rte_wait_until_equal_relaxed_16(volatile uint16_t *addr, uint16_t expe= cted); > > + > > +/** > > + * Wait for *addr to be updated with a 32-bit expected value, with a r= elaxed > > + * memory ordering model meaning the loads around this API can be reor= dered. > > + * > > + * @param addr > > + * A pointer to the memory location. > > + * @param expected > > + * A 32-bit expected value to be in the memory location. > > + */ > > +__rte_always_inline > > +static void > > +rte_wait_until_equal_relaxed_32(volatile uint32_t *addr, uint32_t expe= cted); > > + > > +/** > > + * Wait for *addr to be updated with a 64-bit expected value, with a r= elaxed > > + * memory ordering model meaning the loads around this API can be reor= dered. > > + * > > + * @param addr > > + * A pointer to the memory location. > > + * @param expected > > + * A 64-bit expected value to be in the memory location. > > + */ > > +__rte_always_inline > > +static void > > +rte_wait_until_equal_relaxed_64(volatile uint64_t *addr, uint64_t expe= cted); > > + > > +/** > > + * Wait for *addr to be updated with a 16-bit expected value, with an = acquire > > + * memory ordering model meaning the loads after this API can't be obs= erved > > + * before this API. > > + * > > + * @param addr > > + * A pointer to the memory location. > > + * @param expected > > + * A 16-bit expected value to be in the memory location. > > + */ > > +__rte_always_inline > > +static void > > +rte_wait_until_equal_acquire_16(volatile uint16_t *addr, uint16_t expe= cted); > > + > > +/** > > + * Wait for *addr to be updated with a 32-bit expected value, with an = acquire > > + * memory ordering model meaning the loads after this API can't be obs= erved > > + * before this API. > > + * > > + * @param addr > > + * A pointer to the memory location. > > + * @param expected > > + * A 32-bit expected value to be in the memory location. > > + */ > > +__rte_always_inline > > +static void > > +rte_wait_until_equal_acquire_32(volatile uint32_t *addr, uint32_t expe= cted); >=20 > LGTM in general. > One stylish thing: wouldn't it be better to have an API like that: > rte_wait_until_equal_acquire_X(addr, expected, memory_order) > ? >=20 > I.E. - pass memorder as parameter, not to incorporate it into function na= me? > Less functions, plus user can specify order himself. > Plus looks similar to C11 atomic instrincts. >=20 >=20 > > + > > +/** > > + * Wait for *addr to be updated with a 64-bit expected value, with an = acquire > > + * memory ordering model meaning the loads after this API can't be obs= erved > > + * before this API. > > + * > > + * @param addr > > + * A pointer to the memory location. > > + * @param expected > > + * A 64-bit expected value to be in the memory location. > > + */ > > +__rte_always_inline > > +static void > > +rte_wait_until_equal_acquire_64(volatile uint64_t *addr, uint64_t expe= cted); > > + > > +#if !defined(RTE_ARM_USE_WFE) > > +#define __WAIT_UNTIL_EQUAL(op_name, size, type, memorder) \ > > +__rte_always_inline \ > > +static void \ > > +rte_wait_until_equal_##op_name##_##size(volatile type *addr, \ > > + type expected) \ > > +{ \ > > + while (__atomic_load_n(addr, memorder) !=3D expected) \ > > + rte_pause(); \ > > +} > > + > > +/* Wait for *addr to be updated with expected value */ > > +__WAIT_UNTIL_EQUAL(relaxed, 16, uint16_t, __ATOMIC_RELAXED) > > +__WAIT_UNTIL_EQUAL(acquire, 16, uint16_t, __ATOMIC_ACQUIRE) > > +__WAIT_UNTIL_EQUAL(relaxed, 32, uint32_t, __ATOMIC_RELAXED) > > +__WAIT_UNTIL_EQUAL(acquire, 32, uint32_t, __ATOMIC_ACQUIRE) > > +__WAIT_UNTIL_EQUAL(relaxed, 64, uint64_t, __ATOMIC_RELAXED) > > +__WAIT_UNTIL_EQUAL(acquire, 64, uint64_t, __ATOMIC_ACQUIRE) > > +#endif /* RTE_ARM_USE_WFE */ > > + > > #endif /* _RTE_PAUSE_H_ */ > > -- > > 2.7.4