From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id CA190A317C for ; Thu, 17 Oct 2019 15:15:07 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 881981C0CE; Thu, 17 Oct 2019 15:15:07 +0200 (CEST) Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by dpdk.org (Postfix) with ESMTP id B8F901C02C for ; Thu, 17 Oct 2019 15:15:05 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga107.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 17 Oct 2019 06:15:04 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.67,307,1566889200"; d="scan'208";a="186484416" Received: from irsmsx109.ger.corp.intel.com ([163.33.3.23]) by orsmga007.jf.intel.com with ESMTP; 17 Oct 2019 06:15:01 -0700 Received: from irsmsx104.ger.corp.intel.com ([169.254.5.252]) by IRSMSX109.ger.corp.intel.com ([169.254.13.52]) with mapi id 14.03.0439.000; Thu, 17 Oct 2019 14:15:00 +0100 From: "Ananyev, Konstantin" To: Gavin Hu , "dev@dpdk.org" CC: "nd@arm.com" , "thomas@monjalon.net" , "stephen@networkplumber.org" , "hemant.agrawal@nxp.com" , "jerinj@marvell.com" , "pbhagavatula@marvell.com" , "Honnappa.Nagarahalli@arm.com" , "ruifeng.wang@arm.com" , "phil.yang@arm.com" , "steve.capper@arm.com" Thread-Topic: [dpdk-dev] [PATCH v7 2/7] eal: add the APIs to wait until equal Thread-Index: AQHVdPZsIn7e7nyRGEuInrt1oftpBKde7MQg Date: Thu, 17 Oct 2019 13:14:59 +0000 Message-ID: <2601191342CEEE43887BDE71AB97725801A8C6A834@IRSMSX104.ger.corp.intel.com> References: <1561911676-37718-1-git-send-email-gavin.hu@arm.com> <1569562904-43950-3-git-send-email-gavin.hu@arm.com> In-Reply-To: <1569562904-43950-3-git-send-email-gavin.hu@arm.com> Accept-Language: en-IE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiMTQxOTdhMjUtNDdlZi00MTIxLWJhZWQtZGY3YjlhNmRmYmQzIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoiQVdSbHlyaGJCMjlTWmIxMTdQbVZHbWRxWDViaUpIMnY3Ykw1em1qOUplR0UrZTVzZVozd1ZicGwzYWJSRFVIeiJ9 x-ctpclassification: CTP_NT dlp-product: dlpe-windows dlp-version: 11.2.0.6 dlp-reaction: no-action x-originating-ip: [163.33.239.180] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH v7 2/7] eal: add the APIs to wait until equal X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Gavin, >=20 > The rte_wait_until_equal_xx APIs abstract the functionality of > 'polling for a memory location to become equal to a given value'. >=20 > Add the RTE_ARM_USE_WFE configuration entry for aarch64, disabled > by default. When it is enabled, the above APIs will call WFE instruction > to save CPU cycles and power. >=20 > Signed-off-by: Gavin Hu > Reviewed-by: Ruifeng Wang > Reviewed-by: Steve Capper > Reviewed-by: Ola Liljedahl > Reviewed-by: Honnappa Nagarahalli > Reviewed-by: Phil Yang > Acked-by: Pavan Nikhilesh > --- > config/arm/meson.build | 1 + > config/common_base | 5 + > .../common/include/arch/arm/rte_pause_64.h | 30 ++++++ > lib/librte_eal/common/include/generic/rte_pause.h | 106 +++++++++++++++= ++++++ > 4 files changed, 142 insertions(+) >=20 > diff --git a/config/arm/meson.build b/config/arm/meson.build > index 979018e..b4b4cac 100644 > --- a/config/arm/meson.build > +++ b/config/arm/meson.build > @@ -26,6 +26,7 @@ flags_common_default =3D [ > ['RTE_LIBRTE_AVP_PMD', false], >=20 > ['RTE_SCHED_VECTOR', false], > + ['RTE_ARM_USE_WFE', false], > ] >=20 > flags_generic =3D [ > diff --git a/config/common_base b/config/common_base > index 8ef75c2..8861713 100644 > --- a/config/common_base > +++ b/config/common_base > @@ -111,6 +111,11 @@ CONFIG_RTE_MAX_VFIO_CONTAINERS=3D64 > CONFIG_RTE_MALLOC_DEBUG=3Dn > CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES=3Dn > CONFIG_RTE_USE_LIBBSD=3Dn > +# Use WFE instructions to implement the rte_wait_for_equal_xxx APIs, > +# calling these APIs put the cores in low power state while waiting > +# for the memory address to become equal to the expected value. > +# This is supported only by aarch64. > +CONFIG_RTE_ARM_USE_WFE=3Dn >=20 > # > # Recognize/ignore the AVX/AVX512 CPU flags for performance/power testin= g. > diff --git a/lib/librte_eal/common/include/arch/arm/rte_pause_64.h b/lib/= librte_eal/common/include/arch/arm/rte_pause_64.h > index 93895d3..dabde17 100644 > --- a/lib/librte_eal/common/include/arch/arm/rte_pause_64.h > +++ b/lib/librte_eal/common/include/arch/arm/rte_pause_64.h > @@ -1,5 +1,6 @@ > /* SPDX-License-Identifier: BSD-3-Clause > * Copyright(c) 2017 Cavium, Inc > + * Copyright(c) 2019 Arm Limited > */ >=20 > #ifndef _RTE_PAUSE_ARM64_H_ > @@ -17,6 +18,35 @@ static inline void rte_pause(void) > asm volatile("yield" ::: "memory"); > } >=20 > +#ifdef RTE_ARM_USE_WFE > +#define __WAIT_UNTIL_EQUAL(name, asm_op, wide, type) \ > +static __rte_always_inline void \ > +rte_wait_until_equal_##name(volatile type * addr, type expected) \ > +{ \ > + type tmp; \ > + asm volatile( \ > + #asm_op " %" #wide "[tmp], %[addr]\n" \ > + "cmp %" #wide "[tmp], %" #wide "[expected]\n" \ > + "b.eq 2f\n" \ > + "sevl\n" \ > + "1: wfe\n" \ > + #asm_op " %" #wide "[tmp], %[addr]\n" \ > + "cmp %" #wide "[tmp], %" #wide "[expected]\n" \ > + "bne 1b\n" \ > + "2:\n" \ > + : [tmp] "=3D&r" (tmp) \ > + : [addr] "Q"(*addr), [expected] "r"(expected) \ > + : "cc", "memory"); \ > +} > +/* Wait for *addr to be updated with expected value */ > +__WAIT_UNTIL_EQUAL(relaxed_16, ldxrh, w, uint16_t) > +__WAIT_UNTIL_EQUAL(acquire_16, ldaxrh, w, uint16_t) > +__WAIT_UNTIL_EQUAL(relaxed_32, ldxr, w, uint32_t) > +__WAIT_UNTIL_EQUAL(acquire_32, ldaxr, w, uint32_t) > +__WAIT_UNTIL_EQUAL(relaxed_64, ldxr, x, uint64_t) > +__WAIT_UNTIL_EQUAL(acquire_64, ldaxr, x, uint64_t) > +#endif > + > #ifdef __cplusplus > } > #endif > diff --git a/lib/librte_eal/common/include/generic/rte_pause.h b/lib/libr= te_eal/common/include/generic/rte_pause.h > index 52bd4db..8906473 100644 > --- a/lib/librte_eal/common/include/generic/rte_pause.h > +++ b/lib/librte_eal/common/include/generic/rte_pause.h > @@ -1,5 +1,6 @@ > /* SPDX-License-Identifier: BSD-3-Clause > * Copyright(c) 2017 Cavium, Inc > + * Copyright(c) 2019 Arm Limited > */ >=20 > #ifndef _RTE_PAUSE_H_ > @@ -12,6 +13,10 @@ > * > */ >=20 > +#include > +#include > +#include > + > /** > * Pause CPU execution for a short while > * > @@ -20,4 +25,105 @@ > */ > static inline void rte_pause(void); >=20 > +/** > + * Wait for *addr to be updated with a 16-bit expected value, with a rel= axed > + * memory ordering model meaning the loads around this API can be reorde= red. > + * > + * @param addr > + * A pointer to the memory location. > + * @param expected > + * A 16-bit expected value to be in the memory location. > + */ > +__rte_always_inline > +static void > +rte_wait_until_equal_relaxed_16(volatile uint16_t *addr, uint16_t expect= ed); > + > +/** > + * Wait for *addr to be updated with a 32-bit expected value, with a rel= axed > + * memory ordering model meaning the loads around this API can be reorde= red. > + * > + * @param addr > + * A pointer to the memory location. > + * @param expected > + * A 32-bit expected value to be in the memory location. > + */ > +__rte_always_inline > +static void > +rte_wait_until_equal_relaxed_32(volatile uint32_t *addr, uint32_t expect= ed); > + > +/** > + * Wait for *addr to be updated with a 64-bit expected value, with a rel= axed > + * memory ordering model meaning the loads around this API can be reorde= red. > + * > + * @param addr > + * A pointer to the memory location. > + * @param expected > + * A 64-bit expected value to be in the memory location. > + */ > +__rte_always_inline > +static void > +rte_wait_until_equal_relaxed_64(volatile uint64_t *addr, uint64_t expect= ed); > + > +/** > + * Wait for *addr to be updated with a 16-bit expected value, with an ac= quire > + * memory ordering model meaning the loads after this API can't be obser= ved > + * before this API. > + * > + * @param addr > + * A pointer to the memory location. > + * @param expected > + * A 16-bit expected value to be in the memory location. > + */ > +__rte_always_inline > +static void > +rte_wait_until_equal_acquire_16(volatile uint16_t *addr, uint16_t expect= ed); > + > +/** > + * Wait for *addr to be updated with a 32-bit expected value, with an ac= quire > + * memory ordering model meaning the loads after this API can't be obser= ved > + * before this API. > + * > + * @param addr > + * A pointer to the memory location. > + * @param expected > + * A 32-bit expected value to be in the memory location. > + */ > +__rte_always_inline > +static void > +rte_wait_until_equal_acquire_32(volatile uint32_t *addr, uint32_t expect= ed); LGTM in general. One stylish thing: wouldn't it be better to have an API like that: rte_wait_until_equal_acquire_X(addr, expected, memory_order) ? I.E. - pass memorder as parameter, not to incorporate it into function name= ? Less functions, plus user can specify order himself. Plus looks similar to C11 atomic instrincts. > + > +/** > + * Wait for *addr to be updated with a 64-bit expected value, with an ac= quire > + * memory ordering model meaning the loads after this API can't be obser= ved > + * before this API. > + * > + * @param addr > + * A pointer to the memory location. > + * @param expected > + * A 64-bit expected value to be in the memory location. > + */ > +__rte_always_inline > +static void > +rte_wait_until_equal_acquire_64(volatile uint64_t *addr, uint64_t expect= ed); > + > +#if !defined(RTE_ARM_USE_WFE) > +#define __WAIT_UNTIL_EQUAL(op_name, size, type, memorder) \ > +__rte_always_inline \ > +static void \ > +rte_wait_until_equal_##op_name##_##size(volatile type *addr, \ > + type expected) \ > +{ \ > + while (__atomic_load_n(addr, memorder) !=3D expected) \ > + rte_pause(); \ > +} > + > +/* Wait for *addr to be updated with expected value */ > +__WAIT_UNTIL_EQUAL(relaxed, 16, uint16_t, __ATOMIC_RELAXED) > +__WAIT_UNTIL_EQUAL(acquire, 16, uint16_t, __ATOMIC_ACQUIRE) > +__WAIT_UNTIL_EQUAL(relaxed, 32, uint32_t, __ATOMIC_RELAXED) > +__WAIT_UNTIL_EQUAL(acquire, 32, uint32_t, __ATOMIC_ACQUIRE) > +__WAIT_UNTIL_EQUAL(relaxed, 64, uint64_t, __ATOMIC_RELAXED) > +__WAIT_UNTIL_EQUAL(acquire, 64, uint64_t, __ATOMIC_ACQUIRE) > +#endif /* RTE_ARM_USE_WFE */ > + > #endif /* _RTE_PAUSE_H_ */ > -- > 2.7.4