From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id D7FC6A32A2 for ; Thu, 24 Oct 2019 15:57:22 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id E93891EAE9; Thu, 24 Oct 2019 15:57:21 +0200 (CEST) Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by dpdk.org (Postfix) with ESMTP id E21921EAD5 for ; Thu, 24 Oct 2019 15:57:19 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga107.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 24 Oct 2019 06:57:18 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.68,224,1569308400"; d="scan'208";a="204258869" Received: from irsmsx102.ger.corp.intel.com ([163.33.3.155]) by FMSMGA003.fm.intel.com with ESMTP; 24 Oct 2019 06:57:16 -0700 Received: from irsmsx104.ger.corp.intel.com ([169.254.5.252]) by IRSMSX102.ger.corp.intel.com ([169.254.2.40]) with mapi id 14.03.0439.000; Thu, 24 Oct 2019 14:57:15 +0100 From: "Ananyev, Konstantin" To: "Ananyev, Konstantin" , Gavin Hu , "dev@dpdk.org" CC: "nd@arm.com" , "david.marchand@redhat.com" , "thomas@monjalon.net" , "stephen@networkplumber.org" , "hemant.agrawal@nxp.com" , "jerinj@marvell.com" , "pbhagavatula@marvell.com" , "Honnappa.Nagarahalli@arm.com" , "ruifeng.wang@arm.com" , "phil.yang@arm.com" , "steve.capper@arm.com" Thread-Topic: [PATCH v9 2/5] eal: add the APIs to wait until equal Thread-Index: AQHVilfZJ0AKHNOFT0CkCkMKJltek6dpySSggAAHc3A= Date: Thu, 24 Oct 2019 13:57:14 +0000 Message-ID: <2601191342CEEE43887BDE71AB97725801A8C6F8F8@IRSMSX104.ger.corp.intel.com> References: <1561911676-37718-1-git-send-email-gavin.hu@arm.com> <1571913748-51735-3-git-send-email-gavin.hu@arm.com> <2601191342CEEE43887BDE71AB97725801A8C6F8D0@IRSMSX104.ger.corp.intel.com> In-Reply-To: <2601191342CEEE43887BDE71AB97725801A8C6F8D0@IRSMSX104.ger.corp.intel.com> Accept-Language: en-IE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiNTY3NGY0MWItNGQ5NS00ZWMyLTg4YjgtZjQ0NWRhNjdmNzA2IiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoidUNqaTdoVlVRSWNFOVJ4WFowc1F2cXR2K0tEZFdCd3EyRklxWXZLSHJOSldXMURmOVNXRzF3bGxDSTdEZlpMdyJ9 x-ctpclassification: CTP_NT dlp-product: dlpe-windows dlp-version: 11.2.0.6 dlp-reaction: no-action x-originating-ip: [163.33.239.181] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH v9 2/5] eal: add the APIs to wait until equal X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" >=20 > Hi Gavin, >=20 > > The rte_wait_until_equal_xx APIs abstract the functionality of > > 'polling for a memory location to become equal to a given value'. > > > > Add the RTE_ARM_USE_WFE configuration entry for aarch64, disabled > > by default. When it is enabled, the above APIs will call WFE instructio= n > > to save CPU cycles and power. > > > > From a VM, when calling this API on aarch64, it may trap in and out to > > release vCPUs whereas cause high exit latency. Since kernel 4.18.20 an > > adaptive trapping mechanism is introduced to balance the latency and > > workload. > > > > Signed-off-by: Gavin Hu > > Reviewed-by: Ruifeng Wang > > Reviewed-by: Steve Capper > > Reviewed-by: Ola Liljedahl > > Reviewed-by: Honnappa Nagarahalli > > Reviewed-by: Phil Yang > > Acked-by: Pavan Nikhilesh > > Acked-by: Jerin Jacob > > --- > > config/arm/meson.build | 1 + > > config/common_base | 5 + > > .../common/include/arch/arm/rte_pause_64.h | 70 +++++++ > > lib/librte_eal/common/include/generic/rte_pause.h | 217 +++++++++++++= ++++++++ > > 4 files changed, 293 insertions(+) > > > > diff --git a/config/arm/meson.build b/config/arm/meson.build > > index 979018e..b4b4cac 100644 > > --- a/config/arm/meson.build > > +++ b/config/arm/meson.build > > @@ -26,6 +26,7 @@ flags_common_default =3D [ > > ['RTE_LIBRTE_AVP_PMD', false], > > > > ['RTE_SCHED_VECTOR', false], > > + ['RTE_ARM_USE_WFE', false], > > ] > > > > flags_generic =3D [ > > diff --git a/config/common_base b/config/common_base > > index e843a21..c812156 100644 > > --- a/config/common_base > > +++ b/config/common_base > > @@ -111,6 +111,11 @@ CONFIG_RTE_MAX_VFIO_CONTAINERS=3D64 > > CONFIG_RTE_MALLOC_DEBUG=3Dn > > CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES=3Dn > > CONFIG_RTE_USE_LIBBSD=3Dn > > +# Use WFE instructions to implement the rte_wait_for_equal_xxx APIs, > > +# calling these APIs put the cores in low power state while waiting > > +# for the memory address to become equal to the expected value. > > +# This is supported only by aarch64. > > +CONFIG_RTE_ARM_USE_WFE=3Dn > > > > # > > # Recognize/ignore the AVX/AVX512 CPU flags for performance/power test= ing. > > diff --git a/lib/librte_eal/common/include/arch/arm/rte_pause_64.h b/li= b/librte_eal/common/include/arch/arm/rte_pause_64.h > > index 93895d3..7bc8efb 100644 > > --- a/lib/librte_eal/common/include/arch/arm/rte_pause_64.h > > +++ b/lib/librte_eal/common/include/arch/arm/rte_pause_64.h > > @@ -1,5 +1,6 @@ > > /* SPDX-License-Identifier: BSD-3-Clause > > * Copyright(c) 2017 Cavium, Inc > > + * Copyright(c) 2019 Arm Limited > > */ > > > > #ifndef _RTE_PAUSE_ARM64_H_ > > @@ -17,6 +18,75 @@ static inline void rte_pause(void) > > asm volatile("yield" ::: "memory"); > > } > > > > +#ifdef RTE_WAIT_UNTIL_EQUAL_ARCH_DEFINED > > +static inline void rte_sevl(void) > > +{ > > + asm volatile("sevl" : : : "memory"); > > +} > > + > > +static inline void rte_wfe(void) > > +{ > > + asm volatile("wfe" : : : "memory"); > > +} > > + > > +static __rte_always_inline uint16_t > > +__atomic_load_ex_16(volatile uint16_t *addr, int memorder) > > +{ > > + uint16_t tmp; > > + assert((memorder =3D=3D __ATOMIC_ACQUIRE) > > + || (memorder =3D=3D __ATOMIC_RELAXED)); > > + if (memorder =3D=3D __ATOMIC_ACQUIRE) > > + asm volatile("ldaxrh %w[tmp], [%x[addr]]" > > + : [tmp] "=3D&r" (tmp) > > + : [addr] "r"(addr) > > + : "memory"); > > + else if (memorder =3D=3D __ATOMIC_RELAXED) > > + asm volatile("ldxrh %w[tmp], [%x[addr]]" > > + : [tmp] "=3D&r" (tmp) > > + : [addr] "r"(addr) > > + : "memory"); > > + return tmp; > > +} > > + > > +static __rte_always_inline uint32_t > > +__atomic_load_ex_32(volatile uint32_t *addr, int memorder) > > +{ > > + uint32_t tmp; > > + assert((memorder =3D=3D __ATOMIC_ACQUIRE) > > + || (memorder =3D=3D __ATOMIC_RELAXED)); > > + if (memorder =3D=3D __ATOMIC_ACQUIRE) > > + asm volatile("ldaxr %w[tmp], [%x[addr]]" > > + : [tmp] "=3D&r" (tmp) > > + : [addr] "r"(addr) > > + : "memory"); > > + else if (memorder =3D=3D __ATOMIC_RELAXED) > > + asm volatile("ldxr %w[tmp], [%x[addr]]" > > + : [tmp] "=3D&r" (tmp) > > + : [addr] "r"(addr) > > + : "memory"); > > + return tmp; > > +} > > + > > +static __rte_always_inline uint64_t > > +__atomic_load_ex_64(volatile uint64_t *addr, int memorder) > > +{ > > + uint64_t tmp; > > + assert((memorder =3D=3D __ATOMIC_ACQUIRE) > > + || (memorder =3D=3D __ATOMIC_RELAXED)); > > + if (memorder =3D=3D __ATOMIC_ACQUIRE) > > + asm volatile("ldaxr %x[tmp], [%x[addr]]" > > + : [tmp] "=3D&r" (tmp) > > + : [addr] "r"(addr) > > + : "memory"); > > + else if (memorder =3D=3D __ATOMIC_RELAXED) > > + asm volatile("ldxr %x[tmp], [%x[addr]]" > > + : [tmp] "=3D&r" (tmp) > > + : [addr] "r"(addr) > > + : "memory"); > > + return tmp; > > +} > > +#endif > > + >=20 > The function themselves seems good to me... > But I think it was some misunderstanding about code layout/placement. > I think arm specific functionsand defines need to be defined in arm spec= ific headers only. > But we still can have one instance of rte_wait_until_equal_* for arm. >=20 > To be more specific, I am talking about something like that here: >=20 > lib/librte_eal/common/include/generic/rte_pause.h: > ... > #ifndef RTE_WAIT_UNTIL_EQUAL_ARCH_DEFINED > static __rte_always_inline void > rte_wait_until_equal_32(volatile type * addr, type expected, int memorder= ) \ > { > while (__atomic_load_n(addr, memorder) !=3D expected) { > rte_pause(); \ > \ > } > .... > #endif > ... >=20 > lib/librte_eal/common/include/arch/arm/rte_pause_64.h: >=20 > ... > #ifdef RTE_ARM_USE_WFE > #define RTE_WAIT_UNTIL_EQUAL_ARCH_DEFINED > #endif > #include "generic/rte_pause.h" >=20 > ... > #ifdef RTE_ARM_USE_WFE > static inline void rte_sevl(void) > { > asm volatile("sevl" : : : "memory"); > } > static inline void rte_wfe(void) > { > asm volatile("wfe" : : : "memory"); > } > #else > static inline void rte_sevl(void) > { > } > static inline void rte_wfe(void) > { > rte_pause(); > } > ... >=20 > static __rte_always_inline void > rte_wait_until_equal_32(volatile uint32_t *addr, uint32_t expected, int m= emorder) > { > if (__atomic_load_ex_32(addr, memorder) !=3D expected) { > rte_sevl(); > do { > rte_wfe(); > } while (__atomic_load_ex_32(addr, memorder) !=3D expected); > } > } One more nit (nearly forgot): I think it is better to have rte_ (or __rte__= ) prefix for all functions defined in public files, so: __rte_atomic_load_ex_32() or just rt= e_atomic_load_ex_32(). >=20 > #endif >=20 >=20 > > #ifdef __cplusplus > > } > > #endif > > diff --git a/lib/librte_eal/common/include/generic/rte_pause.h b/lib/li= brte_eal/common/include/generic/rte_pause.h > > index 52bd4db..4db44f9 100644 > > --- a/lib/librte_eal/common/include/generic/rte_pause.h > > +++ b/lib/librte_eal/common/include/generic/rte_pause.h > > @@ -1,5 +1,6 @@ > > /* SPDX-License-Identifier: BSD-3-Clause > > * Copyright(c) 2017 Cavium, Inc > > + * Copyright(c) 2019 Arm Limited > > */ > > > > #ifndef _RTE_PAUSE_H_ > > @@ -12,6 +13,12 @@ > > * > > */ > > > > +#include > > +#include > > +#include > > +#include > > +#include > > + > > /** > > * Pause CPU execution for a short while > > * > > @@ -20,4 +27,214 @@ > > */ > > static inline void rte_pause(void); > > > > +static inline void rte_sevl(void); > > +static inline void rte_wfe(void); > > +/** > > + * @warning > > + * @b EXPERIMENTAL: this API may change, or be removed, without prior = notice > > + * > > + * Atomic load from addr, it returns the 16-bit content of *addr. > > + * > > + * @param addr > > + * A pointer to the memory location. > > + * @param memorder > > + * The valid memory order variants are __ATOMIC_ACQUIRE and __ATOMIC_= RELAXED. > > + * These map to C++11 memory orders with the same names, see the C++1= 1 standard > > + * the GCC wiki on atomic synchronization for detailed definitions. > > + */ > > +static __rte_always_inline uint16_t > > +__atomic_load_ex_16(volatile uint16_t *addr, int memorder); > > + > > +/** > > + * @warning > > + * @b EXPERIMENTAL: this API may change, or be removed, without prior = notice > > + * > > + * Atomic load from addr, it returns the 32-bit content of *addr. > > + * > > + * @param addr > > + * A pointer to the memory location. > > + * @param memorder > > + * The valid memory order variants are __ATOMIC_ACQUIRE and __ATOMIC_= RELAXED. > > + * These map to C++11 memory orders with the same names, see the C++1= 1 standard > > + * the GCC wiki on atomic synchronization for detailed definitions. > > + */ > > +static __rte_always_inline uint32_t > > +__atomic_load_ex_32(volatile uint32_t *addr, int memorder); > > + > > +/** > > + * @warning > > + * @b EXPERIMENTAL: this API may change, or be removed, without prior = notice > > + * > > + * Atomic load from addr, it returns the 64-bit content of *addr. > > + * > > + * @param addr > > + * A pointer to the memory location. > > + * @param memorder > > + * The valid memory order variants are __ATOMIC_ACQUIRE and __ATOMIC_= RELAXED. > > + * These map to C++11 memory orders with the same names, see the C++1= 1 standard > > + * the GCC wiki on atomic synchronization for detailed definitions. > > + */ > > +static __rte_always_inline uint64_t > > +__atomic_load_ex_64(volatile uint64_t *addr, int memorder); > > + > > +/** > > + * @warning > > + * @b EXPERIMENTAL: this API may change, or be removed, without prior = notice > > + * > > + * Wait for *addr to be updated with a 16-bit expected value, with a r= elaxed > > + * memory ordering model meaning the loads around this API can be reor= dered. > > + * > > + * @param addr > > + * A pointer to the memory location. > > + * @param expected > > + * A 16-bit expected value to be in the memory location. > > + * @param memorder > > + * Two different memory orders that can be specified: > > + * __ATOMIC_ACQUIRE and __ATOMIC_RELAXED. These map to > > + * C++11 memory orders with the same names, see the C++11 standard or > > + * the GCC wiki on atomic synchronization for detailed definition. > > + */ > > +__rte_experimental > > +static __rte_always_inline void > > +rte_wait_until_equal_16(volatile uint16_t *addr, uint16_t expected, > > +int memorder); > > + > > +/** > > + * @warning > > + * @b EXPERIMENTAL: this API may change, or be removed, without prior = notice > > + * > > + * Wait for *addr to be updated with a 32-bit expected value, with a r= elaxed > > + * memory ordering model meaning the loads around this API can be reor= dered. > > + * > > + * @param addr > > + * A pointer to the memory location. > > + * @param expected > > + * A 32-bit expected value to be in the memory location. > > + * @param memorder > > + * Two different memory orders that can be specified: > > + * __ATOMIC_ACQUIRE and __ATOMIC_RELAXED. These map to > > + * C++11 memory orders with the same names, see the C++11 standard or > > + * the GCC wiki on atomic synchronization for detailed definition. > > + */ > > +__rte_experimental > > +static __rte_always_inline void > > +rte_wait_until_equal_32(volatile uint32_t *addr, uint32_t expected, > > +int memorder); > > + > > +/** > > + * @warning > > + * @b EXPERIMENTAL: this API may change, or be removed, without prior = notice > > + * > > + * Wait for *addr to be updated with a 64-bit expected value, with a r= elaxed > > + * memory ordering model meaning the loads around this API can be reor= dered. > > + * > > + * @param addr > > + * A pointer to the memory location. > > + * @param expected > > + * A 64-bit expected value to be in the memory location. > > + * @param memorder > > + * Two different memory orders that can be specified: > > + * __ATOMIC_ACQUIRE and __ATOMIC_RELAXED. These map to > > + * C++11 memory orders with the same names, see the C++11 standard or > > + * the GCC wiki on atomic synchronization for detailed definition. > > + */ > > +__rte_experimental > > +static __rte_always_inline void > > +rte_wait_until_equal_64(volatile uint64_t *addr, uint64_t expected, > > +int memorder); > > + > > +#ifdef RTE_ARM_USE_WFE > > +#define RTE_WAIT_UNTIL_EQUAL_ARCH_DEFINED > > +#endif > > + > > +#ifndef RTE_WAIT_UNTIL_EQUAL_ARCH_DEFINED > > +static inline void rte_sevl(void) > > +{ > > +} > > + > > +static inline void rte_wfe(void) > > +{ > > + rte_pause(); > > +} > > + > > +/** > > + * @warning > > + * @b EXPERIMENTAL: this API may change, or be removed, without prior = notice > > + * > > + * Atomic load from addr, it returns the 16-bit content of *addr. > > + * > > + * @param addr > > + * A pointer to the memory location. > > + * @param memorder > > + * The valid memory order variants are __ATOMIC_ACQUIRE and __ATOMIC_= RELAXED. > > + * These map to C++11 memory orders with the same names, see the C++1= 1 standard > > + * the GCC wiki on atomic synchronization for detailed definitions. > > + */ > > +static __rte_always_inline uint16_t > > +__atomic_load_ex_16(volatile uint16_t *addr, int memorder) > > +{ > > + uint16_t tmp; > > + assert((memorder =3D=3D __ATOMIC_ACQUIRE) > > + || (memorder =3D=3D __ATOMIC_RELAXED)); > > + tmp =3D __atomic_load_n(addr, memorder); > > + return tmp; > > +} > > + > > +static __rte_always_inline uint32_t > > +__atomic_load_ex_32(volatile uint32_t *addr, int memorder) > > +{ > > + uint32_t tmp; > > + assert((memorder =3D=3D __ATOMIC_ACQUIRE) > > + || (memorder =3D=3D __ATOMIC_RELAXED)); > > + tmp =3D __atomic_load_n(addr, memorder); > > + return tmp; > > +} > > + > > +static __rte_always_inline uint64_t > > +__atomic_load_ex_64(volatile uint64_t *addr, int memorder) > > +{ > > + uint64_t tmp; > > + assert((memorder =3D=3D __ATOMIC_ACQUIRE) > > + || (memorder =3D=3D __ATOMIC_RELAXED)); > > + tmp =3D __atomic_load_n(addr, memorder); > > + return tmp; > > +} > > + > > +static __rte_always_inline void > > +rte_wait_until_equal_16(volatile uint16_t *addr, uint16_t expected, > > +int memorder) > > +{ > > + if (__atomic_load_n(addr, memorder) !=3D expected) { > > + rte_sevl(); > > + do { > > + rte_wfe(); > > + } while (__atomic_load_ex_16(addr, memorder) !=3D expected); > > + } > > +} > > + > > +static __rte_always_inline void > > +rte_wait_until_equal_32(volatile uint32_t *addr, uint32_t expected, > > +int memorder) > > +{ > > + if (__atomic_load_ex_32(addr, memorder) !=3D expected) { > > + rte_sevl(); > > + do { > > + rte_wfe(); > > + } while (__atomic_load_ex_32(addr, memorder) !=3D expected); > > + } > > +} > > + > > +static __rte_always_inline void > > +rte_wait_until_equal_64(volatile uint64_t *addr, uint64_t expected, > > +int memorder) > > +{ > > + if (__atomic_load_ex_64(addr, memorder) !=3D expected) { > > + rte_sevl(); > > + do { > > + rte_wfe(); > > + } while (__atomic_load_ex_64(addr, memorder) !=3D expected); > > + } > > +} > > +#endif > > + > > #endif /* _RTE_PAUSE_H_ */ > > -- > > 2.7.4