From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 64C26A0540; Mon, 13 Jul 2020 08:24:27 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 995391D167; Mon, 13 Jul 2020 08:24:18 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by dpdk.org (Postfix) with ESMTP id A1B861C1BB for ; Mon, 13 Jul 2020 08:24:17 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1D6D01FB; Sun, 12 Jul 2020 23:24:17 -0700 (PDT) Received: from phil-VirtualBox.shanghai.arm.com (phil-VirtualBox.shanghai.arm.com [10.169.108.144]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id D8F2A3F887; Sun, 12 Jul 2020 23:24:12 -0700 (PDT) From: Phil Yang To: thomas@monjalon.net, john.mcnamara@intel.com, Honnappa.Nagarahalli@arm.com, drc@linux.vnet.ibm.com, dev@dpdk.org Cc: david.marchand@redhat.com, jerinj@marvell.com, konstantin.ananyev@intel.com, Ola.Liljedahl@arm.com, bruce.richardson@intel.com, Ruifeng.Wang@arm.com, nd@arm.com, Jan Viktorin , Ruifeng Wang Date: Mon, 13 Jul 2020 14:23:42 +0800 Message-Id: <1594621423-14796-4-git-send-email-phil.yang@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1594621423-14796-1-git-send-email-phil.yang@arm.com> References: <1594115449-13750-1-git-send-email-phil.yang@arm.com> <1594621423-14796-1-git-send-email-phil.yang@arm.com> Subject: [dpdk-dev] [PATCH v7 3/3] eal/atomic: add wrapper for C11 atomic thread fence X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Provide a wrapper for __atomic_thread_fence built-in to support optimized code for __ATOMIC_SEQ_CST memory order for x86 platforms. Suggested-by: Honnappa Nagarahalli Signed-off-by: Phil Yang Reviewed-by: Ola Liljedahl Acked-by: Konstantin Ananyev --- lib/librte_eal/arm/include/rte_atomic_32.h | 6 ++++++ lib/librte_eal/arm/include/rte_atomic_64.h | 6 ++++++ lib/librte_eal/include/generic/rte_atomic.h | 6 ++++++ lib/librte_eal/ppc/include/rte_atomic.h | 6 ++++++ lib/librte_eal/x86/include/rte_atomic.h | 17 +++++++++++++++++ 5 files changed, 41 insertions(+) diff --git a/lib/librte_eal/arm/include/rte_atomic_32.h b/lib/librte_eal/arm/include/rte_atomic_32.h index 7dc0d06..dbe7cc6 100644 --- a/lib/librte_eal/arm/include/rte_atomic_32.h +++ b/lib/librte_eal/arm/include/rte_atomic_32.h @@ -37,6 +37,12 @@ extern "C" { #define rte_cio_rmb() rte_rmb() +static __rte_always_inline void +rte_atomic_thread_fence(int mo) +{ + __atomic_thread_fence(mo); +} + #ifdef __cplusplus } #endif diff --git a/lib/librte_eal/arm/include/rte_atomic_64.h b/lib/librte_eal/arm/include/rte_atomic_64.h index e42f69e..bf885ad 100644 --- a/lib/librte_eal/arm/include/rte_atomic_64.h +++ b/lib/librte_eal/arm/include/rte_atomic_64.h @@ -41,6 +41,12 @@ extern "C" { #define rte_cio_rmb() rte_rmb() +static __rte_always_inline void +rte_atomic_thread_fence(int mo) +{ + __atomic_thread_fence(mo); +} + /*------------------------ 128 bit atomic operations -------------------------*/ #if defined(__ARM_FEATURE_ATOMICS) || defined(RTE_ARM_FEATURE_ATOMICS) diff --git a/lib/librte_eal/include/generic/rte_atomic.h b/lib/librte_eal/include/generic/rte_atomic.h index e6ab15a..5b941db 100644 --- a/lib/librte_eal/include/generic/rte_atomic.h +++ b/lib/librte_eal/include/generic/rte_atomic.h @@ -158,6 +158,12 @@ static inline void rte_cio_rmb(void); asm volatile ("" : : : "memory"); \ } while(0) +/** + * Synchronization fence between threads based on the specified + * memory order. + */ +static inline void rte_atomic_thread_fence(int mo); + /*------------------------- 16 bit atomic operations -------------------------*/ /** diff --git a/lib/librte_eal/ppc/include/rte_atomic.h b/lib/librte_eal/ppc/include/rte_atomic.h index 7e3e131..91c5f30 100644 --- a/lib/librte_eal/ppc/include/rte_atomic.h +++ b/lib/librte_eal/ppc/include/rte_atomic.h @@ -40,6 +40,12 @@ extern "C" { #define rte_cio_rmb() rte_rmb() +static __rte_always_inline void +rte_atomic_thread_fence(int mo) +{ + __atomic_thread_fence(mo); +} + /*------------------------- 16 bit atomic operations -------------------------*/ /* To be compatible with Power7, use GCC built-in functions for 16 bit * operations */ diff --git a/lib/librte_eal/x86/include/rte_atomic.h b/lib/librte_eal/x86/include/rte_atomic.h index b9dcd30..bd256e7 100644 --- a/lib/librte_eal/x86/include/rte_atomic.h +++ b/lib/librte_eal/x86/include/rte_atomic.h @@ -83,6 +83,23 @@ rte_smp_mb(void) #define rte_cio_rmb() rte_compiler_barrier() +/** + * Synchronization fence between threads based on the specified + * memory order. + * + * On x86 the __atomic_thread_fence(__ATOMIC_SEQ_CST) generates + * full 'mfence' which is quite expensive. The optimized + * implementation of rte_smp_mb is used instead. + */ +static __rte_always_inline void +rte_atomic_thread_fence(int mo) +{ + if (mo == __ATOMIC_SEQ_CST) + rte_smp_mb(); + else + __atomic_thread_fence(mo); +} + /*------------------------- 16 bit atomic operations -------------------------*/ #ifndef RTE_FORCE_INTRINSICS -- 2.7.4