From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 811FAA0528; Fri, 17 Jul 2020 06:37:44 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 05A7F1BEBC; Fri, 17 Jul 2020 06:37:43 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by dpdk.org (Postfix) with ESMTP id DE5F01BEB4 for ; Fri, 17 Jul 2020 06:37:40 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 31215D6E; Thu, 16 Jul 2020 21:37:40 -0700 (PDT) Received: from phil-VirtualBox.shanghai.arm.com (phil-VirtualBox.shanghai.arm.com [10.169.108.167]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 0C1553F66E; Thu, 16 Jul 2020 21:37:36 -0700 (PDT) From: Phil Yang To: david.marchand@redhat.com, dev@dpdk.org Cc: olivier.matz@6wind.com, stephen@networkplumber.org, drc@linux.vnet.ibm.com, Honnappa.Nagarahalli@arm.com, Ruifeng.Wang@arm.com, nd@arm.com, Ray Kinsella , Neil Horman Date: Fri, 17 Jul 2020 12:36:50 +0800 Message-Id: <1594960611-19470-1-git-send-email-phil.yang@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1594310331-23345-1-git-send-email-phil.yang@arm.com> References: <1594310331-23345-1-git-send-email-phil.yang@arm.com> Subject: [dpdk-dev] [PATCH v5 1/2] mbuf: use C11 atomic builtins for refcnt operations X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Use C11 atomic builtins with explicit ordering instead of rte_atomic ops which enforce unnecessary barriers on aarch64. Suggested-by: Olivier Matz Suggested-by: Dodji Seketeli Signed-off-by: Phil Yang Reviewed-by: Ruifeng Wang Acked-by: Olivier Matz --- v5: 1. Change built-ins to builtins. 2. Ignore updates of rte_mbuf_ext_shared_info refcnt_atomic in ABI checker. v4: 1. Add union for refcnt_atomic and refcnt in rte_mbuf_ext_shared_info to avoid ABI breakage. (Olivier) 2. Add notice of refcnt_atomic deprecation. (Honnappa) v3: 1.Fix ABI breakage. 2.Simplify data type cast. v2: Fix ABI issue: revert the rte_mbuf_ext_shared_info struct refcnt field to refcnt_atomic. devtools/libabigail.abignore | 4 ++++ lib/librte_mbuf/rte_mbuf.c | 1 - lib/librte_mbuf/rte_mbuf.h | 19 ++++++++++--------- lib/librte_mbuf/rte_mbuf_core.h | 6 +++++- 4 files changed, 19 insertions(+), 11 deletions(-) diff --git a/devtools/libabigail.abignore b/devtools/libabigail.abignore index daa4631..9fea822 100644 --- a/devtools/libabigail.abignore +++ b/devtools/libabigail.abignore @@ -52,6 +52,10 @@ [suppress_type] type_kind = struct name = rte_epoll_event +; Ignore updates of rte_mbuf_ext_shared_info refcnt_atomic +[suppress_type] + name = rte_mbuf_ext_shared_info + has_data_member_inserted_between = {offset_of(refcnt_atomic), offset_of(refcnt_atomic)} ;;;;;;;;;;;;;;;;;;;;;; ; Temporary exceptions till DPDK 20.11 diff --git a/lib/librte_mbuf/rte_mbuf.c b/lib/librte_mbuf/rte_mbuf.c index ae91ae2..8a456e5 100644 --- a/lib/librte_mbuf/rte_mbuf.c +++ b/lib/librte_mbuf/rte_mbuf.c @@ -22,7 +22,6 @@ #include #include #include -#include #include #include #include diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h index f8e492e..7259575 100644 --- a/lib/librte_mbuf/rte_mbuf.h +++ b/lib/librte_mbuf/rte_mbuf.h @@ -37,7 +37,6 @@ #include #include #include -#include #include #include #include @@ -365,7 +364,7 @@ rte_pktmbuf_priv_flags(struct rte_mempool *mp) static inline uint16_t rte_mbuf_refcnt_read(const struct rte_mbuf *m) { - return (uint16_t)(rte_atomic16_read(&m->refcnt_atomic)); + return __atomic_load_n(&m->refcnt, __ATOMIC_RELAXED); } /** @@ -378,14 +377,15 @@ rte_mbuf_refcnt_read(const struct rte_mbuf *m) static inline void rte_mbuf_refcnt_set(struct rte_mbuf *m, uint16_t new_value) { - rte_atomic16_set(&m->refcnt_atomic, (int16_t)new_value); + __atomic_store_n(&m->refcnt, new_value, __ATOMIC_RELAXED); } /* internal */ static inline uint16_t __rte_mbuf_refcnt_update(struct rte_mbuf *m, int16_t value) { - return (uint16_t)(rte_atomic16_add_return(&m->refcnt_atomic, value)); + return __atomic_add_fetch(&m->refcnt, (uint16_t)value, + __ATOMIC_ACQ_REL); } /** @@ -466,7 +466,7 @@ rte_mbuf_refcnt_set(struct rte_mbuf *m, uint16_t new_value) static inline uint16_t rte_mbuf_ext_refcnt_read(const struct rte_mbuf_ext_shared_info *shinfo) { - return (uint16_t)(rte_atomic16_read(&shinfo->refcnt_atomic)); + return __atomic_load_n(&shinfo->refcnt, __ATOMIC_RELAXED); } /** @@ -481,7 +481,7 @@ static inline void rte_mbuf_ext_refcnt_set(struct rte_mbuf_ext_shared_info *shinfo, uint16_t new_value) { - rte_atomic16_set(&shinfo->refcnt_atomic, (int16_t)new_value); + __atomic_store_n(&shinfo->refcnt, new_value, __ATOMIC_RELAXED); } /** @@ -505,7 +505,8 @@ rte_mbuf_ext_refcnt_update(struct rte_mbuf_ext_shared_info *shinfo, return (uint16_t)value; } - return (uint16_t)rte_atomic16_add_return(&shinfo->refcnt_atomic, value); + return __atomic_add_fetch(&shinfo->refcnt, (uint16_t)value, + __ATOMIC_ACQ_REL); } /** Mbuf prefetch */ @@ -1304,8 +1305,8 @@ static inline int __rte_pktmbuf_pinned_extbuf_decref(struct rte_mbuf *m) * Direct usage of add primitive to avoid * duplication of comparing with one. */ - if (likely(rte_atomic16_add_return - (&shinfo->refcnt_atomic, -1))) + if (likely(__atomic_add_fetch(&shinfo->refcnt, (uint16_t)-1, + __ATOMIC_ACQ_REL))) return 1; /* Reinitialize counter before mbuf freeing. */ diff --git a/lib/librte_mbuf/rte_mbuf_core.h b/lib/librte_mbuf/rte_mbuf_core.h index 16600f1..8cd7137 100644 --- a/lib/librte_mbuf/rte_mbuf_core.h +++ b/lib/librte_mbuf/rte_mbuf_core.h @@ -679,7 +679,11 @@ typedef void (*rte_mbuf_extbuf_free_callback_t)(void *addr, void *opaque); struct rte_mbuf_ext_shared_info { rte_mbuf_extbuf_free_callback_t free_cb; /**< Free callback function */ void *fcb_opaque; /**< Free callback argument */ - rte_atomic16_t refcnt_atomic; /**< Atomically accessed refcnt */ + RTE_STD_C11 + union { + rte_atomic16_t refcnt_atomic; /**< Atomically accessed refcnt */ + uint16_t refcnt; + }; }; /**< Maximum number of nb_segs allowed. */ -- 2.7.4