From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
	by inbox.dpdk.org (Postfix) with ESMTP id 0402C44024;
	Tue, 14 May 2024 18:39:00 +0200 (CEST)
Received: from mails.dpdk.org (localhost [127.0.0.1])
	by mails.dpdk.org (Postfix) with ESMTP id 51C3E410E8;
	Tue, 14 May 2024 18:36:36 +0200 (CEST)
Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182])
 by mails.dpdk.org (Postfix) with ESMTP id 422BB406FF
 for <dev@dpdk.org>; Tue, 14 May 2024 18:36:06 +0200 (CEST)
Received: by linux.microsoft.com (Postfix, from userid 1086)
 id A8DB420B2C9E; Tue, 14 May 2024 09:36:00 -0700 (PDT)
DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com A8DB420B2C9E
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com;
 s=default; t=1715704561;
 bh=pcK/itH1S3yVJp7OnuITB1MwqY//rn9dHPNOayUftj8=;
 h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
 b=fTPsXPsoKAzIch2rY9Yn82VkoWxIcS0CMJsuTtXGw4GAiCuLexvjEzzmX6wmByyd5
 iLK/B1+aj/ZMJe5VdhBm0IAAtsP3zPYfOhT3/+I8qdP+la1WXiGsuy2vGBXL5kbUTf
 IShirw5Gie7SoFk1lUOzZMAbYCU2jgAWTsKNfblM=
From: Tyler Retzlaff <roretzla@linux.microsoft.com>
To: dev@dpdk.org
Cc: =?UTF-8?q?Mattias=20R=C3=B6nnblom?= <mattias.ronnblom@ericsson.com>,
 =?UTF-8?q?Morten=20Br=C3=B8rup?= <mb@smartsharesystems.com>,
 Abdullah Sevincer <abdullah.sevincer@intel.com>,
 Ajit Khaparde <ajit.khaparde@broadcom.com>,
 Alok Prasad <palok@marvell.com>,
 Anatoly Burakov <anatoly.burakov@intel.com>,
 Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>,
 Anoob Joseph <anoobj@marvell.com>,
 Bruce Richardson <bruce.richardson@intel.com>,
 Byron Marohn <byron.marohn@intel.com>, Chenbo Xia <chenbox@nvidia.com>,
 Chengwen Feng <fengchengwen@huawei.com>,
 Ciara Loftus <ciara.loftus@intel.com>, Ciara Power <ciara.power@intel.com>,
 Dariusz Sosnowski <dsosnowski@nvidia.com>,
 David Hunt <david.hunt@intel.com>,
 Devendra Singh Rawat <dsinghrawat@marvell.com>,
 Erik Gabriel Carrillo <erik.g.carrillo@intel.com>,
 Guoyang Zhou <zhouguoyang@huawei.com>, Harman Kalra <hkalra@marvell.com>,
 Harry van Haaren <harry.van.haaren@intel.com>,
 Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>,
 Jakub Grajciar <jgrajcia@cisco.com>, Jerin Jacob <jerinj@marvell.com>,
 Jeroen de Borst <jeroendb@google.com>, Jian Wang <jianwang@trustnetic.com>,
 Jiawen Wu <jiawenwu@trustnetic.com>, Jie Hai <haijie1@huawei.com>,
 Jingjing Wu <jingjing.wu@intel.com>,
 Joshua Washington <joshwash@google.com>, Joyce Kong <joyce.kong@arm.com>,
 Junfeng Guo <junfeng.guo@intel.com>, Kevin Laatz <kevin.laatz@intel.com>,
 Konstantin Ananyev <konstantin.v.ananyev@yandex.ru>,
 Liang Ma <liangma@liangbit.com>, Long Li <longli@microsoft.com>,
 Maciej Czekaj <mczekaj@marvell.com>, Matan Azrad <matan@nvidia.com>,
 Maxime Coquelin <maxime.coquelin@redhat.com>,
 Nicolas Chautru <nicolas.chautru@intel.com>, Ori Kam <orika@nvidia.com>,
 Pavan Nikhilesh <pbhagavatula@marvell.com>,
 Peter Mccarthy <peter.mccarthy@intel.com>,
 Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>,
 Reshma Pattan <reshma.pattan@intel.com>, Rosen Xu <rosen.xu@intel.com>,
 Ruifeng Wang <ruifeng.wang@arm.com>, Rushil Gupta <rushilg@google.com>,
 Sameh Gobriel <sameh.gobriel@intel.com>,
 Sivaprasad Tummala <sivaprasad.tummala@amd.com>,
 Somnath Kotur <somnath.kotur@broadcom.com>,
 Stephen Hemminger <stephen@networkplumber.org>,
 Suanming Mou <suanmingm@nvidia.com>, Sunil Kumar Kori <skori@marvell.com>,
 Sunil Uttarwar <sunilprakashrao.uttarwar@amd.com>,
 Tetsuya Mukawa <mtetsuyah@gmail.com>,
 Vamsi Attunuru <vattunuru@marvell.com>,
 Viacheslav Ovsiienko <viacheslavo@nvidia.com>,
 Vladimir Medvedkin <vladimir.medvedkin@intel.com>,
 Xiaoyun Wang <cloud.wangxiaoyun@huawei.com>,
 Yipeng Wang <yipeng1.wang@intel.com>,
 Yisen Zhuang <yisen.zhuang@huawei.com>,
 Ziyang Xuan <xuanziyang2@huawei.com>,
 Tyler Retzlaff <roretzla@linux.microsoft.com>
Subject: [PATCH v6 23/45] event/opdl: use rte stdatomic API
Date: Tue, 14 May 2024 09:35:37 -0700
Message-Id: <1715704559-15839-24-git-send-email-roretzla@linux.microsoft.com>
X-Mailer: git-send-email 1.8.3.1
In-Reply-To: <1715704559-15839-1-git-send-email-roretzla@linux.microsoft.com>
References: <1710967892-7046-1-git-send-email-roretzla@linux.microsoft.com>
 <1715704559-15839-1-git-send-email-roretzla@linux.microsoft.com>
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org

Replace the use of gcc builtin __atomic_xxx intrinsics with
corresponding rte_atomic_xxx optional rte stdatomic API.

Signed-off-by: Tyler Retzlaff <roretzla@linux.microsoft.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
---
 drivers/event/opdl/opdl_ring.c | 80 +++++++++++++++++++++---------------------
 1 file changed, 40 insertions(+), 40 deletions(-)

diff --git a/drivers/event/opdl/opdl_ring.c b/drivers/event/opdl/opdl_ring.c
index e87ffd5..3476f6b 100644
--- a/drivers/event/opdl/opdl_ring.c
+++ b/drivers/event/opdl/opdl_ring.c
@@ -47,12 +47,12 @@ struct __rte_cache_aligned shared_state {
 	/* Last known minimum sequence number of dependencies, used for multi
 	 * thread operation
 	 */
-	uint32_t available_seq;
+	RTE_ATOMIC(uint32_t) available_seq;
 	char _pad1[RTE_CACHE_LINE_SIZE * 3];
-	uint32_t head;  /* Head sequence number (for multi thread operation) */
+	RTE_ATOMIC(uint32_t) head;  /* Head sequence number (for multi thread operation) */
 	char _pad2[RTE_CACHE_LINE_SIZE * 3];
 	struct opdl_stage *stage;  /* back pointer */
-	uint32_t tail;  /* Tail sequence number */
+	RTE_ATOMIC(uint32_t) tail;  /* Tail sequence number */
 	char _pad3[RTE_CACHE_LINE_SIZE * 2];
 };
 
@@ -149,10 +149,10 @@ struct opdl_ring {
 available(const struct opdl_stage *s)
 {
 	if (s->threadsafe == true) {
-		uint32_t n = __atomic_load_n(&s->shared.available_seq,
-				__ATOMIC_ACQUIRE) -
-				__atomic_load_n(&s->shared.head,
-				__ATOMIC_ACQUIRE);
+		uint32_t n = rte_atomic_load_explicit(&s->shared.available_seq,
+				rte_memory_order_acquire) -
+				rte_atomic_load_explicit(&s->shared.head,
+				rte_memory_order_acquire);
 
 		/* Return 0 if available_seq needs to be updated */
 		return (n <= s->num_slots) ? n : 0;
@@ -168,7 +168,7 @@ struct opdl_ring {
 {
 	uint32_t i;
 	uint32_t this_tail = s->shared.tail;
-	uint32_t min_seq = __atomic_load_n(&s->deps[0]->tail, __ATOMIC_ACQUIRE);
+	uint32_t min_seq = rte_atomic_load_explicit(&s->deps[0]->tail, rte_memory_order_acquire);
 	/* Input stage sequence numbers are greater than the sequence numbers of
 	 * its dependencies so an offset of t->num_slots is needed when
 	 * calculating available slots and also the condition which is used to
@@ -179,16 +179,16 @@ struct opdl_ring {
 	if (is_input_stage(s)) {
 		wrap = s->num_slots;
 		for (i = 1; i < s->num_deps; i++) {
-			uint32_t seq = __atomic_load_n(&s->deps[i]->tail,
-					__ATOMIC_ACQUIRE);
+			uint32_t seq = rte_atomic_load_explicit(&s->deps[i]->tail,
+					rte_memory_order_acquire);
 			if ((this_tail - seq) > (this_tail - min_seq))
 				min_seq = seq;
 		}
 	} else {
 		wrap = 0;
 		for (i = 1; i < s->num_deps; i++) {
-			uint32_t seq = __atomic_load_n(&s->deps[i]->tail,
-					__ATOMIC_ACQUIRE);
+			uint32_t seq = rte_atomic_load_explicit(&s->deps[i]->tail,
+					rte_memory_order_acquire);
 			if ((seq - this_tail) < (min_seq - this_tail))
 				min_seq = seq;
 		}
@@ -197,8 +197,8 @@ struct opdl_ring {
 	if (s->threadsafe == false)
 		s->available_seq = min_seq + wrap;
 	else
-		__atomic_store_n(&s->shared.available_seq, min_seq + wrap,
-				__ATOMIC_RELEASE);
+		rte_atomic_store_explicit(&s->shared.available_seq, min_seq + wrap,
+				rte_memory_order_release);
 }
 
 /* Wait until the number of available slots reaches number requested */
@@ -298,7 +298,7 @@ struct opdl_ring {
 	copy_entries_in(t, head, entries, num_entries);
 
 	s->head += num_entries;
-	__atomic_store_n(&s->shared.tail, s->head, __ATOMIC_RELEASE);
+	rte_atomic_store_explicit(&s->shared.tail, s->head, rte_memory_order_release);
 
 	return num_entries;
 }
@@ -381,18 +381,18 @@ struct opdl_ring {
 		/* There should be no race condition here. If shared.tail
 		 * matches, no other core can update it until this one does.
 		 */
-		if (__atomic_load_n(&s->shared.tail, __ATOMIC_ACQUIRE) ==
+		if (rte_atomic_load_explicit(&s->shared.tail, rte_memory_order_acquire) ==
 				tail) {
 			if (num_entries >= (head - tail)) {
 				claim_mgr_remove(disclaims);
-				__atomic_store_n(&s->shared.tail, head,
-						__ATOMIC_RELEASE);
+				rte_atomic_store_explicit(&s->shared.tail, head,
+						rte_memory_order_release);
 				num_entries -= (head - tail);
 			} else {
 				claim_mgr_move_tail(disclaims, num_entries);
-				__atomic_store_n(&s->shared.tail,
+				rte_atomic_store_explicit(&s->shared.tail,
 						num_entries + tail,
-						__ATOMIC_RELEASE);
+						rte_memory_order_release);
 				num_entries = 0;
 			}
 		} else if (block == false)
@@ -420,7 +420,7 @@ struct opdl_ring {
 	opdl_stage_disclaim_multithread_n(s, disclaims->num_to_disclaim,
 			false);
 
-	*old_head = __atomic_load_n(&s->shared.head, __ATOMIC_ACQUIRE);
+	*old_head = rte_atomic_load_explicit(&s->shared.head, rte_memory_order_acquire);
 	while (true) {
 		bool success;
 		/* If called by opdl_ring_input(), claim does not need to be
@@ -440,11 +440,10 @@ struct opdl_ring {
 		if (*num_entries == 0)
 			return;
 
-		success = __atomic_compare_exchange_n(&s->shared.head, old_head,
+		success = rte_atomic_compare_exchange_weak_explicit(&s->shared.head, old_head,
 				*old_head + *num_entries,
-				true,  /* may fail spuriously */
-				__ATOMIC_RELEASE,  /* memory order on success */
-				__ATOMIC_ACQUIRE);  /* memory order on fail */
+				rte_memory_order_release,  /* memory order on success */
+				rte_memory_order_acquire);  /* memory order on fail */
 		if (likely(success))
 			break;
 		rte_pause();
@@ -472,10 +471,11 @@ struct opdl_ring {
 	/* If another thread started inputting before this one, but hasn't
 	 * finished, we need to wait for it to complete to update the tail.
 	 */
-	rte_wait_until_equal_32(&s->shared.tail, old_head, __ATOMIC_ACQUIRE);
+	rte_wait_until_equal_32((uint32_t *)(uintptr_t)&s->shared.tail, old_head,
+	    rte_memory_order_acquire);
 
-	__atomic_store_n(&s->shared.tail, old_head + num_entries,
-			__ATOMIC_RELEASE);
+	rte_atomic_store_explicit(&s->shared.tail, old_head + num_entries,
+			rte_memory_order_release);
 
 	return num_entries;
 }
@@ -525,8 +525,8 @@ struct opdl_ring {
 		for (j = 0; j < num_entries; j++) {
 			ev = (struct rte_event *)get_slot(t, s->head+j);
 
-			event  = __atomic_load_n(&(ev->event),
-					__ATOMIC_ACQUIRE);
+			event  = rte_atomic_load_explicit((uint64_t __rte_atomic *)&ev->event,
+					rte_memory_order_acquire);
 
 			opa_id = OPDL_OPA_MASK & (event >> OPDL_OPA_OFFSET);
 			flow_id  = OPDL_FLOWID_MASK & event;
@@ -627,8 +627,8 @@ struct opdl_ring {
 				num_entries, s->head - old_tail);
 		num_entries = s->head - old_tail;
 	}
-	__atomic_store_n(&s->shared.tail, num_entries + old_tail,
-			__ATOMIC_RELEASE);
+	rte_atomic_store_explicit(&s->shared.tail, num_entries + old_tail,
+			rte_memory_order_release);
 }
 
 uint32_t
@@ -657,7 +657,7 @@ struct opdl_ring {
 	copy_entries_in(t, head, entries, num_entries);
 
 	s->head += num_entries;
-	__atomic_store_n(&s->shared.tail, s->head, __ATOMIC_RELEASE);
+	rte_atomic_store_explicit(&s->shared.tail, s->head, rte_memory_order_release);
 
 	return num_entries;
 
@@ -676,7 +676,7 @@ struct opdl_ring {
 	copy_entries_out(t, head, entries, num_entries);
 
 	s->head += num_entries;
-	__atomic_store_n(&s->shared.tail, s->head, __ATOMIC_RELEASE);
+	rte_atomic_store_explicit(&s->shared.tail, s->head, rte_memory_order_release);
 
 	return num_entries;
 }
@@ -755,7 +755,7 @@ struct opdl_ring {
 		return 0;
 	}
 	if (s->threadsafe == false) {
-		__atomic_store_n(&s->shared.tail, s->head, __ATOMIC_RELEASE);
+		rte_atomic_store_explicit(&s->shared.tail, s->head, rte_memory_order_release);
 		s->seq += s->num_claimed;
 		s->shadow_head = s->head;
 		s->num_claimed = 0;
@@ -1008,8 +1008,8 @@ struct opdl_ring *
 			ev_orig = (struct rte_event *)
 				get_slot(t, s->shadow_head+i);
 
-			event  = __atomic_load_n(&(ev_orig->event),
-					__ATOMIC_ACQUIRE);
+			event  = rte_atomic_load_explicit((uint64_t __rte_atomic *)&ev_orig->event,
+					rte_memory_order_acquire);
 
 			opa_id = OPDL_OPA_MASK & (event >> OPDL_OPA_OFFSET);
 			flow_id  = OPDL_FLOWID_MASK & event;
@@ -1026,9 +1026,9 @@ struct opdl_ring *
 
 				if ((event & OPDL_EVENT_MASK) !=
 						ev_temp) {
-					__atomic_store_n(&(ev_orig->event),
-							ev_update,
-							__ATOMIC_RELEASE);
+					rte_atomic_store_explicit(
+						(uint64_t __rte_atomic *)&ev_orig->event,
+						ev_update, rte_memory_order_release);
 					ev_updated = true;
 				}
 				if (ev_orig->u64 != ev->u64) {
-- 
1.8.3.1