From: "Mattias Rönnblom" <mattias.ronnblom@ericsson.com>
To: <dev@dpdk.org>
Cc: hofors@lysator.liu.se, "Heng Wang" <heng.wang@ericsson.com>,
"Stephen Hemminger" <stephen@networkplumber.org>,
"Tyler Retzlaff" <roretzla@linux.microsoft.com>,
"Mattias Rönnblom" <mattias.ronnblom@ericsson.com>
Subject: [RFC v2 5/6] eal: add atomic bit operations
Date: Thu, 25 Apr 2024 10:58:52 +0200 [thread overview]
Message-ID: <20240425085853.97888-6-mattias.ronnblom@ericsson.com> (raw)
In-Reply-To: <20240425085853.97888-1-mattias.ronnblom@ericsson.com>
Add atomic bit test/set/clear/assign and test-and-set/clear functions.
All atomic bit functions allow (and indeed, require) the caller to
specify a memory order.
RFC v2:
o Add rte_bit_atomic_test_and_assign() (for consistency).
o Fix bugs in rte_bit_atomic_test_and_[set|clear]().
o Use <rte_stdatomics.h> to support MSVC.
Signed-off-by: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
---
lib/eal/include/rte_bitops.h | 297 +++++++++++++++++++++++++++++++++++
1 file changed, 297 insertions(+)
diff --git a/lib/eal/include/rte_bitops.h b/lib/eal/include/rte_bitops.h
index a2746e657f..8c38a1ac03 100644
--- a/lib/eal/include/rte_bitops.h
+++ b/lib/eal/include/rte_bitops.h
@@ -21,6 +21,7 @@
#include <rte_compat.h>
#include <rte_debug.h>
+#include <rte_stdatomic.h>
#ifdef __cplusplus
extern "C" {
@@ -342,6 +343,177 @@ extern "C" {
uint32_t *: __rte_bit_once_assign32, \
uint64_t *: __rte_bit_once_assign64)(addr, nr, value)
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Test if a particular bit in a word is set with a particular memory
+ * order.
+ *
+ * Test a bit with the resulting memory load ordered as per the
+ * specified memory order.
+ *
+ * @param addr
+ * A pointer to the word to query.
+ * @param nr
+ * The index of the bit.
+ * @param memory_order
+ * The memory order to use. See <rte_stdatomics.h> for details.
+ * @return
+ * Returns true if the bit is set, and false otherwise.
+ */
+#define rte_bit_atomic_test(addr, nr, memory_order) \
+ _Generic((addr), \
+ uint32_t *: __rte_bit_atomic_test32, \
+ uint64_t *: __rte_bit_atomic_test64)(addr, nr, memory_order)
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Atomically set bit in word.
+ *
+ * Atomically set bit specified by @c nr in the word pointed to by @c
+ * addr to '1', with the memory ordering as specified by @c
+ * memory_order.
+ *
+ * @param addr
+ * A pointer to the word to modify.
+ * @param nr
+ * The index of the bit.
+ * @param memory_order
+ * The memory order to use. See <rte_stdatomics.h> for details.
+ */
+#define rte_bit_atomic_set(addr, nr, memory_order) \
+ _Generic((addr), \
+ uint32_t *: __rte_bit_atomic_set32, \
+ uint64_t *: __rte_bit_atomic_set64)(addr, nr, memory_order)
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Atomically clear bit in word.
+ *
+ * Atomically set bit specified by @c nr in the word pointed to by @c
+ * addr to '0', with the memory ordering as specified by @c
+ * memory_order.
+ *
+ * @param addr
+ * A pointer to the word to modify.
+ * @param nr
+ * The index of the bit.
+ * @param memory_order
+ * The memory order to use. See <rte_stdatomics.h> for details.
+ */
+#define rte_bit_atomic_clear(addr, nr, memory_order) \
+ _Generic((addr), \
+ uint32_t *: __rte_bit_atomic_clear32, \
+ uint64_t *: __rte_bit_atomic_clear64)(addr, nr, memory_order)
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Atomically assign a value to bit in word.
+ *
+ * Atomically set bit specified by @c nr in the word pointed to by @c
+ * addr to the value indicated by @c value, with the memory ordering
+ * as specified with @c memory_order.
+ *
+ * @param addr
+ * A pointer to the word to modify.
+ * @param nr
+ * The index of the bit.
+ * @param value
+ * The new value of the bit - true for '1', or false for '0'.
+ * @param memory_order
+ * The memory order to use. See <rte_stdatomics.h> for details.
+ */
+#define rte_bit_atomic_assign(addr, nr, value, memory_order) \
+ _Generic((addr), \
+ uint32_t *: __rte_bit_atomic_assign32, \
+ uint64_t *: __rte_bit_atomic_assign64)(addr, nr, value, \
+ memory_order)
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Atomically test and set a bit in word.
+ *
+ * Atomically test and set bit specified by @c nr in the word pointed
+ * to by @c addr to '1', with the memory ordering as specified with @c
+ * memory_order.
+ *
+ * @param addr
+ * A pointer to the word to modify.
+ * @param nr
+ * The index of the bit.
+ * @param memory_order
+ * The memory order to use. See <rte_stdatomics.h> for details.
+ * @return
+ * Returns true if the bit was set, and false otherwise.
+ */
+#define rte_bit_atomic_test_and_set(addr, nr, memory_order) \
+ _Generic((addr), \
+ uint32_t *: __rte_bit_atomic_test_and_set32, \
+ uint64_t *: __rte_bit_atomic_test_and_set64)(addr, nr, \
+ memory_order)
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Atomically test and clear a bit in word.
+ *
+ * Atomically test and clear bit specified by @c nr in the word
+ * pointed to by @c addr to '0', with the memory ordering as specified
+ * with @c memory_order.
+ *
+ * @param addr
+ * A pointer to the word to modify.
+ * @param nr
+ * The index of the bit.
+ * @param memory_order
+ * The memory order to use. See <rte_stdatomics.h> for details.
+ * @return
+ * Returns true if the bit was set, and false otherwise.
+ */
+#define rte_bit_atomic_test_and_clear(addr, nr, memory_order) \
+ _Generic((addr), \
+ uint32_t *: __rte_bit_atomic_test_and_clear32, \
+ uint64_t *: __rte_bit_atomic_test_and_clear64)(addr, nr, \
+ memory_order)
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Atomically test and assign a bit in word.
+ *
+ * Atomically test and assign bit specified by @c nr in the word
+ * pointed to by @c addr the value specified by @c value, with the
+ * memory ordering as specified with @c memory_order.
+ *
+ * @param addr
+ * A pointer to the word to modify.
+ * @param nr
+ * The index of the bit.
+ * @param value
+ * The new value of the bit - true for '1', or false for '0'.
+ * @param memory_order
+ * The memory order to use. See <rte_stdatomics.h> for details.
+ * @return
+ * Returns true if the bit was set, and false otherwise.
+ */
+#define rte_bit_atomic_test_and_assign(addr, nr, value, memory_order) \
+ _Generic((addr), \
+ uint32_t *: __rte_bit_atomic_test_and_assign32, \
+ uint64_t *: __rte_bit_atomic_test_and_assign64)(addr, nr, \
+ value, \
+ memory_order)
+
#define __RTE_GEN_BIT_TEST(name, size, qualifier) \
static inline bool \
name(const qualifier uint ## size ## _t *addr, unsigned int nr) \
@@ -429,6 +601,131 @@ __rte_bit_once_assign64(volatile uint64_t *addr, unsigned int nr, bool value)
__rte_bit_once_clear64(addr, nr);
}
+#define __RTE_GEN_BIT_ATOMIC_TEST(size) \
+ static inline bool \
+ __rte_bit_atomic_test ## size(const uint ## size ## _t *addr, \
+ unsigned int nr, int memory_order) \
+ { \
+ RTE_ASSERT(nr < size); \
+ \
+ const RTE_ATOMIC(uint ## size ## _t) *a_addr = \
+ (const RTE_ATOMIC(uint ## size ## _t) *)addr; \
+ uint ## size ## _t mask = (uint ## size ## _t)1 << nr; \
+ return rte_atomic_load_explicit(a_addr, memory_order) & mask; \
+ }
+
+#define __RTE_GEN_BIT_ATOMIC_SET(size) \
+ static inline void \
+ __rte_bit_atomic_set ## size(uint ## size ## _t *addr, \
+ unsigned int nr, int memory_order) \
+ { \
+ RTE_ASSERT(nr < size); \
+ \
+ RTE_ATOMIC(uint ## size ## _t) *a_addr = \
+ (RTE_ATOMIC(uint ## size ## _t) *)addr; \
+ uint ## size ## _t mask = (uint ## size ## _t)1 << nr; \
+ rte_atomic_fetch_or_explicit(a_addr, mask, memory_order); \
+ }
+
+#define __RTE_GEN_BIT_ATOMIC_CLEAR(size) \
+ static inline void \
+ __rte_bit_atomic_clear ## size(uint ## size ## _t *addr, \
+ unsigned int nr, int memory_order) \
+ { \
+ RTE_ASSERT(nr < size); \
+ \
+ RTE_ATOMIC(uint ## size ## _t) *a_addr = \
+ (RTE_ATOMIC(uint ## size ## _t) *)addr; \
+ uint ## size ## _t mask = (uint ## size ## _t)1 << nr; \
+ rte_atomic_fetch_and_explicit(a_addr, ~mask, memory_order); \
+ }
+
+#define __RTE_GEN_BIT_ATOMIC_ASSIGN(size) \
+ static inline void \
+ __rte_bit_atomic_assign ## size(uint ## size ## _t *addr, \
+ unsigned int nr, bool value, \
+ int memory_order) \
+ { \
+ if (value) \
+ __rte_bit_atomic_set ## size(addr, nr, memory_order); \
+ else \
+ __rte_bit_atomic_clear ## size(addr, nr, \
+ memory_order); \
+ }
+
+#define __RTE_GEN_BIT_ATOMIC_TEST_AND_ASSIGN(size) \
+ static inline bool \
+ __rte_bit_atomic_test_and_assign ## size(uint ## size ## _t *addr, \
+ unsigned int nr, \
+ bool value, \
+ int memory_order) \
+ { \
+ RTE_ASSERT(nr < size); \
+ \
+ RTE_ATOMIC(uint ## size ## _t) *a_addr = \
+ (RTE_ATOMIC(uint ## size ## _t) *)addr; \
+ uint ## size ## _t before; \
+ uint ## size ## _t target; \
+ \
+ before = rte_atomic_load_explicit(a_addr, \
+ rte_memory_order_relaxed); \
+ \
+ do { \
+ target = before; \
+ __rte_bit_assign ## size(&target, nr, value); \
+ } while (!rte_atomic_compare_exchange_weak_explicit( \
+ a_addr, &before, target, \
+ rte_memory_order_relaxed, \
+ memory_order)); \
+ return __rte_bit_test ## size(&before, nr); \
+ }
+
+#define __RTE_GEN_BIT_ATOMIC_OPS(size) \
+ __RTE_GEN_BIT_ATOMIC_TEST(size) \
+ __RTE_GEN_BIT_ATOMIC_SET(size) \
+ __RTE_GEN_BIT_ATOMIC_CLEAR(size) \
+ __RTE_GEN_BIT_ATOMIC_ASSIGN(size) \
+ __RTE_GEN_BIT_ATOMIC_TEST_AND_ASSIGN(size)
+
+__RTE_GEN_BIT_ATOMIC_OPS(32)
+__RTE_GEN_BIT_ATOMIC_OPS(64)
+
+__rte_experimental
+static inline bool
+__rte_bit_atomic_test_and_set32(uint32_t *addr, unsigned int nr,
+ int memory_order)
+{
+ return __rte_bit_atomic_test_and_assign32(addr, nr, true,
+ memory_order);
+}
+
+__rte_experimental
+static inline bool
+__rte_bit_atomic_test_and_clear32(uint32_t *addr, unsigned int nr,
+ int memory_order)
+{
+ return __rte_bit_atomic_test_and_assign32(addr, nr, false,
+ memory_order);
+}
+
+__rte_experimental
+static inline bool
+__rte_bit_atomic_test_and_set64(uint64_t *addr, unsigned int nr,
+ int memory_order)
+{
+ return __rte_bit_atomic_test_and_assign64(addr, nr, true,
+ memory_order);
+}
+
+__rte_experimental
+static inline bool
+__rte_bit_atomic_test_and_clear64(uint64_t *addr, unsigned int nr,
+ int memory_order)
+{
+ return __rte_bit_atomic_test_and_assign64(addr, nr, false,
+ memory_order);
+}
+
/*------------------------ 32-bit relaxed operations ------------------------*/
/**
--
2.34.1
next prev parent reply other threads:[~2024-04-25 9:10 UTC|newest]
Thread overview: 74+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-03-02 13:53 [RFC 0/7] Improve EAL bit operations API Mattias Rönnblom
2024-03-02 13:53 ` [RFC 1/7] eal: extend bit manipulation functions Mattias Rönnblom
2024-03-02 17:05 ` Stephen Hemminger
2024-03-03 6:26 ` Mattias Rönnblom
2024-03-04 16:34 ` Tyler Retzlaff
2024-03-05 18:01 ` Mattias Rönnblom
2024-03-05 18:06 ` Tyler Retzlaff
2024-04-25 8:58 ` [RFC v2 0/6] Improve EAL bit operations API Mattias Rönnblom
2024-04-25 8:58 ` [RFC v2 1/6] eal: extend bit manipulation functionality Mattias Rönnblom
2024-04-29 9:51 ` [RFC v3 0/6] Improve EAL bit operations API Mattias Rönnblom
2024-04-29 9:51 ` [RFC v3 1/6] eal: extend bit manipulation functionality Mattias Rönnblom
2024-04-29 11:12 ` Morten Brørup
2024-04-30 9:55 ` [RFC v4 0/6] Improve EAL bit operations API Mattias Rönnblom
2024-04-30 9:55 ` [RFC v4 1/6] eal: extend bit manipulation functionality Mattias Rönnblom
2024-04-30 12:08 ` [RFC v5 0/6] Improve EAL bit operations API Mattias Rönnblom
2024-04-30 12:08 ` [RFC v5 1/6] eal: extend bit manipulation functionality Mattias Rönnblom
2024-05-02 5:57 ` [RFC v6 0/6] Improve EAL bit operations API Mattias Rönnblom
2024-05-02 5:57 ` [RFC v6 1/6] eal: extend bit manipulation functionality Mattias Rönnblom
2024-05-02 5:57 ` [RFC v6 2/6] eal: add unit tests for bit operations Mattias Rönnblom
2024-05-02 5:57 ` [RFC v6 3/6] eal: add exactly-once bit access functions Mattias Rönnblom
2024-05-02 5:57 ` [RFC v6 4/6] eal: add unit tests for " Mattias Rönnblom
2024-05-02 5:57 ` [RFC v6 5/6] eal: add atomic bit operations Mattias Rönnblom
2024-05-03 6:41 ` Mattias Rönnblom
2024-05-03 23:30 ` Tyler Retzlaff
2024-05-04 15:36 ` Mattias Rönnblom
2024-05-02 5:57 ` [RFC v6 6/6] eal: add unit tests for atomic bit access functions Mattias Rönnblom
2024-04-30 12:08 ` [RFC v5 2/6] eal: add unit tests for bit operations Mattias Rönnblom
2024-04-30 12:08 ` [RFC v5 3/6] eal: add exactly-once bit access functions Mattias Rönnblom
2024-04-30 12:08 ` [RFC v5 4/6] eal: add unit tests for " Mattias Rönnblom
2024-04-30 12:08 ` [RFC v5 5/6] eal: add atomic bit operations Mattias Rönnblom
2024-04-30 12:08 ` [RFC v5 6/6] eal: add unit tests for atomic bit access functions Mattias Rönnblom
2024-04-30 9:55 ` [RFC v4 2/6] eal: add unit tests for bit operations Mattias Rönnblom
2024-04-30 9:55 ` [RFC v4 3/6] eal: add exactly-once bit access functions Mattias Rönnblom
2024-04-30 9:55 ` [RFC v4 4/6] eal: add unit tests for " Mattias Rönnblom
2024-04-30 10:37 ` Morten Brørup
2024-04-30 11:58 ` Mattias Rönnblom
2024-04-30 9:55 ` [RFC v4 5/6] eal: add atomic bit operations Mattias Rönnblom
2024-04-30 9:55 ` [RFC v4 6/6] eal: add unit tests for atomic bit access functions Mattias Rönnblom
2024-04-29 9:51 ` [RFC v3 2/6] eal: add unit tests for bit operations Mattias Rönnblom
2024-04-29 9:51 ` [RFC v3 3/6] eal: add exactly-once bit access functions Mattias Rönnblom
2024-04-29 9:51 ` [RFC v3 4/6] eal: add unit tests for " Mattias Rönnblom
2024-04-29 9:51 ` [RFC v3 5/6] eal: add atomic bit operations Mattias Rönnblom
2024-04-29 9:51 ` [RFC v3 6/6] eal: add unit tests for atomic bit access functions Mattias Rönnblom
2024-04-25 8:58 ` [RFC v2 2/6] eal: add unit tests for bit operations Mattias Rönnblom
2024-04-25 8:58 ` [RFC v2 3/6] eal: add exactly-once bit access functions Mattias Rönnblom
2024-04-25 8:58 ` [RFC v2 4/6] eal: add unit tests for " Mattias Rönnblom
2024-04-25 8:58 ` Mattias Rönnblom [this message]
2024-04-25 10:25 ` [RFC v2 5/6] eal: add atomic bit operations Morten Brørup
2024-04-25 14:36 ` Mattias Rönnblom
2024-04-25 16:18 ` Morten Brørup
2024-04-26 9:39 ` Mattias Rönnblom
2024-04-26 12:00 ` Morten Brørup
2024-04-28 15:37 ` Mattias Rönnblom
2024-04-29 7:24 ` Morten Brørup
2024-04-30 16:52 ` Tyler Retzlaff
2024-04-25 8:58 ` [RFC v2 6/6] eal: add unit tests for atomic bit access functions Mattias Rönnblom
2024-04-25 18:05 ` [RFC v2 0/6] Improve EAL bit operations API Tyler Retzlaff
2024-04-26 11:17 ` Mattias Rönnblom
2024-04-26 21:35 ` Patrick Robb
2024-03-02 13:53 ` [RFC 2/7] eal: add generic bit manipulation macros Mattias Rönnblom
2024-03-04 8:16 ` Heng Wang
2024-03-04 15:41 ` Mattias Rönnblom
2024-03-04 16:42 ` Tyler Retzlaff
2024-03-05 18:08 ` Mattias Rönnblom
2024-03-05 18:22 ` Tyler Retzlaff
2024-03-05 20:02 ` Mattias Rönnblom
2024-03-05 20:53 ` Tyler Retzlaff
2024-03-02 13:53 ` [RFC 3/7] eal: add bit manipulation functions which read or write once Mattias Rönnblom
2024-03-02 13:53 ` [RFC 4/7] eal: add generic once-type bit operations macros Mattias Rönnblom
2024-03-02 13:53 ` [RFC 5/7] eal: add atomic bit operations Mattias Rönnblom
2024-03-02 13:53 ` [RFC 6/7] eal: add generic " Mattias Rönnblom
2024-03-02 13:53 ` [RFC 7/7] eal: deprecate relaxed family of " Mattias Rönnblom
2024-03-02 17:07 ` Stephen Hemminger
2024-03-03 6:30 ` Mattias Rönnblom
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240425085853.97888-6-mattias.ronnblom@ericsson.com \
--to=mattias.ronnblom@ericsson.com \
--cc=dev@dpdk.org \
--cc=heng.wang@ericsson.com \
--cc=hofors@lysator.liu.se \
--cc=roretzla@linux.microsoft.com \
--cc=stephen@networkplumber.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).