From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 417D543F49; Mon, 29 Apr 2024 12:02:27 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id DFE5E402EB; Mon, 29 Apr 2024 12:02:12 +0200 (CEST) Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-vi1eur04on2042.outbound.protection.outlook.com [40.107.8.42]) by mails.dpdk.org (Postfix) with ESMTP id E8453402D0 for ; Mon, 29 Apr 2024 12:02:08 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=bP/9wu0FKmp/zj09tCyP8uTY3eEq+pmk830sRGekS0RS1V2h0fn49JdT6SdfzbJbpUxMgtykc2xQz1KhEWzCVtp1GveOtaewdwFxaK+vkvlHuljyXHXqEhW1hOAd6Ec0tC3zK5YtW/jiA4PxXJ3l/J73r07Iz+yF6bclNhr+F6ccQMtYK7DnTWERlRsFVdj/c6gZBESQ/jpHlTxMuzleyqSLgTEGKnIo7NcJskE9y36RIH057rjicfCi/BEtzL9BZPq/Sm+RqN7ao6KNImje/xY7MbuD6J/Xb7uPnmEzEvUCbpW/xT8gWms7R4vxP6uc/MTHRc/RU7Ls+aKqJWei1w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=vV0MxvxvrsgxLRzfbnzW7OwWkR3i8PuvgA6na5XXKQo=; b=W/SOdHsj3wE6BOC1aBfR/5WihWGkcZOc21vHfEwyaBSnBf9OiSHIRxbrVVxe66l3gxeBf8vjG3APPOhBc3/MLGqOi4SGFf0jgoe9uSCaV2uaYD8BzEjZyR9EAQ+4xZviCKF8TolmSwq+029qtZDbhVbMCSpDcg+mQOogwBaFylLNwdR8X/C7+lf4KjG5cGlrsmnZsDExXyKFo7i5jYpZzpyh3asoIfoX3WuWJUnF5JFrXSXVnkOJGcw6NqCeuYWVofuw+pai/SArgQuzRe2dEN1QS98tQWIj55i0Vmx3HgEdJqiT+HdTGSRuf2OmzCVwbSMsIjGcUY2aDEguVDmcyg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=dpdk.org smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vV0MxvxvrsgxLRzfbnzW7OwWkR3i8PuvgA6na5XXKQo=; b=YkjhvR/MUbu2qwrekLMEbWW/gq8dv7AT7R8UAunas2nemWixPVz7j6HiUqLZTNj77qf4ph8oZAOGnRrFlxTaYlmN5irPFk/78q1ZPX1NU+GMT9ad4JWKjHqb7h8ax3AKBfYHTyuOYzrGGgYSYhSOQb1+emHAHaCaJk4FZK6aUzIkZdnCVazwqpQlWsbEcwkvH/1kLOFf1gwWmJgO/9xAAqNElTEKGJ2josVZPoWfpdnDbfPJCZ+1AKVvXC14lSyUlkN2C+e8KuX7b7xbVnhP1Ted25I9PoRmwTSDpCYPzruAXMT4A1+3nKsw4AQcyxCqx+dq9Ad/HjZW+mj0KKzNLA== Received: from DU2PR04CA0069.eurprd04.prod.outlook.com (2603:10a6:10:232::14) by AM7PR07MB6232.eurprd07.prod.outlook.com (2603:10a6:20b:135::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7472.44; Mon, 29 Apr 2024 10:02:07 +0000 Received: from DB1PEPF00039231.eurprd03.prod.outlook.com (2603:10a6:10:232:cafe::ed) by DU2PR04CA0069.outlook.office365.com (2603:10a6:10:232::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7519.34 via Frontend Transport; Mon, 29 Apr 2024 10:02:07 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by DB1PEPF00039231.mail.protection.outlook.com (10.167.8.104) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7519.19 via Frontend Transport; Mon, 29 Apr 2024 10:02:07 +0000 Received: from seliicinfr00050.seli.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.66) with Microsoft SMTP Server id 15.2.1544.9; Mon, 29 Apr 2024 12:02:06 +0200 Received: from breslau.. (seliicwb00002.seli.gic.ericsson.se [10.156.25.100]) by seliicinfr00050.seli.gic.ericsson.se (Postfix) with ESMTP id B50331C006A; Mon, 29 Apr 2024 12:02:06 +0200 (CEST) From: =?UTF-8?q?Mattias=20R=C3=B6nnblom?= To: CC: , Heng Wang , "Stephen Hemminger" , Tyler Retzlaff , =?UTF-8?q?Morten=20Br=C3=B8rup?= , =?UTF-8?q?Mattias=20R=C3=B6nnblom?= Subject: [RFC v3 3/6] eal: add exactly-once bit access functions Date: Mon, 29 Apr 2024 11:51:35 +0200 Message-ID: <20240429095138.106849-4-mattias.ronnblom@ericsson.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240429095138.106849-1-mattias.ronnblom@ericsson.com> References: <20240425085853.97888-2-mattias.ronnblom@ericsson.com> <20240429095138.106849-1-mattias.ronnblom@ericsson.com> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 8bit X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DB1PEPF00039231:EE_|AM7PR07MB6232:EE_ X-MS-Office365-Filtering-Correlation-Id: 5124dfca-bda7-494c-1f61-08dc68336d9f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: =?utf-8?B?TE1UTUxra0tWVHdJVXNZZFRGVms0KytlSUhkQzVNeFRuQytaWXlic2ZwMUVE?= =?utf-8?B?WWlBZzRNVlE4TCtKTUlsMUR1bjR6VndUR01SbWt5UEgxTERMTmJndW5MVTRa?= =?utf-8?B?OWI2dEE5VXd0SU1Gd2k5YnpQR08rdkJzSmw1cW1LU0JkUDI1d2t0Q0M1VGV1?= =?utf-8?B?eCs1ZkRaT0pHRFFIeDNNbTFBendSSHpSbkYvbGd0QmJPSmtaeEJVcjBnNUZW?= =?utf-8?B?eERFNG9hTitGVi9qU0F2QUlSWUV2K2lXZ1QycTNHb2g0U0I3WWZlR2oxOThj?= =?utf-8?B?d1NMRlJ1VkxMSktGOHNhemo4bHlZRENQRnBsSnRQNTRrL3lxZ0V3Yk9aVUt6?= =?utf-8?B?RGxKaFEzY0tTT2dsNlFPMld4ZlpvU3MvN0ZLcE9BbXJVWHlwd1M4QURQUzAr?= =?utf-8?B?aTZPclMzS0pPbkJmUHpZc2FucnY2Nmo4eWRINGloQ2F5Q2dwODNkTk5CaGY0?= =?utf-8?B?WUR3TGtvbTkrZXdzcE53bEp4U2YzUGQzQUtUSThOa3dOVE50dVplNHk2N1FT?= =?utf-8?B?c28xbEpIbThpRzRtUHpBWEt6S0w0UTNGR2ZyTllCVWZETUlUVUg2UGNvUFlW?= =?utf-8?B?R1NYSFZCSG9XQkFXZWh4RTkvQ0h3V0RiRW9IRHpKL2l4ak02WkYyek5FaDFk?= =?utf-8?B?WlV0YzhNOC83RFlINjBUUnVQaCtZYlN3bkI3OERvZ2dZL3V0TGpKQzRqSFVp?= =?utf-8?B?Z09MdDBXYnBDRXFtQmZ4d2kwV3czOTkvN2YrRmNXcDdaVEZMMXEwOENJUm5k?= =?utf-8?B?TWo4Nk03K2M3NVZQdGNxbkVVNTY3cnlJYzQ1QUlSUmNsdi8za1gyTlhsWlI0?= =?utf-8?B?L3hIOFlHNi9Ca0l6d3l5ZllHK21nOUVpclRCN24yZWJSREFRYUxPcFJQQ0Nr?= =?utf-8?B?blRBOExWcWU0WS9hbHlaSmlJVmxud25tUGRnNlc3WEhoeFBPUW9xWEtheW9S?= =?utf-8?B?MG9SM0hybjAyNmZTQUVUWDAzc25IcG5vMlE5QUxQTlRzQ3BFc1BwMkRHMDY3?= =?utf-8?B?Q0JSZkdyMDdoejV0WWM1STNMWW1vVU1XQVpob3JVeDR1bXBhUzk1VUtFREZa?= =?utf-8?B?MXVHeTh1SWRTdkwwV2xBWDNqUE81R2wvTzEwcGJwZHlVKzkrOHp0T2o2SEs2?= =?utf-8?B?RWZWL2NGUHZDK2pYMTJzbXFWZ0dhdnpXZHVsRS83cjNQbEdDUFgwczVZNFE3?= =?utf-8?B?cVVxUjlXMDZoQTBRR0dxaWZEN0JDS2NJa2hCOXVqVWQ1R1RFTjg2OVJIbzFn?= =?utf-8?B?NHJoQmN3U3kxeGUvSlZmbEwvczNmUXdjWGllOHlSMDdtN0hGOFk1S3VFTDZw?= =?utf-8?B?QkxTUk43U2RHUldsQTJyYjBMOHlwV0JOOWFxMmEyMkRoRmJhdDRtR3NrNHlS?= =?utf-8?B?cnZwU1NsWldDS3MwWmJQbDd6eWFVSkJVdGhnOWJaaWVOZis1cldxRVBQK2Nh?= =?utf-8?B?UFpCRUdmNEtTZFBHOW5LdncxQ3IyTnNsay8yVnZpeDVMeEdrNGgweWpTcnRS?= =?utf-8?B?MDBmK2FuUy9oU1RMWUFFeklDc1pyeGNBd0crcTNUeDdwQlpnVXBRUWJlNkdm?= =?utf-8?B?RkF5bnFNWHkwckxscmJDaU5xOWhHL0YwSGxWREhWSWg1dW1jVDJUcDRXenRw?= =?utf-8?B?Um1wR1R1TEFXTTI4NnJCNkZCcWowZGo1U21yUmQ1clJnV3ZCRDF0dC8yMkRG?= =?utf-8?B?QnllNWowb09Nb0NpVmpRS1VCMkdXZFYyUzBIM28zdXJISUNCV0Z1cDZrUTBR?= =?utf-8?B?MVU4OHU5U0tsSytLZ0F1QUtvYkg4VmFWcGQzWDIzRHl4L1hXdjFXOXFFSzN3?= =?utf-8?B?cEMvUEVjYUpzVW9oSEozc2Mxb1djaS9qZ2gybktMTEx5TzRGQ3VSTUNqVDdI?= =?utf-8?Q?ONaVd2hYEG7fx?= X-Forefront-Antispam-Report: CIP:192.176.1.74; CTRY:SE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:oa.msg.ericsson.com; PTR:office365.se.ericsson.net; CAT:NONE; SFS:(13230031)(1800799015)(82310400014)(376005)(36860700004); DIR:OUT; SFP:1101; X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Apr 2024 10:02:07.0595 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 5124dfca-bda7-494c-1f61-08dc68336d9f X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f; Ip=[192.176.1.74]; Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: DB1PEPF00039231.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR07MB6232 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add bit test/set/clear/assign functions which prevents certain compiler optimizations and guarantees that program-level memory loads and/or stores will actually occur. These functions are useful when interacting with memory-mapped hardware devices. The "once" family of functions does not promise atomicity and provides no memory ordering guarantees beyond the C11 relaxed memory model. RFC v3: * Work around lack of C++ support for _Generic (Tyler Retzlaff). Signed-off-by: Mattias Rönnblom --- lib/eal/include/rte_bitops.h | 180 +++++++++++++++++++++++++++++++++++ 1 file changed, 180 insertions(+) diff --git a/lib/eal/include/rte_bitops.h b/lib/eal/include/rte_bitops.h index fb2e3dae7b..eac3f8b86a 100644 --- a/lib/eal/include/rte_bitops.h +++ b/lib/eal/include/rte_bitops.h @@ -201,6 +201,147 @@ extern "C" { uint32_t *: __rte_bit_assign32, \ uint64_t *: __rte_bit_assign64)(addr, nr, value) +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Generic selection macro to test exactly once the value of a bit in + * a 32-bit or 64-bit word. The type of operation depends on the type + * of the @c addr parameter. + * + * This function is guaranteed to result in exactly one memory load + * (e.g., it may not be eliminate or merged by the compiler). + * + * \code{.c} + * rte_bit_once_set(addr, 17); + * if (rte_bit_once_test(addr, 17)) { + * ... + * } + * \endcode + * + * In the above example, rte_bit_once_set() may not be removed by + * the compiler, which would be allowed in case rte_bit_set() and + * rte_bit_test() was used. + * + * \code{.c} + * while (rte_bit_once_test(addr, 17); + * ; + * \endcode + * + * In case rte_bit_test(addr, 17) was used instead, the resulting + * object code could (and in many cases would be) replaced with + * the equivalent to + * \code{.c} + * if (rte_bit_test(addr, 17)) { + * for (;;) // spin forever + * ; + * } + * \endcode + * + * rte_bit_once_test() does not give any guarantees in regards to + * memory ordering or atomicity. + * + * The regular bit set operations (e.g., rte_bit_test()) should be + * preferred over the "once" family of operations (e.g., + * rte_bit_once_test()) if possible, since the latter may prevent + * optimizations crucial for run-time performance. + * + * @param addr + * A pointer to the word to query. + * @param nr + * The index of the bit. + * @return + * Returns true if the bit is set, and false otherwise. + */ + +#define rte_bit_once_test(addr, nr) \ + _Generic((addr), \ + uint32_t *: __rte_bit_once_test32, \ + uint64_t *: __rte_bit_once_test64)(addr, nr) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Set bit in word exactly once. + * + * Set bit specified by @c nr in the word pointed to by @c addr to '1' + * exactly once. + * + * This function is guaranteed to result in exactly one memory load + * and exactly one memory store, *or* an atomic bit set operation. + * + * See rte_bit_test_once32() for more information and uses cases for + * the "once" class of functions. + * + * This macro does not give any guarantees in regards to memory + * ordering or atomicity. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + */ +#define rte_bit_once_set(addr, nr) \ + _Generic((addr), \ + uint32_t *: __rte_bit_once_set32, \ + uint64_t *: __rte_bit_once_set64)(addr, nr) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Clear bit in word exactly once. + * + * Set bit specified by @c nr in the word pointed to by @c addr to '0' + * exactly once. + * + * This function is guaranteed to result in exactly one memory load + * and exactly one memory store, *or* an atomic bit clear operation. + * + * See rte_bit_test_once32() for more information and uses cases for + * the "once" class of functions. + * + * This macro does not give any guarantees in regards to memory + * ordering or atomicity. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + */ +#define rte_bit_once_clear(addr, nr) \ + _Generic((addr), \ + uint32_t *: __rte_bit_once_clear32, \ + uint64_t *: __rte_bit_once_clear64)(addr, nr) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Assign a value to bit in a word exactly once. + * + * Set bit specified by @c nr in the word pointed to by @c addr to the + * value indicated by @c value exactly once. + * + * This function is guaranteed to result in exactly one memory load + * and exactly one memory store, *or* an atomic bit clear operation. + * + * This function does not give any guarantees in regards to memory + * ordering or atomicity. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + * @param value + * The new value of the bit - true for '1', or false for '0'. + */ +#define rte_bit_once_assign(addr, nr, value) \ + _Generic((addr), \ + uint32_t *: __rte_bit_once_assign32, \ + uint64_t *: __rte_bit_once_assign64)(addr, nr, value) + #define __RTE_GEN_BIT_TEST(name, size, qualifier) \ static inline bool \ name(const qualifier uint ## size ## _t *addr, unsigned int nr) \ @@ -239,6 +380,14 @@ __RTE_GEN_BIT_TEST(__rte_bit_test64, 64, ) __RTE_GEN_BIT_SET(__rte_bit_set64, 64, ) __RTE_GEN_BIT_CLEAR(__rte_bit_clear64, 64, ) +__RTE_GEN_BIT_TEST(__rte_bit_once_test32, 32, volatile) +__RTE_GEN_BIT_SET(__rte_bit_once_set32, 32, volatile) +__RTE_GEN_BIT_CLEAR(__rte_bit_once_clear32, 32, volatile) + +__RTE_GEN_BIT_TEST(__rte_bit_once_test64, 64, volatile) +__RTE_GEN_BIT_SET(__rte_bit_once_set64, 64, volatile) +__RTE_GEN_BIT_CLEAR(__rte_bit_once_clear64, 64, volatile) + __rte_experimental static inline void __rte_bit_assign32(uint32_t *addr, unsigned int nr, bool value) @@ -259,6 +408,27 @@ __rte_bit_assign64(uint64_t *addr, unsigned int nr, bool value) __rte_bit_clear64(addr, nr); } + +__rte_experimental +static inline void +__rte_bit_once_assign32(uint32_t *addr, unsigned int nr, bool value) +{ + if (value) + __rte_bit_once_set32(addr, nr); + else + __rte_bit_once_clear32(addr, nr); +} + +__rte_experimental +static inline void +__rte_bit_once_assign64(volatile uint64_t *addr, unsigned int nr, bool value) +{ + if (value) + __rte_bit_once_set64(addr, nr); + else + __rte_bit_once_clear64(addr, nr); +} + /*------------------------ 32-bit relaxed operations ------------------------*/ /** @@ -953,6 +1123,11 @@ rte_log2_u64(uint64_t v) #undef rte_bit_clear #undef rte_bit_assign +#undef rte_bit_once_test +#undef rte_bit_once_set +#undef rte_bit_once_clear +#undef rte_bit_once_assign + #define __RTE_BIT_OVERLOAD_SZ_2(fun, qualifier, size, arg1_type, arg1_name) \ static inline void \ rte_bit_ ## fun(qualifier uint ## size ## _t *addr, \ @@ -1001,6 +1176,11 @@ __RTE_BIT_OVERLOAD_2(set, , unsigned int, nr) __RTE_BIT_OVERLOAD_2(clear, , unsigned int, nr) __RTE_BIT_OVERLOAD_3(assign, , unsigned int, nr, bool, value) +__RTE_BIT_OVERLOAD_2R(once_test, const volatile, bool, unsigned int, nr) +__RTE_BIT_OVERLOAD_2(once_set, volatile, unsigned int, nr) +__RTE_BIT_OVERLOAD_2(once_clear, volatile, unsigned int, nr) +__RTE_BIT_OVERLOAD_3(once_assign, volatile, unsigned int, nr, bool, value) + #endif #endif /* _RTE_BITOPS_H_ */ -- 2.34.1