From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 30DB343FAE; Sun, 5 May 2024 10:48:22 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1EDD4402E7; Sun, 5 May 2024 10:48:05 +0200 (CEST) Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05on2047.outbound.protection.outlook.com [40.107.20.47]) by mails.dpdk.org (Postfix) with ESMTP id AFE064028C for ; Sun, 5 May 2024 10:47:59 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JxY07GRHYrtMsU+kENXt/L6b5oVqff8jnexTIL7aceb/HDjMsyollIPLFoh8h7GV52d9hcMY4RVQww6hHB6ZXss14p0oBN4cBLttiLETJnqEHVmVQc3q5glRW3eeJzhNNPvkqA9CY0Ib7ePE5c2CuD3ybkqejsceajs/oH/6dwLGcSiHQ9Bs/6WsMdJvl8dHce4ZCuGxui6C/hKt8wKWoTqf+Y/EnFritOTtfznhOqhpfrBnDnq+T+RK18MspFvngl1W9ePLuobxdx+3+frgxShE7M9+3zjGsHDcFOykz8gUHE/4VXQquPsf3un8CtE7aEi6b3z+8YWTF+xU6O83RQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=2NtFc/85D5XqMWuHibOU2UGt9NdiuGqxaSS34YzRHUc=; b=cjvMoYsAvskAQyyrfCh/jHsWNQnUDrIth1xNXbdT06VKRkwQQ04WXfk+HoVrFI07FFbBnoDn37tlddkG6bcl17a+Dc9D7guw9UyW9fuJkblG8lx1CkSyFqWjgWL2IY3yIR6KXtj1IaRvd2SM29DsI//4TFkto4F+YMEFyT/l3gKi+HkzqF6xZZyR0ZTfE3UJu5Ukwr6jVT292u8pQQmOhvrr21klSzE2FDKbvphmbyoZJphhozFcblg/VAlWW8UiT7iXjRtoX4cnI6y46M6JhlQBjzlfq3gBOhHXttrvAZnmdEluQTGI8tp3JfVy68AkgVVt/iZ1ROtQ+zRRrsiC1Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=dpdk.org smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2NtFc/85D5XqMWuHibOU2UGt9NdiuGqxaSS34YzRHUc=; b=mTGL1TPLYV6/KV7F/Xm0/+QC1sdsSdcA0uGzGlHtvG894M6wbFRUdFCNDGNNSuSfIlfbXRTqskGu3izZ3OulTP4STBTApO4dZtxAaq+Lg8V49ckAoqRtStoJ0NX9XwqcH2WX62hvQxPhDSix/3BjZWsi+LhSc86X1RcCfXgBU6HbYTYKXUsbXe8fv8mpv3Vi7HSC/e0poH+U5X3mpVaojhF5lyTLiV9KTlx/fxDRKzePLpPTXi55nwOY7/DpwyKg7XMLeCf65POydy4sEMc6DFdHhYhiyegbIgqEg8GsqQN25LF0n9E7FZvw0FGd5exIPgJUKI+4Y/agq1Vf1x1lzw== Received: from DB7PR05CA0042.eurprd05.prod.outlook.com (2603:10a6:10:2e::19) by PAXPR07MB8793.eurprd07.prod.outlook.com (2603:10a6:102:244::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.40; Sun, 5 May 2024 08:47:58 +0000 Received: from DU2PEPF0001E9C1.eurprd03.prod.outlook.com (2603:10a6:10:2e:cafe::42) by DB7PR05CA0042.outlook.office365.com (2603:10a6:10:2e::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.39 via Frontend Transport; Sun, 5 May 2024 08:47:58 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by DU2PEPF0001E9C1.mail.protection.outlook.com (10.167.8.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7519.19 via Frontend Transport; Sun, 5 May 2024 08:47:58 +0000 Received: from seliicinfr00049.seli.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.64) with Microsoft SMTP Server id 15.2.1544.9; Sun, 5 May 2024 10:47:57 +0200 Received: from breslau.. (seliicwb00002.seli.gic.ericsson.se [10.156.25.100]) by seliicinfr00049.seli.gic.ericsson.se (Postfix) with ESMTP id 7E85038007A; Sun, 5 May 2024 10:47:57 +0200 (CEST) From: =?UTF-8?q?Mattias=20R=C3=B6nnblom?= To: CC: , Heng Wang , "Stephen Hemminger" , Tyler Retzlaff , =?UTF-8?q?Morten=20Br=C3=B8rup?= , =?UTF-8?q?Mattias=20R=C3=B6nnblom?= Subject: [RFC v7 3/6] eal: add exactly-once bit access functions Date: Sun, 5 May 2024 10:37:34 +0200 Message-ID: <20240505083737.118649-4-mattias.ronnblom@ericsson.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240505083737.118649-1-mattias.ronnblom@ericsson.com> References: <20240502055706.112443-2-mattias.ronnblom@ericsson.com> <20240505083737.118649-1-mattias.ronnblom@ericsson.com> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 8bit X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DU2PEPF0001E9C1:EE_|PAXPR07MB8793:EE_ X-MS-Office365-Filtering-Correlation-Id: cf9d5499-03d0-42a5-bc26-08dc6ce0104f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230031|376005|36860700004|82310400017|1800799015; X-Microsoft-Antispam-Message-Info: =?utf-8?B?QmlCRit6WGZaYUNBWlYzaFNVTmlqUlp5RUhOdnNVT0VvN1VWWFd5TDFKR1Vt?= =?utf-8?B?VFdpeDFZUnZiTE03R2NvQVZCRUIyWXVUNGJudS93WlMwNHFRSVQyS0diVy9P?= =?utf-8?B?bW0xbks1UVhzbEpZV3VpWTJCOHlpNWp5VGxtdWhDbHIwYTNVeUkwMEYwMFhO?= =?utf-8?B?TjY1YlR4VGVKdHIvZXNTZ3Nwem53WDRsTTlTNnJjWmd6ck1YZm9rR0xNWmZm?= =?utf-8?B?cnlzK1pHenJkZU5oUFIvV0paVVF6dDFjSlUreEJveVNTaHA3NmpyVmg3VFVH?= =?utf-8?B?ZHdpL3FIekpacTZDWkdudUJFTlRLeHJEZDNBQkUxSlNLR1R2RG1zeWJCeW1O?= =?utf-8?B?SzhsNURaU21BNm8yZ2hRaERvSERjYmQvOHc2cElYVThxMEppUTVIZXh4OXJw?= =?utf-8?B?UzY2MTFRblRGNmtvRmtRYkpIbFJSM0NZM3RjVk1HcUgyY0xRVFZRVFdSZVVp?= =?utf-8?B?ZlBRbWcxYVZvNTJRSitHUm1FZnZsdzlMNkRQbkVKS1V0cCtDTGIwSTlEbWFB?= =?utf-8?B?R1R4N3pnaTlDNUh2b05sYUpyWFR1MkFZWFM3TFVnWHIvcGV0a2VMWVFvSXl1?= =?utf-8?B?ZU1oMWpvQkxPbVZoaklRR0tuQUhtMmFUSWk1Z3lXRnl6OUxaYytKVW12ZDdy?= =?utf-8?B?RENuYllVZk1HaUVockpYRU5EUEVZbkZtMmEvdXFXQU8xUkk5dnZ0eDhVY0dI?= =?utf-8?B?c3hkTnZxOWFEa0hDT055OFAvTVNPSzc2YktkcllZUGg1QXJFanA2OVZBWUFo?= =?utf-8?B?U0xwMlhWNC85U0F0QnlWOGVIMU5sRnUzRmd4b3Facm5kVTNYRzJBYjF4V3pE?= =?utf-8?B?ZVlWc09UVmR1WGUwT1QyVUtpajUvalduZVRUanlpa1B4Qm1ISUlxT0NuUnhL?= =?utf-8?B?bTJMcmoxRmZxVFNJWWNvUGVmVHo3YXB1bUJ4M2RGRXQyYXhwZVZVZFhsazFh?= =?utf-8?B?NkNwcnlUM1A2NjdENEo2RTVaZGtKWGg5VXQ2cFJFZGJFbVU1U1MrendaQUpv?= =?utf-8?B?WTZkRERNTG4vN0g4SVJhaVhoai8vZHV3a0F4NUQvbWJ4ZEdoaGlEMTJxcjdP?= =?utf-8?B?Y3g2dFVhbXBKTStEWUFNeFNSR09YMjFVRVo1ZFZlSURjdTZFSmxUS3dFN3hJ?= =?utf-8?B?QXVzUUhtcDdVMU5FeURJb25JV2Jpcmhwc0J4ajBhMlFsL3VtekVYZnpBSzBO?= =?utf-8?B?RTRuOG15SjNYelNRN1M5cUlRbGpaQXpOa2s0SnFGZHFlWllkMis2dVQ1Qm1C?= =?utf-8?B?a3g4UVQ1bTBNdDhicnZXVlNHRlE5MkpKSTIyWXdNQXF6b2czMytBMkNwVUxF?= =?utf-8?B?Yk1FVnVuVysvYTNPUlRLeU5QQnBDSVM3RVEwdWYvNWdMbzB5WXYzMGhJRDky?= =?utf-8?B?RXh6UlZhdTI5Mm5VYkJYMHdTZkNZWkExNkphTkhaVitKcWpLYmdXSUVGZzAy?= =?utf-8?B?UWpISjlXZmhDOEppb09icG1DU205NnVzQ0dVT3kyTTNvRFFSZHN4djFudDAx?= =?utf-8?B?aTJRZ3EvOWdmWkloRkxMS3JQd2dnWGV3MEJiN1NPbldsRlFyNW0wRVZ3YnFh?= =?utf-8?B?eXAzeUI4YWRORVVVcXlqeDRpV2NrK2ZhdHpyalBvQitCTEZVVE8zbHJwbmF3?= =?utf-8?B?cGtseTMvQ3VMeHlDRnRNK21BZy9jRzBZN0dsY0RQVTlIY3l0bjU3U1RHRTdG?= =?utf-8?B?cXhGNjBiQ3BWU2lQWVNsdlBNQmJ4a0JYWlU0bFh4S05SQmsvclhQUXBrcStl?= =?utf-8?B?V0NuRTAva2FYRGZaTFZLZDJ5ZE0zY1pkdm5leGxvNkFPdWYwb2tqRTRISUFk?= =?utf-8?B?bno5M2FYaHJCYzVORit0NXIyVDRrM1lHODJXZlgxUFk3aWVXRGwrVThXQ05m?= =?utf-8?Q?vdCbQXsLHAjQx?= X-Forefront-Antispam-Report: CIP:192.176.1.74; CTRY:SE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:oa.msg.ericsson.com; PTR:office365.se.ericsson.net; CAT:NONE; SFS:(13230031)(376005)(36860700004)(82310400017)(1800799015); DIR:OUT; SFP:1101; X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 May 2024 08:47:58.1121 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: cf9d5499-03d0-42a5-bc26-08dc6ce0104f X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f; Ip=[192.176.1.74]; Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: DU2PEPF0001E9C1.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR07MB8793 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add test/set/clear/assign/flip functions which prevents certain compiler optimizations and guarantees that program-level memory loads and/or stores will actually occur. These functions are useful when interacting with memory-mapped hardware devices. The "once" family of functions does not promise atomicity and provides no memory ordering guarantees beyond the C11 relaxed memory model. RFC v7: * Fix various minor issues in documentation. RFC v6: * Have rte_bit_once_test() accept const-marked bitsets. RFC v3: * Work around lack of C++ support for _Generic (Tyler Retzlaff). Signed-off-by: Mattias Rönnblom Acked-by: Morten Brørup Acked-by: Tyler Retzlaff --- lib/eal/include/rte_bitops.h | 201 +++++++++++++++++++++++++++++++++++ 1 file changed, 201 insertions(+) diff --git a/lib/eal/include/rte_bitops.h b/lib/eal/include/rte_bitops.h index 3297133e22..3644aa115c 100644 --- a/lib/eal/include/rte_bitops.h +++ b/lib/eal/include/rte_bitops.h @@ -226,6 +226,183 @@ extern "C" { uint32_t *: __rte_bit_flip32, \ uint64_t *: __rte_bit_flip64)(addr, nr) +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Generic selection macro to test exactly once the value of a bit in + * a 32-bit or 64-bit word. The type of operation depends on the type + * of the @c addr parameter. + * + * rte_bit_once_test() is guaranteed to result in exactly one memory + * load (e.g., it may not be eliminate or merged by the compiler). + * + * \code{.c} + * rte_bit_once_set(addr, 17); + * if (rte_bit_once_test(addr, 17)) { + * ... + * } + * \endcode + * + * In the above example, rte_bit_once_set() may not be removed by + * the compiler, which would be allowed in case rte_bit_set() and + * rte_bit_test() was used. + * + * \code{.c} + * while (rte_bit_once_test(addr, 17); + * ; + * \endcode + * + * In case rte_bit_test(addr, 17) was used instead, the resulting + * object code could (and in many cases would be) replaced with + * the equivalent to + * \code{.c} + * if (rte_bit_test(addr, 17)) { + * for (;;) // spin forever + * ; + * } + * \endcode + * + * rte_bit_once_test() does not give any guarantees in regards to + * memory ordering or atomicity. + * + * The regular bit set operations (e.g., rte_bit_test()) should be + * preferred over the "once" family of operations (e.g., + * rte_bit_once_test()) if possible, since the latter may prevent + * optimizations crucial for run-time performance. + * + * @param addr + * A pointer to the word to query. + * @param nr + * The index of the bit. + * @return + * Returns true if the bit is set, and false otherwise. + */ + +#define rte_bit_once_test(addr, nr) \ + _Generic((addr), \ + uint32_t *: __rte_bit_once_test32, \ + const uint32_t *: __rte_bit_once_test32, \ + uint64_t *: __rte_bit_once_test64, \ + const uint64_t *: __rte_bit_once_test64)(addr, nr) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Set bit in word exactly once. + * + * Generic selection macro to set bit specified by @c nr in the word + * pointed to by @c addr to '1' exactly once. + * + * rte_bit_once_set() is guaranteed to result in exactly one memory + * load and exactly one memory store, *or* an atomic bit set + * operation. + * + * See rte_bit_test_once32() for more information and uses cases for + * the "once" class of functions. + * + * This macro does not give any guarantees in regards to memory + * ordering or atomicity. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + */ + +#define rte_bit_once_set(addr, nr) \ + _Generic((addr), \ + uint32_t *: __rte_bit_once_set32, \ + uint64_t *: __rte_bit_once_set64)(addr, nr) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Clear bit in word exactly once. + * + * Generic selection macro to set bit specified by @c nr in the word + * pointed to by @c addr to '0' exactly once. + * + * rte_bit_once_clear() is guaranteed to result in exactly one memory load + * and exactly one memory store, *or* an atomic bit clear operation. + * + * See rte_bit_test_once() for more information and uses cases for + * the "once" class of functions. + * + * This macro does not give any guarantees in regards to memory + * ordering or atomicity. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + */ +#define rte_bit_once_clear(addr, nr) \ + _Generic((addr), \ + uint32_t *: __rte_bit_once_clear32, \ + uint64_t *: __rte_bit_once_clear64)(addr, nr) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Assign a value to bit in a word exactly once. + * + * Generic selection macro to set bit specified by @c nr in the word + * pointed to by @c addr to the value indicated by @c value exactly + * once. + * + * rte_bit_once_assign() is guaranteed to result in exactly one memory + * load and exactly one memory store, *or* an atomic bit clear + * operation. + * + * This macro does not give any guarantees in regards to memory + * ordering or atomicity. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + * @param value + * The new value of the bit - true for '1', or false for '0'. + */ +#define rte_bit_once_assign(addr, nr, value) \ + _Generic((addr), \ + uint32_t *: __rte_bit_once_assign32, \ + uint64_t *: __rte_bit_once_assign64)(addr, nr, value) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Flip bit in word, reading and writing exactly once. + * + * Generic selection macro to change the value of a bit to '0' if '1' + * or '1' if '0' in a 32-bit or 64-bit word. The type of operation + * depends on the type of the @c addr parameter. + * + * rte_bit_once_flip() is guaranteed to result in exactly one memory + * load and exactly one memory store, *or* an atomic bit flip + * operation. + * + * See rte_bit_test_once() for more information and uses cases for the + * "once" class of functions. + * + * This macro does not give any guarantees in regards to memory + * ordering or atomicity. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + */ +#define rte_bit_once_flip(addr, nr) \ + _Generic((addr), \ + uint32_t *: __rte_bit_once_flip32, \ + uint64_t *: __rte_bit_once_flip64)(addr, nr) + #define __RTE_GEN_BIT_TEST(family, fun, qualifier, size) \ __rte_experimental \ static inline bool \ @@ -298,6 +475,18 @@ __RTE_GEN_BIT_CLEAR(, clear,, 64) __RTE_GEN_BIT_ASSIGN(, assign,, 64) __RTE_GEN_BIT_FLIP(, flip,, 64) +__RTE_GEN_BIT_TEST(once_, test, volatile, 32) +__RTE_GEN_BIT_SET(once_, set, volatile, 32) +__RTE_GEN_BIT_CLEAR(once_, clear, volatile, 32) +__RTE_GEN_BIT_ASSIGN(once_, assign, volatile, 32) +__RTE_GEN_BIT_FLIP(once_, flip, volatile, 32) + +__RTE_GEN_BIT_TEST(once_, test, volatile, 64) +__RTE_GEN_BIT_SET(once_, set, volatile, 64) +__RTE_GEN_BIT_CLEAR(once_, clear, volatile, 64) +__RTE_GEN_BIT_ASSIGN(once_, assign, volatile, 64) +__RTE_GEN_BIT_FLIP(once_, flip, volatile, 64) + /*------------------------ 32-bit relaxed operations ------------------------*/ /** @@ -993,6 +1182,12 @@ rte_log2_u64(uint64_t v) #undef rte_bit_assign #undef rte_bit_flip +#undef rte_bit_once_test +#undef rte_bit_once_set +#undef rte_bit_once_clear +#undef rte_bit_once_assign +#undef rte_bit_once_flip + #define __RTE_BIT_OVERLOAD_SZ_2(fun, qualifier, size, arg1_type, arg1_name) \ static inline void \ rte_bit_ ## fun(qualifier uint ## size ## _t *addr, \ @@ -1042,6 +1237,12 @@ __RTE_BIT_OVERLOAD_2(clear,, unsigned int, nr) __RTE_BIT_OVERLOAD_3(assign,, unsigned int, nr, bool, value) __RTE_BIT_OVERLOAD_2(flip,, unsigned int, nr) +__RTE_BIT_OVERLOAD_2R(once_test, const volatile, bool, unsigned int, nr) +__RTE_BIT_OVERLOAD_2(once_set, volatile, unsigned int, nr) +__RTE_BIT_OVERLOAD_2(once_clear, volatile, unsigned int, nr) +__RTE_BIT_OVERLOAD_3(once_assign, volatile, unsigned int, nr, bool, value) +__RTE_BIT_OVERLOAD_2(once_flip, volatile, unsigned int, nr) + #endif #endif /* _RTE_BITOPS_H_ */ -- 2.34.1