From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9CCB143C3A; Sat, 2 Mar 2024 15:02:47 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4DFA042DC3; Sat, 2 Mar 2024 15:02:22 +0100 (CET) Received: from EUR02-VI1-obe.outbound.protection.outlook.com (mail-vi1eur02on2051.outbound.protection.outlook.com [40.107.241.51]) by mails.dpdk.org (Postfix) with ESMTP id 3981242D7D for ; Sat, 2 Mar 2024 15:02:18 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=UGTK7sORqRqccFFUEz6w6CvAmnyO0FS+NMY69/YLqJ1f/jSh3DnfT9XatyAbPQKvDmg/lN3xywtXQcUi5oaVmGo7Y7PPEYxaeFXwz+JDCG/N4DC0NmOiJPY2K9lnVlPXSrspHOfwIloHOjYWxghIhVff8bS+GQ2dzx7sjNzjW1w+kJwkRX39gexI9r/w+tb9dbSc3Tp2OxSb6vT1lz0NRlHBSraU4eIFa0yCbJQmVaQtdHhb5n6aHy6TVTPz7MbJ6U4SPy199wB3jsvDGcGvZCc2dbMeeUVBf4FebqT/b8swj3izwaMtUJBegA9ozq9w4MQ5tkVVSEwNv457vbeokQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=4U8aKmI7T9xrmV77i4eS/o7kXhqw+nMOJ0MHBBGteR0=; b=nxxf0YxQijIHNijx48wTewYVnuESVsOmTnh+Pwfz1nBINdqIHHuQPasaekwixUc9IHViBAxht2q4wKkAAxHqdD8dW/6oD6P9ropG8ImmrZ8zVjXfhHChaTSk3FyTCWf7kBBzgmc8aLB80t9MaH/opqs9UzlMeXVskVbgv1mQTf2r546+v53JRes1u5ILZkvx4XnkqvM7pBGISzCt+CAv1Gy0ke6YDTuuAASRRNLT0UZ3jq5Oam2wYh7FMbOd/437/nGMHzoiLEUmzuSMIe5dNn5Y3cTERgXa+gDD1XJcbjZWZgmUcLeVcBhpZ2wcnLZw7EwZiOU31qSuTXzDEYU9pQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=dpdk.org smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4U8aKmI7T9xrmV77i4eS/o7kXhqw+nMOJ0MHBBGteR0=; b=CeW9AyF3maJszxozaSZfvNECOAE6FgxH+xhepQG+ogEMwBiY1wz/cp0ProDPs+22rm5Srhq0YzVJbZnDDx4ddEQiuEvf6aLZfALBJ9oprF3C+DIXkpmSnTA4m/NkJHVAmqy+B3hQltniZYS5BQ+SBz8umnVoku6Bi2IdP5vrw17pOAWDppBMq/2xwE8+13lLudijaaogbDSn6warH5S66HZtE+4M6ZmC3Rff3VofR3ortwm12DBzi3bQRU9sDppUWHyrLtY0mMvi5YerCn+6UG200QrtY9N1Ki4j5gfHBn+exu645/NaibvOBQfyH0ZrBf3AkgtLLbKRSHNsy3hPNQ== Received: from AM7PR03CA0010.eurprd03.prod.outlook.com (2603:10a6:20b:130::20) by DB8PR07MB6428.eurprd07.prod.outlook.com (2603:10a6:10:132::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7339.32; Sat, 2 Mar 2024 14:02:16 +0000 Received: from AMS0EPF00000197.eurprd05.prod.outlook.com (2603:10a6:20b:130:cafe::a2) by AM7PR03CA0010.outlook.office365.com (2603:10a6:20b:130::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7339.34 via Frontend Transport; Sat, 2 Mar 2024 14:02:16 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by AMS0EPF00000197.mail.protection.outlook.com (10.167.16.219) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7362.11 via Frontend Transport; Sat, 2 Mar 2024 14:02:15 +0000 Received: from seliicinfr00050.seli.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.66) with Microsoft SMTP Server id 15.2.1258.12; Sat, 2 Mar 2024 15:02:15 +0100 Received: from breslau.. (seliicwb00002.seli.gic.ericsson.se [10.156.25.100]) by seliicinfr00050.seli.gic.ericsson.se (Postfix) with ESMTP id 502561C006A; Sat, 2 Mar 2024 15:02:15 +0100 (CET) From: =?UTF-8?q?Mattias=20R=C3=B6nnblom?= To: CC: , Heng Wang , =?UTF-8?q?Mattias=20R=C3=B6nnblom?= Subject: [RFC 3/7] eal: add bit manipulation functions which read or write once Date: Sat, 2 Mar 2024 14:53:24 +0100 Message-ID: <20240302135328.531940-4-mattias.ronnblom@ericsson.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240302135328.531940-1-mattias.ronnblom@ericsson.com> References: <20240302135328.531940-1-mattias.ronnblom@ericsson.com> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 8bit X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AMS0EPF00000197:EE_|DB8PR07MB6428:EE_ X-MS-Office365-Filtering-Correlation-Id: 06f2e0a1-1799-4b23-2a8f-08dc3ac15df7 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: OEWnNlWyM9puiP48P6Ga6ynEJ1TTioPwFCQoSLa2GMoxD05YWIP82X9MwukFALeK/NkYWVHuC95/pcO/l6dMU1mZjCwUsLOvzr0fR+wSYPOo8S9/7ADYrFEme7+27+AoPGErfrzrDfECxx//E0foBI+eB7wgAMpyXsD+SSJ3/nm80oIKsHOSiUl2X+BrcJpBMReD5N+bC22zBgoHFM7G+22LxVASwDOyVs/R1OGEgV/0ub05fUzkHgZsm1bFrHK3nV5YPLjFypoqeemcvI+hSVLSGbWFl8IgZBf9RumUXYt+ayih5diNGORNN30FUYYdu9Z/KLC1Vx0vyGDu92AaSrH9rvdnKwWUkaBdmWfFaugFC/qVbIDS4CVChQ/A4/JBHAhvKBb8JEn93Xx1I1lAETFs8pgfpNHVKlo4BC1bcNrr2lY8ZXmaJU98rDIBQ+UFkTTOykU0ErUIc7cloJSN5eNLtlHK/+rn2Q/rLEZuAbaRae/WjY4bZ8TQ6l3DHMt19EjMaKT6PWfIqjHedgzF3YcDT6H8co6+O4Gw/ROo3OjfTq+Jy7QJrIoNqJv1zA8ENiKzgeSqytobhg7pYkEItKh+ClLSbbhd9a+n3/K9SEC983Uoq18CcSfmrpeSFhSrsks+DQ0CNPlCYOfutfI0FlU21OraFjXZ7TuHlP9ejRj1xy7KTEcC60hEnPQKxA32lN7LY6XPjwFGhG3obmRK9k8WLIKhctJuX8mRiB/2BF+HuUJCC9Oo1U5rFpq6ynEw X-Forefront-Antispam-Report: CIP:192.176.1.74; CTRY:SE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:oa.msg.ericsson.com; PTR:office365.se.ericsson.net; CAT:NONE; SFS:(13230031)(376005)(36860700004)(82310400014); DIR:OUT; SFP:1101; X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Mar 2024 14:02:15.8721 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 06f2e0a1-1799-4b23-2a8f-08dc3ac15df7 X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f; Ip=[192.176.1.74]; Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: AMS0EPF00000197.eurprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB8PR07MB6428 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add bit test/set/clear/assign functions which prevents certain compiler optimizations and guarantees that program-level memory loads and/or stores will actually occur. These functions are useful when interacting with memory-mapped hardware devices. The "once" family of functions does not promise atomicity and provides no memory ordering guarantees beyond the C11 relaxed memory model. Signed-off-by: Mattias Rönnblom --- lib/eal/include/rte_bitops.h | 229 +++++++++++++++++++++++++++++++++++ 1 file changed, 229 insertions(+) diff --git a/lib/eal/include/rte_bitops.h b/lib/eal/include/rte_bitops.h index afd0f11033..3118c51748 100644 --- a/lib/eal/include/rte_bitops.h +++ b/lib/eal/include/rte_bitops.h @@ -338,6 +338,227 @@ rte_bit_assign64(uint64_t *addr, unsigned int nr, bool value) rte_bit_clear64(addr, nr); } +/** + * Test exactly once if a particular bit in a 32-bit word is set. + * + * This function is guaranteed to result in exactly one memory load + * (e.g., it may not be eliminate or merged by the compiler). + * + * \code{.c} + * rte_bit_once_set32(addr, 17); + * if (rte_bit_once_test32(addr, 17)) { + * ... + * } + * \endcode + * + * In the above example, rte_bit_once_set32() may not be removed by + * the compiler, which would be allowed in case rte_bit_set32() and + * rte_bit_test32() was used. + * + * \code{.c} + * while (rte_bit_once_test32(addr, 17); + * ; + * \endcode + * + * In case rte_bit_test32(addr, 17) was used instead, the resulting + * object code could (and in many cases would be) replaced with + * with the equivalent to + * \code{.c} + * if (rte_bit_test32(addr, 17)) { + * for (;;) // spin forever + * ; + * } + * \endcode + * + * The regular bit set operations (e.g., rte_bit_test32()) should be + * preffered over the "once" family of operations (e.g., + * rte_bit_once_test32()), since the latter may prevent optimizations + * crucial for run-time performance. + * + * This function does not give any guarantees in regards to memory + * ordering (except ordering from the same thread to the same memory + * location) or atomicity. + * + * @param addr + * A pointer to the 32-bit word to query. + * @param nr + * The index of the bit (0-31). + * @return + * Returns true if the bit is set, and false otherwise. + */ + +static inline bool +rte_bit_once_test32(const volatile uint32_t *addr, unsigned int nr); + +/** + * Set bit in 32-bit word exactly once. + * + * Set bit specified by @c nr in the 32-bit word pointed to by + * @c addr to '1'. + * + * This function is guaranteed to result in exactly one memory load + * and exactly one memory store, *or* an atomic bit set operation. + * + * See rte_bit_test_once32() for more information and uses cases for + * the "once" class of functions. + * + * This function does not give any guarantees in regards to memory + * ordering or atomicity. + * + * @param addr + * A pointer to the 32-bit word to modify. + * @param nr + * The index of the bit (0-31). + */ +static inline void +rte_bit_once_set32(volatile uint32_t *addr, unsigned int nr); + +/** + * Clear bit in 32-bit word exactly once. + * + * Set bit specified by @c nr in the 32-bit word pointed to by @c addr + * to '0'. + * + * This function is guaranteed to result in exactly one memory load + * and exactly one memory store, *or* an atomic bit clear operation. + * + * See rte_bit_once_test32() for more information and uses cases for the + * "once" class of functions. + * + * This function does not give any guarantees in regards to memory + * ordering or atomicity. + * + * @param addr + * A pointer to the 32-bit word to modify. + * @param nr + * The index of the bit (0-31). + */ +static inline void +rte_bit_once_clear32(volatile uint32_t *addr, unsigned int nr); + +/** + * Assign a value to bit in a 32-bit word exactly once. + * + * Set bit specified by @c nr in the 32-bit word pointed to by + * @c addr to the value indicated by @c value. + * + * This function is guaranteed to result in exactly one memory load + * and exactly one memory store, *or* an atomic bit clear operation. + * + * This function does not give any guarantees in regards to memory + * ordering or atomicity. + * + * @param addr + * A pointer to the 32-bit word to modify. + * @param nr + * The index of the bit (0-31). + * @param value + * The new value of the bit - true for '1', or false for '0'. + */ +static inline void +rte_bit_once_assign32(uint32_t *addr, unsigned int nr, bool value) +{ + if (value) + rte_bit_once_set32(addr, nr); + else + rte_bit_once_clear32(addr, nr); +} + +/** + * Test exactly once if a particular bit in a 64-bit word is set. + * + * This function is guaranteed to result in exactly one memory load. + * See rte_bit_once_test32() for more information and uses cases for the + * "once" class of functions. + * + * rte_v_bit_test64() does give any guarantees in regards to memory + * ordering or atomicity. + * + * @param addr + * A pointer to the 64-bit word to query. + * @param nr + * The index of the bit (0-63). + * @return + * Returns true if the bit is set, and false otherwise. + */ + +static inline bool +rte_bit_once_test64(const volatile uint64_t *addr, unsigned int nr); + +/** + * Set bit in 64-bit word exactly once. + * + * Set bit specified by @c nr in the 64-bit word pointed to by + * @c addr to '1'. + * + * This function is guaranteed to result in exactly one memory load + * and exactly one memory store, *or* an atomic bit set operation. + * + * See rte_bit_once_test32() for more information and uses cases for the + * "once" class of functions. + * + * This function does not give any guarantees in regards to memory + * ordering or atomicity. + * + * @param addr + * A pointer to the 64-bit word to modify. + * @param nr + * The index of the bit (0-63). + */ +static inline void +rte_bit_once_set64(volatile uint64_t *addr, unsigned int nr); + +/** + * Clear bit in 64-bit word exactly once. + * + * Set bit specified by @c nr in the 64-bit word pointed to by @c addr + * to '0'. + * + * This function is guaranteed to result in exactly one memory load + * and exactly one memory store, *or* an atomic bit clear operation. + * + * See rte_bit_once_test32() for more information and uses cases for the + * "once" class of functions. + * + * This function does not give any guarantees in regards to memory + * ordering or atomicity. + * + * @param addr + * A pointer to the 64-bit word to modify. + * @param nr + * The index of the bit (0-63). + */ +static inline void +rte_bit_once_clear64(volatile uint64_t *addr, unsigned int nr); + +/** + * Assign a value to bit in a 64-bit word exactly once. + * + * Set bit specified by @c nr in the 64-bit word pointed to by + * @c addr to the value indicated by @c value. + * + * This function is guaranteed to result in exactly one memory load + * and exactly one memory store, *or* an atomic bit clear operation. + * + * This function does not give any guarantees in regards to memory + * ordering or atomicity. + * + * @param addr + * A pointer to the 64-bit word to modify. + * @param nr + * The index of the bit (0-63). + * @param value + * The new value of the bit - true for '1', or false for '0'. + */ +static inline void +rte_bit_once_assign64(volatile uint64_t *addr, unsigned int nr, bool value) +{ + if (value) + rte_bit_once_set64(addr, nr); + else + rte_bit_once_clear64(addr, nr); +} + #define __RTE_GEN_BIT_TEST(name, size, qualifier) \ static inline bool \ name(const qualifier uint ## size ## _t *addr, unsigned int nr) \ @@ -376,6 +597,14 @@ __RTE_GEN_BIT_TEST(rte_bit_test64, 64,) __RTE_GEN_BIT_SET(rte_bit_set64, 64,) __RTE_GEN_BIT_CLEAR(rte_bit_clear64, 64,) +__RTE_GEN_BIT_TEST(rte_bit_once_test32, 32, volatile) +__RTE_GEN_BIT_SET(rte_bit_once_set32, 32, volatile) +__RTE_GEN_BIT_CLEAR(rte_bit_once_clear32, 32, volatile) + +__RTE_GEN_BIT_TEST(rte_bit_once_test64, 64, volatile) +__RTE_GEN_BIT_SET(rte_bit_once_set64, 64, volatile) +__RTE_GEN_BIT_CLEAR(rte_bit_once_clear64, 64, volatile) + /*------------------------ 32-bit relaxed operations ------------------------*/ /** -- 2.34.1