From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2BD9843FAC; Sun, 5 May 2024 09:44:13 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 371824067E; Sun, 5 May 2024 09:43:45 +0200 (CEST) Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05on2064.outbound.protection.outlook.com [40.107.20.64]) by mails.dpdk.org (Postfix) with ESMTP id CC132402E1 for ; Sun, 5 May 2024 09:43:37 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SRpqcUrcdlASfEUy6XWrTjsaplVX2VGImWIKVsyhcDO1OvFDTdSv+Wki5/POb7GawBhlThvWbnKBQ8t0OB9OwB/9DHmDfPsDi0YJ/hW9D91L0WXrGryqR9Bl+30xQH3TifbiCbMF+/f6TmJklfP7enWEks4YMyVLiv9EWYj/RCx1D65cqMtb8nYzMTEvscd13QICUfdhcxlDbG5zoVXDq9AB9YSUdA/EqH9cJ8yfnH+qrF4/7A25tnmw5iOrhsyXx/waHq0vIOWNE9tolPHValNSvptxzKuy5/h7Z04SaXPI3l6VvbjB2heSG8yHm8IstyG/R27+bDYcrDeZQ2OOJw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Mx2k72mHtMfvaE5BD+qrppgugwcEHxL9T3OLZibFkgE=; b=EI0LpDyEybiPrmEhYAUj2Wrms2ABOIC6CgCpTV+v+8bjd7OeffnjbwCM+ZZjCzhB/ovdBXXgyh9bbJX6hLB0LMgmG33Ewy8EnBLIn/gw4dz2XWNBeWU27iKEeWdUECTyvbZeF/RsUMrCq3tZwjG13AJxWu7sP3M9oj95O4S6ZVwux2GE4YRtd7b/DPaT5nOIIj9t02vbdoJzb2mSn86A0kiCSgBOFuN+xlej3EwDI6GLQ3vVSOmBKF+VqcIiJAKd/2yvtRl9w7eRjomF+6m/DI/aYDHpU1wu+eMZVz0WynwyAeTmv1rPdQiyzzwvk2SnL6nkLMBsZP57f5ngvnuyIA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=dpdk.org smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Mx2k72mHtMfvaE5BD+qrppgugwcEHxL9T3OLZibFkgE=; b=qaU7zNhxSRBSAOdXKQ1KrJst9E3O4mPwlQptaj4fsbvnM3AwUzvXJZu0CrCiAlyW9gqIWCszmVvn+2hi7jssQ1Njr9zvzsZb7hOwVQnUu6TC0dgirkWC/ONv95UuXmu4Kk3L7FmtoOKUwfx+C4Z1q/XXbgzUDASBllIG4zmw2JwOyNyL0WV9q+EP632krfSZ+CfW0kzWwaK0Zw7fJAi8e1EMtwOjJIGczjVKy6yjgetLNN88e9OACAoGRykZI7gjpUiRCi25zudycGx9JNN0bKt75ud0xh+nZ/VVrvAmB+ziy84ht0Lu1nRMF310uJiMjHXDZgvZJxkOnBxByXeNFg== Received: from DUZPR01CA0337.eurprd01.prod.exchangelabs.com (2603:10a6:10:4b8::10) by AM9PR07MB7812.eurprd07.prod.outlook.com (2603:10a6:20b:2fe::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.39; Sun, 5 May 2024 07:43:35 +0000 Received: from DB1PEPF000509E7.eurprd03.prod.outlook.com (2603:10a6:10:4b8:cafe::af) by DUZPR01CA0337.outlook.office365.com (2603:10a6:10:4b8::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.39 via Frontend Transport; Sun, 5 May 2024 07:43:35 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by DB1PEPF000509E7.mail.protection.outlook.com (10.167.242.57) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7519.19 via Frontend Transport; Sun, 5 May 2024 07:43:35 +0000 Received: from seliicinfr00049.seli.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.69) with Microsoft SMTP Server id 15.2.1544.9; Sun, 5 May 2024 09:43:34 +0200 Received: from breslau.. (seliicwb00002.seli.gic.ericsson.se [10.156.25.100]) by seliicinfr00049.seli.gic.ericsson.se (Postfix) with ESMTP id 884D038007A; Sun, 5 May 2024 09:43:34 +0200 (CEST) From: =?UTF-8?q?Mattias=20R=C3=B6nnblom?= To: CC: , =?UTF-8?q?Morten=20Br=C3=B8rup?= , Tyler Retzlaff , Stephen Hemminger , Harry van Haaren , =?UTF-8?q?Mattias=20R=C3=B6nnblom?= Subject: [RFC v5 5/6] service: use multi-word bitset to represent service flags Date: Sun, 5 May 2024 09:33:12 +0200 Message-ID: <20240505073313.118515-5-mattias.ronnblom@ericsson.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240505073313.118515-1-mattias.ronnblom@ericsson.com> References: <20240216102348.480407-1-mattias.ronnblom@ericsson.com> <20240505073313.118515-1-mattias.ronnblom@ericsson.com> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 8bit X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DB1PEPF000509E7:EE_|AM9PR07MB7812:EE_ X-MS-Office365-Filtering-Correlation-Id: 3c325b78-45fd-41cb-61d5-08dc6cd711cd X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230031|1800799015|376005|36860700004|82310400017; X-Microsoft-Antispam-Message-Info: =?utf-8?B?KzZ5eWVlRC91SCsyc1lOOFkxL1RQNFJpTlB6a3BuMTVTRnBNMFhlSGtqbUtY?= =?utf-8?B?c0dndDUydVM3L3kvRVVXYkdFd3lQd2hjUWFIRVFLb0FWY2ZyNmtiM25FalA4?= =?utf-8?B?bEVXUWRRaDZFSFQxdFdGdkNzMVFSa2s4V21LZnAwQ3REOWRXQ1hBbWlYZHFk?= =?utf-8?B?SDE3R1NHanpGbmlpSVFnUGdKNUNxejN1Y1N1YWY1dzVKY2pYNGc5SnFqMEI1?= =?utf-8?B?WElDd0FaR2xZMktCbEtmYTUyN21IZ3pDQ09qemN6MDFJTVE5dENOY0YrdjF1?= =?utf-8?B?eHkySURQamFIVlVsL1hFaTZNbHZldmdQRGtudVQySUl4QUxBYlFWa1Vndzdo?= =?utf-8?B?YlVJTy9ZY2R3Y0NQOWJTT0Yxck5PSytDdjlVNkM4cXptVTFFcWNES1NRanJK?= =?utf-8?B?b1JYZTloRDNSdDkrbEQwcm05TFlZVDZFNGE1WTRXNnRaU3FHdW0zbFRpR0Iw?= =?utf-8?B?ZmU4UGEyL054cmdBZGFjS0dRMTlUM0xZRCtEejdscDZNUjlCNUw3VG9oMVlQ?= =?utf-8?B?elZUc2lDTmJvbnQ2MzcvTERNY3IxVDE5RU9kOFJrV0lpOFg3dk1IR1hVNUlH?= =?utf-8?B?eHJ4L2p0N09lK2Q1NTFvbUVQdTkzcFFFTUJqUTJ0K3UxNk1TWmVLaW0zeTR4?= =?utf-8?B?RXQ5bnQyMXpzL1JVbERXWEovRDEvNUNsay9icGRyWGdFdlJLV0gzcndCVkgr?= =?utf-8?B?dkgySndMUFRzbzBaS0g2cTRySERJc0JFckMvMG5MdnduQ1pqSkJkNUtpYzRi?= =?utf-8?B?c2J1TlZWbWVpVjNhU3VQZjFEd2pHcnh4eWpJSk1LdDRrVDZBNENmNXJqNjR0?= =?utf-8?B?VGpxSUNidjA1Qlk0YzV3ZFE2dWxyRTdlZ2RScklVcE8yWVN4NVh2MDVvK0hD?= =?utf-8?B?R1VaQ1lHc2RxTTU1ZFB6RGpoYStrbm9CdFdZc0Y5Q2xrVUplRzZ1Qi9hc2hp?= =?utf-8?B?NEhNbjlWUzRaM0dxYmhreE4zbDlNNmJRNnBMR2t6Z3ZYUHJaYS9KN0V1Z212?= =?utf-8?B?d1FTS3Z2Tk5hTEFqaDEzVWZMcFUyV0tBQjR0aEsxWFNnaE03VkZBeUF0aS8r?= =?utf-8?B?SUNueU1CdkV6QkgyQXVEeXNYU25vK1FnKzdyaFUyRGdnVklwSHh6Q1UwakpR?= =?utf-8?B?T3lLMGpPRW5iUUlickJlV1VWcXdUY0ZUSU1TVCtBNEZjZWZyaGtvbnY1cGlQ?= =?utf-8?B?d08vN1ZmMkhTQ2duL0NTSUJYU3dSOCtXU29XWXpXL2x0czlyOGMrUGJlRDdJ?= =?utf-8?B?dW1NaVB3NTlTSjhjMGpHRXhyOUM3Q1FIdytTdTZucm5RaVZFUU9EMjhSQ1pa?= =?utf-8?B?VEM5c0FQRUR2cTc2Nzg1YmJna29JTDd0aVpuLy9KS3hzY0VCME9NQ1dGTVhi?= =?utf-8?B?K2J2MXFuN3FZakQ4T3RndWJYNjBkY3lnemthczBsRmxhbVZPbHF2ZzJJd3p3?= =?utf-8?B?N29xRnZpMWIrWTFGUitHMzA5ZlNhek1yWHRjUGhPWnY3a2k1dkJDbS9qQ05z?= =?utf-8?B?OUFNQkJjY0dIREt6b1lRSHQ0b1dtZGY0NjZuOGQ0R0hkcVQwR0VlYk5TQzlq?= =?utf-8?B?bWswaVI4enl0dkJYN2MzTUV1OUZ2Nmh3YjdYZjEwWTUxazYwU2VHTWFLMVJh?= =?utf-8?B?akVKclo2SDJZRXVsc2VoekV6SCtid1YvMjJLRFV3WUIyZi9MeFBMeWh5Z3Jw?= =?utf-8?B?NGR6OURvSmRlNTdTS0hrNWpNMlVHMUxvWTN2ZGUxa29pU3hEKzdGQzY4anZl?= =?utf-8?B?enZLSGQ2djVKYlBGNk9oQnExUFI0a3lxeDh0RHU2cEQ3Yld3RmtWY2RnNVly?= =?utf-8?B?ejFxcXFhMmFrTmM1amZLRnI4ckZUS2JBOXQyZ0txalpweGV5YmYzS3NwUkpm?= =?utf-8?Q?h+13k0VZkpXAN?= X-Forefront-Antispam-Report: CIP:192.176.1.74; CTRY:SE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:oa.msg.ericsson.com; PTR:office365.se.ericsson.net; CAT:NONE; SFS:(13230031)(1800799015)(376005)(36860700004)(82310400017); DIR:OUT; SFP:1101; X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 May 2024 07:43:35.1427 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 3c325b78-45fd-41cb-61d5-08dc6cd711cd X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f; Ip=[192.176.1.74]; Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: DB1PEPF000509E7.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR07MB7812 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Use a multi-word bitset to track which services are mapped to which lcores, allowing the RTE_SERVICE_NUM_MAX compile-time constant to be > 64. Replace array-of-bytes service-currently-active flags with a more compact multi-word bitset-based representation, reducing memory footprint somewhat. Signed-off-by: Mattias Rönnblom --- lib/eal/common/rte_service.c | 70 ++++++++++++++---------------------- 1 file changed, 27 insertions(+), 43 deletions(-) diff --git a/lib/eal/common/rte_service.c b/lib/eal/common/rte_service.c index 56379930b6..ec0f47e141 100644 --- a/lib/eal/common/rte_service.c +++ b/lib/eal/common/rte_service.c @@ -11,6 +11,7 @@ #include #include +#include #include #include #include @@ -63,11 +64,11 @@ struct service_stats { /* the internal values of a service core */ struct __rte_cache_aligned core_state { /* map of services IDs are run on this core */ - uint64_t service_mask; + RTE_BITSET_DECLARE(mapped_services, RTE_SERVICE_NUM_MAX); RTE_ATOMIC(uint8_t) runstate; /* running or stopped */ RTE_ATOMIC(uint8_t) thread_active; /* indicates when thread is in service_run() */ uint8_t is_service_core; /* set if core is currently a service core */ - uint8_t service_active_on_lcore[RTE_SERVICE_NUM_MAX]; + RTE_BITSET_DECLARE(service_active_on_lcore, RTE_SERVICE_NUM_MAX); RTE_ATOMIC(uint64_t) loops; RTE_ATOMIC(uint64_t) cycles; struct service_stats service_stats[RTE_SERVICE_NUM_MAX]; @@ -81,11 +82,6 @@ static uint32_t rte_service_library_initialized; int32_t rte_service_init(void) { - /* Hard limit due to the use of an uint64_t-based bitmask (and the - * clzl intrinsic). - */ - RTE_BUILD_BUG_ON(RTE_SERVICE_NUM_MAX > 64); - if (rte_service_library_initialized) { EAL_LOG(NOTICE, "service library init() called, init flag %d", @@ -296,7 +292,7 @@ rte_service_component_unregister(uint32_t id) /* clear the run-bit in all cores */ for (i = 0; i < RTE_MAX_LCORE; i++) - lcore_states[i].service_mask &= ~(UINT64_C(1) << id); + rte_bitset_clear(lcore_states[i].mapped_services, id); memset(&rte_services[id], 0, sizeof(struct rte_service_spec_impl)); @@ -410,7 +406,7 @@ service_runner_do_callback(struct rte_service_spec_impl *s, /* Expects the service 's' is valid. */ static int32_t -service_run(uint32_t i, struct core_state *cs, uint64_t service_mask, +service_run(uint32_t i, struct core_state *cs, const uint64_t *mapped_services, struct rte_service_spec_impl *s, uint32_t serialize_mt_unsafe) { if (!s) @@ -424,12 +420,12 @@ service_run(uint32_t i, struct core_state *cs, uint64_t service_mask, RUNSTATE_RUNNING || rte_atomic_load_explicit(&s->app_runstate, rte_memory_order_acquire) != RUNSTATE_RUNNING || - !(service_mask & (UINT64_C(1) << i))) { - cs->service_active_on_lcore[i] = 0; + !rte_bitset_test(mapped_services, i)) { + rte_bitset_clear(cs->service_active_on_lcore, i); return -ENOEXEC; } - cs->service_active_on_lcore[i] = 1; + rte_bitset_set(cs->service_active_on_lcore, i); if ((service_mt_safe(s) == 0) && (serialize_mt_unsafe == 1)) { if (!rte_spinlock_trylock(&s->execute_lock)) @@ -454,7 +450,7 @@ rte_service_may_be_active(uint32_t id) return -EINVAL; for (i = 0; i < lcore_count; i++) { - if (lcore_states[ids[i]].service_active_on_lcore[id]) + if (rte_bitset_test(lcore_states[ids[i]].service_active_on_lcore, id)) return 1; } @@ -474,7 +470,9 @@ rte_service_run_iter_on_app_lcore(uint32_t id, uint32_t serialize_mt_unsafe) */ rte_atomic_fetch_add_explicit(&s->num_mapped_cores, 1, rte_memory_order_relaxed); - int ret = service_run(id, cs, UINT64_MAX, s, serialize_mt_unsafe); + RTE_BITSET_DECLARE(all_services, RTE_SERVICE_NUM_MAX); + rte_bitset_set_all(all_services, RTE_SERVICE_NUM_MAX); + int ret = service_run(id, cs, all_services, s, serialize_mt_unsafe); rte_atomic_fetch_sub_explicit(&s->num_mapped_cores, 1, rte_memory_order_relaxed); @@ -485,7 +483,6 @@ static int32_t service_runner_func(void *arg) { RTE_SET_USED(arg); - uint8_t i; const int lcore = rte_lcore_id(); struct core_state *cs = &lcore_states[lcore]; @@ -497,20 +494,11 @@ service_runner_func(void *arg) */ while (rte_atomic_load_explicit(&cs->runstate, rte_memory_order_acquire) == RUNSTATE_RUNNING) { + ssize_t id; - const uint64_t service_mask = cs->service_mask; - uint8_t start_id; - uint8_t end_id; - - if (service_mask == 0) - continue; - - start_id = rte_ctz64(service_mask); - end_id = 64 - rte_clz64(service_mask); - - for (i = start_id; i < end_id; i++) { + RTE_BITSET_FOREACH_SET(id, cs->mapped_services, RTE_SERVICE_NUM_MAX) { /* return value ignored as no change to code flow */ - service_run(i, cs, service_mask, service_get(i), 1); + service_run(id, cs, cs->mapped_services, service_get(id), 1); } rte_atomic_store_explicit(&cs->loops, cs->loops + 1, rte_memory_order_relaxed); @@ -519,8 +507,7 @@ service_runner_func(void *arg) /* Switch off this core for all services, to ensure that future * calls to may_be_active() know this core is switched off. */ - for (i = 0; i < RTE_SERVICE_NUM_MAX; i++) - cs->service_active_on_lcore[i] = 0; + rte_bitset_clear_all(cs->service_active_on_lcore, RTE_SERVICE_NUM_MAX); /* Use SEQ CST memory ordering to avoid any re-ordering around * this store, ensuring that once this store is visible, the service @@ -586,7 +573,7 @@ rte_service_lcore_count_services(uint32_t lcore) if (!cs->is_service_core) return -ENOTSUP; - return rte_popcount64(cs->service_mask); + return rte_bitset_count_set(cs->mapped_services, RTE_SERVICE_NUM_MAX); } int32_t @@ -639,25 +626,23 @@ service_update(uint32_t sid, uint32_t lcore, uint32_t *set, uint32_t *enabled) !lcore_states[lcore].is_service_core) return -EINVAL; - uint64_t sid_mask = UINT64_C(1) << sid; if (set) { - uint64_t lcore_mapped = lcore_states[lcore].service_mask & - sid_mask; + uint64_t lcore_mapped = rte_bitset_test(lcore_states[lcore].mapped_services, sid); if (*set && !lcore_mapped) { - lcore_states[lcore].service_mask |= sid_mask; + rte_bitset_set(lcore_states[lcore].mapped_services, sid); rte_atomic_fetch_add_explicit(&rte_services[sid].num_mapped_cores, 1, rte_memory_order_relaxed); } if (!*set && lcore_mapped) { - lcore_states[lcore].service_mask &= ~(sid_mask); + rte_bitset_clear(lcore_states[lcore].mapped_services, sid); rte_atomic_fetch_sub_explicit(&rte_services[sid].num_mapped_cores, 1, rte_memory_order_relaxed); } } if (enabled) - *enabled = !!(lcore_states[lcore].service_mask & (sid_mask)); + *enabled = rte_bitset_test(lcore_states[lcore].mapped_services, sid); return 0; } @@ -699,11 +684,11 @@ set_lcore_state(uint32_t lcore, int32_t state) int32_t rte_service_lcore_reset_all(void) { - /* loop over cores, reset all to mask 0 */ + /* loop over cores, reset all mapped services */ uint32_t i; for (i = 0; i < RTE_MAX_LCORE; i++) { if (lcore_states[i].is_service_core) { - lcore_states[i].service_mask = 0; + rte_bitset_clear_all(lcore_states[i].mapped_services, RTE_SERVICE_NUM_MAX); set_lcore_state(i, ROLE_RTE); /* runstate act as guard variable Use * store-release memory order here to synchronize @@ -731,7 +716,7 @@ rte_service_lcore_add(uint32_t lcore) set_lcore_state(lcore, ROLE_SERVICE); /* ensure that after adding a core the mask and state are defaults */ - lcore_states[lcore].service_mask = 0; + rte_bitset_clear_all(lcore_states[lcore].mapped_services, RTE_SERVICE_NUM_MAX); /* Use store-release memory order here to synchronize with * load-acquire in runstate read functions. */ @@ -814,12 +799,11 @@ rte_service_lcore_stop(uint32_t lcore) uint32_t i; struct core_state *cs = &lcore_states[lcore]; - uint64_t service_mask = cs->service_mask; for (i = 0; i < RTE_SERVICE_NUM_MAX; i++) { - int32_t enabled = service_mask & (UINT64_C(1) << i); - int32_t service_running = rte_service_runstate_get(i); - int32_t only_core = (1 == + bool enabled = rte_bitset_test(cs->mapped_services, i); + bool service_running = rte_service_runstate_get(i); + bool only_core = (1 == rte_atomic_load_explicit(&rte_services[i].num_mapped_cores, rte_memory_order_relaxed)); -- 2.34.1