From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3A9FE45BAE; Wed, 23 Oct 2024 10:02:16 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2CE5E40E5A; Wed, 23 Oct 2024 10:02:16 +0200 (CEST) Received: from EUR03-DBA-obe.outbound.protection.outlook.com (mail-dbaeur03on2051.outbound.protection.outlook.com [40.107.104.51]) by mails.dpdk.org (Postfix) with ESMTP id 2433E4065B for ; Wed, 23 Oct 2024 10:02:08 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=GIJZLW9kHDxnm3jDT225CxWgNZ2cD6SmUEWj0JWY6fraLIN2LxetICj2cFswSo+EVwGQFxV6zIvZlq8oMzISmXf11cxxMnciGd7MjYA1r7NhQdE2DVy3Q3lt9wgWGvp3k8xn/FXPbPgUIiTj3LAJ+x9lgt8KCST5ejZYD5sE6FN0dtCZJDuFoQrn9uCxD6x8nPJEeNKbYk22V6fVHyF08UFAYpkzqT2P0V288Q/dnLlQm03jjnK51HhOqat5BUTmyKQ8YiD9nyceTXqRdWKF2GynZzKEmF+R8gyLlwRYdevrl2eDqCoGRpWhf7iyRW6W2NglVpvecJ/nYMYnh9HdLw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=bZe54tHV1BWuXkkOzb0KIXBGV4vegSrp2GI9ZzhuZjE=; b=e/JSF2+UkV+jImSz5Ilf9CeYW0Bh7cOcp7twC3J2qzzfGBgMV3AvBtJFt/nRaKLj2xbBbK35z69Ld46o7bUghvUEgdrHa0kGMDFJImfXhhyYiATY9f9epl13hddtvGf07ywtzq1V2HA3LwnVtSFoDK5A7Cq4ADeFq+x/frQt+VpitepdVPbv7V+kCMcdWTPUUn69c4owxyj6rwThkaG1IMvful1k7AlAgVSzaGUaYONECULdNqqcpDSoms20DYvUCH4n9t2T8FiodgnMMoW5Pvif8+rqrBvjTw/uyi//KVZoGymmbEMdJfaI/j/tDQ26WHXFDLausvEfU6RsfGdCfw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=dpdk.org smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bZe54tHV1BWuXkkOzb0KIXBGV4vegSrp2GI9ZzhuZjE=; b=oZIarwFuF+Y6TEh+AnDhDgOSl26obJlvenOOwRLWd6QWGXAB95GIU37buelfbsZmc3AICk4P2IvpAyY5479OW2h8XNjL1o1uRFgJ6DVoQBjy2XqLQpg/VCSMG3UiswW6COKrVo91htE+eudaJfjJVK1lH3/3Dq4zK5go66lBRoSZXp7eM27gzXBAoyCuVgVDcElS0PMUAtGZudaqW4wbpXaejtXF5Ixfjb6D8lkWPnIhpNnTr27hCOol+86VUFCjBcp98+eGFjdjlxiPM8MGpROeCqdPv3Vaut7ZMdHIBAJtWbcbO/CcECiMXIrPwQY1+C/FnTmbtMk1cPwzxhVhVA== Received: from DUZPR01CA0314.eurprd01.prod.exchangelabs.com (2603:10a6:10:4ba::21) by AS8PR07MB7254.eurprd07.prod.outlook.com (2603:10a6:20b:25f::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8069.28; Wed, 23 Oct 2024 08:02:06 +0000 Received: from DU2PEPF00028D0A.eurprd03.prod.outlook.com (2603:10a6:10:4ba:cafe::f6) by DUZPR01CA0314.outlook.office365.com (2603:10a6:10:4ba::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.17 via Frontend Transport; Wed, 23 Oct 2024 08:02:05 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by DU2PEPF00028D0A.mail.protection.outlook.com (10.167.242.170) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.14 via Frontend Transport; Wed, 23 Oct 2024 08:02:05 +0000 Received: from seliicinfr00049.seli.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.60) with Microsoft SMTP Server id 15.2.1544.11; Wed, 23 Oct 2024 10:02:03 +0200 Received: from breslau.. (seliicwb00002.seli.gic.ericsson.se [10.156.25.100]) by seliicinfr00049.seli.gic.ericsson.se (Postfix) with ESMTP id 6EBFF3800B8; Wed, 23 Oct 2024 10:02:03 +0200 (CEST) From: =?UTF-8?q?Mattias=20R=C3=B6nnblom?= To: CC: , =?UTF-8?q?Morten=20Br=C3=B8rup?= , Stephen Hemminger , Konstantin Ananyev , David Marchand , Jerin Jacob , Luka Jankovic , Thomas Monjalon , =?UTF-8?q?Mattias=20R=C3=B6nnblom?= , Konstantin Ananyev , Chengwen Feng Subject: [PATCH v16 7/8] service: keep per-lcore state in lcore variable Date: Wed, 23 Oct 2024 09:53:01 +0200 Message-ID: <20241023075302.869008-8-mattias.ronnblom@ericsson.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241023075302.869008-1-mattias.ronnblom@ericsson.com> References: <20241017055743.848292-1-mattias.ronnblom@ericsson.com> <20241023075302.869008-1-mattias.ronnblom@ericsson.com> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 8bit X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DU2PEPF00028D0A:EE_|AS8PR07MB7254:EE_ X-MS-Office365-Filtering-Correlation-Id: 7f39eabf-814c-411f-a6a9-08dcf338fc7d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|36860700013|376014|7416014|82310400026|1800799024; X-Microsoft-Antispam-Message-Info: =?utf-8?B?Z2UrbnBZb09VcHJDS2JFdGtYZndka2t2T0lDbWZ1RHU4ZC9UMnpGQUFSbkNI?= =?utf-8?B?S0tjNThBYStFdFlTNHcxaGNSS2JDVXh2REluc2xpYm1WanFMYVdXTmRoU2RK?= =?utf-8?B?cndXcWRrRCtudkZYMWN1NGF3WjdWM0Q1ODE1M050TXVLQnlxWG1uRDNCUjdy?= =?utf-8?B?RlFKWHVvS2hUSDdqeW94WkprRHE0eHJWb3pSUW41aXFrd0xUa21DZHJQT21I?= =?utf-8?B?TEtpQ3doU1ZVc2tnb3VCMWVDVGFCbzRkbjNwSHZnanR1MHNUVFBJYjM2Ylkz?= =?utf-8?B?RXpxWUVBRXNpTndjckJUWjNzd01UcmJtYzNSTDYyeW42RVpHOVAxOVoxL1oy?= =?utf-8?B?TnJWUUpmZW5LOExjbldhamp1cnNZVVQydzVHVFg5bC82Sy80NGtod0htb2Zs?= =?utf-8?B?QkltTFFRYVZ2cHNlTXVFM2NIRUx6V1E1d3QwUTVwODF6UFNnQURNVzdiSkx2?= =?utf-8?B?ME5XTGpVd1E3d3RjcFY5KzE4RlI0cnZtV2dnVy9PUVd0aXpUNUt0OW43MnRw?= =?utf-8?B?Z0ZIcGNPYjQ4d0IwU3A2cEJFd0wwNE5YRDR1UXIzRGZCYXpnaTdzSlZuYkR6?= =?utf-8?B?VGJ1bFFhNFJpdEJoRThKMlNkK1ZQZHBXQUIrMll4cjk1TE1INTdUSHpnYklr?= =?utf-8?B?RUtlekluaXZmM1VwMi8vUkZBeGNLNkF6YVQ4OUg2MldycWZsY0FmY0hoN01S?= =?utf-8?B?ZmFYUndQREsyZUc1aXRVU0pna0IvYWY1Vll4TW1Td0tPV3IvNmJwTnYzUitQ?= =?utf-8?B?R2MxYlg4R0ZDTnFSOHBRZTRjM1prSzZNMjVhdWw4VVRPZmRBM3dnaVI4WFRF?= =?utf-8?B?TmVPRUhvRTNOUFNYcjFMNEd3UXRpL0ZFckpxalN2N1VQRllGazNvSnh3MHlR?= =?utf-8?B?bTRVWDlDeU1SS05mWlF1Y3FRUDRmY2ZPclZSZXZ3dFFlL2dXTmpPYTRVZDVl?= =?utf-8?B?MWs4aWpOWDU3UXZ4eFR4aDlzdUFtUGVxbU9HUkJ0cEFmTDRZd3FoZzN1M0hQ?= =?utf-8?B?amRKUnV0Nlpjc1N5dVhCOUY2VmxSUUJobDdMUUozaGFLWGpmSEk4dzgzcUc4?= =?utf-8?B?MEdsa2pvUXpob1AvRVNXeDdnNjJWYnZjdEViZjJ5NUlVSnNRZ0Z6eEI1RGV6?= =?utf-8?B?eGI5OHVkL24vWmNBdkpBVDdURUVGbUZRVE1Ydm5JbUIyZDV1RjJLOGVQZWhp?= =?utf-8?B?QzBSZkNPbXJydFE5SUpCL0k4azY4NnEwZkVkWnFsUHRPY3RHelNUdloyVkNO?= =?utf-8?B?S00yUFBabHBvUEFJY2s3R2pzUEFHSEZMUzZCOCtBc0RiTlM1OWZPWWJDaDU5?= =?utf-8?B?TjZqOFo1bnZRUCtBNUYyUFZvNTNuOFJQcCs2VER4aHBuNkhEME9qZ0k0bmht?= =?utf-8?B?cVlTUTFVRHJ2clA3WEo0MldyOGlHRGZpSzJWZVNZMjFvbVJGNUFQMm0zdzZQ?= =?utf-8?B?OUhqZVVaZVo1VXdXTllXeE1WMmpSUU0wQkpjMzVraWc2cEY0ME1lOE1qbmJX?= =?utf-8?B?SC9aVWViVCt0KzU1dENNcXpZQ25ScWVXUFZCK01zb0N2R3F3Um5VZ3pTZ1Bm?= =?utf-8?B?Rnl1Wk1XdFVpbzc1V0FoTDhCOG9JS1VKcmdKTmdvZGhUR2x5eHJ3dGZlbXJH?= =?utf-8?B?UUU5S2xJZi9ORjZLN2Vac3JWZXBna0Vnc0o1VGloM2J5ZkIxWm56bm93K2NN?= =?utf-8?B?NGpkQWlkTDdTU0J0YzQwbE1MenpFTFpzUVZkM3hTMG4zeGo1OVo3cHJYdGFi?= =?utf-8?B?UUVncnhyYzJGY0tvYXQxQTNtd1lXNng4SWlPdEFxMS9oK2JQMktRMFE5cnZI?= =?utf-8?B?WE1jM0k5bWJtbW1kVjI2VEtwcC9CNkZrbGFzeXNkM0dFTnhPK1dSMUJ3MDFl?= =?utf-8?B?WEI0bkFpVEZkN2lxdVZTdlFtOXBDTnJWNWJrS2pwdk4rRlE9PQ==?= X-Forefront-Antispam-Report: CIP:192.176.1.74; CTRY:SE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:oa.msg.ericsson.com; PTR:office365.se.ericsson.net; CAT:NONE; SFS:(13230040)(36860700013)(376014)(7416014)(82310400026)(1800799024); DIR:OUT; SFP:1101; X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Oct 2024 08:02:05.7860 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 7f39eabf-814c-411f-a6a9-08dcf338fc7d X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f; Ip=[192.176.1.74]; Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: DU2PEPF00028D0A.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR07MB7254 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Replace static array of cache-aligned structs with an lcore variable, to slightly benefit code simplicity and performance. Signed-off-by: Mattias Rönnblom Acked-by: Morten Brørup Acked-by: Konstantin Ananyev Acked-by: Chengwen Feng Acked-by: Stephen Hemminger -- PATCH v14: * Merge with bitset-related changes. PATCH v7: * Update to match new FOREACH API. RFC v6: * Remove a now-redundant lcore variable value memset(). RFC v5: * Fix lcore value pointer bug introduced by RFC v4. RFC v4: * Remove strange-looking lcore value lookup potentially containing invalid lcore id. (Morten Brørup) * Replace misplaced tab with space. (Morten Brørup) --- lib/eal/common/rte_service.c | 116 ++++++++++++++++++++--------------- 1 file changed, 65 insertions(+), 51 deletions(-) diff --git a/lib/eal/common/rte_service.c b/lib/eal/common/rte_service.c index 324471e897..dad3150df9 100644 --- a/lib/eal/common/rte_service.c +++ b/lib/eal/common/rte_service.c @@ -11,6 +11,7 @@ #include #include +#include #include #include #include @@ -78,7 +79,7 @@ struct __rte_cache_aligned core_state { static uint32_t rte_service_count; static struct rte_service_spec_impl *rte_services; -static struct core_state *lcore_states; +static RTE_LCORE_VAR_HANDLE(struct core_state, lcore_states); static uint32_t rte_service_library_initialized; int32_t @@ -99,12 +100,8 @@ rte_service_init(void) goto fail_mem; } - lcore_states = rte_calloc("rte_service_core_states", RTE_MAX_LCORE, - sizeof(struct core_state), RTE_CACHE_LINE_SIZE); - if (!lcore_states) { - EAL_LOG(ERR, "error allocating core states array"); - goto fail_mem; - } + if (lcore_states == NULL) + RTE_LCORE_VAR_ALLOC(lcore_states); int i; struct rte_config *cfg = rte_eal_get_configuration(); @@ -120,7 +117,6 @@ rte_service_init(void) return 0; fail_mem: rte_free(rte_services); - rte_free(lcore_states); return -ENOMEM; } @@ -134,7 +130,6 @@ rte_service_finalize(void) rte_eal_mp_wait_lcore(); rte_free(rte_services); - rte_free(lcore_states); rte_service_library_initialized = 0; } @@ -284,7 +279,6 @@ rte_service_component_register(const struct rte_service_spec *spec, int32_t rte_service_component_unregister(uint32_t id) { - uint32_t i; struct rte_service_spec_impl *s; SERVICE_VALID_GET_OR_ERR_RET(id, s, -EINVAL); @@ -292,9 +286,11 @@ rte_service_component_unregister(uint32_t id) s->internal_flags &= ~(SERVICE_F_REGISTERED); + unsigned int lcore_id; + struct core_state *cs; /* clear the run-bit in all cores */ - for (i = 0; i < RTE_MAX_LCORE; i++) - rte_bitset_clear(lcore_states[i].mapped_services, id); + RTE_LCORE_VAR_FOREACH(lcore_id, cs, lcore_states) + rte_bitset_clear(cs->mapped_services, id); memset(&rte_services[id], 0, sizeof(struct rte_service_spec_impl)); @@ -463,7 +459,10 @@ rte_service_may_be_active(uint32_t id) return -EINVAL; for (i = 0; i < lcore_count; i++) { - if (rte_bitset_test(lcore_states[ids[i]].service_active_on_lcore, id)) + struct core_state *cs = + RTE_LCORE_VAR_LCORE(ids[i], lcore_states); + + if (rte_bitset_test(cs->service_active_on_lcore, id)) return 1; } @@ -473,7 +472,7 @@ rte_service_may_be_active(uint32_t id) int32_t rte_service_run_iter_on_app_lcore(uint32_t id, uint32_t serialize_mt_unsafe) { - struct core_state *cs = &lcore_states[rte_lcore_id()]; + struct core_state *cs = RTE_LCORE_VAR(lcore_states); struct rte_service_spec_impl *s; SERVICE_VALID_GET_OR_ERR_RET(id, s, -EINVAL); @@ -496,8 +495,7 @@ static int32_t service_runner_func(void *arg) { RTE_SET_USED(arg); - const int lcore = rte_lcore_id(); - struct core_state *cs = &lcore_states[lcore]; + struct core_state *cs = RTE_LCORE_VAR(lcore_states); rte_atomic_store_explicit(&cs->thread_active, 1, rte_memory_order_seq_cst); @@ -533,13 +531,15 @@ service_runner_func(void *arg) int32_t rte_service_lcore_may_be_active(uint32_t lcore) { - if (lcore >= RTE_MAX_LCORE || !lcore_states[lcore].is_service_core) + struct core_state *cs = RTE_LCORE_VAR_LCORE(lcore, lcore_states); + + if (lcore >= RTE_MAX_LCORE || !cs->is_service_core) return -EINVAL; /* Load thread_active using ACQUIRE to avoid instructions dependent on * the result being re-ordered before this load completes. */ - return rte_atomic_load_explicit(&lcore_states[lcore].thread_active, + return rte_atomic_load_explicit(&cs->thread_active, rte_memory_order_acquire); } @@ -547,9 +547,12 @@ int32_t rte_service_lcore_count(void) { int32_t count = 0; - uint32_t i; - for (i = 0; i < RTE_MAX_LCORE; i++) - count += lcore_states[i].is_service_core; + + unsigned int lcore_id; + struct core_state *cs; + RTE_LCORE_VAR_FOREACH(lcore_id, cs, lcore_states) + count += cs->is_service_core; + return count; } @@ -566,7 +569,8 @@ rte_service_lcore_list(uint32_t array[], uint32_t n) uint32_t i; uint32_t idx = 0; for (i = 0; i < RTE_MAX_LCORE; i++) { - struct core_state *cs = &lcore_states[i]; + struct core_state *cs = + RTE_LCORE_VAR_LCORE(i, lcore_states); if (cs->is_service_core) { array[idx] = i; idx++; @@ -582,7 +586,7 @@ rte_service_lcore_count_services(uint32_t lcore) if (lcore >= RTE_MAX_LCORE) return -EINVAL; - struct core_state *cs = &lcore_states[lcore]; + struct core_state *cs = RTE_LCORE_VAR_LCORE(lcore, lcore_states); if (!cs->is_service_core) return -ENOTSUP; @@ -634,28 +638,30 @@ rte_service_start_with_defaults(void) static int32_t service_update(uint32_t sid, uint32_t lcore, uint32_t *set, uint32_t *enabled) { + struct core_state *cs = RTE_LCORE_VAR_LCORE(lcore, lcore_states); + /* validate ID, or return error value */ if (!service_valid(sid) || lcore >= RTE_MAX_LCORE || - !lcore_states[lcore].is_service_core) + !cs->is_service_core) return -EINVAL; if (set) { - uint64_t lcore_mapped = rte_bitset_test(lcore_states[lcore].mapped_services, sid); + bool lcore_mapped = rte_bitset_test(cs->mapped_services, sid); if (*set && !lcore_mapped) { - rte_bitset_set(lcore_states[lcore].mapped_services, sid); + rte_bitset_set(cs->mapped_services, sid); rte_atomic_fetch_add_explicit(&rte_services[sid].num_mapped_cores, 1, rte_memory_order_relaxed); } if (!*set && lcore_mapped) { - rte_bitset_clear(lcore_states[lcore].mapped_services, sid); + rte_bitset_clear(cs->mapped_services, sid); rte_atomic_fetch_sub_explicit(&rte_services[sid].num_mapped_cores, 1, rte_memory_order_relaxed); } } if (enabled) - *enabled = rte_bitset_test(lcore_states[lcore].mapped_services, sid); + *enabled = rte_bitset_test(cs->mapped_services, sid); return 0; } @@ -683,13 +689,14 @@ set_lcore_state(uint32_t lcore, int32_t state) { /* mark core state in hugepage backed config */ struct rte_config *cfg = rte_eal_get_configuration(); + struct core_state *cs = RTE_LCORE_VAR_LCORE(lcore, lcore_states); cfg->lcore_role[lcore] = state; /* mark state in process local lcore_config */ lcore_config[lcore].core_role = state; /* update per-lcore optimized state tracking */ - lcore_states[lcore].is_service_core = (state == ROLE_SERVICE); + cs->is_service_core = (state == ROLE_SERVICE); rte_eal_trace_service_lcore_state_change(lcore, state); } @@ -700,14 +707,16 @@ rte_service_lcore_reset_all(void) /* loop over cores, reset all mapped services */ uint32_t i; for (i = 0; i < RTE_MAX_LCORE; i++) { - if (lcore_states[i].is_service_core) { - rte_bitset_clear_all(lcore_states[i].mapped_services, RTE_SERVICE_NUM_MAX); + struct core_state *cs = RTE_LCORE_VAR_LCORE(i, lcore_states); + + if (cs->is_service_core) { + rte_bitset_clear_all(cs->mapped_services, RTE_SERVICE_NUM_MAX); set_lcore_state(i, ROLE_RTE); /* runstate act as guard variable Use * store-release memory order here to synchronize * with load-acquire in runstate read functions. */ - rte_atomic_store_explicit(&lcore_states[i].runstate, + rte_atomic_store_explicit(&cs->runstate, RUNSTATE_STOPPED, rte_memory_order_release); } } @@ -723,17 +732,19 @@ rte_service_lcore_add(uint32_t lcore) { if (lcore >= RTE_MAX_LCORE) return -EINVAL; - if (lcore_states[lcore].is_service_core) + + struct core_state *cs = RTE_LCORE_VAR_LCORE(lcore, lcore_states); + if (cs->is_service_core) return -EALREADY; set_lcore_state(lcore, ROLE_SERVICE); /* ensure that after adding a core the mask and state are defaults */ - rte_bitset_clear_all(lcore_states[lcore].mapped_services, RTE_SERVICE_NUM_MAX); + rte_bitset_clear_all(cs->mapped_services, RTE_SERVICE_NUM_MAX); /* Use store-release memory order here to synchronize with * load-acquire in runstate read functions. */ - rte_atomic_store_explicit(&lcore_states[lcore].runstate, RUNSTATE_STOPPED, + rte_atomic_store_explicit(&cs->runstate, RUNSTATE_STOPPED, rte_memory_order_release); return rte_eal_wait_lcore(lcore); @@ -745,7 +756,7 @@ rte_service_lcore_del(uint32_t lcore) if (lcore >= RTE_MAX_LCORE) return -EINVAL; - struct core_state *cs = &lcore_states[lcore]; + struct core_state *cs = RTE_LCORE_VAR_LCORE(lcore, lcore_states); if (!cs->is_service_core) return -EINVAL; @@ -769,7 +780,7 @@ rte_service_lcore_start(uint32_t lcore) if (lcore >= RTE_MAX_LCORE) return -EINVAL; - struct core_state *cs = &lcore_states[lcore]; + struct core_state *cs = RTE_LCORE_VAR_LCORE(lcore, lcore_states); if (!cs->is_service_core) return -EINVAL; @@ -799,6 +810,8 @@ rte_service_lcore_start(uint32_t lcore) int32_t rte_service_lcore_stop(uint32_t lcore) { + struct core_state *cs = RTE_LCORE_VAR_LCORE(lcore, lcore_states); + if (lcore >= RTE_MAX_LCORE) return -EINVAL; @@ -806,12 +819,11 @@ rte_service_lcore_stop(uint32_t lcore) * memory order here to synchronize with store-release * in runstate update functions. */ - if (rte_atomic_load_explicit(&lcore_states[lcore].runstate, rte_memory_order_acquire) == + if (rte_atomic_load_explicit(&cs->runstate, rte_memory_order_acquire) == RUNSTATE_STOPPED) return -EALREADY; uint32_t i; - struct core_state *cs = &lcore_states[lcore]; for (i = 0; i < RTE_SERVICE_NUM_MAX; i++) { bool enabled = rte_bitset_test(cs->mapped_services, i); @@ -831,7 +843,7 @@ rte_service_lcore_stop(uint32_t lcore) /* Use store-release memory order here to synchronize with * load-acquire in runstate read functions. */ - rte_atomic_store_explicit(&lcore_states[lcore].runstate, RUNSTATE_STOPPED, + rte_atomic_store_explicit(&cs->runstate, RUNSTATE_STOPPED, rte_memory_order_release); rte_eal_trace_service_lcore_stop(lcore); @@ -842,7 +854,7 @@ rte_service_lcore_stop(uint32_t lcore) static uint64_t lcore_attr_get_loops(unsigned int lcore) { - struct core_state *cs = &lcore_states[lcore]; + struct core_state *cs = RTE_LCORE_VAR_LCORE(lcore, lcore_states); return rte_atomic_load_explicit(&cs->loops, rte_memory_order_relaxed); } @@ -850,7 +862,7 @@ lcore_attr_get_loops(unsigned int lcore) static uint64_t lcore_attr_get_cycles(unsigned int lcore) { - struct core_state *cs = &lcore_states[lcore]; + struct core_state *cs = RTE_LCORE_VAR_LCORE(lcore, lcore_states); return rte_atomic_load_explicit(&cs->cycles, rte_memory_order_relaxed); } @@ -858,7 +870,7 @@ lcore_attr_get_cycles(unsigned int lcore) static uint64_t lcore_attr_get_service_calls(uint32_t service_id, unsigned int lcore) { - struct core_state *cs = &lcore_states[lcore]; + struct core_state *cs = RTE_LCORE_VAR_LCORE(lcore, lcore_states); return rte_atomic_load_explicit(&cs->service_stats[service_id].calls, rte_memory_order_relaxed); @@ -885,7 +897,7 @@ lcore_attr_get_service_error_calls(uint32_t service_id, unsigned int lcore) static uint64_t lcore_attr_get_service_cycles(uint32_t service_id, unsigned int lcore) { - struct core_state *cs = &lcore_states[lcore]; + struct core_state *cs = RTE_LCORE_VAR_LCORE(lcore, lcore_states); return rte_atomic_load_explicit(&cs->service_stats[service_id].cycles, rte_memory_order_relaxed); @@ -901,7 +913,10 @@ attr_get(uint32_t id, lcore_attr_get_fun lcore_attr_get) uint64_t sum = 0; for (lcore = 0; lcore < RTE_MAX_LCORE; lcore++) { - if (lcore_states[lcore].is_service_core) + struct core_state *cs = + RTE_LCORE_VAR_LCORE(lcore, lcore_states); + + if (cs->is_service_core) sum += lcore_attr_get(id, lcore); } @@ -963,12 +978,11 @@ int32_t rte_service_lcore_attr_get(uint32_t lcore, uint32_t attr_id, uint64_t *attr_value) { - struct core_state *cs; + struct core_state *cs = RTE_LCORE_VAR_LCORE(lcore, lcore_states); if (lcore >= RTE_MAX_LCORE || !attr_value) return -EINVAL; - cs = &lcore_states[lcore]; if (!cs->is_service_core) return -ENOTSUP; @@ -993,7 +1007,8 @@ rte_service_attr_reset_all(uint32_t id) return -EINVAL; for (lcore = 0; lcore < RTE_MAX_LCORE; lcore++) { - struct core_state *cs = &lcore_states[lcore]; + struct core_state *cs = + RTE_LCORE_VAR_LCORE(lcore, lcore_states); cs->service_stats[id] = (struct service_stats) {}; } @@ -1004,12 +1019,11 @@ rte_service_attr_reset_all(uint32_t id) int32_t rte_service_lcore_attr_reset_all(uint32_t lcore) { - struct core_state *cs; + struct core_state *cs = RTE_LCORE_VAR_LCORE(lcore, lcore_states); if (lcore >= RTE_MAX_LCORE) return -EINVAL; - cs = &lcore_states[lcore]; if (!cs->is_service_core) return -ENOTSUP; @@ -1044,7 +1058,7 @@ static void service_dump_calls_per_lcore(FILE *f, uint32_t lcore) { uint32_t i; - struct core_state *cs = &lcore_states[lcore]; + struct core_state *cs = RTE_LCORE_VAR_LCORE(lcore, lcore_states); fprintf(f, "%02d\t", lcore); for (i = 0; i < RTE_SERVICE_NUM_MAX; i++) { -- 2.43.0