From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E731945A7E; Tue, 1 Oct 2024 18:35:52 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9B5D140299; Tue, 1 Oct 2024 18:35:52 +0200 (CEST) Received: from EUR02-DB5-obe.outbound.protection.outlook.com (mail-db5eur02on2080.outbound.protection.outlook.com [40.107.249.80]) by mails.dpdk.org (Postfix) with ESMTP id D22D740273; Tue, 1 Oct 2024 18:35:50 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=WpsSeMi5JQmPonF26u5R9PqrRHCwrWzrAJsCcEa9Xd90yt2w2nJesaY45+LEkdv1oM55MxlIuCXJGa+0T9pGzt64ZsR5XrbItxPI02XLzCHSepX7OorZ1c7OE/FYVcskQTclcXWgeqkhiadEVmgL+lANqMaFMOenB1eplzL0e44RoZZvS9oXCFMAkecw4sCB0zTfCTmnmbkLHLttHrg7Co34fPSuCwmmmqa6CMOGa74LIzl5wdYwvGrx2/W5tG/0dWqiycDPs1k9KuXqW3MYZmyvDWGmtnH1J5QXBcY7m6ZmqdUMOZW63iXAloXPbufdKZaZDJe4z7ZrxrF18e0oMw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=niZXTGS6L0EyGe/DU5WtpHT4uUh1OdsQSzJRUvb+bos=; b=Gun/mvX57tvO7BzHhbtHypcrFoE/MWXHYHjQ0kfJM7n3VDtRUiYLNYDR8gWjPQRQ1Qg22S0caN53uTfOz2e2zvzw1Z8GB61SmtoIMmO/bLBKlNSR64EmQExCwUIlC6ASbJpDJAmScWT59s0xUeaE3CuEUi/HCJTrNUJd+C6OWCURxOhUV0Ut089QYBZhhhMa0TRjKqIxqhWKZ/amu6BasF2gofqWSc6ewrxh7iciSrRewYruzkpWlU+EOk+erCN002w7Pb4f7VPqJ9bb9Fr2/wF8Q45VV8DfvI2XdDEWFBdTmc6zltRSz7ONySo+TCQWf4rJ9SFXXTCJem5VWE3O8A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=dpdk.org smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=niZXTGS6L0EyGe/DU5WtpHT4uUh1OdsQSzJRUvb+bos=; b=QrK7y4aFsD1KABjivFsCsw5eX18yZQ2kR2LTK+1DHnnmGYege+iyvCw4KN4MgCZV4Ow77bg9quTJ20Cd+o0h05StFhceiZxitIa9Ypm+OT7YPKpNBNsr30NqEttfbLEeMrhe2iZbBKxF/bjzguaSN/v4fnZcJhO9t2GCWdRDAnlD9RMbKz6xAJAIFfDrOmhqNnCjH028GS/y4jv02a6LrOFK+571CCj6w8l15Uz6UZfZx4z5kLjkShmeQjiH7GTJ/hy7sq2S3rYI7DXHr5RNGzj45X18Hcjl1J82cjzMDSVIT2Xg92BzZlZK7ZpW1agLeEqYSS52jlzAm6FzKDvN3g== Received: from DUZP191CA0067.EURP191.PROD.OUTLOOK.COM (2603:10a6:10:4fa::26) by PA4PR07MB7423.eurprd07.prod.outlook.com (2603:10a6:102:cb::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8005.27; Tue, 1 Oct 2024 16:35:48 +0000 Received: from DB1PEPF0003922D.eurprd03.prod.outlook.com (2603:10a6:10:4fa:cafe::8f) by DUZP191CA0067.outlook.office365.com (2603:10a6:10:4fa::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8026.15 via Frontend Transport; Tue, 1 Oct 2024 16:35:48 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by DB1PEPF0003922D.mail.protection.outlook.com (10.167.8.100) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8026.11 via Frontend Transport; Tue, 1 Oct 2024 16:35:46 +0000 Received: from seliicinfr00049.seli.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.64) with Microsoft SMTP Server id 15.2.1544.11; Tue, 1 Oct 2024 18:35:15 +0200 Received: from breslau.. (seliicwb00002.seli.gic.ericsson.se [10.156.25.100]) by seliicinfr00049.seli.gic.ericsson.se (Postfix) with ESMTP id 212D0380070; Tue, 1 Oct 2024 18:35:15 +0200 (CEST) From: =?UTF-8?q?Mattias=20R=C3=B6nnblom?= To: Harry van Haaren , Stephen Hemminger CC: , , Suanming Mou , , , =?UTF-8?q?Mattias=20R=C3=B6nnblom?= , Subject: [PATCH] service: avoid worker lcore exit deadlock Date: Tue, 1 Oct 2024 18:26:03 +0200 Message-ID: <20241001162603.793853-1-mattias.ronnblom@ericsson.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 8bit X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DB1PEPF0003922D:EE_|PA4PR07MB7423:EE_ X-MS-Office365-Filtering-Correlation-Id: e46011ff-aed4-46aa-6870-08dce23719eb X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|82310400026|376014|1800799024|36860700013; X-Microsoft-Antispam-Message-Info: =?utf-8?B?di9GazdXUmZMbDVpUzFZcFB3d3BIemw1UTROaDNWcXhOeEMrbWQyZENzd0FS?= =?utf-8?B?SDZQQjlMYnBNUU9FVVpqWnFJZnlVaW5LVCtTOW9McW4wRDJwd0wyK1dPdXg0?= =?utf-8?B?UVJBTzFEeVRHMFRTT3E1eVRzWTJXR2s0NGU4UzRrY0ZxRCtQL1pjVVBDeUhp?= =?utf-8?B?OWNIRkd4elVlSVROYnNlNnpvUmI2clozUVNGNXJUbFJmaTFncGxud3V6eit0?= =?utf-8?B?bnlIeVREb1luMUpMSTBSKzV5M1FiZmJ4VWs0K2ZOTlVkcDVzVWFWbU51OTUr?= =?utf-8?B?N0d0ZEtwMkw5QXowSXMzb3dMK1VwcFdJenBYbHVJYkpRWVhjSXdBQzUxQTkx?= =?utf-8?B?cnZ6WGJ4cnNMTzBPTWlJWEpvWkY3ZHhLTXJlUk1JM1NOZVZGNkxXVk81RjVo?= =?utf-8?B?M0V2Q0JzMXBJSjR0NEFtZElRVFFra3FVWjQ2ODEzVWI0YXpqbzlwNmowVnd0?= =?utf-8?B?OGZUTWNaci9RV1hUK0ZxWEVUalZxUTZFSzJzWlMxL2tqZnZKd0ZYMFZUeHNt?= =?utf-8?B?am42UGRRTHhWWXlubWpSVERiLzJlSXE2S1VlTDlDb0JUeDJ2WWU4ejJpSEky?= =?utf-8?B?L0U1ajdPdnFsa0J5aWxNNWlQK1NmdkowV1hCTEQyUkU0bkdWaFNsZCtyUEh2?= =?utf-8?B?L09QYlFMVjJWTHBZNEJzcmRrMXRMZG01SngxZ01kbXROMzkwYzk0OS80Skgr?= =?utf-8?B?ZW9oSi9hRmxnS0d0VlZaN2ZTZ0wyQ3hVNGNZRjc5aklxTE8xVFNTOGdzMzlR?= =?utf-8?B?K2pHbWNNbkJoZVYza1ZBQXMwblgyT29PS2REbnBvbUNRdkJEOXhSVWdBcW8x?= =?utf-8?B?WWlCbUdqaGNIZE93UjgyM2tKc2FaQU5Pc3BnNkZoT0JiaEVReEdwcVMvVnNM?= =?utf-8?B?RU5MN2xMNW0vZEJIa00rRXhHeDBSbUpva0w5clJjZGdETitDbVcrYnZBNERN?= =?utf-8?B?QVVYdWY5OVJPcEZUdEtmdFp2b2VSUm5WTnUvWVNBNDg1YTlKSHBBS1ZTdVFD?= =?utf-8?B?UUFhTW5sWU1sZUpMQXdmOFBDQ3B3TFNYYVQ4eGY5MHBNM0pqa1A5dnV6c2Q0?= =?utf-8?B?MDRnK3FxVnAwcWs2SXdXK1phV2k0MW5QOWlJWlFGOXFvTHc5Q09SSUhWbGxK?= =?utf-8?B?a2FXRm10azNoZ1R0cENOWm44U2RZM3pGYXZtc0E4UlorVGEwbmdPU0FvMVFl?= =?utf-8?B?OTVabTlwN1Rsb2RUUzVlekRiVGcvMWc5cGJBUk4vMzlYSzNkaWwzU05vdW9B?= =?utf-8?B?Y2JQRlU5NWV4ak9QS0xUdWI5M1l3Z1dLQkVBT0ZSMXNaUXRhQ2VSb1BTNnM0?= =?utf-8?B?WXJBdHFNM2dhWlVydXgzVHFsa0ZrRmZsaVU0UllLNFR4RzZSdXpvZGpEM042?= =?utf-8?B?VjFDbW4vN2MyQlZnTmp2QlQ4a2pKMmpFZWJOc3F0RXBUMlcwSmlBOEhUT0Z4?= =?utf-8?B?OUNhS2JkRFhkMzR1RUlvdTBmdU5haFZBOC93WEszK0IrWGxaRXQya0ZrRkVJ?= =?utf-8?B?bVZBeWJQU3dXTU5VWTRHNm1qR3JoNlVZWDkxa0d4MGdlajZDQmI5OFBscExi?= =?utf-8?B?NkJRdytwM1JuVGpZQ1p1ZWM1cm8wMzZVZjlHY2dIYmZHMjFEaGp3aitZekoz?= =?utf-8?B?NDZDelZMNVd6ZCt6TTR5cGZBSGovYXV4aDJURnB2VGdiMklCZHUyb1UrTmpp?= =?utf-8?B?bm9uc08rUEVRbWxZSVRVZlhDZ1lEbGc1WDdoVGYwQ0RsV0tVek9CbEZVSFF4?= =?utf-8?B?clJvOTc1N3lJVjlmdDA3bmVjWkN4MXlzbkx3RlltVDI3Q3RJMms3WFRFL3ky?= =?utf-8?B?cEFUc2tvYmZqNXF6RjVMeXhmdUFvdXpYYkFWQThZMFNuK2pLSmxmUmdYYUtZ?= =?utf-8?Q?P2Iq1XCUh8nTs?= X-Forefront-Antispam-Report: CIP:192.176.1.74; CTRY:SE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:oa.msg.ericsson.com; PTR:office365.se.ericsson.net; CAT:NONE; SFS:(13230040)(82310400026)(376014)(1800799024)(36860700013); DIR:OUT; SFP:1101; X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Oct 2024 16:35:46.4923 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e46011ff-aed4-46aa-6870-08dce23719eb X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f; Ip=[192.176.1.74]; Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: DB1PEPF0003922D.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR07MB7423 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: 20230629203720.682f90c3@hermes.local Errors-To: dev-bounces@dpdk.org Calling rte_exit() from a worker lcore thread causes a deadlock in rte_service_finalize(). This patch makes rte_service_finalize() deadlock-free by avoiding the need to synchronize with service lcore threads, which in turn is achieved by moving service and per-lcore state from the heap to being statically allocated. The BSS segment increases with ~156 kB (on x86_64 with default RTE_MAX_LCORE and RTE_SERVICE_NUM_MAX). According to the service perf autotest, this change also results in a slight reduction of service framework overhead. Fixes: 33666b448f15 ("service: fix crash on exit") Cc: harry.van.haaren@intel.com Cc: stable@dpdk.org Signed-off-by: Mattias Rönnblom --- lib/eal/common/rte_service.c | 28 ++-------------------------- 1 file changed, 2 insertions(+), 26 deletions(-) diff --git a/lib/eal/common/rte_service.c b/lib/eal/common/rte_service.c index 94e872a08a..2ea0c0c0d9 100644 --- a/lib/eal/common/rte_service.c +++ b/lib/eal/common/rte_service.c @@ -15,7 +15,6 @@ #include #include #include -#include #include #include @@ -74,8 +73,8 @@ struct core_state { } __rte_cache_aligned; static uint32_t rte_service_count; -static struct rte_service_spec_impl *rte_services; -static struct core_state *lcore_states; +static struct rte_service_spec_impl rte_services[RTE_SERVICE_NUM_MAX]; +static struct core_state lcore_states[RTE_MAX_LCORE]; static uint32_t rte_service_library_initialized; int32_t @@ -93,21 +92,6 @@ rte_service_init(void) return -EALREADY; } - rte_services = rte_calloc("rte_services", RTE_SERVICE_NUM_MAX, - sizeof(struct rte_service_spec_impl), - RTE_CACHE_LINE_SIZE); - if (!rte_services) { - RTE_LOG(ERR, EAL, "error allocating rte services array\n"); - goto fail_mem; - } - - lcore_states = rte_calloc("rte_service_core_states", RTE_MAX_LCORE, - sizeof(struct core_state), RTE_CACHE_LINE_SIZE); - if (!lcore_states) { - RTE_LOG(ERR, EAL, "error allocating core states array\n"); - goto fail_mem; - } - int i; struct rte_config *cfg = rte_eal_get_configuration(); for (i = 0; i < RTE_MAX_LCORE; i++) { @@ -120,10 +104,6 @@ rte_service_init(void) rte_service_library_initialized = 1; return 0; -fail_mem: - rte_free(rte_services); - rte_free(lcore_states); - return -ENOMEM; } void @@ -133,10 +113,6 @@ rte_service_finalize(void) return; rte_service_lcore_reset_all(); - rte_eal_mp_wait_lcore(); - - rte_free(rte_services); - rte_free(lcore_states); rte_service_library_initialized = 0; } -- 2.34.1