From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5CF9F45A2C; Wed, 25 Sep 2024 21:34:09 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7C0A84025D; Wed, 25 Sep 2024 21:34:08 +0200 (CEST) Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.12]) by mails.dpdk.org (Postfix) with ESMTP id A98FA400EF for ; Wed, 25 Sep 2024 21:34:06 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1727292847; x=1758828847; h=from:to:cc:subject:date:message-id: content-transfer-encoding:mime-version; bh=Iw2uIMBGZLQs5a8ig2g08v1dQxpeYf0yoFdvl/Q0VyE=; b=f9DLTO8mYc73WJA/jANakvqDaYJp+VbK/VcsSKxjEOqmS9i6T9zrfBc4 wkcBtgMIJprjW/MH/bOD2tx6wue78B4FaD8XltLrqu5DrnfTRRzetO9IT qTGsMsxVUSS8V7UuAn5ibpBFPGkqGx1/2BhMmIMMczRSDXuEPox12E5Lm MFs3cU3dELTrkQiJp60jzeV9whxsBPZCNTDieux+OskBmyjZq79Bafb9k YQgfJCMU6myOhXbDAYAr6IBTe8xP/adkMgeCZSr0n2y3/qtWLzg7pogUa gFHBXAcBrwP2JGfGJB8nD9M8vHEmj5zCyehusmoPjVIN02IxEscz0gzkO w==; X-CSE-ConnectionGUID: 5m4I14myRwCxHhUXVc89ZQ== X-CSE-MsgGUID: wl7PTmWoTWO7Nqjv2rjVgw== X-IronPort-AV: E=McAfee;i="6700,10204,11206"; a="30249569" X-IronPort-AV: E=Sophos;i="6.10,258,1719903600"; d="scan'208";a="30249569" Received: from orviesa006.jf.intel.com ([10.64.159.146]) by fmvoesa106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Sep 2024 12:34:05 -0700 X-CSE-ConnectionGUID: cHjkRud4T32FgryP+8AzQg== X-CSE-MsgGUID: zNhnkZEsRnCmKXB0aitBQw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,258,1719903600"; d="scan'208";a="72188573" Received: from fmsmsx602.amr.corp.intel.com ([10.18.126.82]) by orviesa006.jf.intel.com with ESMTP/TLS/AES256-GCM-SHA384; 25 Sep 2024 12:34:04 -0700 Received: from fmsmsx602.amr.corp.intel.com (10.18.126.82) by fmsmsx602.amr.corp.intel.com (10.18.126.82) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Wed, 25 Sep 2024 12:34:04 -0700 Received: from FMSEDG603.ED.cps.intel.com (10.1.192.133) by fmsmsx602.amr.corp.intel.com (10.18.126.82) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39 via Frontend Transport; Wed, 25 Sep 2024 12:34:04 -0700 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (104.47.56.170) by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.39; Wed, 25 Sep 2024 12:34:03 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=DeLXnbiMRWxPHpKTMIr7W177klt2x+reuJ3lg0456J/5WOx/045yfD9SJseEZqBCKEYs8eix3lffvEzy3YYD/Tf1zMI0De0SMKgzHbpDryqJPXJBT7fbELr5OgD3FJJNHK10MAZcoH2w4+GWJ1uxVkY6i7K0NZrp1IaK6NKOH+6KSrnCgQmZCAgAwXN0DgyuCUFR1RQS+LtX0McZcPu6+qIUxPv+/DHpL6qXlKi6GHgIDwkhb9jdKuFs1RqMPZ1y1Qj68yiCMumjme+FQfYX1lnUvNiN3KCg8Wl/oUgEE5iMX8QoKyG3Yy6NWmKxCiknw6cHYq1q+gKWUp2f/a0/rQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=lOl+Y6kEadMcoHPz2lohr0K+pu+8wBjeGLu1XkiW5N8=; b=YM8K2UpBzGmG5afrUIYFzuN0BhFOVOfU0A3Xac0T2+WaFlp1DshAof88vZ8ttIxMqEa0IR44dcpwDFxIF4sDt8lfRt4pO2ig8TTRU+78x9UDqx7TKB0Cqn6+uN8aTf8+G7WmLeyZ4a69hzzvr5CnU7D78K72B3gDPrGYryu8vYsUkSymwkEMTytliImx8ild2tN47nXL2zhJzs9FFJjN+D1VTgjFYW/f/EbrIkZnfXXfuIZYDajsj/3mhxcx9qbPYj7MgKdlRl6TS5KrGlnmjlneRNROSi96h5/GH7G2X2iM6wHq19849V7OJZMoJm8wLklyw3eW1OQagZrLQKEMCw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from MW3PR11MB4521.namprd11.prod.outlook.com (2603:10b6:303:55::13) by MN2PR11MB4677.namprd11.prod.outlook.com (2603:10b6:208:24e::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7982.26; Wed, 25 Sep 2024 19:34:01 +0000 Received: from MW3PR11MB4521.namprd11.prod.outlook.com ([fe80::c706:b86b:2b3:10da]) by MW3PR11MB4521.namprd11.prod.outlook.com ([fe80::c706:b86b:2b3:10da%3]) with mapi id 15.20.7982.022; Wed, 25 Sep 2024 19:34:01 +0000 From: Wojciech Panfil To: CC: , , , , , , Subject: [PATCH] eal/alarm_cancel: Fix thread starvation Date: Wed, 25 Sep 2024 21:42:06 +0200 Message-ID: <20240925194206.106825-1-wojciech.panfil@intel.com> X-Mailer: git-send-email 2.46.0 Organization: Intel Corporation Content-Transfer-Encoding: 8bit Content-Type: text/plain X-ClientProxiedBy: MI1P293CA0003.ITAP293.PROD.OUTLOOK.COM (2603:10a6:290:2::19) To MW3PR11MB4521.namprd11.prod.outlook.com (2603:10b6:303:55::13) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MW3PR11MB4521:EE_|MN2PR11MB4677:EE_ X-MS-Office365-Filtering-Correlation-Id: 6b26f886-23ac-49ad-c5ab-08dcdd9901ec X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?Ciq425xyVqI6ZmhxgAsUYnsZPJHr4Vlp+x33NvILCx7WTZGwLlbqfF3Z4oJm?= =?us-ascii?Q?PzxriiQERS88KTzsNzeGtUJmhCDVrbRtLmR4j4fFxx3Pb01avLWscvCZWYzJ?= =?us-ascii?Q?jKcq7hbsFlBFkuz8H52vBW7EhVFpO5DV7341UTUk/UkT7mS3Nl9CEcpAblYp?= =?us-ascii?Q?PHF9u3/Ueh5gfGxbMhQ3zXiWxNJd5ybxR2f7Lx1/hAunyiw20fQX/dBgMBrv?= =?us-ascii?Q?Yb/Mp4BDA6camx12algeSAJEV2ptVrh9VxGPNRlxLg6sjVmvAWtuQ7RiPiNl?= =?us-ascii?Q?bpvbC4JXcLY2oTnN5r26yGP7JYRElS/3a/tpr5zO+cnjtp5UZygKUFG3NKR7?= =?us-ascii?Q?i/SHuz9c+X85/cZyqIvMA+sdlwLlCC34Joo1iSmI5e27M5o6bvUqANEe0xsc?= =?us-ascii?Q?t6UuPWuWDbt3hsBdhQmP/q8D1cL7bn5mWZpMbBYvOKpoQjimRmHux03MQI7A?= =?us-ascii?Q?Nj9O2ESueLsd0h7QHh/7irkOrMKU70SaE9WAAlnzxbwXGDj88VyE961F2VO2?= =?us-ascii?Q?F+OTNx7W/47DIlUv4eBf2po2XFDJ2oU6odSiuVYzDKMIGc79z2WbpZSzq9XV?= =?us-ascii?Q?PS6Khl8xzOdfdJkaPNh3iuLu+6sKhtp2yLWgqx8UHL/U7yu7hGW9vd0hbNTz?= =?us-ascii?Q?ZZNLMvfmUndfc4hOYiWcpZz6/+7MYp3ZuCdeBLkql1wpeZbJMZxgbKI8hq4k?= =?us-ascii?Q?GR08BBbUKBjY6Cp8lBONzkV1PYQMUX9I9FTmYPjnWgWG5yaphkjlJjA0jrPW?= =?us-ascii?Q?UoC+LtujSw5XLX9SF/1YWP7JFHlgqVGrEm3Fe8CCBUdD2MXnBrdckihRpIAS?= =?us-ascii?Q?eIzrSIuEYAR9k7SZ4YAbPhFLWbDvVk9/PkeY2zU9O42bWHbYlMEly2z6zGQH?= =?us-ascii?Q?O6wZBCjrKfhwuXp/S3rJPwOR9ZQiJR55dTEmEf9JQxlsJV9lBX1kMQJGO9BP?= =?us-ascii?Q?0DHDki2cxQBDpjyzzADmPvHaRbrjMOi4/mL+WmjzPyM4H+/VOVAcSY8kA84P?= =?us-ascii?Q?1ioZRO0thiQmXTo+pc5uUyiQNCd0Hx9Mdmej6gS5qq17E5V+B7DqHvf7rP7X?= =?us-ascii?Q?9i8aHassPU3KX1dYoh+g0Mn83/aH0JP9zcyNZrI5ION0LAe/RJqeKEDtBz8N?= =?us-ascii?Q?MjMuVqEMsUJJ6sd4fhDx8cckrft5nHk7pHprgzaSkZbidw1JGmoTJuI754dn?= =?us-ascii?Q?w/w9LIh68A7p0Fqee47/OFAfHT1FGxhe5gv9eRDt4dp6vnbIWSzBmFmX8Ib8?= =?us-ascii?Q?AuAO3hmxOcTxANERSbgFPj6pZZWEki5sXbHinVgRtg=3D=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MW3PR11MB4521.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(376014)(366016); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?Su2TBnERx0cUEwiVMqmiXPMdRmqs1+6ilBhUO3UO9T+OnLOCZWg0Dy/eEbsj?= =?us-ascii?Q?ycfb24QGhYQApJE7gaqTLbBAtRlin8SVCa3eHO6gwidlXBYtj59+dfJ4nKys?= =?us-ascii?Q?3dYoX2mqjzz+sButyjX64ge2HJt1XD8BNBuNyh0r1B+Oz4LkgHF6k9SlMFSe?= =?us-ascii?Q?cWGoV+jcm6ozjdYN7VR+XJeJ4/yH9x0n1PTYCIOhjhLtZa4lJnz5UQDM7xhz?= =?us-ascii?Q?otUvDxlaE72ddmfOHiNNAZ6QgJqM0E+0uo5ckkTXgDUAdWL52aSf+V8ROHni?= =?us-ascii?Q?Sm6QmpMn9pkQViuF0KrYBcsRClhlxydBmyVLJfGPzeBIVGLCJat+LHXYpbxu?= =?us-ascii?Q?Mtw1A7Qx0YkYyU+souaPGtf+FhLHqtSJ/8BFVso261f9p/F6WkKPOxrtDy2E?= =?us-ascii?Q?hhZU8wTnkiuRparBbnfT99rGh/L8ZXdchu0c8/ylA1lSJmDTx3f2TOa6jadp?= =?us-ascii?Q?8AFnj8aiXwFCnua236Pd8VVvFyk6SYiHII+uOXgz/j84HU9xCLO9PxUfQT8S?= =?us-ascii?Q?GPG2DWR/qKqdOJPfRoioQkiiNE0j+7yhlVxoluNqG43hJtxzoSUgDh8c2RXJ?= =?us-ascii?Q?bTJWQ5vpo4Gj63LE65y38OTNO0KYT/pp/exYMczUHOBa5uE8GNAlvC908PDe?= =?us-ascii?Q?A7kOPKiJWbub9THyxpOJZPvavVexZ1wIWbdgyxhVTmoB2tcD2uFzTNNbi6Hx?= =?us-ascii?Q?Qz+JcvkpX/99DAi1+QgZYwGosptL0gicos4/ijKjQ70cGELqXsuPIwAjbBpt?= =?us-ascii?Q?StghaDRgKiL7x8x5iJU01cyCLYZ+npKOevxrftYQH32mVeLXnEYS3qKaxaTC?= =?us-ascii?Q?mwPQMbB1HtyrB5JPj0G5iE3JAXbeqA9CQkebHuxZlvQf5K52TldSsgRe+3CP?= =?us-ascii?Q?uySqF2SC1j+/ZjzwHAR68n8x7TRQGF5br2IMKwkFDUXfKMW1FpRAaShvv/m6?= =?us-ascii?Q?Tuu/gNAiLGuqssUq0pZdiFp4+DuhZaxnXrPTPnzuh+pioWcMoxD7WMFr7tiy?= =?us-ascii?Q?MFubVGq/8/9Gt2P+NZTKDFtY9EDsHUPZ7XAORvZH6cvGrjxgjfCYUjzfsP84?= =?us-ascii?Q?v2DFpmcvLj3GchzvYCPflpbttMmX6k9ozz1iRjjHyRKeExRHRzC2uCWZJdKX?= =?us-ascii?Q?iRTK+Xnu8yYYm9Ir6yLPfWw9vTMcazwcf1JaT6NCaY5j3joGa9yxS/lHfQGi?= =?us-ascii?Q?zIHyeuTfpHyCsukgYuEcl1OpYw0ZA6LWQnIfSXVh+A074EpYf/Z9nB3FEN7a?= =?us-ascii?Q?EEo8lEZsgLVi+m6cJZ1FJ1dm3ULxHpdwXjzLFNYeqxP8OLlumMY87tTizG7U?= =?us-ascii?Q?uPTEbJ7L8xZVpb/FJP1bgEhXmagdEsjctzmFVl4FV7l8NBX4DfSlabucgfUV?= =?us-ascii?Q?xIldHs4zzU8L90czok7eEYnLRMqn19/rMQLdUD+C+K3dWkHl0Ckx5kuhQDRi?= =?us-ascii?Q?XcGxfdkxlzZ5bHa5Wf3YvF+9JjZYzPT6vnQlnDVpjwo4POIGzXb3dvsPDKvK?= =?us-ascii?Q?wL9MfoYr/Jm4BxDynSziKfsEngyWdsu60+wLp0+XrQt/za4NAVghIBNSxI2L?= =?us-ascii?Q?WE3LRZcpRWRkQe/VcHuCMggWDe/DYpTLhzBB6j3CVYLmFjSSoxAN+hLuQR7E?= =?us-ascii?Q?kQ=3D=3D?= X-MS-Exchange-CrossTenant-Network-Message-Id: 6b26f886-23ac-49ad-c5ab-08dcdd9901ec X-MS-Exchange-CrossTenant-AuthSource: MW3PR11MB4521.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Sep 2024 19:34:01.3080 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 6v417Desvldh0vMOda2AIRMBvTvMMwRo8uXkI+QDWx/TmQBDBtlMrbVjv96WwK+rF+wbx1rYe+9P9He3hK+FmuC+l6kcpyNJBTxVdk+Evns= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR11MB4677 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Issue: Two threads: - A, executing rte_eal_alarm_cancel, - B, executing eal_alarm_callback. Such case can cause starvation of thread B. Please see that there is a small time window between lock and unlock in thread A, so thread B must be switched to within a very small time window, so that it can obtain the lock. Solution to this problem is use sched_yield(), which puts current thread (A) at the end of thread execution priority queue and allows thread B to execute. The issue can be observed e.g. on hot-pluggable device detach path. On such path, rte_alarm can used to check if DPDK has completed the detachment. Waiting for completion, rte_eal_alarm_cancel is called, while another thread periodically calls eal_alarm_callback causing the issue to occur. Signed-off-by: Wojciech Panfil --- lib/eal/freebsd/eal_alarm.c | 6 ++++++ lib/eal/linux/eal_alarm.c | 6 ++++++ lib/eal/windows/eal_alarm.c | 5 +++++ 3 files changed, 17 insertions(+) diff --git a/lib/eal/freebsd/eal_alarm.c b/lib/eal/freebsd/eal_alarm.c index 94cae5f4b6..3680f5caba 100644 --- a/lib/eal/freebsd/eal_alarm.c +++ b/lib/eal/freebsd/eal_alarm.c @@ -318,7 +318,13 @@ rte_eal_alarm_cancel(rte_eal_alarm_callback cb_fn, void *cb_arg) } ap_prev = ap; } + rte_spinlock_unlock(&alarm_list_lk); + + /* Yield control to a second thread executing eal_alarm_callback to avoid + * its starvation, as it is waiting for the lock we have just released. + */ + sched_yield(); } while (executing != 0); if (count == 0 && err == 0) diff --git a/lib/eal/linux/eal_alarm.c b/lib/eal/linux/eal_alarm.c index eeb096213b..9fe14ade63 100644 --- a/lib/eal/linux/eal_alarm.c +++ b/lib/eal/linux/eal_alarm.c @@ -248,7 +248,13 @@ rte_eal_alarm_cancel(rte_eal_alarm_callback cb_fn, void *cb_arg) } ap_prev = ap; } + rte_spinlock_unlock(&alarm_list_lk); + + /* Yield control to a second thread executing eal_alarm_callback to avoid + * its starvation, as it is waiting for the lock we have just released. + */ + sched_yield(); } while (executing != 0); if (count == 0 && err == 0) diff --git a/lib/eal/windows/eal_alarm.c b/lib/eal/windows/eal_alarm.c index 052af4b21b..9ad530dd31 100644 --- a/lib/eal/windows/eal_alarm.c +++ b/lib/eal/windows/eal_alarm.c @@ -211,6 +211,11 @@ rte_eal_alarm_cancel(rte_eal_alarm_callback cb_fn, void *cb_arg) } rte_spinlock_unlock(&alarm_lock); + + /* Yield control to a second thread executing eal_alarm_callback to avoid + * its starvation, as it is waiting for the lock we have just released. + */ + SwitchToThread(); } while (executing); rte_eal_trace_alarm_cancel(cb_fn, cb_arg, removed); -- 2.46.0