From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A22AF428F8 for ; Sun, 9 Apr 2023 17:34:31 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9DECA42BFE; Sun, 9 Apr 2023 17:34:31 +0200 (CEST) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2066.outbound.protection.outlook.com [40.107.93.66]) by mails.dpdk.org (Postfix) with ESMTP id B7D0241149 for ; Sun, 9 Apr 2023 17:34:29 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=UQk1VLcEJejyRqUyAJqS4sVvuyprIVKYk7mpm68wm+se4/e5cUOzZccis3wOrK2qpSxqSdGNgGbPGEErEIaT1o/LyLjNrqfoKPkVRj+fqQcpMRRSh6goT/BGRJhUmkjfHqqRB3La0iim0SaBMYAxmt22WZSsTiX5FgEbKfmuwuosnuNr5dWpyffjgnQnDJaxlCx6YIB0eX+YMOv5SPqkTg2Lfsnl+QYMD5YTKzPKd2BaN0IqymeEtmqNDuut0Q6pelN+1IlJFM7E/j21GvALMyHAAXk3gvlVqBtogoVgNkc7mcFMqb5iSWOb+kguGUd/ppQpPcEIIBeVubqyLn07bQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=cNatLXsrSM+Vp5NLptZTdgymaAnMEsOfCwfSF6ZlLw8=; b=bsdeck0UNhNPYp+frRkI9G9A3InEy5KQDEHlgI14s74w69rMTKQp/V2KBlyoC/aD5IqR1pJw1IS/c9PXUsYSSrgN2KrsbzJ+CeDPwMv+6gIWvFXF6F5uSQG1N7DjbR4Y679kNjCdN6qy+aXMTQvmonLToLVTChq3GnJUcNJcXqrGQfAJ5WE1SXd/bganEMnMPZAHake0zLVwo3qyTv1YZmy5iz3Pz7NFBoT7MGdoKZMhOiCCk1yhKGWfIljcu6c7jMRg37GqPF45CDhCJFrYXHgmNt0tAMo+VQI2Simc80YQFsfCruZqKhZyyukz2g3Q6C5y+7mkAjmQecGZFCKRhQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=linux.microsoft.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cNatLXsrSM+Vp5NLptZTdgymaAnMEsOfCwfSF6ZlLw8=; b=gpm7gNflriovaIFKbZxKxcrbVmdf2FD0++as2epjKrCUlgO7Gf4qZdZEqfdCbgx0XEv4KUyY2lFxG07ooOxqaYqNT21tjx6inkhMbbO56xfazJXgbC+7/tlnqpm6IR1YiYX4SjNcnPoZCGlkbCzFRIAqZDJqPHTf/s37A07t4jVTFWAtmGqf6F2GzkGvgQe4Qrs0iouYytWo22DqNWho96Ae0auEY+HGjpOoFhSc8RGaZbhvXf3RAN9O2blACmoajX5ErCvKz5QD4g0qdrjgFHYSU4sa7Fbrojh4P9OpO5JvpP+NZRwU7EhGEAo4LAzKs794mU3Zd6a8nP2cE4YZuQ== Received: from BN9PR03CA0705.namprd03.prod.outlook.com (2603:10b6:408:ef::20) by IA0PR12MB8225.namprd12.prod.outlook.com (2603:10b6:208:408::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6277.38; Sun, 9 Apr 2023 15:34:27 +0000 Received: from BN8NAM11FT093.eop-nam11.prod.protection.outlook.com (2603:10b6:408:ef:cafe::c2) by BN9PR03CA0705.outlook.office365.com (2603:10b6:408:ef::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6277.38 via Frontend Transport; Sun, 9 Apr 2023 15:34:27 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by BN8NAM11FT093.mail.protection.outlook.com (10.13.177.22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6298.20 via Frontend Transport; Sun, 9 Apr 2023 15:34:27 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.5; Sun, 9 Apr 2023 08:34:16 -0700 Received: from nvidia.com (10.126.230.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.37; Sun, 9 Apr 2023 08:34:15 -0700 From: Xueming Li To: Tyler Retzlaff CC: David Marchand , dpdk stable Subject: patch 'eal/unix: fix thread creation' has been queued to stable release 22.11.2 Date: Sun, 9 Apr 2023 23:24:48 +0800 Message-ID: <20230409152529.5308-101-xuemingl@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230409152529.5308-1-xuemingl@nvidia.com> References: <20230227062349.13764-1-xuemingl@nvidia.com> <20230409152529.5308-1-xuemingl@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [10.126.230.37] X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT093:EE_|IA0PR12MB8225:EE_ X-MS-Office365-Filtering-Correlation-Id: 9b548de8-a2eb-41b5-ddca-08db390fe7c1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: kGvB7S5xQz8Ltx7NfDuI8Xv+QNfcn+3M0wEZcLKnhnWaDVAhX3A/+4Pzk3/3qPyUT99UJ1kP0XldBnKoMpnJC+HqMIOOAbK2pLgHmS03M7zxri9L70urqMPzcZd9+i3hTmAHZ4vdG0SBhMbmqDlerKHBlInrhtGv3iVxk7mQumD0uExoU13OcfEemKuQzF/HAs4zHxVqlSxa1Tz/6A+XUX3otJbyxGpsE0NMljANFMG0uCFEep16RiuomT/f8ijWnyevRWBgvxtNBOqsGgKhWqgn6GU1m2zoKlLMf5OShP4itR+D3SP2XVrXc9h5Kg/8pfs+XJKYTiFAdck9hzJqLXOtSdggfy70+UgNPAc7wINsyb+1cX9bZeAzTvaO0bVbfCJo8mio3ZwMDBNZBIakGHrYWbud32qbUIdjIhN6B5NABsmxgBZA0/YtOoGcjWUI4rTJsK6JiFy60qrT+FicTxsT/xHLtawV9Lh7vi84rLdxmBnK0H3hadfFaZHvJqaGea+zFDgGAovY8KB/qeR4ASQ0J9HvBqjBn/8BnoFt6rieHJwKMQ1tLyY7O5ux7S2+siUY58sC5qWJswGmrYte/gGdHtIYOTMX0yMlpH5nvnbqJtClIF131tEdO5q8bnpQQAisFXRq6K/r5LT5MMN+1k5XFS1FAL9KWj/JkMl3fp77JuCgpXvU5ZmpN5P8CNdflsAMdOeDCckXkf1SjF6VOZ2Bkui0Lg4+L15jxUt2TdRoSei05Eoc+3xB/MEJVxTpXbV64sKBNgyjETc2zBq5pWrV9ZKcpUon+YtGrUmKq6kDk0MwVSlgQ+NQHSgI0Qevsvla3RvAXXguDgLEWkMfRQ== X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230028)(4636009)(346002)(136003)(39860400002)(376002)(396003)(451199021)(46966006)(36840700001)(40470700004)(86362001)(36756003)(316002)(41300700001)(70586007)(8676002)(4326008)(6916009)(966005)(7696005)(70206006)(54906003)(478600001)(40480700001)(55016003)(82310400005)(5660300002)(2906002)(8936002)(36860700001)(82740400003)(356005)(7636003)(16526019)(6286002)(186003)(6666004)(53546011)(1076003)(26005)(336012)(2616005)(426003)(47076005)(83380400001)(40460700003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Apr 2023 15:34:27.6560 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 9b548de8-a2eb-41b5-ddca-08db390fe7c1 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT093.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB8225 X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Hi, FYI, your patch has been queued to stable release 22.11.2 Note it hasn't been pushed to http://dpdk.org/browse/dpdk-stable yet. It will be pushed if I get no objections before 04/11/23. So please shout if anyone has objections. Also note that after the patch there's a diff of the upstream commit vs the patch applied to the branch. This will indicate if there was any rebasing needed to apply to the stable branch. If there were code changes for rebasing (ie: not only metadata diffs), please double check that the rebase was correctly done. Queued patches are on a temporary branch at: https://git.dpdk.org/dpdk-stable/log/?h=22.11-staging This queued commit can be viewed at: https://git.dpdk.org/dpdk-stable/log/?h=22.11-staging/commit/db12268a953bbacdb8b518cb1138ed6b79d38491 Thanks. Xueming Li --- >From db12268a953bbacdb8b518cb1138ed6b79d38491 Mon Sep 17 00:00:00 2001 From: Tyler Retzlaff Date: Wed, 15 Mar 2023 17:07:04 -0700 Subject: [PATCH] eal/unix: fix thread creation Cc: Xueming Li [ upstream commit b28c6196b132d1f25cb8c1bf781520fc41556b3a ] In rte_thread_create setting affinity after pthread_create may fail. Such a failure should result in the entire rte_thread_create failing but doesn't. Additionally if there is a failure to set affinity a race exists where the creating thread will free ctx and depending on scheduling of the new thread it may also free ctx (double free). Resolve the above by setting the affinity from the newly created thread using a condition variable to signal the completion of the thread start wrapper having completed. Since we are now waiting for the thread start wrapper to complete we can allocate the thread start wrapper context on the stack. While here clean up the variable naming in the context to better highlight the fields of the context require synchronization between the creating and created thread. Fixes: ce6e911d20f6 ("eal: add thread lifetime API") Signed-off-by: Tyler Retzlaff Signed-off-by: David Marchand --- lib/eal/unix/rte_thread.c | 73 ++++++++++++++++++++++++--------------- 1 file changed, 45 insertions(+), 28 deletions(-) diff --git a/lib/eal/unix/rte_thread.c b/lib/eal/unix/rte_thread.c index 37ebfcfca1..f4076122a4 100644 --- a/lib/eal/unix/rte_thread.c +++ b/lib/eal/unix/rte_thread.c @@ -5,6 +5,7 @@ #include #include +#include #include #include @@ -16,9 +17,14 @@ struct eal_tls_key { pthread_key_t thread_index; }; -struct thread_routine_ctx { +struct thread_start_context { rte_thread_func thread_func; - void *routine_args; + void *thread_args; + const rte_thread_attr_t *thread_attr; + pthread_mutex_t wrapper_mutex; + pthread_cond_t wrapper_cond; + int wrapper_ret; + bool wrapper_done; }; static int @@ -81,13 +87,29 @@ thread_map_os_priority_to_eal_priority(int policy, int os_pri, } static void * -thread_func_wrapper(void *arg) +thread_start_wrapper(void *arg) { - struct thread_routine_ctx ctx = *(struct thread_routine_ctx *)arg; + struct thread_start_context *ctx = (struct thread_start_context *)arg; + rte_thread_func thread_func = ctx->thread_func; + void *thread_args = ctx->thread_args; + int ret = 0; + + if (ctx->thread_attr != NULL && CPU_COUNT(&ctx->thread_attr->cpuset) > 0) { + ret = rte_thread_set_affinity_by_id(rte_thread_self(), &ctx->thread_attr->cpuset); + if (ret != 0) + RTE_LOG(DEBUG, EAL, "rte_thread_set_affinity_by_id failed\n"); + } - free(arg); + pthread_mutex_lock(&ctx->wrapper_mutex); + ctx->wrapper_ret = ret; + ctx->wrapper_done = true; + pthread_cond_signal(&ctx->wrapper_cond); + pthread_mutex_unlock(&ctx->wrapper_mutex); - return (void *)(uintptr_t)ctx.thread_func(ctx.routine_args); + if (ret != 0) + return NULL; + + return (void *)(uintptr_t)thread_func(thread_args); } int @@ -98,20 +120,18 @@ rte_thread_create(rte_thread_t *thread_id, int ret = 0; pthread_attr_t attr; pthread_attr_t *attrp = NULL; - struct thread_routine_ctx *ctx; struct sched_param param = { .sched_priority = 0, }; int policy = SCHED_OTHER; - - ctx = calloc(1, sizeof(*ctx)); - if (ctx == NULL) { - RTE_LOG(DEBUG, EAL, "Insufficient memory for thread context allocations\n"); - ret = ENOMEM; - goto cleanup; - } - ctx->routine_args = args; - ctx->thread_func = thread_func; + struct thread_start_context ctx = { + .thread_func = thread_func, + .thread_args = args, + .thread_attr = thread_attr, + .wrapper_done = false, + .wrapper_mutex = PTHREAD_MUTEX_INITIALIZER, + .wrapper_cond = PTHREAD_COND_INITIALIZER, + }; if (thread_attr != NULL) { ret = pthread_attr_init(&attr); @@ -133,7 +153,6 @@ rte_thread_create(rte_thread_t *thread_id, goto cleanup; } - if (thread_attr->priority == RTE_THREAD_PRIORITY_REALTIME_CRITICAL) { ret = ENOTSUP; @@ -158,24 +177,22 @@ rte_thread_create(rte_thread_t *thread_id, } ret = pthread_create((pthread_t *)&thread_id->opaque_id, attrp, - thread_func_wrapper, ctx); + thread_start_wrapper, &ctx); if (ret != 0) { RTE_LOG(DEBUG, EAL, "pthread_create failed\n"); goto cleanup; } - if (thread_attr != NULL && CPU_COUNT(&thread_attr->cpuset) > 0) { - ret = rte_thread_set_affinity_by_id(*thread_id, - &thread_attr->cpuset); - if (ret != 0) { - RTE_LOG(DEBUG, EAL, "rte_thread_set_affinity_by_id failed\n"); - goto cleanup; - } - } + pthread_mutex_lock(&ctx.wrapper_mutex); + while (!ctx.wrapper_done) + pthread_cond_wait(&ctx.wrapper_cond, &ctx.wrapper_mutex); + ret = ctx.wrapper_ret; + pthread_mutex_unlock(&ctx.wrapper_mutex); + + if (ret != 0) + pthread_join((pthread_t)thread_id->opaque_id, NULL); - ctx = NULL; cleanup: - free(ctx); if (attrp != NULL) pthread_attr_destroy(&attr); -- 2.25.1 --- Diff of the applied patch vs upstream commit (please double-check if non-empty: --- --- - 2023-04-09 21:45:41.320353400 +0800 +++ 0100-eal-unix-fix-thread-creation.patch 2023-04-09 21:45:38.719042200 +0800 @@ -1 +1 @@ -From b28c6196b132d1f25cb8c1bf781520fc41556b3a Mon Sep 17 00:00:00 2001 +From db12268a953bbacdb8b518cb1138ed6b79d38491 Mon Sep 17 00:00:00 2001 @@ -4,0 +5,3 @@ +Cc: Xueming Li + +[ upstream commit b28c6196b132d1f25cb8c1bf781520fc41556b3a ] @@ -25 +27,0 @@ -Cc: stable@dpdk.org