From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id CB89345C9B for ; Tue, 12 Nov 2024 23:08:28 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C6A444064C; Tue, 12 Nov 2024 23:08:28 +0100 (CET) Received: from mail-wr1-f49.google.com (mail-wr1-f49.google.com [209.85.221.49]) by mails.dpdk.org (Postfix) with ESMTP id 0094240676 for ; Tue, 12 Nov 2024 23:08:27 +0100 (CET) Received: by mail-wr1-f49.google.com with SMTP id ffacd0b85a97d-37d518f9abcso4465422f8f.2 for ; Tue, 12 Nov 2024 14:08:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1731449307; x=1732054107; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=K5iBmfzlEmUTz7KoxlobrYHxlwXaJZzZfM99zA2jTx4=; b=D0YeVe6Xtqk1Bdf/4ME6uOfEG6hIYTgFf6Ex9O+kKWqSfjthN95fyvl1Ao+hUftrlT EkAIzSEQ0OM1wY8EJTaFhCgi02FLw7xtxrfEx3dH3O+3ZIvNJDdjF9UY4o5DNuKhWOo0 AP0F+apIK+8imKaOmyuTNIwRsJC5ChRzquH1cv6QOSj+FepL8sxZI+eo8pVjrsoPLGeY Df8y9NTXplz25E9sscYUodmWqOrh5A6w4o7oKXLa5x6ZE8Vjtas6PWMVEhw+NroLLxiW AgrZxevNi/N2oX5R+mW1EhCtxh8xZeJMQ81wUtKYaJDRpWE2tjrvAo7aQCjbMlh04cIp y5Vg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731449307; x=1732054107; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=K5iBmfzlEmUTz7KoxlobrYHxlwXaJZzZfM99zA2jTx4=; b=tK1bxKX/+sh+CZHYahu5hJm15Tu2JNRaRyGV04MNk4gupA80ucDi/VnsUPa6ScIRee Zk+07ExeUP6pMhk+UaxbPa1Ffj8MO9CXtW483Cq0goU7Gkw3KuY4DuvWNV/Dj5GJBpIg jhW1HsDBbGYGNGtwjwwnsu+lfkKYEZdprBSmBY882eACZdJnH9y/3Qo7BKqIg0TL2gM7 XS1wPoqAxPGOZgEQ1f9tUTzG6EjcnJjNoc9jwh1uob9nx9a3ZUjxyJmD/RR+quiCoXgj ITxOuIq3wE+ZA3aY7fRvZgHhggRxGbyf2DCuj6fzUrl+o2OLS6fkh9MsE/LRGph5FmjE HobA== X-Forwarded-Encrypted: i=1; AJvYcCWbheP07wp3swzA+HqFkx62SKFTRHJ2ITQtmiJQCBOV0osV1DZgYlRaxloAyQVtYjPuktS95as=@dpdk.org X-Gm-Message-State: AOJu0Yz63heNzkj3/V7c3avNcWuOOOc3vFdxrthHBKxP8e4Rb3sKtaKM iTshKsPAXxz1nbXRoAl59tyjejNXzGF45fAbjgZvJhmUnOYWnp/TWnhseA== X-Google-Smtp-Source: AGHT+IGCZGmJKwVvJFYs27+dfIQk1MaxgtUYBM19UbHNHAET6bkEwTn2P1OkSxExBpjW+GsIFZh57Q== X-Received: by 2002:a5d:5986:0:b0:37d:53dd:4dec with SMTP id ffacd0b85a97d-381f186cbe0mr15300544f8f.15.1731449306047; Tue, 12 Nov 2024 14:08:26 -0800 (PST) Received: from localhost ([2a01:4b00:d036:ae00:e89d:e9a1:da72:2f9d]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-381ed997313sm16336617f8f.53.2024.11.12.14.08.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Nov 2024 14:08:25 -0800 (PST) From: luca.boccassi@gmail.com To: David Marchand Cc: Luca Boccassi , Stephen Hemminger , Chengwen Feng , dpdk stable Subject: patch 'eal/unix: optimize thread creation' has been queued to stable release 22.11.7 Date: Tue, 12 Nov 2024 22:07:19 +0000 Message-ID: <20241112220754.666489-9-luca.boccassi@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241112220754.666489-1-luca.boccassi@gmail.com> References: <20241023211704.1216956-79-luca.boccassi@gmail.com> <20241112220754.666489-1-luca.boccassi@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Hi, FYI, your patch has been queued to stable release 22.11.7 Note it hasn't been pushed to http://dpdk.org/browse/dpdk-stable yet. It will be pushed if I get no objections before 11/14/24. So please shout if anyone has objections. Also note that after the patch there's a diff of the upstream commit vs the patch applied to the branch. This will indicate if there was any rebasing needed to apply to the stable branch. If there were code changes for rebasing (ie: not only metadata diffs), please double check that the rebase was correctly done. Queued patches are on a temporary branch at: https://github.com/bluca/dpdk-stable This queued commit can be viewed at: https://github.com/bluca/dpdk-stable/commit/02097752bb88526197af8b0dd0a8da9e6321cf9a Thanks. Luca Boccassi --- >From 02097752bb88526197af8b0dd0a8da9e6321cf9a Mon Sep 17 00:00:00 2001 From: David Marchand Date: Sat, 2 Nov 2024 10:38:16 +0100 Subject: [PATCH] eal/unix: optimize thread creation [ upstream commit 64f27886b8bf127cd365a8a3ed5c05852a5ae81d ] Setting the cpu affinity of the child thread from the parent thread is racy when using pthread_setaffinity_np, as the child thread may start running and initialize before affinity is set. On the other hand, setting the cpu affinity from the child thread itself may fail, so the parent thread waits for the child thread to report whether this call succeeded. This synchronisation point resulted in a significant slow down of rte_thread_create() (as seen in the lcores_autotest unit tests, in OBS for some ARM systems). Another option for setting cpu affinity is to use the not portable pthread_attr_setaffinity_np available in FreeBSD and glibc, but not available in musl. Fixes: b28c6196b132 ("eal/unix: fix thread creation") Signed-off-by: David Marchand Acked-by: Luca Boccassi Acked-by: Stephen Hemminger Acked-by: Chengwen Feng --- lib/eal/unix/meson.build | 5 +++++ lib/eal/unix/rte_thread.c | 25 +++++++++++++++++++++++++ 2 files changed, 30 insertions(+) diff --git a/lib/eal/unix/meson.build b/lib/eal/unix/meson.build index cc7d67dd32..f1eb82e16a 100644 --- a/lib/eal/unix/meson.build +++ b/lib/eal/unix/meson.build @@ -11,3 +11,8 @@ sources += files( 'eal_unix_timer.c', 'rte_thread.c', ) + +if is_freebsd or cc.has_function('pthread_attr_setaffinity_np', args: '-D_GNU_SOURCE', + prefix : '#include ') + cflags += '-DRTE_EAL_PTHREAD_ATTR_SETAFFINITY_NP' +endif diff --git a/lib/eal/unix/rte_thread.c b/lib/eal/unix/rte_thread.c index f4076122a4..e546329120 100644 --- a/lib/eal/unix/rte_thread.c +++ b/lib/eal/unix/rte_thread.c @@ -17,6 +17,7 @@ struct eal_tls_key { pthread_key_t thread_index; }; +#ifndef RTE_EAL_PTHREAD_ATTR_SETAFFINITY_NP struct thread_start_context { rte_thread_func thread_func; void *thread_args; @@ -26,6 +27,7 @@ struct thread_start_context { int wrapper_ret; bool wrapper_done; }; +#endif static int thread_map_priority_to_os_value(enum rte_thread_priority eal_pri, int *os_pri, @@ -86,6 +88,7 @@ thread_map_os_priority_to_eal_priority(int policy, int os_pri, return 0; } +#ifndef RTE_EAL_PTHREAD_ATTR_SETAFFINITY_NP static void * thread_start_wrapper(void *arg) { @@ -111,6 +114,7 @@ thread_start_wrapper(void *arg) return (void *)(uintptr_t)thread_func(thread_args); } +#endif int rte_thread_create(rte_thread_t *thread_id, @@ -124,6 +128,7 @@ rte_thread_create(rte_thread_t *thread_id, .sched_priority = 0, }; int policy = SCHED_OTHER; +#ifndef RTE_EAL_PTHREAD_ATTR_SETAFFINITY_NP struct thread_start_context ctx = { .thread_func = thread_func, .thread_args = args, @@ -132,6 +137,7 @@ rte_thread_create(rte_thread_t *thread_id, .wrapper_mutex = PTHREAD_MUTEX_INITIALIZER, .wrapper_cond = PTHREAD_COND_INITIALIZER, }; +#endif if (thread_attr != NULL) { ret = pthread_attr_init(&attr); @@ -142,6 +148,16 @@ rte_thread_create(rte_thread_t *thread_id, attrp = &attr; +#ifdef RTE_EAL_PTHREAD_ATTR_SETAFFINITY_NP + if (CPU_COUNT(&thread_attr->cpuset) > 0) { + ret = pthread_attr_setaffinity_np(attrp, sizeof(thread_attr->cpuset), + &thread_attr->cpuset); + if (ret != 0) { + RTE_LOG(DEBUG, EAL, "pthread_attr_setaffinity_np failed"); + goto cleanup; + } + } +#endif /* * Set the inherit scheduler parameter to explicit, * otherwise the priority attribute is ignored. @@ -176,6 +192,14 @@ rte_thread_create(rte_thread_t *thread_id, } } +#ifdef RTE_EAL_PTHREAD_ATTR_SETAFFINITY_NP + ret = pthread_create((pthread_t *)&thread_id->opaque_id, attrp, + (void *)(void *)thread_func, args); + if (ret != 0) { + RTE_LOG(DEBUG, EAL, "pthread_create failed\n"); + goto cleanup; + } +#else /* !RTE_EAL_PTHREAD_ATTR_SETAFFINITY_NP */ ret = pthread_create((pthread_t *)&thread_id->opaque_id, attrp, thread_start_wrapper, &ctx); if (ret != 0) { @@ -191,6 +215,7 @@ rte_thread_create(rte_thread_t *thread_id, if (ret != 0) pthread_join((pthread_t)thread_id->opaque_id, NULL); +#endif /* RTE_EAL_PTHREAD_ATTR_SETAFFINITY_NP */ cleanup: if (attrp != NULL) -- 2.45.2 --- Diff of the applied patch vs upstream commit (please double-check if non-empty: --- --- - 2024-11-12 22:06:58.987392626 +0000 +++ 0009-eal-unix-optimize-thread-creation.patch 2024-11-12 22:06:58.639306763 +0000 @@ -1 +1 @@ -From 64f27886b8bf127cd365a8a3ed5c05852a5ae81d Mon Sep 17 00:00:00 2001 +From 02097752bb88526197af8b0dd0a8da9e6321cf9a Mon Sep 17 00:00:00 2001 @@ -5,0 +6,2 @@ +[ upstream commit 64f27886b8bf127cd365a8a3ed5c05852a5ae81d ] + @@ -23 +24,0 @@ -Cc: stable@dpdk.org @@ -48 +49 @@ -index 1b4c73f58e..ea629c2065 100644 +index f4076122a4..e546329120 100644 @@ -51 +52 @@ -@@ -19,6 +19,7 @@ struct eal_tls_key { +@@ -17,6 +17,7 @@ struct eal_tls_key { @@ -59 +60 @@ -@@ -28,6 +29,7 @@ struct thread_start_context { +@@ -26,6 +27,7 @@ struct thread_start_context { @@ -67 +68 @@ -@@ -88,6 +90,7 @@ thread_map_os_priority_to_eal_priority(int policy, int os_pri, +@@ -86,6 +88,7 @@ thread_map_os_priority_to_eal_priority(int policy, int os_pri, @@ -75 +76 @@ -@@ -113,6 +116,7 @@ thread_start_wrapper(void *arg) +@@ -111,6 +114,7 @@ thread_start_wrapper(void *arg) @@ -83 +84 @@ -@@ -126,6 +130,7 @@ rte_thread_create(rte_thread_t *thread_id, +@@ -124,6 +128,7 @@ rte_thread_create(rte_thread_t *thread_id, @@ -91 +92 @@ -@@ -134,6 +139,7 @@ rte_thread_create(rte_thread_t *thread_id, +@@ -132,6 +137,7 @@ rte_thread_create(rte_thread_t *thread_id, @@ -99 +100 @@ -@@ -144,6 +150,16 @@ rte_thread_create(rte_thread_t *thread_id, +@@ -142,6 +148,16 @@ rte_thread_create(rte_thread_t *thread_id, @@ -108 +109 @@ -+ EAL_LOG(DEBUG, "pthread_attr_setaffinity_np failed"); ++ RTE_LOG(DEBUG, EAL, "pthread_attr_setaffinity_np failed"); @@ -116 +117 @@ -@@ -178,6 +194,14 @@ rte_thread_create(rte_thread_t *thread_id, +@@ -176,6 +192,14 @@ rte_thread_create(rte_thread_t *thread_id, @@ -124 +125 @@ -+ EAL_LOG(DEBUG, "pthread_create failed"); ++ RTE_LOG(DEBUG, EAL, "pthread_create failed\n"); @@ -131 +132 @@ -@@ -193,6 +217,7 @@ rte_thread_create(rte_thread_t *thread_id, +@@ -191,6 +215,7 @@ rte_thread_create(rte_thread_t *thread_id, @@ -134 +135 @@ - rte_thread_join(*thread_id, NULL); + pthread_join((pthread_t)thread_id->opaque_id, NULL);