From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 61D6C45C29 for ; Sat, 2 Nov 2024 11:09:13 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 3DA634021E; Sat, 2 Nov 2024 11:09:13 +0100 (CET) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id E16BD4021E for ; Sat, 2 Nov 2024 11:09:11 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1730542151; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=h3Hn/7IAhiRdDok3yBlaSfVeMzz022Mrs52KHxE+p0M=; b=gNgikYdAtrChVVQ7ZnzrTkoz8R9qNkZAv6SNyRmRbfTSuC3cNjQc22LryS8qiDpuzeBGJG 33YzQawpQy4BmRWhi0AnUCI1Mna9kxXRW+Ab9Wj2AZ7zDjDy0vo6y/sKufVnf8F6Xphn8H abvAA9tFwu3k8d3rJ7jmt/DBmujS8PU= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-27-j6NKd_w2PDipTyMXHEdIKA-1; Sat, 02 Nov 2024 06:09:08 -0400 X-MC-Unique: j6NKd_w2PDipTyMXHEdIKA-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id E95BA19560B1; Sat, 2 Nov 2024 10:09:06 +0000 (UTC) Received: from dmarchan.redhat.com (unknown [10.45.224.57]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 77F3A195605A; Sat, 2 Nov 2024 10:09:04 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: thomas@monjalon.net, stephen@networkplumber.org, luca.boccassi@gmail.com, stable@dpdk.org, Tyler Retzlaff Subject: [PATCH] eal/unix: optimize thread creation with glibc Date: Sat, 2 Nov 2024 11:08:39 +0100 Message-ID: <20241102100839.2325651-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Setting the cpu affinity of the child thread from the parent thread is racy when using pthread_setaffinity_np, as the child thread may start running and initialize before affinity is set. On the other hand, setting the cpu affinity from the child thread itself may fail, so the parent thread waits for the child thread to report whether this call succeeded. This synchronisation point resulted in a significant slow down of rte_thread_create() (as seen in the lcores_autotest unit tests, in OBS for some ARM systems). Another option for setting cpu affinity is to use the not portable pthread_attr_setaffinity_np, but it is not available with musl. Assume availability by relying on __USE_GNU that is not set with musl. Fixes: b28c6196b132 ("eal/unix: fix thread creation") Cc: stable@dpdk.org Signed-off-by: David Marchand --- lib/eal/unix/rte_thread.c | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/lib/eal/unix/rte_thread.c b/lib/eal/unix/rte_thread.c index 1b4c73f58e..e42b6c37a2 100644 --- a/lib/eal/unix/rte_thread.c +++ b/lib/eal/unix/rte_thread.c @@ -4,6 +4,7 @@ */ #include +#include #include #include #include @@ -19,6 +20,7 @@ struct eal_tls_key { pthread_key_t thread_index; }; +#ifndef __USE_GNU struct thread_start_context { rte_thread_func thread_func; void *thread_args; @@ -28,6 +30,7 @@ struct thread_start_context { int wrapper_ret; bool wrapper_done; }; +#endif static int thread_map_priority_to_os_value(enum rte_thread_priority eal_pri, int *os_pri, @@ -88,6 +91,7 @@ thread_map_os_priority_to_eal_priority(int policy, int os_pri, return 0; } +#ifndef __USE_GNU static void * thread_start_wrapper(void *arg) { @@ -113,6 +117,7 @@ thread_start_wrapper(void *arg) return (void *)(uintptr_t)thread_func(thread_args); } +#endif int rte_thread_create(rte_thread_t *thread_id, @@ -126,6 +131,7 @@ rte_thread_create(rte_thread_t *thread_id, .sched_priority = 0, }; int policy = SCHED_OTHER; +#ifndef __USE_GNU struct thread_start_context ctx = { .thread_func = thread_func, .thread_args = args, @@ -134,6 +140,7 @@ rte_thread_create(rte_thread_t *thread_id, .wrapper_mutex = PTHREAD_MUTEX_INITIALIZER, .wrapper_cond = PTHREAD_COND_INITIALIZER, }; +#endif if (thread_attr != NULL) { ret = pthread_attr_init(&attr); @@ -144,6 +151,16 @@ rte_thread_create(rte_thread_t *thread_id, attrp = &attr; +#ifdef __USE_GNU + if (CPU_COUNT(&thread_attr->cpuset) > 0) { + ret = pthread_attr_setaffinity_np(attrp, sizeof(thread_attr->cpuset), + &thread_attr->cpuset); + if (ret != 0) { + EAL_LOG(DEBUG, "pthread_attr_setaffinity_np failed"); + goto cleanup; + } + } +#endif /* * Set the inherit scheduler parameter to explicit, * otherwise the priority attribute is ignored. @@ -178,6 +195,14 @@ rte_thread_create(rte_thread_t *thread_id, } } +#ifdef __USE_GNU + ret = pthread_create((pthread_t *)&thread_id->opaque_id, attrp, + (void *)(void *)thread_func, args); + if (ret != 0) { + EAL_LOG(DEBUG, "pthread_create failed"); + goto cleanup; + } +#else /* !__USE_GNU */ ret = pthread_create((pthread_t *)&thread_id->opaque_id, attrp, thread_start_wrapper, &ctx); if (ret != 0) { @@ -193,6 +218,7 @@ rte_thread_create(rte_thread_t *thread_id, if (ret != 0) rte_thread_join(*thread_id, NULL); +#endif /* __USE_GNU */ cleanup: if (attrp != NULL) -- 2.46.2