From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <stable-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
by inbox.dpdk.org (Postfix) with ESMTP id 70ED341E95
for <public@inbox.dpdk.org>; Tue, 14 Mar 2023 23:50:45 +0100 (CET)
Received: from mails.dpdk.org (localhost [127.0.0.1])
by mails.dpdk.org (Postfix) with ESMTP id 6ABB14282D;
Tue, 14 Mar 2023 23:50:45 +0100 (CET)
Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182])
by mails.dpdk.org (Postfix) with ESMTP id 5363F40F18;
Tue, 14 Mar 2023 23:50:41 +0100 (CET)
Received: by linux.microsoft.com (Postfix, from userid 1086)
id A43BB205700B; Tue, 14 Mar 2023 15:50:40 -0700 (PDT)
DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com A43BB205700B
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com;
s=default; t=1678834240;
bh=ZFrn2EfpicGDY2XF3sREWl6QiNpP3JKa5t+OJ802J8w=;
h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
b=q+Jh4J8Id/xAO0U9eQ8g/TNPTdT6a6VJUXGMcQ9G17Bv7MUP6p/lMSbOjco9XYySS
2/omsHB0chIGmqKelKRz1rsWGh7CmvvuK5eNRLcIaEFG7qsCiuiSOweILWTElP8DKy
naWhOFPsWBLAoRuo9fUwlJA+QIDI94Ffv57qG5E4=
From: Tyler Retzlaff <roretzla@linux.microsoft.com>
To: dev@dpdk.org
Cc: thomas@monjalon.net, david.marchand@redhat.com,
Tyler Retzlaff <roretzla@linux.microsoft.com>, stable@dpdk.org
Subject: [PATCH v4 2/2] eal: fix failure path race setting new thread affinity
Date: Tue, 14 Mar 2023 15:50:39 -0700
Message-Id: <1678834239-11569-3-git-send-email-roretzla@linux.microsoft.com>
X-Mailer: git-send-email 1.8.3.1
In-Reply-To: <1678834239-11569-1-git-send-email-roretzla@linux.microsoft.com>
References: <1677782682-27200-1-git-send-email-roretzla@linux.microsoft.com>
<1678834239-11569-1-git-send-email-roretzla@linux.microsoft.com>
X-BeenThere: stable@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: patches for DPDK stable branches <stable.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/stable>,
<mailto:stable-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/stable/>
List-Post: <mailto:stable@dpdk.org>
List-Help: <mailto:stable-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/stable>,
<mailto:stable-request@dpdk.org?subject=subscribe>
Errors-To: stable-bounces@dpdk.org
In rte_thread_create setting affinity after pthread_create may fail.
Such a failure should result in the entire rte_thread_create failing
but doesn't.
Additionally if there is a failure to set affinity a race exists where
the creating thread will free ctx and depending on scheduling of the new
thread it may also free ctx (double free).
Resolve both of the above issues by setting the affinity from the newly
created thread instead of after thread creation. To achieve this modify
the existing thread wrapper to allow the creating thread to wait on the
result of the set affinity operation.
Fixes: ce6e911d20f6 ("eal: add thread lifetime API")
Cc: stable@dpdk.org
Signed-off-by: Tyler Retzlaff <roretzla@linux.microsoft.com>
---
lib/eal/unix/rte_thread.c | 51 +++++++++++++++++++++++++++++++++++++----------
1 file changed, 40 insertions(+), 11 deletions(-)
diff --git a/lib/eal/unix/rte_thread.c b/lib/eal/unix/rte_thread.c
index 37ebfcf..9fcf53b 100644
--- a/lib/eal/unix/rte_thread.c
+++ b/lib/eal/unix/rte_thread.c
@@ -16,8 +16,17 @@ struct eal_tls_key {
pthread_key_t thread_index;
};
+enum __rte_thread_wrapper_status {
+ THREAD_WRAPPER_LAUNCHING, /* Yet to call thread_func function */
+ THREAD_WRAPPER_RUNNING, /* Thread is running successfully */
+ THREAD_WRAPPER_ERROR /* Thread thread_wrapper encountered an error */
+};
+
struct thread_routine_ctx {
rte_thread_func thread_func;
+ const rte_thread_attr_t *thread_attr;
+ int thread_wrapper_ret;
+ enum __rte_thread_wrapper_status thread_wrapper_status;
void *routine_args;
};
@@ -83,11 +92,22 @@ struct thread_routine_ctx {
static void *
thread_func_wrapper(void *arg)
{
- struct thread_routine_ctx ctx = *(struct thread_routine_ctx *)arg;
-
- free(arg);
+ struct thread_routine_ctx *ctx = (struct thread_routine_ctx *)arg;
+ rte_thread_func thread_func = ctx->thread_func;
+ void *thread_args = ctx->routine_args;
+
+ if (ctx->thread_attr != NULL && CPU_COUNT(&ctx->thread_attr->cpuset) > 0) {
+ ctx->thread_wrapper_ret = rte_thread_set_affinity(&ctx->thread_attr->cpuset);
+ if (ctx->thread_wrapper_ret != 0) {
+ RTE_LOG(DEBUG, EAL, "rte_thread_set_affinity failed\n");
+ __atomic_store_n(&ctx->thread_wrapper_status,
+ THREAD_WRAPPER_ERROR, __ATOMIC_RELEASE);
+ }
+ }
+ __atomic_store_n(&ctx->thread_wrapper_status,
+ THREAD_WRAPPER_RUNNING, __ATOMIC_RELEASE);
- return (void *)(uintptr_t)ctx.thread_func(ctx.routine_args);
+ return (void *)(uintptr_t)thread_func(thread_args);
}
int
@@ -98,6 +118,7 @@ struct thread_routine_ctx {
int ret = 0;
pthread_attr_t attr;
pthread_attr_t *attrp = NULL;
+ enum __rte_thread_wrapper_status thread_wrapper_status;
struct thread_routine_ctx *ctx;
struct sched_param param = {
.sched_priority = 0,
@@ -111,7 +132,10 @@ struct thread_routine_ctx {
goto cleanup;
}
ctx->routine_args = args;
+ ctx->thread_attr = thread_attr;
ctx->thread_func = thread_func;
+ ctx->thread_wrapper_ret = 0;
+ ctx->thread_wrapper_status = THREAD_WRAPPER_LAUNCHING;
if (thread_attr != NULL) {
ret = pthread_attr_init(&attr);
@@ -164,14 +188,19 @@ struct thread_routine_ctx {
goto cleanup;
}
- if (thread_attr != NULL && CPU_COUNT(&thread_attr->cpuset) > 0) {
- ret = rte_thread_set_affinity_by_id(*thread_id,
- &thread_attr->cpuset);
- if (ret != 0) {
- RTE_LOG(DEBUG, EAL, "rte_thread_set_affinity_by_id failed\n");
- goto cleanup;
- }
+ /* Wait for the thread wrapper to initialize thread successfully */
+ while ((thread_wrapper_status =
+ __atomic_load_n(&ctx->thread_wrapper_status,
+ __ATOMIC_ACQUIRE)) == THREAD_WRAPPER_LAUNCHING)
+ sched_yield();
+
+ /* Check if the control thread encountered an error */
+ if (thread_wrapper_status == THREAD_WRAPPER_ERROR) {
+ /* thread wrapper is exiting */
+ pthread_join((pthread_t)thread_id->opaque_id, NULL);
+ ret = ctx->thread_wrapper_ret;
}
+ free(ctx);
ctx = NULL;
cleanup:
--
1.8.3.1