From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
by inbox.dpdk.org (Postfix) with ESMTP id E2618A0C55;
Wed, 13 Oct 2021 22:18:22 +0200 (CEST)
Received: from [217.70.189.124] (localhost [127.0.0.1])
by mails.dpdk.org (Postfix) with ESMTP id D22324116A;
Wed, 13 Oct 2021 22:18:22 +0200 (CEST)
Received: from foss.arm.com (foss.arm.com [217.140.110.172])
by mails.dpdk.org (Postfix) with ESMTP id CE011410FA
for <dev@dpdk.org>; Wed, 13 Oct 2021 22:18:20 +0200 (CEST)
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4E5941063;
Wed, 13 Oct 2021 13:18:20 -0700 (PDT)
Received: from qc2400f-1.austin.arm.com (qc2400f-1.austin.arm.com
[10.118.12.44])
by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 41A533F70D;
Wed, 13 Oct 2021 13:18:20 -0700 (PDT)
From: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
To: dev@dpdk.org, honnappa.nagarahalli@arm.com, olivier.matz@6wind.com,
lucp.at.work@gmail.com, stephen@networkplumber.org,
david.marchand@redhat.com, thomas@monjalon.net
Cc: ruifeng.wang@arm.com,
nd@arm.com
Date: Wed, 13 Oct 2021 15:18:10 -0500
Message-Id: <20211013201811.15247-1-honnappa.nagarahalli@arm.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <20210730213709.19400-1-honnappa.nagarahalli@arm.com>
References: <20210730213709.19400-1-honnappa.nagarahalli@arm.com>
Subject: [dpdk-dev] [PATCH v4 1/2] eal: simplify the implementation of
rte_ctrl_thread_create
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>
Remove the usage of pthread barrier and replace it with
synchronization using atomic variable. This also removes
the use of reference count required to synchronize freeing
the memory.
Signed-off-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
---
v4:
1) Used rte_delay_us_sleep instead of sched_yield. Using sched_yield
requires to maintain a Windows version of the rte_ctrl_thread_create
API.
v3:
1) RFC to Patch
2) Changed commit log and API description [Olivier]
3) Used sched_yield while spinning [Stephen]
4) Fixed use after free
5) Added a test case
v2:
1) Used compiler's C++11 atomic built-ins to access the shared variable.
lib/eal/common/eal_common_thread.c | 87 +++++++++++++++---------------
lib/eal/include/rte_lcore.h | 9 ++--
2 files changed, 49 insertions(+), 47 deletions(-)
diff --git a/lib/eal/common/eal_common_thread.c b/lib/eal/common/eal_common_thread.c
index 1a52f42a2b..e8e4149f69 100644
--- a/lib/eal/common/eal_common_thread.c
+++ b/lib/eal/common/eal_common_thread.c
@@ -166,38 +166,44 @@ __rte_thread_uninit(void)
RTE_PER_LCORE(_lcore_id) = LCORE_ID_ANY;
}
+enum __rte_ctrl_thread_status {
+ __RTE_CTRL_THREAD_LAUNCHING, /* Yet to call pthread_create function */
+ __RTE_CTRL_THREAD_RUNNING, /* Control thread is running successfully */
+ __RTE_CTRL_THREAD_ERROR /* Control thread encountered an error */
+};
+
struct rte_thread_ctrl_params {
void *(*start_routine)(void *);
void *arg;
- pthread_barrier_t configured;
- unsigned int refcnt;
+ int ret;
+ /* Control thread status. If the status is __RTE_CTRL_THREAD_ERROR
+ * 'ret' has the error code.
+ */
+ enum __rte_ctrl_thread_status ctrl_thread_status;
};
-static void ctrl_params_free(struct rte_thread_ctrl_params *params)
-{
- if (__atomic_sub_fetch(¶ms->refcnt, 1, __ATOMIC_ACQ_REL) == 0) {
- (void)pthread_barrier_destroy(¶ms->configured);
- free(params);
- }
-}
-
static void *ctrl_thread_init(void *arg)
{
struct internal_config *internal_conf =
eal_get_internal_configuration();
rte_cpuset_t *cpuset = &internal_conf->ctrl_cpuset;
struct rte_thread_ctrl_params *params = arg;
- void *(*start_routine)(void *);
+ void *(*start_routine)(void *) = params->start_routine;
void *routine_arg = params->arg;
__rte_thread_init(rte_lcore_id(), cpuset);
-
- pthread_barrier_wait(¶ms->configured);
- start_routine = params->start_routine;
- ctrl_params_free(params);
-
- if (start_routine == NULL)
+ params->ret = pthread_setaffinity_np(pthread_self(),
+ sizeof(*cpuset), cpuset);
+ if (params->ret != 0) {
+ __atomic_store_n(¶ms->ctrl_thread_status,
+ __RTE_CTRL_THREAD_ERROR,
+ __ATOMIC_RELEASE);
return NULL;
+ }
+
+ __atomic_store_n(¶ms->ctrl_thread_status,
+ __RTE_CTRL_THREAD_RUNNING,
+ __ATOMIC_RELEASE);
return start_routine(routine_arg);
}
@@ -207,10 +213,8 @@ rte_ctrl_thread_create(pthread_t *thread, const char *name,
const pthread_attr_t *attr,
void *(*start_routine)(void *), void *arg)
{
- struct internal_config *internal_conf =
- eal_get_internal_configuration();
- rte_cpuset_t *cpuset = &internal_conf->ctrl_cpuset;
struct rte_thread_ctrl_params *params;
+ enum __rte_ctrl_thread_status ctrl_thread_status;
int ret;
params = malloc(sizeof(*params));
@@ -219,15 +223,14 @@ rte_ctrl_thread_create(pthread_t *thread, const char *name,
params->start_routine = start_routine;
params->arg = arg;
- params->refcnt = 2;
-
- ret = pthread_barrier_init(¶ms->configured, NULL, 2);
- if (ret != 0)
- goto fail_no_barrier;
+ params->ret = 0;
+ params->ctrl_thread_status = __RTE_CTRL_THREAD_LAUNCHING;
ret = pthread_create(thread, attr, ctrl_thread_init, (void *)params);
- if (ret != 0)
- goto fail_with_barrier;
+ if (ret != 0) {
+ free(params);
+ return -ret;
+ }
if (name != NULL) {
ret = rte_thread_setname(*thread, name);
@@ -236,24 +239,22 @@ rte_ctrl_thread_create(pthread_t *thread, const char *name,
"Cannot set name for ctrl thread\n");
}
- ret = pthread_setaffinity_np(*thread, sizeof(*cpuset), cpuset);
- if (ret != 0)
- params->start_routine = NULL;
-
- pthread_barrier_wait(¶ms->configured);
- ctrl_params_free(params);
-
- if (ret != 0)
- /* start_routine has been set to NULL above; */
- /* ctrl thread will exit immediately */
+ /* Wait for the control thread to initialize successfully */
+ while ((ctrl_thread_status =
+ __atomic_load_n(¶ms->ctrl_thread_status, __ATOMIC_ACQUIRE))
+ == __RTE_CTRL_THREAD_LAUNCHING)
+ /* Yield the CPU. Using sched_yield call requires maintaining
+ * another implementation for Windows as sched_yield is not
+ * supported on Windows.
+ */
+ rte_delay_us_sleep(1);
+
+ /* Check if the control thread encountered an error */
+ if (ctrl_thread_status == __RTE_CTRL_THREAD_ERROR)
+ /* ctrl thread is exiting */
pthread_join(*thread, NULL);
- return -ret;
-
-fail_with_barrier:
- (void)pthread_barrier_destroy(¶ms->configured);
-
-fail_no_barrier:
+ ret = params->ret;
free(params);
return -ret;
diff --git a/lib/eal/include/rte_lcore.h b/lib/eal/include/rte_lcore.h
index 1550b75da0..b81c8b2685 100644
--- a/lib/eal/include/rte_lcore.h
+++ b/lib/eal/include/rte_lcore.h
@@ -420,10 +420,11 @@ rte_thread_unregister(void);
/**
* Create a control thread.
*
- * Wrapper to pthread_create(), pthread_setname_np() and
- * pthread_setaffinity_np(). The affinity of the new thread is based
- * on the CPU affinity retrieved at the time rte_eal_init() was called,
- * the dataplane and service lcores are then excluded.
+ * Creates a control thread with the given name and attributes. The
+ * affinity of the new thread is based on the CPU affinity retrieved
+ * at the time rte_eal_init() was called, the dataplane and service
+ * lcores are then excluded. If setting the name of the thread fails,
+ * the error is ignored and a debug message is logged.
*
* @param thread
* Filled with the thread id of the new created thread.
--
2.25.1