From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 18AD943214 for ; Fri, 27 Oct 2023 10:12:32 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1014440A67; Fri, 27 Oct 2023 10:12:32 +0200 (CEST) Received: from wout3-smtp.messagingengine.com (wout3-smtp.messagingengine.com [64.147.123.19]) by mails.dpdk.org (Postfix) with ESMTP id E17DA40272; Fri, 27 Oct 2023 10:12:30 +0200 (CEST) Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailout.west.internal (Postfix) with ESMTP id F42263200928; Fri, 27 Oct 2023 04:12:26 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute6.internal (MEProxy); Fri, 27 Oct 2023 04:12:28 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= cc:cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm3; t= 1698394346; x=1698480746; bh=CTwSuxgwI2YfYxp24yuewCwJHp4E+iC6dbc wpQQqPig=; b=nG6qm1iPkZ5ydEU7D2cxehRm0GhOFnBM7U+5yczlLUGVqULKxsE pyxUyvOpZdAqWZtlggJWJmBFTpCMluiQ7q4jxJ+0d7ohhF6FGUr39zBY1apaIsA5 KC9sPYSjGEsoSbrNEsXO791q3xrIbpIuGcoJt7xhig2IQTlfftTKndTh5GZHq53c iMTxmmwYWrbrV8Im9UGmRVFpDGFZnlA8V8b6gSI1mrDHOq1Tz9C2zTFoOj574R6J K9UQR2eukIdWTxwVKA0v+38LQzQzAqdCBLwfdqeMkAxHeNdDCEqZ8k2f8mCE8miw JIW6uzXe7TxVoqj+OeCq04Zq3MS+MD0oghQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t= 1698394346; x=1698480746; bh=CTwSuxgwI2YfYxp24yuewCwJHp4E+iC6dbc wpQQqPig=; b=mZ8TlWCS5OwUH+VWYqy/jzcWsWGERtmZd+husKaedWfo/4iZnAv PL9yCWaIyWpoNSxiXBp/9e8mYWHm8HXKmmsnMYrTECE75M/vEwh2u53dzOpaMuet g6zzS8WoILhOCVXP19omKR0xaIGg6RDj64YUbcF3noJkxFi0NEJEv5o99qF6j1F8 iHK9ADxzrWxdMZTFC79bPKNRUpUWsdGMYV5KG2qphTljlLn2QQ0yuP5bm5TZaM2o W73BkSqm4l2TR4M59A/iH9MHiuVCyS5Bd7GgqyV+tCXWoqN4Imnc/0tNp9NX+uo/ vhiADw33328LkY49wrj2UWgoaQ/y3RvJXNg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrleeggddtudcutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvfevufffkffojghfgggtgfesthekredtredtjeenucfhrhhomhepvfhhohhm rghsucfoohhnjhgrlhhonhcuoehthhhomhgrshesmhhonhhjrghlohhnrdhnvghtqeenuc ggtffrrghtthgvrhhnpedtieffffegfeetlefhkeeuteeuudffjefgleevtdeijedukefg veehteehheegjeenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfh hrohhmpehthhhomhgrshesmhhonhhjrghlohhnrdhnvght X-ME-Proxy: Feedback-ID: i47234305:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 27 Oct 2023 04:12:24 -0400 (EDT) From: Thomas Monjalon To: dev@dpdk.org Cc: David Marchand , stable@dpdk.org, =?UTF-8?q?Morten=20Br=C3=B8rup?= , Anatoly Burakov , Tyler Retzlaff , Narcisa Vasile , Stephen Hemminger , Dmitry Kozlyuk , Konstantin Ananyev , Andrew Rybchenko Subject: [PATCH v6 1/1] eal/unix: allow creating thread with real-time priority Date: Fri, 27 Oct 2023 10:08:52 +0200 Message-ID: <20231027081158.1358064-1-thomas@monjalon.net> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231024125416.798897-1-thomas@monjalon.net> References: <20231024125416.798897-1-thomas@monjalon.net> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org When adding an API for creating threads, the real-time priority has been forbidden on Unix. There is a known issue with ring behaviour, but it should not be completely forbidden. Real-time thread can block some kernel threads on the same core, making the system unstable. That's why a sleep is added in the test thread, and a warning is logged when using real-time priority. Fixes: ca04c78b6262 ("eal: get/set thread priority per thread identifier") Fixes: ce6e911d20f6 ("eal: add thread lifetime API") Fixes: a7ba40b2b1bf ("drivers: convert to internal control threads") Cc: stable@dpdk.org Signed-off-by: Thomas Monjalon Acked-by: Morten Brørup --- v1: no yield at all v2: more comments, sched_yield() and Sleep(0) on Windows v3: 2 yield functions with sleep in realtime version v4: runtime warning, longer sleep on Unix and lighter yield on Windows v5: fix build and increase Unix sleep to 1 ms v6: no yield helper, use a big sleep for testing purpose It is not clear how to implement helpers which would work on all systems (different configuration, arch and OS). So the first patch adding yield functions is dropped for now. --- app/test/test_threads.c | 12 ++-------- .../prog_guide/env_abstraction_layer.rst | 4 +++- lib/eal/include/rte_thread.h | 8 +++++-- lib/eal/unix/rte_thread.c | 22 +++++++++++-------- 4 files changed, 24 insertions(+), 22 deletions(-) diff --git a/app/test/test_threads.c b/app/test/test_threads.c index 4ac3f2671a..a863214137 100644 --- a/app/test/test_threads.c +++ b/app/test/test_threads.c @@ -5,6 +5,7 @@ #include #include +#include #include #include "test.h" @@ -22,7 +23,7 @@ thread_main(void *arg) __atomic_store_n(&thread_id_ready, 1, __ATOMIC_RELEASE); while (__atomic_load_n(&thread_id_ready, __ATOMIC_ACQUIRE) == 1) - ; + rte_delay_us_sleep(1000); /* required for RT priority */ return 0; } @@ -97,21 +98,12 @@ test_thread_priority(void) "Priority set mismatches priority get"); priority = RTE_THREAD_PRIORITY_REALTIME_CRITICAL; -#ifndef RTE_EXEC_ENV_WINDOWS - RTE_TEST_ASSERT(rte_thread_set_priority(thread_id, priority) == ENOTSUP, - "Priority set to critical should fail"); - RTE_TEST_ASSERT(rte_thread_get_priority(thread_id, &priority) == 0, - "Failed to get thread priority"); - RTE_TEST_ASSERT(priority == RTE_THREAD_PRIORITY_NORMAL, - "Failed set to critical should have retained normal"); -#else RTE_TEST_ASSERT(rte_thread_set_priority(thread_id, priority) == 0, "Priority set to critical should succeed"); RTE_TEST_ASSERT(rte_thread_get_priority(thread_id, &priority) == 0, "Failed to get thread priority"); RTE_TEST_ASSERT(priority == RTE_THREAD_PRIORITY_REALTIME_CRITICAL, "Priority set mismatches priority get"); -#endif priority = RTE_THREAD_PRIORITY_NORMAL; RTE_TEST_ASSERT(rte_thread_set_priority(thread_id, priority) == 0, diff --git a/doc/guides/prog_guide/env_abstraction_layer.rst b/doc/guides/prog_guide/env_abstraction_layer.rst index 6debf54efb..d1f7cae7cd 100644 --- a/doc/guides/prog_guide/env_abstraction_layer.rst +++ b/doc/guides/prog_guide/env_abstraction_layer.rst @@ -815,7 +815,9 @@ Known Issues 4. It MAY be used by preemptible multi-producer and/or preemptible multi-consumer pthreads whose scheduling policy are all SCHED_OTHER(cfs), SCHED_IDLE or SCHED_BATCH. User SHOULD be aware of the performance penalty before using it. - 5. It MUST not be used by multi-producer/consumer pthreads, whose scheduling policies are SCHED_FIFO or SCHED_RR. + 5. It MUST not be used by multi-producer/consumer pthreads + whose scheduling policies are ``SCHED_FIFO`` + or ``SCHED_RR`` (``RTE_THREAD_PRIORITY_REALTIME_CRITICAL``). Alternatively, applications can use the lock-free stack mempool handler. When considering this handler, note that: diff --git a/lib/eal/include/rte_thread.h b/lib/eal/include/rte_thread.h index f2581fe152..7ff031e1b2 100644 --- a/lib/eal/include/rte_thread.h +++ b/lib/eal/include/rte_thread.h @@ -56,10 +56,14 @@ typedef uint32_t (*rte_thread_func) (void *arg); * Thread priority values. */ enum rte_thread_priority { + /** Normal thread priority, the default. */ RTE_THREAD_PRIORITY_NORMAL = 0, - /**< normal thread priority, the default */ + /** + * Highest thread priority, use with caution. + * WARNING: System may be unstable because of a real-time busy loop. + * @see rte_thread_yield_realtime(). + */ RTE_THREAD_PRIORITY_REALTIME_CRITICAL = 1, - /**< highest thread priority allowed */ }; /** diff --git a/lib/eal/unix/rte_thread.c b/lib/eal/unix/rte_thread.c index 278d8d342d..17ffb86c17 100644 --- a/lib/eal/unix/rte_thread.c +++ b/lib/eal/unix/rte_thread.c @@ -33,6 +33,8 @@ static int thread_map_priority_to_os_value(enum rte_thread_priority eal_pri, int *os_pri, int *pol) { + static bool warned; + /* Clear the output parameters. */ *os_pri = sched_get_priority_min(SCHED_OTHER) - 1; *pol = -1; @@ -51,6 +53,17 @@ thread_map_priority_to_os_value(enum rte_thread_priority eal_pri, int *os_pri, sched_get_priority_max(SCHED_OTHER)) / 2; break; case RTE_THREAD_PRIORITY_REALTIME_CRITICAL: + /* + * WARNING: Real-time busy loop takes priority on kernel threads, + * making the system unstable. + * There is also a known issue when using rte_ring. + */ + if (!warned) { + RTE_LOG(NOTICE, EAL, + "Real-time thread is unstable if polling without sleep.\n"); + warned = true; + } + *pol = SCHED_RR; *os_pri = sched_get_priority_max(SCHED_RR); break; @@ -155,11 +168,6 @@ rte_thread_create(rte_thread_t *thread_id, goto cleanup; } - if (thread_attr->priority == - RTE_THREAD_PRIORITY_REALTIME_CRITICAL) { - ret = ENOTSUP; - goto cleanup; - } ret = thread_map_priority_to_os_value(thread_attr->priority, ¶m.sched_priority, &policy); if (ret != 0) @@ -291,10 +299,6 @@ rte_thread_set_priority(rte_thread_t thread_id, int policy; int ret; - /* Realtime priority can cause crashes on non-Windows platforms. */ - if (priority == RTE_THREAD_PRIORITY_REALTIME_CRITICAL) - return ENOTSUP; - ret = thread_map_priority_to_os_value(priority, ¶m.sched_priority, &policy); if (ret != 0) -- 2.42.0