From: Thomas Monjalon <thomas@monjalon.net>
To: dev@dpdk.org
Cc: "David Marchand" <david.marchand@redhat.com>,
stable@dpdk.org, "Morten Brørup" <mb@smartsharesystems.com>,
"Anatoly Burakov" <anatoly.burakov@intel.com>,
"Dmitry Kozlyuk" <dmitry.kozliuk@gmail.com>,
"Narcisa Ana Maria Vasile" <navasile@linux.microsoft.com>,
"Dmitry Malloy" <dmitrym@microsoft.com>,
"Pallavi Kadam" <pallavi.kadam@intel.com>,
"Stephen Hemminger" <stephen@networkplumber.org>,
"Tyler Retzlaff" <roretzla@linux.microsoft.com>,
"Andrew Rybchenko" <andrew.rybchenko@oktetlabs.ru>,
"Konstantin Ananyev" <konstantin.v.ananyev@yandex.ru>
Subject: [PATCH v2] eal/unix: allow creating thread with real-time priority
Date: Wed, 25 Oct 2023 17:13:14 +0200 [thread overview]
Message-ID: <20231025151352.995318-1-thomas@monjalon.net> (raw)
In-Reply-To: <20231024125416.798897-1-thomas@monjalon.net>
When adding an API for creating threads,
the real-time priority has been forbidden on Unix.
There is a known issue with ring behaviour,
but it should not be completely forbidden.
Real-time thread can block some kernel threads on the same core,
making the system unstable.
That's why a pause is added in the test thread.
This pause is a new API function rte_thread_yield(),
compatible with both Unix and Windows.
Fixes: ca04c78b6262 ("eal: get/set thread priority per thread identifier")
Fixes: ce6e911d20f6 ("eal: add thread lifetime API")
Fixes: a7ba40b2b1bf ("drivers: convert to internal control threads")
Cc: stable@dpdk.org
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
---
app/test/test_threads.c | 11 +---------
.../prog_guide/env_abstraction_layer.rst | 4 +++-
lib/eal/include/rte_thread.h | 13 ++++++++++--
lib/eal/unix/rte_thread.c | 21 +++++++++++--------
lib/eal/version.map | 3 +++
lib/eal/windows/rte_thread.c | 6 ++++++
6 files changed, 36 insertions(+), 22 deletions(-)
diff --git a/app/test/test_threads.c b/app/test/test_threads.c
index 4ac3f2671a..9a449ba9c5 100644
--- a/app/test/test_threads.c
+++ b/app/test/test_threads.c
@@ -22,7 +22,7 @@ thread_main(void *arg)
__atomic_store_n(&thread_id_ready, 1, __ATOMIC_RELEASE);
while (__atomic_load_n(&thread_id_ready, __ATOMIC_ACQUIRE) == 1)
- ;
+ rte_thread_yield(); /* required in case of real-time priority */
return 0;
}
@@ -97,21 +97,12 @@ test_thread_priority(void)
"Priority set mismatches priority get");
priority = RTE_THREAD_PRIORITY_REALTIME_CRITICAL;
-#ifndef RTE_EXEC_ENV_WINDOWS
- RTE_TEST_ASSERT(rte_thread_set_priority(thread_id, priority) == ENOTSUP,
- "Priority set to critical should fail");
- RTE_TEST_ASSERT(rte_thread_get_priority(thread_id, &priority) == 0,
- "Failed to get thread priority");
- RTE_TEST_ASSERT(priority == RTE_THREAD_PRIORITY_NORMAL,
- "Failed set to critical should have retained normal");
-#else
RTE_TEST_ASSERT(rte_thread_set_priority(thread_id, priority) == 0,
"Priority set to critical should succeed");
RTE_TEST_ASSERT(rte_thread_get_priority(thread_id, &priority) == 0,
"Failed to get thread priority");
RTE_TEST_ASSERT(priority == RTE_THREAD_PRIORITY_REALTIME_CRITICAL,
"Priority set mismatches priority get");
-#endif
priority = RTE_THREAD_PRIORITY_NORMAL;
RTE_TEST_ASSERT(rte_thread_set_priority(thread_id, priority) == 0,
diff --git a/doc/guides/prog_guide/env_abstraction_layer.rst b/doc/guides/prog_guide/env_abstraction_layer.rst
index 6debf54efb..d1f7cae7cd 100644
--- a/doc/guides/prog_guide/env_abstraction_layer.rst
+++ b/doc/guides/prog_guide/env_abstraction_layer.rst
@@ -815,7 +815,9 @@ Known Issues
4. It MAY be used by preemptible multi-producer and/or preemptible multi-consumer pthreads whose scheduling policy are all SCHED_OTHER(cfs), SCHED_IDLE or SCHED_BATCH. User SHOULD be aware of the performance penalty before using it.
- 5. It MUST not be used by multi-producer/consumer pthreads, whose scheduling policies are SCHED_FIFO or SCHED_RR.
+ 5. It MUST not be used by multi-producer/consumer pthreads
+ whose scheduling policies are ``SCHED_FIFO``
+ or ``SCHED_RR`` (``RTE_THREAD_PRIORITY_REALTIME_CRITICAL``).
Alternatively, applications can use the lock-free stack mempool handler. When
considering this handler, note that:
diff --git a/lib/eal/include/rte_thread.h b/lib/eal/include/rte_thread.h
index 8da9d4d3fb..eeccc40532 100644
--- a/lib/eal/include/rte_thread.h
+++ b/lib/eal/include/rte_thread.h
@@ -56,10 +56,11 @@ typedef uint32_t (*rte_thread_func) (void *arg);
* Thread priority values.
*/
enum rte_thread_priority {
+ /** Normal thread priority, the default. */
RTE_THREAD_PRIORITY_NORMAL = 0,
- /**< normal thread priority, the default */
+ /** Highest thread priority, use with caution.
+ * WARNING: System may be unstable because of a real-time busy loop. */
RTE_THREAD_PRIORITY_REALTIME_CRITICAL = 1,
- /**< highest thread priority allowed */
};
/**
@@ -183,6 +184,14 @@ int rte_thread_join(rte_thread_t thread_id, uint32_t *value_ptr);
*/
int rte_thread_detach(rte_thread_t thread_id);
+/**
+ * Allow another thread to run on the same CPU core.
+ *
+ * Especially useful in real-time thread priority.
+ * @see RTE_THREAD_PRIORITY_REALTIME_CRITICAL
+ */
+void rte_thread_yield(void);
+
/**
* Get the id of the calling thread.
*
diff --git a/lib/eal/unix/rte_thread.c b/lib/eal/unix/rte_thread.c
index 36a21ab2f9..399acf2fa0 100644
--- a/lib/eal/unix/rte_thread.c
+++ b/lib/eal/unix/rte_thread.c
@@ -5,6 +5,7 @@
#include <errno.h>
#include <pthread.h>
+#include <sched.h>
#include <stdbool.h>
#include <stdlib.h>
#include <string.h>
@@ -49,6 +50,11 @@ thread_map_priority_to_os_value(enum rte_thread_priority eal_pri, int *os_pri,
sched_get_priority_max(SCHED_OTHER)) / 2;
break;
case RTE_THREAD_PRIORITY_REALTIME_CRITICAL:
+ /*
+ * WARNING: Real-time busy loop takes priority on kernel threads,
+ * making the system unstable.
+ * There is also a known issue when using rte_ring.
+ */
*pol = SCHED_RR;
*os_pri = sched_get_priority_max(SCHED_RR);
break;
@@ -153,11 +159,6 @@ rte_thread_create(rte_thread_t *thread_id,
goto cleanup;
}
- if (thread_attr->priority ==
- RTE_THREAD_PRIORITY_REALTIME_CRITICAL) {
- ret = ENOTSUP;
- goto cleanup;
- }
ret = thread_map_priority_to_os_value(thread_attr->priority,
¶m.sched_priority, &policy);
if (ret != 0)
@@ -227,6 +228,12 @@ rte_thread_detach(rte_thread_t thread_id)
return pthread_detach((pthread_t)thread_id.opaque_id);
}
+void
+rte_thread_yield(void)
+{
+ sched_yield();
+}
+
int
rte_thread_equal(rte_thread_t t1, rte_thread_t t2)
{
@@ -275,10 +282,6 @@ rte_thread_set_priority(rte_thread_t thread_id,
int policy;
int ret;
- /* Realtime priority can cause crashes on non-Windows platforms. */
- if (priority == RTE_THREAD_PRIORITY_REALTIME_CRITICAL)
- return ENOTSUP;
-
ret = thread_map_priority_to_os_value(priority, ¶m.sched_priority,
&policy);
if (ret != 0)
diff --git a/lib/eal/version.map b/lib/eal/version.map
index e00a844805..0b4a503c5f 100644
--- a/lib/eal/version.map
+++ b/lib/eal/version.map
@@ -413,6 +413,9 @@ EXPERIMENTAL {
# added in 23.07
rte_memzone_max_get;
rte_memzone_max_set;
+
+ # added in 23.11
+ rte_thread_yield;
};
INTERNAL {
diff --git a/lib/eal/windows/rte_thread.c b/lib/eal/windows/rte_thread.c
index acf648456c..b0373b1a55 100644
--- a/lib/eal/windows/rte_thread.c
+++ b/lib/eal/windows/rte_thread.c
@@ -304,6 +304,12 @@ rte_thread_detach(rte_thread_t thread_id)
return 0;
}
+void
+rte_thread_yield(void)
+{
+ Sleep(0);
+}
+
int
rte_thread_equal(rte_thread_t t1, rte_thread_t t2)
{
--
2.42.0
next prev parent reply other threads:[~2023-10-25 15:15 UTC|newest]
Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-10-24 12:54 [PATCH] " Thomas Monjalon
2023-10-24 13:55 ` Morten Brørup
2023-10-24 16:04 ` Stephen Hemminger
2023-10-25 13:15 ` Thomas Monjalon
2023-10-25 13:34 ` Bruce Richardson
2023-10-25 13:44 ` Thomas Monjalon
2023-10-25 15:08 ` Stephen Hemminger
2023-10-25 15:14 ` Bruce Richardson
2023-10-25 15:18 ` Thomas Monjalon
2023-10-25 15:32 ` Thomas Monjalon
2023-10-25 15:13 ` Thomas Monjalon [this message]
2023-10-25 15:37 ` [PATCH v2] " Stephen Hemminger
2023-10-25 16:46 ` Thomas Monjalon
2023-10-25 17:54 ` Morten Brørup
2023-10-25 21:33 ` Stephen Hemminger
2023-10-26 7:33 ` Morten Brørup
2023-10-26 16:32 ` Stephen Hemminger
2023-10-26 17:07 ` Morten Brørup
2023-10-26 0:00 ` Stephen Hemminger
[not found] ` <20231025163352.1076755-1-thomas@monjalon.net>
2023-10-25 16:31 ` [PATCH v3 2/2] " Thomas Monjalon
[not found] ` <20231026134313.1165954-1-thomas@monjalon.net>
2023-10-26 13:37 ` [PATCH v4 " Thomas Monjalon
[not found] ` <20231026142749.1174372-1-thomas@monjalon.net>
2023-10-26 14:19 ` [PATCH v5 " Thomas Monjalon
2023-10-27 8:08 ` [PATCH v6 1/1] " Thomas Monjalon
2023-10-27 8:45 ` Morten Brørup
2023-10-27 9:11 ` Thomas Monjalon
2023-10-27 18:15 ` Stephen Hemminger
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20231025151352.995318-1-thomas@monjalon.net \
--to=thomas@monjalon.net \
--cc=anatoly.burakov@intel.com \
--cc=andrew.rybchenko@oktetlabs.ru \
--cc=david.marchand@redhat.com \
--cc=dev@dpdk.org \
--cc=dmitry.kozliuk@gmail.com \
--cc=dmitrym@microsoft.com \
--cc=konstantin.v.ananyev@yandex.ru \
--cc=mb@smartsharesystems.com \
--cc=navasile@linux.microsoft.com \
--cc=pallavi.kadam@intel.com \
--cc=roretzla@linux.microsoft.com \
--cc=stable@dpdk.org \
--cc=stephen@networkplumber.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).