From: Thomas Monjalon <thomas@monjalon.net>
To: "Morten Brørup" <mb@smartsharesystems.com>,
"Bruce Richardson" <bruce.richardson@intel.com>
Cc: David Marchand <david.marchand@redhat.com>,
dev@dpdk.org, stephen@networkplumber.org
Subject: Re: [PATCH v3 0/2] allow creating thread with real-time priority
Date: Thu, 26 Oct 2023 18:07:06 +0200 [thread overview]
Message-ID: <2651241.BddDVKsqQX@thomas> (raw)
In-Reply-To: <ZTqLnPs8o1ybKpFZ@bricha3-MOBL.ger.corp.intel.com>
26/10/2023 17:54, Bruce Richardson:
> On Thu, Oct 26, 2023 at 04:59:51PM +0200, Morten Brørup wrote:
> > > From: Morten Brørup [mailto:mb@smartsharesystems.com]
> > > Sent: Thursday, 26 October 2023 16.50
> > >
> > > > From: Thomas Monjalon [mailto:thomas@monjalon.net]
> > > > Sent: Thursday, 26 October 2023 16.31
> > > >
> > > > 26/10/2023 16:08, Morten Brørup:
> > > > > > From: Thomas Monjalon [mailto:thomas@monjalon.net]
> > > > > > Sent: Thursday, 26 October 2023 16.05
> > > > > >
> > > > > > 26/10/2023 15:57, Morten Brørup:
> > > > > > > > From: Morten Brørup [mailto:mb@smartsharesystems.com]
> > > > > > > > Sent: Thursday, 26 October 2023 15.45
> > > > > > > >
> > > > > > > > > From: Thomas Monjalon [mailto:thomas@monjalon.net]
> > > > > > > > > Sent: Thursday, 26 October 2023 15.37
> > > > > > > > >
> > > > > > > > > 25/10/2023 18:31, Thomas Monjalon:
> > > > > > > > > > Real-time thread priority was been forbidden on Unix
> > > > > > > > > > because of problems they can cause.
> > > > > > > > > > Warnings and helpers are added to avoid deadlocks,
> > > > > > > > > > so real-time can be allowed on all systems.
> > > > > > > > >
> > > > > > > > > Unit test is failing:
> > > > > > > > > DPDK:fast-tests / threads_autotest TIMEOUT 600.01 s
> > > > > > > > >
> > > > > > > > > It is seen in only 1 target (maybe the failure occurence is
> > > random):
> > > > > > > > > Debian 11 (Buster) (ARM) | PASS
> > > > > > > > > Fedora 37 (ARM) | PASS
> > > > > > > > > CentOS Stream 9 (ARM) | FAIL
> > > > > > > > > Fedora 38 (ARM) | PASS
> > > > > > > > > Fedora 38 (ARM Clang) | PASS
> > > > > > > > > Ubuntu 20.04 (ARM) | PASS
> > > > > > > > >
> > > > > > > > > I need to send a v4 with new implementation and better comments.
> > > > > > > > > The Unix sleep will be upgraded from 1 ns to 1 us in case it makes
> > > a
> > > > > > > > > difference.
> > > > > > > >
> > > > > > > > It will not make a difference. The kernel will go through the
> > > sleeping
> > > > > > steps,
> > > > > > > > then wake up again and see the real-time thread is ready to run, and
> > > > then
> > > > > > > > immediately schedule it.
> > > > > > > >
> > > > > > > > For testing purposes, consider sleeping 10 milliseconds or something
> > > > > > > > significant like that.
> > > > > > >
> > > > > > > A bit more details...
> > > > > > >
> > > > > > > In our recent tests, nanosleep() itself took around 50 us. So you need
> > > > to
> > > > > > sleep longer than that for your thread not to be runnable when the
> > > > nanosleep()
> > > > > > wakes up again, because 50 us has already passed in "nanosleep
> > > overhead".
> > > > > > > 10 milliseconds provides plenty of margin, and corresponds to 10
> > > jiffies
> > > > on
> > > > > > a 1000 Hz kernel. (I don't know if it makes any difference for the
> > > kernel
> > > > > > scheduler if the timer crosses a jiffy border or not.)
> > > > > >
> > > > > > 10 ms looks like an eternity.
> > > > >
> > > > > Agree. It is only for functional testing, not for production!
> > > >
> > > > Realtime thread won't make any sense if we have to insert a long sleep.
> > >
> > > It seems David came to our rescue here!
> > >
> > > I have just tried running our test again with prctl(PR_SET_TIMERSLACK) of 1
> > > ns, and the nanosleep(1 ns) delay dropped from ca. 50 us to ca. 2.5 us.
> > >
> > > The timeout parameter to epoll_wait() is in milliseconds, which is useless for
> > > low-latency.
> > > Perhaps real-time threads can be used with epoll() combined with timerfd for
> > > nanosecond resolution timeout.
> >
> > Or epoll_pwait2(), which has nanosecond resolution timeout.
> >
> > Unfortunately, rte_epoll_wait() is not an experimental API anymore, so we cannot change its timeout parameter from milliseconds to micro- or nanoseconds. We would have to introduce a new API for this.
> >
>
> Just an idea - can we change the timeout parameter to float rather than int,
> and then use function versioning for backward compatibility for any
> binaries passing int?
> That way the actual meaning of the parameter doesn't change, but it still
> allows sub-millisecond values (all-be-it with some loss of accuracy due to
> float).
Sorry I'm not following why you want to use rte_epoll_wait()?
If the realtime thread has some blocking system calls,
no sleep is needed I think.
For average realtime thread, I suggest the API rte_thread_yield_realtime()
which could wait for 1 ms or less by default.
For smaller sleep, we can use PR_SET_TIMERSLACK and rte_delay_us_sleep().
If we provide an API for PR_SET_TIMERSLACK, we could adapt the duration
of rte_thread_yield_realtime() dynamically after calling prctl().
next prev parent reply other threads:[~2023-10-26 16:07 UTC|newest]
Thread overview: 54+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-10-24 12:54 [PATCH] eal/unix: " Thomas Monjalon
2023-10-24 13:55 ` Morten Brørup
2023-10-24 16:04 ` Stephen Hemminger
2023-10-25 13:15 ` Thomas Monjalon
2023-10-25 13:34 ` Bruce Richardson
2023-10-25 13:44 ` Thomas Monjalon
2023-10-25 15:08 ` Stephen Hemminger
2023-10-25 15:14 ` Bruce Richardson
2023-10-25 15:18 ` Thomas Monjalon
2023-10-25 15:32 ` Thomas Monjalon
2023-10-25 15:13 ` [PATCH v2] " Thomas Monjalon
2023-10-25 15:37 ` Stephen Hemminger
2023-10-25 16:46 ` Thomas Monjalon
2023-10-25 17:54 ` Morten Brørup
2023-10-25 21:33 ` Stephen Hemminger
2023-10-26 7:33 ` Morten Brørup
2023-10-26 16:32 ` Stephen Hemminger
2023-10-26 17:07 ` Morten Brørup
2023-10-26 0:00 ` Stephen Hemminger
2023-10-25 16:31 ` [PATCH v3 0/2] " Thomas Monjalon
2023-10-25 16:31 ` [PATCH v3 1/2] eal: add thread yield functions Thomas Monjalon
2023-10-25 16:40 ` Bruce Richardson
2023-10-25 17:06 ` Thomas Monjalon
2023-10-25 18:07 ` Morten Brørup
2023-10-25 16:31 ` [PATCH v3 2/2] eal/unix: allow creating thread with real-time priority Thomas Monjalon
2023-10-26 13:36 ` [PATCH v3 0/2] " Thomas Monjalon
2023-10-26 13:44 ` Morten Brørup
2023-10-26 13:57 ` Morten Brørup
2023-10-26 14:04 ` Thomas Monjalon
2023-10-26 14:08 ` Morten Brørup
2023-10-26 14:30 ` Thomas Monjalon
2023-10-26 14:50 ` Morten Brørup
2023-10-26 14:59 ` Morten Brørup
2023-10-26 15:54 ` Bruce Richardson
2023-10-26 16:07 ` Thomas Monjalon [this message]
2023-10-26 16:50 ` Morten Brørup
2023-10-26 19:51 ` Thomas Monjalon
2023-10-27 7:19 ` Morten Brørup
2023-10-26 16:28 ` Stephen Hemminger
2023-10-26 19:55 ` Thomas Monjalon
2023-10-26 21:10 ` Stephen Hemminger
2023-10-26 14:10 ` David Marchand
2023-10-26 13:37 ` [PATCH v4 " Thomas Monjalon
2023-10-26 13:37 ` [PATCH v4 1/2] eal: add thread yield functions Thomas Monjalon
2023-10-26 13:37 ` [PATCH v4 2/2] eal/unix: allow creating thread with real-time priority Thomas Monjalon
2023-10-26 14:19 ` [PATCH v5 0/2] " Thomas Monjalon
2023-10-26 14:19 ` [PATCH v5 1/2] eal: add thread yield functions Thomas Monjalon
2023-10-26 14:19 ` [PATCH v5 2/2] eal/unix: allow creating thread with real-time priority Thomas Monjalon
2023-10-26 20:00 ` [PATCH v5 0/2] " Thomas Monjalon
2023-10-26 23:35 ` Tyler Retzlaff
2023-10-27 8:08 ` [PATCH v6 1/1] eal/unix: " Thomas Monjalon
2023-10-27 8:45 ` Morten Brørup
2023-10-27 9:11 ` Thomas Monjalon
2023-10-27 18:15 ` Stephen Hemminger
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=2651241.BddDVKsqQX@thomas \
--to=thomas@monjalon.net \
--cc=bruce.richardson@intel.com \
--cc=david.marchand@redhat.com \
--cc=dev@dpdk.org \
--cc=mb@smartsharesystems.com \
--cc=stephen@networkplumber.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).