DPDK patches and discussions
 help / color / mirror / Atom feed
From: Bruce Richardson <bruce.richardson@intel.com>
To: "Morten Brørup" <mb@smartsharesystems.com>
Cc: Thomas Monjalon <thomas@monjalon.net>,
	David Marchand <david.marchand@redhat.com>, <dev@dpdk.org>,
	<stephen@networkplumber.org>
Subject: Re: [PATCH v3 0/2] allow creating thread with real-time priority
Date: Thu, 26 Oct 2023 16:54:04 +0100	[thread overview]
Message-ID: <ZTqLnPs8o1ybKpFZ@bricha3-MOBL.ger.corp.intel.com> (raw)
In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC35E9EF9D@smartserver.smartshare.dk>

On Thu, Oct 26, 2023 at 04:59:51PM +0200, Morten Brørup wrote:
> > From: Morten Brørup [mailto:mb@smartsharesystems.com]
> > Sent: Thursday, 26 October 2023 16.50
> > 
> > > From: Thomas Monjalon [mailto:thomas@monjalon.net]
> > > Sent: Thursday, 26 October 2023 16.31
> > >
> > > 26/10/2023 16:08, Morten Brørup:
> > > > > From: Thomas Monjalon [mailto:thomas@monjalon.net]
> > > > > Sent: Thursday, 26 October 2023 16.05
> > > > >
> > > > > 26/10/2023 15:57, Morten Brørup:
> > > > > > > From: Morten Brørup [mailto:mb@smartsharesystems.com]
> > > > > > > Sent: Thursday, 26 October 2023 15.45
> > > > > > >
> > > > > > > > From: Thomas Monjalon [mailto:thomas@monjalon.net]
> > > > > > > > Sent: Thursday, 26 October 2023 15.37
> > > > > > > >
> > > > > > > > 25/10/2023 18:31, Thomas Monjalon:
> > > > > > > > > Real-time thread priority was been forbidden on Unix
> > > > > > > > > because of problems they can cause.
> > > > > > > > > Warnings and helpers are added to avoid deadlocks,
> > > > > > > > > so real-time can be allowed on all systems.
> > > > > > > >
> > > > > > > > Unit test is failing:
> > > > > > > > DPDK:fast-tests / threads_autotest      TIMEOUT 600.01 s
> > > > > > > >
> > > > > > > > It is seen in only 1 target (maybe the failure occurence is
> > random):
> > > > > > > >   Debian 11 (Buster) (ARM) | PASS
> > > > > > > >   Fedora 37 (ARM)          | PASS
> > > > > > > >   CentOS Stream 9 (ARM)    | FAIL
> > > > > > > >   Fedora 38 (ARM)          | PASS
> > > > > > > >   Fedora 38 (ARM Clang)    | PASS
> > > > > > > >   Ubuntu 20.04 (ARM)       | PASS
> > > > > > > >
> > > > > > > > I need to send a v4 with new implementation and better comments.
> > > > > > > > The Unix sleep will be upgraded from 1 ns to 1 us in case it makes
> > a
> > > > > > > > difference.
> > > > > > >
> > > > > > > It will not make a difference. The kernel will go through the
> > sleeping
> > > > > steps,
> > > > > > > then wake up again and see the real-time thread is ready to run, and
> > > then
> > > > > > > immediately schedule it.
> > > > > > >
> > > > > > > For testing purposes, consider sleeping 10 milliseconds or something
> > > > > > > significant like that.
> > > > > >
> > > > > > A bit more details...
> > > > > >
> > > > > > In our recent tests, nanosleep() itself took around 50 us. So you need
> > > to
> > > > > sleep longer than that for your thread not to be runnable when the
> > > nanosleep()
> > > > > wakes up again, because 50 us has already passed in "nanosleep
> > overhead".
> > > > > > 10 milliseconds provides plenty of margin, and corresponds to 10
> > jiffies
> > > on
> > > > > a 1000 Hz kernel. (I don't know if it makes any difference for the
> > kernel
> > > > > scheduler if the timer crosses a jiffy border or not.)
> > > > >
> > > > > 10 ms looks like an eternity.
> > > >
> > > > Agree. It is only for functional testing, not for production!
> > >
> > > Realtime thread won't make any sense if we have to insert a long sleep.
> > 
> > It seems David came to our rescue here!
> > 
> > I have just tried running our test again with prctl(PR_SET_TIMERSLACK) of 1
> > ns, and the nanosleep(1 ns) delay dropped from ca. 50 us to ca. 2.5 us.
> > 
> > The timeout parameter to epoll_wait() is in milliseconds, which is useless for
> > low-latency.
> > Perhaps real-time threads can be used with epoll() combined with timerfd for
> > nanosecond resolution timeout.
> 
> Or epoll_pwait2(), which has nanosecond resolution timeout.
> 
> Unfortunately, rte_epoll_wait() is not an experimental API anymore, so we cannot change its timeout parameter from milliseconds to micro- or nanoseconds. We would have to introduce a new API for this.
> 

Just an idea - can we change the timeout parameter to float rather than int,
and then use function versioning for backward compatibility for any
binaries passing int?
That way the actual meaning of the parameter doesn't change, but it still
allows sub-millisecond values (all-be-it with some loss of accuracy due to
float).

/Bruce


  reply	other threads:[~2023-10-26 15:54 UTC|newest]

Thread overview: 54+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-10-24 12:54 [PATCH] eal/unix: " Thomas Monjalon
2023-10-24 13:55 ` Morten Brørup
2023-10-24 16:04   ` Stephen Hemminger
2023-10-25 13:15     ` Thomas Monjalon
2023-10-25 13:34       ` Bruce Richardson
2023-10-25 13:44         ` Thomas Monjalon
2023-10-25 15:08           ` Stephen Hemminger
2023-10-25 15:14             ` Bruce Richardson
2023-10-25 15:18               ` Thomas Monjalon
2023-10-25 15:32                 ` Thomas Monjalon
2023-10-25 15:13 ` [PATCH v2] " Thomas Monjalon
2023-10-25 15:37   ` Stephen Hemminger
2023-10-25 16:46     ` Thomas Monjalon
2023-10-25 17:54       ` Morten Brørup
2023-10-25 21:33         ` Stephen Hemminger
2023-10-26  7:33           ` Morten Brørup
2023-10-26 16:32             ` Stephen Hemminger
2023-10-26 17:07               ` Morten Brørup
2023-10-26  0:00         ` Stephen Hemminger
2023-10-25 16:31 ` [PATCH v3 0/2] " Thomas Monjalon
2023-10-25 16:31   ` [PATCH v3 1/2] eal: add thread yield functions Thomas Monjalon
2023-10-25 16:40     ` Bruce Richardson
2023-10-25 17:06       ` Thomas Monjalon
2023-10-25 18:07     ` Morten Brørup
2023-10-25 16:31   ` [PATCH v3 2/2] eal/unix: allow creating thread with real-time priority Thomas Monjalon
2023-10-26 13:36   ` [PATCH v3 0/2] " Thomas Monjalon
2023-10-26 13:44     ` Morten Brørup
2023-10-26 13:57       ` Morten Brørup
2023-10-26 14:04         ` Thomas Monjalon
2023-10-26 14:08           ` Morten Brørup
2023-10-26 14:30             ` Thomas Monjalon
2023-10-26 14:50               ` Morten Brørup
2023-10-26 14:59                 ` Morten Brørup
2023-10-26 15:54                   ` Bruce Richardson [this message]
2023-10-26 16:07                     ` Thomas Monjalon
2023-10-26 16:50                       ` Morten Brørup
2023-10-26 19:51                         ` Thomas Monjalon
2023-10-27  7:19                           ` Morten Brørup
2023-10-26 16:28             ` Stephen Hemminger
2023-10-26 19:55               ` Thomas Monjalon
2023-10-26 21:10                 ` Stephen Hemminger
2023-10-26 14:10         ` David Marchand
2023-10-26 13:37 ` [PATCH v4 " Thomas Monjalon
2023-10-26 13:37   ` [PATCH v4 1/2] eal: add thread yield functions Thomas Monjalon
2023-10-26 13:37   ` [PATCH v4 2/2] eal/unix: allow creating thread with real-time priority Thomas Monjalon
2023-10-26 14:19 ` [PATCH v5 0/2] " Thomas Monjalon
2023-10-26 14:19   ` [PATCH v5 1/2] eal: add thread yield functions Thomas Monjalon
2023-10-26 14:19   ` [PATCH v5 2/2] eal/unix: allow creating thread with real-time priority Thomas Monjalon
2023-10-26 20:00   ` [PATCH v5 0/2] " Thomas Monjalon
2023-10-26 23:35     ` Tyler Retzlaff
2023-10-27  8:08 ` [PATCH v6 1/1] eal/unix: " Thomas Monjalon
2023-10-27  8:45   ` Morten Brørup
2023-10-27  9:11     ` Thomas Monjalon
2023-10-27 18:15     ` Stephen Hemminger

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=ZTqLnPs8o1ybKpFZ@bricha3-MOBL.ger.corp.intel.com \
    --to=bruce.richardson@intel.com \
    --cc=david.marchand@redhat.com \
    --cc=dev@dpdk.org \
    --cc=mb@smartsharesystems.com \
    --cc=stephen@networkplumber.org \
    --cc=thomas@monjalon.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).