* RE: EAL threads
2022-09-21 9:05 ` Bruce Richardson
@ 2022-09-21 10:36 ` Morten Brørup
2022-09-30 20:49 ` Mattias Rönnblom
1 sibling, 0 replies; 4+ messages in thread
From: Morten Brørup @ 2022-09-21 10:36 UTC (permalink / raw)
To: Bruce Richardson, Mattias Rönnblom; +Cc: dev
> From: Bruce Richardson [mailto:bruce.richardson@intel.com]
> Sent: Wednesday, 21 September 2022 11.06
>
> On Wed, Sep 21, 2022 at 10:39:15AM +0200, Mattias Rönnblom wrote:
> > I have some lcore-related questions:
> >
> > The documentation make use of the term "non-EAL thread". Is a non-EAL
> thread
> > the same thing as a non-lcore thread? I.e., are there EAL threads
> that are
> > not lcore threads?
>
> Yes, there are some threads created by EAL which are not lcore threads.
> These are generally for background tasks, such as interrupts or
> telemetry,
> and are given a coremask to be kept away from the dataplane lcore
> threads.
> Therefore, I think you are right to try and get clear terminology for
> the
> threads. In most cases, I think non-EAL thread is referring to threads
> created by the app itself, but it may in some cases refer to the 'non-
> lcore
> threads' as you call them.
>
> Personally, I would suggest terms like:
> * dataplane threads
> * background eal threads
> * app threads
Assuming that "app threads" are not performing any dataplane packet processing, I think "control plane threads" might be a better name. Consider this: The purpose of a DPDK application is to perform dataplane tasks, so "app threads" might easily be confused with "dataplane threads".
>
> for clarity, since I think the term EAL thread is ambiguous. However,
> you or others might have been ideas for terms.
>
> >
> > I also have a question related to rte_register_thread(): shouldn't
> the
> > documentation say the user is assumed to pin the calling thread to
> some
> > core? That is the expectation, correct?
>
> Probably, but it's not mandatory.
For this terminology, I think both pinning and isolation of lcores should be considered.
Unfortunately, the confusion is much broader...
Most of the DPDK documentation is based on the architecture before the introduction of various mechanisms for cooperative time sharing of lcores, such as Service Cores and Graphs.
This also goes for the names of DPDK types, functions and macros. E.g. RTE_PER_LCORE [1] actually means per thread, regardless which lcore runs the thread (perhaps even multiple lcores may be scheduled from time to time to run the thread). And why aren't there similar RTE_PER_SEVICE and RTE_PER_NODE macros for Services and Graph nodes...
It would be nice settling on an expanded set of well defined terms, and cleaning up the documentation (and type/function/macro names) accordingly.
[1]: https://doc.dpdk.org/api/rte__per__lcore_8h.html
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: EAL threads
2022-09-21 9:05 ` Bruce Richardson
2022-09-21 10:36 ` Morten Brørup
@ 2022-09-30 20:49 ` Mattias Rönnblom
1 sibling, 0 replies; 4+ messages in thread
From: Mattias Rönnblom @ 2022-09-30 20:49 UTC (permalink / raw)
To: Bruce Richardson; +Cc: dev, Mattias Rönnblom
On 2022-09-21 11:05, Bruce Richardson wrote:
> On Wed, Sep 21, 2022 at 10:39:15AM +0200, Mattias Rönnblom wrote:
>> I have some lcore-related questions:
>>
>> The documentation make use of the term "non-EAL thread". Is a non-EAL thread
>> the same thing as a non-lcore thread? I.e., are there EAL threads that are
>> not lcore threads?
>
> Yes, there are some threads created by EAL which are not lcore threads.
> These are generally for background tasks, such as interrupts or telemetry,
> and are given a coremask to be kept away from the dataplane lcore threads.
> Therefore, I think you are right to try and get clear terminology for the
> threads. In most cases, I think non-EAL thread is referring to threads
> created by the app itself, but it may in some cases refer to the 'non-lcore
> threads' as you call them.
>
> Personally, I would suggest terms like:
> * dataplane threads
> * background eal threads
> * app threads
>
> for clarity, since I think the term EAL thread is ambiguous. However,
> you or others might have been ideas for terms.
>
It's confusing to have something called an EAL thread to have a lcore id
and generally being referred to as "a lcore", when it is in fact not
necessarily tied to a logical core (in the generic hardware sense of the
word).
What's important to various DPDK code (PMDs, libraries) is
a) If the calling thread has a lcore id, and thus can be trusted to have
its own, per-lcore data structure.
b) Whether or not the thread may be (involuntarily) preempted, in
particular by other threads which it might be
communicating/synchronizing-with.
As of now, an EAL thread qualifies for a), and usually, but not always,
for b) (not in a strict sense, but it's rarely interrupted and if so,
only by threads or ISRs outside the DPDK process).
A lot of things in DPDK assumes b) from EAL threads. Any library or PMD
that uses spinlocks, the default rings, etc. There might be more subtle
issues as well, I'm guessing. Just take the new busyness-patch set. If a
thread which is in a busy state it's being interrupted, it will be
considered busy even when it's not running, but some other thread is. If
this other thread is also an EAL thread, that does work and is
considered busy, the telemetry data might be wildly inaccurate and even
suggest that the system spends more CPU cycles than are actually
available. DSW port load measurement, using the same scheme, will also
fall into this trap, when being used by preemptable EAL threads.
Thanks for your answer.
>>
>> I also have a question related to rte_register_thread(): shouldn't the
>> documentation say the user is assumed to pin the calling thread to some
>> core? That is the expectation, correct?
>
> Probably, but it's not mandatory.
^ permalink raw reply [flat|nested] 4+ messages in thread