Re: EAL threads

["Mattias Rönnblom" <hofors@lysator.liu.se>]

On 2022-09-21 11:05, Bruce Richardson wrote:
> On Wed, Sep 21, 2022 at 10:39:15AM +0200, Mattias Rönnblom wrote:
>> I have some lcore-related questions:
>>
>> The documentation make use of the term "non-EAL thread". Is a non-EAL thread
>> the same thing as a non-lcore thread? I.e., are there EAL threads that are
>> not lcore threads?
> 
> Yes, there are some threads created by EAL which are not lcore threads.
> These are generally for background tasks, such as interrupts or telemetry,
> and are given a coremask to be kept away from the dataplane lcore threads.
> Therefore, I think you are right to try and get clear terminology for the
> threads. In most cases, I think non-EAL thread is referring to threads
> created by the app itself, but it may in some cases refer to the 'non-lcore
> threads' as you call them.
> 
> Personally, I would suggest terms like:
> * dataplane threads
> * background eal threads
> * app threads
> 
> for clarity, since I think the term EAL thread is ambiguous. However,
> you or others might have been ideas for terms.
> 

It's confusing to have something called an EAL thread to have a lcore id 
and generally being referred to as "a lcore", when it is in fact not 
necessarily tied to a logical core (in the generic hardware sense of the 
word).

What's important to various DPDK code (PMDs, libraries) is
a) If the calling thread has a lcore id, and thus can be trusted to have 
its own, per-lcore data structure.
b) Whether or not the thread may be (involuntarily) preempted, in 
particular by other threads which it might be 
communicating/synchronizing-with.

As of now, an EAL thread qualifies for a), and usually, but not always, 
for b) (not in a strict sense, but it's rarely interrupted and if so, 
only by threads or ISRs outside the DPDK process).

A lot of things in DPDK assumes b) from EAL threads. Any library or PMD 
that uses spinlocks, the default rings, etc. There might be more subtle 
issues as well, I'm guessing. Just take the new busyness-patch set. If a 
thread which is in a busy state it's being interrupted, it will be 
considered busy even when it's not running, but some other thread is. If 
this other thread is also an EAL thread, that does work and is 
considered busy, the telemetry data might be wildly inaccurate and even 
suggest that the system spends more CPU cycles than are actually 
available. DSW port load measurement, using the same scheme, will also 
fall into this trap, when being used by preemptable EAL threads.

Thanks for your answer.

>>
>> I also have a question related to rte_register_thread(): shouldn't the
>> documentation say the user is assumed to pin the calling thread to some
>> core? That is the expectation, correct?
> 
> Probably, but it's not mandatory.