CPU affinity of control threads

CPU affinity of control threads

[Morten Brørup]

Bruce,

I found another gap in the documentation of CPU affinities.

Let's assume a 6-core system.

If a DPDK application is run using:
taskset --cpu-list 0-2 dpdk-app --lcores 3-5

Lcores 3-5 are EAL cores, and each of the three lcores is bound to one cpu, respectively cpu 3, 4 and 5?
Lcore 3 is the main lcore, and lcores 4-5 are worker lcores?
And the original thread of the process (a.k.a. the main lcore) gets its cpuset is changed from 0-2 to 3?

Now, calling rte_thread_create_control() starts a control thread.
Its documentation says:
"Creates a control thread with the given name and attributes. The
affinity of the new thread is based on the CPU affinity retrieved
at the time rte_eal_init() was called, the EAL threads are then
excluded. If setting the name of the thread fails, the error is
ignored and a debug message is logged."

What does that mean?
I.e., what is the affinity of that control thread?

A) Cpu 0-2, controlled by taskset, or
B) cpu 3-5, controlled by the EAL --lcores parameter?


Now, what if the application was run with:
taskset --cpu-list 0-5 dpdk-app --lcores 0-5

Since all cpus are now EAL threads, what happens with rte_thread_create_control() when "the EAL treads are then excluded", and there are no cpus left?


-Morten

Re: CPU affinity of control threads

[Bruce Richardson]

On Thu, May 15, 2025 at 05:22:00PM +0200, Morten Brørup wrote:
> Bruce,
> 
> I found another gap in the documentation of CPU affinities.
> 
> Let's assume a 6-core system.
> 
> If a DPDK application is run using:
> taskset --cpu-list 0-2 dpdk-app --lcores 3-5
> 
> Lcores 3-5 are EAL cores, and each of the three lcores is bound to one cpu, respectively cpu 3, 4 and 5?
> Lcore 3 is the main lcore, and lcores 4-5 are worker lcores?
> And the original thread of the process (a.k.a. the main lcore) gets its cpuset is changed from 0-2 to 3?
> 
> Now, calling rte_thread_create_control() starts a control thread.
> Its documentation says:
> "Creates a control thread with the given name and attributes. The
> affinity of the new thread is based on the CPU affinity retrieved
> at the time rte_eal_init() was called, the EAL threads are then
> excluded. If setting the name of the thread fails, the error is
> ignored and a debug message is logged."
> 
> What does that mean?
> I.e., what is the affinity of that control thread?
> 
> A) Cpu 0-2, controlled by taskset, or
> B) cpu 3-5, controlled by the EAL --lcores parameter?
> 
> 
> Now, what if the application was run with:
> taskset --cpu-list 0-5 dpdk-app --lcores 0-5
> 
> Since all cpus are now EAL threads, what happens with rte_thread_create_control() when "the EAL treads are then excluded", and there are no cpus left?
> 
No idea! :-)

This is one of those scenarios where we need to a) test to see what current
behaviour is, and b) try and work out what the correct (or best) behaviour
should be.

/Bruce