All non-dpdk application threads assigned to the same core after calling rte_eal_init()

All non-dpdk application threads assigned to the same core after calling rte_eal_init()

[Daniel May]

[-- Attachment #1: Type: text/plain, Size: 1023 bytes --]

Hello,

I have a C++ application which uses DPDK to receive packets over two 100
Gbps interfaces from an FPGA device.

I've isolated cores 0-2. DPDK is using cores 0-2, main lcore on 0, and two
receive threads on cores 1 and 2.  The receive threads read packets from
mbufs and move the packets to a couple of circular buffers.  This works as
expected.

The application starts another set of threads (worker threads) to read from
the circular buffers and handle the data. However, these threads are all
being assigned to core 0, instead of cores 3-n.

If I start the worker threads first, before calling rte_eal_init(), they
get assigned to non isolated cores as expected.  However, any thread I
start from the main thread after calling rte_eal_init() gets assigned to
the main lcore core (0).  I can set affinity manually, but I'd rather the
kernel scheduler do its thing.

Is there something I need to do to hand control back to the kernel
scheduler for assigning threads to cores after initializing DPDK?

Thanks,
Daniel

[-- Attachment #2: Type: text/html, Size: 1219 bytes --]

Re: All non-dpdk application threads assigned to the same core after calling rte_eal_init()

[Stephen Hemminger]

On Mon, 28 Jul 2025 09:57:28 -0500
Daniel May <danielmay83@gmail.com> wrote:

> Hello,
> 
> I have a C++ application which uses DPDK to receive packets over two 100
> Gbps interfaces from an FPGA device.
> 
> I've isolated cores 0-2. DPDK is using cores 0-2, main lcore on 0, and two
> receive threads on cores 1 and 2.  The receive threads read packets from
> mbufs and move the packets to a couple of circular buffers.  This works as
> expected.
> 
> The application starts another set of threads (worker threads) to read from
> the circular buffers and handle the data. However, these threads are all
> being assigned to core 0, instead of cores 3-n.

If you are creating the threads from another thread, the new thread inherits
the affinity mask of the parent. In you example, it looks like you are spawning
the threads from the main lcore (0).

> If I start the worker threads first, before calling rte_eal_init(), they
> get assigned to non isolated cores as expected.  However, any thread I
> start from the main thread after calling rte_eal_init() gets assigned to
> the main lcore core (0).  I can set affinity manually, but I'd rather the
> kernel scheduler do its thing.
> 
> Is there something I need to do to hand control back to the kernel
> scheduler for assigning threads to cores after initializing DPDK?

The way to fix is to call set affinity to tell kernel what you want.