[dpdk-users] DPDK and isolcpus

[dpdk-users] DPDK and isolcpus

[Filip Janiszewski]

Hi,

I've configured a bunch of my box cores with isolcpus and nohz_full in
the attempt to squeeze a little more performance out of them, but,
apparently I can't use those core in DPDK anymore as it seems that
rte_lcore_count reports only core which are not isolated, also I can't
launch any thread (rte_eal_remote_launch) on those cores.

In my understanding
(http://doc.dpdk.org/spp-18.02/setup/performance_opt.html) I should be
able to "Use the isolcpus Linux kernel parameter to isolate them from
Linux scheduler to reduce context switches. It prevents workloads of
other processes than DPDK running on reserved cores with isolcpus
parameter." but can't make it work.

Which is the correct way to bind DPDK threads to isolated cpus?

Thanks

-- 
BR, Filip
+48 666 369 823

Re: [dpdk-users] DPDK and isolcpus

[曾懷恩]

Hi Filip,

Do you use Mellanox NIC?

In my experience, rte_lcore_count only counts those lcores that are not isolated when using mlx5 driver.

And Intel ixgbe counts lcores correctly even I use isolcpus flag

BR, 

Sent from my iPhone

> From: Filip Janiszewski <contact@filipjaniszewski.com>
> To: "users@dpdk.org" <users@dpdk.org>
> Subject: [dpdk-users] DPDK and isolcpus
> Message-ID:
>    <e98abf2a-b1cf-5fee-7929-778f70390d1d@filipjaniszewski.com>
> Content-Type: text/plain; charset=iso-8859-15
> 
> Hi,
> 
> I've configured a bunch of my box cores with isolcpus and nohz_full in
> the attempt to squeeze a little more performance out of them, but,
> apparently I can't use those core in DPDK anymore as it seems that
> rte_lcore_count reports only core which are not isolated, also I can't
> launch any thread (rte_eal_remote_launch) on those cores.
> 
> In my understanding
> (http://doc.dpdk.org/spp-18.02/setup/performance_opt.html) I should be
> able to "Use the isolcpus Linux kernel parameter to isolate them from
> Linux scheduler to reduce context switches. It prevents workloads of
> other processes than DPDK running on reserved cores with isolcpus
> parameter." but can't make it work.
> 
> Which is the correct way to bind DPDK threads to isolated cpus?
> 
> Thanks
> 
> -- 
> BR, Filip
> +48 666 369 823
> 
> 
> End of users Digest, Vol 196, Issue 5
> *************************************

Re: [dpdk-users] DPDK and isolcpus

[Stephen Hemminger]

On Fri, 26 Jul 2019 22:50:47 +0800
曾懷恩 <the@csie.io> wrote:

> Hi Filip,
> 
> Do you use Mellanox NIC?
> 
> In my experience, rte_lcore_count only counts those lcores that are not isolated when using mlx5 driver.
> 
> And Intel ixgbe counts lcores correctly even I use isolcpus flag
> 
> BR, 
> 
> Sent from my iPhone
> 
> > From: Filip Janiszewski <contact@filipjaniszewski.com>
> > To: "users@dpdk.org" <users@dpdk.org>
> > Subject: [dpdk-users] DPDK and isolcpus
> > Message-ID:
> >    <e98abf2a-b1cf-5fee-7929-778f70390d1d@filipjaniszewski.com>
> > Content-Type: text/plain; charset=iso-8859-15
> > 
> > Hi,
> > 
> > I've configured a bunch of my box cores with isolcpus and nohz_full in
> > the attempt to squeeze a little more performance out of them, but,
> > apparently I can't use those core in DPDK anymore as it seems that
> > rte_lcore_count reports only core which are not isolated, also I can't
> > launch any thread (rte_eal_remote_launch) on those cores.
> > 
> > In my understanding
> > (http://doc.dpdk.org/spp-18.02/setup/performance_opt.html) I should be
> > able to "Use the isolcpus Linux kernel parameter to isolate them from
> > Linux scheduler to reduce context switches. It prevents workloads of
> > other processes than DPDK running on reserved cores with isolcpus
> > parameter." but can't make it work.
> > 
> > Which is the correct way to bind DPDK threads to isolated cpus?
> > 
> > Thanks
> > 
> > -- 
> > BR, Filip
> > +48 666 369 823
> > 
> > 
> > End of users Digest, Vol 196, Issue 5
> > *************************************  

The default behavior of DPDK is to use all CPU's it finds.
To use isolated CPU's you need to pass the cpu's to use on the command line.

dpdk-XXX -l 4-7 ...

[dpdk-users] DPDK and isolcpus

[曾懷恩]

Hi Filip,

Do you use Mellanox NIC?

In my experience, rte_lcore_count only counts those lcores that are not isolated when using mlx5 driver.

And Intel ixgbe counts lcores correctly even I use isolcpus flag

BR, 

Sent from my iPhone

> From: Filip Janiszewski <contact@filipjaniszewski.com>
> To: "users@dpdk.org" <users@dpdk.org>
> Subject: [dpdk-users] DPDK and isolcpus
> Message-ID:
>   <e98abf2a-b1cf-5fee-7929-778f70390d1d@filipjaniszewski.com>
> Content-Type: text/plain; charset=iso-8859-15
> 
> Hi,
> 
> I've configured a bunch of my box cores with isolcpus and nohz_full in
> the attempt to squeeze a little more performance out of them, but,
> apparently I can't use those core in DPDK anymore as it seems that
> rte_lcore_count reports only core which are not isolated, also I can't
> launch any thread (rte_eal_remote_launch) on those cores.
> 
> In my understanding
> (http://doc.dpdk.org/spp-18.02/setup/performance_opt.html) I should be
> able to "Use the isolcpus Linux kernel parameter to isolate them from
> Linux scheduler to reduce context switches. It prevents workloads of
> other processes than DPDK running on reserved cores with isolcpus
> parameter." but can't make it work.
> 
> Which is the correct way to bind DPDK threads to isolated cpus?
> 
> Thanks
> 
> -- 
> BR, Filip
> +48 666 369 823
> 
> 
> End of users Digest, Vol 196, Issue 5
> *************************************