* quick question about core affinity
@ 2023-04-26 11:37 이재홍
2023-04-26 12:20 ` Lukáš Šišmiš
2023-04-26 12:31 ` Jamie Fargen
0 siblings, 2 replies; 5+ messages in thread
From: 이재홍 @ 2023-04-26 11:37 UTC (permalink / raw)
To: users
[-- Attachment #1: Type: text/plain, Size: 605 bytes --]
Hello, I'm new to DPDK
I've tried to run samples and got a query about core affinity.
As I understand, if a lcore has affinity to a CPU set, it will run only on
the CPU set.
And I thought If I run a dpdk sample with core 0-2, none process can use
the core (0-2). but when I try to run a simple app(not dpdk app) with
taskset command, it runs on 0, 1, 2 cores..
what I want was if I use cores for dpdk apps none other process can access
the cores.. but it seems possible..
I've googled to find out this but I couldn't find anything I wanted.
Is there anyone can explain about this...?
BR.
Jaehong Lee
[-- Attachment #2: Type: text/html, Size: 797 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: quick question about core affinity
2023-04-26 11:37 quick question about core affinity 이재홍
@ 2023-04-26 12:20 ` Lukáš Šišmiš
2023-04-26 12:26 ` 이재홍
2023-04-26 16:27 ` Stephen Hemminger
2023-04-26 12:31 ` Jamie Fargen
1 sibling, 2 replies; 5+ messages in thread
From: Lukáš Šišmiš @ 2023-04-26 12:20 UTC (permalink / raw)
To: users, ljh890322
Hi,
DPDK core affinity runs your application on the selected cores. But that
doesn't stop other applications from running on the same cores.
To get closer to your goal of really isolating the application from
other processes you would need to add isolcpus to your boot parameters.
That instructs the scheduler to not use the mentioned cores. After
booting with this parameter you could run your DPDK application and
scheduler would not schedule any process to the cores that DPDK
application would use.
However, if you run a separate application and with the taskset command
pin it to the cores your DPDK application uses that will still run and
will be in conflict with your DPDK app.
Best regards,
Lukas
On 26. 04. 23 13:37, 이재홍 wrote:
> Hello, I'm new to DPDK
>
> I've tried to run samples and got a query about core affinity.
> As I understand, if a lcore has affinity to a CPU set, it will run
> only on the CPU set.
> And I thought If I run a dpdk sample with core 0-2, none process can
> use the core (0-2). but when I try to run a simple app(not dpdk app)
> with taskset command, it runs on 0, 1, 2 cores..
>
> what I want was if I use cores for dpdk apps none other process can
> access the cores.. but it seems possible..
>
> I've googled to find out this but I couldn't find anything I wanted.
> Is there anyone can explain about this...?
>
>
> BR.
> Jaehong Lee
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: quick question about core affinity
2023-04-26 12:20 ` Lukáš Šišmiš
@ 2023-04-26 12:26 ` 이재홍
2023-04-26 16:27 ` Stephen Hemminger
1 sibling, 0 replies; 5+ messages in thread
From: 이재홍 @ 2023-04-26 12:26 UTC (permalink / raw)
To: Lukáš Šišmiš; +Cc: users
[-- Attachment #1: Type: text/plain, Size: 92 bytes --]
Hello Lukas,
It was exactly what I wanted to know!
Thanks really a lot :)
BR,
Jaehong Lee
[-- Attachment #2: Type: text/html, Size: 184 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: quick question about core affinity
2023-04-26 11:37 quick question about core affinity 이재홍
2023-04-26 12:20 ` Lukáš Šišmiš
@ 2023-04-26 12:31 ` Jamie Fargen
1 sibling, 0 replies; 5+ messages in thread
From: Jamie Fargen @ 2023-04-26 12:31 UTC (permalink / raw)
To: 이재홍; +Cc: users
[-- Attachment #1: Type: text/plain, Size: 2525 bytes --]
Depending on the OS there maybe other ways to accomplish core isolation,
but it is documented in this page, section 7.1.3
<https://doc.dpdk.org/guides-16.04/linux_gsg/nic_perf_intel_platform.html#linux-boot-command-line>
[1].
You need to add the cores to the linux command line,
In the example you have given will need to add isolcpus=0-2 to the
linux command line.
You will likely want to inspect system cores and make sure they are on the
same numa node by ensuring that those cores are on the same 'physical core'
and you can do that by examining /proc/cpuinfo and verifying the cores have
the same 'physical id'.
You can see on this system, core 0 is on numa 0 and core 1, is on numa 1,
this would not be an optimal configuration, and instead it would be better
to use 2,4,6 or 3,5,7. You maybe asking why cores 0,1 were omitted, and
that is because some clock and timing functions of the kernel run on the
first core of the physical cpu.
$ cat /proc/cpuinfo | grep 'processor\|physical'
processor : 0
physical id : 0
address sizes : 46 bits physical, 57 bits virtual
processor : 1
physical id : 1
address sizes : 46 bits physical, 57 bits virtual
processor : 2
physical id : 0
address sizes : 46 bits physical, 57 bits virtual
processor : 3
physical id : 1
Aother tip is you will want the cores to be on the same path as the
datapath nic. You can determine which numa node the network device is
attached to by inspecting the following file for the nic you are using for
the datapath.
# cat /sys/class/net/eno12399/device/numa_node
0
Reference Link(s):
1 -
https://doc.dpdk.org/guides-16.04/linux_gsg/nic_perf_intel_platform.html#linux-boot-command-line
On Wed, Apr 26, 2023 at 7:38 AM 이재홍 <ljh890322@gmail.com> wrote:
> Hello, I'm new to DPDK
>
> I've tried to run samples and got a query about core affinity.
> As I understand, if a lcore has affinity to a CPU set, it will run only on
> the CPU set.
> And I thought If I run a dpdk sample with core 0-2, none process can use
> the core (0-2). but when I try to run a simple app(not dpdk app) with
> taskset command, it runs on 0, 1, 2 cores..
>
> what I want was if I use cores for dpdk apps none other process can access
> the cores.. but it seems possible..
>
> I've googled to find out this but I couldn't find anything I wanted.
> Is there anyone can explain about this...?
>
>
> BR.
> Jaehong Lee
>
--
Jamie Fargen
Senior Engineer
jfargen@redhat.com
813-580-0718
[-- Attachment #2: Type: text/html, Size: 4211 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: quick question about core affinity
2023-04-26 12:20 ` Lukáš Šišmiš
2023-04-26 12:26 ` 이재홍
@ 2023-04-26 16:27 ` Stephen Hemminger
1 sibling, 0 replies; 5+ messages in thread
From: Stephen Hemminger @ 2023-04-26 16:27 UTC (permalink / raw)
To: Lukáš Šišmiš; +Cc: users, ljh890322
On Wed, 26 Apr 2023 14:20:30 +0200
Lukáš Šišmiš <sismis@cesnet.cz> wrote:
> Hi,
>
>
> DPDK core affinity runs your application on the selected cores. But that
> doesn't stop other applications from running on the same cores.
>
> To get closer to your goal of really isolating the application from
> other processes you would need to add isolcpus to your boot parameters.
>
> That instructs the scheduler to not use the mentioned cores. After
> booting with this parameter you could run your DPDK application and
> scheduler would not schedule any process to the cores that DPDK
> application would use.
>
> However, if you run a separate application and with the taskset command
> pin it to the cores your DPDK application uses that will still run and
> will be in conflict with your DPDK app.
>
>
> Best regards,
>
> Lukas
>
> On 26. 04. 23 13:37, 이재홍 wrote:
> > Hello, I'm new to DPDK
> >
> > I've tried to run samples and got a query about core affinity.
> > As I understand, if a lcore has affinity to a CPU set, it will run
> > only on the CPU set.
> > And I thought If I run a dpdk sample with core 0-2, none process can
> > use the core (0-2). but when I try to run a simple app(not dpdk app)
> > with taskset command, it runs on 0, 1, 2 cores..
> >
> > what I want was if I use cores for dpdk apps none other process can
> > access the cores.. but it seems possible..
> >
> > I've googled to find out this but I couldn't find anything I wanted.
> > Is there anyone can explain about this...?
Look up "DPDK core isolation".
More detail in here: https://www.suse.com/c/cpu-isolation-introduction-part-1/
There are multiple ways to do this, the simplest one is to set the kernel
command line so that on boot the scheduler does not use the isolated cores.
The more complex one recommended for production is to use cgroups
and systemd.
You can't isolate CPU 0. It is special and used for system interrupts etc.
In general, don't use CPU 0 for DPDK applications.
There are other performance tuning considerations such as IRQ affinity,
nohz_full and rcu isolation that are also worth looking at.
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2023-04-26 16:27 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-04-26 11:37 quick question about core affinity 이재홍
2023-04-26 12:20 ` Lukáš Šišmiš
2023-04-26 12:26 ` 이재홍
2023-04-26 16:27 ` Stephen Hemminger
2023-04-26 12:31 ` Jamie Fargen
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).