* [dpdk-dev] DPDK 20.08 meson build testpmd not detecting all NUMA nodes
@ 2020-08-28 9:26 Juraj Linkeš
2020-08-28 10:20 ` Bruce Richardson
0 siblings, 1 reply; 6+ messages in thread
From: Juraj Linkeš @ 2020-08-28 9:26 UTC (permalink / raw)
To: dev
Hello DPDK devs,
I've meson built (using just meson build && ninja -C build) dpdk-20.08 (downloaded from the site) on a Taishan arm server and dpdk-testpmd is incorrectly detecting NUMA nodes on the system:
EAL: Detected 16 lcore(s)
EAL: Detected 1 NUMA nodes
lscpu report 4 NUMA nodes:
NUMA node0 CPU(s): 0-15
NUMA node1 CPU(s): 16-31
NUMA node2 CPU(s): 32-47
NUMA node3 CPU(s): 48-63
When built with make T=arm64-armv8a-linuxapp-gcc testpmd is detecting NUMA nodes correctly:
EAL: Detected 64 lcore(s)
EAL: Detected 4 NUMA nodes
Any idea what's the difference between the builds? I've looked at NUMA config options but those seem ok to me:
grep NUMA arm64-armv8a-linuxapp-gcc/.config build/rte_build_config.h
arm64-armv8a-linuxapp-gcc/.config:CONFIG_RTE_MAX_NUMA_NODES=8
arm64-armv8a-linuxapp-gcc/.config:# a "type" is a combination of page size and NUMA node. total number of memseg
arm64-armv8a-linuxapp-gcc/.config:CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES=y
arm64-armv8a-linuxapp-gcc/.config:CONFIG_RTE_LIBRTE_VHOST_NUMA=y
build/rte_build_config.h:#define RTE_EAL_NUMA_AWARE_HUGEPAGES 1
build/rte_build_config.h:#define RTE_HAS_LIBNUMA 1
build/rte_build_config.h:#define RTE_LIBRTE_VHOST_NUMA 1
build/rte_build_config.h:#define RTE_MAX_NUMA_NODES 4
Any pointers are welcome.
Thanks,
Juraj
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-dev] DPDK 20.08 meson build testpmd not detecting all NUMA nodes
2020-08-28 9:26 [dpdk-dev] DPDK 20.08 meson build testpmd not detecting all NUMA nodes Juraj Linkeš
@ 2020-08-28 10:20 ` Bruce Richardson
2020-08-28 10:29 ` Bruce Richardson
0 siblings, 1 reply; 6+ messages in thread
From: Bruce Richardson @ 2020-08-28 10:20 UTC (permalink / raw)
To: Juraj Linkeš; +Cc: dev
On Fri, Aug 28, 2020 at 09:26:12AM +0000, Juraj Linkeš wrote:
> Hello DPDK devs,
>
> I've meson built (using just meson build && ninja -C build) dpdk-20.08 (downloaded from the site) on a Taishan arm server and dpdk-testpmd is incorrectly detecting NUMA nodes on the system:
> EAL: Detected 16 lcore(s)
> EAL: Detected 1 NUMA nodes
>
> lscpu report 4 NUMA nodes:
> NUMA node0 CPU(s): 0-15
> NUMA node1 CPU(s): 16-31
> NUMA node2 CPU(s): 32-47
> NUMA node3 CPU(s): 48-63
>
> When built with make T=arm64-armv8a-linuxapp-gcc testpmd is detecting NUMA nodes correctly:
> EAL: Detected 64 lcore(s)
> EAL: Detected 4 NUMA nodes
>
> Any idea what's the difference between the builds? I've looked at NUMA config options but those seem ok to me:
> grep NUMA arm64-armv8a-linuxapp-gcc/.config build/rte_build_config.h
> arm64-armv8a-linuxapp-gcc/.config:CONFIG_RTE_MAX_NUMA_NODES=8
> arm64-armv8a-linuxapp-gcc/.config:# a "type" is a combination of page size and NUMA node. total number of memseg
> arm64-armv8a-linuxapp-gcc/.config:CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES=y
> arm64-armv8a-linuxapp-gcc/.config:CONFIG_RTE_LIBRTE_VHOST_NUMA=y
> build/rte_build_config.h:#define RTE_EAL_NUMA_AWARE_HUGEPAGES 1
> build/rte_build_config.h:#define RTE_HAS_LIBNUMA 1
> build/rte_build_config.h:#define RTE_LIBRTE_VHOST_NUMA 1
> build/rte_build_config.h:#define RTE_MAX_NUMA_NODES 4
>
> Any pointers are welcome.
>
What is the max lcores value in your build config? Do you know what
settings are being chosen for your platform in config/arm/meson.build?
Regards,
/Bruce
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-dev] DPDK 20.08 meson build testpmd not detecting all NUMA nodes
2020-08-28 10:20 ` Bruce Richardson
@ 2020-08-28 10:29 ` Bruce Richardson
2020-08-28 11:11 ` Juraj Linkeš
0 siblings, 1 reply; 6+ messages in thread
From: Bruce Richardson @ 2020-08-28 10:29 UTC (permalink / raw)
To: Juraj Linkeš; +Cc: dev
On Fri, Aug 28, 2020 at 11:20:02AM +0100, Bruce Richardson wrote:
> On Fri, Aug 28, 2020 at 09:26:12AM +0000, Juraj Linkeš wrote:
> > Hello DPDK devs,
> >
> > I've meson built (using just meson build && ninja -C build) dpdk-20.08 (downloaded from the site) on a Taishan arm server and dpdk-testpmd is incorrectly detecting NUMA nodes on the system:
> > EAL: Detected 16 lcore(s)
> > EAL: Detected 1 NUMA nodes
> >
> > lscpu report 4 NUMA nodes:
> > NUMA node0 CPU(s): 0-15
> > NUMA node1 CPU(s): 16-31
> > NUMA node2 CPU(s): 32-47
> > NUMA node3 CPU(s): 48-63
> >
> > When built with make T=arm64-armv8a-linuxapp-gcc testpmd is detecting NUMA nodes correctly:
> > EAL: Detected 64 lcore(s)
> > EAL: Detected 4 NUMA nodes
> >
> > Any idea what's the difference between the builds? I've looked at NUMA config options but those seem ok to me:
> > grep NUMA arm64-armv8a-linuxapp-gcc/.config build/rte_build_config.h
> > arm64-armv8a-linuxapp-gcc/.config:CONFIG_RTE_MAX_NUMA_NODES=8
> > arm64-armv8a-linuxapp-gcc/.config:# a "type" is a combination of page size and NUMA node. total number of memseg
> > arm64-armv8a-linuxapp-gcc/.config:CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES=y
> > arm64-armv8a-linuxapp-gcc/.config:CONFIG_RTE_LIBRTE_VHOST_NUMA=y
> > build/rte_build_config.h:#define RTE_EAL_NUMA_AWARE_HUGEPAGES 1
> > build/rte_build_config.h:#define RTE_HAS_LIBNUMA 1
> > build/rte_build_config.h:#define RTE_LIBRTE_VHOST_NUMA 1
> > build/rte_build_config.h:#define RTE_MAX_NUMA_NODES 4
> >
> > Any pointers are welcome.
> >
> What is the max lcores value in your build config? Do you know what
> settings are being chosen for your platform in config/arm/meson.build?
>
By way of follow-up, the reason to look at the cores is that the number of
numa nodes is detected simply by iterating through all the cores discovered
and finding out what NUMA node they are on. If only 16 cores are allowed,
then only one NUMA node will be found because all cores discovered will
report that as their NUMA location.
/Bruce
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-dev] DPDK 20.08 meson build testpmd not detecting all NUMA nodes
2020-08-28 10:29 ` Bruce Richardson
@ 2020-08-28 11:11 ` Juraj Linkeš
2020-08-28 13:10 ` Bruce Richardson
0 siblings, 1 reply; 6+ messages in thread
From: Juraj Linkeš @ 2020-08-28 11:11 UTC (permalink / raw)
To: Bruce Richardson; +Cc: dev
> -----Original Message-----
> From: Bruce Richardson <bruce.richardson@intel.com>
> Sent: Friday, August 28, 2020 12:30 PM
> To: Juraj Linkeš <juraj.linkes@pantheon.tech>
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] DPDK 20.08 meson build testpmd not detecting all
> NUMA nodes
>
> On Fri, Aug 28, 2020 at 11:20:02AM +0100, Bruce Richardson wrote:
> > On Fri, Aug 28, 2020 at 09:26:12AM +0000, Juraj Linkeš wrote:
> > > Hello DPDK devs,
> > >
> > > I've meson built (using just meson build && ninja -C build) dpdk-20.08
> (downloaded from the site) on a Taishan arm server and dpdk-testpmd is
> incorrectly detecting NUMA nodes on the system:
> > > EAL: Detected 16 lcore(s)
> > > EAL: Detected 1 NUMA nodes
> > >
> > > lscpu report 4 NUMA nodes:
> > > NUMA node0 CPU(s): 0-15
> > > NUMA node1 CPU(s): 16-31
> > > NUMA node2 CPU(s): 32-47
> > > NUMA node3 CPU(s): 48-63
> > >
> > > When built with make T=arm64-armv8a-linuxapp-gcc testpmd is detecting
> NUMA nodes correctly:
> > > EAL: Detected 64 lcore(s)
> > > EAL: Detected 4 NUMA nodes
> > >
> > > Any idea what's the difference between the builds? I've looked at NUMA
> config options but those seem ok to me:
> > > grep NUMA arm64-armv8a-linuxapp-gcc/.config build/rte_build_config.h
> > > arm64-armv8a-linuxapp-gcc/.config:CONFIG_RTE_MAX_NUMA_NODES=8
> > > arm64-armv8a-linuxapp-gcc/.config:# a "type" is a combination of
> > > page size and NUMA node. total number of memseg
> > > arm64-armv8a-linuxapp-
> gcc/.config:CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGE
> > > S=y arm64-armv8a-linuxapp-
> gcc/.config:CONFIG_RTE_LIBRTE_VHOST_NUMA=y
> > > build/rte_build_config.h:#define RTE_EAL_NUMA_AWARE_HUGEPAGES 1
> > > build/rte_build_config.h:#define RTE_HAS_LIBNUMA 1
> > > build/rte_build_config.h:#define RTE_LIBRTE_VHOST_NUMA 1
> > > build/rte_build_config.h:#define RTE_MAX_NUMA_NODES 4
> > >
> > > Any pointers are welcome.
> > >
> > What is the max lcores value in your build config? Do you know what
> > settings are being chosen for your platform in config/arm/meson.build?
> >
> By way of follow-up, the reason to look at the cores is that the number of numa
> nodes is detected simply by iterating through all the cores discovered and finding
> out what NUMA node they are on. If only 16 cores are allowed, then only one
> NUMA node will be found because all cores discovered will report that as their
> NUMA location.
>
> /Bruce
Thanks Bruce, RTE_MAX_LCORE is being set only to 16 (line 39 in config/arm/meson.build), so that's the cause. Why is this used in native builds? Shouldn't this be discovered?
I any case, setting it statically for implementer 0x41 seems wrong, as there's a big variety different processors under it: https://github.com/bp0/armids/blob/master/arm.ids
Juraj
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-dev] DPDK 20.08 meson build testpmd not detecting all NUMA nodes
2020-08-28 11:11 ` Juraj Linkeš
@ 2020-08-28 13:10 ` Bruce Richardson
2020-08-28 15:15 ` Ruifeng Wang
0 siblings, 1 reply; 6+ messages in thread
From: Bruce Richardson @ 2020-08-28 13:10 UTC (permalink / raw)
To: Juraj Linkeš; +Cc: dev, yskoh, honnappa.nagarahalli
On Fri, Aug 28, 2020 at 11:11:36AM +0000, Juraj Linkeš wrote:
>
>
> > -----Original Message----- From: Bruce Richardson
> > <bruce.richardson@intel.com> Sent: Friday, August 28, 2020 12:30 PM To:
> > Juraj Linkeš <juraj.linkes@pantheon.tech> Cc: dev@dpdk.org Subject: Re:
> > [dpdk-dev] DPDK 20.08 meson build testpmd not detecting all NUMA nodes
> >
> > On Fri, Aug 28, 2020 at 11:20:02AM +0100, Bruce Richardson wrote:
> > > On Fri, Aug 28, 2020 at 09:26:12AM +0000, Juraj Linkeš wrote:
> > > > Hello DPDK devs,
> > > >
> > > > I've meson built (using just meson build && ninja -C build)
> > > > dpdk-20.08
> > (downloaded from the site) on a Taishan arm server and dpdk-testpmd is
> > incorrectly detecting NUMA nodes on the system:
> > > > EAL: Detected 16 lcore(s) EAL: Detected 1 NUMA nodes
> > > >
> > > > lscpu report 4 NUMA nodes: NUMA node0 CPU(s): 0-15 NUMA node1
> > > > CPU(s): 16-31 NUMA node2 CPU(s): 32-47 NUMA node3 CPU(s):
> > > > 48-63
> > > >
> > > > When built with make T=arm64-armv8a-linuxapp-gcc testpmd is
> > > > detecting
> > NUMA nodes correctly:
> > > > EAL: Detected 64 lcore(s) EAL: Detected 4 NUMA nodes
> > > >
> > > > Any idea what's the difference between the builds? I've looked at
> > > > NUMA
> > config options but those seem ok to me:
> > > > grep NUMA arm64-armv8a-linuxapp-gcc/.config
> > > > build/rte_build_config.h
> > > > arm64-armv8a-linuxapp-gcc/.config:CONFIG_RTE_MAX_NUMA_NODES=8
> > > > arm64-armv8a-linuxapp-gcc/.config:# a "type" is a combination of
> > > > page size and NUMA node. total number of memseg
> > > > arm64-armv8a-linuxapp-
> > gcc/.config:CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGE
> > > > S=y arm64-armv8a-linuxapp-
> > gcc/.config:CONFIG_RTE_LIBRTE_VHOST_NUMA=y
> > > > build/rte_build_config.h:#define RTE_EAL_NUMA_AWARE_HUGEPAGES 1
> > > > build/rte_build_config.h:#define RTE_HAS_LIBNUMA 1
> > > > build/rte_build_config.h:#define RTE_LIBRTE_VHOST_NUMA 1
> > > > build/rte_build_config.h:#define RTE_MAX_NUMA_NODES 4
> > > >
> > > > Any pointers are welcome.
> > > >
> > > What is the max lcores value in your build config? Do you know what
> > > settings are being chosen for your platform in
> > > config/arm/meson.build?
> > >
> > By way of follow-up, the reason to look at the cores is that the number
> > of numa nodes is detected simply by iterating through all the cores
> > discovered and finding out what NUMA node they are on. If only 16 cores
> > are allowed, then only one NUMA node will be found because all cores
> > discovered will report that as their NUMA location.
> >
> > /Bruce
>
> Thanks Bruce, RTE_MAX_LCORE is being set only to 16 (line 39 in
> config/arm/meson.build), so that's the cause. Why is this used in native
> builds? Shouldn't this be discovered?
>
> I any case, setting it statically for implementer 0x41 seems wrong, as
> there's a big variety different processors under it:
> https://github.com/bp0/armids/blob/master/arm.ids
>
Sorry, no idea about this, since I don't generally work on/with the
arm/meson.build file. Looking at git, that particular value seems to come
from commit d97108a33231 ("config: change defaults of armv8"), so perhaps
check with the author and reviewer of that patch. [Adding them on CC]
Regards,
/Bruce
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-dev] DPDK 20.08 meson build testpmd not detecting all NUMA nodes
2020-08-28 13:10 ` Bruce Richardson
@ 2020-08-28 15:15 ` Ruifeng Wang
0 siblings, 0 replies; 6+ messages in thread
From: Ruifeng Wang @ 2020-08-28 15:15 UTC (permalink / raw)
To: Bruce Richardson, Juraj Linkeš
Cc: dev, yskoh, Honnappa Nagarahalli, Dharmik Thakkar, nd
Hi Juraj,
> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Bruce Richardson
> Sent: Friday, August 28, 2020 9:10 PM
> To: Juraj Linkeš <juraj.linkes@pantheon.tech>
> Cc: dev@dpdk.org; yskoh@mellanox.com; Honnappa Nagarahalli
> <Honnappa.Nagarahalli@arm.com>
> Subject: Re: [dpdk-dev] DPDK 20.08 meson build testpmd not detecting all
> NUMA nodes
>
> On Fri, Aug 28, 2020 at 11:11:36AM +0000, Juraj Linkeš wrote:
> >
> >
> > > -----Original Message----- From: Bruce Richardson
> > > <bruce.richardson@intel.com> Sent: Friday, August 28, 2020 12:30 PM To:
> > > Juraj Linkeš <juraj.linkes@pantheon.tech> Cc: dev@dpdk.org Subject: Re:
> > > [dpdk-dev] DPDK 20.08 meson build testpmd not detecting all NUMA
> > > nodes
> > >
> > > On Fri, Aug 28, 2020 at 11:20:02AM +0100, Bruce Richardson wrote:
> > > > On Fri, Aug 28, 2020 at 09:26:12AM +0000, Juraj Linkeš wrote:
> > > > > Hello DPDK devs,
> > > > >
> > > > > I've meson built (using just meson build && ninja -C build)
> > > > > dpdk-20.08
> > > (downloaded from the site) on a Taishan arm server and dpdk-testpmd
> > > is incorrectly detecting NUMA nodes on the system:
> > > > > EAL: Detected 16 lcore(s) EAL: Detected 1 NUMA nodes
> > > > >
> > > > > lscpu report 4 NUMA nodes: NUMA node0 CPU(s): 0-15 NUMA
> node1
> > > > > CPU(s): 16-31 NUMA node2 CPU(s): 32-47 NUMA node3 CPU(s):
> > > > > 48-63
> > > > >
> > > > > When built with make T=arm64-armv8a-linuxapp-gcc testpmd is
> > > > > detecting
> > > NUMA nodes correctly:
> > > > > EAL: Detected 64 lcore(s) EAL: Detected 4 NUMA nodes
> > > > >
> > > > > Any idea what's the difference between the builds? I've looked
> > > > > at NUMA
> > > config options but those seem ok to me:
> > > > > grep NUMA arm64-armv8a-linuxapp-gcc/.config
> > > > > build/rte_build_config.h
> > > > > arm64-armv8a-linuxapp-
> gcc/.config:CONFIG_RTE_MAX_NUMA_NODES=8
> > > > > arm64-armv8a-linuxapp-gcc/.config:# a "type" is a combination of
> > > > > page size and NUMA node. total number of memseg
> > > > > arm64-armv8a-linuxapp-
> > > gcc/.config:CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGE
> > > > > S=y arm64-armv8a-linuxapp-
> > > gcc/.config:CONFIG_RTE_LIBRTE_VHOST_NUMA=y
> > > > > build/rte_build_config.h:#define
> RTE_EAL_NUMA_AWARE_HUGEPAGES 1
> > > > > build/rte_build_config.h:#define RTE_HAS_LIBNUMA 1
> > > > > build/rte_build_config.h:#define RTE_LIBRTE_VHOST_NUMA 1
> > > > > build/rte_build_config.h:#define RTE_MAX_NUMA_NODES 4
> > > > >
> > > > > Any pointers are welcome.
> > > > >
> > > > What is the max lcores value in your build config? Do you know
> > > > what settings are being chosen for your platform in
> > > > config/arm/meson.build?
> > > >
> > > By way of follow-up, the reason to look at the cores is that the
> > > number of numa nodes is detected simply by iterating through all the
> > > cores discovered and finding out what NUMA node they are on. If only
> > > 16 cores are allowed, then only one NUMA node will be found because
> > > all cores discovered will report that as their NUMA location.
> > >
> > > /Bruce
> >
> > Thanks Bruce, RTE_MAX_LCORE is being set only to 16 (line 39 in
> > config/arm/meson.build), so that's the cause. Why is this used in
> > native builds? Shouldn't this be discovered?
There is a patch set from Dharmik that detects lcore count for native builds:
http://patches.dpdk.org/project/dpdk/list/?series=11784
It will be good if you can try and test it.
Thanks.
/Ruifeng
> >
> > I any case, setting it statically for implementer 0x41 seems wrong, as
> > there's a big variety different processors under it:
> > https://github.com/bp0/armids/blob/master/arm.ids
> >
> Sorry, no idea about this, since I don't generally work on/with the
> arm/meson.build file. Looking at git, that particular value seems to come
> from commit d97108a33231 ("config: change defaults of armv8"), so perhaps
> check with the author and reviewer of that patch. [Adding them on CC]
>
> Regards,
> /Bruce
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2020-08-28 15:15 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-08-28 9:26 [dpdk-dev] DPDK 20.08 meson build testpmd not detecting all NUMA nodes Juraj Linkeš
2020-08-28 10:20 ` Bruce Richardson
2020-08-28 10:29 ` Bruce Richardson
2020-08-28 11:11 ` Juraj Linkeš
2020-08-28 13:10 ` Bruce Richardson
2020-08-28 15:15 ` Ruifeng Wang
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).