[PATCH v1] buildtools: ensure the NUMA nodes are counted correct

[PATCH v1] buildtools: ensure the NUMA nodes are counted correct

[Chaoyong He]

From: Peng Zhang <peng.zhang@corigine.com>

Sorting a list of strings with the format "node[0-9]+" in order to find the
largest integer by looking at the last item after the sort breaks. But if
there are more then 10 items as a string sort will sort "node10" before
"node2", it will get the error NUMA nodes.

Solve this by sorting the list based on the integer part of the string.

Fixes: 8ef09fdc506b ("build: add optional NUMA and CPU counts detection")
Cc: stable@dpdk.org

Signed-off-by: Peng Zhang <peng.zhang@corigine.com>
Signed-off-by: Chaoyong He <chaoyong.he@corigine.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@corigine.com>
---
 buildtools/get-numa-count.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/buildtools/get-numa-count.py b/buildtools/get-numa-count.py
index 1b7787787f..2f243886cd 100644
--- a/buildtools/get-numa-count.py
+++ b/buildtools/get-numa-count.py
@@ -6,11 +6,12 @@
 import glob
 import os
 import subprocess
+import re
 
 if os.name == 'posix':
     if os.path.isdir('/sys/devices/system/node'):
         numa_nodes = glob.glob('/sys/devices/system/node/node*')
-        numa_nodes.sort()
+        numa_nodes.sort(key=lambda l: int(re.findall('\d+', l)[0]))
         print(int(os.path.basename(numa_nodes[-1])[4:]) + 1)
     else:
         subprocess.run(['sysctl', '-n', 'vm.ndomains'], check=False)
-- 
2.27.0

Re: [PATCH v1] buildtools: ensure the NUMA nodes are counted correct

[Thomas Monjalon]

02/08/2022 09:54, Chaoyong He:
> From: Peng Zhang <peng.zhang@corigine.com>
> 
> Sorting a list of strings with the format "node[0-9]+" in order to find the
> largest integer by looking at the last item after the sort breaks. But if
> there are more then 10 items as a string sort will sort "node10" before
> "node2", it will get the error NUMA nodes.

What is the error you are seeing?

FW: [PATCH v1] buildtools: ensure the NUMA nodes are counted correct

[Nole Zhang]

> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: 2022年8月29日 21:15
> To: Nole Zhang <peng.zhang@corigine.com>; Chaoyong He 
> <chaoyong.he@corigine.com>
> Subject: Re: [PATCH v1] buildtools: ensure the NUMA nodes are counted 
> correct
> 
> 29/08/2022 13:17, Nole Zhang:
> > From: Thomas Monjalon <thomas@monjalon.net>
> > > 02/08/2022 09:54, Chaoyong He:
> > > > From: Peng Zhang <peng.zhang@corigine.com>
> > > >
> > > > Sorting a list of strings with the format "node[0-9]+" in order 
> > > > to find the largest integer by looking at the last item after 
> > > > the sort breaks. But if there are more then 10 items as a string 
> > > > sort will sort "node10" before "node2", it will get the error NUMA nodes.
> > >
> > > What is the error you are seeing?
> > >
> > >
> > We get the error NUMA, in this example, we get the NUMA nodes is 10, 
> > But at fact, it has 11 NUMA.
> 
> Please give more details, where do you see this error?
> We should know how to reproduce and check we have the same issue.
> Thanks
> 
> Please reply with a detailed answer on the mailing list.
> 
In the China Phytium S2500 CPU + INSPUR server, it has 16 NUMA.
The details are as follows:

~#: lscpu

Architecture:        aarch64
CPU op-mode(s):      64-bit
Byte Order:          Little Endian
CPU(s):              128
On-line CPU(s) list: 0-127
Thread(s) per core:  1
Core(s) per socket:  64
Socket(s):           2
NUMA node(s):        16
Vendor ID:           0x70
Model:               3
Model name:          S2500
Stepping:            0x1
BogoMIPS:            100.00
L1d cache:           4 MiB
L1i cache:           4 MiB
L2 cache:            64 MiB
L3 cache:            128 MiB
NUMA node0 CPU(s):   0-7
NUMA node1 CPU(s):   8-15
NUMA node2 CPU(s):   16-23
NUMA node3 CPU(s):   24-31
NUMA node4 CPU(s):   32-39
NUMA node5 CPU(s):   40-47
NUMA node6 CPU(s):   48-55
NUMA node7 CPU(s):   56-63
NUMA node8 CPU(s):   64-71
NUMA node9 CPU(s):   72-79
NUMA node10 CPU(s):  80-87
NUMA node11 CPU(s):  88-95
NUMA node12 CPU(s):  96-103
NUMA node13 CPU(s):  104-111
NUMA node14 CPU(s):  112-119
NUMA node15 CPU(s):  120-127
Flags:               half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt lpae evtstrm


I use  meson build -Dmax_lcores=detect -Dmax_numa_nodes=detect to compile, then dpdk initialization only shows 10 numa. 

Re: FW: [PATCH v1] buildtools: ensure the NUMA nodes are counted correct

[Niklas Soderlund]

Hi Thomas,

Have you checked if this address the same issue you where seeing? Do you 
think we can move forward with this fix?

On 2022-08-31 10:47:24 +0200, Nole Zhang wrote:
> 
> 
> 
> > -----Original Message-----
> > From: Thomas Monjalon <thomas@monjalon.net>
> > Sent: 2022年8月29日 21:15
> > To: Nole Zhang <peng.zhang@corigine.com>; Chaoyong He 
> > <chaoyong.he@corigine.com>
> > Subject: Re: [PATCH v1] buildtools: ensure the NUMA nodes are counted 
> > correct
> > 
> > 29/08/2022 13:17, Nole Zhang:
> > > From: Thomas Monjalon <thomas@monjalon.net>
> > > > 02/08/2022 09:54, Chaoyong He:
> > > > > From: Peng Zhang <peng.zhang@corigine.com>
> > > > >
> > > > > Sorting a list of strings with the format "node[0-9]+" in order 
> > > > > to find the largest integer by looking at the last item after 
> > > > > the sort breaks. But if there are more then 10 items as a string 
> > > > > sort will sort "node10" before "node2", it will get the error NUMA nodes.
> > > >
> > > > What is the error you are seeing?
> > > >
> > > >
> > > We get the error NUMA, in this example, we get the NUMA nodes is 10, 
> > > But at fact, it has 11 NUMA.
> > 
> > Please give more details, where do you see this error?
> > We should know how to reproduce and check we have the same issue.
> > Thanks
> > 
> > Please reply with a detailed answer on the mailing list.
> > 
> In the China Phytium S2500 CPU + INSPUR server, it has 16 NUMA.
> The details are as follows:
> 
> ~#: lscpu
> 
> Architecture:        aarch64
> CPU op-mode(s):      64-bit
> Byte Order:          Little Endian
> CPU(s):              128
> On-line CPU(s) list: 0-127
> Thread(s) per core:  1
> Core(s) per socket:  64
> Socket(s):           2
> NUMA node(s):        16
> Vendor ID:           0x70
> Model:               3
> Model name:          S2500
> Stepping:            0x1
> BogoMIPS:            100.00
> L1d cache:           4 MiB
> L1i cache:           4 MiB
> L2 cache:            64 MiB
> L3 cache:            128 MiB
> NUMA node0 CPU(s):   0-7
> NUMA node1 CPU(s):   8-15
> NUMA node2 CPU(s):   16-23
> NUMA node3 CPU(s):   24-31
> NUMA node4 CPU(s):   32-39
> NUMA node5 CPU(s):   40-47
> NUMA node6 CPU(s):   48-55
> NUMA node7 CPU(s):   56-63
> NUMA node8 CPU(s):   64-71
> NUMA node9 CPU(s):   72-79
> NUMA node10 CPU(s):  80-87
> NUMA node11 CPU(s):  88-95
> NUMA node12 CPU(s):  96-103
> NUMA node13 CPU(s):  104-111
> NUMA node14 CPU(s):  112-119
> NUMA node15 CPU(s):  120-127
> Flags:               half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt lpae evtstrm
> 
> 
> I use  meson build -Dmax_lcores=detect -Dmax_numa_nodes=detect to compile, then dpdk initialization only shows 10 numa. 

-- 
Kind Regards,
Niklas Söderlund

Re: FW: [PATCH v1] buildtools: ensure the NUMA nodes are counted correct

[Thomas Monjalon]

Hi,

20/09/2022 12:11, Niklas Soderlund:
> Hi Thomas,
> 
> Have you checked if this address the same issue you where seeing? Do you 
> think we can move forward with this fix?

Yes. No need to show lscpu output,
but the output of the DPDK initialization would be more interesting.


> On 2022-08-31 10:47:24 +0200, Nole Zhang wrote:
> > 
> > 
> > 
> > > -----Original Message-----
> > > From: Thomas Monjalon <thomas@monjalon.net>
> > > Sent: 2022年8月29日 21:15
> > > To: Nole Zhang <peng.zhang@corigine.com>; Chaoyong He 
> > > <chaoyong.he@corigine.com>
> > > Subject: Re: [PATCH v1] buildtools: ensure the NUMA nodes are counted 
> > > correct
> > > 
> > > 29/08/2022 13:17, Nole Zhang:
> > > > From: Thomas Monjalon <thomas@monjalon.net>
> > > > > 02/08/2022 09:54, Chaoyong He:
> > > > > > From: Peng Zhang <peng.zhang@corigine.com>
> > > > > >
> > > > > > Sorting a list of strings with the format "node[0-9]+" in order 
> > > > > > to find the largest integer by looking at the last item after 
> > > > > > the sort breaks. But if there are more then 10 items as a string 
> > > > > > sort will sort "node10" before "node2", it will get the error NUMA nodes.
> > > > >
> > > > > What is the error you are seeing?
> > > > >
> > > > >
> > > > We get the error NUMA, in this example, we get the NUMA nodes is 10, 
> > > > But at fact, it has 11 NUMA.
> > > 
> > > Please give more details, where do you see this error?
> > > We should know how to reproduce and check we have the same issue.
> > > Thanks
> > > 
> > > Please reply with a detailed answer on the mailing list.
> > > 
> > In the China Phytium S2500 CPU + INSPUR server, it has 16 NUMA.
> > The details are as follows:
> > 
> > ~#: lscpu
> > 
> > Architecture:        aarch64
> > CPU op-mode(s):      64-bit
> > Byte Order:          Little Endian
> > CPU(s):              128
> > On-line CPU(s) list: 0-127
> > Thread(s) per core:  1
> > Core(s) per socket:  64
> > Socket(s):           2
> > NUMA node(s):        16
> > Vendor ID:           0x70
> > Model:               3
> > Model name:          S2500
> > Stepping:            0x1
> > BogoMIPS:            100.00
> > L1d cache:           4 MiB
> > L1i cache:           4 MiB
> > L2 cache:            64 MiB
> > L3 cache:            128 MiB
> > NUMA node0 CPU(s):   0-7
> > NUMA node1 CPU(s):   8-15
> > NUMA node2 CPU(s):   16-23
> > NUMA node3 CPU(s):   24-31
> > NUMA node4 CPU(s):   32-39
> > NUMA node5 CPU(s):   40-47
> > NUMA node6 CPU(s):   48-55
> > NUMA node7 CPU(s):   56-63
> > NUMA node8 CPU(s):   64-71
> > NUMA node9 CPU(s):   72-79
> > NUMA node10 CPU(s):  80-87
> > NUMA node11 CPU(s):  88-95
> > NUMA node12 CPU(s):  96-103
> > NUMA node13 CPU(s):  104-111
> > NUMA node14 CPU(s):  112-119
> > NUMA node15 CPU(s):  120-127
> > Flags:               half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt lpae evtstrm
> > 
> > 
> > I use  meson build -Dmax_lcores=detect -Dmax_numa_nodes=detect to compile, then dpdk initialization only shows 10 numa. 
> 
> 

RE: FW: [PATCH v1] buildtools: ensure the NUMA nodes are counted correct

[Nole Zhang]

> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: 2022年9月20日 21:23
> To: Niklas Soderlund <niklas.soderlund@corigine.com>
> Cc: Nole Zhang <peng.zhang@corigine.com>; dev@dpdk.org; Chaoyong He
> <chaoyong.he@corigine.com>; oss-drivers <oss-drivers@corigine.com>
> Subject: Re: FW: [PATCH v1] buildtools: ensure the NUMA nodes are counted
> correct
> 
> Hi,
> 
> 20/09/2022 12:11, Niklas Soderlund:
> > Hi Thomas,
> >
> > Have you checked if this address the same issue you where seeing? Do
> > you think we can move forward with this fix?
> 
> Yes. No need to show lscpu output,
> but the output of the DPDK initialization would be more interesting.
> 

OK, this is the DPDK initialization.

The original code, the dpdkd initialized info:
EAL: Detected CPU lcores: 128
EAL: Detected NUMA nodes: 10
EAL: Static memory layout is selected, amount of reserved memory can be adjusted with -m or --socket-mem
EAL: Detected static linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: VFIO support initialized

Revised code, the dpdkd initialized info:
EAL: Detected CPU lcores: 128
EAL: Detected NUMA nodes: 16
EAL: Static memory layout is selected, amount of reserved memory can be adjusted with -m or --socket-mem
EAL: Detected static linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: VFIO support initialized

> 
> > On 2022-08-31 10:47:24 +0200, Nole Zhang wrote:
> > >
> > >
> > >
> > > > -----Original Message-----
> > > > From: Thomas Monjalon <thomas@monjalon.net>
> > > > Sent: 2022年8月29日 21:15
> > > > To: Nole Zhang <peng.zhang@corigine.com>; Chaoyong He
> > > > <chaoyong.he@corigine.com>
> > > > Subject: Re: [PATCH v1] buildtools: ensure the NUMA nodes are
> > > > counted correct
> > > >
> > > > 29/08/2022 13:17, Nole Zhang:
> > > > > From: Thomas Monjalon <thomas@monjalon.net>
> > > > > > 02/08/2022 09:54, Chaoyong He:
> > > > > > > From: Peng Zhang <peng.zhang@corigine.com>
> > > > > > >
> > > > > > > Sorting a list of strings with the format "node[0-9]+" in
> > > > > > > order to find the largest integer by looking at the last
> > > > > > > item after the sort breaks. But if there are more then 10
> > > > > > > items as a string sort will sort "node10" before "node2", it will get
> the error NUMA nodes.
> > > > > >
> > > > > > What is the error you are seeing?
> > > > > >
> > > > > >
> > > > > We get the error NUMA, in this example, we get the NUMA nodes is
> > > > > 10, But at fact, it has 11 NUMA.
> > > >
> > > > Please give more details, where do you see this error?
> > > > We should know how to reproduce and check we have the same issue.
> > > > Thanks
> > > >
> > > > Please reply with a detailed answer on the mailing list.
> > > >
> > > In the China Phytium S2500 CPU + INSPUR server, it has 16 NUMA.
> > > The details are as follows:
> > >
> > > ~#: lscpu
> > >
> > > Architecture:        aarch64
> > > CPU op-mode(s):      64-bit
> > > Byte Order:          Little Endian
> > > CPU(s):              128
> > > On-line CPU(s) list: 0-127
> > > Thread(s) per core:  1
> > > Core(s) per socket:  64
> > > Socket(s):           2
> > > NUMA node(s):        16
> > > Vendor ID:           0x70
> > > Model:               3
> > > Model name:          S2500
> > > Stepping:            0x1
> > > BogoMIPS:            100.00
> > > L1d cache:           4 MiB
> > > L1i cache:           4 MiB
> > > L2 cache:            64 MiB
> > > L3 cache:            128 MiB
> > > NUMA node0 CPU(s):   0-7
> > > NUMA node1 CPU(s):   8-15
> > > NUMA node2 CPU(s):   16-23
> > > NUMA node3 CPU(s):   24-31
> > > NUMA node4 CPU(s):   32-39
> > > NUMA node5 CPU(s):   40-47
> > > NUMA node6 CPU(s):   48-55
> > > NUMA node7 CPU(s):   56-63
> > > NUMA node8 CPU(s):   64-71
> > > NUMA node9 CPU(s):   72-79
> > > NUMA node10 CPU(s):  80-87
> > > NUMA node11 CPU(s):  88-95
> > > NUMA node12 CPU(s):  96-103
> > > NUMA node13 CPU(s):  104-111
> > > NUMA node14 CPU(s):  112-119
> > > NUMA node15 CPU(s):  120-127
> > > Flags:               half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt
> lpae evtstrm
> > >
> > >
> > > I use  meson build -Dmax_lcores=detect -Dmax_numa_nodes=detect to
> compile, then dpdk initialization only shows 10 numa.
> >
> >
> 
> 
> 
> 

Re: FW: [PATCH v1] buildtools: ensure the NUMA nodes are counted correct

[Thomas Monjalon]

Please send a v2 with the proper explanations.


21/09/2022 09:19, Nole Zhang:
> 
> > -----Original Message-----
> > From: Thomas Monjalon <thomas@monjalon.net>
> > Sent: 2022年9月20日 21:23
> > To: Niklas Soderlund <niklas.soderlund@corigine.com>
> > Cc: Nole Zhang <peng.zhang@corigine.com>; dev@dpdk.org; Chaoyong He
> > <chaoyong.he@corigine.com>; oss-drivers <oss-drivers@corigine.com>
> > Subject: Re: FW: [PATCH v1] buildtools: ensure the NUMA nodes are counted
> > correct
> > 
> > Hi,
> > 
> > 20/09/2022 12:11, Niklas Soderlund:
> > > Hi Thomas,
> > >
> > > Have you checked if this address the same issue you where seeing? Do
> > > you think we can move forward with this fix?
> > 
> > Yes. No need to show lscpu output,
> > but the output of the DPDK initialization would be more interesting.
> > 
> 
> OK, this is the DPDK initialization.
> 
> The original code, the dpdkd initialized info:
> EAL: Detected CPU lcores: 128
> EAL: Detected NUMA nodes: 10
> EAL: Static memory layout is selected, amount of reserved memory can be adjusted with -m or --socket-mem
> EAL: Detected static linkage of DPDK
> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> EAL: Selected IOVA mode 'VA'
> EAL: VFIO support initialized
> 
> Revised code, the dpdkd initialized info:
> EAL: Detected CPU lcores: 128
> EAL: Detected NUMA nodes: 16
> EAL: Static memory layout is selected, amount of reserved memory can be adjusted with -m or --socket-mem
> EAL: Detected static linkage of DPDK
> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> EAL: Selected IOVA mode 'VA'
> EAL: VFIO support initialized
> 
> > 
> > > On 2022-08-31 10:47:24 +0200, Nole Zhang wrote:
> > > >
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: Thomas Monjalon <thomas@monjalon.net>
> > > > > Sent: 2022年8月29日 21:15
> > > > > To: Nole Zhang <peng.zhang@corigine.com>; Chaoyong He
> > > > > <chaoyong.he@corigine.com>
> > > > > Subject: Re: [PATCH v1] buildtools: ensure the NUMA nodes are
> > > > > counted correct
> > > > >
> > > > > 29/08/2022 13:17, Nole Zhang:
> > > > > > From: Thomas Monjalon <thomas@monjalon.net>
> > > > > > > 02/08/2022 09:54, Chaoyong He:
> > > > > > > > From: Peng Zhang <peng.zhang@corigine.com>
> > > > > > > >
> > > > > > > > Sorting a list of strings with the format "node[0-9]+" in
> > > > > > > > order to find the largest integer by looking at the last
> > > > > > > > item after the sort breaks. But if there are more then 10
> > > > > > > > items as a string sort will sort "node10" before "node2", it will get
> > the error NUMA nodes.
> > > > > > >
> > > > > > > What is the error you are seeing?
> > > > > > >
> > > > > > >
> > > > > > We get the error NUMA, in this example, we get the NUMA nodes is
> > > > > > 10, But at fact, it has 11 NUMA.
> > > > >
> > > > > Please give more details, where do you see this error?
> > > > > We should know how to reproduce and check we have the same issue.
> > > > > Thanks
> > > > >
> > > > > Please reply with a detailed answer on the mailing list.
> > > > >
> > > > In the China Phytium S2500 CPU + INSPUR server, it has 16 NUMA.
> > > > The details are as follows:
> > > >
> > > > ~#: lscpu
> > > >
> > > > Architecture:        aarch64
> > > > CPU op-mode(s):      64-bit
> > > > Byte Order:          Little Endian
> > > > CPU(s):              128
> > > > On-line CPU(s) list: 0-127
> > > > Thread(s) per core:  1
> > > > Core(s) per socket:  64
> > > > Socket(s):           2
> > > > NUMA node(s):        16
> > > > Vendor ID:           0x70
> > > > Model:               3
> > > > Model name:          S2500
> > > > Stepping:            0x1
> > > > BogoMIPS:            100.00
> > > > L1d cache:           4 MiB
> > > > L1i cache:           4 MiB
> > > > L2 cache:            64 MiB
> > > > L3 cache:            128 MiB
> > > > NUMA node0 CPU(s):   0-7
> > > > NUMA node1 CPU(s):   8-15
> > > > NUMA node2 CPU(s):   16-23
> > > > NUMA node3 CPU(s):   24-31
> > > > NUMA node4 CPU(s):   32-39
> > > > NUMA node5 CPU(s):   40-47
> > > > NUMA node6 CPU(s):   48-55
> > > > NUMA node7 CPU(s):   56-63
> > > > NUMA node8 CPU(s):   64-71
> > > > NUMA node9 CPU(s):   72-79
> > > > NUMA node10 CPU(s):  80-87
> > > > NUMA node11 CPU(s):  88-95
> > > > NUMA node12 CPU(s):  96-103
> > > > NUMA node13 CPU(s):  104-111
> > > > NUMA node14 CPU(s):  112-119
> > > > NUMA node15 CPU(s):  120-127
> > > > Flags:               half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt
> > lpae evtstrm
> > > >
> > > >
> > > > I use  meson build -Dmax_lcores=detect -Dmax_numa_nodes=detect to
> > compile, then dpdk initialization only shows 10 numa.
> > >
> > >
> > 
> > 
> > 
> > 
> 
> 

[PATCH v2] buildtools: ensure the NUMA nodes are counted correct

[Niklas Söderlund]

From: Peng Zhang <peng.zhang@corigine.com>

The method to fetch, sort and read the last entry of a list to figure
out the total number of NUMA nodes in the system fails with 10 or more
nodes. The reason being the usage of string compare while sorting, hence
node 'node10' will be sorted before 'node2'.

Solve this by sorting the list based on integer comparison of the
numerical part of the node name.

Before this change on a system with 16 NUMA nodes,

    EAL: Detected CPU lcores: 128
    EAL: Detected NUMA nodes: 10
    EAL: Static memory layout is selected, amount of reserved memory can
	 be adjusted with -m or --socket-mem
    EAL: Detected static linkage of DPDK
    EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
    EAL: Selected IOVA mode 'VA'
    EAL: VFIO support initialized

With this change on the same system,

    EAL: Detected CPU lcores: 128
    EAL: Detected NUMA nodes: 16
    EAL: Static memory layout is selected, amount of reserved memory can
	 be adjusted with -m or --socket-mem
    EAL: Detected static linkage of DPDK
    EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
    EAL: Selected IOVA mode 'VA'
    EAL: VFIO support initialized

Fixes: 8ef09fdc506b ("build: add optional NUMA and CPU counts detection")
Cc: stable@dpdk.org

Signed-off-by: Peng Zhang <peng.zhang@corigine.com>
Signed-off-by: Chaoyong He <chaoyong.he@corigine.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@corigine.com>
---
 buildtools/get-numa-count.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/buildtools/get-numa-count.py b/buildtools/get-numa-count.py
index 1b7787787f71..2f243886cd21 100644
--- a/buildtools/get-numa-count.py
+++ b/buildtools/get-numa-count.py
@@ -6,11 +6,12 @@
 import glob
 import os
 import subprocess
+import re
 
 if os.name == 'posix':
     if os.path.isdir('/sys/devices/system/node'):
         numa_nodes = glob.glob('/sys/devices/system/node/node*')
-        numa_nodes.sort()
+        numa_nodes.sort(key=lambda l: int(re.findall('\d+', l)[0]))
         print(int(os.path.basename(numa_nodes[-1])[4:]) + 1)
     else:
         subprocess.run(['sysctl', '-n', 'vm.ndomains'], check=False)
-- 
2.37.3

Re: [PATCH v2] buildtools: ensure the NUMA nodes are counted correct

[Niklas Söderlund]

Hi,

Genteel ping on this.

On 2022-09-22 10:06:42 +0200, Niklas Söderlund wrote:
> From: Peng Zhang <peng.zhang@corigine.com>
> 
> The method to fetch, sort and read the last entry of a list to figure
> out the total number of NUMA nodes in the system fails with 10 or more
> nodes. The reason being the usage of string compare while sorting, hence
> node 'node10' will be sorted before 'node2'.
> 
> Solve this by sorting the list based on integer comparison of the
> numerical part of the node name.
> 
> Before this change on a system with 16 NUMA nodes,
> 
>     EAL: Detected CPU lcores: 128
>     EAL: Detected NUMA nodes: 10
>     EAL: Static memory layout is selected, amount of reserved memory can
> 	 be adjusted with -m or --socket-mem
>     EAL: Detected static linkage of DPDK
>     EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
>     EAL: Selected IOVA mode 'VA'
>     EAL: VFIO support initialized
> 
> With this change on the same system,
> 
>     EAL: Detected CPU lcores: 128
>     EAL: Detected NUMA nodes: 16
>     EAL: Static memory layout is selected, amount of reserved memory can
> 	 be adjusted with -m or --socket-mem
>     EAL: Detected static linkage of DPDK
>     EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
>     EAL: Selected IOVA mode 'VA'
>     EAL: VFIO support initialized
> 
> Fixes: 8ef09fdc506b ("build: add optional NUMA and CPU counts detection")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Peng Zhang <peng.zhang@corigine.com>
> Signed-off-by: Chaoyong He <chaoyong.he@corigine.com>
> Reviewed-by: Niklas Söderlund <niklas.soderlund@corigine.com>
> ---
>  buildtools/get-numa-count.py | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/buildtools/get-numa-count.py b/buildtools/get-numa-count.py
> index 1b7787787f71..2f243886cd21 100644
> --- a/buildtools/get-numa-count.py
> +++ b/buildtools/get-numa-count.py
> @@ -6,11 +6,12 @@
>  import glob
>  import os
>  import subprocess
> +import re
>  
>  if os.name == 'posix':
>      if os.path.isdir('/sys/devices/system/node'):
>          numa_nodes = glob.glob('/sys/devices/system/node/node*')
> -        numa_nodes.sort()
> +        numa_nodes.sort(key=lambda l: int(re.findall('\d+', l)[0]))
>          print(int(os.path.basename(numa_nodes[-1])[4:]) + 1)
>      else:
>          subprocess.run(['sysctl', '-n', 'vm.ndomains'], check=False)
> -- 
> 2.37.3
> 

-- 
Kind Regards,
Niklas Söderlund

Re: [PATCH v2] buildtools: ensure the NUMA nodes are counted correct

[Thomas Monjalon]

22/09/2022 10:06, Niklas Söderlund:
> From: Peng Zhang <peng.zhang@corigine.com>
> 
> The method to fetch, sort and read the last entry of a list to figure
> out the total number of NUMA nodes in the system fails with 10 or more
> nodes. The reason being the usage of string compare while sorting, hence
> node 'node10' will be sorted before 'node2'.
> 
> Solve this by sorting the list based on integer comparison of the
> numerical part of the node name.
> 
> Before this change on a system with 16 NUMA nodes,
> 
>     EAL: Detected CPU lcores: 128
>     EAL: Detected NUMA nodes: 10
>     EAL: Static memory layout is selected, amount of reserved memory can
> 	 be adjusted with -m or --socket-mem
>     EAL: Detected static linkage of DPDK
>     EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
>     EAL: Selected IOVA mode 'VA'
>     EAL: VFIO support initialized
> 
> With this change on the same system,
> 
>     EAL: Detected CPU lcores: 128
>     EAL: Detected NUMA nodes: 16
>     EAL: Static memory layout is selected, amount of reserved memory can
> 	 be adjusted with -m or --socket-mem
>     EAL: Detected static linkage of DPDK
>     EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
>     EAL: Selected IOVA mode 'VA'
>     EAL: VFIO support initialized
> 
> Fixes: 8ef09fdc506b ("build: add optional NUMA and CPU counts detection")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Peng Zhang <peng.zhang@corigine.com>
> Signed-off-by: Chaoyong He <chaoyong.he@corigine.com>
> Reviewed-by: Niklas Söderlund <niklas.soderlund@corigine.com>

Applied, thanks.