DPDK patches and discussions
 help / color / mirror / Atom feed
From: Juraj Linkeš <juraj.linkes@pantheon.tech>
To: David Christensen <drc@linux.vnet.ibm.com>,
	"dev@dpdk.org" <dev@dpdk.org>
Cc: "thomas@monjalon.net" <thomas@monjalon.net>,
	"david.marchand@redhat.com" <david.marchand@redhat.com>,
	"bruce.richardson@intel.com" <bruce.richardson@intel.com>,
	"Honnappa.Nagarahalli@arm.com" <Honnappa.Nagarahalli@arm.com>
Subject: Re: [dpdk-dev] [PATCH v2] build: optional NUMA and cpu counts detection
Date: Fri, 16 Apr 2021 12:43:39 +0000
Message-ID: <dd061536581f4104bdb430270de7a02e@pantheon.tech> (raw)
In-Reply-To: <fbbabb7b-d011-55ed-e6a8-23321aef8279@linux.vnet.ibm.com>

I didn't notice this e-mail, since it was sent just to dev@dpdk.org and not to me explicitly, which left it unnoticed by my filters, unfortunately.

> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of David Christensen
> Sent: Wednesday, March 31, 2021 9:07 PM
> To: dev@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH v2] build: optional NUMA and cpu counts
> detection
> 
> 
> 
> On 3/31/21 2:06 AM, Juraj Linkeš wrote:
> > Add an option to automatically discover the host's numa and cpu counts
> > and use those values for a non cross-build.
> > Give users the option to override the per-arch default values or
> > values from cross files by specifying them on the command line with
> > -Dmax_lcores and -Dmax_numa_nodes.
> >
> > Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> > Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
> > ---
> >   MAINTAINERS                  |  2 ++
> >   buildtools/get-cpu-count.py  |  7 ++++++
> >   buildtools/get-numa-count.py | 22 +++++++++++++++++
> >   buildtools/meson.build       |  2 ++
> >   config/meson.build           | 47 ++++++++++++++++++++++++++++++++++--
> >   config/x86/meson.build       |  2 ++
> >   meson_options.txt            |  8 +++---
> >   7 files changed, 84 insertions(+), 6 deletions(-)
> >   create mode 100644 buildtools/get-cpu-count.py
> >   create mode 100644 buildtools/get-numa-count.py
> >
> > diff --git a/MAINTAINERS b/MAINTAINERS index 0ec5588540..7270f33cf5
> > 100644
> > --- a/MAINTAINERS
> > +++ b/MAINTAINERS
> > @@ -99,6 +99,8 @@ F: meson_options.txt
> >   F: config/
> >   F: buildtools/chkincs/
> >   F: buildtools/call-sphinx-build.py
> > +F: buildtools/get-cpu-count.py
> > +F: buildtools/get-numa-count.py
> >   F: buildtools/list-dir-globs.py
> >   F: buildtools/pkg-config/
> >   F: buildtools/symlink-drivers-solibs.sh
> > diff --git a/buildtools/get-cpu-count.py b/buildtools/get-cpu-count.py
> > new file mode 100644 index 0000000000..317b32088f
> > --- /dev/null
> > +++ b/buildtools/get-cpu-count.py
> > @@ -0,0 +1,7 @@
> > +#!/usr/bin/env python3
> > +# SPDX-License-Identifier: BSD-3-Clause # Copyright (c) 2021
> > +PANTHEON.tech s.r.o.
> > +
> > +import os
> > +
> > +print(os.cpu_count())
> 
> Returns the expected value on a P9 system.
> 
> > diff --git a/buildtools/get-numa-count.py
> > b/buildtools/get-numa-count.py new file mode 100644 index
> > 0000000000..77ef2b9f24
> > --- /dev/null
> > +++ b/buildtools/get-numa-count.py
> > @@ -0,0 +1,22 @@
> > +#!/usr/bin/env python3
> > +# SPDX-License-Identifier: BSD-3-Clause # Copyright (c) 2021
> > +PANTHEON.tech s.r.o.
> > +
> > +import ctypes
> > +import glob
> > +import os
> > +import subprocess
> > +
> > +if os.name == 'posix':
> > +    if os.path.isdir('/sys/devices/system/node'):
> > +        print(len(glob.glob('/sys/devices/system/node/node*')))
> > +    else:
> > +        subprocess.run(['sysctl', '-n', 'vm.ndomains'])
> > +
> > +elif os.name == 'nt':
> > +    libkernel32 = ctypes.windll.kernel32
> > +
> > +    count = ctypes.c_ulong()
> > +
> > +    libkernel32.GetNumaHighestNodeNumber(ctypes.pointer(count))
> > +    print(count.value + 1)
> 
> Does not return the expected value on my P9 system (NUMA nodes are not
> contiguous).  Got 6, expect to see at least 8 otherwise I can't use lcores on
> NUMA node 8.
> 

I guess the problem here is that we're not really setting the number of NUMA nodes that are usable, but the highest NUMA that is usable?

In that case, the proper config in should be 256 so that even node255 is usable, is that right?

> $ python3 ./get-numa-count.py
> 6
> $ lscpu
> Architecture:        ppc64le
> Byte Order:          Little Endian
> CPU(s):              128
> On-line CPU(s) list: 0-127
> Thread(s) per core:  4
> Core(s) per socket:  16
> Socket(s):           2
> NUMA node(s):        6
> Model:               2.3 (pvr 004e 1203)
> Model name:          POWER9, altivec supported
> CPU max MHz:         3800.0000
> CPU min MHz:         2300.0000
> L1d cache:           32K
> L1i cache:           32K
> L2 cache:            512K
> L3 cache:            10240K
> NUMA node0 CPU(s):   0-63
> NUMA node8 CPU(s):   64-127
> NUMA node252 CPU(s):
> NUMA node253 CPU(s):
> NUMA node254 CPU(s):
> NUMA node255 CPU(s):
> 
> See attached ls-node.txt.gz for full directory structure on a P9 system.
> 
> Dave

  reply	other threads:[~2021-04-16 12:43 UTC|newest]

Thread overview: 32+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-11-20 12:55 [dpdk-dev] [PATCH v1 1/1] " Juraj Linkeš
2020-12-23 11:32 ` Juraj Linkeš
2021-03-31  9:06 ` [dpdk-dev] [PATCH v2] " Juraj Linkeš
2021-03-31 19:07   ` David Christensen
2021-04-16 12:43     ` Juraj Linkeš [this message]
2021-04-19 10:18   ` [dpdk-dev] [PATCH v3] " Juraj Linkeš
2021-04-28 19:33     ` David Christensen
2021-06-29 10:55     ` [dpdk-dev] [PATCH v4] " Juraj Linkeš
2021-06-29 11:28       ` Bruce Richardson
2021-07-06  8:56         ` Juraj Linkeš
2021-07-06  9:08           ` Bruce Richardson
2021-07-06 18:10             ` David Christensen
2021-07-16 13:53               ` Juraj Linkeš
2021-07-16 14:24                 ` Bruce Richardson
2021-07-20 20:49                 ` David Christensen
2021-07-21 12:41                   ` Juraj Linkeš
2021-06-29 18:00       ` Stephen Hemminger
2021-07-21 13:04       ` [dpdk-dev] [PATCH v5] " Juraj Linkeš
2021-08-02 12:44         ` Juraj Linkeš
2021-08-02 23:29           ` David Christensen
2021-08-03 10:21             ` Juraj Linkeš
2021-08-31  0:54               ` Piotr Kubaj
2021-08-31  7:54                 ` Juraj Linkeš
2021-08-31  8:02                   ` Bruce Richardson
2021-09-09  7:20                     ` Juraj Linkeš
2021-09-09  8:01                       ` Bruce Richardson
2021-09-09  8:08                         ` Thomas Monjalon
2021-09-09  8:42                           ` Juraj Linkeš
2021-09-09 16:44                         ` David Christensen
2021-08-31 12:32                   ` Piotr Kubaj
2021-08-17 10:45         ` [dpdk-dev] [PATCH v6] " Juraj Linkeš
2021-09-16  7:46           ` Thomas Monjalon

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=dd061536581f4104bdb430270de7a02e@pantheon.tech \
    --to=juraj.linkes@pantheon.tech \
    --cc=Honnappa.Nagarahalli@arm.com \
    --cc=bruce.richardson@intel.com \
    --cc=david.marchand@redhat.com \
    --cc=dev@dpdk.org \
    --cc=drc@linux.vnet.ibm.com \
    --cc=thomas@monjalon.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

DPDK patches and discussions

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://inbox.dpdk.org/dev/0 dev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dev dev/ https://inbox.dpdk.org/dev \
		dev@dpdk.org
	public-inbox-index dev

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.dev


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git