From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 13CAEA04B7; Wed, 14 Oct 2020 10:16:25 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id E53231DBC7; Wed, 14 Oct 2020 10:16:23 +0200 (CEST) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by dpdk.org (Postfix) with ESMTP id 286E31DB81 for ; Wed, 14 Oct 2020 10:16:22 +0200 (CEST) IronPort-SDR: YYNZQZ3mbNq7SJu/9sOGdnkhj0snnliwuv8ILrsxQlx0eHUVrEYsIExx4tq1pcR1KFLcW7vc1H VbGFs42RRMfw== X-IronPort-AV: E=McAfee;i="6000,8403,9773"; a="230233602" X-IronPort-AV: E=Sophos;i="5.77,374,1596524400"; d="scan'208";a="230233602" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Oct 2020 01:16:19 -0700 IronPort-SDR: E8ZX0WHwmtLBjHx7OKNP5/0PvGz1KZPtNak1PZzWz8No9fXJkKo4ZBaVP5JMSipELf/z6DsC7v L3SYSekbEVKA== X-IronPort-AV: E=Sophos;i="5.77,374,1596524400"; d="scan'208";a="521339342" Received: from ifreidki-mobl1.ger.corp.intel.com (HELO bricha3-MOBL.ger.corp.intel.com) ([10.254.145.91]) by fmsmga005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-SHA; 14 Oct 2020 01:16:15 -0700 Date: Wed, 14 Oct 2020 09:16:10 +0100 From: Bruce Richardson To: Juraj =?utf-8?Q?Linke=C5=A1?= Cc: "Ruifeng.Wang@arm.com" , "Honnappa.Nagarahalli@arm.com" , "Phil.Yang@arm.com" , "vcchunga@amazon.com" , "Dharmik.Thakkar@arm.com" , "jerinjacobk@gmail.com" , "hemant.agrawal@nxp.com" , "dev@dpdk.org" Message-ID: <20201014081610.GA1513@bricha3-MOBL.ger.corp.intel.com> References: <1600867161-15673-1-git-send-email-juraj.linkes@pantheon.tech> <1602600882-695-1-git-send-email-juraj.linkes@pantheon.tech> <1602600882-695-4-git-send-email-juraj.linkes@pantheon.tech> <20201013154757.GC1496@bricha3-MOBL.ger.corp.intel.com> <42b8d77b7d764e0789fb74ef2f9401d4@pantheon.tech> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <42b8d77b7d764e0789fb74ef2f9401d4@pantheon.tech> Subject: Re: [dpdk-dev] [RFC PATCH v2 3/6] build: automatic NUMA and cpu counts detection X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Wed, Oct 14, 2020 at 06:04:52AM +0000, Juraj Linkeš wrote: > > > > -----Original Message----- > > From: Bruce Richardson > > Sent: Tuesday, October 13, 2020 5:48 PM > > To: Juraj Linkeš > > Cc: Ruifeng.Wang@arm.com; Honnappa.Nagarahalli@arm.com; > > Phil.Yang@arm.com; vcchunga@amazon.com; Dharmik.Thakkar@arm.com; > > jerinjacobk@gmail.com; hemant.agrawal@nxp.com; dev@dpdk.org > > Subject: Re: [RFC PATCH v2 3/6] build: automatic NUMA and cpu counts > > detection > > > > On Tue, Oct 13, 2020 at 04:54:39PM +0200, Juraj Linkeš wrote: > > > The build machine's number of cpus and numa nodes vary, resulting in > > > mismatched counts of RTE_MAX_LCORE and RTE_MAX_NUMA_NODES for > > many > > > builds. Automatically discover the host's numa and cpu counts to > > > remove this mismatch for native builds. Use current defaults for cross builds. > > > Leave users the option to override both if the specify a non-zero > > > amount on the command line. > > > > > > Signed-off-by: Juraj Linkeš > > > --- > > > buildtools/get_cpu_count.py | 7 +++++++ > > > buildtools/get_numa_count.py | 17 +++++++++++++++++ > > > buildtools/meson.build | 2 ++ > > > config/meson.build | 20 ++++++++++++++++++-- > > > meson_options.txt | 8 ++++---- > > > 5 files changed, 48 insertions(+), 6 deletions(-) create mode 100644 > > > buildtools/get_cpu_count.py create mode 100644 > > > buildtools/get_numa_count.py > > > > > > diff --git a/buildtools/get_cpu_count.py b/buildtools/get_cpu_count.py > > > new file mode 100644 index 000000000..386f85f8b > > > --- /dev/null > > > +++ b/buildtools/get_cpu_count.py > > > @@ -0,0 +1,7 @@ > > > +#!/usr/bin/python3 > > > +# SPDX-License-Identifier: BSD-3-Clause # Copyright (c) 2020 > > > +PANTHEON.tech s.r.o. > > > + > > > +import os > > > + > > > +print(os.cpu_count()) > > > diff --git a/buildtools/get_numa_count.py > > > b/buildtools/get_numa_count.py new file mode 100644 index > > > 000000000..f2ad35532 > > > --- /dev/null > > > +++ b/buildtools/get_numa_count.py > > > @@ -0,0 +1,17 @@ > > > +#!/usr/bin/python3 > > > +# SPDX-License-Identifier: BSD-3-Clause # Copyright (c) 2020 > > > +PANTHEON.tech s.r.o. > > > + > > > +import ctypes > > > +import glob > > > +import os > > > + > > > +if os.name == 'posix': > > > + print(len(glob.glob('/sys/devices/system/node/node*'))) > > > > FreeBSD also reports it's os.name as posix, but doesn't have a /sys/devices path > > to query. > > > > Do you know how do I get numa count on FreeBDS? I don't want to use the numa library if we can to avoid unneccesary dependencies. Do we need to cover some other cases other than FreeBDS, other Linux and Windows? > To get numa count on freebsd, you can query it using "sysctl vm.ndomains" command. However, the bigger issue here is that you need something different other than os.name to distinguish between linux and freebsd. Regards, /Bruce