From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1D214A0C47; Tue, 6 Jul 2021 11:08:46 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 053554120E; Tue, 6 Jul 2021 11:08:46 +0200 (CEST) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by mails.dpdk.org (Postfix) with ESMTP id 160FD4067E for ; Tue, 6 Jul 2021 11:08:43 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10036"; a="270203634" X-IronPort-AV: E=Sophos;i="5.83,328,1616482800"; d="scan'208";a="270203634" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Jul 2021 02:08:42 -0700 X-IronPort-AV: E=Sophos;i="5.83,328,1616482800"; d="scan'208";a="456996623" Received: from bricha3-mobl.ger.corp.intel.com ([10.252.23.242]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-SHA; 06 Jul 2021 02:08:40 -0700 Date: Tue, 6 Jul 2021 10:08:37 +0100 From: Bruce Richardson To: Juraj =?utf-8?Q?Linke=C5=A1?= Cc: "thomas@monjalon.net" , "david.marchand@redhat.com" , "Honnappa.Nagarahalli@arm.com" , "Ruifeng.Wang@arm.com" , "ferruh.yigit@intel.com" , "jerinjacobk@gmail.com" , "dev@dpdk.org" Message-ID: References: <1618827522-31828-1-git-send-email-juraj.linkes@pantheon.tech> <1624964105-6525-1-git-send-email-juraj.linkes@pantheon.tech> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: Subject: Re: [dpdk-dev] [PATCH v4] build: optional NUMA and cpu counts detection X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Tue, Jul 06, 2021 at 08:56:37AM +0000, Juraj Linkeš wrote: > > > > -----Original Message----- > > From: Bruce Richardson > > Sent: Tuesday, June 29, 2021 1:29 PM > > To: Juraj Linkeš > > Cc: thomas@monjalon.net; david.marchand@redhat.com; > > Honnappa.Nagarahalli@arm.com; Ruifeng.Wang@arm.com; > > ferruh.yigit@intel.com; jerinjacobk@gmail.com; dev@dpdk.org > > Subject: Re: [PATCH v4] build: optional NUMA and cpu counts detection > > > > On Tue, Jun 29, 2021 at 12:55:05PM +0200, Juraj Linkeš wrote: > > > Add an option to automatically discover the host's numa and cpu counts > > > and use those values for a non cross-build. > > > Give users the option to override the per-arch default values or > > > values from cross files by specifying them on the command line with > > > -Dmax_lcores and -Dmax_numa_nodes. > > > > > > Signed-off-by: Juraj Linkeš > > > Reviewed-by: Honnappa Nagarahalli > > > --- > > Two very minor suggestions inline below. > > > > Acked-by: Bruce Richardson > > > > > > > > > > +max_lcores = get_option('max_lcores') if max_lcores == 'auto' > > > > Rather than "auto", would "detect" be a clearer name for this option value? > > > > > > > +option('max_lcores', type: 'string', value: 'default', description: > > > + 'Set maximum number of cores/threads supported by EAL. The > > > +default is different per-arch. Set to auto to detect the number of cores on the > > build machine.') option('max_numa_nodes', type: 'string', value: 'default', > > description: > > > + 'Set highest NUMA node supported by EAL. The default is > > > +different per-arch. Set to auto to detect the highest numa node on > > > +the build machine.') > > > > I'd put the explicit values of "default" and "auto"(or "detect") in quotes "" to > > make clear they are literal values. > > > > Thanks, Bruce, I'll change it. I have one extra question now that I'm looking at the patch: > What does subprocess.run(['sysctl', '-n', 'vm.ndomains'], check=False) return exactly? Is the the number of NUMA nodes (looks like it) or the highest NUMA node on the system (the highest number of all NUMA nodes)? I'm asking because of how NUMA works on P9: > NUMA node0 CPU(s): 0-63 > NUMA node8 CPU(s): 64-127 > NUMA node252 CPU(s): > NUMA node253 CPU(s): > NUMA node254 CPU(s): > NUMA node255 CPU(s): > > Here we need not just two NUMA nodes, but at least 9 (0-8). Linux and Windows should return the highest NUMA, not sure about FreeBSD. Or maybe we should return the highest NUMA on which there are actual CPUs? I'm not sure, and I think to be really sure we'd need it tested on a P9 system. The help text for the sysctl node says "Number of physical memory domains available", which would imply 2 in the case above. [However, we also would need to find out how BSD numbers the domains, too, as it's possible an OS could just call them 0 and 1, rather than 0 and 8 if it wanted to.] In short, we'd need to test to be sure. Is FreeBSD on P9 a supported config, and if so can the P9 maintainer perhaps help out with testing? /Bruce