From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 61384A0521; Tue, 3 Nov 2020 10:44:59 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 445FAC832; Tue, 3 Nov 2020 10:44:58 +0100 (CET) Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by dpdk.org (Postfix) with ESMTP id 93E64C82C for ; Tue, 3 Nov 2020 10:44:56 +0100 (CET) IronPort-SDR: pjRvsrKcmiJK6daDxmyRgRhzFRJcsu6k5iQspMvvO9lTfMkhAB+8O3yfJXHg9eae62pmo4b6fq rfj/gprMV1GA== X-IronPort-AV: E=McAfee;i="6000,8403,9793"; a="253735585" X-IronPort-AV: E=Sophos;i="5.77,447,1596524400"; d="scan'208";a="253735585" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Nov 2020 01:44:54 -0800 IronPort-SDR: uzMEe8mycvd99M0g8ggVcCJBwRKbVkUpqstz1Mlo0lWXGNhKrAvVuRbAu/xIQ+n77Fu4pGWDEC P9A2jsYyDkKg== X-IronPort-AV: E=Sophos;i="5.77,447,1596524400"; d="scan'208";a="470735013" Received: from bricha3-mobl.ger.corp.intel.com ([10.249.45.202]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-SHA; 03 Nov 2020 01:44:50 -0800 Date: Tue, 3 Nov 2020 09:44:46 +0000 From: Bruce Richardson To: Honnappa Nagarahalli Cc: Juraj =?utf-8?Q?Linke=C5=A1?= , Ruifeng Wang , Phil Yang , "vcchunga@amazon.com" , Dharmik Thakkar , "jerinjacobk@gmail.com" , "hemant.agrawal@nxp.com" , "dev@dpdk.org" , nd Message-ID: <20201103094446.GA1132@bricha3-MOBL.ger.corp.intel.com> References: <1602600882-695-1-git-send-email-juraj.linkes@pantheon.tech> <1603280261-20206-1-git-send-email-juraj.linkes@pantheon.tech> <1603280261-20206-4-git-send-email-juraj.linkes@pantheon.tech> <20201021120220.GC592@bricha3-MOBL.ger.corp.intel.com> <54e20244e1e841148669eb93b6b8876c@pantheon.tech> <20201021141319.GD592@bricha3-MOBL.ger.corp.intel.com> <20201102135520.GA1454@bricha3-MOBL.ger.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Subject: Re: [dpdk-dev] [RFC PATCH v3 3/6] build: automatic NUMA and cpu counts detection X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Mon, Nov 02, 2020 at 07:01:44PM +0000, Honnappa Nagarahalli wrote: > > > > > > > Part of the confusion arises from the fact that originally that was the only > > parameter set by this - and on x86 it still is. Perhaps this parameter needs to > Just wondering, for x86, what does it mean if we set the max_num_cores and max_numa_nodes based on dynamic detection for 'native' build? > ISA still remains the same as before. But, the build might not work on machines with higher number of cores and numa nodes. > At the same time, the build also might not work on a machine with a different ISA. The users need to be aware that the target machine has the same ISA and same number of cores/numa nodes as the target machine. > Yes, that is a fair summary. > > be renamed to "isa-level" or "architecture-flag" or similar to reflect its > > meaning. This would then allow a new "machine" setting, which can be > > considered separately. The question then is how much that helps with the > > main issue under discussion, that of cores and numa node values. > If we rename it, we will have backward compatibility issue (i.e. 'native' build on x86 will have different meaning and whoever wants the original meaning, have to change to using this new name). Not sure about the complexity in meson scripts. > Yep, it was just a thought to see if it could help in this situation. > > > > > > But, I think other DPDK specific parameters should also be considered. > > > For ex: RTE_MAX_LCORE should have a particular value for 'generic' build in > > all the supported architectures. The value could be different for each > > architecture, but it is fixed for the 'generic' build for a given architecture. > > Otherwise, the 'generic' build might not run on all the machines of that > > architecture. > > > > > > Similarly, for 'native' build, is there any reason not to include other DPDK > > parameters as part of the definition? IMO, 'native' should refer to the entire > > build machine, not just the ISA. i.e. build on the target machine. > > > > > > > While I understand the idea here, it is somewhat complicated by the fact that > > the meson options given in "meson_options.txt" cannot be set by meson > > code, which means that when we change the machine flag to "native" we > > can only use or ignore the user-provided lcores and numa nodes setting - we > > have no way to change them and reflect those changes back to the user. :-( > > This leads to the situation in the discussion in this thread, where we start > > needing all sorts of magic values to indicate use of machine-type defaults or > > detected defaults. > I am wondering why we need to take the max_num_cores and max_numa_nodes from the user? This option was not provided in the make build system. I ask this question because for 'generic' this has to be a static/known configuration. For cross builds, this info can come (or derived) from the cross build file. > Was it supposed to be used in conjunction with 'native' build? > Well, it was configurable in the build config files same as all other DPDK build settings with make. When working first on meson, I felt it was a setting the user might be likely to want to tune, which is why I put it into the meson_options.txt and nobody suggested otherwise on review [which is the reason why many of the current options are the way they are :-)]. >From my side, I have a couple of unknowns: 1. How big a difference in terms of memory use etc. of DPDK does it make by having really big values for these core/numa counts? If there is not much difference, then there is indeed little value in having them configurable at all, and we should just use big defaults and be done with it. 2. If there is a noticable difference in these settings, how many users are going to want to actually go to the trouble of tweaking these? 3. How big an effort is it to switch to having these settings made entirely dynamic at runtime? Doing so would naturally make the need for these settings completely go away. With all that said, I'd be ok with a number of solutions. I'm ok to have these dropped as meson options and just have them specified in other ways, e.g. cross-file, or from meson.build files. [For x86, I'd tend towards having them defined in rte_config.h inside x86-specific ifdefs]. Alternatively, I'm also happy enough with the proposed scheme here of allowing user override, with platform defaults using "0"-value and detection using "-1". Regards, /Bruce