From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3D773A04B1; Thu, 5 Nov 2020 10:23:15 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 79C58A932; Thu, 5 Nov 2020 10:23:13 +0100 (CET) Received: from mailgw01.pantheon.sk (mailgw01.pantheon.sk [46.229.239.26]) by dpdk.org (Postfix) with ESMTP id A68B8A8E9 for ; Thu, 5 Nov 2020 10:23:10 +0100 (CET) Received: from mailgw01.pantheon.sk (localhost.localdomain [127.0.0.1]) by mailgw01.pantheon.sk (Proxmox) with ESMTP id 4F82C2027CA; Thu, 5 Nov 2020 10:23:09 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pantheon.tech; h=cc:cc:content-transfer-encoding:content-type:content-type :date:from:from:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to; s=dkim; bh=mLAahPkZPVQ8trZ5wRI+ GUukVpba5pOKl1lToNA56hU=; b=pXtTL8O8CgEtmFH/BraFvpe95qAfDMAqBs1P Sn4VOYNbi7ZukZHr/AxBUdTulGEK4f8j7hx9rPuxRyDR0onSjcJROuWfR0e6A7Yq QY/MZfhToxZ8/R7AG2MCAe1tJidiG2W66hTxZoP/zRRXo1vWZQUzROJ2Gu6AQFy3 pm5qo2IzvH4gILJMFUokBMsUlHYncDYpyLgqUbM3bdm6Wr+E4ySoxGPo3bBZaDjG FQgwmp/Bk1DGLDOrKkve4p9NrGVDeDfKvoWTWuvKI4+hJ9xvix/O9ANdRkV44G7r IWcvn8k1/8dY2L0VwcUYMfs0s5gEKHZeD9EnBrULsVPxehIIlg== From: =?iso-8859-2?Q?Juraj_Linke=B9?= To: Bruce Richardson , Honnappa Nagarahalli CC: Ruifeng Wang , Phil Yang , "vcchunga@amazon.com" , Dharmik Thakkar , "jerinjacobk@gmail.com" , "hemant.agrawal@nxp.com" , "dev@dpdk.org" , nd Thread-Topic: [RFC PATCH v3 3/6] build: automatic NUMA and cpu counts detection Thread-Index: AQHWp56fojjDhTEMUUC082zv7zxFhKmh00kAgAAlvLD///7dgIAMAN+AgAbm2ACAAFWcAIAA9rcAgAMalUA= Date: Thu, 5 Nov 2020 09:23:08 +0000 Message-ID: <8a8ef54d7ab64f62b1bc180fcc6b2939@pantheon.tech> References: <1602600882-695-1-git-send-email-juraj.linkes@pantheon.tech> <1603280261-20206-1-git-send-email-juraj.linkes@pantheon.tech> <1603280261-20206-4-git-send-email-juraj.linkes@pantheon.tech> <20201021120220.GC592@bricha3-MOBL.ger.corp.intel.com> <54e20244e1e841148669eb93b6b8876c@pantheon.tech> <20201021141319.GD592@bricha3-MOBL.ger.corp.intel.com> <20201102135520.GA1454@bricha3-MOBL.ger.corp.intel.com> <20201103094446.GA1132@bricha3-MOBL.ger.corp.intel.com> In-Reply-To: <20201103094446.GA1132@bricha3-MOBL.ger.corp.intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.101.4.10] Content-Type: text/plain; charset="iso-8859-2" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [RFC PATCH v3 3/6] build: automatic NUMA and cpu counts detection X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > -----Original Message----- > From: Bruce Richardson > Sent: Tuesday, November 3, 2020 10:45 AM > To: Honnappa Nagarahalli > Cc: Juraj Linke=B9 ; Ruifeng Wang > ; Phil Yang ; > vcchunga@amazon.com; Dharmik Thakkar ; > jerinjacobk@gmail.com; hemant.agrawal@nxp.com; dev@dpdk.org; nd > > Subject: Re: [RFC PATCH v3 3/6] build: automatic NUMA and cpu counts > detection >=20 > On Mon, Nov 02, 2020 at 07:01:44PM +0000, Honnappa Nagarahalli wrote: > > > > > > > > > > > Part of the confusion arises from the fact that originally that was > > > the only parameter set by this - and on x86 it still is. Perhaps > > > this parameter needs to > > Just wondering, for x86, what does it mean if we set the max_num_cores = and > max_numa_nodes based on dynamic detection for 'native' build? > > ISA still remains the same as before. But, the build might not work on > machines with higher number of cores and numa nodes. > > At the same time, the build also might not work on a machine with a dif= ferent > ISA. The users need to be aware that the target machine has the same ISA = and > same number of cores/numa nodes as the target machine. > > > Yes, that is a fair summary. >=20 There's also additional confusion in what the build should produce with no = user input. You mentioned that builds from CI are used outside of CI and wh= en a user compiles up a binary without settings these [cpu and numa counts]= explicitly it should run on 95%+ of systems of that type (why not 100%?). = These requirements/expectations suggest that generic settings should be the= default (machine=3Dgeneric so that CI builds are usable on machines with d= ifferent ISA and then using generic defaults for other settings (we're talk= ing mainly about cpu and numa counts, but it should apply to everything - t= here are more settings that need generic defaults for Arm)). If we change the default to generic, then setting machine=3Dnative would on= ly set the machine args and not change the behavior of other vars (i.e. use= defaults for the build machine). > > > be renamed to "isa-level" or "architecture-flag" or similar to > > > reflect its meaning. This would then allow a new "machine" setting, > > > which can be considered separately. The question then is how much > > > that helps with the main issue under discussion, that of cores and nu= ma node > values. > > If we rename it, we will have backward compatibility issue (i.e. 'nativ= e' build on > x86 will have different meaning and whoever wants the original meaning, h= ave > to change to using this new name). Not sure about the complexity in meson > scripts. > > >=20 > Yep, it was just a thought to see if it could help in this situation. >=20 I also don't think renaming or repurposing the machine meson option is the = way to go, not only because of backwards compatibility but also because the= option sets RTE_MACHINE and machine args, which lines up with the name nic= ely. As a side note, I grepped RTE_MACHINE and I don't see it being used an= ywhere - do we need that config? But this doesn't work for Arm, since setting just machine args doesn't make= sense - setting that also necessitates setting of some other dpdk config a= s well. What could help is new meson option. I'm working on adding an option where = the user could specify the set of target configuration, or soc configuratio= n. I could add the configuration for generic build as one of the available = sets and then we'd be able to use that variable instead of machine for Arm = builds. > > > > > > > > > But, I think other DPDK specific parameters should also be consider= ed. > > > > For ex: RTE_MAX_LCORE should have a particular value for 'generic' > > > > build in > > > all the supported architectures. The value could be different for > > > each architecture, but it is fixed for the 'generic' build for a give= n > architecture. > > > Otherwise, the 'generic' build might not run on all the machines of > > > that architecture. > > > > > > > > Similarly, for 'native' build, is there any reason not to include > > > > other DPDK > > > parameters as part of the definition? IMO, 'native' should refer to > > > the entire build machine, not just the ISA. i.e. build on the target = machine. > > > > > > > > > > While I understand the idea here, it is somewhat complicated by the > > > fact that the meson options given in "meson_options.txt" cannot be > > > set by meson code, which means that when we change the machine flag > > > to "native" we can only use or ignore the user-provided lcores and > > > numa nodes setting - we have no way to change them and reflect those > > > changes back to the user. :-( This leads to the situation in the > > > discussion in this thread, where we start needing all sorts of magic > > > values to indicate use of machine-type defaults or detected defaults. > > I am wondering why we need to take the max_num_cores and > max_numa_nodes from the user? This option was not provided in the make bu= ild > system. I ask this question because for 'generic' this has to be a static= /known > configuration. For cross builds, this info can come (or derived) from the= cross > build file. > > Was it supposed to be used in conjunction with 'native' build? > > >=20 > Well, it was configurable in the build config files same as all other DPD= K build > settings with make. When working first on meson, I felt it was a setting = the user > might be likely to want to tune, which is why I put it into the meson_opt= ions.txt > and nobody suggested otherwise on review [which is the reason why many of > the current options are the way they are :-)]. >=20 > From my side, I have a couple of unknowns: > 1. How big a difference in terms of memory use etc. of DPDK does it make = by > having really big values for these core/numa counts? If there is not m= uch > difference, then there is indeed little value in having them configura= ble > at all, and we should just use big defaults and be done with it. > 2. If there is a noticable difference in these settings, how many users a= re > going to want to actually go to the trouble of tweaking these? > 3. How big an effort is it to switch to having these settings made entire= ly > dynamic at runtime? Doing so would naturally make the need for these > settings completely go away. >=20 > With all that said, I'd be ok with a number of solutions. I'm ok to have = these > dropped as meson options and just have them specified in other ways, e.g. > cross-file, or from meson.build files. [For x86, I'd tend towards having = them > defined in rte_config.h inside x86-specific ifdefs]. > Alternatively, I'm also happy enough with the proposed scheme here of all= owing > user override, with platform defaults using "0"-value and detection using= "-1". >=20 > Regards, > /Bruce