From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id C35BBA04B7; Tue, 13 Oct 2020 16:31:57 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id C0A7E1BD1F; Tue, 13 Oct 2020 16:31:55 +0200 (CEST) Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) by dpdk.org (Postfix) with ESMTP id 343921BC8B for ; Tue, 13 Oct 2020 16:31:54 +0200 (CEST) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.nyi.internal (Postfix) with ESMTP id C1AB75C00B8; Tue, 13 Oct 2020 10:31:52 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Tue, 13 Oct 2020 10:31:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s=fm2; bh= a5+0t0Ub52HJUkw9jPhuEakz2jRHyYTEpKsAFpoPmks=; b=ZNGBDnsiIUi0VC+R pZGkM+k5c7T39LBjXRa4bR3MEOEcf+0cDb1AoMjj89aHhZLH/EGxoVIQSKiVj/h1 9ZSmEsVuQMu7XnqBolO5xbjA9oyNqp4YASVxKtYib7enIMxlp8pFLlrdp+0/GMUD 9Vk8PYEglTOu5yD2Mt5pzEym4fXB5FsA6/4sS6XGlsJam8v+GO/jusOA1tPMtnec g+Iu4ZLqemC+V6Af0nRYdVZDrL6bkYCiGyIfaoaWp0OfAuM8TEtemI9WPVYBG7Md +H8lG6Nyl4xCb++8rVGg4pgS1tWBX888DE6k6wbhG8TWE4/Z0HRGUsmhJGCsFkws bn7s9g== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; bh=a5+0t0Ub52HJUkw9jPhuEakz2jRHyYTEpKsAFpoPm ks=; b=TJ9VGNmxvTdC6tVq0ksf4y3SUzE2iMAWMM6+00r6hT7Fg+1pVQfK9eZW5 mBA2ON8MIfd9gdr3Dom7wcuDrIq6BF6pijbURZMKsLCS2oHSp0OXveHei1h2d4ca HfOshMccgSXteOO8noDdUv1bhTC+17jHliaZObgD+NKB7mmD27mV8+aa9zjcy/jw 4UkFoqhwVBPX0uTD6BlTUp0nT5E0y1PRJgjZfoY1UzK/QF9cgoWXmhRwRMK8GLvL MPpfyQUAoaI6rzKN6pYeglCMUv5NLw/Y53doCGYWgRDwvqsLhldfcszT/uukV725 6R3xWiZ5lTb7nTwKVhX6JEBJEFheQ== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedujedrheelgdejkecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvffufffkjghfggfgtgesthhqredttddtjeenucfhrhhomhepvfhhohhmrghs ucfoohhnjhgrlhhonhcuoehthhhomhgrshesmhhonhhjrghlohhnrdhnvghtqeenucggtf frrghtthgvrhhnpedvkeetveeihfegfedtfeejueekkeekueevgfejuedviedvvdevuefg teevtdefveenucffohhmrghinhepughpughkrdhorhhgnecukfhppeejjedrudefgedrvd dtfedrudekgeenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhr ohhmpehthhhomhgrshesmhhonhhjrghlohhnrdhnvght X-ME-Proxy: Received: from xps.localnet (184.203.134.77.rev.sfr.net [77.134.203.184]) by mail.messagingengine.com (Postfix) with ESMTPA id 58BA7328005A; Tue, 13 Oct 2020 10:31:50 -0400 (EDT) From: Thomas Monjalon To: Dharmik Thakkar Cc: Juraj =?utf-8?B?TGlua2XFoQ==?= , dev@dpdk.org, Stephen Hemminger , Jerin Jacob , nd Date: Tue, 13 Oct 2020 16:31:49 +0200 Message-ID: <12050806.9cV25Y2rms@thomas> In-Reply-To: References: <20200825211317.8358-1-dharmik.thakkar@arm.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="UTF-8" Subject: Re: [dpdk-dev] [PATCH 2/2] build: find max lcore programmatically X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Please, what is the conclusion here? 18/09/2020 07:47, Dharmik Thakkar: >=20 > > On Sep 17, 2020, at 4:56 AM, Juraj Linke=C5=A1 wrote: > >=20 > >=20 > >=20 > >> -----Original Message----- > >> From: Dharmik Thakkar > >> Sent: Friday, September 4, 2020 7:44 AM > >> To: Stephen Hemminger > >> Cc: Juraj Linke=C5=A1 ; Jerin Jacob > >> ; thomas@monjalon.net; dpdk-dev ; > >> nd > >> Subject: Re: [dpdk-dev] [PATCH 2/2] build: find max lcore programmatic= ally > >>=20 > >>=20 > >>=20 > >>> On Sep 3, 2020, at 5:52 PM, Stephen Hemminger > >> wrote: > >>>=20 > >>> On Thu, 3 Sep 2020 06:20:17 +0000 > >>> Juraj Linke=C5=A1 wrote: > >>>=20 > >>>>> -----Original Message----- > >>>>> From: dev On Behalf Of Dharmik Thakkar > >>>>> Sent: Wednesday, August 26, 2020 6:56 AM > >>>>> To: Jerin Jacob > >>>>> Cc: thomas@monjalon.net; dpdk-dev ; nd > >>>>> Subject: Re: [dpdk-dev] [PATCH 2/2] build: find max lcore > >>>>> programmatically > >>>>>=20 > >>>>>=20 > >>>>>=20 > >>>>>> On Aug 25, 2020, at 11:47 PM, Jerin Jacob > >> wrote: > >>>>>>=20 > >>>>>> On Wed, Aug 26, 2020 at 2:44 AM Dharmik Thakkar > >>>>> wrote: > >>>>>>>=20 > >>>>>>> For Arm, RTE_MAX_LCORE is hard-coded into the config. It leads to > >>>>>>> incorrect RTE_MAX_LCORE when machines have same Implemener and > >>>>>>> part number but different number of CPUs. > >>>>>>> For x86, RTE_MAX_LCORE is always set to 128 (using the value set > >>>>>>> in > >>>>>>> meson_options.txt) > >>>>>>>=20 > >>>>>>> Use python script to find max lcore when using native build to > >>>>>>> correctly set RTE_MAX_LCORE. > >>>>>>=20 > >>>>>> We may need to build on the native arm64 machine and use it on > >>>>>> another > >>>>>> arm64 machine(Just like x86). > >>>>>> So I think, at least for default config(which will be used by > >>>>>> distribution) to support max > >>>>>> lcores as fixed. I am not sure this patch changes those aspects or > >>>>>> not? Please check. > >>>>>=20 > >>>>> This patch does *not* affect =E2=80=98default=E2=80=99 build type a= nd cross-compilation. > >>>>>=20 > >>>>>>=20 > >>>>>>>=20 > >>>>>>> Signed-off-by: Dharmik Thakkar > >>>>>>> Reviewed-by: Ruifeng Wang > >>>>>>> --- > >>>>>>> config/get_max_lcores.py | 13 +++++++++++++ > >>>>>>> config/meson.build | 13 ++++++++++++- > >>>>>>> 2 files changed, 25 insertions(+), 1 deletion(-) create mode > >>>>>>> 100755 config/get_max_lcores.py > >>>>>>>=20 > >>>>>>> diff --git a/config/get_max_lcores.py b/config/get_max_lcores.py > >>>>>>> new file mode 100755 index 000000000000..ebf1c7efdadd > >>>>>>> --- /dev/null > >>>>>>> +++ b/config/get_max_lcores.py > >>>>>>> @@ -0,0 +1,13 @@ > >>>>>>> +#!/usr/bin/python3 > >>>>>>> +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2020 Arm > >>>>>>> +Limited > >>>>>>> + > >>>>>>> +import os > >>>>>>> + > >>>>>>> +max_lcores =3D [] > >>>>>>> + > >>>>>>> +nCPU =3D os.cpu_count() > >>>>>>> + > >>>>>>> +max_lcores.append(str(nCPU & 0xFFF)) # Number of CPUs > >>>>>>> + > >>>>>>> +print(' '.join(max_lcores)) > >>>>>>> diff --git a/config/meson.build b/config/meson.build index > >>>>>>> 6996e5cbeaa5..80c05bc15d2f 100644 > >>>>>>> --- a/config/meson.build > >>>>>>> +++ b/config/meson.build > >>>>>>> @@ -237,11 +237,22 @@ else # for 32-bit we need smaller reserved > >>>>>>> memory > >>>>> areas > >>>>>>> dpdk_conf.set('RTE_MAX_MEM_MB', 2048) endif > >>>>>>>=20 > >>>>>>> - > >>>>>>> compile_time_cpuflags =3D [] > >>>>>>> subdir(arch_subdir) > >>>>>>> dpdk_conf.set('RTE_COMPILE_TIME_CPUFLAGS', > >>>>>>> ','.join(compile_time_cpuflags)) > >>>>>>>=20 > >>>>>>> +# set max lcores > >>>>>>> +if machine !=3D 'default' and not meson.is_cross_build() > >>>>>>> + # The script returns max lcores > >>>>>>> + params =3D files('get_max_lcores.py') > >>>>>>> + cmd_out =3D run_command(params) > >>>>=20 > >>>> Have you considered running just a shell command, such as "nproc --a= ll"? > >>>=20 > >>> Is this really a good idea? > >>> For real distributions and NFV products, the build and runtime > >>> environment will usually be different even if on same CPU architectur= e. > >>>=20 > >>> In many cases there maybe a huge build machine (128 CPU) or in a > >>> container (reported as single cpu) even if not doing cross build. > >>=20 > >> That=E2=80=99s a great point, Stephen. IMO, this patch is useful when = building and > >> running natively. > >> For all other purposes (like the ones you mentioned), do you think it = is a good > >> idea to set RTE_MAX_LCORE using -Dmax_lcores? > >=20 > > We should only use this native builds, as that would be consistent with= the current meson build philosophy of "meson figuring as much as possible = on its own". Any build other than native implies the user wants to deviate = from the build machine. > >=20 >=20 > The MIDR value-based probing doesn=E2=80=99t quite work well for Arm IP (= currently being discussed on this patch: https://patches.dpdk.org/patch/769= 81/). >=20 > > One way to do this automatic core count is when max_lcores=3D0 (0 would= have the special meaning of "figure core count automatically"). We can set= that as default in meson_option.txt and then users will have the ability t= o set it to whatever they want, even for native builds. What do you think? > >=20 >=20 > Yes, agreed. >=20 > > Currently the -Dmax_lcores option doesn't work for arm builds (the valu= e of RTE_MAX_LCORE is overwritten in config/arm/meson.build). I believe the= patch tries to address this, but still, we need to be mindful of that. > >=20 > > Juraj