Re: [dpdk-dev] [PATCH v5] build: use platform option for generic and native

["Juraj Linkeš" <juraj.linkes@pantheon.tech>]

> -----Original Message-----
> From: Juraj Linkeš <juraj.linkes@pantheon.tech>
> Sent: Tuesday, April 20, 2021 10:08 AM
> To: thomas@monjalon.net; david.marchand@redhat.com;
> bruce.richardson@intel.com; Honnappa.Nagarahalli@arm.com;
> Ruifeng.Wang@arm.com
> Cc: dev@dpdk.org; Juraj Linkeš <juraj.linkes@pantheon.tech>
> Subject: [PATCH v5] build: use platform option for generic and native
> 
> The current meson option 'machine' should only specify the ISA, which is not
> sufficient for Arm, where setting ISA implies other setting as well.
> Use the existing 'platform' meson option to differentiate the type of the build
> (native/generic) and set machine accordingly, unless the user chooses to
> override it.
> The 'machine' option also doesn't describe very well what it sets, so introduce a
> new option 'cpu_instruction_set', but keep 'machine' for backwards
> compatibility.
> 'machine' was setting the ISA in x86 builds and setting native/default 'build type'
> in aarch64 builds. These two new variables, taken together, are now properly
> setting both for all architectures in a uniform manner.
> 
> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> ---
>  config/arm/meson.build              |  6 ++--
>  config/meson.build                  | 53 ++++++++++++++++++++---------
>  devtools/test-meson-builds.sh       |  9 ++---
>  doc/guides/linux_gsg/build_dpdk.rst | 28 ++++++++++++++-
>  meson_options.txt                   | 10 +++---
>  5 files changed, 78 insertions(+), 28 deletions(-)
> 
> diff --git a/config/arm/meson.build b/config/arm/meson.build index
> 86cdb9b53b..59c788f1fa 100644
> --- a/config/arm/meson.build
> +++ b/config/arm/meson.build
> @@ -347,7 +347,7 @@ else
>  	soc = get_option('platform')
>  	soc_config = {}
>  	if not meson.is_cross_build()
> -		if machine == 'generic'
> +		if platform == 'generic'
>  			# generic build
>  			if soc != ''
>  				error('Building for a particular platform is ' +
> @@ -402,7 +402,7 @@ else
>  	else
>  		error('Unsupported Arm implementer: @0@.
> '.format(implementer_id) +
>  		      'Please add support for it or use the generic ' +
> -		      '(-Dmachine=generic) build.')
> +		      '(-Dplatform=generic) build.')
>  	endif
> 
>  	message('Arm implementer: ' + implementer_config['description'])
> @@ -417,7 +417,7 @@ else
>  		error('Unsupported part number @0@ of implementer @1@. '
>  		      .format(part_number, implementer_id) +
>  		      'Please add support for it or use the generic ' +
> -		      '(-Dmachine=generic) build.')
> +		      '(-Dplatform=generic) build.')
>  	endif
> 
>  	# add/overwrite flags in the proper order diff --git a/config/meson.build
> b/config/meson.build index 6e6ef8c0e1..1b6dcab8f5 100644
> --- a/config/meson.build
> +++ b/config/meson.build
> @@ -67,43 +67,64 @@ endif
>  disable_drivers = ''
>  enable_drivers = ''
> 
> -# set the machine type and cflags for it
> +platform = get_option('platform')
> +
> +# set the cpu_instruction_set and cflags for it
>  if meson.is_cross_build()
> -	machine = host_machine.cpu()
> +	cpu_instruction_set = host_machine.cpu()
>  else
> -	machine = get_option('machine')
> +	cpu_instruction_set = get_option('cpu_instruction_set')
> +	if get_option('machine') != 'auto'
> +		warning('The "machine" option is deprecated. ' +
> +		        'Please use "cpu_instruction_set" instead.')
> +		if cpu_instruction_set != 'auto'
> +			error('Setting both "machine" and ' +
> +			      '"cpu_instruction_set" is unsupported.')
> +		endif
> +		cpu_instruction_set = get_option('machine')
> +	endif
> +endif
> +
> +if platform == 'native'
> +	if cpu_instruction_set == 'auto'
> +		cpu_instruction_set = 'native'
> +	endif
> +elif platform == 'generic'
> +	if cpu_instruction_set == 'auto'
> +		cpu_instruction_set = 'generic'
> +	endif
>  endif
> 
> -# machine type 'generic' is special, it selects the per arch agreed common -#
> minimal baseline needed for DPDK. Machine type 'default' is also supported -#
> with the same meaning for backwards compatibility.
> +# cpu_instruction_set 'generic' is special, it selects the per arch
> +agreed # common minimal baseline needed for DPDK. cpu_instruction_set
> +'default' is # also supported with the same meaning for backwards
> compatibility.
>  # That might not be the most optimized, but the most portable version while  #
> still being able to support the CPU features required for DPDK.
>  # This can be bumped up by the DPDK project, but it can never be an  # invariant
> like 'native'
> -if machine == 'default' or machine == 'generic'
> +if cpu_instruction_set == 'generic' or cpu_instruction_set == 'default'
>  	if host_machine.cpu_family().startswith('x86')
> -		# matches the old pre-meson build systems generic machine
> -		machine = 'corei7'
> +		# matches the old pre-meson build systems generic
> cpu_instruction_set
> +		cpu_instruction_set = 'corei7'
>  	elif host_machine.cpu_family().startswith('arm')
> -		machine = 'armv7-a'
> +		cpu_instruction_set = 'armv7-a'
>  	elif host_machine.cpu_family().startswith('aarch')
>  		# arm64 manages generic config in config/arm/meson.build
> -		machine = 'generic'
> +		cpu_instruction_set = 'generic'
>  	elif host_machine.cpu_family().startswith('ppc')
> -		machine = 'power8'
> +		cpu_instruction_set = 'power8'
>  	endif
>  endif
> 
> -dpdk_conf.set('RTE_MACHINE', machine)
> +dpdk_conf.set('RTE_MACHINE', cpu_instruction_set)
>  machine_args = []
> 
>  # ppc64 does not support -march= at all, use -mcpu and -mtune for that  if
> host_machine.cpu_family().startswith('ppc')
> -	machine_args += '-mcpu=' + machine
> -	machine_args += '-mtune=' + machine
> +	machine_args += '-mcpu=' + cpu_instruction_set
> +	machine_args += '-mtune=' + cpu_instruction_set
>  else
> -	machine_args += '-march=' + machine
> +	machine_args += '-march=' + cpu_instruction_set
>  endif
> 
>  toolchain = cc.get_id()
> diff --git a/devtools/test-meson-builds.sh b/devtools/test-meson-builds.sh index
> daf817ac3e..7edbd6c717 100755
> --- a/devtools/test-meson-builds.sh
> +++ b/devtools/test-meson-builds.sh
> @@ -223,12 +223,13 @@ done
>  # test compilation with minimal x86 instruction set  # Set the install path for
> libraries to "lib" explicitly to prevent problems  # with pkg-config prefixes if
> installed in "lib/x86_64-linux-gnu" later.
> -generic_machine='nehalem'
> -if ! check_cc_flags "-march=$generic_machine" ; then
> -	generic_machine='corei7'
> +generic_isa='nehalem'
> +if ! check_cc_flags "-march=$generic_isa" ; then
> +	generic_isa='corei7'
>  fi
>  build build-x86-generic cc skipABI -Dcheck_includes=true \
> -	-Dlibdir=lib -Dmachine=$generic_machine $use_shared
> +	-Dlibdir=lib -Dcpu_instruction_set=$generic_isa \
> +	$use_shared
> 
>  # 32-bit with default compiler
>  if check_cc_flags '-m32' ; then
> diff --git a/doc/guides/linux_gsg/build_dpdk.rst
> b/doc/guides/linux_gsg/build_dpdk.rst
> index f78eef2517..787b2c2d1f 100644
> --- a/doc/guides/linux_gsg/build_dpdk.rst
> +++ b/doc/guides/linux_gsg/build_dpdk.rst
> @@ -89,7 +89,33 @@ to a regular "debug" build, you can either:
>  * run ``meson configure -Dbuildtype=debug`` inside the build folder after the
> initial meson run.
> 
>  Other options are specific to the DPDK project but can be adjusted similarly.
> -To set the "max_lcores" value to 256, for example, you can either:
> +The "platform" option specifies a set a configuration parameters that
> +will be used. The valid values are:
> +
> +* ``-Dplatform=native`` will tailor the configuration to the build machine.
> +
> +* ``-Dplatform=generic`` will use configuration that works on all
> +machines of the same architecture as the build machine.
> +
> +* ``-Dplatform=<Arm_SoC>`` will use configuration optimized for a
> +particular Arm SoC. Consult the "socs" dictionary in
> +config/arm/meson.build to see which SoC are supported.
> +
> +An important configuration parameter that "platform" sets is the
> +instruction set to use in x86 and ppc builds:
> +
> +* ``-Dplatform=native`` sets "cpu_instruction_set" to "native", which
> +configures -march, -mcpu, -mtune to "native".
> +
> +* ``-Dplatform=generic`` sets "cpu_instruction_set" to "generic", which
> +configures -march, -mcpu, -mtune to a common minimal baseline needed for
> DPDK.
> +
> +"cpu_instruction_set" is not used in Arm builds, as setting the
> +instruction set without other parameters leads to inferior builds. The
> +way to tailor Arm builds is to build for an SoC or a native build using the
> "platform" option.
> +
> +The values determined by the "platform" parameter may be overwritten.
> +For example, to set the "max_lcores" value to 256, you can either:
> 

I documented the new options in more detail here. I don't think this is the right place to document these, since this is only the Linux guide and the options should be os agnostic. I'm not sure where the better place to put this would be.

>  * pass ``-Dmax_lcores=256`` to meson when configuring the build folder initially
> 
> diff --git a/meson_options.txt b/meson_options.txt index
> b78f3bd9d5..07db5146db 100644
> --- a/meson_options.txt
> +++ b/meson_options.txt
> @@ -2,6 +2,8 @@
> 
>  option('check_includes', type: 'boolean', value: false,
>  	description: 'build "chkincs" to verify each header file can compile
> alone')
> +option('cpu_instruction_set', type: 'string', value: 'auto',
> +	description: 'Set the target machine ISA (instruction set
> +architecture). Will be set according to the platform option by
> +default.')
>  option('developer_mode', type: 'feature',
>  	description: 'turn on additional build checks relevant for DPDK
> developers')  option('disable_drivers', type: 'string', value: '', @@ -26,16 +28,16
> @@ option('include_subdir_arch', type: 'string', value: '',
>  	description: 'subdirectory where to install arch-dependent headers')
> option('kernel_dir', type: 'string', value: '',
>  	description: 'Path to the kernel for building kernel modules. Headers
> must be in $kernel_dir or $kernel_dir/build. Modules will be installed in
> /lib/modules.') -option('machine', type: 'string', value: 'native',
> -	description: 'set the target machine type or "generic", a build usable on
> all machines of the build machine architecture or "native", which lets the
> compiler pick the architecture of the build machine.')
> +option('machine', type: 'string', value: 'auto',
> +	description: 'Alias of cpu_instruction_set.')
>  option('max_ethports', type: 'integer', value: 32,
>  	description: 'maximum number of Ethernet devices')
> option('max_lcores', type: 'integer', value: 128,
>  	description: 'maximum number of cores/threads supported by EAL')
> option('max_numa_nodes', type: 'integer', value: 32,
>  	description: 'maximum number of NUMA nodes supported by EAL') -
> option('platform', type: 'string', value: '',
> -	description: 'use configuration for a particular platform (such as a SoC).')
> +option('platform', type: 'string', value: 'generic',
> +	description: 'Platform to build, either "native", "generic" or an Arm
> +SoC. Please refer to the Linux build guide for more information.')

The other thing we need to discuss is the fact that we're changing the behavior in this patch from machine=native to platform=generic (and the implied cpu_instruction_set=genetic which equals to machine=generic). We need more feedback on this - Thomas, David? Should we include more people? The reason for this change is we want the build without any meson options (specified on cmdline) to be usable on as many different machines of the same architecture (this is my understanding form the discussion with Bruce), which is the generic platform. We could preserve the current behavior by setting the default to native, so we just need more discussion.

>  option('enable_trace_fp', type: 'boolean', value: false,
>  	description: 'enable fast path trace points.')  option('tests', type:
> 'boolean', value: true,
> --
> 2.20.1