[dpdk-dev] [PATCH] devtools: select targets in build test

[dpdk-dev] [PATCH] devtools: select targets in build test

[David Marchand]

When a target compilation is broken, one way to skip the target is to
uninstall the associated toolchain.
But it is not always possible and you end up with hacking the script to
avoid this target until a fix is ready.

It is also often quicker to check a fix on a failing target before
checking compilation on all targets.

Introduce a variable to select targets.

Example:
$ DPDK_BUILD_TEST_TARGETS=build-x86-mingw \
    ./devtools/test-meson-builds.sh
ninja: Entering directory `/home/dmarchan/builds/build-x86-mingw'
[...]
Found ninja-1.10.1 at /usr/bin/ninja
[19/19] Linking target examples/dpdk-helloworld.exe

Signed-off-by: David Marchand <david.marchand@redhat.com>
---
 devtools/test-meson-builds.sh | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/devtools/test-meson-builds.sh b/devtools/test-meson-builds.sh
index c11ae87e0d..3e88e8291e 100755
--- a/devtools/test-meson-builds.sh
+++ b/devtools/test-meson-builds.sh
@@ -92,6 +92,15 @@ load_env () # <target compiler>
 	command -v $targetcc >/dev/null 2>&1 || return 1
 }
 
+target_is_selected()
+{
+	if [ -z "${DPDK_BUILD_TEST_TARGETS:-}" ]; then
+		return 0
+	fi
+	target_filter=" $DPDK_BUILD_TEST_TARGETS "
+	! [ "${target_filter##* $1 }" = "${target_filter}" ]
+}
+
 config () # <dir> <builddir> <meson options>
 {
 	dir=$1
@@ -149,6 +158,7 @@ install_target () # <builddir> <installdir>
 build () # <directory> <target cc | cross file> <ABI check> [meson options]
 {
 	targetdir=$1
+	target_is_selected $targetdir || return 0
 	shift
 	crossfile=
 	[ -r $1 ] && crossfile=$1 || targetcc=$1
@@ -271,6 +281,8 @@ for f in $srcdir/config/ppc/ppc* ; do
 	build $targetdir $f ABI $use_shared
 done
 
+target_is_selected build-x86-default || exit 0
+
 # Test installation of the x86-default target, to be used for checking
 # the sample apps build using the pkg-config file for cflags and libs
 load_env cc
-- 
2.23.0

Re: [dpdk-dev] [PATCH] devtools: select targets in build test

[Thomas Monjalon]

04/02/2021 14:34, David Marchand:
> When a target compilation is broken, one way to skip the target is to
> uninstall the associated toolchain.
> But it is not always possible and you end up with hacking the script to
> avoid this target until a fix is ready.
> 
> It is also often quicker to check a fix on a failing target before
> checking compilation on all targets.
> 
> Introduce a variable to select targets.
> 
> Example:
> $ DPDK_BUILD_TEST_TARGETS=build-x86-mingw \
>     ./devtools/test-meson-builds.sh

With this solution, you need to list all targets you want to compile.

An alternative could be to disable a target in the config file
based on the variable DPDK_TARGET set by load_env.
One hack, which does not need any change in the script I think,
is to set targetcc=disabled.
Or we could check a well defined variable after calling load-devel-config.

[...]
> +target_is_selected build-x86-default || exit 0

Why this line?

Re: [dpdk-dev] [PATCH] devtools: select targets in build test

[David Marchand]

On Thu, Feb 4, 2021 at 2:59 PM Thomas Monjalon <thomas@monjalon.net> wrote:
>
> 04/02/2021 14:34, David Marchand:
> > When a target compilation is broken, one way to skip the target is to
> > uninstall the associated toolchain.
> > But it is not always possible and you end up with hacking the script to
> > avoid this target until a fix is ready.
> >
> > It is also often quicker to check a fix on a failing target before
> > checking compilation on all targets.
> >
> > Introduce a variable to select targets.
> >
> > Example:
> > $ DPDK_BUILD_TEST_TARGETS=build-x86-mingw \
> >     ./devtools/test-meson-builds.sh
>
> With this solution, you need to list all targets you want to compile.

To fill the list, it is easy, with no understand of the script internals:
$ ls $HOME/builds
build-32b              build-arm64-dpaa        build-arm64-octeontx2
build-clang-static  build-gcc-static      build-x86-default
build-arm64-bluefield  build-arm64-host-clang  build-clang-shared
build-gcc-shared    build-ppc64le-power8  build-x86-mingw

>
> An alternative could be to disable a target in the config file
> based on the variable DPDK_TARGET set by load_env.
> One hack, which does not need any change in the script I think,
> is to set targetcc=disabled.
> Or we could check a well defined variable after calling load-devel-config.

A bit fragile since you are bound to this internal shell variable.
Putting logic in ~/.config/dpdk/devel.build is undocumented and more
tedious than passing an environment variable when running the script.


>
> [...]
> > +target_is_selected build-x86-default || exit 0
>
> Why this line?

If the build-x86-default was not compiled in this run because you did
not select it, the call to the install target after this check
triggers a compilation of this target.
This is not wanted from my pov, or at best confusing, because you
don't see anything with the default verbose:

$ DPDK_BUILD_TEST_TARGETS=build-x86-mingw ./devtools/test-meson-builds.sh
ninja: Entering directory `/home/dmarchan/builds/build-x86-mingw'
[...]
Found ninja-1.10.1 at /usr/bin/ninja
[19/19] Linking target examples/dpdk-helloworld.exe

^^ for some time you get no output, you have the impression the script
is stuck, while it is actually compiling the build-x86-default target.

Then,
## Building cmdline
## Building helloworld
## Building l2fwd
## Building l3fwd
## Building multi_process
## Building skeleton
## Building timer

Compiling those examples had nothing to do with the build-x86-mingw
target I was expecting.


-- 
David Marchand

Re: [dpdk-dev] [PATCH] devtools: select targets in build test

[Thomas Monjalon]

04/02/2021 15:48, David Marchand:
> On Thu, Feb 4, 2021 at 2:59 PM Thomas Monjalon <thomas@monjalon.net> wrote:
> >
> > 04/02/2021 14:34, David Marchand:
> > > When a target compilation is broken, one way to skip the target is to
> > > uninstall the associated toolchain.
> > > But it is not always possible and you end up with hacking the script to
> > > avoid this target until a fix is ready.
> > >
> > > It is also often quicker to check a fix on a failing target before
> > > checking compilation on all targets.
> > >
> > > Introduce a variable to select targets.
> > >
> > > Example:
> > > $ DPDK_BUILD_TEST_TARGETS=build-x86-mingw \
> > >     ./devtools/test-meson-builds.sh
> >
> > With this solution, you need to list all targets you want to compile.
> 
> To fill the list, it is easy, with no understand of the script internals:
> $ ls $HOME/builds
> build-32b              build-arm64-dpaa        build-arm64-octeontx2
> build-clang-static  build-gcc-static      build-x86-default
> build-arm64-bluefield  build-arm64-host-clang  build-clang-shared
> build-gcc-shared    build-ppc64le-power8  build-x86-mingw

Yes easy.

> > An alternative could be to disable a target in the config file
> > based on the variable DPDK_TARGET set by load_env.
> > One hack, which does not need any change in the script I think,
> > is to set targetcc=disabled.
> > Or we could check a well defined variable after calling load-devel-config.
> 
> A bit fragile since you are bound to this internal shell variable.
> Putting logic in ~/.config/dpdk/devel.build is undocumented and more
> tedious than passing an environment variable when running the script.

Yes

> > [...]
> > > +target_is_selected build-x86-default || exit 0
> >
> > Why this line?
> 
> If the build-x86-default was not compiled in this run because you did
> not select it, the call to the install target after this check
> triggers a compilation of this target.

OK, please add a comment.

> This is not wanted from my pov, or at best confusing, because you
> don't see anything with the default verbose:
> 
> $ DPDK_BUILD_TEST_TARGETS=build-x86-mingw ./devtools/test-meson-builds.sh
> ninja: Entering directory `/home/dmarchan/builds/build-x86-mingw'
> [...]
> Found ninja-1.10.1 at /usr/bin/ninja
> [19/19] Linking target examples/dpdk-helloworld.exe
> 
> ^^ for some time you get no output, you have the impression the script
> is stuck, while it is actually compiling the build-x86-default target.
> 
> Then,
> ## Building cmdline
> ## Building helloworld
> ## Building l2fwd
> ## Building l3fwd
> ## Building multi_process
> ## Building skeleton
> ## Building timer
> 
> Compiling those examples had nothing to do with the build-x86-mingw
> target I was expecting.