[dpdk-dev] [PATCH 0/4] add PPC and Windows to meson test

[dpdk-dev] [PATCH 0/4] add PPC and Windows to meson test

[Thomas Monjalon]

In order to better support PPC and Windows,
their compilation is tested on Linux with Meson
with the script test-meson-builds.sh,
supposed to be called in every CI labs.

Thomas Monjalon (4):
  devtools: shrink cross-compilation test definition
  devtools: allow non-standard toolchain in meson test
  devtools: add ppc64 in meson build test
  devtools: add Windows cross-build test with MinGW

 config/ppc/ppc64le-power8-linux-gcc         | 11 ++++++
 config/x86/{meson_mingw.txt => cross-mingw} |  0
 devtools/test-meson-builds.sh               | 42 +++++++++++++++------
 doc/guides/windows_gsg/build_dpdk.rst       |  2 +-
 4 files changed, 43 insertions(+), 12 deletions(-)
 create mode 100644 config/ppc/ppc64le-power8-linux-gcc
 rename config/x86/{meson_mingw.txt => cross-mingw} (100%)

-- 
2.26.2

[dpdk-dev] [PATCH 1/4] devtools: shrink cross-compilation test definition

[Thomas Monjalon]

Each cross-compilation case needs to define the target compiler
and the meson cross file.
Given the compiler is already defined in the cross file,
the latter is enough.

The function "build" is changed to accept a cross file alternatively
to the compiler name. In the case of a file (detected if readable),
the compiler is extracted with sed and tr, and the option --cross-file
is automatically added.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 devtools/test-meson-builds.sh | 24 +++++++++++++++---------
 1 file changed, 15 insertions(+), 9 deletions(-)

diff --git a/devtools/test-meson-builds.sh b/devtools/test-meson-builds.sh
index 18b874fac5..602167e43a 100755
--- a/devtools/test-meson-builds.sh
+++ b/devtools/test-meson-builds.sh
@@ -117,16 +117,24 @@ install_target () # <builddir> <installdir>
 	fi
 }
 
-build () # <directory> <target compiler> <meson options>
+build () # <directory> <target compiler | cross file> <meson options>
 {
 	targetdir=$1
 	shift
-	targetcc=$1
+	crossfile=
+	[ -r $1 ] && crossfile=$1 || targetcc=$1
 	shift
 	# skip build if compiler not available
 	command -v ${CC##* } >/dev/null 2>&1 || return 0
+	if [ -n "$crossfile" ] ; then
+		cross="--cross-file $crossfile"
+		targetcc=$(sed -n 's,^c[[:space:]]*=[[:space:]]*,,p' \
+			$crossfile | tr -d "'" | tr -d '"')
+	else
+		cross=
+	fi
 	load_env $targetcc || return 0
-	config $srcdir $builds_dir/$targetdir --werror $*
+	config $srcdir $builds_dir/$targetdir $cross --werror $*
 	compile $builds_dir/$targetdir
 	if [ -n "$DPDK_ABI_REF_VERSION" ]; then
 		abirefdir=${DPDK_ABI_REF_DIR:-reference}/$DPDK_ABI_REF_VERSION
@@ -186,17 +194,15 @@ if [ "$ok" = "false" ] ; then
 fi
 build build-x86-default cc -Dlibdir=lib -Dmachine=$default_machine $use_shared
 
-c=aarch64-linux-gnu-gcc
 # generic armv8a with clang as host compiler
+f=$srcdir/config/arm/arm64_armv8_linux_gcc
 export CC="clang"
-build build-arm64-host-clang $c $use_shared \
-	--cross-file $srcdir/config/arm/arm64_armv8_linux_gcc
+build build-arm64-host-clang $f $use_shared
 unset CC
-# all gcc/arm configurations
+# some gcc/arm configurations
 for f in $srcdir/config/arm/arm64_[bdo]*gcc ; do
 	export CC="$CCACHE gcc"
-	build build-$(basename $f | tr '_' '-' | cut -d'-' -f-2) $c \
-		$use_shared --cross-file $f
+	build build-$(basename $f | tr '_' '-' | cut -d'-' -f-2) $f $use_shared
 	unset CC
 done
 
-- 
2.26.2

[dpdk-dev] [PATCH 2/4] devtools: allow non-standard toolchain in meson test

[Thomas Monjalon]

If a compiler is not found in $PATH, the compilation test is skipped.
In some cases, the compiler could be found after extending $PATH
in an environment configuration script (called by load-devel-config).

The decision to skip is deferred to a later stage, after loading the
configuration script.

In such case, the variable DPDK_TARGET, used by the configuration script
as input, is the compiler name.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 devtools/test-meson-builds.sh | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/devtools/test-meson-builds.sh b/devtools/test-meson-builds.sh
index 602167e43a..1d80a029aa 100755
--- a/devtools/test-meson-builds.sh
+++ b/devtools/test-meson-builds.sh
@@ -52,9 +52,15 @@ load_env () # <target compiler>
 	export CFLAGS=$default_cflags
 	export LDFLAGS=$default_ldflags
 	unset DPDK_MESON_OPTIONS
-	command -v $targetcc >/dev/null 2>&1 || return 1
-	DPDK_TARGET=$($targetcc -v 2>&1 | sed -n 's,^Target: ,,p')
+	if command -v $targetcc >/dev/null 2>&1 ; then
+		DPDK_TARGET=$($targetcc -v 2>&1 | sed -n 's,^Target: ,,p')
+	else # toolchain not yet in PATH: its name should be enough
+		DPDK_TARGET=$targetcc
+	fi
+	# config input: $DPDK_TARGET
 	. $srcdir/devtools/load-devel-config
+	# config output: $DPDK_MESON_OPTIONS, $PATH, $PKG_CONFIG_PATH, etc
+	command -v $targetcc >/dev/null 2>&1 || return 1
 }
 
 config () # <dir> <builddir> <meson options>
-- 
2.26.2

[dpdk-dev] [PATCH 3/4] devtools: add ppc64 in meson build test

[Thomas Monjalon]

The CPU is defined as Power8, running as little endian.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 config/ppc/ppc64le-power8-linux-gcc | 11 +++++++++++
 devtools/test-meson-builds.sh       |  5 +++++
 2 files changed, 16 insertions(+)
 create mode 100644 config/ppc/ppc64le-power8-linux-gcc

diff --git a/config/ppc/ppc64le-power8-linux-gcc b/config/ppc/ppc64le-power8-linux-gcc
new file mode 100644
index 0000000000..51f7ceebf3
--- /dev/null
+++ b/config/ppc/ppc64le-power8-linux-gcc
@@ -0,0 +1,11 @@
+[binaries]
+c = 'powerpc64le-linux-gcc'
+cpp = 'powerpc64le-linux-cpp'
+ar = 'powerpc64le-linux-gcc-ar'
+strip = 'powerpc64le-linux-strip'
+
+[host_machine]
+system = 'linux'
+cpu_family = 'ppc64'
+cpu = 'power8'
+endian = 'little'
diff --git a/devtools/test-meson-builds.sh b/devtools/test-meson-builds.sh
index 1d80a029aa..1cde17a2e5 100755
--- a/devtools/test-meson-builds.sh
+++ b/devtools/test-meson-builds.sh
@@ -212,6 +212,11 @@ for f in $srcdir/config/arm/arm64_[bdo]*gcc ; do
 	unset CC
 done
 
+# ppc configurations
+for f in $srcdir/config/ppc/ppc* ; do
+	build build-$(basename $f | cut -d'-' -f-2) $f $use_shared
+done
+
 # Test installation of the x86-default target, to be used for checking
 # the sample apps build using the pkg-config file for cflags and libs
 build_path=$(readlink -f $builds_dir/build-x86-default)
-- 
2.26.2

[dpdk-dev] [PATCH 4/4] devtools: add Windows cross-build test with MinGW

[Thomas Monjalon]

The Meson cross file is renamed from meson_mingw.txt to cross-mingw,
and is added to test-meson-builds.sh.

The only example supported on Windows so far is "helloworld",
that's why the default list of examples is overriden.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 config/x86/{meson_mingw.txt => cross-mingw} | 0
 devtools/test-meson-builds.sh               | 3 +++
 doc/guides/windows_gsg/build_dpdk.rst       | 2 +-
 3 files changed, 4 insertions(+), 1 deletion(-)
 rename config/x86/{meson_mingw.txt => cross-mingw} (100%)

diff --git a/config/x86/meson_mingw.txt b/config/x86/cross-mingw
similarity index 100%
rename from config/x86/meson_mingw.txt
rename to config/x86/cross-mingw
diff --git a/devtools/test-meson-builds.sh b/devtools/test-meson-builds.sh
index 1cde17a2e5..52635c73d1 100755
--- a/devtools/test-meson-builds.sh
+++ b/devtools/test-meson-builds.sh
@@ -200,6 +200,9 @@ if [ "$ok" = "false" ] ; then
 fi
 build build-x86-default cc -Dlibdir=lib -Dmachine=$default_machine $use_shared
 
+# x86 MinGW
+build build-x86-mingw $srcdir/config/x86/cross-mingw -Dexamples=helloworld
+
 # generic armv8a with clang as host compiler
 f=$srcdir/config/arm/arm64_armv8_linux_gcc
 export CC="clang"
diff --git a/doc/guides/windows_gsg/build_dpdk.rst b/doc/guides/windows_gsg/build_dpdk.rst
index d46e84e3fb..d517f08f0d 100644
--- a/doc/guides/windows_gsg/build_dpdk.rst
+++ b/doc/guides/windows_gsg/build_dpdk.rst
@@ -109,7 +109,7 @@ Depending on the distribution, paths in this file may need adjustments.
 
 .. code-block:: console
 
-    meson --cross-file config/x86/meson_mingw.txt -Dexamples=helloworld build
+    meson --cross-file config/x86/cross-mingw -Dexamples=helloworld build
     ninja -C build
 
 
-- 
2.26.2

Re: [dpdk-dev] [PATCH 4/4] devtools: add Windows cross-build test with MinGW

[Thomas Monjalon]

15/06/2020 00:57, Thomas Monjalon:
> The Meson cross file is renamed from meson_mingw.txt to cross-mingw,
> and is added to test-meson-builds.sh.
> 
> The only example supported on Windows so far is "helloworld",
> that's why the default list of examples is overriden.
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> ---

There is a warning with MinGW on Linux:

In file included from lib/librte_eal/include/rte_eal.h:15,
                 from lib/librte_eal/common/eal_common_options.c:26:
lib/librte_eal/common/eal_common_options.c:
	In function ‘eal_adjust_config’:
lib/librte_eal/windows/include/sched.h:63:55: error:
	‘default_set._bits[1]’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
   63 |   (dst)->_bits[_i] = (src1)->_bits[_i] & (src2)->_bits[_i]; \
      |                                                       ^


% x86_64-w64-mingw32-gcc --version
x86_64-w64-mingw32-gcc (GCC) 10.1.0


It looks to be a false positive, but we need to find a way to remove
the warning with a code change.

This workaround in lib/librte_eal/common/eal_common_options.c
makes 200% sure the default_set is initialized:

-       if (pthread_getaffinity_np(pthread_self(), sizeof(rte_cpuset_t),
-                               &default_set))
-               CPU_ZERO(&default_set);
-
+       CPU_ZERO(&default_set);
+       pthread_getaffinity_np(pthread_self(),
+                       sizeof default_set, &default_set);

Re: [dpdk-dev] [PATCH 4/4] devtools: add Windows cross-build test with MinGW

[Dmitry Kozlyuk]

> It looks to be a false positive, but we need to find a way to remove
> the warning with a code change.
> 
> This workaround in lib/librte_eal/common/eal_common_options.c
> makes 200% sure the default_set is initialized:
> 
> -       if (pthread_getaffinity_np(pthread_self(), sizeof(rte_cpuset_t),
> -                               &default_set))
> -               CPU_ZERO(&default_set);
> -
> +       CPU_ZERO(&default_set);
> +       pthread_getaffinity_np(pthread_self(),
> +                       sizeof default_set, &default_set);
>

Doesn't look like a false-positive to me. That's where it's been spotted
and explained before:

	https://mails.dpdk.org/archives/dev/2020-May/168634.html

And this series eliminates it along with other improvements:

	http://patchwork.dpdk.org/patch/70727/

-- 
Dmitry Kozlyuk

Re: [dpdk-dev] [PATCH 4/4] devtools: add Windows cross-build test with MinGW

[Thomas Monjalon]

15/06/2020 03:05, Dmitry Kozlyuk:
> > It looks to be a false positive, but we need to find a way to remove
> > the warning with a code change.
> > 
> > This workaround in lib/librte_eal/common/eal_common_options.c
> > makes 200% sure the default_set is initialized:
> > 
> > -       if (pthread_getaffinity_np(pthread_self(), sizeof(rte_cpuset_t),
> > -                               &default_set))
> > -               CPU_ZERO(&default_set);
> > -
> > +       CPU_ZERO(&default_set);
> > +       pthread_getaffinity_np(pthread_self(),
> > +                       sizeof default_set, &default_set);
> >
> 
> Doesn't look like a false-positive to me. That's where it's been spotted
> and explained before:
> 
> 	https://mails.dpdk.org/archives/dev/2020-May/168634.html
> 
> And this series eliminates it along with other improvements:
> 
> 	http://patchwork.dpdk.org/patch/70727/

I see.
So the fix must be explained (and probably a separate patch).

Re: [dpdk-dev] [PATCH 3/4] devtools: add ppc64 in meson build test

[David Christensen]

On 6/14/20 3:57 PM, Thomas Monjalon wrote:
> diff --git a/devtools/test-meson-builds.sh b/devtools/test-meson-builds.sh
> index 1d80a029aa..1cde17a2e5 100755
> --- a/devtools/test-meson-builds.sh
> +++ b/devtools/test-meson-builds.sh
> @@ -212,6 +212,11 @@ for f in $srcdir/config/arm/arm64_[bdo]*gcc ; do
>   	unset CC
>   done
> 
> +# ppc configurations
> +for f in $srcdir/config/ppc/ppc* ; do
> +	build build-$(basename $f | cut -d'-' -f-2) $f $use_shared
> +done
> +

The entire script file is poorly structured for multi-architecture 
builds.  The script fails on PPC64LE even before your changes:

------------------------------------
# shared and static linked builds with gcc and clang
for c in gcc clang ; do
        command -v $c >/dev/null 2>&1 || continue
        for s in static shared ; do
                export CC="$CCACHE $c"
                build build-$c-$s $c --default-library=$s
                unset CC
        done
done
------------------------------------

Compilation on clang is not currently supported due to build failures on 
Altivec data types for vectorization:

In file included from ../lib/librte_eal/common/eal_common_options.c:33:
In file included from ../lib/librte_eal/ppc/include/rte_memcpy.h:12:
In file included from ../lib/librte_eal/ppc/include/rte_altivec.h:10:
/usr/lib64/clang/9.0.1/include/altivec.h:55:19: error: unknown type name 
'vector'
static __inline__ vector bool char __ATTRS_o_ai
                   ^
/usr/lib64/clang/9.0.1/include/altivec.h:56:10: error: unknown type name 
'vector'
vec_perm(vector bool char __a, vector bool char __b, vector unsigned 
char __c);

and C99 atomics:

../lib/librte_eal/ppc/include/rte_atomic.h:85:9: error: implicit 
declaration of function '__atomic_exchange_2' is invalid in C99 
[-Werror,-Wimplicit-function-declaration]
         return __atomic_exchange_2(dst, val, __ATOMIC_SEQ_CST);
                ^

The script then immediately launches into an x86 build no matter what 
architecture the system is running on:

------------------------------------
default_machine='nehalem'
ok=$(cc -march=$default_machine -E - < /dev/null > /dev/null 2>&1 || 
echo false)
if [ "$ok" = "false" ] ; then
         default_machine='corei7'
fi
build build-x86-default cc -Dlibdir=lib -Dmachine=$default_machine 
$use_shared
------------------------------------

Which also fails on a Power system:

ninja -v -C ./build-x86-default
ninja: Entering directory `./build-x86-default'
[1/2265] cc -Ilib/76b5a35@@rte_kvargs@sta -Ilib -I../lib -I. -I../ 
-Iconfig -I../config -Ilib/librte_eal/include 
-I../lib/librte_eal/include -Ilib/librte_eal/linux/include 
-I../lib/librte_eal/linux/include -Ilib/librte_eal/ppc/include 
-I../lib/librte_eal/ppc/include -Ilib/librte_kvargs 
-I../lib/librte_kvargs -fdiagnostics-color=always -pipe 
-D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Werror -O2 -g -include 
rte_config.h -Wextra -Wcast-qual -Wdeprecated -Wformat-nonliteral 
-Wformat-security -Wmissing-declarations -Wmissing-prototypes 
-Wnested-externs -Wold-style-definition -Wpointer-arith -Wsign-compare 
-Wstrict-prototypes -Wundef -Wwrite-strings -Wno-packed-not-aligned 
-Wno-missing-field-initializers -D_GNU_SOURCE -fPIC -mcpu=corei7 
-mtune=corei7 -DALLOW_EXPERIMENTAL_API -DALLOW_INTERNAL_API 
-Wno-format-truncation  -MD -MQ 
'lib/76b5a35@@rte_kvargs@sta/librte_kvargs_rte_kvargs.c.o' -MF 
'lib/76b5a35@@rte_kvargs@sta/librte_kvargs_rte_kvargs.c.o.d' -o 
'lib/76b5a35@@rte_kvargs@sta/librte_kvargs_rte_kvargs.c.o' -c 
../lib/librte_kvargs/rte_kvargs.c
FAILED: lib/76b5a35@@rte_kvargs@sta/librte_kvargs_rte_kvargs.c.o


I think the logic of looping through supported configs for test builds 
is the right one, but more changes are required for multi-architecture 
support in this script.  For now I'd abandon the PPC/ARM support 
entirely in this script.

Dave

Re: [dpdk-dev] [PATCH 3/4] devtools: add ppc64 in meson build test

[Thomas Monjalon]

15/06/2020 23:43, David Christensen:
> On 6/14/20 3:57 PM, Thomas Monjalon wrote:
> > diff --git a/devtools/test-meson-builds.sh b/devtools/test-meson-builds.sh
> > index 1d80a029aa..1cde17a2e5 100755
> > --- a/devtools/test-meson-builds.sh
> > +++ b/devtools/test-meson-builds.sh
> > @@ -212,6 +212,11 @@ for f in $srcdir/config/arm/arm64_[bdo]*gcc ; do
> >   	unset CC
> >   done
> > 
> > +# ppc configurations
> > +for f in $srcdir/config/ppc/ppc* ; do
> > +	build build-$(basename $f | cut -d'-' -f-2) $f $use_shared
> > +done
> > +
> 
> The entire script file is poorly structured for multi-architecture 
> builds.  The script fails on PPC64LE even before your changes:

This script supports only builds from x86 Linux so far.
This patch is adding cross-compilation of ppc from x86,
and it works fine.


> ------------------------------------
> # shared and static linked builds with gcc and clang
> for c in gcc clang ; do
>         command -v $c >/dev/null 2>&1 || continue
>         for s in static shared ; do
>                 export CC="$CCACHE $c"
>                 build build-$c-$s $c --default-library=$s
>                 unset CC
>         done
> done
> ------------------------------------
> 
> Compilation on clang is not currently supported due to build failures on 
> Altivec data types for vectorization:
> 
> In file included from ../lib/librte_eal/common/eal_common_options.c:33:
> In file included from ../lib/librte_eal/ppc/include/rte_memcpy.h:12:
> In file included from ../lib/librte_eal/ppc/include/rte_altivec.h:10:
> /usr/lib64/clang/9.0.1/include/altivec.h:55:19: error: unknown type name 
> 'vector'
> static __inline__ vector bool char __ATTRS_o_ai
>                    ^
> /usr/lib64/clang/9.0.1/include/altivec.h:56:10: error: unknown type name 
> 'vector'
> vec_perm(vector bool char __a, vector bool char __b, vector unsigned 
> char __c);
> 
> and C99 atomics:
> 
> ../lib/librte_eal/ppc/include/rte_atomic.h:85:9: error: implicit 
> declaration of function '__atomic_exchange_2' is invalid in C99 
> [-Werror,-Wimplicit-function-declaration]
>          return __atomic_exchange_2(dst, val, __ATOMIC_SEQ_CST);
>                 ^
> 
> The script then immediately launches into an x86 build no matter what 
> architecture the system is running on:
> 
> ------------------------------------
> default_machine='nehalem'
> ok=$(cc -march=$default_machine -E - < /dev/null > /dev/null 2>&1 || 
> echo false)
> if [ "$ok" = "false" ] ; then
>          default_machine='corei7'
> fi
> build build-x86-default cc -Dlibdir=lib -Dmachine=$default_machine 
> $use_shared
> ------------------------------------
> 
> Which also fails on a Power system:
> 
> ninja -v -C ./build-x86-default
> ninja: Entering directory `./build-x86-default'
> [1/2265] cc -Ilib/76b5a35@@rte_kvargs@sta -Ilib -I../lib -I. -I../ 
> -Iconfig -I../config -Ilib/librte_eal/include 
> -I../lib/librte_eal/include -Ilib/librte_eal/linux/include 
> -I../lib/librte_eal/linux/include -Ilib/librte_eal/ppc/include 
> -I../lib/librte_eal/ppc/include -Ilib/librte_kvargs 
> -I../lib/librte_kvargs -fdiagnostics-color=always -pipe 
> -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Werror -O2 -g -include 
> rte_config.h -Wextra -Wcast-qual -Wdeprecated -Wformat-nonliteral 
> -Wformat-security -Wmissing-declarations -Wmissing-prototypes 
> -Wnested-externs -Wold-style-definition -Wpointer-arith -Wsign-compare 
> -Wstrict-prototypes -Wundef -Wwrite-strings -Wno-packed-not-aligned 
> -Wno-missing-field-initializers -D_GNU_SOURCE -fPIC -mcpu=corei7 
> -mtune=corei7 -DALLOW_EXPERIMENTAL_API -DALLOW_INTERNAL_API 
> -Wno-format-truncation  -MD -MQ 
> 'lib/76b5a35@@rte_kvargs@sta/librte_kvargs_rte_kvargs.c.o' -MF 
> 'lib/76b5a35@@rte_kvargs@sta/librte_kvargs_rte_kvargs.c.o.d' -o 
> 'lib/76b5a35@@rte_kvargs@sta/librte_kvargs_rte_kvargs.c.o' -c 
> ../lib/librte_kvargs/rte_kvargs.c
> FAILED: lib/76b5a35@@rte_kvargs@sta/librte_kvargs_rte_kvargs.c.o
> 
> 
> I think the logic of looping through supported configs for test builds 
> is the right one, but more changes are required for multi-architecture 
> support in this script.  For now I'd abandon the PPC/ARM support 
> entirely in this script.

I am adding PPC compilation check to avoid having PPC broken,
because after asking for many years to IBM execs,
there is still no PPC CI integrated in DPDK patchwork.
If you don't think this small step is going in the right direction,
then I will stop bothering about PPC support,
and we should probably completely drop PPC support I guess.

[dpdk-dev] [PATCH v2 0/4] add PPC and Windows cross-compilation to meson test

[Thomas Monjalon]

In order to better support PPC and Windows,
their compilation is tested on Linux with Meson
with the script test-meson-builds.sh,
supposed to be called in every CI labs.


Thomas Monjalon (4):
  devtools: shrink cross-compilation test definition
  devtools: allow non-standard toolchain in meson test
  devtools: add ppc64 in meson build test
  devtools: add Windows cross-build test with MinGW


v2: update some explanations and fix ABI check


 config/ppc/ppc64le-power8-linux-gcc         | 11 ++++++
 config/x86/{meson_mingw.txt => cross-mingw} |  0
 devtools/test-meson-builds.sh               | 44 +++++++++++++++------
 doc/guides/windows_gsg/build_dpdk.rst       |  2 +-
 4 files changed, 44 insertions(+), 13 deletions(-)
 create mode 100644 config/ppc/ppc64le-power8-linux-gcc
 rename config/x86/{meson_mingw.txt => cross-mingw} (100%)

-- 
2.26.2

[dpdk-dev] [PATCH v2 1/4] devtools: shrink cross-compilation test definition

[Thomas Monjalon]

Each cross-compilation case needs to define the target compiler
and the meson cross file.
Given the compiler is already defined in the cross file,
the latter is enough.

The function "build" is changed to accept a cross file alternatively
to the compiler name. In the case of a file (detected if readable),
the compiler is extracted with sed and tr, and the option --cross-file
is automatically added.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
v2: fix ABI check config (thanks David)
---
 devtools/test-meson-builds.sh | 26 ++++++++++++++++----------
 1 file changed, 16 insertions(+), 10 deletions(-)

diff --git a/devtools/test-meson-builds.sh b/devtools/test-meson-builds.sh
index 18b874fac5..bee55ec038 100755
--- a/devtools/test-meson-builds.sh
+++ b/devtools/test-meson-builds.sh
@@ -117,16 +117,24 @@ install_target () # <builddir> <installdir>
 	fi
 }
 
-build () # <directory> <target compiler> <meson options>
+build () # <directory> <target compiler | cross file> <meson options>
 {
 	targetdir=$1
 	shift
-	targetcc=$1
+	crossfile=
+	[ -r $1 ] && crossfile=$1 || targetcc=$1
 	shift
 	# skip build if compiler not available
 	command -v ${CC##* } >/dev/null 2>&1 || return 0
+	if [ -n "$crossfile" ] ; then
+		cross="--cross-file $crossfile"
+		targetcc=$(sed -n 's,^c[[:space:]]*=[[:space:]]*,,p' \
+			$crossfile | tr -d "'" | tr -d '"')
+	else
+		cross=
+	fi
 	load_env $targetcc || return 0
-	config $srcdir $builds_dir/$targetdir --werror $*
+	config $srcdir $builds_dir/$targetdir $cross --werror $*
 	compile $builds_dir/$targetdir
 	if [ -n "$DPDK_ABI_REF_VERSION" ]; then
 		abirefdir=${DPDK_ABI_REF_DIR:-reference}/$DPDK_ABI_REF_VERSION
@@ -140,7 +148,7 @@ build () # <directory> <target compiler> <meson options>
 			fi
 
 			rm -rf $abirefdir/build
-			config $abirefdir/src $abirefdir/build $*
+			config $abirefdir/src $abirefdir/build $cross $*
 			compile $abirefdir/build
 			install_target $abirefdir/build $abirefdir/$targetdir
 			$srcdir/devtools/gen-abi.sh $abirefdir/$targetdir
@@ -186,17 +194,15 @@ if [ "$ok" = "false" ] ; then
 fi
 build build-x86-default cc -Dlibdir=lib -Dmachine=$default_machine $use_shared
 
-c=aarch64-linux-gnu-gcc
 # generic armv8a with clang as host compiler
+f=$srcdir/config/arm/arm64_armv8_linux_gcc
 export CC="clang"
-build build-arm64-host-clang $c $use_shared \
-	--cross-file $srcdir/config/arm/arm64_armv8_linux_gcc
+build build-arm64-host-clang $f $use_shared
 unset CC
-# all gcc/arm configurations
+# some gcc/arm configurations
 for f in $srcdir/config/arm/arm64_[bdo]*gcc ; do
 	export CC="$CCACHE gcc"
-	build build-$(basename $f | tr '_' '-' | cut -d'-' -f-2) $c \
-		$use_shared --cross-file $f
+	build build-$(basename $f | tr '_' '-' | cut -d'-' -f-2) $f $use_shared
 	unset CC
 done
 
-- 
2.26.2

[dpdk-dev] [PATCH v2 2/4] devtools: allow non-standard toolchain in meson test

[Thomas Monjalon]

If a compiler is not found in $PATH, the compilation test is skipped.
In some cases, the compiler could be found after extending $PATH
in an environment configuration script (called by load-devel-config).

The decision to skip is deferred to a later stage, after loading the
configuration script.

In such case, the variable DPDK_TARGET, used by the configuration script
as input, is the compiler name.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 devtools/test-meson-builds.sh | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/devtools/test-meson-builds.sh b/devtools/test-meson-builds.sh
index bee55ec038..9781eaf415 100755
--- a/devtools/test-meson-builds.sh
+++ b/devtools/test-meson-builds.sh
@@ -52,9 +52,15 @@ load_env () # <target compiler>
 	export CFLAGS=$default_cflags
 	export LDFLAGS=$default_ldflags
 	unset DPDK_MESON_OPTIONS
-	command -v $targetcc >/dev/null 2>&1 || return 1
-	DPDK_TARGET=$($targetcc -v 2>&1 | sed -n 's,^Target: ,,p')
+	if command -v $targetcc >/dev/null 2>&1 ; then
+		DPDK_TARGET=$($targetcc -v 2>&1 | sed -n 's,^Target: ,,p')
+	else # toolchain not yet in PATH: its name should be enough
+		DPDK_TARGET=$targetcc
+	fi
+	# config input: $DPDK_TARGET
 	. $srcdir/devtools/load-devel-config
+	# config output: $DPDK_MESON_OPTIONS, $PATH, $PKG_CONFIG_PATH, etc
+	command -v $targetcc >/dev/null 2>&1 || return 1
 }
 
 config () # <dir> <builddir> <meson options>
-- 
2.26.2

[dpdk-dev] [PATCH v2 3/4] devtools: add ppc64 in meson build test

[Thomas Monjalon]

Add cross-compilation support of a PPC target in the build test matrix.
The CPU is defined as Power8, running as little endian.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 config/ppc/ppc64le-power8-linux-gcc | 11 +++++++++++
 devtools/test-meson-builds.sh       |  5 +++++
 2 files changed, 16 insertions(+)
 create mode 100644 config/ppc/ppc64le-power8-linux-gcc

diff --git a/config/ppc/ppc64le-power8-linux-gcc b/config/ppc/ppc64le-power8-linux-gcc
new file mode 100644
index 0000000000..51f7ceebf3
--- /dev/null
+++ b/config/ppc/ppc64le-power8-linux-gcc
@@ -0,0 +1,11 @@
+[binaries]
+c = 'powerpc64le-linux-gcc'
+cpp = 'powerpc64le-linux-cpp'
+ar = 'powerpc64le-linux-gcc-ar'
+strip = 'powerpc64le-linux-strip'
+
+[host_machine]
+system = 'linux'
+cpu_family = 'ppc64'
+cpu = 'power8'
+endian = 'little'
diff --git a/devtools/test-meson-builds.sh b/devtools/test-meson-builds.sh
index 9781eaf415..eaa215f13e 100755
--- a/devtools/test-meson-builds.sh
+++ b/devtools/test-meson-builds.sh
@@ -212,6 +212,11 @@ for f in $srcdir/config/arm/arm64_[bdo]*gcc ; do
 	unset CC
 done
 
+# ppc configurations
+for f in $srcdir/config/ppc/ppc* ; do
+	build build-$(basename $f | cut -d'-' -f-2) $f $use_shared
+done
+
 # Test installation of the x86-default target, to be used for checking
 # the sample apps build using the pkg-config file for cflags and libs
 build_path=$(readlink -f $builds_dir/build-x86-default)
-- 
2.26.2

[dpdk-dev] [PATCH v2 4/4] devtools: add Windows cross-build test with MinGW

[Thomas Monjalon]

The Meson cross file is renamed from meson_mingw.txt to cross-mingw,
and is added to test-meson-builds.sh.

The only example supported on Windows so far is "helloworld",
that's why the default list of examples is overridden.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 config/x86/{meson_mingw.txt => cross-mingw} | 0
 devtools/test-meson-builds.sh               | 3 +++
 doc/guides/windows_gsg/build_dpdk.rst       | 2 +-
 3 files changed, 4 insertions(+), 1 deletion(-)
 rename config/x86/{meson_mingw.txt => cross-mingw} (100%)

diff --git a/config/x86/meson_mingw.txt b/config/x86/cross-mingw
similarity index 100%
rename from config/x86/meson_mingw.txt
rename to config/x86/cross-mingw
diff --git a/devtools/test-meson-builds.sh b/devtools/test-meson-builds.sh
index eaa215f13e..64a022ccf1 100755
--- a/devtools/test-meson-builds.sh
+++ b/devtools/test-meson-builds.sh
@@ -200,6 +200,9 @@ if [ "$ok" = "false" ] ; then
 fi
 build build-x86-default cc -Dlibdir=lib -Dmachine=$default_machine $use_shared
 
+# x86 MinGW
+build build-x86-mingw $srcdir/config/x86/cross-mingw -Dexamples=helloworld
+
 # generic armv8a with clang as host compiler
 f=$srcdir/config/arm/arm64_armv8_linux_gcc
 export CC="clang"
diff --git a/doc/guides/windows_gsg/build_dpdk.rst b/doc/guides/windows_gsg/build_dpdk.rst
index 650483e3be..249e029c4e 100644
--- a/doc/guides/windows_gsg/build_dpdk.rst
+++ b/doc/guides/windows_gsg/build_dpdk.rst
@@ -109,5 +109,5 @@ Depending on the distribution, paths in this file may need adjustments.
 
 .. code-block:: console
 
-    meson --cross-file config/x86/meson_mingw.txt -Dexamples=helloworld build
+    meson --cross-file config/x86/cross-mingw -Dexamples=helloworld build
     ninja -C build
-- 
2.26.2

Re: [dpdk-dev] [PATCH 3/4] devtools: add ppc64 in meson build test

[David Christensen]

>>> +# ppc configurations
>>> +for f in $srcdir/config/ppc/ppc* ; do
>>> +	build build-$(basename $f | cut -d'-' -f-2) $f $use_shared
>>> +done
>>> +
>>
>> The entire script file is poorly structured for multi-architecture
>> builds.  The script fails on PPC64LE even before your changes:
> 
> This script supports only builds from x86 Linux so far.
> This patch is adding cross-compilation of ppc from x86,
> and it works fine.

Which distro and packages are used for testing?  I tried it with Ubuntu 
20.04 but the cross-compiler is named differently 
(powerpc64le-linux-gnu-gcc instead of powerpc64le-linux-gcc).

>> I think the logic of looping through supported configs for test builds
>> is the right one, but more changes are required for multi-architecture
>> support in this script.  For now I'd abandon the PPC/ARM support
>> entirely in this script.
> 
> I am adding PPC compilation check to avoid having PPC broken,
> because after asking for many years to IBM execs,
> there is still no PPC CI integrated in DPDK patchwork.
> If you don't think this small step is going in the right direction,
> then I will stop bothering about PPC support,
> and we should probably completely drop PPC support I guess.

Sorry, I didn't understand the context behind the change request nor how 
the script is used.  I appreciate the effort behind the change and would 
like to help where I can.

Dave

Re: [dpdk-dev] [PATCH 3/4] devtools: add ppc64 in meson build test

[Thomas Monjalon]

16/06/2020 22:35, David Christensen:
> Thomas Monjalon wrote:
> > This script supports only builds from x86 Linux so far.
> > This patch is adding cross-compilation of ppc from x86,
> > and it works fine.
> 
> Which distro and packages are used for testing?  I tried it with Ubuntu 
> 20.04 but the cross-compiler is named differently 
> (powerpc64le-linux-gnu-gcc instead of powerpc64le-linux-gcc).

I'm using a buildroot toolchain from bootlin:
https://toolchains.bootlin.com/releases_powerpc64le-power8.html

If it is more common to have "gnu" in the prefix,
no problem to change it in this patch.
I can create "gnu" symbolic links in my toolchain.

Ideally I would like to have an option to specify the toolchain prefix
with meson, instead of hardcoding it in a cross file.

Re: [dpdk-dev] [PATCH 3/4] devtools: add ppc64 in meson build test

[David Christensen]

> 16/06/2020 22:35, David Christensen:
>> Thomas Monjalon wrote:
>>> This script supports only builds from x86 Linux so far.
>>> This patch is adding cross-compilation of ppc from x86,
>>> and it works fine.
>>
>> Which distro and packages are used for testing?  I tried it with Ubuntu
>> 20.04 but the cross-compiler is named differently
>> (powerpc64le-linux-gnu-gcc instead of powerpc64le-linux-gcc).
> 
> I'm using a buildroot toolchain from bootlin:
> https://toolchains.bootlin.com/releases_powerpc64le-power8.html
> 
> If it is more common to have "gnu" in the prefix,
> no problem to change it in this patch.
> I can create "gnu" symbolic links in my toolchain.
> 
> Ideally I would like to have an option to specify the toolchain prefix
> with meson, instead of hardcoding it in a cross file.

If there's no defined distro where this tool is expected to be used I'd 
be happy with a comment in the cross file indicating which set of 
cross-compilers you chose for the implementation.

Reviewed-by: David Christensen <drc@linux.vnet.ibm.com>

Re: [dpdk-dev] [PATCH v2 1/4] devtools: shrink cross-compilation test definition

[David Christensen]

On 6/15/20 3:22 PM, Thomas Monjalon wrote:
> Each cross-compilation case needs to define the target compiler
> and the meson cross file.
> Given the compiler is already defined in the cross file,
> the latter is enough.
> 
> The function "build" is changed to accept a cross file alternatively
> to the compiler name. In the case of a file (detected if readable),
> the compiler is extracted with sed and tr, and the option --cross-file
> is automatically added.
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> ---
> v2: fix ABI check config (thanks David)

Reviewed-by: David Christensen <drc@linux.vnet.ibm.com>

Re: [dpdk-dev] [PATCH v2 2/4] devtools: allow non-standard toolchain in meson test

[David Christensen]

On 6/15/20 3:22 PM, Thomas Monjalon wrote:
> If a compiler is not found in $PATH, the compilation test is skipped.
> In some cases, the compiler could be found after extending $PATH
> in an environment configuration script (called by load-devel-config).
> 
> The decision to skip is deferred to a later stage, after loading the
> configuration script.
> 
> In such case, the variable DPDK_TARGET, used by the configuration script
> as input, is the compiler name.
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>

Reviewed-by: David Christensen <drc@linux.vnet.ibm.com>

Re: [dpdk-dev] [PATCH v2 3/4] devtools: add ppc64 in meson build test

[David Christensen]

> Add cross-compilation support of a PPC target in the build test matrix.
> The CPU is defined as Power8, running as little endian.
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>

Reviewed-by: David Christensen <drc@linux.vnet.ibm.com>

Re: [dpdk-dev] [PATCH v2 0/4] add PPC and Windows cross-compilation to meson test

[Thomas Monjalon]

16/06/2020 00:22, Thomas Monjalon:
> In order to better support PPC and Windows,
> their compilation is tested on Linux with Meson
> with the script test-meson-builds.sh,
> supposed to be called in every CI labs.
> 
> 
> Thomas Monjalon (4):
>   devtools: shrink cross-compilation test definition
>   devtools: allow non-standard toolchain in meson test
>   devtools: add ppc64 in meson build test
>   devtools: add Windows cross-build test with MinGW
> 
> 
> v2: update some explanations and fix ABI check


Applied