[dpdk-dev] [PATCH 0/2] add travis ci support for aarch64

[dpdk-dev] [PATCH 0/2] add travis ci support for aarch64

[Ruifeng Wang]

This patch set is to enable native aarch64 build in Travis CI.
It leverages Travis CI multi arch support.

As the first step, compilation jobs are added.
Unit test is not added for now due to service limitation. We are
planning to run unit test with no-huge in future.


Ruifeng Wang (2):
  ci: add travis ci support for aarch64
  devtools: add path to additional shared object files

 .ci/linux-setup.sh    | 11 +++++++----
 .travis.yml           | 42 +++++++++++++++++++++++++++++++++++++++++-
 devtools/test-null.sh |  2 +-
 3 files changed, 49 insertions(+), 6 deletions(-)

-- 
2.17.1

[dpdk-dev] [PATCH 1/2] ci: add travis ci support for aarch64

[Ruifeng Wang]

Add Travis compilation jobs for aarch64. gcc/clang compilations for
static/shared libraries are added.

Some limitations for current aarch64 Travis support:
1. Container is used. Huge page is not available due to security reason.
2. Missing kernel header package in Xenial distribution.

Solutions to address the limitations:
1. Not to add unit test for now. And run tests with no-huge in future.
2. Use Bionic distribution for all aarch64 jobs.

Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>
---
 .ci/linux-setup.sh | 11 +++++++----
 .travis.yml        | 42 +++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 48 insertions(+), 5 deletions(-)

diff --git a/.ci/linux-setup.sh b/.ci/linux-setup.sh
index dfb9d4a20..a92978037 100755
--- a/.ci/linux-setup.sh
+++ b/.ci/linux-setup.sh
@@ -3,7 +3,10 @@
 # need to install as 'root' since some of the unit tests won't run without it
 sudo python3 -m pip install --upgrade meson
 
-# setup hugepages
-cat /proc/meminfo
-sudo sh -c 'echo 1024 > /proc/sys/vm/nr_hugepages'
-cat /proc/meminfo
+# hugepage settings are skipped on aarch64 due to environment limitation
+if [ "$TRAVIS_ARCH" != "aarch64" ]; then
+    # setup hugepages
+    cat /proc/meminfo
+    sudo sh -c 'echo 1024 > /proc/sys/vm/nr_hugepages'
+    cat /proc/meminfo
+fi
diff --git a/.travis.yml b/.travis.yml
index 8f90d06f2..980c7605d 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -115,6 +115,46 @@ matrix:
       apt:
         packages:
           - *extra_packages
-
+  - env: DEF_LIB="static"
+    arch: arm64
+    compiler: gcc
+    dist: bionic
+    addons:
+      apt:
+        packages:
+          - *required_packages
+  - env: DEF_LIB="shared"
+    arch: arm64
+    compiler: gcc
+    dist: bionic
+    addons:
+      apt:
+        packages:
+          - *required_packages
+  - env: DEF_LIB="static"
+    arch: arm64
+    dist: bionic
+    compiler: clang
+    addons:
+      apt:
+        packages:
+          - *required_packages
+  - env: DEF_LIB="shared"
+    arch: arm64
+    dist: bionic
+    compiler: clang
+    addons:
+      apt:
+        packages:
+          - *required_packages
+  - env: DEF_LIB="shared" OPTS="-Denable_kmods=false" BUILD_DOCS=1
+    arch: arm64
+    compiler: gcc
+    dist: bionic
+    addons:
+      apt:
+        packages:
+          - *required_packages
+          - *doc_packages
 
 script: ./.ci/${TRAVIS_OS_NAME}-build.sh
-- 
2.17.1

[dpdk-dev] [PATCH 2/2] devtools: add path to additional shared object files

[Ruifeng Wang]

librte_mempool_ring.so and librte_pmd_null.so are in 'drivers' folder.
Add 'drivers' into LD_LIBRARY_PATH so that testpmd can find and make
use of these shared libraries.

Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>
---
 devtools/test-null.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/devtools/test-null.sh b/devtools/test-null.sh
index f39af2c06..548de8113 100755
--- a/devtools/test-null.sh
+++ b/devtools/test-null.sh
@@ -20,7 +20,7 @@ if [ ! -f "$testpmd" ] ; then
 fi
 
 if ldd $testpmd | grep -q librte_ ; then
-	export LD_LIBRARY_PATH=$build/lib:$LD_LIBRARY_PATH
+	export LD_LIBRARY_PATH=$build/drivers:$build/lib:$LD_LIBRARY_PATH
 	libs='-d librte_mempool_ring.so -d librte_pmd_null.so'
 else
 	libs=
-- 
2.17.1

Re: [dpdk-dev] [PATCH 2/2] devtools: add path to additional shared object files

[David Marchand]

On Wed, Dec 18, 2019 at 6:39 AM Ruifeng Wang <ruifeng.wang@arm.com> wrote:
>
> librte_mempool_ring.so and librte_pmd_null.so are in 'drivers' folder.
> Add 'drivers' into LD_LIBRARY_PATH so that testpmd can find and make
> use of these shared libraries.
>
> Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> ---
>  devtools/test-null.sh | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/devtools/test-null.sh b/devtools/test-null.sh
> index f39af2c06..548de8113 100755
> --- a/devtools/test-null.sh
> +++ b/devtools/test-null.sh
> @@ -20,7 +20,7 @@ if [ ! -f "$testpmd" ] ; then
>  fi
>
>  if ldd $testpmd | grep -q librte_ ; then
> -       export LD_LIBRARY_PATH=$build/lib:$LD_LIBRARY_PATH
> +       export LD_LIBRARY_PATH=$build/drivers:$build/lib:$LD_LIBRARY_PATH
>         libs='-d librte_mempool_ring.so -d librte_pmd_null.so'
>  else
>         libs=
> --
> 2.17.1
>

I'm surprised to see this.
So far, the CI ran fine without it, so something is different in the
environment.

I can see that the RPATH entry disappeared from the testpmd binary.
Xenial:
# readelf -d build/app/dpdk-testpmd |grep RPATH
 0x000000000000000f (RPATH)              Library rpath:
[$ORIGIN/../lib:$ORIGIN/../drivers:XXXXXXXXXXXXX]

(not sure what XXXX purpose is, but different topic)

Bionic:
# readelf -d build/app/dpdk-testpmd |grep RPATH

Adding Bruce and Kevin, as I think this is the same issue than:
http://mails.dpdk.org/archives/dev/2019-December/153627.html
Could it be a change in meson?


--
David Marchand

Re: [dpdk-dev] [PATCH 2/2] devtools: add path to additional shared object files

[Laatz, Kevin]

On 18/12/2019 08:23, David Marchand wrote:
> On Wed, Dec 18, 2019 at 6:39 AM Ruifeng Wang <ruifeng.wang@arm.com> wrote:
>> librte_mempool_ring.so and librte_pmd_null.so are in 'drivers' folder.
>> Add 'drivers' into LD_LIBRARY_PATH so that testpmd can find and make
>> use of these shared libraries.
>>
>> Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
>> Reviewed-by: Gavin Hu <gavin.hu@arm.com>
>> ---
>>   devtools/test-null.sh | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/devtools/test-null.sh b/devtools/test-null.sh
>> index f39af2c06..548de8113 100755
>> --- a/devtools/test-null.sh
>> +++ b/devtools/test-null.sh
>> @@ -20,7 +20,7 @@ if [ ! -f "$testpmd" ] ; then
>>   fi
>>
>>   if ldd $testpmd | grep -q librte_ ; then
>> -       export LD_LIBRARY_PATH=$build/lib:$LD_LIBRARY_PATH
>> +       export LD_LIBRARY_PATH=$build/drivers:$build/lib:$LD_LIBRARY_PATH
>>          libs='-d librte_mempool_ring.so -d librte_pmd_null.so'
>>   else
>>          libs=
>> --
>> 2.17.1
>>
> I'm surprised to see this.
> So far, the CI ran fine without it, so something is different in the
> environment.
>
> I can see that the RPATH entry disappeared from the testpmd binary.
> Xenial:
> # readelf -d build/app/dpdk-testpmd |grep RPATH
>   0x000000000000000f (RPATH)              Library rpath:
> [$ORIGIN/../lib:$ORIGIN/../drivers:XXXXXXXXXXXXX]
>
> (not sure what XXXX purpose is, but different topic)
>
> Bionic:
> # readelf -d build/app/dpdk-testpmd |grep RPATH

It looks like RPATH just changed to be named RUNPATH in Bionic:

# readelf -d build/app/dpdk-testpmd | grep R.*PATH
  0x000000000000001d (RUNPATH)            Library runpath: 
[$ORIGIN/../lib:$ORIGIN/../drivers:XXXXXXXXXXXXX]

> Adding Bruce and Kevin, as I think this is the same issue than:
> http://mails.dpdk.org/archives/dev/2019-December/153627.html
> Could it be a change in meson?

Yes, looks like the same error to me.

I'm not sure this is solely a meson issue, I often need to pass -d 
librte_mempool_ring.so when using make builds too. Maybe I'm missing 
something here?

---
Kevin

Re: [dpdk-dev] [PATCH 2/2] devtools: add path to additional shared object files

[David Marchand]

On Wed, Dec 18, 2019 at 2:43 PM Laatz, Kevin <kevin.laatz@intel.com> wrote:
>
> On 18/12/2019 08:23, David Marchand wrote:
> > On Wed, Dec 18, 2019 at 6:39 AM Ruifeng Wang <ruifeng.wang@arm.com> wrote:
> >> librte_mempool_ring.so and librte_pmd_null.so are in 'drivers' folder.
> >> Add 'drivers' into LD_LIBRARY_PATH so that testpmd can find and make
> >> use of these shared libraries.
> >>
> >> Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> >> Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> >> ---
> >>   devtools/test-null.sh | 2 +-
> >>   1 file changed, 1 insertion(+), 1 deletion(-)
> >>
> >> diff --git a/devtools/test-null.sh b/devtools/test-null.sh
> >> index f39af2c06..548de8113 100755
> >> --- a/devtools/test-null.sh
> >> +++ b/devtools/test-null.sh
> >> @@ -20,7 +20,7 @@ if [ ! -f "$testpmd" ] ; then
> >>   fi
> >>
> >>   if ldd $testpmd | grep -q librte_ ; then
> >> -       export LD_LIBRARY_PATH=$build/lib:$LD_LIBRARY_PATH
> >> +       export LD_LIBRARY_PATH=$build/drivers:$build/lib:$LD_LIBRARY_PATH
> >>          libs='-d librte_mempool_ring.so -d librte_pmd_null.so'
> >>   else
> >>          libs=
> >> --
> >> 2.17.1
> >>
> > I'm surprised to see this.
> > So far, the CI ran fine without it, so something is different in the
> > environment.
> >
> > I can see that the RPATH entry disappeared from the testpmd binary.
> > Xenial:
> > # readelf -d build/app/dpdk-testpmd |grep RPATH
> >   0x000000000000000f (RPATH)              Library rpath:
> > [$ORIGIN/../lib:$ORIGIN/../drivers:XXXXXXXXXXXXX]
> >
> > (not sure what XXXX purpose is, but different topic)
> >
> > Bionic:
> > # readelf -d build/app/dpdk-testpmd |grep RPATH
>
> It looks like RPATH just changed to be named RUNPATH in Bionic:
>
> # readelf -d build/app/dpdk-testpmd | grep R.*PATH
>   0x000000000000001d (RUNPATH)            Library runpath:
> [$ORIGIN/../lib:$ORIGIN/../drivers:XXXXXXXXXXXXX]

Did some experiment with some test program and .so of mine.
TL;DR, if I understand correctly, RPATH on the binary applies to all
lookups, even in a subsequent .so code.
But RUNPATH only applies to the current ELF, meaning that the dlopen()
in my intermediate .so does not get it.

dlopen() is called from librte_eal.so, and RUNPATH on testpmd is not enough.


Details:
[dmarchan@dmarchan plop]$ cat main.c
extern void loader(void);

int main(int argc, char *argv[])
{
    loader();
    return 0;
}
[dmarchan@dmarchan plop]$ cat loader/loader.c
#include <dlfcn.h>
#include <stdio.h>

void loader(void)
{
    if (dlopen("lib.so", RTLD_NOW) == NULL)
        fprintf(stderr, "%s\n", dlerror());
}

[dmarchan@dmarchan plop]$ cat lib/lib.c
#include <stdio.h>

__attribute__((constructor))
static void plop(void)
{
    fprintf(stdout, "plop\n");
}


# no rpath/runpath
[dmarchan@dmarchan plop]$ gcc -o lib/lib.so -Wall -Werror -shared
-fPIC lib/lib.c
[dmarchan@dmarchan plop]$ gcc -o loader/loader.so -Wall -Werror
-shared -fPIC loader/loader.c -ldl
[dmarchan@dmarchan plop]$ gcc -o main -Wall -Werror main.c loader/loader.so
[dmarchan@dmarchan plop]$ readelf -d main |grep R.*PATH
[dmarchan@dmarchan plop]$ ./main
lib.so: cannot open shared object file: No such file or directory

# using rpath on final binary
[dmarchan@dmarchan plop]$ gcc -o main -Wall -Werror main.c
loader/loader.so -Wl,-rpath,loader:lib
[dmarchan@dmarchan plop]$ readelf -d main |grep R.*PATH
 0x000000000000000f (RPATH)              Library rpath: [loader:lib]
[dmarchan@dmarchan plop]$ ./main
plop

# using runpath on final binary
[dmarchan@dmarchan plop]$ gcc -o main -Wall -Werror main.c
loader/loader.so -Wl,-enable-new-dtag,-rpath,loader:lib
[dmarchan@dmarchan plop]$ readelf -d main |grep R.*PATH
 0x000000000000001d (RUNPATH)            Library runpath: [loader:lib]
[dmarchan@dmarchan plop]$ ./main
lib.so: cannot open shared object file: No such file or directory

# using runpath on loader
[dmarchan@dmarchan plop]$ gcc -o loader/loader.so -Wall -Werror
-shared -fPIC loader/loader.c -ldl -Wl,-enable-new-dtag,-rpath,lib
[dmarchan@dmarchan plop]$ readelf -d loader/loader.so |grep R.*PATH
 0x000000000000001d (RUNPATH)            Library runpath: [lib]
[dmarchan@dmarchan plop]$ gcc -o main -Wall -Werror main.c
loader/loader.so -Wl,-enable-new-dtag,-rpath,loader
[dmarchan@dmarchan plop]$ readelf -d main |grep R.*PATH
 0x000000000000001d (RUNPATH)            Library runpath: [loader]
[dmarchan@dmarchan plop]$ ./main
plop


> > Adding Bruce and Kevin, as I think this is the same issue than:
> > http://mails.dpdk.org/archives/dev/2019-December/153627.html
> > Could it be a change in meson?
>
> Yes, looks like the same error to me.
>
> I'm not sure this is solely a meson issue, I often need to pass -d
> librte_mempool_ring.so when using make builds too. Maybe I'm missing
> something here?

In a make build directory, all libraries ends up in the same lib/
directory and the test-null.sh script work with the current
LD_LIBRARY_PATH.


Ruifeng patch seems valid to me, but I would love some explanations in
the commitlog.

--
David Marchand

Re: [dpdk-dev] [PATCH 2/2] devtools: add path to additional shared object files

[Richardson, Bruce]

> -----Original Message-----
> From: David Marchand <david.marchand@redhat.com>
> Sent: Wednesday, December 18, 2019 3:32 PM
> To: Laatz, Kevin <kevin.laatz@intel.com>
> Cc: Ruifeng Wang <ruifeng.wang@arm.com>; Richardson, Bruce
> <bruce.richardson@intel.com>; Aaron Conole <aconole@redhat.com>; Michael
> Santana <maicolgabriel@hotmail.com>; Thomas Monjalon
> <thomas@monjalon.net>; Yigit, Ferruh <ferruh.yigit@intel.com>; Andrew
> Rybchenko <arybchenko@solarflare.com>; dev <dev@dpdk.org>; Gavin Hu
> <gavin.hu@arm.com>; Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>;
> nd <nd@arm.com>
> Subject: Re: [dpdk-dev] [PATCH 2/2] devtools: add path to additional
> shared object files
> 
> On Wed, Dec 18, 2019 at 2:43 PM Laatz, Kevin <kevin.laatz@intel.com>
> wrote:
> >
> > On 18/12/2019 08:23, David Marchand wrote:
> > > On Wed, Dec 18, 2019 at 6:39 AM Ruifeng Wang <ruifeng.wang@arm.com>
> wrote:
> > >> librte_mempool_ring.so and librte_pmd_null.so are in 'drivers'
> folder.
> > >> Add 'drivers' into LD_LIBRARY_PATH so that testpmd can find and
> > >> make use of these shared libraries.
> > >>
> > >> Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> > >> Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> > >> ---
> > >>   devtools/test-null.sh | 2 +-
> > >>   1 file changed, 1 insertion(+), 1 deletion(-)
> > >>
> > >> diff --git a/devtools/test-null.sh b/devtools/test-null.sh index
> > >> f39af2c06..548de8113 100755
> > >> --- a/devtools/test-null.sh
> > >> +++ b/devtools/test-null.sh
> > >> @@ -20,7 +20,7 @@ if [ ! -f "$testpmd" ] ; then
> > >>   fi
> > >>
> > >>   if ldd $testpmd | grep -q librte_ ; then
> > >> -       export LD_LIBRARY_PATH=$build/lib:$LD_LIBRARY_PATH
> > >> +       export
> > >> + LD_LIBRARY_PATH=$build/drivers:$build/lib:$LD_LIBRARY_PATH
> > >>          libs='-d librte_mempool_ring.so -d librte_pmd_null.so'
> > >>   else
> > >>          libs=
> > >> --
> > >> 2.17.1
> > >>
> > > I'm surprised to see this.
> > > So far, the CI ran fine without it, so something is different in the
> > > environment.
> > >
> > > I can see that the RPATH entry disappeared from the testpmd binary.
> > > Xenial:
> > > # readelf -d build/app/dpdk-testpmd |grep RPATH
> > >   0x000000000000000f (RPATH)              Library rpath:
> > > [$ORIGIN/../lib:$ORIGIN/../drivers:XXXXXXXXXXXXX]
> > >
> > > (not sure what XXXX purpose is, but different topic)
> > >
> > > Bionic:
> > > # readelf -d build/app/dpdk-testpmd |grep RPATH
> >
> > It looks like RPATH just changed to be named RUNPATH in Bionic:
> >
> > # readelf -d build/app/dpdk-testpmd | grep R.*PATH
> >   0x000000000000001d (RUNPATH)            Library runpath:
> > [$ORIGIN/../lib:$ORIGIN/../drivers:XXXXXXXXXXXXX]
> 
> Did some experiment with some test program and .so of mine.
> TL;DR, if I understand correctly, RPATH on the binary applies to all
> lookups, even in a subsequent .so code.
> But RUNPATH only applies to the current ELF, meaning that the dlopen() in
> my intermediate .so does not get it.
> 
> dlopen() is called from librte_eal.so, and RUNPATH on testpmd is not
> enough.
> 
> 
> Details:
> [dmarchan@dmarchan plop]$ cat main.c
> extern void loader(void);
> 
> int main(int argc, char *argv[])
> {
>     loader();
>     return 0;
> }
> [dmarchan@dmarchan plop]$ cat loader/loader.c #include <dlfcn.h> #include
> <stdio.h>
> 
> void loader(void)
> {
>     if (dlopen("lib.so", RTLD_NOW) == NULL)
>         fprintf(stderr, "%s\n", dlerror()); }
> 
> [dmarchan@dmarchan plop]$ cat lib/lib.c
> #include <stdio.h>
> 
> __attribute__((constructor))
> static void plop(void)
> {
>     fprintf(stdout, "plop\n");
> }
> 
> 
> # no rpath/runpath
> [dmarchan@dmarchan plop]$ gcc -o lib/lib.so -Wall -Werror -shared -fPIC
> lib/lib.c [dmarchan@dmarchan plop]$ gcc -o loader/loader.so -Wall -Werror
> -shared -fPIC loader/loader.c -ldl [dmarchan@dmarchan plop]$ gcc -o main -
> Wall -Werror main.c loader/loader.so [dmarchan@dmarchan plop]$ readelf -d
> main |grep R.*PATH [dmarchan@dmarchan plop]$ ./main
> lib.so: cannot open shared object file: No such file or directory
> 
> # using rpath on final binary
> [dmarchan@dmarchan plop]$ gcc -o main -Wall -Werror main.c
> loader/loader.so -Wl,-rpath,loader:lib [dmarchan@dmarchan plop]$ readelf -
> d main |grep R.*PATH
>  0x000000000000000f (RPATH)              Library rpath: [loader:lib]
> [dmarchan@dmarchan plop]$ ./main
> plop
> 
> # using runpath on final binary
> [dmarchan@dmarchan plop]$ gcc -o main -Wall -Werror main.c
> loader/loader.so -Wl,-enable-new-dtag,-rpath,loader:lib
> [dmarchan@dmarchan plop]$ readelf -d main |grep R.*PATH
>  0x000000000000001d (RUNPATH)            Library runpath: [loader:lib]
> [dmarchan@dmarchan plop]$ ./main
> lib.so: cannot open shared object file: No such file or directory
> 
> # using runpath on loader
> [dmarchan@dmarchan plop]$ gcc -o loader/loader.so -Wall -Werror -shared -
> fPIC loader/loader.c -ldl -Wl,-enable-new-dtag,-rpath,lib
> [dmarchan@dmarchan plop]$ readelf -d loader/loader.so |grep R.*PATH
>  0x000000000000001d (RUNPATH)            Library runpath: [lib]
> [dmarchan@dmarchan plop]$ gcc -o main -Wall -Werror main.c
> loader/loader.so -Wl,-enable-new-dtag,-rpath,loader
> [dmarchan@dmarchan plop]$ readelf -d main |grep R.*PATH
>  0x000000000000001d (RUNPATH)            Library runpath: [loader]
> [dmarchan@dmarchan plop]$ ./main
> plop
> 
> 
> > > Adding Bruce and Kevin, as I think this is the same issue than:
> > > http://mails.dpdk.org/archives/dev/2019-December/153627.html
> > > Could it be a change in meson?
> >
> > Yes, looks like the same error to me.
> >
> > I'm not sure this is solely a meson issue, I often need to pass -d
> > librte_mempool_ring.so when using make builds too. Maybe I'm missing
> > something here?
> 
> In a make build directory, all libraries ends up in the same lib/
> directory and the test-null.sh script work with the current
> LD_LIBRARY_PATH.
> 
> 
> Ruifeng patch seems valid to me, but I would love some explanations in the
> commitlog.
> 
> --
> David Marchand

Really, if we are running a shared-library build, all sorts of problems are
to be expected unless we do an "install" to place the .so files in their correct
locations on the filesystem.

For example, on install, the drivers are copied to a drivers directory off the
lib folder, but then symlinked to lib too, so that e.g. the bus drivers can be
found as dependencies of the other drivers.

For running binaries within the build directory, I always recommend using a 
static build instead, to avoid all these issues - this is why static linking
is still the DPDK default.

In terms of this patch, I have no problem with it if it fixed the issue.

/Bruce

Re: [dpdk-dev] [PATCH 2/2] devtools: add path to additional shared object files

[Ruifeng Wang]

> -----Original Message-----
> From: David Marchand <david.marchand@redhat.com>
> Sent: Wednesday, December 18, 2019 23:32
> To: Laatz, Kevin <kevin.laatz@intel.com>
> Cc: Ruifeng Wang <Ruifeng.Wang@arm.com>; Bruce Richardson
> <bruce.richardson@intel.com>; Aaron Conole <aconole@redhat.com>;
> Michael Santana <maicolgabriel@hotmail.com>; thomas@monjalon.net; Yigit,
> Ferruh <ferruh.yigit@intel.com>; Andrew Rybchenko
> <arybchenko@solarflare.com>; dev <dev@dpdk.org>; Gavin Hu
> <Gavin.Hu@arm.com>; Honnappa Nagarahalli
> <Honnappa.Nagarahalli@arm.com>; nd <nd@arm.com>
> Subject: Re: [dpdk-dev] [PATCH 2/2] devtools: add path to additional shared
> object files
> 
> On Wed, Dec 18, 2019 at 2:43 PM Laatz, Kevin <kevin.laatz@intel.com> wrote:
> >
> > On 18/12/2019 08:23, David Marchand wrote:
> > > On Wed, Dec 18, 2019 at 6:39 AM Ruifeng Wang <ruifeng.wang@arm.com>
> wrote:
> > >> librte_mempool_ring.so and librte_pmd_null.so are in 'drivers' folder.
> > >> Add 'drivers' into LD_LIBRARY_PATH so that testpmd can find and
> > >> make use of these shared libraries.
> > >>
> > >> Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> > >> Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> > >> ---
> > >>   devtools/test-null.sh | 2 +-
> > >>   1 file changed, 1 insertion(+), 1 deletion(-)
> > >>
> > >> diff --git a/devtools/test-null.sh b/devtools/test-null.sh index
> > >> f39af2c06..548de8113 100755
> > >> --- a/devtools/test-null.sh
> > >> +++ b/devtools/test-null.sh
> > >> @@ -20,7 +20,7 @@ if [ ! -f "$testpmd" ] ; then
> > >>   fi
> > >>
> > >>   if ldd $testpmd | grep -q librte_ ; then
> > >> -       export LD_LIBRARY_PATH=$build/lib:$LD_LIBRARY_PATH
> > >> +       export
> > >> + LD_LIBRARY_PATH=$build/drivers:$build/lib:$LD_LIBRARY_PATH
> > >>          libs='-d librte_mempool_ring.so -d librte_pmd_null.so'
> > >>   else
> > >>          libs=
> > >> --
> > >> 2.17.1
> > >>
> > > I'm surprised to see this.
> > > So far, the CI ran fine without it, so something is different in the
> > > environment.
> > >
> > > I can see that the RPATH entry disappeared from the testpmd binary.
> > > Xenial:
> > > # readelf -d build/app/dpdk-testpmd |grep RPATH
> > >   0x000000000000000f (RPATH)              Library rpath:
> > > [$ORIGIN/../lib:$ORIGIN/../drivers:XXXXXXXXXXXXX]
> > >
> > > (not sure what XXXX purpose is, but different topic)
> > >
> > > Bionic:
> > > # readelf -d build/app/dpdk-testpmd |grep RPATH
> >
> > It looks like RPATH just changed to be named RUNPATH in Bionic:
> >
> > # readelf -d build/app/dpdk-testpmd | grep R.*PATH
> >   0x000000000000001d (RUNPATH)            Library runpath:
> > [$ORIGIN/../lib:$ORIGIN/../drivers:XXXXXXXXXXXXX]
> 
> Did some experiment with some test program and .so of mine.
> TL;DR, if I understand correctly, RPATH on the binary applies to all lookups,
> even in a subsequent .so code.
> But RUNPATH only applies to the current ELF, meaning that the dlopen() in
> my intermediate .so does not get it.
> 
> dlopen() is called from librte_eal.so, and RUNPATH on testpmd is not enough.
> 
Thanks for your experiment and analysis. Really happy to know more around the issue.

> 
> Details:
> [dmarchan@dmarchan plop]$ cat main.c
> extern void loader(void);
> 
> int main(int argc, char *argv[])
> {
>     loader();
>     return 0;
> }
> [dmarchan@dmarchan plop]$ cat loader/loader.c #include <dlfcn.h>
> #include <stdio.h>
> 
> void loader(void)
> {
>     if (dlopen("lib.so", RTLD_NOW) == NULL)
>         fprintf(stderr, "%s\n", dlerror()); }
> 
> [dmarchan@dmarchan plop]$ cat lib/lib.c
> #include <stdio.h>
> 
> __attribute__((constructor))
> static void plop(void)
> {
>     fprintf(stdout, "plop\n");
> }
> 
> 
> # no rpath/runpath
> [dmarchan@dmarchan plop]$ gcc -o lib/lib.so -Wall -Werror -shared -fPIC
> lib/lib.c [dmarchan@dmarchan plop]$ gcc -o loader/loader.so -Wall -Werror -
> shared -fPIC loader/loader.c -ldl [dmarchan@dmarchan plop]$ gcc -o main -
> Wall -Werror main.c loader/loader.so [dmarchan@dmarchan plop]$ readelf -
> d main |grep R.*PATH [dmarchan@dmarchan plop]$ ./main
> lib.so: cannot open shared object file: No such file or directory
> 
> # using rpath on final binary
> [dmarchan@dmarchan plop]$ gcc -o main -Wall -Werror main.c
> loader/loader.so -Wl,-rpath,loader:lib [dmarchan@dmarchan plop]$ readelf -
> d main |grep R.*PATH
>  0x000000000000000f (RPATH)              Library rpath: [loader:lib]
> [dmarchan@dmarchan plop]$ ./main
> plop
> 
> # using runpath on final binary
> [dmarchan@dmarchan plop]$ gcc -o main -Wall -Werror main.c
> loader/loader.so -Wl,-enable-new-dtag,-rpath,loader:lib
> [dmarchan@dmarchan plop]$ readelf -d main |grep R.*PATH
>  0x000000000000001d (RUNPATH)            Library runpath: [loader:lib]
> [dmarchan@dmarchan plop]$ ./main
> lib.so: cannot open shared object file: No such file or directory
> 
> # using runpath on loader
> [dmarchan@dmarchan plop]$ gcc -o loader/loader.so -Wall -Werror -shared -
> fPIC loader/loader.c -ldl -Wl,-enable-new-dtag,-rpath,lib
> [dmarchan@dmarchan plop]$ readelf -d loader/loader.so |grep R.*PATH
>  0x000000000000001d (RUNPATH)            Library runpath: [lib]
> [dmarchan@dmarchan plop]$ gcc -o main -Wall -Werror main.c
> loader/loader.so -Wl,-enable-new-dtag,-rpath,loader
> [dmarchan@dmarchan plop]$ readelf -d main |grep R.*PATH
>  0x000000000000001d (RUNPATH)            Library runpath: [loader]
> [dmarchan@dmarchan plop]$ ./main
> plop
> 
> 
> > > Adding Bruce and Kevin, as I think this is the same issue than:
> > > http://mails.dpdk.org/archives/dev/2019-December/153627.html
> > > Could it be a change in meson?
> >
> > Yes, looks like the same error to me.
> >
> > I'm not sure this is solely a meson issue, I often need to pass -d
> > librte_mempool_ring.so when using make builds too. Maybe I'm missing
> > something here?
> 
> In a make build directory, all libraries ends up in the same lib/ directory and
> the test-null.sh script work with the current LD_LIBRARY_PATH.
> 
> 
> Ruifeng patch seems valid to me, but I would love some explanations in the
> commitlog.
> 
Sure. I will re-spin to add explanations.

> --
> David Marchand

[dpdk-dev] [PATCH v2 0/2] add travis ci support for aarch64

[Ruifeng Wang]

This patch set is to enable native aarch64 build in Travis CI.
It leverages Travis CI multi arch support.

As the first step, compilation jobs are added.
Unit test is not added for now due to service limitation. We are
planning to run unit test with no-huge in future.

This patch set has dependency on:
http://patches.dpdk.org/patch/63969/


v2:
Removed dist designation from matrix since base dist is changing.
Add explanation for library path issue.

Ruifeng Wang (2):
  ci: add travis ci support for aarch64
  devtools: add path to additional shared object files

 .ci/linux-setup.sh    | 11 +++++++----
 .travis.yml           | 37 ++++++++++++++++++++++++++++++++++++-
 devtools/test-null.sh |  2 +-
 3 files changed, 44 insertions(+), 6 deletions(-)

-- 
2.17.1

[dpdk-dev] [PATCH v2 1/2] ci: add travis ci support for aarch64

[Ruifeng Wang]

Add Travis compilation jobs for aarch64. gcc/clang compilations for
static/shared libraries are added.

Some limitations for current aarch64 Travis support:
1. Container is used. Huge page is not available due to security reason.
2. Missing kernel header package in Xenial distribution.

Solutions to address the limitations:
1. Not to add unit test for now. And run tests with no-huge in future.
2. Use Bionic distribution for all aarch64 jobs.

Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>
---
 .ci/linux-setup.sh | 11 +++++++----
 .travis.yml        | 37 ++++++++++++++++++++++++++++++++++++-
 2 files changed, 43 insertions(+), 5 deletions(-)

diff --git a/.ci/linux-setup.sh b/.ci/linux-setup.sh
index dfb9d4a20..a92978037 100755
--- a/.ci/linux-setup.sh
+++ b/.ci/linux-setup.sh
@@ -3,7 +3,10 @@
 # need to install as 'root' since some of the unit tests won't run without it
 sudo python3 -m pip install --upgrade meson
 
-# setup hugepages
-cat /proc/meminfo
-sudo sh -c 'echo 1024 > /proc/sys/vm/nr_hugepages'
-cat /proc/meminfo
+# hugepage settings are skipped on aarch64 due to environment limitation
+if [ "$TRAVIS_ARCH" != "aarch64" ]; then
+    # setup hugepages
+    cat /proc/meminfo
+    sudo sh -c 'echo 1024 > /proc/sys/vm/nr_hugepages'
+    cat /proc/meminfo
+fi
diff --git a/.travis.yml b/.travis.yml
index 8f90d06f2..048cb6ba5 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -115,6 +115,41 @@ matrix:
       apt:
         packages:
           - *extra_packages
-
+  - env: DEF_LIB="static"
+    arch: arm64
+    compiler: gcc
+    addons:
+      apt:
+        packages:
+          - *required_packages
+  - env: DEF_LIB="shared"
+    arch: arm64
+    compiler: gcc
+    addons:
+      apt:
+        packages:
+          - *required_packages
+  - env: DEF_LIB="static"
+    arch: arm64
+    compiler: clang
+    addons:
+      apt:
+        packages:
+          - *required_packages
+  - env: DEF_LIB="shared"
+    arch: arm64
+    compiler: clang
+    addons:
+      apt:
+        packages:
+          - *required_packages
+  - env: DEF_LIB="shared" OPTS="-Denable_kmods=false" BUILD_DOCS=1
+    arch: arm64
+    compiler: gcc
+    addons:
+      apt:
+        packages:
+          - *required_packages
+          - *doc_packages
 
 script: ./.ci/${TRAVIS_OS_NAME}-build.sh
-- 
2.17.1

Re: [dpdk-dev] [PATCH v2 1/2] ci: add travis ci support for aarch64

[dwilder]

On 2019-12-20 01:37, Ruifeng Wang wrote:
> Add Travis compilation jobs for aarch64. gcc/clang compilations for
> static/shared libraries are added.
> 
> Some limitations for current aarch64 Travis support:
> 1. Container is used. Huge page is not available due to security 
> reason.
> 2. Missing kernel header package in Xenial distribution.
> 
> Solutions to address the limitations:
> 1. Not to add unit test for now. And run tests with no-huge in future.
> 2. Use Bionic distribution for all aarch64 jobs.
> 
> Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> ---
>  .ci/linux-setup.sh | 11 +++++++----
>  .travis.yml        | 37 ++++++++++++++++++++++++++++++++++++-
>  2 files changed, 43 insertions(+), 5 deletions(-)
> 
> diff --git a/.ci/linux-setup.sh b/.ci/linux-setup.sh
> index dfb9d4a20..a92978037 100755
> --- a/.ci/linux-setup.sh
> +++ b/.ci/linux-setup.sh
> @@ -3,7 +3,10 @@
>  # need to install as 'root' since some of the unit tests won't run 
> without it
>  sudo python3 -m pip install --upgrade meson
> 
> -# setup hugepages
> -cat /proc/meminfo
> -sudo sh -c 'echo 1024 > /proc/sys/vm/nr_hugepages'
> -cat /proc/meminfo


dont test for TRAVIS_ARCH here as multiple archs may need to avoid the 
hugepage setup as well.

How about:
if [ "$RUN_TESTS" = "1" ]; then
    # setup hugepages

If your planning to add a tests using --no-huge option could we add a 
NOHUGEPAGES option to the matrix?

> +# hugepage settings are skipped on aarch64 due to environment 
> limitation
> +if [ "$TRAVIS_ARCH" != "aarch64" ]; then
> +    # setup hugepages
> +    cat /proc/meminfo
> +    sudo sh -c 'echo 1024 > /proc/sys/vm/nr_hugepages'
> +    cat /proc/meminfo
> +fi
> diff --git a/.travis.yml b/.travis.yml
> index 8f90d06f2..048cb6ba5 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -115,6 +115,41 @@ matrix:
>        apt:
>          packages:
>            - *extra_packages
> -
> +  - env: DEF_LIB="static"
> +    arch: arm64
> +    compiler: gcc
> +    addons:
> +      apt:
> +        packages:
> +          - *required_packages
> +  - env: DEF_LIB="shared"
> +    arch: arm64
> +    compiler: gcc
> +    addons:
> +      apt:
> +        packages:
> +          - *required_packages
> +  - env: DEF_LIB="static"
> +    arch: arm64
> +    compiler: clang
> +    addons:
> +      apt:
> +        packages:
> +          - *required_packages
> +  - env: DEF_LIB="shared"
> +    arch: arm64
> +    compiler: clang
> +    addons:
> +      apt:
> +        packages:
> +          - *required_packages
> +  - env: DEF_LIB="shared" OPTS="-Denable_kmods=false" BUILD_DOCS=1
> +    arch: arm64
> +    compiler: gcc
> +    addons:
> +      apt:
> +        packages:
> +          - *required_packages
> +          - *doc_packages
> 
>  script: ./.ci/${TRAVIS_OS_NAME}-build.sh

Re: [dpdk-dev] [PATCH v2 1/2] ci: add travis ci support for aarch64

[Ruifeng Wang]

> -----Original Message-----
> From: dwilder <dwilder@us.ibm.com>
> Sent: Tuesday, January 7, 2020 04:17
> To: Ruifeng Wang <Ruifeng.Wang@arm.com>
> Cc: aconole@redhat.com; maicolgabriel@hotmail.com;
> thomas@monjalon.net; ferruh.yigit@intel.com; arybchenko@solarflare.com;
> dev@dpdk.org; Gavin Hu <Gavin.Hu@arm.com>; Honnappa Nagarahalli
> <Honnappa.Nagarahalli@arm.com>; nd <nd@arm.com>
> Subject: Re: [dpdk-dev] [PATCH v2 1/2] ci: add travis ci support for aarch64
> 
> On 2019-12-20 01:37, Ruifeng Wang wrote:
> > Add Travis compilation jobs for aarch64. gcc/clang compilations for
> > static/shared libraries are added.
> >
> > Some limitations for current aarch64 Travis support:
> > 1. Container is used. Huge page is not available due to security
> > reason.
> > 2. Missing kernel header package in Xenial distribution.
> >
> > Solutions to address the limitations:
> > 1. Not to add unit test for now. And run tests with no-huge in future.
> > 2. Use Bionic distribution for all aarch64 jobs.
> >
> > Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> > Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> > ---
> >  .ci/linux-setup.sh | 11 +++++++----
> >  .travis.yml        | 37 ++++++++++++++++++++++++++++++++++++-
> >  2 files changed, 43 insertions(+), 5 deletions(-)
> >
> > diff --git a/.ci/linux-setup.sh b/.ci/linux-setup.sh index
> > dfb9d4a20..a92978037 100755
> > --- a/.ci/linux-setup.sh
> > +++ b/.ci/linux-setup.sh
> > @@ -3,7 +3,10 @@
> >  # need to install as 'root' since some of the unit tests won't run
> > without it  sudo python3 -m pip install --upgrade meson
> >
> > -# setup hugepages
> > -cat /proc/meminfo
> > -sudo sh -c 'echo 1024 > /proc/sys/vm/nr_hugepages'
> > -cat /proc/meminfo
> 
> 
> dont test for TRAVIS_ARCH here as multiple archs may need to avoid the
> hugepage setup as well.
> 
> How about:
> if [ "$RUN_TESTS" = "1" ]; then
>     # setup hugepages

This should  work. I can try this approach in next version.
My concern is the name may cause confusion. 'RUN_TESTS' will mean to
run default tests (with hugepage).
> 
> If your planning to add a tests using --no-huge option could we add a
> NOHUGEPAGES option to the matrix?

Yes, this will give control to choose test suite. I will take this into consideration.

> 
> > +# hugepage settings are skipped on aarch64 due to environment
> > limitation
> > +if [ "$TRAVIS_ARCH" != "aarch64" ]; then
> > +    # setup hugepages
> > +    cat /proc/meminfo
> > +    sudo sh -c 'echo 1024 > /proc/sys/vm/nr_hugepages'
> > +    cat /proc/meminfo
> > +fi
> > diff --git a/.travis.yml b/.travis.yml index 8f90d06f2..048cb6ba5
> > 100644
> > --- a/.travis.yml
> > +++ b/.travis.yml
> > @@ -115,6 +115,41 @@ matrix:
> >        apt:
> >          packages:
> >            - *extra_packages
> > -
> > +  - env: DEF_LIB="static"
> > +    arch: arm64
> > +    compiler: gcc
> > +    addons:
> > +      apt:
> > +        packages:
> > +          - *required_packages
> > +  - env: DEF_LIB="shared"
> > +    arch: arm64
> > +    compiler: gcc
> > +    addons:
> > +      apt:
> > +        packages:
> > +          - *required_packages
> > +  - env: DEF_LIB="static"
> > +    arch: arm64
> > +    compiler: clang
> > +    addons:
> > +      apt:
> > +        packages:
> > +          - *required_packages
> > +  - env: DEF_LIB="shared"
> > +    arch: arm64
> > +    compiler: clang
> > +    addons:
> > +      apt:
> > +        packages:
> > +          - *required_packages
> > +  - env: DEF_LIB="shared" OPTS="-Denable_kmods=false" BUILD_DOCS=1
> > +    arch: arm64
> > +    compiler: gcc
> > +    addons:
> > +      apt:
> > +        packages:
> > +          - *required_packages
> > +          - *doc_packages
> >
> >  script: ./.ci/${TRAVIS_OS_NAME}-build.sh

[dpdk-dev] [PATCH v2 2/2] devtools: add path to additional shared object files

[Ruifeng Wang]

Drivers librte_mempool_ring.so and librte_pmd_null.so are loaded by
librte_eal.so when running testpmd.
In Ubuntu Xenial, driver path is installed to RPATH on testpmd. This
allows librte_eal.so to find drivers by using the RPATH.
However, in Ubuntu Bionic, driver path is installed to RUNPATH instead.
The RUNPATH on testpmd is not available by librte_eal.so and therefore
lead to driver load failure:

EAL: Detected 32 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: librte_mempool_ring.so: cannot open shared object file:
					No such file or directory
EAL: FATAL: Cannot init plugins
EAL: Cannot init plugins

Add 'drivers' into LD_LIBRARY_PATH so that testpmd can find and make
use of these shared libraries.

Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>
---
 devtools/test-null.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/devtools/test-null.sh b/devtools/test-null.sh
index f39af2c06..548de8113 100755
--- a/devtools/test-null.sh
+++ b/devtools/test-null.sh
@@ -20,7 +20,7 @@ if [ ! -f "$testpmd" ] ; then
 fi
 
 if ldd $testpmd | grep -q librte_ ; then
-	export LD_LIBRARY_PATH=$build/lib:$LD_LIBRARY_PATH
+	export LD_LIBRARY_PATH=$build/drivers:$build/lib:$LD_LIBRARY_PATH
 	libs='-d librte_mempool_ring.so -d librte_pmd_null.so'
 else
 	libs=
-- 
2.17.1

Re: [dpdk-dev] [PATCH v2 0/2] add travis ci support for aarch64

[David Marchand]

On Fri, Dec 20, 2019 at 10:38 AM Ruifeng Wang <ruifeng.wang@arm.com> wrote:
>
> This patch set is to enable native aarch64 build in Travis CI.
> It leverages Travis CI multi arch support.
>
> As the first step, compilation jobs are added.
> Unit test is not added for now due to service limitation. We are
> planning to run unit test with no-huge in future.
>
> This patch set has dependency on:
> http://patches.dpdk.org/patch/63969/
>
>
> v2:
> Removed dist designation from matrix since base dist is changing.

*If* (I did not think about the implications yet, hence the *If*) we
switch the base distribution to Bionic, we can't push it anyway
without your second patch.

Patches order must be reversed.
And for the jobs in travis, the distribution for aarch64 native jobs
must be explicitely bionic distribution (you did not update the
commitlog in the v2 but not a problem if you restore the dist: bionic
entries).
We can then take your series.

Kevin patch that upgrades globally from xenial to bionic, can then be
integrated after yours.


> Add explanation for library path issue.

Thanks for this.


-- 
David Marchand

[dpdk-dev] [PATCH v3 0/2] add travis ci support for aarch64

[Ruifeng Wang]

This patch set is to enable native aarch64 build in Travis CI.
It leverages Travis CI multi arch support.

As the first step, compilation jobs are added.
Unit test is not added for now due to service limitation. We are
planning to run unit test with no-huge in future.


v3:
Reverse patches order. Not depending on other patches.
Restore distribution designation for aarch64 native jobs.

v2:
Removed dist designation from matrix since base dist is changing.
Add explanation for library path issue.

Ruifeng Wang (2):
  devtools: add path to additional shared object files
  ci: add travis ci support for aarch64

 .ci/linux-setup.sh    | 11 +++++++----
 .travis.yml           | 42 +++++++++++++++++++++++++++++++++++++++++-
 devtools/test-null.sh |  2 +-
 3 files changed, 49 insertions(+), 6 deletions(-)

-- 
2.17.1

[dpdk-dev] [PATCH v3 1/2] devtools: add path to additional shared object files

[Ruifeng Wang]

Drivers librte_mempool_ring.so and librte_pmd_null.so are loaded by
librte_eal.so when running testpmd.
In Ubuntu Xenial, driver path is installed to RPATH on testpmd. This
allows librte_eal.so to find drivers by using the RPATH.
However, in Ubuntu Bionic, driver path is installed to RUNPATH instead.
The RUNPATH on testpmd is not available by librte_eal.so and therefore
lead to driver load failure:

EAL: Detected 32 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: librte_mempool_ring.so: cannot open shared object file:
					No such file or directory
EAL: FATAL: Cannot init plugins
EAL: Cannot init plugins

Add 'drivers' into LD_LIBRARY_PATH so that testpmd can find and make
use of these shared libraries.

Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>
---
 devtools/test-null.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/devtools/test-null.sh b/devtools/test-null.sh
index f39af2c06..548de8113 100755
--- a/devtools/test-null.sh
+++ b/devtools/test-null.sh
@@ -20,7 +20,7 @@ if [ ! -f "$testpmd" ] ; then
 fi
 
 if ldd $testpmd | grep -q librte_ ; then
-	export LD_LIBRARY_PATH=$build/lib:$LD_LIBRARY_PATH
+	export LD_LIBRARY_PATH=$build/drivers:$build/lib:$LD_LIBRARY_PATH
 	libs='-d librte_mempool_ring.so -d librte_pmd_null.so'
 else
 	libs=
-- 
2.17.1

[dpdk-dev] [PATCH v3 2/2] ci: add travis ci support for aarch64

[Ruifeng Wang]

Add Travis compilation jobs for aarch64. gcc/clang compilations for
static/shared libraries are added.

Some limitations for current aarch64 Travis support:
1. Container is used. Huge page is not available due to security reason.
2. Missing kernel header package in Xenial distribution.

Solutions to address the limitations:
1. Not to add unit test for now. And run tests with no-huge in future.
2. Use Bionic distribution for all aarch64 jobs.

Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>
---
 .ci/linux-setup.sh | 11 +++++++----
 .travis.yml        | 42 +++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 48 insertions(+), 5 deletions(-)

diff --git a/.ci/linux-setup.sh b/.ci/linux-setup.sh
index dfb9d4a20..a92978037 100755
--- a/.ci/linux-setup.sh
+++ b/.ci/linux-setup.sh
@@ -3,7 +3,10 @@
 # need to install as 'root' since some of the unit tests won't run without it
 sudo python3 -m pip install --upgrade meson
 
-# setup hugepages
-cat /proc/meminfo
-sudo sh -c 'echo 1024 > /proc/sys/vm/nr_hugepages'
-cat /proc/meminfo
+# hugepage settings are skipped on aarch64 due to environment limitation
+if [ "$TRAVIS_ARCH" != "aarch64" ]; then
+    # setup hugepages
+    cat /proc/meminfo
+    sudo sh -c 'echo 1024 > /proc/sys/vm/nr_hugepages'
+    cat /proc/meminfo
+fi
diff --git a/.travis.yml b/.travis.yml
index 8f90d06f2..980c7605d 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -115,6 +115,46 @@ matrix:
       apt:
         packages:
           - *extra_packages
-
+  - env: DEF_LIB="static"
+    arch: arm64
+    compiler: gcc
+    dist: bionic
+    addons:
+      apt:
+        packages:
+          - *required_packages
+  - env: DEF_LIB="shared"
+    arch: arm64
+    compiler: gcc
+    dist: bionic
+    addons:
+      apt:
+        packages:
+          - *required_packages
+  - env: DEF_LIB="static"
+    arch: arm64
+    dist: bionic
+    compiler: clang
+    addons:
+      apt:
+        packages:
+          - *required_packages
+  - env: DEF_LIB="shared"
+    arch: arm64
+    dist: bionic
+    compiler: clang
+    addons:
+      apt:
+        packages:
+          - *required_packages
+  - env: DEF_LIB="shared" OPTS="-Denable_kmods=false" BUILD_DOCS=1
+    arch: arm64
+    compiler: gcc
+    dist: bionic
+    addons:
+      apt:
+        packages:
+          - *required_packages
+          - *doc_packages
 
 script: ./.ci/${TRAVIS_OS_NAME}-build.sh
-- 
2.17.1

Re: [dpdk-dev] [PATCH v3 2/2] ci: add travis ci support for aarch64

[Aaron Conole]

Ruifeng Wang <ruifeng.wang@arm.com> writes:

> Add Travis compilation jobs for aarch64. gcc/clang compilations for
> static/shared libraries are added.
>
> Some limitations for current aarch64 Travis support:
> 1. Container is used. Huge page is not available due to security reason.
> 2. Missing kernel header package in Xenial distribution.
>
> Solutions to address the limitations:
> 1. Not to add unit test for now. And run tests with no-huge in future.
> 2. Use Bionic distribution for all aarch64 jobs.
>
> Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> ---

Can't we achieve the same thing by setting

arch:
  - amd64
  - arm64

in the build matrix?  Or will that also force the intel builds to use
the container infrastructure (in which case the no-huge support needs to
be fixed)?

One thing I wonder, isn't is possible to use qemu-user to do the amd64
unit tests?  Then do we really need some changes to do the native build?
Does it buy us anything *today* given the cost of the hugepage
restriction?  Will that ever be resolved (I didn't see so from the
docs on travis)?

>  .ci/linux-setup.sh | 11 +++++++----
>  .travis.yml        | 42 +++++++++++++++++++++++++++++++++++++++++-
>  2 files changed, 48 insertions(+), 5 deletions(-)
>
> diff --git a/.ci/linux-setup.sh b/.ci/linux-setup.sh
> index dfb9d4a20..a92978037 100755
> --- a/.ci/linux-setup.sh
> +++ b/.ci/linux-setup.sh
> @@ -3,7 +3,10 @@
>  # need to install as 'root' since some of the unit tests won't run without it
>  sudo python3 -m pip install --upgrade meson
>  
> -# setup hugepages
> -cat /proc/meminfo
> -sudo sh -c 'echo 1024 > /proc/sys/vm/nr_hugepages'
> -cat /proc/meminfo
> +# hugepage settings are skipped on aarch64 due to environment limitation
> +if [ "$TRAVIS_ARCH" != "aarch64" ]; then
> +    # setup hugepages
> +    cat /proc/meminfo
> +    sudo sh -c 'echo 1024 > /proc/sys/vm/nr_hugepages'
> +    cat /proc/meminfo
> +fi
> diff --git a/.travis.yml b/.travis.yml
> index 8f90d06f2..980c7605d 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -115,6 +115,46 @@ matrix:
>        apt:
>          packages:
>            - *extra_packages
> -
> +  - env: DEF_LIB="static"
> +    arch: arm64
> +    compiler: gcc
> +    dist: bionic
> +    addons:
> +      apt:
> +        packages:
> +          - *required_packages
> +  - env: DEF_LIB="shared"
> +    arch: arm64
> +    compiler: gcc
> +    dist: bionic
> +    addons:
> +      apt:
> +        packages:
> +          - *required_packages
> +  - env: DEF_LIB="static"
> +    arch: arm64
> +    dist: bionic
> +    compiler: clang
> +    addons:
> +      apt:
> +        packages:
> +          - *required_packages
> +  - env: DEF_LIB="shared"
> +    arch: arm64
> +    dist: bionic
> +    compiler: clang
> +    addons:
> +      apt:
> +        packages:
> +          - *required_packages
> +  - env: DEF_LIB="shared" OPTS="-Denable_kmods=false" BUILD_DOCS=1
> +    arch: arm64
> +    compiler: gcc
> +    dist: bionic
> +    addons:
> +      apt:
> +        packages:
> +          - *required_packages
> +          - *doc_packages
>  
>  script: ./.ci/${TRAVIS_OS_NAME}-build.sh

Re: [dpdk-dev] [PATCH v3 2/2] ci: add travis ci support for aarch64

[Ruifeng Wang]

> -----Original Message-----
> From: Aaron Conole <aconole@redhat.com>
> Sent: Monday, January 6, 2020 21:34
> To: Ruifeng Wang <Ruifeng.Wang@arm.com>
> Cc: maicolgabriel@hotmail.com; thomas@monjalon.net;
> ferruh.yigit@intel.com; arybchenko@solarflare.com; dev@dpdk.org;
> david.marchand@redhat.com; Gavin Hu <Gavin.Hu@arm.com>; Honnappa
> Nagarahalli <Honnappa.Nagarahalli@arm.com>; nd <nd@arm.com>
> Subject: Re: [PATCH v3 2/2] ci: add travis ci support for aarch64
> 
> Ruifeng Wang <ruifeng.wang@arm.com> writes:
> 
> > Add Travis compilation jobs for aarch64. gcc/clang compilations for
> > static/shared libraries are added.
> >
> > Some limitations for current aarch64 Travis support:
> > 1. Container is used. Huge page is not available due to security reason.
> > 2. Missing kernel header package in Xenial distribution.
> >
> > Solutions to address the limitations:
> > 1. Not to add unit test for now. And run tests with no-huge in future.
> > 2. Use Bionic distribution for all aarch64 jobs.
> >
> > Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> > Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> > ---
> 
> Can't we achieve the same thing by setting
> 
> arch:
>   - amd64
>   - arm64
> 
> in the build matrix?  Or will that also force the intel builds to use the container
> infrastructure (in which case the no-huge support needs to be fixed)?

No, container infrastructure will not be imposed to intel builds. 
AFAIN, Travis infrastructure for a specific CPU arch is provided as is, and there is no config option to control.
The problem with just adding 'arch' in build matrix is that RUN_TESTS on arm64 is not supported
by now (Travis limitation). 'env' with RUN_TESTS will fail.
> 
> One thing I wonder, isn't is possible to use qemu-user to do the amd64 unit
> tests?  Then do we really need some changes to do the native build?

Do you mean to use qemu-user to do unit tests for non-x86 arch?
Changes will be needed as well to enable qemu-user to do unit test.
Since Travis support multi CPU arch, I think native build and test is simpler and more natural. 

> Does it buy us anything *today* given the cost of the hugepage restriction?
> Will that ever be resolved (I didn't see so from the docs on travis)?

The hugepage issue has been reported to Travis. I think it will be resolved. But no set dates yet.
> 
> >  .ci/linux-setup.sh | 11 +++++++----
> >  .travis.yml        | 42 +++++++++++++++++++++++++++++++++++++++++-
> >  2 files changed, 48 insertions(+), 5 deletions(-)
> >
> > diff --git a/.ci/linux-setup.sh b/.ci/linux-setup.sh index
> > dfb9d4a20..a92978037 100755
> > --- a/.ci/linux-setup.sh
> > +++ b/.ci/linux-setup.sh
> > @@ -3,7 +3,10 @@
> >  # need to install as 'root' since some of the unit tests won't run
> > without it  sudo python3 -m pip install --upgrade meson
> >
> > -# setup hugepages
> > -cat /proc/meminfo
> > -sudo sh -c 'echo 1024 > /proc/sys/vm/nr_hugepages'
> > -cat /proc/meminfo
> > +# hugepage settings are skipped on aarch64 due to environment
> > +limitation if [ "$TRAVIS_ARCH" != "aarch64" ]; then
> > +    # setup hugepages
> > +    cat /proc/meminfo
> > +    sudo sh -c 'echo 1024 > /proc/sys/vm/nr_hugepages'
> > +    cat /proc/meminfo
> > +fi
> > diff --git a/.travis.yml b/.travis.yml index 8f90d06f2..980c7605d
> > 100644
> > --- a/.travis.yml
> > +++ b/.travis.yml
> > @@ -115,6 +115,46 @@ matrix:
> >        apt:
> >          packages:
> >            - *extra_packages
> > -
> > +  - env: DEF_LIB="static"
> > +    arch: arm64
> > +    compiler: gcc
> > +    dist: bionic
> > +    addons:
> > +      apt:
> > +        packages:
> > +          - *required_packages
> > +  - env: DEF_LIB="shared"
> > +    arch: arm64
> > +    compiler: gcc
> > +    dist: bionic
> > +    addons:
> > +      apt:
> > +        packages:
> > +          - *required_packages
> > +  - env: DEF_LIB="static"
> > +    arch: arm64
> > +    dist: bionic
> > +    compiler: clang
> > +    addons:
> > +      apt:
> > +        packages:
> > +          - *required_packages
> > +  - env: DEF_LIB="shared"
> > +    arch: arm64
> > +    dist: bionic
> > +    compiler: clang
> > +    addons:
> > +      apt:
> > +        packages:
> > +          - *required_packages
> > +  - env: DEF_LIB="shared" OPTS="-Denable_kmods=false" BUILD_DOCS=1
> > +    arch: arm64
> > +    compiler: gcc
> > +    dist: bionic
> > +    addons:
> > +      apt:
> > +        packages:
> > +          - *required_packages
> > +          - *doc_packages
> >
> >  script: ./.ci/${TRAVIS_OS_NAME}-build.sh

Re: [dpdk-dev] [PATCH v3 2/2] ci: add travis ci support for aarch64

[Honnappa Nagarahalli]

<snip>

> >
> > > Add Travis compilation jobs for aarch64. gcc/clang compilations for
> > > static/shared libraries are added.
> > >
> > > Some limitations for current aarch64 Travis support:
> > > 1. Container is used. Huge page is not available due to security reason.
> > > 2. Missing kernel header package in Xenial distribution.
> > >
> > > Solutions to address the limitations:
> > > 1. Not to add unit test for now. And run tests with no-huge in future.
> > > 2. Use Bionic distribution for all aarch64 jobs.
> > >
> > > Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> > > Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> > > ---
> >
> > Can't we achieve the same thing by setting
> >
> > arch:
> >   - amd64
> >   - arm64
> >
> > in the build matrix?  Or will that also force the intel builds to use
> > the container infrastructure (in which case the no-huge support needs to be
> fixed)?
> 
> No, container infrastructure will not be imposed to intel builds.
> AFAIN, Travis infrastructure for a specific CPU arch is provided as is, and
> there is no config option to control.
> The problem with just adding 'arch' in build matrix is that RUN_TESTS on
> arm64 is not supported by now (Travis limitation). 'env' with RUN_TESTS will
> fail.
> >
> > One thing I wonder, isn't is possible to use qemu-user to do the amd64
> > unit tests?  Then do we really need some changes to do the native build?
> 
> Do you mean to use qemu-user to do unit tests for non-x86 arch?
> Changes will be needed as well to enable qemu-user to do unit test.
> Since Travis support multi CPU arch, I think native build and test is simpler
> and more natural.
Yes, prefer to run the tests natively as the infrastructure is available and will improve further from here.

> 
> > Does it buy us anything *today* given the cost of the hugepage restriction?
> > Will that ever be resolved (I didn't see so from the docs on travis)?
> 
> The hugepage issue has been reported to Travis. I think it will be resolved.
> But no set dates yet.
> >
> > >  .ci/linux-setup.sh | 11 +++++++----
> > >  .travis.yml        | 42 +++++++++++++++++++++++++++++++++++++++++-
> > >  2 files changed, 48 insertions(+), 5 deletions(-)
> > >
> > > diff --git a/.ci/linux-setup.sh b/.ci/linux-setup.sh index
> > > dfb9d4a20..a92978037 100755
> > > --- a/.ci/linux-setup.sh
> > > +++ b/.ci/linux-setup.sh
> > > @@ -3,7 +3,10 @@
> > >  # need to install as 'root' since some of the unit tests won't run
> > > without it  sudo python3 -m pip install --upgrade meson
> > >
> > > -# setup hugepages
> > > -cat /proc/meminfo
> > > -sudo sh -c 'echo 1024 > /proc/sys/vm/nr_hugepages'
> > > -cat /proc/meminfo
> > > +# hugepage settings are skipped on aarch64 due to environment
> > > +limitation if [ "$TRAVIS_ARCH" != "aarch64" ]; then
> > > +    # setup hugepages
> > > +    cat /proc/meminfo
> > > +    sudo sh -c 'echo 1024 > /proc/sys/vm/nr_hugepages'
> > > +    cat /proc/meminfo
> > > +fi
> > > diff --git a/.travis.yml b/.travis.yml index 8f90d06f2..980c7605d
> > > 100644
> > > --- a/.travis.yml
> > > +++ b/.travis.yml
> > > @@ -115,6 +115,46 @@ matrix:
> > >        apt:
> > >          packages:
> > >            - *extra_packages
> > > -
> > > +  - env: DEF_LIB="static"
> > > +    arch: arm64
> > > +    compiler: gcc
> > > +    dist: bionic
> > > +    addons:
> > > +      apt:
> > > +        packages:
> > > +          - *required_packages
> > > +  - env: DEF_LIB="shared"
> > > +    arch: arm64
> > > +    compiler: gcc
> > > +    dist: bionic
> > > +    addons:
> > > +      apt:
> > > +        packages:
> > > +          - *required_packages
> > > +  - env: DEF_LIB="static"
> > > +    arch: arm64
> > > +    dist: bionic
> > > +    compiler: clang
> > > +    addons:
> > > +      apt:
> > > +        packages:
> > > +          - *required_packages
> > > +  - env: DEF_LIB="shared"
> > > +    arch: arm64
> > > +    dist: bionic
> > > +    compiler: clang
> > > +    addons:
> > > +      apt:
> > > +        packages:
> > > +          - *required_packages
> > > +  - env: DEF_LIB="shared" OPTS="-Denable_kmods=false" BUILD_DOCS=1
> > > +    arch: arm64
> > > +    compiler: gcc
> > > +    dist: bionic
> > > +    addons:
> > > +      apt:
> > > +        packages:
> > > +          - *required_packages
> > > +          - *doc_packages
> > >
> > >  script: ./.ci/${TRAVIS_OS_NAME}-build.sh
> 

Re: [dpdk-dev] [PATCH v3 2/2] ci: add travis ci support for aarch64

[Aaron Conole]

Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com> writes:

> <snip>
>
>> >
>> > > Add Travis compilation jobs for aarch64. gcc/clang compilations for
>> > > static/shared libraries are added.
>> > >
>> > > Some limitations for current aarch64 Travis support:
>> > > 1. Container is used. Huge page is not available due to security reason.
>> > > 2. Missing kernel header package in Xenial distribution.
>> > >
>> > > Solutions to address the limitations:
>> > > 1. Not to add unit test for now. And run tests with no-huge in future.
>> > > 2. Use Bionic distribution for all aarch64 jobs.
>> > >
>> > > Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
>> > > Reviewed-by: Gavin Hu <gavin.hu@arm.com>
>> > > ---
>> >
>> > Can't we achieve the same thing by setting
>> >
>> > arch:
>> >   - amd64
>> >   - arm64
>> >
>> > in the build matrix?  Or will that also force the intel builds to use
>> > the container infrastructure (in which case the no-huge support needs to be
>> fixed)?
>> 
>> No, container infrastructure will not be imposed to intel builds.
>> AFAIN, Travis infrastructure for a specific CPU arch is provided as is, and
>> there is no config option to control.
>> The problem with just adding 'arch' in build matrix is that RUN_TESTS on
>> arm64 is not supported by now (Travis limitation). 'env' with RUN_TESTS will
>> fail.
>> >
>> > One thing I wonder, isn't is possible to use qemu-user to do the amd64
>> > unit tests?  Then do we really need some changes to do the native build?
>> 
>> Do you mean to use qemu-user to do unit tests for non-x86 arch?
>> Changes will be needed as well to enable qemu-user to do unit test.
>> Since Travis support multi CPU arch, I think native build and test is simpler
>> and more natural.
> Yes, prefer to run the tests natively as the infrastructure is
> available and will improve further from here.

Agreed.  But we can't run the aarch64 tests for now, no matter which way
we slice it.

>> 
>> > Does it buy us anything *today* given the cost of the hugepage restriction?
>> > Will that ever be resolved (I didn't see so from the docs on travis)?
>> 
>> The hugepage issue has been reported to Travis. I think it will be resolved.
>> But no set dates yet.
>> >
>> > >  .ci/linux-setup.sh | 11 +++++++----
>> > >  .travis.yml        | 42 +++++++++++++++++++++++++++++++++++++++++-
>> > >  2 files changed, 48 insertions(+), 5 deletions(-)
>> > >
>> > > diff --git a/.ci/linux-setup.sh b/.ci/linux-setup.sh index
>> > > dfb9d4a20..a92978037 100755
>> > > --- a/.ci/linux-setup.sh
>> > > +++ b/.ci/linux-setup.sh
>> > > @@ -3,7 +3,10 @@
>> > >  # need to install as 'root' since some of the unit tests won't run
>> > > without it  sudo python3 -m pip install --upgrade meson
>> > >
>> > > -# setup hugepages
>> > > -cat /proc/meminfo
>> > > -sudo sh -c 'echo 1024 > /proc/sys/vm/nr_hugepages'
>> > > -cat /proc/meminfo
>> > > +# hugepage settings are skipped on aarch64 due to environment
>> > > +limitation if [ "$TRAVIS_ARCH" != "aarch64" ]; then
>> > > +    # setup hugepages
>> > > +    cat /proc/meminfo
>> > > +    sudo sh -c 'echo 1024 > /proc/sys/vm/nr_hugepages'
>> > > +    cat /proc/meminfo
>> > > +fi
>> > > diff --git a/.travis.yml b/.travis.yml index 8f90d06f2..980c7605d
>> > > 100644
>> > > --- a/.travis.yml
>> > > +++ b/.travis.yml
>> > > @@ -115,6 +115,46 @@ matrix:
>> > >        apt:
>> > >          packages:
>> > >            - *extra_packages
>> > > -
>> > > +  - env: DEF_LIB="static"
>> > > +    arch: arm64
>> > > +    compiler: gcc
>> > > +    dist: bionic
>> > > +    addons:
>> > > +      apt:
>> > > +        packages:
>> > > +          - *required_packages
>> > > +  - env: DEF_LIB="shared"
>> > > +    arch: arm64
>> > > +    compiler: gcc
>> > > +    dist: bionic
>> > > +    addons:
>> > > +      apt:
>> > > +        packages:
>> > > +          - *required_packages
>> > > +  - env: DEF_LIB="static"
>> > > +    arch: arm64
>> > > +    dist: bionic
>> > > +    compiler: clang
>> > > +    addons:
>> > > +      apt:
>> > > +        packages:
>> > > +          - *required_packages
>> > > +  - env: DEF_LIB="shared"
>> > > +    arch: arm64
>> > > +    dist: bionic
>> > > +    compiler: clang
>> > > +    addons:
>> > > +      apt:
>> > > +        packages:
>> > > +          - *required_packages
>> > > +  - env: DEF_LIB="shared" OPTS="-Denable_kmods=false" BUILD_DOCS=1
>> > > +    arch: arm64
>> > > +    compiler: gcc
>> > > +    dist: bionic
>> > > +    addons:
>> > > +      apt:
>> > > +        packages:
>> > > +          - *required_packages
>> > > +          - *doc_packages
>> > >
>> > >  script: ./.ci/${TRAVIS_OS_NAME}-build.sh
>> 

Re: [dpdk-dev] [PATCH v3 2/2] ci: add travis ci support for aarch64

[Aaron Conole]

Ruifeng Wang <Ruifeng.Wang@arm.com> writes:

>> -----Original Message-----
>> From: Aaron Conole <aconole@redhat.com>
>> Sent: Monday, January 6, 2020 21:34
>> To: Ruifeng Wang <Ruifeng.Wang@arm.com>
>> Cc: maicolgabriel@hotmail.com; thomas@monjalon.net;
>> ferruh.yigit@intel.com; arybchenko@solarflare.com; dev@dpdk.org;
>> david.marchand@redhat.com; Gavin Hu <Gavin.Hu@arm.com>; Honnappa
>> Nagarahalli <Honnappa.Nagarahalli@arm.com>; nd <nd@arm.com>
>> Subject: Re: [PATCH v3 2/2] ci: add travis ci support for aarch64
>> 
>> Ruifeng Wang <ruifeng.wang@arm.com> writes:
>> 
>> > Add Travis compilation jobs for aarch64. gcc/clang compilations for
>> > static/shared libraries are added.
>> >
>> > Some limitations for current aarch64 Travis support:
>> > 1. Container is used. Huge page is not available due to security reason.
>> > 2. Missing kernel header package in Xenial distribution.
>> >
>> > Solutions to address the limitations:
>> > 1. Not to add unit test for now. And run tests with no-huge in future.
>> > 2. Use Bionic distribution for all aarch64 jobs.
>> >
>> > Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
>> > Reviewed-by: Gavin Hu <gavin.hu@arm.com>
>> > ---
>> 
>> Can't we achieve the same thing by setting
>> 
>> arch:
>>   - amd64
>>   - arm64
>> 
>> in the build matrix?  Or will that also force the intel builds to use the container
>> infrastructure (in which case the no-huge support needs to be fixed)?
>
> No, container infrastructure will not be imposed to intel builds. 
> AFAIN, Travis infrastructure for a specific CPU arch is provided as
> is, and there is no config option to control.
> The problem with just adding 'arch' in build matrix is that RUN_TESTS on arm64 is not supported
> by now (Travis limitation). 'env' with RUN_TESTS will fail.

Okay I see.

>> 
>> One thing I wonder, isn't is possible to use qemu-user to do the amd64 unit
>> tests?  Then do we really need some changes to do the native build?
>
> Do you mean to use qemu-user to do unit tests for non-x86 arch?

Yes.  This has the advantage of giving users a way to also do the
multi-arch checks on their own systems (so a developer with just an x86
could at least do some testing on arm or ppc).

> Changes will be needed as well to enable qemu-user to do unit test.
> Since Travis support multi CPU arch, I think native build and test is simpler and more natural. 

I agree, some script changes might be needed, but maybe not as many as
you fear (can't we use binfmt_misc infrastructure to do this with
qemu-user and then the actual 'execute' would work).

>> Does it buy us anything *today* given the cost of the hugepage restriction?
>> Will that ever be resolved (I didn't see so from the docs on travis)?
>
> The hugepage issue has been reported to Travis. I think it will be
> resolved. But no set dates yet.

Is there a plan for them to address?  I guess probably not.  So we
either need the ability for tests to run in the no-huge environment (and
detect that no hugepages are available to run the tests that way), or we
need the travis environment supporting hugepages.  Is there something I
missed?

>> 
>> >  .ci/linux-setup.sh | 11 +++++++----
>> >  .travis.yml        | 42 +++++++++++++++++++++++++++++++++++++++++-
>> >  2 files changed, 48 insertions(+), 5 deletions(-)
>> >
>> > diff --git a/.ci/linux-setup.sh b/.ci/linux-setup.sh index
>> > dfb9d4a20..a92978037 100755
>> > --- a/.ci/linux-setup.sh
>> > +++ b/.ci/linux-setup.sh
>> > @@ -3,7 +3,10 @@
>> >  # need to install as 'root' since some of the unit tests won't run
>> > without it  sudo python3 -m pip install --upgrade meson
>> >
>> > -# setup hugepages
>> > -cat /proc/meminfo
>> > -sudo sh -c 'echo 1024 > /proc/sys/vm/nr_hugepages'
>> > -cat /proc/meminfo
>> > +# hugepage settings are skipped on aarch64 due to environment
>> > +limitation if [ "$TRAVIS_ARCH" != "aarch64" ]; then
>> > +    # setup hugepages
>> > +    cat /proc/meminfo
>> > +    sudo sh -c 'echo 1024 > /proc/sys/vm/nr_hugepages'
>> > +    cat /proc/meminfo
>> > +fi
>> > diff --git a/.travis.yml b/.travis.yml index 8f90d06f2..980c7605d
>> > 100644
>> > --- a/.travis.yml
>> > +++ b/.travis.yml
>> > @@ -115,6 +115,46 @@ matrix:
>> >        apt:
>> >          packages:
>> >            - *extra_packages
>> > -
>> > +  - env: DEF_LIB="static"
>> > +    arch: arm64
>> > +    compiler: gcc
>> > +    dist: bionic
>> > +    addons:
>> > +      apt:
>> > +        packages:
>> > +          - *required_packages
>> > +  - env: DEF_LIB="shared"
>> > +    arch: arm64
>> > +    compiler: gcc
>> > +    dist: bionic
>> > +    addons:
>> > +      apt:
>> > +        packages:
>> > +          - *required_packages
>> > +  - env: DEF_LIB="static"
>> > +    arch: arm64
>> > +    dist: bionic
>> > +    compiler: clang
>> > +    addons:
>> > +      apt:
>> > +        packages:
>> > +          - *required_packages
>> > +  - env: DEF_LIB="shared"
>> > +    arch: arm64
>> > +    dist: bionic
>> > +    compiler: clang
>> > +    addons:
>> > +      apt:
>> > +        packages:
>> > +          - *required_packages
>> > +  - env: DEF_LIB="shared" OPTS="-Denable_kmods=false" BUILD_DOCS=1
>> > +    arch: arm64
>> > +    compiler: gcc
>> > +    dist: bionic
>> > +    addons:
>> > +      apt:
>> > +        packages:
>> > +          - *required_packages
>> > +          - *doc_packages
>> >
>> >  script: ./.ci/${TRAVIS_OS_NAME}-build.sh

Re: [dpdk-dev] [PATCH v3 2/2] ci: add travis ci support for aarch64

[Bruce Richardson]

On Wed, Jan 08, 2020 at 11:05:21AM -0500, Aaron Conole wrote:
> Ruifeng Wang <Ruifeng.Wang@arm.com> writes:
> 
> >> -----Original Message-----
> >> From: Aaron Conole <aconole@redhat.com>
> >> Sent: Monday, January 6, 2020 21:34
> >> To: Ruifeng Wang <Ruifeng.Wang@arm.com>
> >> Cc: maicolgabriel@hotmail.com; thomas@monjalon.net;
> >> ferruh.yigit@intel.com; arybchenko@solarflare.com; dev@dpdk.org;
> >> david.marchand@redhat.com; Gavin Hu <Gavin.Hu@arm.com>; Honnappa
> >> Nagarahalli <Honnappa.Nagarahalli@arm.com>; nd <nd@arm.com>
> >> Subject: Re: [PATCH v3 2/2] ci: add travis ci support for aarch64
> >> 
> >> Ruifeng Wang <ruifeng.wang@arm.com> writes:
> >> 
> >> > Add Travis compilation jobs for aarch64. gcc/clang compilations for
> >> > static/shared libraries are added.
> >> >
> >> > Some limitations for current aarch64 Travis support:
> >> > 1. Container is used. Huge page is not available due to security reason.
> >> > 2. Missing kernel header package in Xenial distribution.
> >> >
> >> > Solutions to address the limitations:
> >> > 1. Not to add unit test for now. And run tests with no-huge in future.
> >> > 2. Use Bionic distribution for all aarch64 jobs.
> >> >
> >> > Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> >> > Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> >> > ---
> >> 
> >> Can't we achieve the same thing by setting
> >> 
> >> arch:
> >>   - amd64
> >>   - arm64
> >> 
> >> in the build matrix?  Or will that also force the intel builds to use the container
> >> infrastructure (in which case the no-huge support needs to be fixed)?
> >
> > No, container infrastructure will not be imposed to intel builds. 
> > AFAIN, Travis infrastructure for a specific CPU arch is provided as
> > is, and there is no config option to control.
> > The problem with just adding 'arch' in build matrix is that RUN_TESTS on arm64 is not supported
> > by now (Travis limitation). 'env' with RUN_TESTS will fail.
> 
> Okay I see.
> 
> >> 
> >> One thing I wonder, isn't is possible to use qemu-user to do the amd64 unit
> >> tests?  Then do we really need some changes to do the native build?
> >
> > Do you mean to use qemu-user to do unit tests for non-x86 arch?
> 
> Yes.  This has the advantage of giving users a way to also do the
> multi-arch checks on their own systems (so a developer with just an x86
> could at least do some testing on arm or ppc).
> 
> > Changes will be needed as well to enable qemu-user to do unit test.
> > Since Travis support multi CPU arch, I think native build and test is simpler and more natural. 
> 
> I agree, some script changes might be needed, but maybe not as many as
> you fear (can't we use binfmt_misc infrastructure to do this with
> qemu-user and then the actual 'execute' would work).
> 
> >> Does it buy us anything *today* given the cost of the hugepage restriction?
> >> Will that ever be resolved (I didn't see so from the docs on travis)?
> >
> > The hugepage issue has been reported to Travis. I think it will be
> > resolved. But no set dates yet.
> 
> Is there a plan for them to address?  I guess probably not.  So we
> either need the ability for tests to run in the no-huge environment (and
> detect that no hugepages are available to run the tests that way), or we
> need the travis environment supporting hugepages.  Is there something I
> missed?
> 
I think a reasonable number of tests should already run in a no-huge
environment. Ideally we could have autotest detect the fact it's running
with no-huge and skip all unsupported tests, but I think that would be
quite a bit of work to undertake, given the number of tests there are.

/Bruce

Re: [dpdk-dev] [PATCH v3 2/2] ci: add travis ci support for aarch64

[Ruifeng Wang]

> -----Original Message-----
> From: Aaron Conole <aconole@redhat.com>
> Sent: Thursday, January 9, 2020 00:05
> To: Ruifeng Wang <Ruifeng.Wang@arm.com>
> Cc: maicolgabriel@hotmail.com; thomas@monjalon.net;
> ferruh.yigit@intel.com; arybchenko@solarflare.com; dev@dpdk.org;
> david.marchand@redhat.com; Gavin Hu <Gavin.Hu@arm.com>; Honnappa
> Nagarahalli <Honnappa.Nagarahalli@arm.com>; nd <nd@arm.com>
> Subject: Re: [PATCH v3 2/2] ci: add travis ci support for aarch64
> 
> Ruifeng Wang <Ruifeng.Wang@arm.com> writes:
> 
> >> -----Original Message-----
> >> From: Aaron Conole <aconole@redhat.com>
> >> Sent: Monday, January 6, 2020 21:34
> >> To: Ruifeng Wang <Ruifeng.Wang@arm.com>
> >> Cc: maicolgabriel@hotmail.com; thomas@monjalon.net;
> >> ferruh.yigit@intel.com; arybchenko@solarflare.com; dev@dpdk.org;
> >> david.marchand@redhat.com; Gavin Hu <Gavin.Hu@arm.com>;
> Honnappa
> >> Nagarahalli <Honnappa.Nagarahalli@arm.com>; nd <nd@arm.com>
> >> Subject: Re: [PATCH v3 2/2] ci: add travis ci support for aarch64
> >>
> >> Ruifeng Wang <ruifeng.wang@arm.com> writes:
> >>
> >> > Add Travis compilation jobs for aarch64. gcc/clang compilations for
> >> > static/shared libraries are added.
> >> >
> >> > Some limitations for current aarch64 Travis support:
> >> > 1. Container is used. Huge page is not available due to security reason.
> >> > 2. Missing kernel header package in Xenial distribution.
> >> >
> >> > Solutions to address the limitations:
> >> > 1. Not to add unit test for now. And run tests with no-huge in future.
> >> > 2. Use Bionic distribution for all aarch64 jobs.
> >> >
> >> > Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> >> > Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> >> > ---
> >>
> >> Can't we achieve the same thing by setting
> >>
> >> arch:
> >>   - amd64
> >>   - arm64
> >>
> >> in the build matrix?  Or will that also force the intel builds to use
> >> the container infrastructure (in which case the no-huge support needs to
> be fixed)?
> >
> > No, container infrastructure will not be imposed to intel builds.
> > AFAIN, Travis infrastructure for a specific CPU arch is provided as
> > is, and there is no config option to control.
> > The problem with just adding 'arch' in build matrix is that RUN_TESTS
> > on arm64 is not supported by now (Travis limitation). 'env' with RUN_TESTS
> will fail.
> 
> Okay I see.
> 
> >>
> >> One thing I wonder, isn't is possible to use qemu-user to do the
> >> amd64 unit tests?  Then do we really need some changes to do the native
> build?
> >
> > Do you mean to use qemu-user to do unit tests for non-x86 arch?
> 
> Yes.  This has the advantage of giving users a way to also do the multi-arch
> checks on their own systems (so a developer with just an x86 could at least
> do some testing on arm or ppc).
> 
Yes, users can do multi-arch checks *locally* by using qemu. 
This patch aims to enable *public* CI for aarch64. It has no sense to rely on specific arch while infrastructure
supports multi arch.

> > Changes will be needed as well to enable qemu-user to do unit test.
> > Since Travis support multi CPU arch, I think native build and test is simpler
> and more natural.
> 
> I agree, some script changes might be needed, but maybe not as many as
> you fear (can't we use binfmt_misc infrastructure to do this with qemu-user
> and then the actual 'execute' would work).
> 
It is more like a tool for local validation, and should be another story.

> >> Does it buy us anything *today* given the cost of the hugepage
> restriction?
> >> Will that ever be resolved (I didn't see so from the docs on travis)?
> >
> > The hugepage issue has been reported to Travis. I think it will be
> > resolved. But no set dates yet.
> 
> Is there a plan for them to address?  I guess probably not.  So we either need
> the ability for tests to run in the no-huge environment (and detect that no
> hugepages are available to run the tests that way), or we need the travis
> environment supporting hugepages.  Is there something I missed?
> 
Yes, over half of quick tests can run in no-huge environment.

> >>
> >> >  .ci/linux-setup.sh | 11 +++++++----
> >> >  .travis.yml        | 42
> +++++++++++++++++++++++++++++++++++++++++-
> >> >  2 files changed, 48 insertions(+), 5 deletions(-)
> >> >
> >> > diff --git a/.ci/linux-setup.sh b/.ci/linux-setup.sh index
> >> > dfb9d4a20..a92978037 100755
> >> > --- a/.ci/linux-setup.sh
> >> > +++ b/.ci/linux-setup.sh
> >> > @@ -3,7 +3,10 @@
> >> >  # need to install as 'root' since some of the unit tests won't run
> >> > without it  sudo python3 -m pip install --upgrade meson
> >> >
> >> > -# setup hugepages
> >> > -cat /proc/meminfo
> >> > -sudo sh -c 'echo 1024 > /proc/sys/vm/nr_hugepages'
> >> > -cat /proc/meminfo
> >> > +# hugepage settings are skipped on aarch64 due to environment
> >> > +limitation if [ "$TRAVIS_ARCH" != "aarch64" ]; then
> >> > +    # setup hugepages
> >> > +    cat /proc/meminfo
> >> > +    sudo sh -c 'echo 1024 > /proc/sys/vm/nr_hugepages'
> >> > +    cat /proc/meminfo
> >> > +fi
> >> > diff --git a/.travis.yml b/.travis.yml index 8f90d06f2..980c7605d
> >> > 100644
> >> > --- a/.travis.yml
> >> > +++ b/.travis.yml
> >> > @@ -115,6 +115,46 @@ matrix:
> >> >        apt:
> >> >          packages:
> >> >            - *extra_packages
> >> > -
> >> > +  - env: DEF_LIB="static"
> >> > +    arch: arm64
> >> > +    compiler: gcc
> >> > +    dist: bionic
> >> > +    addons:
> >> > +      apt:
> >> > +        packages:
> >> > +          - *required_packages
> >> > +  - env: DEF_LIB="shared"
> >> > +    arch: arm64
> >> > +    compiler: gcc
> >> > +    dist: bionic
> >> > +    addons:
> >> > +      apt:
> >> > +        packages:
> >> > +          - *required_packages
> >> > +  - env: DEF_LIB="static"
> >> > +    arch: arm64
> >> > +    dist: bionic
> >> > +    compiler: clang
> >> > +    addons:
> >> > +      apt:
> >> > +        packages:
> >> > +          - *required_packages
> >> > +  - env: DEF_LIB="shared"
> >> > +    arch: arm64
> >> > +    dist: bionic
> >> > +    compiler: clang
> >> > +    addons:
> >> > +      apt:
> >> > +        packages:
> >> > +          - *required_packages
> >> > +  - env: DEF_LIB="shared" OPTS="-Denable_kmods=false"
> BUILD_DOCS=1
> >> > +    arch: arm64
> >> > +    compiler: gcc
> >> > +    dist: bionic
> >> > +    addons:
> >> > +      apt:
> >> > +        packages:
> >> > +          - *required_packages
> >> > +          - *doc_packages
> >> >
> >> >  script: ./.ci/${TRAVIS_OS_NAME}-build.sh

Re: [dpdk-dev] [PATCH v3 2/2] ci: add travis ci support for aarch64

[Honnappa Nagarahalli]

<snip>

> > >>
> > >> > Add Travis compilation jobs for aarch64. gcc/clang compilations
> > >> > for static/shared libraries are added.
> > >> >
> > >> > Some limitations for current aarch64 Travis support:
> > >> > 1. Container is used. Huge page is not available due to security reason.
> > >> > 2. Missing kernel header package in Xenial distribution.
> > >> >
> > >> > Solutions to address the limitations:
> > >> > 1. Not to add unit test for now. And run tests with no-huge in future.
> > >> > 2. Use Bionic distribution for all aarch64 jobs.
> > >> >
> > >> > Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> > >> > Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> > >> > ---
> > >>
> > >> Can't we achieve the same thing by setting
> > >>
> > >> arch:
> > >>   - amd64
> > >>   - arm64
> > >>
> > >> in the build matrix?  Or will that also force the intel builds to
> > >> use the container infrastructure (in which case the no-huge support
> > >> needs to
> > be fixed)?
> > >
> > > No, container infrastructure will not be imposed to intel builds.
> > > AFAIN, Travis infrastructure for a specific CPU arch is provided as
> > > is, and there is no config option to control.
> > > The problem with just adding 'arch' in build matrix is that
> > > RUN_TESTS on arm64 is not supported by now (Travis limitation).
> > > 'env' with RUN_TESTS
> > will fail.
> >
> > Okay I see.
> >
> > >>
> > >> One thing I wonder, isn't is possible to use qemu-user to do the
> > >> amd64 unit tests?  Then do we really need some changes to do the
> > >> native
> > build?
> > >
> > > Do you mean to use qemu-user to do unit tests for non-x86 arch?
> >
> > Yes.  This has the advantage of giving users a way to also do the
> > multi-arch checks on their own systems (so a developer with just an
> > x86 could at least do some testing on arm or ppc).
> >
> Yes, users can do multi-arch checks *locally* by using qemu.
> This patch aims to enable *public* CI for aarch64. It has no sense to rely on
> specific arch while infrastructure supports multi arch.
> 
> > > Changes will be needed as well to enable qemu-user to do unit test.
> > > Since Travis support multi CPU arch, I think native build and test
> > > is simpler
> > and more natural.
> >
> > I agree, some script changes might be needed, but maybe not as many as
> > you fear (can't we use binfmt_misc infrastructure to do this with
> > qemu-user and then the actual 'execute' would work).
> >
> It is more like a tool for local validation, and should be another story.
> 
> > >> Does it buy us anything *today* given the cost of the hugepage
> > restriction?
> > >> Will that ever be resolved (I didn't see so from the docs on travis)?
> > >
> > > The hugepage issue has been reported to Travis. I think it will be
> > > resolved. But no set dates yet.
> >
> > Is there a plan for them to address?  I guess probably not.  So we
> > either need the ability for tests to run in the no-huge environment
> > (and detect that no hugepages are available to run the tests that
> > way), or we need the travis environment supporting hugepages.  Is there
> something I missed?
> >
> Yes, over half of quick tests can run in no-huge environment.
Plan is to enable the testing for whatever works today and work on fixing the remaining test cases. Is this ok?

<snip>

Re: [dpdk-dev] [PATCH v3 2/2] ci: add travis ci support for aarch64

[Aaron Conole]

Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com> writes:

> <snip>
>
>> > >>
>> > >> > Add Travis compilation jobs for aarch64. gcc/clang compilations
>> > >> > for static/shared libraries are added.
>> > >> >
>> > >> > Some limitations for current aarch64 Travis support:
>> > >> > 1. Container is used. Huge page is not available due to security reason.
>> > >> > 2. Missing kernel header package in Xenial distribution.
>> > >> >
>> > >> > Solutions to address the limitations:
>> > >> > 1. Not to add unit test for now. And run tests with no-huge in future.
>> > >> > 2. Use Bionic distribution for all aarch64 jobs.
>> > >> >
>> > >> > Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
>> > >> > Reviewed-by: Gavin Hu <gavin.hu@arm.com>
>> > >> > ---
>> > >>
>> > >> Can't we achieve the same thing by setting
>> > >>
>> > >> arch:
>> > >>   - amd64
>> > >>   - arm64
>> > >>
>> > >> in the build matrix?  Or will that also force the intel builds to
>> > >> use the container infrastructure (in which case the no-huge support
>> > >> needs to
>> > be fixed)?
>> > >
>> > > No, container infrastructure will not be imposed to intel builds.
>> > > AFAIN, Travis infrastructure for a specific CPU arch is provided as
>> > > is, and there is no config option to control.
>> > > The problem with just adding 'arch' in build matrix is that
>> > > RUN_TESTS on arm64 is not supported by now (Travis limitation).
>> > > 'env' with RUN_TESTS
>> > will fail.
>> >
>> > Okay I see.
>> >
>> > >>
>> > >> One thing I wonder, isn't is possible to use qemu-user to do the
>> > >> amd64 unit tests?  Then do we really need some changes to do the
>> > >> native
>> > build?
>> > >
>> > > Do you mean to use qemu-user to do unit tests for non-x86 arch?
>> >
>> > Yes.  This has the advantage of giving users a way to also do the
>> > multi-arch checks on their own systems (so a developer with just an
>> > x86 could at least do some testing on arm or ppc).
>> >
>> Yes, users can do multi-arch checks *locally* by using qemu.
>> This patch aims to enable *public* CI for aarch64. It has no sense to rely on
>> specific arch while infrastructure supports multi arch.
>> 
>> > > Changes will be needed as well to enable qemu-user to do unit test.
>> > > Since Travis support multi CPU arch, I think native build and test
>> > > is simpler
>> > and more natural.
>> >
>> > I agree, some script changes might be needed, but maybe not as many as
>> > you fear (can't we use binfmt_misc infrastructure to do this with
>> > qemu-user and then the actual 'execute' would work).
>> >
>> It is more like a tool for local validation, and should be another story.
>> 
>> > >> Does it buy us anything *today* given the cost of the hugepage
>> > restriction?
>> > >> Will that ever be resolved (I didn't see so from the docs on travis)?
>> > >
>> > > The hugepage issue has been reported to Travis. I think it will be
>> > > resolved. But no set dates yet.
>> >
>> > Is there a plan for them to address?  I guess probably not.  So we
>> > either need the ability for tests to run in the no-huge environment
>> > (and detect that no hugepages are available to run the tests that
>> > way), or we need the travis environment supporting hugepages.  Is there
>> something I missed?
>> >
>> Yes, over half of quick tests can run in no-huge environment.
> Plan is to enable the testing for whatever works today and work on
> fixing the remaining test cases. Is this ok?

Sounds good then.

> <snip>

[dpdk-dev] [PATCH v4 0/2] add travis ci support for aarch64

[Ruifeng Wang]

This patch set is to enable native aarch64 build in Travis CI.
It leverages Travis CI multi arch support.

As the first step, compilation jobs are added.
Unit test is not added for now due to service limitation. We are
planning to run unit test with no-huge in future. Plan is to
enable the testing for whatever works today and work on fixing
the remaining test cases.

v4:
Test RUN_TESTS flag instead of arch when setting hugepage.

v3:
Reverse patches order. Not depending on other patches.
Restore distribution designation for aarch64 native jobs.

v2:
Removed dist designation from matrix since base dist is changing.
Add explanation for library path issue.

Ruifeng Wang (2):
  devtools: add path to additional shared object files
  ci: add travis ci support for aarch64

 .ci/linux-setup.sh    | 11 +++++++----
 .travis.yml           | 42 +++++++++++++++++++++++++++++++++++++++++-
 devtools/test-null.sh |  2 +-
 3 files changed, 49 insertions(+), 6 deletions(-)

-- 
2.17.1

[dpdk-dev] [PATCH v4 1/2] devtools: add path to additional shared object files

[Ruifeng Wang]

Drivers librte_mempool_ring.so and librte_pmd_null.so are loaded by
librte_eal.so when running testpmd.
In Ubuntu Xenial, driver path is installed to RPATH on testpmd. This
allows librte_eal.so to find drivers by using the RPATH.
However, in Ubuntu Bionic, driver path is installed to RUNPATH instead.
The RUNPATH on testpmd is not available by librte_eal.so and therefore
lead to driver load failure:

EAL: Detected 32 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: librte_mempool_ring.so: cannot open shared object file:
					No such file or directory
EAL: FATAL: Cannot init plugins
EAL: Cannot init plugins

Add 'drivers' into LD_LIBRARY_PATH so that testpmd can find and make
use of these shared libraries.

Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>
---
 devtools/test-null.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/devtools/test-null.sh b/devtools/test-null.sh
index f39af2c06..548de8113 100755
--- a/devtools/test-null.sh
+++ b/devtools/test-null.sh
@@ -20,7 +20,7 @@ if [ ! -f "$testpmd" ] ; then
 fi
 
 if ldd $testpmd | grep -q librte_ ; then
-	export LD_LIBRARY_PATH=$build/lib:$LD_LIBRARY_PATH
+	export LD_LIBRARY_PATH=$build/drivers:$build/lib:$LD_LIBRARY_PATH
 	libs='-d librte_mempool_ring.so -d librte_pmd_null.so'
 else
 	libs=
-- 
2.17.1

Re: [dpdk-dev] [PATCH v4 1/2] devtools: add path to additional shared object files

[Aaron Conole]

Ruifeng Wang <ruifeng.wang@arm.com> writes:

> Drivers librte_mempool_ring.so and librte_pmd_null.so are loaded by
> librte_eal.so when running testpmd.
> In Ubuntu Xenial, driver path is installed to RPATH on testpmd. This
> allows librte_eal.so to find drivers by using the RPATH.
> However, in Ubuntu Bionic, driver path is installed to RUNPATH instead.
> The RUNPATH on testpmd is not available by librte_eal.so and therefore
> lead to driver load failure:
>
> EAL: Detected 32 lcore(s)
> EAL: Detected 1 NUMA nodes
> EAL: librte_mempool_ring.so: cannot open shared object file:
> 					No such file or directory
> EAL: FATAL: Cannot init plugins
> EAL: Cannot init plugins
>
> Add 'drivers' into LD_LIBRARY_PATH so that testpmd can find and make
> use of these shared libraries.
>
> Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> ---

Acked-by: Aaron Conole <aconole@redhat.com>

[dpdk-dev] [PATCH v4 2/2] ci: add travis ci support for aarch64

[Ruifeng Wang]

Add Travis compilation jobs for aarch64. gcc/clang compilations for
static/shared libraries are added.

Some limitations for current aarch64 Travis support:
1. Container is used. Huge page is not available due to security reason.
2. Missing kernel header package in Xenial distribution.

Solutions to address the limitations:
1. Not to add unit test for now. And run tests with no-huge in future.
2. Use Bionic distribution for all aarch64 jobs.

Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>
---
 .ci/linux-setup.sh | 11 +++++++----
 .travis.yml        | 42 +++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 48 insertions(+), 5 deletions(-)

diff --git a/.ci/linux-setup.sh b/.ci/linux-setup.sh
index dfb9d4a20..f93e67664 100755
--- a/.ci/linux-setup.sh
+++ b/.ci/linux-setup.sh
@@ -3,7 +3,10 @@
 # need to install as 'root' since some of the unit tests won't run without it
 sudo python3 -m pip install --upgrade meson
 
-# setup hugepages
-cat /proc/meminfo
-sudo sh -c 'echo 1024 > /proc/sys/vm/nr_hugepages'
-cat /proc/meminfo
+# skip hugepage settings if tests will not run
+if [ "$RUN_TESTS" = "1" ]; then
+    # setup hugepages
+    cat /proc/meminfo
+    sudo sh -c 'echo 1024 > /proc/sys/vm/nr_hugepages'
+    cat /proc/meminfo
+fi
diff --git a/.travis.yml b/.travis.yml
index 8f90d06f2..980c7605d 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -115,6 +115,46 @@ matrix:
       apt:
         packages:
           - *extra_packages
-
+  - env: DEF_LIB="static"
+    arch: arm64
+    compiler: gcc
+    dist: bionic
+    addons:
+      apt:
+        packages:
+          - *required_packages
+  - env: DEF_LIB="shared"
+    arch: arm64
+    compiler: gcc
+    dist: bionic
+    addons:
+      apt:
+        packages:
+          - *required_packages
+  - env: DEF_LIB="static"
+    arch: arm64
+    dist: bionic
+    compiler: clang
+    addons:
+      apt:
+        packages:
+          - *required_packages
+  - env: DEF_LIB="shared"
+    arch: arm64
+    dist: bionic
+    compiler: clang
+    addons:
+      apt:
+        packages:
+          - *required_packages
+  - env: DEF_LIB="shared" OPTS="-Denable_kmods=false" BUILD_DOCS=1
+    arch: arm64
+    compiler: gcc
+    dist: bionic
+    addons:
+      apt:
+        packages:
+          - *required_packages
+          - *doc_packages
 
 script: ./.ci/${TRAVIS_OS_NAME}-build.sh
-- 
2.17.1

Re: [dpdk-dev] [PATCH v4 2/2] ci: add travis ci support for aarch64

[Aaron Conole]

Ruifeng Wang <ruifeng.wang@arm.com> writes:

> Add Travis compilation jobs for aarch64. gcc/clang compilations for
> static/shared libraries are added.
>
> Some limitations for current aarch64 Travis support:
> 1. Container is used. Huge page is not available due to security reason.
> 2. Missing kernel header package in Xenial distribution.
>
> Solutions to address the limitations:
> 1. Not to add unit test for now. And run tests with no-huge in future.
> 2. Use Bionic distribution for all aarch64 jobs.
>
> Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> ---

I think the above text should probably include 'native' since there are
already aarch64 builds that go (but they are cross compiled).  I also am
glad that you didn't remove them.

Acked-by: Aaron Conole <aconole@redhat.com>

Re: [dpdk-dev] [PATCH v4 2/2] ci: add travis ci support for aarch64

[Ruifeng Wang]

> -----Original Message-----
> From: Aaron Conole <aconole@redhat.com>
> Sent: Friday, January 10, 2020 23:13
> To: Ruifeng Wang <Ruifeng.Wang@arm.com>
> Cc: maicolgabriel@hotmail.com; thomas@monjalon.net;
> ferruh.yigit@intel.com; arybchenko@solarflare.com; dev@dpdk.org;
> david.marchand@redhat.com; Gavin Hu <Gavin.Hu@arm.com>; Honnappa
> Nagarahalli <Honnappa.Nagarahalli@arm.com>; nd <nd@arm.com>
> Subject: Re: [PATCH v4 2/2] ci: add travis ci support for aarch64
> 
> Ruifeng Wang <ruifeng.wang@arm.com> writes:
> 
> > Add Travis compilation jobs for aarch64. gcc/clang compilations for
> > static/shared libraries are added.
> >
> > Some limitations for current aarch64 Travis support:
> > 1. Container is used. Huge page is not available due to security reason.
> > 2. Missing kernel header package in Xenial distribution.
> >
> > Solutions to address the limitations:
> > 1. Not to add unit test for now. And run tests with no-huge in future.
> > 2. Use Bionic distribution for all aarch64 jobs.
> >
> > Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> > Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> > ---
> 
> I think the above text should probably include 'native' since there are already
> aarch64 builds that go (but they are cross compiled).  I also am glad that you
> didn't remove them.
> 
Agree. 
I will submit new version with commit message update, and take your Ack.
Thank you.

> Acked-by: Aaron Conole <aconole@redhat.com>

[dpdk-dev] [PATCH v5 0/2] add travis ci support for native aarch64

[Ruifeng Wang]

This patch set is to enable native aarch64 build in Travis CI.
It leverages Travis CI multi arch support.

As the first step, compilation jobs are added.
Unit test is not added for now due to service limitation. We are
planning to run unit test with no-huge in future. Plan is to
enable the testing for whatever works today and work on fixing
the remaining test cases.

v5:
Update commit message for 'native' build.
Taking Ack by Aaron from v4 with suggested commit message change.

v4:
Test RUN_TESTS flag instead of arch when setting hugepage.

v3:
Reverse patches order. Not depending on other patches.
Restore distribution designation for aarch64 native jobs.

v2:
Removed dist designation from matrix since base dist is changing.
Add explanation for library path issue.

Ruifeng Wang (2):
  devtools: add path to additional shared object files
  ci: add travis ci support for native aarch64

 .ci/linux-setup.sh    | 11 +++++++----
 .travis.yml           | 42 +++++++++++++++++++++++++++++++++++++++++-
 devtools/test-null.sh |  2 +-
 3 files changed, 49 insertions(+), 6 deletions(-)

-- 
2.17.1

[dpdk-dev] [PATCH v5 1/2] devtools: add path to additional shared object files

[Ruifeng Wang]

Drivers librte_mempool_ring.so and librte_pmd_null.so are loaded by
librte_eal.so when running testpmd.
In Ubuntu Xenial, driver path is installed to RPATH on testpmd. This
allows librte_eal.so to find drivers by using the RPATH.
However, in Ubuntu Bionic, driver path is installed to RUNPATH instead.
The RUNPATH on testpmd is not available by librte_eal.so and therefore
lead to driver load failure:

EAL: Detected 32 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: librte_mempool_ring.so: cannot open shared object file:
					No such file or directory
EAL: FATAL: Cannot init plugins
EAL: Cannot init plugins

Add 'drivers' into LD_LIBRARY_PATH so that testpmd can find and make
use of these shared libraries.

Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>
Acked-by: Aaron Conole <aconole@redhat.com>
---
 devtools/test-null.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/devtools/test-null.sh b/devtools/test-null.sh
index f39af2c06..548de8113 100755
--- a/devtools/test-null.sh
+++ b/devtools/test-null.sh
@@ -20,7 +20,7 @@ if [ ! -f "$testpmd" ] ; then
 fi
 
 if ldd $testpmd | grep -q librte_ ; then
-	export LD_LIBRARY_PATH=$build/lib:$LD_LIBRARY_PATH
+	export LD_LIBRARY_PATH=$build/drivers:$build/lib:$LD_LIBRARY_PATH
 	libs='-d librte_mempool_ring.so -d librte_pmd_null.so'
 else
 	libs=
-- 
2.17.1

[dpdk-dev] [PATCH v5 2/2] ci: add travis ci support for native aarch64

[Ruifeng Wang]

Add Travis compilation jobs for native aarch64. gcc/clang compilations
for static/shared libraries are added.

Some limitations for current aarch64 Travis support:
1. Container is used. Huge page is not available due to security reason.
2. Missing kernel header package in Xenial distribution.

Solutions to address the limitations:
1. Not to add unit test for now. And run tests with no-huge in future.
2. Use Bionic distribution for all aarch64 jobs.

Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>
Acked-by: Aaron Conole <aconole@redhat.com>
---
 .ci/linux-setup.sh | 11 +++++++----
 .travis.yml        | 42 +++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 48 insertions(+), 5 deletions(-)

diff --git a/.ci/linux-setup.sh b/.ci/linux-setup.sh
index dfb9d4a20..f93e67664 100755
--- a/.ci/linux-setup.sh
+++ b/.ci/linux-setup.sh
@@ -3,7 +3,10 @@
 # need to install as 'root' since some of the unit tests won't run without it
 sudo python3 -m pip install --upgrade meson
 
-# setup hugepages
-cat /proc/meminfo
-sudo sh -c 'echo 1024 > /proc/sys/vm/nr_hugepages'
-cat /proc/meminfo
+# skip hugepage settings if tests will not run
+if [ "$RUN_TESTS" = "1" ]; then
+    # setup hugepages
+    cat /proc/meminfo
+    sudo sh -c 'echo 1024 > /proc/sys/vm/nr_hugepages'
+    cat /proc/meminfo
+fi
diff --git a/.travis.yml b/.travis.yml
index 8f90d06f2..980c7605d 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -115,6 +115,46 @@ matrix:
       apt:
         packages:
           - *extra_packages
-
+  - env: DEF_LIB="static"
+    arch: arm64
+    compiler: gcc
+    dist: bionic
+    addons:
+      apt:
+        packages:
+          - *required_packages
+  - env: DEF_LIB="shared"
+    arch: arm64
+    compiler: gcc
+    dist: bionic
+    addons:
+      apt:
+        packages:
+          - *required_packages
+  - env: DEF_LIB="static"
+    arch: arm64
+    dist: bionic
+    compiler: clang
+    addons:
+      apt:
+        packages:
+          - *required_packages
+  - env: DEF_LIB="shared"
+    arch: arm64
+    dist: bionic
+    compiler: clang
+    addons:
+      apt:
+        packages:
+          - *required_packages
+  - env: DEF_LIB="shared" OPTS="-Denable_kmods=false" BUILD_DOCS=1
+    arch: arm64
+    compiler: gcc
+    dist: bionic
+    addons:
+      apt:
+        packages:
+          - *required_packages
+          - *doc_packages
 
 script: ./.ci/${TRAVIS_OS_NAME}-build.sh
-- 
2.17.1

Re: [dpdk-dev] [PATCH v5 0/2] add travis ci support for native aarch64

[David Marchand]

On Mon, Jan 13, 2020 at 7:26 AM Ruifeng Wang <ruifeng.wang@arm.com> wrote:
>
> This patch set is to enable native aarch64 build in Travis CI.
> It leverages Travis CI multi arch support.
>
> As the first step, compilation jobs are added.
> Unit test is not added for now due to service limitation. We are
> planning to run unit test with no-huge in future. Plan is to
> enable the testing for whatever works today and work on fixing
> the remaining test cases.

Series applied, thanks.


--
David Marchand