DPDK patches and discussions
 help / color / mirror / Atom feed
* [dpdk-dev] [PATCH 0/4] Reorganise Travis jobs
@ 2020-02-19 19:41 David Marchand
  2020-02-19 19:41 ` [dpdk-dev] [PATCH 1/4] ci: remove unnecessary dependency on Linux headers David Marchand
                   ` (4 more replies)
  0 siblings, 5 replies; 24+ messages in thread
From: David Marchand @ 2020-02-19 19:41 UTC (permalink / raw)
  To: thomas, aconole; +Cc: dev

This series tries to make the travis configuration a bit easier to read,
then drops jobs that are not essential or functional duplicates in the
hope we can give some oxygen to the robot who seems to lag quite a bit
and can't catch up.

-- 
David Marchand

David Marchand (4):
  ci: remove unnecessary dependency on linux headers
  ci: fix Travis config warnings
  ci: use an explicit list of Travis jobs
  ci: reorganise Travis jobs

 .travis.yml | 139 ++++++++++++++++++++--------------------------------
 1 file changed, 54 insertions(+), 85 deletions(-)

-- 
2.23.0


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [dpdk-dev] [PATCH 1/4] ci: remove unnecessary dependency on Linux headers
  2020-02-19 19:41 [dpdk-dev] [PATCH 0/4] Reorganise Travis jobs David Marchand
@ 2020-02-19 19:41 ` David Marchand
  2020-02-20 10:44   ` Thomas Monjalon
  2020-02-20 14:37   ` Aaron Conole
  2020-02-19 19:41 ` [dpdk-dev] [PATCH 2/4] ci: fix Travis config warnings David Marchand
                   ` (3 subsequent siblings)
  4 siblings, 2 replies; 24+ messages in thread
From: David Marchand @ 2020-02-19 19:41 UTC (permalink / raw)
  To: thomas, aconole; +Cc: dev, Michael Santana

Following removal of kmod compilation, we don't need to install
linux-headers anymore.

Fixes: ea860973592b ("ci: remove redundant configs disabling kmods")

Signed-off-by: David Marchand <david.marchand@redhat.com>
---
 .travis.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.travis.yml b/.travis.yml
index 1d43f16ae..671447ee8 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -17,7 +17,7 @@ addons:
   apt:
     update: true
     packages: &required_packages
-      - [libnuma-dev, linux-headers-$(uname -r), python3-setuptools, python3-wheel, python3-pip, ninja-build]
+      - [libnuma-dev, python3-setuptools, python3-wheel, python3-pip, ninja-build]
 
 aarch64_packages: &aarch64_packages
   - *required_packages
-- 
2.23.0


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [dpdk-dev] [PATCH 2/4] ci: fix Travis config warnings
  2020-02-19 19:41 [dpdk-dev] [PATCH 0/4] Reorganise Travis jobs David Marchand
  2020-02-19 19:41 ` [dpdk-dev] [PATCH 1/4] ci: remove unnecessary dependency on Linux headers David Marchand
@ 2020-02-19 19:41 ` David Marchand
  2020-02-20 11:03   ` Thomas Monjalon
  2020-02-20 14:36   ` Aaron Conole
  2020-02-19 19:41 ` [dpdk-dev] [PATCH 3/4] ci: use an explicit list of Travis jobs David Marchand
                   ` (2 subsequent siblings)
  4 siblings, 2 replies; 24+ messages in thread
From: David Marchand @ 2020-02-19 19:41 UTC (permalink / raw)
  To: thomas, aconole; +Cc: dev, Michael Santana

Reading https://config.travis-ci.com/ and using
https://config.travis-ci.com/explore to check changes, we can cleanup
some warnings reported by the config validation options in Travis.

Signed-off-by: David Marchand <david.marchand@redhat.com>
---
 .travis.yml | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index 671447ee8..7c1b2ff1b 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -19,22 +19,22 @@ addons:
     packages: &required_packages
       - [libnuma-dev, python3-setuptools, python3-wheel, python3-pip, ninja-build]
 
-aarch64_packages: &aarch64_packages
+_aarch64_packages: &aarch64_packages
   - *required_packages
   - [gcc-aarch64-linux-gnu, libc6-dev-arm64-cross, pkg-config-aarch64-linux-gnu]
 
-extra_packages: &extra_packages
+_extra_packages: &extra_packages
   - *required_packages
   - [libbsd-dev, libpcap-dev, libibverbs-dev, libcrypto++-dev, libfdt-dev, libjansson4]
 
-libabigail_build_packages: &libabigail_build_packages
+_libabigail_build_packages: &libabigail_build_packages
   - [autoconf, automake, libtool, pkg-config, libxml2-dev, libdw-dev]
 
-build_32b_packages: &build_32b_packages
+_build_32b_packages: &build_32b_packages
   - *required_packages
   - [gcc-multilib]
 
-doc_packages: &doc_packages
+_doc_packages: &doc_packages
   - [doxygen, graphviz, python3-sphinx]
 
 before_install: ./.ci/${TRAVIS_OS_NAME}-setup.sh
@@ -44,7 +44,7 @@ env:
   - DEF_LIB="shared"
   - DEF_LIB="shared" RUN_TESTS=1
 
-matrix:
+jobs:
   include:
   - env: DEF_LIB="static" BUILD_32BIT=1
     compiler: gcc
-- 
2.23.0


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [dpdk-dev] [PATCH 3/4] ci: use an explicit list of Travis jobs
  2020-02-19 19:41 [dpdk-dev] [PATCH 0/4] Reorganise Travis jobs David Marchand
  2020-02-19 19:41 ` [dpdk-dev] [PATCH 1/4] ci: remove unnecessary dependency on Linux headers David Marchand
  2020-02-19 19:41 ` [dpdk-dev] [PATCH 2/4] ci: fix Travis config warnings David Marchand
@ 2020-02-19 19:41 ` David Marchand
  2020-02-20 11:05   ` Thomas Monjalon
  2020-02-20 14:35   ` Aaron Conole
  2020-02-19 19:41 ` [dpdk-dev] [PATCH 4/4] ci: reorganise " David Marchand
  2020-02-20 21:05 ` [dpdk-dev] [PATCH 0/4] Reorganise " David Marchand
  4 siblings, 2 replies; 24+ messages in thread
From: David Marchand @ 2020-02-19 19:41 UTC (permalink / raw)
  To: thomas, aconole; +Cc: dev, Michael Santana

Maintaining the .travis.yml requires some knowledge of how Travis
computes the jobs list (combination of os: arch: compiler: etc...).
Let's switch to an explicit list to find all jobs at a glance.

To enhance readability, jobs have been sorted per arch/compiler with
comments to isolate blocks.

Setting required_packages for aarch64 native jobs is unnecessary,
the global addons: values are the same.

This commit does not change the jobs list (21 jobs in total).

Signed-off-by: David Marchand <david.marchand@redhat.com>
---
 .travis.yml | 102 ++++++++++++++++++++++++++--------------------------
 1 file changed, 52 insertions(+), 50 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index 7c1b2ff1b..c174d5f30 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -1,17 +1,14 @@
+# default values for all jobs
 language: c
 cache:
   ccache: true
   directories:
     - libabigail
     - reference
-compiler:
-  - gcc
-  - clang
 
 dist: bionic
 
-os:
-  - linux
+os: linux
 
 addons:
   apt:
@@ -38,98 +35,105 @@ _doc_packages: &doc_packages
   - [doxygen, graphviz, python3-sphinx]
 
 before_install: ./.ci/${TRAVIS_OS_NAME}-setup.sh
-
-env:
-  - DEF_LIB="static"
-  - DEF_LIB="shared"
-  - DEF_LIB="shared" RUN_TESTS=1
+script: ./.ci/${TRAVIS_OS_NAME}-build.sh
 
 jobs:
   include:
-  - env: DEF_LIB="static" BUILD_32BIT=1
+  # x86_64 gcc jobs
+  - env: DEF_LIB="static"
+    arch: amd64
     compiler: gcc
-    addons:
-      apt:
-        packages:
-          - *build_32b_packages
-  - env: DEF_LIB="static" AARCH64=1
+  - env: DEF_LIB="shared"
+    arch: amd64
     compiler: gcc
-    addons:
-      apt:
-        packages:
-          - *aarch64_packages
-  - env: DEF_LIB="shared" AARCH64=1
+  - env: DEF_LIB="shared" RUN_TESTS=1
+    arch: amd64
     compiler: gcc
-    addons:
-      apt:
-        packages:
-          - *aarch64_packages
   - env: DEF_LIB="static" EXTRA_PACKAGES=1
+    arch: amd64
     compiler: gcc
     addons:
       apt:
         packages:
           - *extra_packages
   - env: DEF_LIB="shared" EXTRA_PACKAGES=1 RUN_TESTS=1 BUILD_DOCS=1
+    arch: amd64
     compiler: gcc
     addons:
       apt:
         packages:
           - *extra_packages
           - *doc_packages
-  - env: DEF_LIB="static" EXTRA_PACKAGES=1
+  - env: DEF_LIB="shared" EXTRA_PACKAGES=1 ABI_CHECKS=1
+    arch: amd64
     compiler: gcc
     addons:
       apt:
         packages:
           - *extra_packages
+          - *libabigail_build_packages
+  # x86_64 clang jobs
+  - env: DEF_LIB="static"
+    arch: amd64
+    compiler: clang
+  - env: DEF_LIB="shared"
+    arch: amd64
+    compiler: clang
+  - env: DEF_LIB="shared" RUN_TESTS=1
+    arch: amd64
+    compiler: clang
   - env: DEF_LIB="static" EXTRA_PACKAGES=1
+    arch: amd64
     compiler: clang
     addons:
       apt:
         packages:
           - *extra_packages
   - env: DEF_LIB="shared" EXTRA_PACKAGES=1
+    arch: amd64
     compiler: clang
     addons:
       apt:
         packages:
           - *extra_packages
   - env: DEF_LIB="shared" EXTRA_PACKAGES=1 RUN_TESTS=1 BUILD_DOCS=1
+    arch: amd64
     compiler: clang
     addons:
       apt:
         packages:
           - *extra_packages
           - *doc_packages
-  - env: DEF_LIB="static"
-    arch: arm64
+  # x86_64 cross-compiling 32-bits jobs
+  - env: DEF_LIB="static" BUILD_32BIT=1
+    arch: amd64
     compiler: gcc
     addons:
       apt:
         packages:
-          - *required_packages
-  - env: DEF_LIB="shared"
-    arch: arm64
+          - *build_32b_packages
+  # x86_64 cross-compiling aarch64 jobs
+  - env: DEF_LIB="static" AARCH64=1
+    arch: amd64
     compiler: gcc
     addons:
       apt:
         packages:
-          - *required_packages
-  - env: DEF_LIB="static"
-    arch: arm64
-    compiler: clang
+          - *aarch64_packages
+  - env: DEF_LIB="shared" AARCH64=1
+    arch: amd64
+    compiler: gcc
     addons:
       apt:
         packages:
-          - *required_packages
+          - *aarch64_packages
+  # aarch64 gcc jobs
+  - env: DEF_LIB="static"
+    arch: arm64
+    compiler: gcc
   - env: DEF_LIB="shared"
     arch: arm64
-    compiler: clang
-    addons:
-      apt:
-        packages:
-          - *required_packages
+    compiler: gcc
   - env: DEF_LIB="shared" BUILD_DOCS=1
     arch: arm64
     compiler: gcc
@@ -138,13 +142,6 @@ jobs:
         packages:
           - *required_packages
           - *doc_packages
-  - env: DEF_LIB="shared" EXTRA_PACKAGES=1 ABI_CHECKS=1
-    compiler: gcc
-    addons:
-      apt:
-        packages:
-          - *extra_packages
-          - *libabigail_build_packages
   - env: DEF_LIB="shared" EXTRA_PACKAGES=1 ABI_CHECKS=1
     arch: arm64
     compiler: gcc
@@ -153,5 +150,10 @@ jobs:
         packages:
           - *extra_packages
           - *libabigail_build_packages
-
-script: ./.ci/${TRAVIS_OS_NAME}-build.sh
+  # aarch64 clang jobs
+  - env: DEF_LIB="static"
+    arch: arm64
+    compiler: clang
+  - env: DEF_LIB="shared"
+    arch: arm64
+    compiler: clang
-- 
2.23.0


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [dpdk-dev] [PATCH 4/4] ci: reorganise Travis jobs
  2020-02-19 19:41 [dpdk-dev] [PATCH 0/4] Reorganise Travis jobs David Marchand
                   ` (2 preceding siblings ...)
  2020-02-19 19:41 ` [dpdk-dev] [PATCH 3/4] ci: use an explicit list of Travis jobs David Marchand
@ 2020-02-19 19:41 ` David Marchand
  2020-02-19 21:39   ` Aaron Conole
                     ` (2 more replies)
  2020-02-20 21:05 ` [dpdk-dev] [PATCH 0/4] Reorganise " David Marchand
  4 siblings, 3 replies; 24+ messages in thread
From: David Marchand @ 2020-02-19 19:41 UTC (permalink / raw)
  To: thomas, aconole; +Cc: dev, Michael Santana

Let's prune the jobs list to limit the amount of time spent by the robot
in Travis.

Since meson enables automatically the relevant components, there is not
much gain in testing with extra_packages vs required_packages only.

For a given arch/compiler/env combination, compilation is first tested
in all jobs that run tests or build the docs or run the ABI checks.
In the same context, for jobs that accumulates running tests, building
the docs etc..., those steps are independent and can be split to save
some cpu on Travis.

With this, we go down from 21 to 15 jobs.

Note: this patch requires a flush of the existing caches in Travis.

Signed-off-by: David Marchand <david.marchand@redhat.com>
---
 .travis.yml | 51 +++++++++------------------------------------------
 1 file changed, 9 insertions(+), 42 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index c174d5f30..b64a81bd0 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -15,15 +15,12 @@ addons:
     update: true
     packages: &required_packages
       - [libnuma-dev, python3-setuptools, python3-wheel, python3-pip, ninja-build]
+      - [libbsd-dev, libpcap-dev, libibverbs-dev, libcrypto++-dev, libfdt-dev, libjansson4]
 
 _aarch64_packages: &aarch64_packages
   - *required_packages
   - [gcc-aarch64-linux-gnu, libc6-dev-arm64-cross, pkg-config-aarch64-linux-gnu]
 
-_extra_packages: &extra_packages
-  - *required_packages
-  - [libbsd-dev, libpcap-dev, libibverbs-dev, libcrypto++-dev, libfdt-dev, libjansson4]
-
 _libabigail_build_packages: &libabigail_build_packages
   - [autoconf, automake, libtool, pkg-config, libxml2-dev, libdw-dev]
 
@@ -43,66 +40,39 @@ jobs:
   - env: DEF_LIB="static"
     arch: amd64
     compiler: gcc
-  - env: DEF_LIB="shared"
-    arch: amd64
-    compiler: gcc
   - env: DEF_LIB="shared" RUN_TESTS=1
     arch: amd64
     compiler: gcc
-  - env: DEF_LIB="static" EXTRA_PACKAGES=1
-    arch: amd64
-    compiler: gcc
-    addons:
-      apt:
-        packages:
-          - *extra_packages
-  - env: DEF_LIB="shared" EXTRA_PACKAGES=1 RUN_TESTS=1 BUILD_DOCS=1
+  - env: DEF_LIB="shared" BUILD_DOCS=1
     arch: amd64
     compiler: gcc
     addons:
       apt:
         packages:
-          - *extra_packages
+          - *required_packages
           - *doc_packages
-  - env: DEF_LIB="shared" EXTRA_PACKAGES=1 ABI_CHECKS=1
+  - env: DEF_LIB="shared" ABI_CHECKS=1
     arch: amd64
     compiler: gcc
     addons:
       apt:
         packages:
-          - *extra_packages
+          - *required_packages
           - *libabigail_build_packages
   # x86_64 clang jobs
   - env: DEF_LIB="static"
     arch: amd64
     compiler: clang
-  - env: DEF_LIB="shared"
-    arch: amd64
-    compiler: clang
   - env: DEF_LIB="shared" RUN_TESTS=1
     arch: amd64
     compiler: clang
-  - env: DEF_LIB="static" EXTRA_PACKAGES=1
-    arch: amd64
-    compiler: clang
-    addons:
-      apt:
-        packages:
-          - *extra_packages
-  - env: DEF_LIB="shared" EXTRA_PACKAGES=1
-    arch: amd64
-    compiler: clang
-    addons:
-      apt:
-        packages:
-          - *extra_packages
-  - env: DEF_LIB="shared" EXTRA_PACKAGES=1 RUN_TESTS=1 BUILD_DOCS=1
+  - env: DEF_LIB="shared" BUILD_DOCS=1
     arch: amd64
     compiler: clang
     addons:
       apt:
         packages:
-          - *extra_packages
+          - *required_packages
           - *doc_packages
   # x86_64 cross-compiling 32-bits jobs
   - env: DEF_LIB="static" BUILD_32BIT=1
@@ -131,9 +101,6 @@ jobs:
   - env: DEF_LIB="static"
     arch: arm64
     compiler: gcc
-  - env: DEF_LIB="shared"
-    arch: arm64
-    compiler: gcc
   - env: DEF_LIB="shared" BUILD_DOCS=1
     arch: arm64
     compiler: gcc
@@ -142,13 +109,13 @@ jobs:
         packages:
           - *required_packages
           - *doc_packages
-  - env: DEF_LIB="shared" EXTRA_PACKAGES=1 ABI_CHECKS=1
+  - env: DEF_LIB="shared" ABI_CHECKS=1
     arch: arm64
     compiler: gcc
     addons:
       apt:
         packages:
-          - *extra_packages
+          - *required_packages
           - *libabigail_build_packages
   # aarch64 clang jobs
   - env: DEF_LIB="static"
-- 
2.23.0


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [dpdk-dev] [PATCH 4/4] ci: reorganise Travis jobs
  2020-02-19 19:41 ` [dpdk-dev] [PATCH 4/4] ci: reorganise " David Marchand
@ 2020-02-19 21:39   ` Aaron Conole
  2020-02-20 10:42     ` Thomas Monjalon
  2020-02-20  0:18   ` [dpdk-dev] " dwilder
  2020-02-20 11:07   ` Thomas Monjalon
  2 siblings, 1 reply; 24+ messages in thread
From: Aaron Conole @ 2020-02-19 21:39 UTC (permalink / raw)
  To: David Marchand; +Cc: thomas, dev, Michael Santana, Bruce Richardson

David Marchand <david.marchand@redhat.com> writes:

> Let's prune the jobs list to limit the amount of time spent by the robot
> in Travis.
>
> Since meson enables automatically the relevant components, there is not
> much gain in testing with extra_packages vs required_packages only.
>
> For a given arch/compiler/env combination, compilation is first tested
> in all jobs that run tests or build the docs or run the ABI checks.
> In the same context, for jobs that accumulates running tests, building
> the docs etc..., those steps are independent and can be split to save
> some cpu on Travis.
>
> With this, we go down from 21 to 15 jobs.
>
> Note: this patch requires a flush of the existing caches in Travis.
>
> Signed-off-by: David Marchand <david.marchand@redhat.com>
> ---

In general, I think the idea with required vs. extra was to have a build
that did the minimum required, and one that did all the packages (to
allow a minimum vs. full DPDK).

At least, that's from
http://mails.dpdk.org/archives/dev/2019-January/124007.html

Not sure if that's still something anyone cares about.

>  .travis.yml | 51 +++++++++------------------------------------------
>  1 file changed, 9 insertions(+), 42 deletions(-)
>
> diff --git a/.travis.yml b/.travis.yml
> index c174d5f30..b64a81bd0 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -15,15 +15,12 @@ addons:
>      update: true
>      packages: &required_packages
>        - [libnuma-dev, python3-setuptools, python3-wheel, python3-pip, ninja-build]
> +      - [libbsd-dev, libpcap-dev, libibverbs-dev, libcrypto++-dev, libfdt-dev, libjansson4]
>
>  _aarch64_packages: &aarch64_packages
>    - *required_packages
>    - [gcc-aarch64-linux-gnu, libc6-dev-arm64-cross, pkg-config-aarch64-linux-gnu]
>  
> -_extra_packages: &extra_packages
> -  - *required_packages
> -  - [libbsd-dev, libpcap-dev, libibverbs-dev, libcrypto++-dev, libfdt-dev, libjansson4]
> -
>  _libabigail_build_packages: &libabigail_build_packages
>    - [autoconf, automake, libtool, pkg-config, libxml2-dev, libdw-dev]
>  
> @@ -43,66 +40,39 @@ jobs:
>    - env: DEF_LIB="static"
>      arch: amd64
>      compiler: gcc
> -  - env: DEF_LIB="shared"
> -    arch: amd64
> -    compiler: gcc
>    - env: DEF_LIB="shared" RUN_TESTS=1
>      arch: amd64
>      compiler: gcc
> -  - env: DEF_LIB="static" EXTRA_PACKAGES=1
> -    arch: amd64
> -    compiler: gcc
> -    addons:
> -      apt:
> -        packages:
> -          - *extra_packages
> -  - env: DEF_LIB="shared" EXTRA_PACKAGES=1 RUN_TESTS=1 BUILD_DOCS=1
> +  - env: DEF_LIB="shared" BUILD_DOCS=1
>      arch: amd64
>      compiler: gcc
>      addons:
>        apt:
>          packages:
> -          - *extra_packages
> +          - *required_packages
>            - *doc_packages
> -  - env: DEF_LIB="shared" EXTRA_PACKAGES=1 ABI_CHECKS=1
> +  - env: DEF_LIB="shared" ABI_CHECKS=1
>      arch: amd64
>      compiler: gcc
>      addons:
>        apt:
>          packages:
> -          - *extra_packages
> +          - *required_packages
>            - *libabigail_build_packages
>    # x86_64 clang jobs
>    - env: DEF_LIB="static"
>      arch: amd64
>      compiler: clang
> -  - env: DEF_LIB="shared"
> -    arch: amd64
> -    compiler: clang
>    - env: DEF_LIB="shared" RUN_TESTS=1
>      arch: amd64
>      compiler: clang
> -  - env: DEF_LIB="static" EXTRA_PACKAGES=1
> -    arch: amd64
> -    compiler: clang
> -    addons:
> -      apt:
> -        packages:
> -          - *extra_packages
> -  - env: DEF_LIB="shared" EXTRA_PACKAGES=1
> -    arch: amd64
> -    compiler: clang
> -    addons:
> -      apt:
> -        packages:
> -          - *extra_packages
> -  - env: DEF_LIB="shared" EXTRA_PACKAGES=1 RUN_TESTS=1 BUILD_DOCS=1
> +  - env: DEF_LIB="shared" BUILD_DOCS=1
>      arch: amd64
>      compiler: clang
>      addons:
>        apt:
>          packages:
> -          - *extra_packages
> +          - *required_packages
>            - *doc_packages
>    # x86_64 cross-compiling 32-bits jobs
>    - env: DEF_LIB="static" BUILD_32BIT=1
> @@ -131,9 +101,6 @@ jobs:
>    - env: DEF_LIB="static"
>      arch: arm64
>      compiler: gcc
> -  - env: DEF_LIB="shared"
> -    arch: arm64
> -    compiler: gcc
>    - env: DEF_LIB="shared" BUILD_DOCS=1
>      arch: arm64
>      compiler: gcc
> @@ -142,13 +109,13 @@ jobs:
>          packages:
>            - *required_packages
>            - *doc_packages
> -  - env: DEF_LIB="shared" EXTRA_PACKAGES=1 ABI_CHECKS=1
> +  - env: DEF_LIB="shared" ABI_CHECKS=1
>      arch: arm64
>      compiler: gcc
>      addons:
>        apt:
>          packages:
> -          - *extra_packages
> +          - *required_packages
>            - *libabigail_build_packages

>    # aarch64 clang jobs
>    - env: DEF_LIB="static"


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [dpdk-dev] [PATCH 4/4] ci: reorganise Travis jobs
  2020-02-19 19:41 ` [dpdk-dev] [PATCH 4/4] ci: reorganise " David Marchand
  2020-02-19 21:39   ` Aaron Conole
@ 2020-02-20  0:18   ` dwilder
  2020-02-20 11:07   ` Thomas Monjalon
  2 siblings, 0 replies; 24+ messages in thread
From: dwilder @ 2020-02-20  0:18 UTC (permalink / raw)
  To: David Marchand; +Cc: thomas, aconole, dev, Michael Santana

On 2020-02-19 11:41, David Marchand wrote:
> Let's prune the jobs list to limit the amount of time spent by the 
> robot
> in Travis.
> 
> Since meson enables automatically the relevant components, there is not
> much gain in testing with extra_packages vs required_packages only.
> 
> For a given arch/compiler/env combination, compilation is first tested
> in all jobs that run tests or build the docs or run the ABI checks.
> In the same context, for jobs that accumulates running tests, building
> the docs etc..., those steps are independent and can be split to save
> some cpu on Travis.
> 
> With this, we go down from 21 to 15 jobs.
> 
> Note: this patch requires a flush of the existing caches in Travis.
> 
> Signed-off-by: David Marchand <david.marchand@redhat.com>
> ---
>  .travis.yml | 51 +++++++++------------------------------------------
>  1 file changed, 9 insertions(+), 42 deletions(-)
> 
> diff --git a/.travis.yml b/.travis.yml
> index c174d5f30..b64a81bd0 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -15,15 +15,12 @@ addons:
>      update: true
>      packages: &required_packages
>        - [libnuma-dev, python3-setuptools, python3-wheel, python3-pip,
> ninja-build]
> +      - [libbsd-dev, libpcap-dev, libibverbs-dev, libcrypto++-dev,
> libfdt-dev, libjansson4]
> 
>  _aarch64_packages: &aarch64_packages
>    - *required_packages
>    - [gcc-aarch64-linux-gnu, libc6-dev-arm64-cross,
> pkg-config-aarch64-linux-gnu]
> 
> -_extra_packages: &extra_packages
> -  - *required_packages
> -  - [libbsd-dev, libpcap-dev, libibverbs-dev, libcrypto++-dev,
> libfdt-dev, libjansson4]
> -
>  _libabigail_build_packages: &libabigail_build_packages
>    - [autoconf, automake, libtool, pkg-config, libxml2-dev, libdw-dev]
> 
> @@ -43,66 +40,39 @@ jobs:
>    - env: DEF_LIB="static"
>      arch: amd64
>      compiler: gcc
> -  - env: DEF_LIB="shared"
> -    arch: amd64
> -    compiler: gcc
>    - env: DEF_LIB="shared" RUN_TESTS=1
>      arch: amd64
>      compiler: gcc
> -  - env: DEF_LIB="static" EXTRA_PACKAGES=1
> -    arch: amd64
> -    compiler: gcc
> -    addons:
> -      apt:
> -        packages:
> -          - *extra_packages
> -  - env: DEF_LIB="shared" EXTRA_PACKAGES=1 RUN_TESTS=1 BUILD_DOCS=1
> +  - env: DEF_LIB="shared" BUILD_DOCS=1
>      arch: amd64
>      compiler: gcc
>      addons:
>        apt:
>          packages:
> -          - *extra_packages
> +          - *required_packages
>            - *doc_packages
> -  - env: DEF_LIB="shared" EXTRA_PACKAGES=1 ABI_CHECKS=1
> +  - env: DEF_LIB="shared" ABI_CHECKS=1
>      arch: amd64
>      compiler: gcc
>      addons:
>        apt:
>          packages:
> -          - *extra_packages
> +          - *required_packages
>            - *libabigail_build_packages
>    # x86_64 clang jobs
>    - env: DEF_LIB="static"
>      arch: amd64
>      compiler: clang
> -  - env: DEF_LIB="shared"
> -    arch: amd64
> -    compiler: clang
>    - env: DEF_LIB="shared" RUN_TESTS=1
>      arch: amd64
>      compiler: clang
> -  - env: DEF_LIB="static" EXTRA_PACKAGES=1
> -    arch: amd64
> -    compiler: clang
> -    addons:
> -      apt:
> -        packages:
> -          - *extra_packages
> -  - env: DEF_LIB="shared" EXTRA_PACKAGES=1
> -    arch: amd64
> -    compiler: clang
> -    addons:
> -      apt:
> -        packages:
> -          - *extra_packages
> -  - env: DEF_LIB="shared" EXTRA_PACKAGES=1 RUN_TESTS=1 BUILD_DOCS=1
> +  - env: DEF_LIB="shared" BUILD_DOCS=1
>      arch: amd64
>      compiler: clang
>      addons:
>        apt:
>          packages:
> -          - *extra_packages
> +          - *required_packages
>            - *doc_packages
>    # x86_64 cross-compiling 32-bits jobs
>    - env: DEF_LIB="static" BUILD_32BIT=1
> @@ -131,9 +101,6 @@ jobs:
>    - env: DEF_LIB="static"
>      arch: arm64
>      compiler: gcc
> -  - env: DEF_LIB="shared"
> -    arch: arm64
> -    compiler: gcc
>    - env: DEF_LIB="shared" BUILD_DOCS=1
>      arch: arm64
>      compiler: gcc
> @@ -142,13 +109,13 @@ jobs:
>          packages:
>            - *required_packages
>            - *doc_packages
> -  - env: DEF_LIB="shared" EXTRA_PACKAGES=1 ABI_CHECKS=1
> +  - env: DEF_LIB="shared" ABI_CHECKS=1
>      arch: arm64
>      compiler: gcc
>      addons:
>        apt:
>          packages:
> -          - *extra_packages
> +          - *required_packages
>            - *libabigail_build_packages
>    # aarch64 clang jobs
>    - env: DEF_LIB="static"

Much easier to follow,  I had no issues adding ppc64le jobs.
All my travis jobs ran fine.

LGTM  (the series)


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [dpdk-dev] [PATCH 4/4] ci: reorganise Travis jobs
  2020-02-19 21:39   ` Aaron Conole
@ 2020-02-20 10:42     ` Thomas Monjalon
  2020-02-20 12:22       ` David Marchand
  0 siblings, 1 reply; 24+ messages in thread
From: Thomas Monjalon @ 2020-02-20 10:42 UTC (permalink / raw)
  To: David Marchand, Aaron Conole; +Cc: dev, Michael Santana, Bruce Richardson

19/02/2020 22:39, Aaron Conole:
> David Marchand <david.marchand@redhat.com> writes:
> 
> > Let's prune the jobs list to limit the amount of time spent by the robot
> > in Travis.
> >
> > Since meson enables automatically the relevant components, there is not
> > much gain in testing with extra_packages vs required_packages only.
> >
> > For a given arch/compiler/env combination, compilation is first tested
> > in all jobs that run tests or build the docs or run the ABI checks.
> > In the same context, for jobs that accumulates running tests, building
> > the docs etc..., those steps are independent and can be split to save
> > some cpu on Travis.
> >
> > With this, we go down from 21 to 15 jobs.
> >
> > Note: this patch requires a flush of the existing caches in Travis.
> >
> > Signed-off-by: David Marchand <david.marchand@redhat.com>
> > ---
> 
> In general, I think the idea with required vs. extra was to have a build
> that did the minimum required, and one that did all the packages (to
> allow a minimum vs. full DPDK).
> 
> At least, that's from
> http://mails.dpdk.org/archives/dev/2019-January/124007.html

I think the benefit of a minimum build is to have a quick report,
and easy to setup.

> Not sure if that's still something anyone cares about.

Given that Travis knows how to satisfy the dependencies,
and that we must wait for all jobs to finish,
I don't see any benefit of a minimal setup.




^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [dpdk-dev] [PATCH 1/4] ci: remove unnecessary dependency on Linux headers
  2020-02-19 19:41 ` [dpdk-dev] [PATCH 1/4] ci: remove unnecessary dependency on Linux headers David Marchand
@ 2020-02-20 10:44   ` Thomas Monjalon
  2020-02-20 14:37   ` Aaron Conole
  1 sibling, 0 replies; 24+ messages in thread
From: Thomas Monjalon @ 2020-02-20 10:44 UTC (permalink / raw)
  To: David Marchand; +Cc: aconole, dev, Michael Santana

19/02/2020 20:41, David Marchand:
> Following removal of kmod compilation, we don't need to install
> linux-headers anymore.
> 
> Fixes: ea860973592b ("ci: remove redundant configs disabling kmods")
> 
> Signed-off-by: David Marchand <david.marchand@redhat.com>

Acked-by: Thomas Monjalon <thomas@monjalon.net>




^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [dpdk-dev] [PATCH 2/4] ci: fix Travis config warnings
  2020-02-19 19:41 ` [dpdk-dev] [PATCH 2/4] ci: fix Travis config warnings David Marchand
@ 2020-02-20 11:03   ` Thomas Monjalon
  2020-02-20 12:09     ` David Marchand
  2020-02-20 16:46     ` David Marchand
  2020-02-20 14:36   ` Aaron Conole
  1 sibling, 2 replies; 24+ messages in thread
From: Thomas Monjalon @ 2020-02-20 11:03 UTC (permalink / raw)
  To: David Marchand; +Cc: aconole, dev, Michael Santana

19/02/2020 20:41, David Marchand:
> Reading https://config.travis-ci.com/ and using
> https://config.travis-ci.com/explore to check changes, we can cleanup
> some warnings reported by the config validation options in Travis.

For documentation purpose, would be good to describe issues in the commit log.




^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [dpdk-dev] [PATCH 3/4] ci: use an explicit list of Travis jobs
  2020-02-19 19:41 ` [dpdk-dev] [PATCH 3/4] ci: use an explicit list of Travis jobs David Marchand
@ 2020-02-20 11:05   ` Thomas Monjalon
  2020-02-20 14:35   ` Aaron Conole
  1 sibling, 0 replies; 24+ messages in thread
From: Thomas Monjalon @ 2020-02-20 11:05 UTC (permalink / raw)
  To: David Marchand; +Cc: aconole, dev, Michael Santana

19/02/2020 20:41, David Marchand:
> Maintaining the .travis.yml requires some knowledge of how Travis
> computes the jobs list (combination of os: arch: compiler: etc...).
> Let's switch to an explicit list to find all jobs at a glance.
> 
> To enhance readability, jobs have been sorted per arch/compiler with
> comments to isolate blocks.
> 
> Setting required_packages for aarch64 native jobs is unnecessary,
> the global addons: values are the same.
> 
> This commit does not change the jobs list (21 jobs in total).
> 
> Signed-off-by: David Marchand <david.marchand@redhat.com>

Acked-by: Thomas Monjalon <thomas@monjalon.net>

Thank you for thinking to poor humans like me with no Travis-fu skill.



^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [dpdk-dev] [PATCH 4/4] ci: reorganise Travis jobs
  2020-02-19 19:41 ` [dpdk-dev] [PATCH 4/4] ci: reorganise " David Marchand
  2020-02-19 21:39   ` Aaron Conole
  2020-02-20  0:18   ` [dpdk-dev] " dwilder
@ 2020-02-20 11:07   ` Thomas Monjalon
  2 siblings, 0 replies; 24+ messages in thread
From: Thomas Monjalon @ 2020-02-20 11:07 UTC (permalink / raw)
  To: David Marchand; +Cc: aconole, dev, Michael Santana

19/02/2020 20:41, David Marchand:
> Let's prune the jobs list to limit the amount of time spent by the robot
> in Travis.
> 
> Since meson enables automatically the relevant components, there is not
> much gain in testing with extra_packages vs required_packages only.
> 
> For a given arch/compiler/env combination, compilation is first tested
> in all jobs that run tests or build the docs or run the ABI checks.
> In the same context, for jobs that accumulates running tests, building
> the docs etc..., those steps are independent and can be split to save
> some cpu on Travis.
> 
> With this, we go down from 21 to 15 jobs.
> 
> Note: this patch requires a flush of the existing caches in Travis.
> 
> Signed-off-by: David Marchand <david.marchand@redhat.com>

Acked-by: Thomas Monjalon <thomas@monjalon.net>

A small step for DPDK but a big save for the earth ;)




^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [dpdk-dev] [PATCH 2/4] ci: fix Travis config warnings
  2020-02-20 11:03   ` Thomas Monjalon
@ 2020-02-20 12:09     ` David Marchand
  2020-02-20 16:46     ` David Marchand
  1 sibling, 0 replies; 24+ messages in thread
From: David Marchand @ 2020-02-20 12:09 UTC (permalink / raw)
  To: Thomas Monjalon; +Cc: Aaron Conole, dev, Michael Santana

On Thu, Feb 20, 2020 at 12:03 PM Thomas Monjalon <thomas@monjalon.net> wrote:
>
> 19/02/2020 20:41, David Marchand:
> > Reading https://config.travis-ci.com/ and using
> > https://config.travis-ci.com/explore to check changes, we can cleanup
> > some warnings reported by the config validation options in Travis.
>
> For documentation purpose, would be good to describe issues in the commit log.

Ok, will do.


-- 
David Marchand


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [dpdk-dev] [PATCH 4/4] ci: reorganise Travis jobs
  2020-02-20 10:42     ` Thomas Monjalon
@ 2020-02-20 12:22       ` David Marchand
  2020-02-20 14:35         ` Aaron Conole
  0 siblings, 1 reply; 24+ messages in thread
From: David Marchand @ 2020-02-20 12:22 UTC (permalink / raw)
  To: Thomas Monjalon, Aaron Conole; +Cc: dev, Michael Santana, Bruce Richardson

On Thu, Feb 20, 2020 at 11:42 AM Thomas Monjalon <thomas@monjalon.net> wrote:
>
> 19/02/2020 22:39, Aaron Conole:
> > David Marchand <david.marchand@redhat.com> writes:
> >
> > > Let's prune the jobs list to limit the amount of time spent by the robot
> > > in Travis.
> > >
> > > Since meson enables automatically the relevant components, there is not
> > > much gain in testing with extra_packages vs required_packages only.
> > >
> > > For a given arch/compiler/env combination, compilation is first tested
> > > in all jobs that run tests or build the docs or run the ABI checks.
> > > In the same context, for jobs that accumulates running tests, building
> > > the docs etc..., those steps are independent and can be split to save
> > > some cpu on Travis.
> > >
> > > With this, we go down from 21 to 15 jobs.
> > >
> > > Note: this patch requires a flush of the existing caches in Travis.
> > >
> > > Signed-off-by: David Marchand <david.marchand@redhat.com>
> > > ---
> >
> > In general, I think the idea with required vs. extra was to have a build
> > that did the minimum required, and one that did all the packages (to
> > allow a minimum vs. full DPDK).
> >
> > At least, that's from
> > http://mails.dpdk.org/archives/dev/2019-January/124007.html
>
> I think the benefit of a minimum build is to have a quick report,
> and easy to setup.

Yes, Travis serves as a first gate when submitting patches.
But since Travis is best effort/free, we can't have a full coverage.


> > Not sure if that's still something anyone cares about.
>
> Given that Travis knows how to satisfy the dependencies,
> and that we must wait for all jobs to finish,
> I don't see any benefit of a minimal setup.

This minimal setup also tests that dpdk dependencies are correct.
If a change makes something rely on libX and libX is in the packages
always installed in Travis, the missing dependency would not get
caught.

But here, this adds too many jobs.

UNH, Intel and other CIs should step in and fill this kind of gap.


--
David Marchand


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [dpdk-dev] [PATCH 4/4] ci: reorganise Travis jobs
  2020-02-20 12:22       ` David Marchand
@ 2020-02-20 14:35         ` Aaron Conole
  2020-02-20 16:01           ` David Marchand
  0 siblings, 1 reply; 24+ messages in thread
From: Aaron Conole @ 2020-02-20 14:35 UTC (permalink / raw)
  To: David Marchand; +Cc: Thomas Monjalon, dev, Michael Santana, Bruce Richardson

David Marchand <david.marchand@redhat.com> writes:

> On Thu, Feb 20, 2020 at 11:42 AM Thomas Monjalon <thomas@monjalon.net> wrote:
>>
>> 19/02/2020 22:39, Aaron Conole:
>> > David Marchand <david.marchand@redhat.com> writes:
>> >
>> > > Let's prune the jobs list to limit the amount of time spent by the robot
>> > > in Travis.
>> > >
>> > > Since meson enables automatically the relevant components, there is not
>> > > much gain in testing with extra_packages vs required_packages only.
>> > >
>> > > For a given arch/compiler/env combination, compilation is first tested
>> > > in all jobs that run tests or build the docs or run the ABI checks.
>> > > In the same context, for jobs that accumulates running tests, building
>> > > the docs etc..., those steps are independent and can be split to save
>> > > some cpu on Travis.
>> > >
>> > > With this, we go down from 21 to 15 jobs.
>> > >
>> > > Note: this patch requires a flush of the existing caches in Travis.
>> > >
>> > > Signed-off-by: David Marchand <david.marchand@redhat.com>
>> > > ---
>> >
>> > In general, I think the idea with required vs. extra was to have a build
>> > that did the minimum required, and one that did all the packages (to
>> > allow a minimum vs. full DPDK).
>> >
>> > At least, that's from
>> > http://mails.dpdk.org/archives/dev/2019-January/124007.html
>>
>> I think the benefit of a minimum build is to have a quick report,
>> and easy to setup.
>
> Yes, Travis serves as a first gate when submitting patches.
> But since Travis is best effort/free, we can't have a full coverage.
>
>
>> > Not sure if that's still something anyone cares about.
>>
>> Given that Travis knows how to satisfy the dependencies,
>> and that we must wait for all jobs to finish,
>> I don't see any benefit of a minimal setup.
>
> This minimal setup also tests that dpdk dependencies are correct.
> If a change makes something rely on libX and libX is in the packages
> always installed in Travis, the missing dependency would not get
> caught.
>
> But here, this adds too many jobs.
>
> UNH, Intel and other CIs should step in and fill this kind of gap.

Okay, makes sense to me.  Are one of these CI providers offering to
cover this?

Also,

Acked-by: Aaron Conole <aconole@redhat.com>

>
> --
> David Marchand


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [dpdk-dev] [PATCH 3/4] ci: use an explicit list of Travis jobs
  2020-02-19 19:41 ` [dpdk-dev] [PATCH 3/4] ci: use an explicit list of Travis jobs David Marchand
  2020-02-20 11:05   ` Thomas Monjalon
@ 2020-02-20 14:35   ` Aaron Conole
  1 sibling, 0 replies; 24+ messages in thread
From: Aaron Conole @ 2020-02-20 14:35 UTC (permalink / raw)
  To: David Marchand; +Cc: thomas, dev, Michael Santana

David Marchand <david.marchand@redhat.com> writes:

> Maintaining the .travis.yml requires some knowledge of how Travis
> computes the jobs list (combination of os: arch: compiler: etc...).
> Let's switch to an explicit list to find all jobs at a glance.
>
> To enhance readability, jobs have been sorted per arch/compiler with
> comments to isolate blocks.
>
> Setting required_packages for aarch64 native jobs is unnecessary,
> the global addons: values are the same.
>
> This commit does not change the jobs list (21 jobs in total).
>
> Signed-off-by: David Marchand <david.marchand@redhat.com>
> ---

Acked-by: Aaron Conole <aconole@redhat.com>


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [dpdk-dev] [PATCH 2/4] ci: fix Travis config warnings
  2020-02-19 19:41 ` [dpdk-dev] [PATCH 2/4] ci: fix Travis config warnings David Marchand
  2020-02-20 11:03   ` Thomas Monjalon
@ 2020-02-20 14:36   ` Aaron Conole
  1 sibling, 0 replies; 24+ messages in thread
From: Aaron Conole @ 2020-02-20 14:36 UTC (permalink / raw)
  To: David Marchand; +Cc: thomas, dev, Michael Santana

David Marchand <david.marchand@redhat.com> writes:

> Reading https://config.travis-ci.com/ and using
> https://config.travis-ci.com/explore to check changes, we can cleanup
> some warnings reported by the config validation options in Travis.
>
> Signed-off-by: David Marchand <david.marchand@redhat.com>
> ---

Acked-by: Aaron Conole <aconole@redhat.com>

Echoing what Thomas wrote, please include the issues warnings.


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [dpdk-dev] [PATCH 1/4] ci: remove unnecessary dependency on Linux headers
  2020-02-19 19:41 ` [dpdk-dev] [PATCH 1/4] ci: remove unnecessary dependency on Linux headers David Marchand
  2020-02-20 10:44   ` Thomas Monjalon
@ 2020-02-20 14:37   ` Aaron Conole
  1 sibling, 0 replies; 24+ messages in thread
From: Aaron Conole @ 2020-02-20 14:37 UTC (permalink / raw)
  To: David Marchand; +Cc: thomas, dev, Michael Santana

David Marchand <david.marchand@redhat.com> writes:

> Following removal of kmod compilation, we don't need to install
> linux-headers anymore.
>
> Fixes: ea860973592b ("ci: remove redundant configs disabling kmods")
>
> Signed-off-by: David Marchand <david.marchand@redhat.com>
> ---

Acked-by: Aaron Conole <aconole@redhat.com>


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [dpdk-dev] [PATCH 4/4] ci: reorganise Travis jobs
  2020-02-20 14:35         ` Aaron Conole
@ 2020-02-20 16:01           ` David Marchand
  2020-02-20 19:38             ` [dpdk-dev] [dpdk-ci] " Jeremy Plsek
  0 siblings, 1 reply; 24+ messages in thread
From: David Marchand @ 2020-02-20 16:01 UTC (permalink / raw)
  To: ci; +Cc: Thomas Monjalon, dev, Michael Santana, Aaron Conole, Bruce Richardson

On Thu, Feb 20, 2020 at 3:35 PM Aaron Conole <aconole@redhat.com> wrote:
>
> David Marchand <david.marchand@redhat.com> writes:
>
> > On Thu, Feb 20, 2020 at 11:42 AM Thomas Monjalon <thomas@monjalon.net> wrote:
> >>
> >> 19/02/2020 22:39, Aaron Conole:
> >> > David Marchand <david.marchand@redhat.com> writes:
> >> >
> >> > > Let's prune the jobs list to limit the amount of time spent by the robot
> >> > > in Travis.
> >> > >
> >> > > Since meson enables automatically the relevant components, there is not
> >> > > much gain in testing with extra_packages vs required_packages only.
> >> > >
> >> > > For a given arch/compiler/env combination, compilation is first tested
> >> > > in all jobs that run tests or build the docs or run the ABI checks.
> >> > > In the same context, for jobs that accumulates running tests, building
> >> > > the docs etc..., those steps are independent and can be split to save
> >> > > some cpu on Travis.
> >> > >
> >> > > With this, we go down from 21 to 15 jobs.
> >> > >
> >> > > Note: this patch requires a flush of the existing caches in Travis.
> >> > >
> >> > > Signed-off-by: David Marchand <david.marchand@redhat.com>
> >> > > ---
> >> >
> >> > In general, I think the idea with required vs. extra was to have a build
> >> > that did the minimum required, and one that did all the packages (to
> >> > allow a minimum vs. full DPDK).
> >> >
> >> > At least, that's from
> >> > http://mails.dpdk.org/archives/dev/2019-January/124007.html
> >>
> >> I think the benefit of a minimum build is to have a quick report,
> >> and easy to setup.
> >
> > Yes, Travis serves as a first gate when submitting patches.
> > But since Travis is best effort/free, we can't have a full coverage.
> >
> >
> >> > Not sure if that's still something anyone cares about.
> >>
> >> Given that Travis knows how to satisfy the dependencies,
> >> and that we must wait for all jobs to finish,
> >> I don't see any benefit of a minimal setup.
> >
> > This minimal setup also tests that dpdk dependencies are correct.
> > If a change makes something rely on libX and libX is in the packages
> > always installed in Travis, the missing dependency would not get
> > caught.
> >
> > But here, this adds too many jobs.
> >
> > UNH, Intel and other CIs should step in and fill this kind of gap.
>
> Okay, makes sense to me.  Are one of these CI providers offering to
> cover this?

Maybe it is already covered, the best is to ask, so sending to ci@dpdk.org.

For the CI guys, which packages are installed on the systems/vms that
do compilation tests?
Is it possible to have a summary of the different setups?


Thanks.

--
David Marchand


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [dpdk-dev] [PATCH 2/4] ci: fix Travis config warnings
  2020-02-20 11:03   ` Thomas Monjalon
  2020-02-20 12:09     ` David Marchand
@ 2020-02-20 16:46     ` David Marchand
  2020-02-20 21:01       ` Aaron Conole
  1 sibling, 1 reply; 24+ messages in thread
From: David Marchand @ 2020-02-20 16:46 UTC (permalink / raw)
  To: Thomas Monjalon, Aaron Conole; +Cc: dev, Michael Santana

On Thu, Feb 20, 2020 at 12:03 PM Thomas Monjalon <thomas@monjalon.net> wrote:
>
> 19/02/2020 20:41, David Marchand:
> > Reading https://config.travis-ci.com/ and using
> > https://config.travis-ci.com/explore to check changes, we can cleanup
> > some warnings reported by the config validation options in Travis.
>
> For documentation purpose, would be good to describe issues in the commit log.

Since I only have this commitlog to update, I intend to push the
series with the following addition:

Example on a job in master:
https://travis-ci.com/DPDK/dpdk/builds/149537002/config

Build config validation:
root: deprecated key doc_packages (anchor on a non-private key)
root: deprecated key build_32b_packages (anchor on a non-private key)
root: deprecated key libabigail_build_packages (anchor on a non-private key)
root: deprecated key extra_packages (anchor on a non-private key)
root: deprecated key aarch64_packages (anchor on a non-private key)
root: key matrix is an alias for jobs, using jobs

For the "(anchor on a non-private key)" warnings, the Travis
documentation [1] recommends prefixing private keys with _ (Travis
schema is available at [2]).

The use of the "matrix" key is allowed, but it is just an alias to the
"jobs" key.

1: https://docs.travis-ci.com/user/build-config-yaml#private-keys-as-yaml-anchors-and-aliases-and-external-tooling
2: https://github.com/travis-ci/travis-yml/blob/730a77f4028b91293a55c8c1c52fc72e64379dde/schema.json


-- 
David Marchand


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [dpdk-dev] [dpdk-ci] [PATCH 4/4] ci: reorganise Travis jobs
  2020-02-20 16:01           ` David Marchand
@ 2020-02-20 19:38             ` Jeremy Plsek
  0 siblings, 0 replies; 24+ messages in thread
From: Jeremy Plsek @ 2020-02-20 19:38 UTC (permalink / raw)
  To: David Marchand
  Cc: ci, Thomas Monjalon, dev, Michael Santana, Aaron Conole,
	Bruce Richardson

[-- Attachment #1: Type: text/plain, Size: 3853 bytes --]

On Thu, Feb 20, 2020 at 11:02 AM David Marchand
<david.marchand@redhat.com> wrote:
>
> On Thu, Feb 20, 2020 at 3:35 PM Aaron Conole <aconole@redhat.com> wrote:
> >
> > David Marchand <david.marchand@redhat.com> writes:
> >
> > > On Thu, Feb 20, 2020 at 11:42 AM Thomas Monjalon <thomas@monjalon.net> wrote:
> > >>
> > >> 19/02/2020 22:39, Aaron Conole:
> > >> > David Marchand <david.marchand@redhat.com> writes:
> > >> >
> > >> > > Let's prune the jobs list to limit the amount of time spent by the robot
> > >> > > in Travis.
> > >> > >
> > >> > > Since meson enables automatically the relevant components, there is not
> > >> > > much gain in testing with extra_packages vs required_packages only.
> > >> > >
> > >> > > For a given arch/compiler/env combination, compilation is first tested
> > >> > > in all jobs that run tests or build the docs or run the ABI checks.
> > >> > > In the same context, for jobs that accumulates running tests, building
> > >> > > the docs etc..., those steps are independent and can be split to save
> > >> > > some cpu on Travis.
> > >> > >
> > >> > > With this, we go down from 21 to 15 jobs.
> > >> > >
> > >> > > Note: this patch requires a flush of the existing caches in Travis.
> > >> > >
> > >> > > Signed-off-by: David Marchand <david.marchand@redhat.com>
> > >> > > ---
> > >> >
> > >> > In general, I think the idea with required vs. extra was to have a build
> > >> > that did the minimum required, and one that did all the packages (to
> > >> > allow a minimum vs. full DPDK).
> > >> >
> > >> > At least, that's from
> > >> > http://mails.dpdk.org/archives/dev/2019-January/124007.html
> > >>
> > >> I think the benefit of a minimum build is to have a quick report,
> > >> and easy to setup.
> > >
> > > Yes, Travis serves as a first gate when submitting patches.
> > > But since Travis is best effort/free, we can't have a full coverage.
> > >
> > >
> > >> > Not sure if that's still something anyone cares about.
> > >>
> > >> Given that Travis knows how to satisfy the dependencies,
> > >> and that we must wait for all jobs to finish,
> > >> I don't see any benefit of a minimal setup.
> > >
> > > This minimal setup also tests that dpdk dependencies are correct.
> > > If a change makes something rely on libX and libX is in the packages
> > > always installed in Travis, the missing dependency would not get
> > > caught.
> > >
> > > But here, this adds too many jobs.
> > >
> > > UNH, Intel and other CIs should step in and fill this kind of gap.
> >
> > Okay, makes sense to me.  Are one of these CI providers offering to
> > cover this?
>
> Maybe it is already covered, the best is to ask, so sending to ci@dpdk.org.
>
> For the CI guys, which packages are installed on the systems/vms that
> do compilation tests?

For our compile tests, we compile with pretty much all dpdk
dependencies that can easily be installed from the distro's repository
(the dependencies we installed are based on the meson output). The
thought process was to test compiling as much of dpdk as possible
(with constraints mentioned before).

Here are the dependencies we install on some of the setups:

Ubuntu 18.04: make gcc ninja-build pkg-config libnuma-dev
librdmacm-dev librdmacm1 rdma-core libelf-dev libmnl-dev libpcap-dev
libcrypto++-dev libjansson-dev
(meson installed with pip)

CentOS 8 and Fedora 31: diffutils gcc make ninja-build meson
pkg-config numactl-devel librdmacm rdma-core-devel
elfutils-libelf-devel libmnl-devel libpcap-devel jansson-devel

openSuse Leap 15: gcc make ninja pkg-config libnuma-devel librdmacm1
rdma-core-devel libelf-devel libmnl-devel libpcap-devel
libjansson-devel
(meson installed with pip)

> Is it possible to have a summary of the different setups?
>
I've attached an overview/summary of what we do at the UNH-IOL DPDK
Community Lab.
>
> Thanks.

>
> --
> David Marchand
>

[-- Attachment #2: DPDK Lab Summary February 2020.pdf --]
[-- Type: application/pdf, Size: 846111 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [dpdk-dev] [PATCH 2/4] ci: fix Travis config warnings
  2020-02-20 16:46     ` David Marchand
@ 2020-02-20 21:01       ` Aaron Conole
  2020-02-21 10:17         ` Thomas Monjalon
  0 siblings, 1 reply; 24+ messages in thread
From: Aaron Conole @ 2020-02-20 21:01 UTC (permalink / raw)
  To: David Marchand; +Cc: Thomas Monjalon, dev, Michael Santana

David Marchand <david.marchand@redhat.com> writes:

> On Thu, Feb 20, 2020 at 12:03 PM Thomas Monjalon <thomas@monjalon.net> wrote:
>>
>> 19/02/2020 20:41, David Marchand:
>> > Reading https://config.travis-ci.com/ and using
>> > https://config.travis-ci.com/explore to check changes, we can cleanup
>> > some warnings reported by the config validation options in Travis.
>>
>> For documentation purpose, would be good to describe issues in the commit log.
>
> Since I only have this commitlog to update, I intend to push the
> series with the following addition:
>
> Example on a job in master:
> https://travis-ci.com/DPDK/dpdk/builds/149537002/config
>
> Build config validation:
> root: deprecated key doc_packages (anchor on a non-private key)
> root: deprecated key build_32b_packages (anchor on a non-private key)
> root: deprecated key libabigail_build_packages (anchor on a non-private key)
> root: deprecated key extra_packages (anchor on a non-private key)
> root: deprecated key aarch64_packages (anchor on a non-private key)
> root: key matrix is an alias for jobs, using jobs
>
> For the "(anchor on a non-private key)" warnings, the Travis
> documentation [1] recommends prefixing private keys with _ (Travis
> schema is available at [2]).
>
> The use of the "matrix" key is allowed, but it is just an alias to the
> "jobs" key.
>
> 1:
> https://docs.travis-ci.com/user/build-config-yaml#private-keys-as-yaml-anchors-and-aliases-and-external-tooling
> 2:
> https://github.com/travis-ci/travis-yml/blob/730a77f4028b91293a55c8c1c52fc72e64379dde/schema.json

LGTM


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [dpdk-dev] [PATCH 0/4] Reorganise Travis jobs
  2020-02-19 19:41 [dpdk-dev] [PATCH 0/4] Reorganise Travis jobs David Marchand
                   ` (3 preceding siblings ...)
  2020-02-19 19:41 ` [dpdk-dev] [PATCH 4/4] ci: reorganise " David Marchand
@ 2020-02-20 21:05 ` David Marchand
  4 siblings, 0 replies; 24+ messages in thread
From: David Marchand @ 2020-02-20 21:05 UTC (permalink / raw)
  To: Thomas Monjalon, Aaron Conole; +Cc: dev

On Wed, Feb 19, 2020 at 8:41 PM David Marchand
<david.marchand@redhat.com> wrote:
> This series tries to make the travis configuration a bit easier to read,
> then drops jobs that are not essential or functional duplicates in the
> hope we can give some oxygen to the robot who seems to lag quite a bit
> and can't catch up.

Travis cache for master flushed.
Series applied.


--
David Marchand


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [dpdk-dev] [PATCH 2/4] ci: fix Travis config warnings
  2020-02-20 21:01       ` Aaron Conole
@ 2020-02-21 10:17         ` Thomas Monjalon
  0 siblings, 0 replies; 24+ messages in thread
From: Thomas Monjalon @ 2020-02-21 10:17 UTC (permalink / raw)
  To: David Marchand, Aaron Conole; +Cc: dev, Michael Santana

20/02/2020 22:01, Aaron Conole:
> David Marchand <david.marchand@redhat.com> writes:
> 
> > On Thu, Feb 20, 2020 at 12:03 PM Thomas Monjalon <thomas@monjalon.net> wrote:
> >>
> >> 19/02/2020 20:41, David Marchand:
> >> > Reading https://config.travis-ci.com/ and using
> >> > https://config.travis-ci.com/explore to check changes, we can cleanup
> >> > some warnings reported by the config validation options in Travis.
> >>
> >> For documentation purpose, would be good to describe issues in the commit log.
> >
> > Since I only have this commitlog to update, I intend to push the
> > series with the following addition:
> >
> > Example on a job in master:
> > https://travis-ci.com/DPDK/dpdk/builds/149537002/config
> >
> > Build config validation:
> > root: deprecated key doc_packages (anchor on a non-private key)
> > root: deprecated key build_32b_packages (anchor on a non-private key)
> > root: deprecated key libabigail_build_packages (anchor on a non-private key)
> > root: deprecated key extra_packages (anchor on a non-private key)
> > root: deprecated key aarch64_packages (anchor on a non-private key)
> > root: key matrix is an alias for jobs, using jobs
> >
> > For the "(anchor on a non-private key)" warnings, the Travis
> > documentation [1] recommends prefixing private keys with _ (Travis
> > schema is available at [2]).
> >
> > The use of the "matrix" key is allowed, but it is just an alias to the
> > "jobs" key.
> >
> > 1:
> > https://docs.travis-ci.com/user/build-config-yaml#private-keys-as-yaml-anchors-and-aliases-and-external-tooling
> > 2:
> > https://github.com/travis-ci/travis-yml/blob/730a77f4028b91293a55c8c1c52fc72e64379dde/schema.json
> 
> LGTM

Perfect, thanks




^ permalink raw reply	[flat|nested] 24+ messages in thread

end of thread, other threads:[~2020-02-21 16:49 UTC | newest]

Thread overview: 24+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-02-19 19:41 [dpdk-dev] [PATCH 0/4] Reorganise Travis jobs David Marchand
2020-02-19 19:41 ` [dpdk-dev] [PATCH 1/4] ci: remove unnecessary dependency on Linux headers David Marchand
2020-02-20 10:44   ` Thomas Monjalon
2020-02-20 14:37   ` Aaron Conole
2020-02-19 19:41 ` [dpdk-dev] [PATCH 2/4] ci: fix Travis config warnings David Marchand
2020-02-20 11:03   ` Thomas Monjalon
2020-02-20 12:09     ` David Marchand
2020-02-20 16:46     ` David Marchand
2020-02-20 21:01       ` Aaron Conole
2020-02-21 10:17         ` Thomas Monjalon
2020-02-20 14:36   ` Aaron Conole
2020-02-19 19:41 ` [dpdk-dev] [PATCH 3/4] ci: use an explicit list of Travis jobs David Marchand
2020-02-20 11:05   ` Thomas Monjalon
2020-02-20 14:35   ` Aaron Conole
2020-02-19 19:41 ` [dpdk-dev] [PATCH 4/4] ci: reorganise " David Marchand
2020-02-19 21:39   ` Aaron Conole
2020-02-20 10:42     ` Thomas Monjalon
2020-02-20 12:22       ` David Marchand
2020-02-20 14:35         ` Aaron Conole
2020-02-20 16:01           ` David Marchand
2020-02-20 19:38             ` [dpdk-dev] [dpdk-ci] " Jeremy Plsek
2020-02-20  0:18   ` [dpdk-dev] " dwilder
2020-02-20 11:07   ` Thomas Monjalon
2020-02-20 21:05 ` [dpdk-dev] [PATCH 0/4] Reorganise " David Marchand

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).