[dpdk-dev] [PATCH] ci: hook to Github Actions

[dpdk-dev] [PATCH] ci: hook to Github Actions

[David Marchand]

With the recent changes in terms of free access to the Travis CI, let's
offer an alternative with Github Actions.
Running jobs on ARM is not supported unless using external runners, so
this commit only adds builds for x86_64 and cross compiling for i386 and
aarch64.

Differences with the Travis CI integration:
- All jobs generate documentation.
  This is not that heavy and the default timeout on actions is never
  reached so no reason splitting this into multiple jobs.
- Error logs are not dumped to the console when something goes wrong.
  Instead, they are gathered in a "catch-all" step and attached as
  artifacts.
- A cache entry is stored once and for all, but if no cache is found you
  can inherit from the default branch cache. The cache is 5GB large, for
  the whole git repository.
- The maximum retention of logs and artifacts is 3 months.
- /home/runner is world writable, so a workaround has been added for
  starting dpdk processes.

Signed-off-by: David Marchand <david.marchand@redhat.com>
---
 .ci/linux-build.sh          |  4 +-
 .github/workflows/build.yml | 98 +++++++++++++++++++++++++++++++++++++
 MAINTAINERS                 |  1 +
 3 files changed, 102 insertions(+), 1 deletion(-)
 create mode 100644 .github/workflows/build.yml

diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh
index d079801d78..a2a0e5bf42 100755
--- a/.ci/linux-build.sh
+++ b/.ci/linux-build.sh
@@ -12,7 +12,9 @@ on_error() {
         fi
     done
 }
-trap on_error EXIT
+# We capture the error logs as artifacts in Github Actions, no need to dump
+# them via a EXIT handler.
+[ -n "$GITHUB_WORKFLOW" ] || trap on_error EXIT
 
 install_libabigail() {
     version=$1
diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml
new file mode 100644
index 0000000000..e0a8f1ed52
--- /dev/null
+++ b/.github/workflows/build.yml
@@ -0,0 +1,98 @@
+name: build
+
+on: push
+
+defaults:
+  run:
+    shell: bash --noprofile --norc -exo pipefail {0}
+
+jobs:
+  build:
+    name: ${{ join(matrix.config.*, '-') }}
+    runs-on: ${{ matrix.config.os }}
+    env:
+      PKGS: |
+        ccache libnuma-dev python3-setuptools python3-wheel python3-pip \
+        ninja-build libbsd-dev libpcap-dev libibverbs-dev libcrypto++-dev \
+        libfdt-dev libjansson-dev doxygen graphviz python3-sphinx \
+        python3-sphinx-rtd-theme
+      CC: ccache ${{ matrix.config.compiler }}
+      JOBNAME: ${{ join(matrix.config.*, '-') }}
+
+    strategy:
+      fail-fast: false
+      matrix:
+        config:
+          - os: ubuntu-18.04
+            compiler: gcc
+            library: static
+          - os: ubuntu-18.04
+            compiler: gcc
+            library: shared
+          - os: ubuntu-18.04
+            compiler: clang
+            library: static
+          - os: ubuntu-18.04
+            compiler: clang
+            library: shared
+          - os: ubuntu-18.04
+            compiler: gcc
+            library: static
+            cross: i386
+          - os: ubuntu-18.04
+            compiler: gcc
+            library: static
+            cross: aarch64
+          - os: ubuntu-18.04
+            compiler: gcc
+            library: shared
+            cross: aarch64
+
+    steps:
+    - uses: actions/checkout@v2
+    - uses: actions/cache@v2
+      with:
+        path: ~/.ccache
+        key: ${{ env.JOBNAME }}-${{ github.ref }}
+        restore-keys: |
+          ${{ env.JOBNAME }}-refs/heads/main
+    - name: Install packages
+      run: sudo apt install -y ${{ env.PKGS }}
+    - name: Install i386 cross compiling packages
+      if: matrix.config.cross == 'i386'
+      run: sudo apt install -y gcc-multilib
+    - name: Install aarch64 cross compiling packages
+      if: matrix.config.cross == 'aarch64'
+      run: |
+        sudo apt install -y gcc-aarch64-linux-gnu libc6-dev-arm64-cross \
+          pkg-config-aarch64-linux-gnu
+    - name: Prepare environment
+      run: |
+         .ci/linux-setup.sh
+         # Workaround on $HOME permissions as EAL checks them for plugin loading
+         chmod o-w $HOME
+    - name: Build and test
+      run: |
+        export DEF_LIB=${{ matrix.config.library }}
+        export BUILD_DOCS=1
+        case '${{ matrix.config.cross }}' in
+        'i386')
+            export BUILD_32BIT=1
+        ;;
+        'aarch64')
+            export AARCH64=1
+        ;;
+        '')
+            export RUN_TESTS=1
+        ;;
+        esac
+        .ci/linux-build.sh
+    - name: Upload logs on failure
+      if: failure()
+      uses: actions/upload-artifact@v2
+      with:
+        name: meson-logs-${{ env.JOBNAME }}
+        path: |
+          build/meson-logs/testlog.txt
+          build/.ninja_log
+          build/meson-logs/meson-log.txt
diff --git a/MAINTAINERS b/MAINTAINERS
index 214515060a..95b61085b7 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -109,6 +109,7 @@ Public CI
 M: Aaron Conole <aconole@redhat.com>
 M: Michael Santana <maicolgabriel@hotmail.com>
 F: .travis.yml
+F: .github/workflows/build.yml
 F: .ci/
 
 ABI Policy & Versioning
-- 
2.23.0

Re: [dpdk-dev] [PATCH] ci: hook to Github Actions

[Aaron Conole]

David Marchand <david.marchand@redhat.com> writes:

> With the recent changes in terms of free access to the Travis CI, let's
> offer an alternative with Github Actions.
> Running jobs on ARM is not supported unless using external runners, so
> this commit only adds builds for x86_64 and cross compiling for i386 and
> aarch64.
>
> Differences with the Travis CI integration:
> - All jobs generate documentation.
>   This is not that heavy and the default timeout on actions is never
>   reached so no reason splitting this into multiple jobs.
> - Error logs are not dumped to the console when something goes wrong.
>   Instead, they are gathered in a "catch-all" step and attached as
>   artifacts.
> - A cache entry is stored once and for all, but if no cache is found you
>   can inherit from the default branch cache. The cache is 5GB large, for
>   the whole git repository.
> - The maximum retention of logs and artifacts is 3 months.
> - /home/runner is world writable, so a workaround has been added for
>   starting dpdk processes.
>
> Signed-off-by: David Marchand <david.marchand@redhat.com>
> ---

Thanks for working on this.  Sadly, I think we will have to abandon
Travis soon - given the new changes it is looking very awful.  Robot
already is starved for job time.

Since we don't have ARM test runs, I guess we will have to rely on
something else for that coverage now, but I like that there is coverage
included at least to compile.

I will need to update the robot to pull information from github actions,
so for now it will need to be manually checked (but here's an example of
a run: https://github.com/ovsrobot/dpdk/actions/runs/382073265).  What's
nice is the robot is already primed to run the jobs, so that's good.

Acked-by: Aaron Conole <aconole@redhat.com>

>  .ci/linux-build.sh          |  4 +-
>  .github/workflows/build.yml | 98 +++++++++++++++++++++++++++++++++++++
>  MAINTAINERS                 |  1 +
>  3 files changed, 102 insertions(+), 1 deletion(-)
>  create mode 100644 .github/workflows/build.yml
>
> diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh
> index d079801d78..a2a0e5bf42 100755
> --- a/.ci/linux-build.sh
> +++ b/.ci/linux-build.sh
> @@ -12,7 +12,9 @@ on_error() {
>          fi
>      done
>  }
> -trap on_error EXIT
> +# We capture the error logs as artifacts in Github Actions, no need to dump
> +# them via a EXIT handler.
> +[ -n "$GITHUB_WORKFLOW" ] || trap on_error EXIT
>  
>  install_libabigail() {
>      version=$1
> diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml
> new file mode 100644
> index 0000000000..e0a8f1ed52
> --- /dev/null
> +++ b/.github/workflows/build.yml
> @@ -0,0 +1,98 @@
> +name: build
> +
> +on: push
> +
> +defaults:
> +  run:
> +    shell: bash --noprofile --norc -exo pipefail {0}
> +
> +jobs:
> +  build:
> +    name: ${{ join(matrix.config.*, '-') }}
> +    runs-on: ${{ matrix.config.os }}
> +    env:
> +      PKGS: |
> +        ccache libnuma-dev python3-setuptools python3-wheel python3-pip \
> +        ninja-build libbsd-dev libpcap-dev libibverbs-dev libcrypto++-dev \
> +        libfdt-dev libjansson-dev doxygen graphviz python3-sphinx \
> +        python3-sphinx-rtd-theme
> +      CC: ccache ${{ matrix.config.compiler }}
> +      JOBNAME: ${{ join(matrix.config.*, '-') }}
> +
> +    strategy:
> +      fail-fast: false
> +      matrix:
> +        config:
> +          - os: ubuntu-18.04
> +            compiler: gcc
> +            library: static
> +          - os: ubuntu-18.04
> +            compiler: gcc
> +            library: shared
> +          - os: ubuntu-18.04
> +            compiler: clang
> +            library: static
> +          - os: ubuntu-18.04
> +            compiler: clang
> +            library: shared
> +          - os: ubuntu-18.04
> +            compiler: gcc
> +            library: static
> +            cross: i386
> +          - os: ubuntu-18.04
> +            compiler: gcc
> +            library: static
> +            cross: aarch64
> +          - os: ubuntu-18.04
> +            compiler: gcc
> +            library: shared
> +            cross: aarch64
> +
> +    steps:
> +    - uses: actions/checkout@v2
> +    - uses: actions/cache@v2
> +      with:
> +        path: ~/.ccache
> +        key: ${{ env.JOBNAME }}-${{ github.ref }}
> +        restore-keys: |
> +          ${{ env.JOBNAME }}-refs/heads/main
> +    - name: Install packages
> +      run: sudo apt install -y ${{ env.PKGS }}
> +    - name: Install i386 cross compiling packages
> +      if: matrix.config.cross == 'i386'
> +      run: sudo apt install -y gcc-multilib
> +    - name: Install aarch64 cross compiling packages
> +      if: matrix.config.cross == 'aarch64'
> +      run: |
> +        sudo apt install -y gcc-aarch64-linux-gnu libc6-dev-arm64-cross \
> +          pkg-config-aarch64-linux-gnu
> +    - name: Prepare environment
> +      run: |
> +         .ci/linux-setup.sh
> +         # Workaround on $HOME permissions as EAL checks them for plugin loading
> +         chmod o-w $HOME
> +    - name: Build and test
> +      run: |
> +        export DEF_LIB=${{ matrix.config.library }}
> +        export BUILD_DOCS=1
> +        case '${{ matrix.config.cross }}' in
> +        'i386')
> +            export BUILD_32BIT=1
> +        ;;
> +        'aarch64')
> +            export AARCH64=1
> +        ;;
> +        '')
> +            export RUN_TESTS=1
> +        ;;
> +        esac
> +        .ci/linux-build.sh
> +    - name: Upload logs on failure
> +      if: failure()
> +      uses: actions/upload-artifact@v2
> +      with:
> +        name: meson-logs-${{ env.JOBNAME }}
> +        path: |
> +          build/meson-logs/testlog.txt
> +          build/.ninja_log
> +          build/meson-logs/meson-log.txt
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 214515060a..95b61085b7 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -109,6 +109,7 @@ Public CI
>  M: Aaron Conole <aconole@redhat.com>
>  M: Michael Santana <maicolgabriel@hotmail.com>
>  F: .travis.yml
> +F: .github/workflows/build.yml
>  F: .ci/
>  
>  ABI Policy & Versioning

Re: [dpdk-dev] [PATCH] ci: hook to Github Actions

[David Marchand]

On Wed, Nov 25, 2020 at 2:45 PM Aaron Conole <aconole@redhat.com> wrote:
> Thanks for working on this.  Sadly, I think we will have to abandon
> Travis soon - given the new changes it is looking very awful.  Robot
> already is starved for job time.
>
> Since we don't have ARM test runs, I guess we will have to rely on
> something else for that coverage now, but I like that there is coverage
> included at least to compile.

For ARM test runs, UNH is a good candidate but nothing prevents other
ARM based CI from being added.


> I will need to update the robot to pull information from github actions,
> so for now it will need to be manually checked (but here's an example of
> a run: https://github.com/ovsrobot/dpdk/actions/runs/382073265).  What's
> nice is the robot is already primed to run the jobs, so that's good.

Thanks.
I added a bookmark to https://github.com/ovsrobot/dpdk/actions for now.


-- 
David Marchand

Re: [dpdk-dev] [PATCH] ci: hook to Github Actions

[Honnappa Nagarahalli]

<snip>

> 
> On Wed, Nov 25, 2020 at 2:45 PM Aaron Conole <aconole@redhat.com> wrote:
> > Thanks for working on this.  Sadly, I think we will have to abandon
> > Travis soon - given the new changes it is looking very awful.  Robot
> > already is starved for job time.
I am looking at [1], is DPDK not considered as open source project?

[1] https://blog.travis-ci.com/oss-announcement

> >
> > Since we don't have ARM test runs, I guess we will have to rely on
> > something else for that coverage now, but I like that there is
> > coverage included at least to compile.
> 
> For ARM test runs, UNH is a good candidate but nothing prevents other ARM
> based CI from being added.
Is it possible to keep Travis CI for Arm?

> 
> 
> > I will need to update the robot to pull information from github
> > actions, so for now it will need to be manually checked (but here's an
> > example of a run:
> > https://github.com/ovsrobot/dpdk/actions/runs/382073265).  What's nice is
> the robot is already primed to run the jobs, so that's good.
Is there any guarantee that GitHub actions will be free forever?

> 
> Thanks.
> I added a bookmark to https://github.com/ovsrobot/dpdk/actions for now.
> 
> 
> --
> David Marchand

Re: [dpdk-dev] [PATCH] ci: hook to Github Actions

[David Marchand]

On Thu, Nov 26, 2020 at 5:47 AM Honnappa Nagarahalli
<Honnappa.Nagarahalli@arm.com> wrote:
> > On Wed, Nov 25, 2020 at 2:45 PM Aaron Conole <aconole@redhat.com> wrote:
> > > Thanks for working on this.  Sadly, I think we will have to abandon
> > > Travis soon - given the new changes it is looking very awful.  Robot
> > > already is starved for job time.
> I am looking at [1], is DPDK not considered as open source project?
>
> [1] https://blog.travis-ci.com/oss-announcement

Ilya (@OVS) contacted the Travis support.
The reply is that a project that has sponsored contributors can not
ask for free tokens on Travis CI.

I personally did not try to contact their support given this response.


> > > Since we don't have ARM test runs, I guess we will have to rely on
> > > something else for that coverage now, but I like that there is
> > > coverage included at least to compile.
> >
> > For ARM test runs, UNH is a good candidate but nothing prevents other ARM
> > based CI from being added.
> Is it possible to keep Travis CI for Arm?

For individuals, the 10k credits with the current DPDK jobs get burned
in something like 4 runs (read: 4 runs a month).
Even if we narrow the configuration to only ARM, this will at best
give us x3, so let's say 12 runs a month.

Now consider the ovsrobot and the number of series that hit the list
on a worst^Wbest day like a week before rc1.


> > > I will need to update the robot to pull information from github
> > > actions, so for now it will need to be manually checked (but here's an
> > > example of a run:
> > > https://github.com/ovsrobot/dpdk/actions/runs/382073265).  What's nice is
> > the robot is already primed to run the jobs, so that's good.
> Is there any guarantee that GitHub actions will be free forever?

There is no "forever".
At least, UNH lab which the project sponsors seems viable on the mid/long term.


-- 
David Marchand

Re: [dpdk-dev] [PATCH] ci: hook to Github Actions

[Honnappa Nagarahalli]

<snip>

> 
> On Thu, Nov 26, 2020 at 5:47 AM Honnappa Nagarahalli
> <Honnappa.Nagarahalli@arm.com> wrote:
> > > On Wed, Nov 25, 2020 at 2:45 PM Aaron Conole <aconole@redhat.com>
> wrote:
> > > > Thanks for working on this.  Sadly, I think we will have to
> > > > abandon Travis soon - given the new changes it is looking very
> > > > awful.  Robot already is starved for job time.
> > I am looking at [1], is DPDK not considered as open source project?
> >
> > [1] https://blog.travis-ci.com/oss-announcement
> 
> Ilya (@OVS) contacted the Travis support.
> The reply is that a project that has sponsored contributors can not ask for free
> tokens on Travis CI.
> 
> I personally did not try to contact their support given this response.
> 
> 
> > > > Since we don't have ARM test runs, I guess we will have to rely on
> > > > something else for that coverage now, but I like that there is
> > > > coverage included at least to compile.
> > >
> > > For ARM test runs, UNH is a good candidate but nothing prevents
> > > other ARM based CI from being added.
> > Is it possible to keep Travis CI for Arm?
> 
> For individuals, the 10k credits with the current DPDK jobs get burned in
> something like 4 runs (read: 4 runs a month).
> Even if we narrow the configuration to only ARM, this will at best give us x3, so
> let's say 12 runs a month.
> 
> Now consider the ovsrobot and the number of series that hit the list on a
> worst^Wbest day like a week before rc1.
> 
> 
> > > > I will need to update the robot to pull information from github
> > > > actions, so for now it will need to be manually checked (but
> > > > here's an example of a run:
> > > > https://github.com/ovsrobot/dpdk/actions/runs/382073265).  What's
> > > > nice is
> > > the robot is already primed to run the jobs, so that's good.
> > Is there any guarantee that GitHub actions will be free forever?
> 
> There is no "forever".
I think we are spending our efforts on things that will not work for the community in the long run (unless the project spends money to buy credits)

> At least, UNH lab which the project sponsors seems viable on the mid/long term.
Yes, agree, all the more reason to continue to maintain the lab.

> 
> 
> --
> David Marchand

[dpdk-dev] [PATCH v2 1/2] ci: hook to GitHub Actions

[David Marchand]

With the recent changes in terms of free access to the Travis CI, let's
offer an alternative with GitHub Actions.
Running jobs on ARM is not supported unless using external runners, so
this commit only adds builds for x86_64 and cross compiling for i386 and
aarch64.

Differences with the Travis CI integration:
- Error logs are not dumped to the console when something goes wrong.
  Instead, they are gathered in a "catch-all" step and attached as
  artifacts.
- A cache entry is stored once and for all, but if no cache is found you
  can inherit from the default branch cache. The cache is 5GB large, for
  the whole git repository.
- The maximum retention of logs and artifacts is 3 months.
- /home/runner is world writable, so a workaround has been added for
  starting dpdk processes.
- Ilya, working on OVS GHA support, noticed that jobs can run with
  processors that don't have the same capabilities. For DPDK, this
  impacts the ccache content since everything was built with
  -march=native so far, and we will end up with binaries that can't run
  in a later build. The problem has not been seen in Travis CI (?) but
  it is safer to use a fixed "-Dmachine=default" in any case.
- Scheduling jobs is part of the configuration and takes the form of a
  crontab. A build is scheduled every Monday at 0:00 (UTC) to provide a
  default ccache for the week (useful for the ovsrobot).

Signed-off-by: David Marchand <david.marchand@redhat.com>
---
Changelog since v1:
- changed shell variables value in CI scripts and Travis configuration
  (s/=[^\$]*1/=\1true), this makes it easier for GHA,
- forced compilation as 'default' to avoid random unit tests issues in
  GHA,
- scheduled a run per week on Monday at 0:00 UTC,
- updated the ccache key:
  - no need to depend on the default-library parameter since this
    parameter only impacts the linking of dpdk binaries,
  - the week when the cache is generated is added so that jobs in
    other branches can benefit from a recent cache (mimicking what we had
    for the robot in Travis),
- realigned documentation generation with what is done in Travis:
  generating the doc in all jobs was a waste of resources,

---
 .ci/linux-build.sh          |  17 +++---
 .github/workflows/build.yml | 100 ++++++++++++++++++++++++++++++++++++
 .travis.yml                 |  24 ++++-----
 MAINTAINERS                 |   1 +
 4 files changed, 123 insertions(+), 19 deletions(-)
 create mode 100644 .github/workflows/build.yml

diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh
index d079801d78..ee8d07f865 100755
--- a/.ci/linux-build.sh
+++ b/.ci/linux-build.sh
@@ -12,7 +12,9 @@ on_error() {
         fi
     done
 }
-trap on_error EXIT
+# We capture the error logs as artifacts in Github Actions, no need to dump
+# them via a EXIT handler.
+[ -n "$GITHUB_WORKFLOW" ] || trap on_error EXIT
 
 install_libabigail() {
     version=$1
@@ -28,16 +30,16 @@ install_libabigail() {
     rm ${version}.tar.gz
 }
 
-if [ "$AARCH64" = "1" ]; then
+if [ "$AARCH64" = "true" ]; then
     # convert the arch specifier
     OPTS="$OPTS --cross-file config/arm/arm64_armv8_linux_gcc"
 fi
 
-if [ "$BUILD_DOCS" = "1" ]; then
+if [ "$BUILD_DOCS" = "true" ]; then
     OPTS="$OPTS -Denable_docs=true"
 fi
 
-if [ "$BUILD_32BIT" = "1" ]; then
+if [ "$BUILD_32BIT" = "true" ]; then
     OPTS="$OPTS -Dc_args=-m32 -Dc_link_args=-m32"
     export PKG_CONFIG_LIBDIR="/usr/lib32/pkgconfig"
 fi
@@ -48,16 +50,17 @@ else
     OPTS="$OPTS -Dexamples=all"
 fi
 
+OPTS="$OPTS -Dmachine=default"
 OPTS="$OPTS --default-library=$DEF_LIB"
 OPTS="$OPTS --buildtype=debugoptimized"
 meson build --werror $OPTS
 ninja -C build
 
-if [ "$AARCH64" != "1" ]; then
+if [ "$AARCH64" != "true" ]; then
     devtools/test-null.sh
 fi
 
-if [ "$ABI_CHECKS" = "1" ]; then
+if [ "$ABI_CHECKS" = "true" ]; then
     LIBABIGAIL_VERSION=${LIBABIGAIL_VERSION:-libabigail-1.6}
 
     if [ "$(cat libabigail/VERSION 2>/dev/null)" != "$LIBABIGAIL_VERSION" ]; then
@@ -95,6 +98,6 @@ if [ "$ABI_CHECKS" = "1" ]; then
     devtools/check-abi.sh reference install ${ABI_CHECKS_WARN_ONLY:-}
 fi
 
-if [ "$RUN_TESTS" = "1" ]; then
+if [ "$RUN_TESTS" = "true" ]; then
     sudo meson test -C build --suite fast-tests -t 3
 fi
diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml
new file mode 100644
index 0000000000..bef6e52372
--- /dev/null
+++ b/.github/workflows/build.yml
@@ -0,0 +1,100 @@
+name: build
+
+on:
+  push:
+  schedule:
+    - cron: '0 0 * * 1'
+
+defaults:
+  run:
+    shell: bash --noprofile --norc -exo pipefail {0}
+
+jobs:
+  build:
+    name: ${{ join(matrix.config.*, '-') }}
+    runs-on: ${{ matrix.config.os }}
+    env:
+      AARCH64: ${{ matrix.config.cross == 'aarch64' }}
+      BUILD_32BIT: ${{ matrix.config.cross == 'i386' }}
+      BUILD_DOCS: ${{ contains(matrix.config.checks, 'doc') }}
+      CC: ccache ${{ matrix.config.compiler }}
+      DEF_LIB: ${{ matrix.config.library }}
+      RUN_TESTS: ${{ contains(matrix.config.checks, 'tests') }}
+
+    strategy:
+      fail-fast: false
+      matrix:
+        config:
+          - os: ubuntu-18.04
+            compiler: gcc
+            library: static
+          - os: ubuntu-18.04
+            compiler: gcc
+            library: shared
+            checks: doc+tests
+          - os: ubuntu-18.04
+            compiler: clang
+            library: static
+          - os: ubuntu-18.04
+            compiler: clang
+            library: shared
+            checks: doc+tests
+          - os: ubuntu-18.04
+            compiler: gcc
+            library: static
+            cross: i386
+          - os: ubuntu-18.04
+            compiler: gcc
+            library: static
+            cross: aarch64
+          - os: ubuntu-18.04
+            compiler: gcc
+            library: shared
+            cross: aarch64
+
+    steps:
+    - name: Checkout sources
+      uses: actions/checkout@v2
+    - name: Generate cache keys
+      id: get_ref_keys
+      run: |
+        echo -n '::set-output name=ccache::'
+        echo 'ccache-${{ matrix.config.os }}-${{ matrix.config.compiler }}-${{ matrix.config.cross }}-'$(date -u +%Y-w%W)
+    - name: Retrieve ccache cache
+      uses: actions/cache@v2
+      with:
+        path: ~/.ccache
+        key: ${{ steps.get_ref_keys.outputs.ccache }}-${{ github.ref }}
+        restore-keys: |
+          ${{ steps.get_ref_keys.outputs.ccache }}-refs/heads/main
+    - name: Install packages
+      run: sudo apt install -y ccache libnuma-dev python3-setuptools
+        python3-wheel python3-pip ninja-build libbsd-dev libpcap-dev
+        libibverbs-dev libcrypto++-dev libfdt-dev libjansson-dev
+    - name: Install i386 cross compiling packages
+      if: env.BUILD_32BIT == 'true'
+      run: sudo apt install -y gcc-multilib
+    - name: Install aarch64 cross compiling packages
+      if: env.AARCH64 == 'true'
+      run: sudo apt install -y gcc-aarch64-linux-gnu libc6-dev-arm64-cross
+        pkg-config-aarch64-linux-gnu
+    - name: Install doc generation packages
+      if: env.BUILD_DOCS == 'true'
+      run: sudo apt install -y doxygen graphviz python3-sphinx
+        python3-sphinx-rtd-theme
+    - name: Run setup
+      run: |
+        .ci/linux-setup.sh
+        # Workaround on $HOME permissions as EAL checks them for plugin loading
+        chmod o-w $HOME
+    - name: Build and test
+      run: .ci/linux-build.sh
+    - name: Upload logs on failure
+      if: failure()
+      uses: actions/upload-artifact@v2
+      with:
+        name: meson-logs-${{ join(matrix.config.*, '-') }}
+        path: |
+          build/meson-logs/testlog.txt
+          build/.ninja_log
+          build/meson-logs/meson-log.txt
diff --git a/.travis.yml b/.travis.yml
index 5e12db23b5..d655e286c3 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -34,10 +34,10 @@ jobs:
   - env: DEF_LIB="static"
     arch: amd64
     compiler: gcc
-  - env: DEF_LIB="shared" RUN_TESTS=1
+  - env: DEF_LIB="shared" RUN_TESTS=true
     arch: amd64
     compiler: gcc
-  - env: DEF_LIB="shared" BUILD_DOCS=1
+  - env: DEF_LIB="shared" BUILD_DOCS=true
     arch: amd64
     compiler: gcc
     addons:
@@ -49,10 +49,10 @@ jobs:
   - env: DEF_LIB="static"
     arch: amd64
     compiler: clang
-  - env: DEF_LIB="shared" RUN_TESTS=1
+  - env: DEF_LIB="shared" RUN_TESTS=true
     arch: amd64
     compiler: clang
-  - env: DEF_LIB="shared" BUILD_DOCS=1
+  - env: DEF_LIB="shared" BUILD_DOCS=true
     arch: amd64
     compiler: clang
     addons:
@@ -61,7 +61,7 @@ jobs:
           - *required_packages
           - *doc_packages
   # x86_64 cross-compiling 32-bits jobs
-  - env: DEF_LIB="static" BUILD_32BIT=1
+  - env: DEF_LIB="static" BUILD_32BIT=true
     arch: amd64
     compiler: gcc
     addons:
@@ -69,14 +69,14 @@ jobs:
         packages:
           - *build_32b_packages
   # x86_64 cross-compiling aarch64 jobs
-  - env: DEF_LIB="static" AARCH64=1
+  - env: DEF_LIB="static" AARCH64=true
     arch: amd64
     compiler: gcc
     addons:
       apt:
         packages:
           - *aarch64_packages
-  - env: DEF_LIB="shared" AARCH64=1
+  - env: DEF_LIB="shared" AARCH64=true
     arch: amd64
     compiler: gcc
     addons:
@@ -87,16 +87,16 @@ jobs:
   - env: DEF_LIB="static"
     arch: arm64
     compiler: gcc
-  - env: DEF_LIB="shared" RUN_TESTS=1
+  - env: DEF_LIB="shared" RUN_TESTS=true
     arch: arm64
     compiler: gcc
-  - env: DEF_LIB="shared" RUN_TESTS=1
+  - env: DEF_LIB="shared" RUN_TESTS=true
     dist: focal
     arch: arm64-graviton2
     virt: vm
     group: edge
     compiler: gcc
-  - env: DEF_LIB="shared" BUILD_DOCS=1
+  - env: DEF_LIB="shared" BUILD_DOCS=true
     arch: arm64
     compiler: gcc
     addons:
@@ -108,10 +108,10 @@ jobs:
   - env: DEF_LIB="static"
     arch: arm64
     compiler: clang
-  - env: DEF_LIB="shared" RUN_TESTS=1
+  - env: DEF_LIB="shared" RUN_TESTS=true
     arch: arm64
     compiler: clang
-  - env: DEF_LIB="shared" RUN_TESTS=1
+  - env: DEF_LIB="shared" RUN_TESTS=true
     dist: focal
     arch: arm64-graviton2
     virt: vm
diff --git a/MAINTAINERS b/MAINTAINERS
index eafe9f8c46..f45c8c1b13 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -109,6 +109,7 @@ Public CI
 M: Aaron Conole <aconole@redhat.com>
 M: Michael Santana <maicolgabriel@hotmail.com>
 F: .travis.yml
+F: .github/workflows/build.yml
 F: .ci/
 
 ABI Policy & Versioning
-- 
2.23.0

[dpdk-dev] [PATCH v2 2/2] ci: enable v21 ABI checks

[David Marchand]

v21 ABI will be maintained until v21.11.

Let's use the latest released libabigail 1.8.

In GitHub Actions, libabigail binaries and the ABI reference are stored
in two shared caches as all branches can use the same.

While at it, we can reproduce changes from the commit 0b8086ce3fe7
("devtools: remove useless files from ABI reference").
This will save some space in the CI caches.

Signed-off-by: David Marchand <david.marchand@redhat.com>
---
 .ci/linux-build.sh          |  5 ++++-
 .github/workflows/build.yml | 26 +++++++++++++++++++++++++-
 .travis.yml                 | 27 +++++++++++++++++++++++++++
 3 files changed, 56 insertions(+), 2 deletions(-)

diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh
index ee8d07f865..d2c821adf3 100755
--- a/.ci/linux-build.sh
+++ b/.ci/linux-build.sh
@@ -86,10 +86,13 @@ if [ "$ABI_CHECKS" = "true" ]; then
     if [ ! -d reference ]; then
         refsrcdir=$(readlink -f $(pwd)/../dpdk-$REF_GIT_TAG)
         git clone --single-branch -b $REF_GIT_TAG $REF_GIT_REPO $refsrcdir
-        meson --werror $OPTS $refsrcdir $refsrcdir/build
+        meson $OPTS -Dexamples= $refsrcdir $refsrcdir/build
         ninja -C $refsrcdir/build
         DESTDIR=$(pwd)/reference ninja -C $refsrcdir/build install
         devtools/gen-abi.sh reference
+        find reference/usr/local -name '*.a' -delete
+        rm -rf reference/usr/local/bin
+        rm -rf reference/usr/local/share
         echo $REF_GIT_TAG > reference/VERSION
     fi
 
diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml
index bef6e52372..05eb59527f 100644
--- a/.github/workflows/build.yml
+++ b/.github/workflows/build.yml
@@ -15,10 +15,13 @@ jobs:
     runs-on: ${{ matrix.config.os }}
     env:
       AARCH64: ${{ matrix.config.cross == 'aarch64' }}
+      ABI_CHECKS: ${{ contains(matrix.config.checks, 'abi') }}
       BUILD_32BIT: ${{ matrix.config.cross == 'i386' }}
       BUILD_DOCS: ${{ contains(matrix.config.checks, 'doc') }}
       CC: ccache ${{ matrix.config.compiler }}
       DEF_LIB: ${{ matrix.config.library }}
+      LIBABIGAIL_VERSION: libabigail-1.8
+      REF_GIT_TAG: v20.11
       RUN_TESTS: ${{ contains(matrix.config.checks, 'tests') }}
 
     strategy:
@@ -31,7 +34,7 @@ jobs:
           - os: ubuntu-18.04
             compiler: gcc
             library: shared
-            checks: doc+tests
+            checks: abi+doc+tests
           - os: ubuntu-18.04
             compiler: clang
             library: static
@@ -60,6 +63,10 @@ jobs:
       run: |
         echo -n '::set-output name=ccache::'
         echo 'ccache-${{ matrix.config.os }}-${{ matrix.config.compiler }}-${{ matrix.config.cross }}-'$(date -u +%Y-w%W)
+        echo -n '::set-output name=libabigail::'
+        echo 'libabigail-${{ matrix.config.os }}'
+        echo -n '::set-output name=abi::'
+        echo 'abi-${{ matrix.config.os }}-${{ matrix.config.compiler }}-${{ matrix.config.cross }}-${{ env.LIBABIGAIL_VERSION }}-${{ env.REF_GIT_TAG }}'
     - name: Retrieve ccache cache
       uses: actions/cache@v2
       with:
@@ -67,10 +74,27 @@ jobs:
         key: ${{ steps.get_ref_keys.outputs.ccache }}-${{ github.ref }}
         restore-keys: |
           ${{ steps.get_ref_keys.outputs.ccache }}-refs/heads/main
+    - name: Retrieve libabigail cache
+      id: libabigail-cache
+      uses: actions/cache@v2
+      if: env.ABI_CHECKS == 'true'
+      with:
+        path: libabigail
+        key: ${{ steps.get_ref_keys.outputs.libabigail }}
+    - name: Retrieve ABI reference cache
+      uses: actions/cache@v2
+      if: env.ABI_CHECKS == 'true'
+      with:
+        path: reference
+        key: ${{ steps.get_ref_keys.outputs.abi }}
     - name: Install packages
       run: sudo apt install -y ccache libnuma-dev python3-setuptools
         python3-wheel python3-pip ninja-build libbsd-dev libpcap-dev
         libibverbs-dev libcrypto++-dev libfdt-dev libjansson-dev
+    - name: Install libabigail build dependencies if no cache is available
+      if: env.ABI_CHECKS == 'true' && steps.libabigail-cache.outputs.cache-hit != 'true'
+      run: sudo apt install -y autoconf automake libtool pkg-config libxml2-dev
+          libdw-dev
     - name: Install i386 cross compiling packages
       if: env.BUILD_32BIT == 'true'
       run: sudo apt install -y gcc-multilib
diff --git a/.travis.yml b/.travis.yml
index d655e286c3..5aa7ad49f1 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -2,6 +2,9 @@
 language: c
 cache:
   ccache: true
+  directories:
+    - libabigail
+    - reference
 
 dist: bionic
 
@@ -18,6 +21,9 @@ _aarch64_packages: &aarch64_packages
   - *required_packages
   - [gcc-aarch64-linux-gnu, libc6-dev-arm64-cross, pkg-config-aarch64-linux-gnu]
 
+_libabigail_build_packages: &libabigail_build_packages
+  - [autoconf, automake, libtool, pkg-config, libxml2-dev, libdw-dev]
+
 _build_32b_packages: &build_32b_packages
   - *required_packages
   - [gcc-multilib]
@@ -28,6 +34,11 @@ _doc_packages: &doc_packages
 before_install: ./.ci/${TRAVIS_OS_NAME}-setup.sh
 script: ./.ci/${TRAVIS_OS_NAME}-build.sh
 
+env:
+  global:
+    - LIBABIGAIL_VERSION=libabigail-1.8
+    - REF_GIT_TAG=v20.11
+
 jobs:
   include:
   # x86_64 gcc jobs
@@ -45,6 +56,14 @@ jobs:
         packages:
           - *required_packages
           - *doc_packages
+  - env: DEF_LIB="shared" ABI_CHECKS=true
+    arch: amd64
+    compiler: gcc
+    addons:
+      apt:
+        packages:
+          - *required_packages
+          - *libabigail_build_packages
   # x86_64 clang jobs
   - env: DEF_LIB="static"
     arch: amd64
@@ -104,6 +123,14 @@ jobs:
         packages:
           - *required_packages
           - *doc_packages
+  - env: DEF_LIB="shared" ABI_CHECKS=true
+    arch: arm64
+    compiler: gcc
+    addons:
+      apt:
+        packages:
+          - *required_packages
+          - *libabigail_build_packages
   # aarch64 clang jobs
   - env: DEF_LIB="static"
     arch: arm64
-- 
2.23.0

Re: [dpdk-dev] [PATCH] ci: hook to Github Actions

[David Marchand]

On Thu, Nov 26, 2020 at 6:01 PM Honnappa Nagarahalli
<Honnappa.Nagarahalli@arm.com> wrote:
> > > Is there any guarantee that GitHub actions will be free forever?
> >
> > There is no "forever".
> I think we are spending our efforts on things that will not work for the community in the long run (unless the project spends money to buy credits)

That was not the initial goal of the patch, but GHA can be used by
developers who work on their github forks too, like for testing before
submitting to the public ml.

On spending efforts, the lab should be the priority.
My main concern was to get ABI checks back quickly since 21.02
proposals started to hit the list.
A ticket has been opened for the lab to handle this, but this can take
some time so in the interim we have GHA support.

I sent the v2 with ABI checks.


--
David Marchand

Re: [dpdk-dev] [PATCH v2 1/2] ci: hook to GitHub Actions

[Ferruh Yigit]

On 12/4/2020 5:36 PM, David Marchand wrote:
> With the recent changes in terms of free access to the Travis CI, let's
> offer an alternative with GitHub Actions.
> Running jobs on ARM is not supported unless using external runners, so
> this commit only adds builds for x86_64 and cross compiling for i386 and
> aarch64.
> 
> Differences with the Travis CI integration:
> - Error logs are not dumped to the console when something goes wrong.
>    Instead, they are gathered in a "catch-all" step and attached as
>    artifacts.
> - A cache entry is stored once and for all, but if no cache is found you
>    can inherit from the default branch cache. The cache is 5GB large, for
>    the whole git repository.
> - The maximum retention of logs and artifacts is 3 months.
> - /home/runner is world writable, so a workaround has been added for
>    starting dpdk processes.
> - Ilya, working on OVS GHA support, noticed that jobs can run with
>    processors that don't have the same capabilities. For DPDK, this
>    impacts the ccache content since everything was built with
>    -march=native so far, and we will end up with binaries that can't run
>    in a later build. The problem has not been seen in Travis CI (?) but
>    it is safer to use a fixed "-Dmachine=default" in any case.
> - Scheduling jobs is part of the configuration and takes the form of a
>    crontab. A build is scheduled every Monday at 0:00 (UTC) to provide a
>    default ccache for the week (useful for the ovsrobot).
> 
> Signed-off-by: David Marchand <david.marchand@redhat.com>
> ---
> Changelog since v1:
> - changed shell variables value in CI scripts and Travis configuration
>    (s/=[^\$]*1/=\1true), this makes it easier for GHA,
> - forced compilation as 'default' to avoid random unit tests issues in
>    GHA,
> - scheduled a run per week on Monday at 0:00 UTC,
> - updated the ccache key:
>    - no need to depend on the default-library parameter since this
>      parameter only impacts the linking of dpdk binaries,
>    - the week when the cache is generated is added so that jobs in
>      other branches can benefit from a recent cache (mimicking what we had
>      for the robot in Travis),
> - realigned documentation generation with what is done in Travis:
>    generating the doc in all jobs was a waste of resources,
> 

For series,
Tested-by: Ferruh Yigit <ferruh.yigit@intel.com>

Confirmed that ABI check script is detecting issues, in the absence of the 
Travis checks I am for having this alternative.

Re: [dpdk-dev] [PATCH v2 1/2] ci: hook to GitHub Actions

[Thomas Monjalon]

11/12/2020 21:07, Ferruh Yigit:
> On 12/4/2020 5:36 PM, David Marchand wrote:
> > With the recent changes in terms of free access to the Travis CI, let's
> > offer an alternative with GitHub Actions.
> > Running jobs on ARM is not supported unless using external runners, so
> > this commit only adds builds for x86_64 and cross compiling for i386 and
> > aarch64.
> > 
> > Differences with the Travis CI integration:
> > - Error logs are not dumped to the console when something goes wrong.
> >    Instead, they are gathered in a "catch-all" step and attached as
> >    artifacts.
> > - A cache entry is stored once and for all, but if no cache is found you
> >    can inherit from the default branch cache. The cache is 5GB large, for
> >    the whole git repository.
> > - The maximum retention of logs and artifacts is 3 months.
> > - /home/runner is world writable, so a workaround has been added for
> >    starting dpdk processes.
> > - Ilya, working on OVS GHA support, noticed that jobs can run with
> >    processors that don't have the same capabilities. For DPDK, this
> >    impacts the ccache content since everything was built with
> >    -march=native so far, and we will end up with binaries that can't run
> >    in a later build. The problem has not been seen in Travis CI (?) but
> >    it is safer to use a fixed "-Dmachine=default" in any case.
> > - Scheduling jobs is part of the configuration and takes the form of a
> >    crontab. A build is scheduled every Monday at 0:00 (UTC) to provide a
> >    default ccache for the week (useful for the ovsrobot).
> > 
> > Signed-off-by: David Marchand <david.marchand@redhat.com>
> > ---
> > Changelog since v1:
> > - changed shell variables value in CI scripts and Travis configuration
> >    (s/=[^\$]*1/=\1true), this makes it easier for GHA,
> > - forced compilation as 'default' to avoid random unit tests issues in
> >    GHA,
> > - scheduled a run per week on Monday at 0:00 UTC,
> > - updated the ccache key:
> >    - no need to depend on the default-library parameter since this
> >      parameter only impacts the linking of dpdk binaries,
> >    - the week when the cache is generated is added so that jobs in
> >      other branches can benefit from a recent cache (mimicking what we had
> >      for the robot in Travis),
> > - realigned documentation generation with what is done in Travis:
> >    generating the doc in all jobs was a waste of resources,
> > 
> 
> For series,
> Tested-by: Ferruh Yigit <ferruh.yigit@intel.com>
> 
> Confirmed that ABI check script is detecting issues, in the absence of the 
> Travis checks I am for having this alternative.

Thanks for offering an interesting CI alternative.
For the series,
Acked-by: Thomas Monjalon <thomas@monjalon.net>

Re: [dpdk-dev] [PATCH v2 1/2] ci: hook to GitHub Actions

[Aaron Conole]

David Marchand <david.marchand@redhat.com> writes:

> With the recent changes in terms of free access to the Travis CI, let's
> offer an alternative with GitHub Actions.
> Running jobs on ARM is not supported unless using external runners, so
> this commit only adds builds for x86_64 and cross compiling for i386 and
> aarch64.
>
> Differences with the Travis CI integration:
> - Error logs are not dumped to the console when something goes wrong.
>   Instead, they are gathered in a "catch-all" step and attached as
>   artifacts.
> - A cache entry is stored once and for all, but if no cache is found you
>   can inherit from the default branch cache. The cache is 5GB large, for
>   the whole git repository.
> - The maximum retention of logs and artifacts is 3 months.
> - /home/runner is world writable, so a workaround has been added for
>   starting dpdk processes.
> - Ilya, working on OVS GHA support, noticed that jobs can run with
>   processors that don't have the same capabilities. For DPDK, this
>   impacts the ccache content since everything was built with
>   -march=native so far, and we will end up with binaries that can't run
>   in a later build. The problem has not been seen in Travis CI (?) but
>   it is safer to use a fixed "-Dmachine=default" in any case.

That's because the build machine and test machine are the same, but I
think GHA uses a different model, and will spawn a new environment for
the steps.  I'm not 100% sure, because it's all supposed to be a
black-box.

> - Scheduling jobs is part of the configuration and takes the form of a
>   crontab. A build is scheduled every Monday at 0:00 (UTC) to provide a
>   default ccache for the week (useful for the ovsrobot).
>
> Signed-off-by: David Marchand <david.marchand@redhat.com>
> ---

Acked-by: Aaron Conole <aconole@redhat.com>

> Changelog since v1:
> - changed shell variables value in CI scripts and Travis configuration
>   (s/=[^\$]*1/=\1true), this makes it easier for GHA,
> - forced compilation as 'default' to avoid random unit tests issues in
>   GHA,
> - scheduled a run per week on Monday at 0:00 UTC,
> - updated the ccache key:
>   - no need to depend on the default-library parameter since this
>     parameter only impacts the linking of dpdk binaries,
>   - the week when the cache is generated is added so that jobs in
>     other branches can benefit from a recent cache (mimicking what we had
>     for the robot in Travis),
> - realigned documentation generation with what is done in Travis:
>   generating the doc in all jobs was a waste of resources,
>
> ---
>  .ci/linux-build.sh          |  17 +++---
>  .github/workflows/build.yml | 100 ++++++++++++++++++++++++++++++++++++
>  .travis.yml                 |  24 ++++-----
>  MAINTAINERS                 |   1 +
>  4 files changed, 123 insertions(+), 19 deletions(-)
>  create mode 100644 .github/workflows/build.yml
>
> diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh
> index d079801d78..ee8d07f865 100755
> --- a/.ci/linux-build.sh
> +++ b/.ci/linux-build.sh
> @@ -12,7 +12,9 @@ on_error() {
>          fi
>      done
>  }
> -trap on_error EXIT
> +# We capture the error logs as artifacts in Github Actions, no need to dump
> +# them via a EXIT handler.
> +[ -n "$GITHUB_WORKFLOW" ] || trap on_error EXIT
>  
>  install_libabigail() {
>      version=$1
> @@ -28,16 +30,16 @@ install_libabigail() {
>      rm ${version}.tar.gz
>  }
>  
> -if [ "$AARCH64" = "1" ]; then
> +if [ "$AARCH64" = "true" ]; then
>      # convert the arch specifier
>      OPTS="$OPTS --cross-file config/arm/arm64_armv8_linux_gcc"
>  fi
>  
> -if [ "$BUILD_DOCS" = "1" ]; then
> +if [ "$BUILD_DOCS" = "true" ]; then
>      OPTS="$OPTS -Denable_docs=true"
>  fi
>  
> -if [ "$BUILD_32BIT" = "1" ]; then
> +if [ "$BUILD_32BIT" = "true" ]; then
>      OPTS="$OPTS -Dc_args=-m32 -Dc_link_args=-m32"
>      export PKG_CONFIG_LIBDIR="/usr/lib32/pkgconfig"
>  fi
> @@ -48,16 +50,17 @@ else
>      OPTS="$OPTS -Dexamples=all"
>  fi
>  
> +OPTS="$OPTS -Dmachine=default"
>  OPTS="$OPTS --default-library=$DEF_LIB"
>  OPTS="$OPTS --buildtype=debugoptimized"
>  meson build --werror $OPTS
>  ninja -C build
>  
> -if [ "$AARCH64" != "1" ]; then
> +if [ "$AARCH64" != "true" ]; then
>      devtools/test-null.sh
>  fi
>  
> -if [ "$ABI_CHECKS" = "1" ]; then
> +if [ "$ABI_CHECKS" = "true" ]; then
>      LIBABIGAIL_VERSION=${LIBABIGAIL_VERSION:-libabigail-1.6}
>  
>      if [ "$(cat libabigail/VERSION 2>/dev/null)" != "$LIBABIGAIL_VERSION" ]; then
> @@ -95,6 +98,6 @@ if [ "$ABI_CHECKS" = "1" ]; then
>      devtools/check-abi.sh reference install ${ABI_CHECKS_WARN_ONLY:-}
>  fi
>  
> -if [ "$RUN_TESTS" = "1" ]; then
> +if [ "$RUN_TESTS" = "true" ]; then
>      sudo meson test -C build --suite fast-tests -t 3
>  fi
> diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml
> new file mode 100644
> index 0000000000..bef6e52372
> --- /dev/null
> +++ b/.github/workflows/build.yml
> @@ -0,0 +1,100 @@
> +name: build
> +
> +on:
> +  push:
> +  schedule:
> +    - cron: '0 0 * * 1'
> +
> +defaults:
> +  run:
> +    shell: bash --noprofile --norc -exo pipefail {0}
> +
> +jobs:
> +  build:
> +    name: ${{ join(matrix.config.*, '-') }}
> +    runs-on: ${{ matrix.config.os }}
> +    env:
> +      AARCH64: ${{ matrix.config.cross == 'aarch64' }}
> +      BUILD_32BIT: ${{ matrix.config.cross == 'i386' }}
> +      BUILD_DOCS: ${{ contains(matrix.config.checks, 'doc') }}
> +      CC: ccache ${{ matrix.config.compiler }}
> +      DEF_LIB: ${{ matrix.config.library }}
> +      RUN_TESTS: ${{ contains(matrix.config.checks, 'tests') }}
> +
> +    strategy:
> +      fail-fast: false
> +      matrix:
> +        config:
> +          - os: ubuntu-18.04
> +            compiler: gcc
> +            library: static
> +          - os: ubuntu-18.04
> +            compiler: gcc
> +            library: shared
> +            checks: doc+tests
> +          - os: ubuntu-18.04
> +            compiler: clang
> +            library: static
> +          - os: ubuntu-18.04
> +            compiler: clang
> +            library: shared
> +            checks: doc+tests
> +          - os: ubuntu-18.04
> +            compiler: gcc
> +            library: static
> +            cross: i386
> +          - os: ubuntu-18.04
> +            compiler: gcc
> +            library: static
> +            cross: aarch64
> +          - os: ubuntu-18.04
> +            compiler: gcc
> +            library: shared
> +            cross: aarch64
> +
> +    steps:
> +    - name: Checkout sources
> +      uses: actions/checkout@v2
> +    - name: Generate cache keys
> +      id: get_ref_keys
> +      run: |
> +        echo -n '::set-output name=ccache::'
> +        echo 'ccache-${{ matrix.config.os }}-${{ matrix.config.compiler }}-${{ matrix.config.cross }}-'$(date -u +%Y-w%W)
> +    - name: Retrieve ccache cache
> +      uses: actions/cache@v2
> +      with:
> +        path: ~/.ccache
> +        key: ${{ steps.get_ref_keys.outputs.ccache }}-${{ github.ref }}
> +        restore-keys: |
> +          ${{ steps.get_ref_keys.outputs.ccache }}-refs/heads/main
> +    - name: Install packages
> +      run: sudo apt install -y ccache libnuma-dev python3-setuptools
> +        python3-wheel python3-pip ninja-build libbsd-dev libpcap-dev
> +        libibverbs-dev libcrypto++-dev libfdt-dev libjansson-dev
> +    - name: Install i386 cross compiling packages
> +      if: env.BUILD_32BIT == 'true'
> +      run: sudo apt install -y gcc-multilib
> +    - name: Install aarch64 cross compiling packages
> +      if: env.AARCH64 == 'true'
> +      run: sudo apt install -y gcc-aarch64-linux-gnu libc6-dev-arm64-cross
> +        pkg-config-aarch64-linux-gnu
> +    - name: Install doc generation packages
> +      if: env.BUILD_DOCS == 'true'
> +      run: sudo apt install -y doxygen graphviz python3-sphinx
> +        python3-sphinx-rtd-theme
> +    - name: Run setup
> +      run: |
> +        .ci/linux-setup.sh
> +        # Workaround on $HOME permissions as EAL checks them for plugin loading
> +        chmod o-w $HOME
> +    - name: Build and test
> +      run: .ci/linux-build.sh
> +    - name: Upload logs on failure
> +      if: failure()
> +      uses: actions/upload-artifact@v2
> +      with:
> +        name: meson-logs-${{ join(matrix.config.*, '-') }}
> +        path: |
> +          build/meson-logs/testlog.txt
> +          build/.ninja_log
> +          build/meson-logs/meson-log.txt
> diff --git a/.travis.yml b/.travis.yml
> index 5e12db23b5..d655e286c3 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -34,10 +34,10 @@ jobs:
>    - env: DEF_LIB="static"
>      arch: amd64
>      compiler: gcc
> -  - env: DEF_LIB="shared" RUN_TESTS=1
> +  - env: DEF_LIB="shared" RUN_TESTS=true
>      arch: amd64
>      compiler: gcc
> -  - env: DEF_LIB="shared" BUILD_DOCS=1
> +  - env: DEF_LIB="shared" BUILD_DOCS=true
>      arch: amd64
>      compiler: gcc
>      addons:
> @@ -49,10 +49,10 @@ jobs:
>    - env: DEF_LIB="static"
>      arch: amd64
>      compiler: clang
> -  - env: DEF_LIB="shared" RUN_TESTS=1
> +  - env: DEF_LIB="shared" RUN_TESTS=true
>      arch: amd64
>      compiler: clang
> -  - env: DEF_LIB="shared" BUILD_DOCS=1
> +  - env: DEF_LIB="shared" BUILD_DOCS=true
>      arch: amd64
>      compiler: clang
>      addons:
> @@ -61,7 +61,7 @@ jobs:
>            - *required_packages
>            - *doc_packages
>    # x86_64 cross-compiling 32-bits jobs
> -  - env: DEF_LIB="static" BUILD_32BIT=1
> +  - env: DEF_LIB="static" BUILD_32BIT=true
>      arch: amd64
>      compiler: gcc
>      addons:
> @@ -69,14 +69,14 @@ jobs:
>          packages:
>            - *build_32b_packages
>    # x86_64 cross-compiling aarch64 jobs
> -  - env: DEF_LIB="static" AARCH64=1
> +  - env: DEF_LIB="static" AARCH64=true
>      arch: amd64
>      compiler: gcc
>      addons:
>        apt:
>          packages:
>            - *aarch64_packages
> -  - env: DEF_LIB="shared" AARCH64=1
> +  - env: DEF_LIB="shared" AARCH64=true
>      arch: amd64
>      compiler: gcc
>      addons:
> @@ -87,16 +87,16 @@ jobs:
>    - env: DEF_LIB="static"
>      arch: arm64
>      compiler: gcc
> -  - env: DEF_LIB="shared" RUN_TESTS=1
> +  - env: DEF_LIB="shared" RUN_TESTS=true
>      arch: arm64
>      compiler: gcc
> -  - env: DEF_LIB="shared" RUN_TESTS=1
> +  - env: DEF_LIB="shared" RUN_TESTS=true
>      dist: focal
>      arch: arm64-graviton2
>      virt: vm
>      group: edge
>      compiler: gcc
> -  - env: DEF_LIB="shared" BUILD_DOCS=1
> +  - env: DEF_LIB="shared" BUILD_DOCS=true
>      arch: arm64
>      compiler: gcc
>      addons:
> @@ -108,10 +108,10 @@ jobs:
>    - env: DEF_LIB="static"
>      arch: arm64
>      compiler: clang
> -  - env: DEF_LIB="shared" RUN_TESTS=1
> +  - env: DEF_LIB="shared" RUN_TESTS=true
>      arch: arm64
>      compiler: clang
> -  - env: DEF_LIB="shared" RUN_TESTS=1
> +  - env: DEF_LIB="shared" RUN_TESTS=true
>      dist: focal
>      arch: arm64-graviton2
>      virt: vm
> diff --git a/MAINTAINERS b/MAINTAINERS
> index eafe9f8c46..f45c8c1b13 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -109,6 +109,7 @@ Public CI
>  M: Aaron Conole <aconole@redhat.com>
>  M: Michael Santana <maicolgabriel@hotmail.com>
>  F: .travis.yml
> +F: .github/workflows/build.yml
>  F: .ci/
>  
>  ABI Policy & Versioning

Re: [dpdk-dev] [PATCH v2 2/2] ci: enable v21 ABI checks

[Aaron Conole]

David Marchand <david.marchand@redhat.com> writes:

> v21 ABI will be maintained until v21.11.
>
> Let's use the latest released libabigail 1.8.
>
> In GitHub Actions, libabigail binaries and the ABI reference are stored
> in two shared caches as all branches can use the same.
>
> While at it, we can reproduce changes from the commit 0b8086ce3fe7
> ("devtools: remove useless files from ABI reference").
> This will save some space in the CI caches.
>
> Signed-off-by: David Marchand <david.marchand@redhat.com>
> ---

Acked-by: Aaron Conole <aconole@redhat.com>

Re: [dpdk-dev] [PATCH v2 1/2] ci: hook to GitHub Actions

[David Marchand]

On Fri, Dec 4, 2020 at 6:37 PM David Marchand <david.marchand@redhat.com> wrote:
>
> With the recent changes in terms of free access to the Travis CI, let's
> offer an alternative with GitHub Actions.
> Running jobs on ARM is not supported unless using external runners, so
> this commit only adds builds for x86_64 and cross compiling for i386 and
> aarch64.
>
> Differences with the Travis CI integration:
> - Error logs are not dumped to the console when something goes wrong.
>   Instead, they are gathered in a "catch-all" step and attached as
>   artifacts.
> - A cache entry is stored once and for all, but if no cache is found you
>   can inherit from the default branch cache. The cache is 5GB large, for
>   the whole git repository.
> - The maximum retention of logs and artifacts is 3 months.
> - /home/runner is world writable, so a workaround has been added for
>   starting dpdk processes.
> - Ilya, working on OVS GHA support, noticed that jobs can run with
>   processors that don't have the same capabilities. For DPDK, this
>   impacts the ccache content since everything was built with
>   -march=native so far, and we will end up with binaries that can't run
>   in a later build. The problem has not been seen in Travis CI (?) but
>   it is safer to use a fixed "-Dmachine=default" in any case.
> - Scheduling jobs is part of the configuration and takes the form of a
>   crontab. A build is scheduled every Monday at 0:00 (UTC) to provide a
>   default ccache for the week (useful for the ovsrobot).
>
> Signed-off-by: David Marchand <david.marchand@redhat.com>

Tested-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Aaron Conole <aconole@redhat.com>

Series applied.

For the time being, we can check jobs status by checking:
https://github.com/ovsrobot/dpdk/actions
Next step is to update the robot to make use of
https://docs.github.com/en/free-pro-team@latest/rest/reference/actions#list-workflow-runs-for-a-repository


-- 
David Marchand