From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 24D2CA09E9; Mon, 14 Dec 2020 15:12:46 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id D5B783257; Mon, 14 Dec 2020 15:12:43 +0100 (CET) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by dpdk.org (Postfix) with ESMTP id 6ADC22B9C for ; Mon, 14 Dec 2020 15:12:41 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1607955159; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=/uxV5oiuM4ZNfMu1Ez+vc7+s4phz/sPj24Gb81a6t48=; b=RYatWS59g2Dy67s5RiYiew+qw+RZ2ETlAaSeew1s4EK7M2KIG+1PinnPd3mi3zojmJt+kM MVLQ8E2GT5wMlhvoKcnj62zVtSSD18XZq7A/kdYXlg3K7SMpujAgGxI+WgM7kEL0kZfug2 fiy+jm5DZ4zPRK2tYaz1CmxKf6LyfT0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-147-VwnnTag9OLWRUKqo-B_lCA-1; Mon, 14 Dec 2020 09:12:37 -0500 X-MC-Unique: VwnnTag9OLWRUKqo-B_lCA-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 69A35192CC56; Mon, 14 Dec 2020 14:12:36 +0000 (UTC) Received: from dhcp-25.97.bos.redhat.com (ovpn-113-60.rdu2.redhat.com [10.10.113.60]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 4CB3118F0A; Mon, 14 Dec 2020 14:12:32 +0000 (UTC) From: Aaron Conole To: David Marchand Cc: dev@dpdk.org, Michael Santana , Thomas Monjalon References: <20201124215700.12126-1-david.marchand@redhat.com> <20201204173622.12053-1-david.marchand@redhat.com> Date: Mon, 14 Dec 2020 09:12:31 -0500 In-Reply-To: <20201204173622.12053-1-david.marchand@redhat.com> (David Marchand's message of "Fri, 4 Dec 2020 18:36:21 +0100") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=aconole@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain Subject: Re: [dpdk-dev] [PATCH v2 1/2] ci: hook to GitHub Actions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" David Marchand writes: > With the recent changes in terms of free access to the Travis CI, let's > offer an alternative with GitHub Actions. > Running jobs on ARM is not supported unless using external runners, so > this commit only adds builds for x86_64 and cross compiling for i386 and > aarch64. > > Differences with the Travis CI integration: > - Error logs are not dumped to the console when something goes wrong. > Instead, they are gathered in a "catch-all" step and attached as > artifacts. > - A cache entry is stored once and for all, but if no cache is found you > can inherit from the default branch cache. The cache is 5GB large, for > the whole git repository. > - The maximum retention of logs and artifacts is 3 months. > - /home/runner is world writable, so a workaround has been added for > starting dpdk processes. > - Ilya, working on OVS GHA support, noticed that jobs can run with > processors that don't have the same capabilities. For DPDK, this > impacts the ccache content since everything was built with > -march=native so far, and we will end up with binaries that can't run > in a later build. The problem has not been seen in Travis CI (?) but > it is safer to use a fixed "-Dmachine=default" in any case. That's because the build machine and test machine are the same, but I think GHA uses a different model, and will spawn a new environment for the steps. I'm not 100% sure, because it's all supposed to be a black-box. > - Scheduling jobs is part of the configuration and takes the form of a > crontab. A build is scheduled every Monday at 0:00 (UTC) to provide a > default ccache for the week (useful for the ovsrobot). > > Signed-off-by: David Marchand > --- Acked-by: Aaron Conole > Changelog since v1: > - changed shell variables value in CI scripts and Travis configuration > (s/=[^\$]*1/=\1true), this makes it easier for GHA, > - forced compilation as 'default' to avoid random unit tests issues in > GHA, > - scheduled a run per week on Monday at 0:00 UTC, > - updated the ccache key: > - no need to depend on the default-library parameter since this > parameter only impacts the linking of dpdk binaries, > - the week when the cache is generated is added so that jobs in > other branches can benefit from a recent cache (mimicking what we had > for the robot in Travis), > - realigned documentation generation with what is done in Travis: > generating the doc in all jobs was a waste of resources, > > --- > .ci/linux-build.sh | 17 +++--- > .github/workflows/build.yml | 100 ++++++++++++++++++++++++++++++++++++ > .travis.yml | 24 ++++----- > MAINTAINERS | 1 + > 4 files changed, 123 insertions(+), 19 deletions(-) > create mode 100644 .github/workflows/build.yml > > diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh > index d079801d78..ee8d07f865 100755 > --- a/.ci/linux-build.sh > +++ b/.ci/linux-build.sh > @@ -12,7 +12,9 @@ on_error() { > fi > done > } > -trap on_error EXIT > +# We capture the error logs as artifacts in Github Actions, no need to dump > +# them via a EXIT handler. > +[ -n "$GITHUB_WORKFLOW" ] || trap on_error EXIT > > install_libabigail() { > version=$1 > @@ -28,16 +30,16 @@ install_libabigail() { > rm ${version}.tar.gz > } > > -if [ "$AARCH64" = "1" ]; then > +if [ "$AARCH64" = "true" ]; then > # convert the arch specifier > OPTS="$OPTS --cross-file config/arm/arm64_armv8_linux_gcc" > fi > > -if [ "$BUILD_DOCS" = "1" ]; then > +if [ "$BUILD_DOCS" = "true" ]; then > OPTS="$OPTS -Denable_docs=true" > fi > > -if [ "$BUILD_32BIT" = "1" ]; then > +if [ "$BUILD_32BIT" = "true" ]; then > OPTS="$OPTS -Dc_args=-m32 -Dc_link_args=-m32" > export PKG_CONFIG_LIBDIR="/usr/lib32/pkgconfig" > fi > @@ -48,16 +50,17 @@ else > OPTS="$OPTS -Dexamples=all" > fi > > +OPTS="$OPTS -Dmachine=default" > OPTS="$OPTS --default-library=$DEF_LIB" > OPTS="$OPTS --buildtype=debugoptimized" > meson build --werror $OPTS > ninja -C build > > -if [ "$AARCH64" != "1" ]; then > +if [ "$AARCH64" != "true" ]; then > devtools/test-null.sh > fi > > -if [ "$ABI_CHECKS" = "1" ]; then > +if [ "$ABI_CHECKS" = "true" ]; then > LIBABIGAIL_VERSION=${LIBABIGAIL_VERSION:-libabigail-1.6} > > if [ "$(cat libabigail/VERSION 2>/dev/null)" != "$LIBABIGAIL_VERSION" ]; then > @@ -95,6 +98,6 @@ if [ "$ABI_CHECKS" = "1" ]; then > devtools/check-abi.sh reference install ${ABI_CHECKS_WARN_ONLY:-} > fi > > -if [ "$RUN_TESTS" = "1" ]; then > +if [ "$RUN_TESTS" = "true" ]; then > sudo meson test -C build --suite fast-tests -t 3 > fi > diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml > new file mode 100644 > index 0000000000..bef6e52372 > --- /dev/null > +++ b/.github/workflows/build.yml > @@ -0,0 +1,100 @@ > +name: build > + > +on: > + push: > + schedule: > + - cron: '0 0 * * 1' > + > +defaults: > + run: > + shell: bash --noprofile --norc -exo pipefail {0} > + > +jobs: > + build: > + name: ${{ join(matrix.config.*, '-') }} > + runs-on: ${{ matrix.config.os }} > + env: > + AARCH64: ${{ matrix.config.cross == 'aarch64' }} > + BUILD_32BIT: ${{ matrix.config.cross == 'i386' }} > + BUILD_DOCS: ${{ contains(matrix.config.checks, 'doc') }} > + CC: ccache ${{ matrix.config.compiler }} > + DEF_LIB: ${{ matrix.config.library }} > + RUN_TESTS: ${{ contains(matrix.config.checks, 'tests') }} > + > + strategy: > + fail-fast: false > + matrix: > + config: > + - os: ubuntu-18.04 > + compiler: gcc > + library: static > + - os: ubuntu-18.04 > + compiler: gcc > + library: shared > + checks: doc+tests > + - os: ubuntu-18.04 > + compiler: clang > + library: static > + - os: ubuntu-18.04 > + compiler: clang > + library: shared > + checks: doc+tests > + - os: ubuntu-18.04 > + compiler: gcc > + library: static > + cross: i386 > + - os: ubuntu-18.04 > + compiler: gcc > + library: static > + cross: aarch64 > + - os: ubuntu-18.04 > + compiler: gcc > + library: shared > + cross: aarch64 > + > + steps: > + - name: Checkout sources > + uses: actions/checkout@v2 > + - name: Generate cache keys > + id: get_ref_keys > + run: | > + echo -n '::set-output name=ccache::' > + echo 'ccache-${{ matrix.config.os }}-${{ matrix.config.compiler }}-${{ matrix.config.cross }}-'$(date -u +%Y-w%W) > + - name: Retrieve ccache cache > + uses: actions/cache@v2 > + with: > + path: ~/.ccache > + key: ${{ steps.get_ref_keys.outputs.ccache }}-${{ github.ref }} > + restore-keys: | > + ${{ steps.get_ref_keys.outputs.ccache }}-refs/heads/main > + - name: Install packages > + run: sudo apt install -y ccache libnuma-dev python3-setuptools > + python3-wheel python3-pip ninja-build libbsd-dev libpcap-dev > + libibverbs-dev libcrypto++-dev libfdt-dev libjansson-dev > + - name: Install i386 cross compiling packages > + if: env.BUILD_32BIT == 'true' > + run: sudo apt install -y gcc-multilib > + - name: Install aarch64 cross compiling packages > + if: env.AARCH64 == 'true' > + run: sudo apt install -y gcc-aarch64-linux-gnu libc6-dev-arm64-cross > + pkg-config-aarch64-linux-gnu > + - name: Install doc generation packages > + if: env.BUILD_DOCS == 'true' > + run: sudo apt install -y doxygen graphviz python3-sphinx > + python3-sphinx-rtd-theme > + - name: Run setup > + run: | > + .ci/linux-setup.sh > + # Workaround on $HOME permissions as EAL checks them for plugin loading > + chmod o-w $HOME > + - name: Build and test > + run: .ci/linux-build.sh > + - name: Upload logs on failure > + if: failure() > + uses: actions/upload-artifact@v2 > + with: > + name: meson-logs-${{ join(matrix.config.*, '-') }} > + path: | > + build/meson-logs/testlog.txt > + build/.ninja_log > + build/meson-logs/meson-log.txt > diff --git a/.travis.yml b/.travis.yml > index 5e12db23b5..d655e286c3 100644 > --- a/.travis.yml > +++ b/.travis.yml > @@ -34,10 +34,10 @@ jobs: > - env: DEF_LIB="static" > arch: amd64 > compiler: gcc > - - env: DEF_LIB="shared" RUN_TESTS=1 > + - env: DEF_LIB="shared" RUN_TESTS=true > arch: amd64 > compiler: gcc > - - env: DEF_LIB="shared" BUILD_DOCS=1 > + - env: DEF_LIB="shared" BUILD_DOCS=true > arch: amd64 > compiler: gcc > addons: > @@ -49,10 +49,10 @@ jobs: > - env: DEF_LIB="static" > arch: amd64 > compiler: clang > - - env: DEF_LIB="shared" RUN_TESTS=1 > + - env: DEF_LIB="shared" RUN_TESTS=true > arch: amd64 > compiler: clang > - - env: DEF_LIB="shared" BUILD_DOCS=1 > + - env: DEF_LIB="shared" BUILD_DOCS=true > arch: amd64 > compiler: clang > addons: > @@ -61,7 +61,7 @@ jobs: > - *required_packages > - *doc_packages > # x86_64 cross-compiling 32-bits jobs > - - env: DEF_LIB="static" BUILD_32BIT=1 > + - env: DEF_LIB="static" BUILD_32BIT=true > arch: amd64 > compiler: gcc > addons: > @@ -69,14 +69,14 @@ jobs: > packages: > - *build_32b_packages > # x86_64 cross-compiling aarch64 jobs > - - env: DEF_LIB="static" AARCH64=1 > + - env: DEF_LIB="static" AARCH64=true > arch: amd64 > compiler: gcc > addons: > apt: > packages: > - *aarch64_packages > - - env: DEF_LIB="shared" AARCH64=1 > + - env: DEF_LIB="shared" AARCH64=true > arch: amd64 > compiler: gcc > addons: > @@ -87,16 +87,16 @@ jobs: > - env: DEF_LIB="static" > arch: arm64 > compiler: gcc > - - env: DEF_LIB="shared" RUN_TESTS=1 > + - env: DEF_LIB="shared" RUN_TESTS=true > arch: arm64 > compiler: gcc > - - env: DEF_LIB="shared" RUN_TESTS=1 > + - env: DEF_LIB="shared" RUN_TESTS=true > dist: focal > arch: arm64-graviton2 > virt: vm > group: edge > compiler: gcc > - - env: DEF_LIB="shared" BUILD_DOCS=1 > + - env: DEF_LIB="shared" BUILD_DOCS=true > arch: arm64 > compiler: gcc > addons: > @@ -108,10 +108,10 @@ jobs: > - env: DEF_LIB="static" > arch: arm64 > compiler: clang > - - env: DEF_LIB="shared" RUN_TESTS=1 > + - env: DEF_LIB="shared" RUN_TESTS=true > arch: arm64 > compiler: clang > - - env: DEF_LIB="shared" RUN_TESTS=1 > + - env: DEF_LIB="shared" RUN_TESTS=true > dist: focal > arch: arm64-graviton2 > virt: vm > diff --git a/MAINTAINERS b/MAINTAINERS > index eafe9f8c46..f45c8c1b13 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -109,6 +109,7 @@ Public CI > M: Aaron Conole > M: Michael Santana > F: .travis.yml > +F: .github/workflows/build.yml > F: .ci/ > > ABI Policy & Versioning