* [dpdk-stable] [PATCH v1 1/1] ci: enable DPDK GHA for arm64 with self-hosted runners [not found] <20211013080341.29561-1-serena.he@arm.com> @ 2021-10-13 8:03 ` Serena He 2021-10-13 11:32 ` [dpdk-stable] [dpdk-dev] " Michael Santana 0 siblings, 1 reply; 3+ messages in thread From: Serena He @ 2021-10-13 8:03 UTC (permalink / raw) To: aconole, maicolgabriel, david.marchand, thomas Cc: dev, nd, honnappa.nagarahalli, ruifeng.wang, Dean.Arnold, Serena He, stable CI jobs are triggered only for repos installed with given GHApp and runners Cc: stable@dpdk.org Signed-off-by: Serena He <serena.he@arm.com> --- .github/workflows/build-arm64.yml | 118 ++++++++++++++++++++++++++++++ 1 file changed, 118 insertions(+) create mode 100644 .github/workflows/build-arm64.yml diff --git a/.github/workflows/build-arm64.yml b/.github/workflows/build-arm64.yml new file mode 100644 index 0000000000..570563f7c8 --- /dev/null +++ b/.github/workflows/build-arm64.yml @@ -0,0 +1,118 @@ +name: build-arm64 + +on: + push: + schedule: + - cron: '0 0 * * 1' + +defaults: + run: + shell: bash --noprofile --norc -exo pipefail {0} + +jobs: + build: + # Here, runners for arm64 are accessed by installed GitHub APP, thus will not be available by fork. + # you can change the following 'if' and 'runs-on' if you have your own runners installed. + # or request to get your repo on the whitelist to use GitHub APP and delete this 'if'. + if: ${{ github.repository == 'DPDK/dpdk' || github.repository == 'ovsrobot/dpdk' }} + name: ${{ join(matrix.config.*, '-') }} + runs-on: ${{ matrix.config.os }} + env: + ABI_CHECKS: ${{ contains(matrix.config.checks, 'abi') }} + BUILD_DOCS: ${{ contains(matrix.config.checks, 'doc') }} + CL: ${{ matrix.config.compiler == 'clang' }} + CC: ccache ${{ matrix.config.compiler }} + DEF_LIB: ${{ matrix.config.library }} + LIBABIGAIL_VERSION: libabigail-1.8 + REF_GIT_TAG: none + + strategy: + fail-fast: false + matrix: + config: + - os: [self-hosted,arm-ubuntu-20.04] + compiler: gcc + library: static + - os: [self-hosted,arm-ubuntu-20.04] + compiler: gcc + library: shared + checks: doc+tests + - os: [self-hosted,arm-ubuntu-20.04] + compiler: clang + library: static + - os: [self-hosted,arm-ubuntu-20.04] + compiler: clang + library: shared + checks: doc+tests + + steps: + - name: Checkout sources + uses: actions/checkout@v2 + - name: Generate cache keys + id: get_ref_keys + run: | + echo -n '::set-output name=ccache::' + echo 'ccache-${{ matrix.config.os }}-${{ matrix.config.compiler }}-${{ matrix.config.cross }}-'$(date -u +%Y-w%W) + echo -n '::set-output name=libabigail::' + echo 'libabigail-${{ matrix.config.os }}' + echo -n '::set-output name=abi::' + echo 'abi-${{ matrix.config.os }}-${{ matrix.config.compiler }}-${{ matrix.config.cross }}-${{ env.LIBABIGAIL_VERSION }}-${{ env.REF_GIT_TAG }}' + - name: Retrieve ccache cache + uses: actions/cache@v2 + with: + path: ~/.ccache + key: ${{ steps.get_ref_keys.outputs.ccache }}-${{ github.ref }} + restore-keys: | + ${{ steps.get_ref_keys.outputs.ccache }}-refs/heads/main + - name: Retrieve libabigail cache + id: libabigail-cache + uses: actions/cache@v2 + if: env.ABI_CHECKS == 'true' + with: + path: libabigail + key: ${{ steps.get_ref_keys.outputs.libabigail }} + - name: Retrieve ABI reference cache + uses: actions/cache@v2 + if: env.ABI_CHECKS == 'true' + with: + path: reference + key: ${{ steps.get_ref_keys.outputs.abi }} + - name: Update APT cache + run: sudo apt update || true + - name: Install packages + run: sudo apt install -y ccache libnuma-dev python3-setuptools + python3-wheel python3-pip python3-pyelftools ninja-build libbsd-dev + libpcap-dev libibverbs-dev libcrypto++-dev libfdt-dev libjansson-dev + libarchive-dev zlib1g-dev pkgconf + - name: Install libabigail build dependencies if no cache is available + if: env.ABI_CHECKS == 'true' && steps.libabigail-cache.outputs.cache-hit != 'true' + run: sudo apt install -y autoconf automake libtool pkg-config libxml2-dev + libdw-dev + + - name: Install test tools packages + run: sudo apt install -y gdb + - name: Install doc generation packages + if: env.BUILD_DOCS == 'true' + run: sudo apt install -y doxygen graphviz python3-sphinx + python3-sphinx-rtd-theme + - name: Run setup + run: | + .ci/linux-setup.sh + # Workaround on $HOME permissions as EAL checks them for plugin loading + chmod o-w $HOME + - name: Install clang + if: env.CL == 'true' + run: sudo apt install -y clang + - name: Build and test + run: .ci/linux-build.sh + - name: Upload logs on failure + if: failure() + uses: actions/upload-artifact@v2 + with: + name: meson-logs-${{ join(matrix.config.*, '-') }} + path: | + build/meson-logs/testlog.txt + build/.ninja_log + build/meson-logs/meson-log.txt + build/gdb.log + -- 2.17.1 ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [dpdk-stable] [dpdk-dev] [PATCH v1 1/1] ci: enable DPDK GHA for arm64 with self-hosted runners 2021-10-13 8:03 ` [dpdk-stable] [PATCH v1 1/1] ci: enable DPDK GHA for arm64 with self-hosted runners Serena He @ 2021-10-13 11:32 ` Michael Santana 2021-10-14 7:04 ` Serena He 0 siblings, 1 reply; 3+ messages in thread From: Michael Santana @ 2021-10-13 11:32 UTC (permalink / raw) To: Serena He, aconole, maicolgabriel, david.marchand, thomas Cc: dev, nd, honnappa.nagarahalli, ruifeng.wang, Dean.Arnold, stable On 10/13/21 4:03 AM, Serena He wrote: > CI jobs are triggered only for repos installed with given GHApp and runners > > Cc: stable@dpdk.org > > Signed-off-by: Serena He <serena.he@arm.com> > > --- > .github/workflows/build-arm64.yml | 118 ++++++++++++++++++++++++++++++ > 1 file changed, 118 insertions(+) > create mode 100644 .github/workflows/build-arm64.yml > > diff --git a/.github/workflows/build-arm64.yml b/.github/workflows/build-arm64.yml > new file mode 100644 > index 0000000000..570563f7c8 > --- /dev/null > +++ b/.github/workflows/build-arm64.yml Adding a new workflow should work on our 0-day-bot. We now support having multiple workflows so this looks good > @@ -0,0 +1,118 @@ > +name: build-arm64 > + > +on: > + push: > + schedule: > + - cron: '0 0 * * 1' nit: Please add a comment for when this is scheduled so we dont have to do cron math :) > + > +defaults: > + run: > + shell: bash --noprofile --norc -exo pipefail {0} > + > +jobs: > + build: > + # Here, runners for arm64 are accessed by installed GitHub APP, thus will not be available by fork. > + # you can change the following 'if' and 'runs-on' if you have your own runners installed. > + # or request to get your repo on the whitelist to use GitHub APP and delete this 'if'. I think I understand. I think you mean s/GitHub APP/GitHub/ . otherwise I dont know what that is. From my understanding you had to request special arm-based runners from github Are DPDK/dpdk and ovsrobot/dpdk whitelisted to use the arm-based runners? Maybe there was a thread about this in the past that I missed, but where and how do you get these arm-based runners from github? > + if: ${{ github.repository == 'DPDK/dpdk' || github.repository == 'ovsrobot/dpdk' }} > + name: ${{ join(matrix.config.*, '-') }} > + runs-on: ${{ matrix.config.os }} > + env: > + ABI_CHECKS: ${{ contains(matrix.config.checks, 'abi') }} > + BUILD_DOCS: ${{ contains(matrix.config.checks, 'doc') }} > + CL: ${{ matrix.config.compiler == 'clang' }} > + CC: ccache ${{ matrix.config.compiler }} > + DEF_LIB: ${{ matrix.config.library }} > + LIBABIGAIL_VERSION: libabigail-1.8 > + REF_GIT_TAG: none > + > + strategy: > + fail-fast: false > + matrix: > + config: > + - os: [self-hosted,arm-ubuntu-20.04] > + compiler: gcc > + library: static > + - os: [self-hosted,arm-ubuntu-20.04] > + compiler: gcc > + library: shared > + checks: doc+tests > + - os: [self-hosted,arm-ubuntu-20.04] > + compiler: clang > + library: static > + - os: [self-hosted,arm-ubuntu-20.04] > + compiler: clang > + library: shared > + checks: doc+tests > + > + steps: > + - name: Checkout sources > + uses: actions/checkout@v2 > + - name: Generate cache keys > + id: get_ref_keys > + run: | > + echo -n '::set-output name=ccache::' > + echo 'ccache-${{ matrix.config.os }}-${{ matrix.config.compiler }}-${{ matrix.config.cross }}-'$(date -u +%Y-w%W) > + echo -n '::set-output name=libabigail::' > + echo 'libabigail-${{ matrix.config.os }}' > + echo -n '::set-output name=abi::' > + echo 'abi-${{ matrix.config.os }}-${{ matrix.config.compiler }}-${{ matrix.config.cross }}-${{ env.LIBABIGAIL_VERSION }}-${{ env.REF_GIT_TAG }}' > + - name: Retrieve ccache cache > + uses: actions/cache@v2 > + with: > + path: ~/.ccache > + key: ${{ steps.get_ref_keys.outputs.ccache }}-${{ github.ref }} > + restore-keys: | > + ${{ steps.get_ref_keys.outputs.ccache }}-refs/heads/main > + - name: Retrieve libabigail cache > + id: libabigail-cache > + uses: actions/cache@v2 > + if: env.ABI_CHECKS == 'true' > + with: > + path: libabigail > + key: ${{ steps.get_ref_keys.outputs.libabigail }} > + - name: Retrieve ABI reference cache > + uses: actions/cache@v2 > + if: env.ABI_CHECKS == 'true' > + with: > + path: reference > + key: ${{ steps.get_ref_keys.outputs.abi }} > + - name: Update APT cache > + run: sudo apt update || true > + - name: Install packages > + run: sudo apt install -y ccache libnuma-dev python3-setuptools > + python3-wheel python3-pip python3-pyelftools ninja-build libbsd-dev > + libpcap-dev libibverbs-dev libcrypto++-dev libfdt-dev libjansson-dev > + libarchive-dev zlib1g-dev pkgconf > + - name: Install libabigail build dependencies if no cache is available > + if: env.ABI_CHECKS == 'true' && steps.libabigail-cache.outputs.cache-hit != 'true' > + run: sudo apt install -y autoconf automake libtool pkg-config libxml2-dev > + libdw-dev Lots of caching stuff. All of it needed? > + > + - name: Install test tools packages > + run: sudo apt install -y gdb > + - name: Install doc generation packages > + if: env.BUILD_DOCS == 'true' > + run: sudo apt install -y doxygen graphviz python3-sphinx > + python3-sphinx-rtd-theme > + - name: Run setup > + run: | > + .ci/linux-setup.sh > + # Workaround on $HOME permissions as EAL checks them for plugin loading > + chmod o-w $HOME > + - name: Install clang > + if: env.CL == 'true' > + run: sudo apt install -y clang > + - name: Build and test > + run: .ci/linux-build.sh > + - name: Upload logs on failure > + if: failure() > + uses: actions/upload-artifact@v2 > + with: > + name: meson-logs-${{ join(matrix.config.*, '-') }} > + path: | > + build/meson-logs/testlog.txt > + build/.ninja_log > + build/meson-logs/meson-log.txt > + build/gdb.log LGTM! > + > ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [dpdk-stable] [dpdk-dev] [PATCH v1 1/1] ci: enable DPDK GHA for arm64 with self-hosted runners 2021-10-13 11:32 ` [dpdk-stable] [dpdk-dev] " Michael Santana @ 2021-10-14 7:04 ` Serena He 0 siblings, 0 replies; 3+ messages in thread From: Serena He @ 2021-10-14 7:04 UTC (permalink / raw) To: Michael Santana Cc: dev, nd, Honnappa Nagarahalli, Ruifeng Wang, Dean Arnold, stable, aconole, maicolgabriel, david.marchand, thomas Hi Michael, thanks for the feedback, and here are some comments below. > On 10/13/21 4:03 AM, Serena He wrote: > > CI jobs are triggered only for repos installed with given GHApp and > > runners > > > > Cc: stable@dpdk.org > > > > Signed-off-by: Serena He <serena.he@arm.com> > > > > --- > > .github/workflows/build-arm64.yml | 118 > ++++++++++++++++++++++++++++++ > > 1 file changed, 118 insertions(+) > > create mode 100644 .github/workflows/build-arm64.yml > > > > diff --git a/.github/workflows/build-arm64.yml > > b/.github/workflows/build-arm64.yml > > new file mode 100644 > > index 0000000000..570563f7c8 > > --- /dev/null > > +++ b/.github/workflows/build-arm64.yml > Adding a new workflow should work on our 0-day-bot. We now support > having multiple workflows so this looks good Great! > > @@ -0,0 +1,118 @@ > > +name: build-arm64 > > + > > +on: > > + push: > > + schedule: > > + - cron: '0 0 * * 1' > nit: Please add a comment for when this is scheduled so we dont have to do > cron math :) Sure, I will add that. > > + > > +defaults: > > + run: > > + shell: bash --noprofile --norc -exo pipefail {0} > > + > > +jobs: > > + build: > > + # Here, runners for arm64 are accessed by installed GitHub APP, thus > will not be available by fork. > > + # you can change the following 'if' and 'runs-on' if you have your own > runners installed. > > + # or request to get your repo on the whitelist to use GitHub APP and > delete this 'if'. > I think I understand. I think you mean s/GitHub APP/GitHub/ . otherwise I > dont know what that is. From my understanding you had to request special > arm-based runners from github > > Are DPDK/dpdk and ovsrobot/dpdk whitelisted to use the arm-based > runners? > > Maybe there was a thread about this in the past that I missed, but where and > how do you get these arm-based runners from github? GitHub APPs are integrations with the GitHub APIs and this one provided here will send requests for arm-based runners from AWS cloud. Document will be provided along with the APP to make better understanding after release. DPDK/dpdk and ovsrobot/dpdk are both whitelisted to use. > > + if: ${{ github.repository == 'DPDK/dpdk' || github.repository == > 'ovsrobot/dpdk' }} [...] > > + - name: Generate cache keys > > + id: get_ref_keys > > + run: | > > + echo -n '::set-output name=ccache::' > > + echo 'ccache-${{ matrix.config.os }}-${{ matrix.config.compiler }}- > ${{ matrix.config.cross }}-'$(date -u +%Y-w%W) > > + echo -n '::set-output name=libabigail::' > > + echo 'libabigail-${{ matrix.config.os }}' > > + echo -n '::set-output name=abi::' > > + echo 'abi-${{ matrix.config.os }}-${{ matrix.config.compiler }}- > ${{ matrix.config.cross }}-${{ env.LIBABIGAIL_VERSION }}- > ${{ env.REF_GIT_TAG }}' > > + - name: Retrieve ccache cache > > + uses: actions/cache@v2 > > + with: > > + path: ~/.ccache > > + key: ${{ steps.get_ref_keys.outputs.ccache }}-${{ github.ref }} > > + restore-keys: | > > + ${{ steps.get_ref_keys.outputs.ccache }}-refs/heads/main > > + - name: Retrieve libabigail cache > > + id: libabigail-cache > > + uses: actions/cache@v2 > > + if: env.ABI_CHECKS == 'true' > > + with: > > + path: libabigail > > + key: ${{ steps.get_ref_keys.outputs.libabigail }} > > + - name: Retrieve ABI reference cache > > + uses: actions/cache@v2 > > + if: env.ABI_CHECKS == 'true' > > + with: > > + path: reference > > + key: ${{ steps.get_ref_keys.outputs.abi }} > > + - name: Update APT cache > > + run: sudo apt update || true > > + - name: Install packages > > + run: sudo apt install -y ccache libnuma-dev python3-setuptools > > + python3-wheel python3-pip python3-pyelftools ninja-build libbsd-dev > > + libpcap-dev libibverbs-dev libcrypto++-dev libfdt-dev libjansson-dev > > + libarchive-dev zlib1g-dev pkgconf > > + - name: Install libabigail build dependencies if no cache is available > > + if: env.ABI_CHECKS == 'true' && steps.libabigail-cache.outputs.cache- > hit != 'true' > > + run: sudo apt install -y autoconf automake libtool pkg-config libxml2- > dev > > + libdw-dev > Lots of caching stuff. All of it needed? All these caching stuff is in consistence with the build workflow on GitHub provided runners, so should be needed. ^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2021-10-14 7:05 UTC | newest] Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <20211013080341.29561-1-serena.he@arm.com> 2021-10-13 8:03 ` [dpdk-stable] [PATCH v1 1/1] ci: enable DPDK GHA for arm64 with self-hosted runners Serena He 2021-10-13 11:32 ` [dpdk-stable] [dpdk-dev] " Michael Santana 2021-10-14 7:04 ` Serena He
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).