* [dpdk-stable] [PATCH v1 1/1] ci: enable DPDK GHA for arm64 with self-hosted runners
[not found] <20211013080341.29561-1-serena.he@arm.com>
@ 2021-10-13 8:03 ` Serena He
2021-10-13 11:32 ` [dpdk-stable] [dpdk-dev] " Michael Santana
0 siblings, 1 reply; 3+ messages in thread
From: Serena He @ 2021-10-13 8:03 UTC (permalink / raw)
To: aconole, maicolgabriel, david.marchand, thomas
Cc: dev, nd, honnappa.nagarahalli, ruifeng.wang, Dean.Arnold,
Serena He, stable
CI jobs are triggered only for repos installed with given GHApp and runners
Cc: stable@dpdk.org
Signed-off-by: Serena He <serena.he@arm.com>
---
.github/workflows/build-arm64.yml | 118 ++++++++++++++++++++++++++++++
1 file changed, 118 insertions(+)
create mode 100644 .github/workflows/build-arm64.yml
diff --git a/.github/workflows/build-arm64.yml b/.github/workflows/build-arm64.yml
new file mode 100644
index 0000000000..570563f7c8
--- /dev/null
+++ b/.github/workflows/build-arm64.yml
@@ -0,0 +1,118 @@
+name: build-arm64
+
+on:
+ push:
+ schedule:
+ - cron: '0 0 * * 1'
+
+defaults:
+ run:
+ shell: bash --noprofile --norc -exo pipefail {0}
+
+jobs:
+ build:
+ # Here, runners for arm64 are accessed by installed GitHub APP, thus will not be available by fork.
+ # you can change the following 'if' and 'runs-on' if you have your own runners installed.
+ # or request to get your repo on the whitelist to use GitHub APP and delete this 'if'.
+ if: ${{ github.repository == 'DPDK/dpdk' || github.repository == 'ovsrobot/dpdk' }}
+ name: ${{ join(matrix.config.*, '-') }}
+ runs-on: ${{ matrix.config.os }}
+ env:
+ ABI_CHECKS: ${{ contains(matrix.config.checks, 'abi') }}
+ BUILD_DOCS: ${{ contains(matrix.config.checks, 'doc') }}
+ CL: ${{ matrix.config.compiler == 'clang' }}
+ CC: ccache ${{ matrix.config.compiler }}
+ DEF_LIB: ${{ matrix.config.library }}
+ LIBABIGAIL_VERSION: libabigail-1.8
+ REF_GIT_TAG: none
+
+ strategy:
+ fail-fast: false
+ matrix:
+ config:
+ - os: [self-hosted,arm-ubuntu-20.04]
+ compiler: gcc
+ library: static
+ - os: [self-hosted,arm-ubuntu-20.04]
+ compiler: gcc
+ library: shared
+ checks: doc+tests
+ - os: [self-hosted,arm-ubuntu-20.04]
+ compiler: clang
+ library: static
+ - os: [self-hosted,arm-ubuntu-20.04]
+ compiler: clang
+ library: shared
+ checks: doc+tests
+
+ steps:
+ - name: Checkout sources
+ uses: actions/checkout@v2
+ - name: Generate cache keys
+ id: get_ref_keys
+ run: |
+ echo -n '::set-output name=ccache::'
+ echo 'ccache-${{ matrix.config.os }}-${{ matrix.config.compiler }}-${{ matrix.config.cross }}-'$(date -u +%Y-w%W)
+ echo -n '::set-output name=libabigail::'
+ echo 'libabigail-${{ matrix.config.os }}'
+ echo -n '::set-output name=abi::'
+ echo 'abi-${{ matrix.config.os }}-${{ matrix.config.compiler }}-${{ matrix.config.cross }}-${{ env.LIBABIGAIL_VERSION }}-${{ env.REF_GIT_TAG }}'
+ - name: Retrieve ccache cache
+ uses: actions/cache@v2
+ with:
+ path: ~/.ccache
+ key: ${{ steps.get_ref_keys.outputs.ccache }}-${{ github.ref }}
+ restore-keys: |
+ ${{ steps.get_ref_keys.outputs.ccache }}-refs/heads/main
+ - name: Retrieve libabigail cache
+ id: libabigail-cache
+ uses: actions/cache@v2
+ if: env.ABI_CHECKS == 'true'
+ with:
+ path: libabigail
+ key: ${{ steps.get_ref_keys.outputs.libabigail }}
+ - name: Retrieve ABI reference cache
+ uses: actions/cache@v2
+ if: env.ABI_CHECKS == 'true'
+ with:
+ path: reference
+ key: ${{ steps.get_ref_keys.outputs.abi }}
+ - name: Update APT cache
+ run: sudo apt update || true
+ - name: Install packages
+ run: sudo apt install -y ccache libnuma-dev python3-setuptools
+ python3-wheel python3-pip python3-pyelftools ninja-build libbsd-dev
+ libpcap-dev libibverbs-dev libcrypto++-dev libfdt-dev libjansson-dev
+ libarchive-dev zlib1g-dev pkgconf
+ - name: Install libabigail build dependencies if no cache is available
+ if: env.ABI_CHECKS == 'true' && steps.libabigail-cache.outputs.cache-hit != 'true'
+ run: sudo apt install -y autoconf automake libtool pkg-config libxml2-dev
+ libdw-dev
+
+ - name: Install test tools packages
+ run: sudo apt install -y gdb
+ - name: Install doc generation packages
+ if: env.BUILD_DOCS == 'true'
+ run: sudo apt install -y doxygen graphviz python3-sphinx
+ python3-sphinx-rtd-theme
+ - name: Run setup
+ run: |
+ .ci/linux-setup.sh
+ # Workaround on $HOME permissions as EAL checks them for plugin loading
+ chmod o-w $HOME
+ - name: Install clang
+ if: env.CL == 'true'
+ run: sudo apt install -y clang
+ - name: Build and test
+ run: .ci/linux-build.sh
+ - name: Upload logs on failure
+ if: failure()
+ uses: actions/upload-artifact@v2
+ with:
+ name: meson-logs-${{ join(matrix.config.*, '-') }}
+ path: |
+ build/meson-logs/testlog.txt
+ build/.ninja_log
+ build/meson-logs/meson-log.txt
+ build/gdb.log
+
--
2.17.1
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [dpdk-stable] [dpdk-dev] [PATCH v1 1/1] ci: enable DPDK GHA for arm64 with self-hosted runners
2021-10-13 8:03 ` [dpdk-stable] [PATCH v1 1/1] ci: enable DPDK GHA for arm64 with self-hosted runners Serena He
@ 2021-10-13 11:32 ` Michael Santana
2021-10-14 7:04 ` Serena He
0 siblings, 1 reply; 3+ messages in thread
From: Michael Santana @ 2021-10-13 11:32 UTC (permalink / raw)
To: Serena He, aconole, maicolgabriel, david.marchand, thomas
Cc: dev, nd, honnappa.nagarahalli, ruifeng.wang, Dean.Arnold, stable
On 10/13/21 4:03 AM, Serena He wrote:
> CI jobs are triggered only for repos installed with given GHApp and runners
>
> Cc: stable@dpdk.org
>
> Signed-off-by: Serena He <serena.he@arm.com>
>
> ---
> .github/workflows/build-arm64.yml | 118 ++++++++++++++++++++++++++++++
> 1 file changed, 118 insertions(+)
> create mode 100644 .github/workflows/build-arm64.yml
>
> diff --git a/.github/workflows/build-arm64.yml b/.github/workflows/build-arm64.yml
> new file mode 100644
> index 0000000000..570563f7c8
> --- /dev/null
> +++ b/.github/workflows/build-arm64.yml
Adding a new workflow should work on our 0-day-bot. We now support
having multiple workflows so this looks good
> @@ -0,0 +1,118 @@
> +name: build-arm64
> +
> +on:
> + push:
> + schedule:
> + - cron: '0 0 * * 1'
nit: Please add a comment for when this is scheduled so we dont have to
do cron math :)
> +
> +defaults:
> + run:
> + shell: bash --noprofile --norc -exo pipefail {0}
> +
> +jobs:
> + build:
> + # Here, runners for arm64 are accessed by installed GitHub APP, thus will not be available by fork.
> + # you can change the following 'if' and 'runs-on' if you have your own runners installed.
> + # or request to get your repo on the whitelist to use GitHub APP and delete this 'if'.
I think I understand. I think you mean s/GitHub APP/GitHub/ . otherwise
I dont know what that is. From my understanding you had to request
special arm-based runners from github
Are DPDK/dpdk and ovsrobot/dpdk whitelisted to use the arm-based runners?
Maybe there was a thread about this in the past that I missed, but where
and how do you get these arm-based runners from github?
> + if: ${{ github.repository == 'DPDK/dpdk' || github.repository == 'ovsrobot/dpdk' }}
> + name: ${{ join(matrix.config.*, '-') }}
> + runs-on: ${{ matrix.config.os }}
> + env:
> + ABI_CHECKS: ${{ contains(matrix.config.checks, 'abi') }}
> + BUILD_DOCS: ${{ contains(matrix.config.checks, 'doc') }}
> + CL: ${{ matrix.config.compiler == 'clang' }}
> + CC: ccache ${{ matrix.config.compiler }}
> + DEF_LIB: ${{ matrix.config.library }}
> + LIBABIGAIL_VERSION: libabigail-1.8
> + REF_GIT_TAG: none
> +
> + strategy:
> + fail-fast: false
> + matrix:
> + config:
> + - os: [self-hosted,arm-ubuntu-20.04]
> + compiler: gcc
> + library: static
> + - os: [self-hosted,arm-ubuntu-20.04]
> + compiler: gcc
> + library: shared
> + checks: doc+tests
> + - os: [self-hosted,arm-ubuntu-20.04]
> + compiler: clang
> + library: static
> + - os: [self-hosted,arm-ubuntu-20.04]
> + compiler: clang
> + library: shared
> + checks: doc+tests
> +
> + steps:
> + - name: Checkout sources
> + uses: actions/checkout@v2
> + - name: Generate cache keys
> + id: get_ref_keys
> + run: |
> + echo -n '::set-output name=ccache::'
> + echo 'ccache-${{ matrix.config.os }}-${{ matrix.config.compiler }}-${{ matrix.config.cross }}-'$(date -u +%Y-w%W)
> + echo -n '::set-output name=libabigail::'
> + echo 'libabigail-${{ matrix.config.os }}'
> + echo -n '::set-output name=abi::'
> + echo 'abi-${{ matrix.config.os }}-${{ matrix.config.compiler }}-${{ matrix.config.cross }}-${{ env.LIBABIGAIL_VERSION }}-${{ env.REF_GIT_TAG }}'
> + - name: Retrieve ccache cache
> + uses: actions/cache@v2
> + with:
> + path: ~/.ccache
> + key: ${{ steps.get_ref_keys.outputs.ccache }}-${{ github.ref }}
> + restore-keys: |
> + ${{ steps.get_ref_keys.outputs.ccache }}-refs/heads/main
> + - name: Retrieve libabigail cache
> + id: libabigail-cache
> + uses: actions/cache@v2
> + if: env.ABI_CHECKS == 'true'
> + with:
> + path: libabigail
> + key: ${{ steps.get_ref_keys.outputs.libabigail }}
> + - name: Retrieve ABI reference cache
> + uses: actions/cache@v2
> + if: env.ABI_CHECKS == 'true'
> + with:
> + path: reference
> + key: ${{ steps.get_ref_keys.outputs.abi }}
> + - name: Update APT cache
> + run: sudo apt update || true
> + - name: Install packages
> + run: sudo apt install -y ccache libnuma-dev python3-setuptools
> + python3-wheel python3-pip python3-pyelftools ninja-build libbsd-dev
> + libpcap-dev libibverbs-dev libcrypto++-dev libfdt-dev libjansson-dev
> + libarchive-dev zlib1g-dev pkgconf
> + - name: Install libabigail build dependencies if no cache is available
> + if: env.ABI_CHECKS == 'true' && steps.libabigail-cache.outputs.cache-hit != 'true'
> + run: sudo apt install -y autoconf automake libtool pkg-config libxml2-dev
> + libdw-dev
Lots of caching stuff. All of it needed?
> +
> + - name: Install test tools packages
> + run: sudo apt install -y gdb
> + - name: Install doc generation packages
> + if: env.BUILD_DOCS == 'true'
> + run: sudo apt install -y doxygen graphviz python3-sphinx
> + python3-sphinx-rtd-theme
> + - name: Run setup
> + run: |
> + .ci/linux-setup.sh
> + # Workaround on $HOME permissions as EAL checks them for plugin loading
> + chmod o-w $HOME
> + - name: Install clang
> + if: env.CL == 'true'
> + run: sudo apt install -y clang
> + - name: Build and test
> + run: .ci/linux-build.sh
> + - name: Upload logs on failure
> + if: failure()
> + uses: actions/upload-artifact@v2
> + with:
> + name: meson-logs-${{ join(matrix.config.*, '-') }}
> + path: |
> + build/meson-logs/testlog.txt
> + build/.ninja_log
> + build/meson-logs/meson-log.txt
> + build/gdb.log
LGTM!
> +
>
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [dpdk-stable] [dpdk-dev] [PATCH v1 1/1] ci: enable DPDK GHA for arm64 with self-hosted runners
2021-10-13 11:32 ` [dpdk-stable] [dpdk-dev] " Michael Santana
@ 2021-10-14 7:04 ` Serena He
0 siblings, 0 replies; 3+ messages in thread
From: Serena He @ 2021-10-14 7:04 UTC (permalink / raw)
To: Michael Santana
Cc: dev, nd, Honnappa Nagarahalli, Ruifeng Wang, Dean Arnold, stable,
aconole, maicolgabriel, david.marchand, thomas
Hi Michael, thanks for the feedback, and here are some comments below.
> On 10/13/21 4:03 AM, Serena He wrote:
> > CI jobs are triggered only for repos installed with given GHApp and
> > runners
> >
> > Cc: stable@dpdk.org
> >
> > Signed-off-by: Serena He <serena.he@arm.com>
> >
> > ---
> > .github/workflows/build-arm64.yml | 118
> ++++++++++++++++++++++++++++++
> > 1 file changed, 118 insertions(+)
> > create mode 100644 .github/workflows/build-arm64.yml
> >
> > diff --git a/.github/workflows/build-arm64.yml
> > b/.github/workflows/build-arm64.yml
> > new file mode 100644
> > index 0000000000..570563f7c8
> > --- /dev/null
> > +++ b/.github/workflows/build-arm64.yml
> Adding a new workflow should work on our 0-day-bot. We now support
> having multiple workflows so this looks good
Great!
> > @@ -0,0 +1,118 @@
> > +name: build-arm64
> > +
> > +on:
> > + push:
> > + schedule:
> > + - cron: '0 0 * * 1'
> nit: Please add a comment for when this is scheduled so we dont have to do
> cron math :)
Sure, I will add that.
> > +
> > +defaults:
> > + run:
> > + shell: bash --noprofile --norc -exo pipefail {0}
> > +
> > +jobs:
> > + build:
> > + # Here, runners for arm64 are accessed by installed GitHub APP, thus
> will not be available by fork.
> > + # you can change the following 'if' and 'runs-on' if you have your own
> runners installed.
> > + # or request to get your repo on the whitelist to use GitHub APP and
> delete this 'if'.
> I think I understand. I think you mean s/GitHub APP/GitHub/ . otherwise I
> dont know what that is. From my understanding you had to request special
> arm-based runners from github
>
> Are DPDK/dpdk and ovsrobot/dpdk whitelisted to use the arm-based
> runners?
>
> Maybe there was a thread about this in the past that I missed, but where and
> how do you get these arm-based runners from github?
GitHub APPs are integrations with the GitHub APIs and this one provided here will send requests for arm-based runners from AWS cloud. Document will be provided along with the APP to make better understanding after release. DPDK/dpdk and ovsrobot/dpdk are both whitelisted to use.
> > + if: ${{ github.repository == 'DPDK/dpdk' || github.repository ==
> 'ovsrobot/dpdk' }}
[...]
> > + - name: Generate cache keys
> > + id: get_ref_keys
> > + run: |
> > + echo -n '::set-output name=ccache::'
> > + echo 'ccache-${{ matrix.config.os }}-${{ matrix.config.compiler }}-
> ${{ matrix.config.cross }}-'$(date -u +%Y-w%W)
> > + echo -n '::set-output name=libabigail::'
> > + echo 'libabigail-${{ matrix.config.os }}'
> > + echo -n '::set-output name=abi::'
> > + echo 'abi-${{ matrix.config.os }}-${{ matrix.config.compiler }}-
> ${{ matrix.config.cross }}-${{ env.LIBABIGAIL_VERSION }}-
> ${{ env.REF_GIT_TAG }}'
> > + - name: Retrieve ccache cache
> > + uses: actions/cache@v2
> > + with:
> > + path: ~/.ccache
> > + key: ${{ steps.get_ref_keys.outputs.ccache }}-${{ github.ref }}
> > + restore-keys: |
> > + ${{ steps.get_ref_keys.outputs.ccache }}-refs/heads/main
> > + - name: Retrieve libabigail cache
> > + id: libabigail-cache
> > + uses: actions/cache@v2
> > + if: env.ABI_CHECKS == 'true'
> > + with:
> > + path: libabigail
> > + key: ${{ steps.get_ref_keys.outputs.libabigail }}
> > + - name: Retrieve ABI reference cache
> > + uses: actions/cache@v2
> > + if: env.ABI_CHECKS == 'true'
> > + with:
> > + path: reference
> > + key: ${{ steps.get_ref_keys.outputs.abi }}
> > + - name: Update APT cache
> > + run: sudo apt update || true
> > + - name: Install packages
> > + run: sudo apt install -y ccache libnuma-dev python3-setuptools
> > + python3-wheel python3-pip python3-pyelftools ninja-build libbsd-dev
> > + libpcap-dev libibverbs-dev libcrypto++-dev libfdt-dev libjansson-dev
> > + libarchive-dev zlib1g-dev pkgconf
> > + - name: Install libabigail build dependencies if no cache is available
> > + if: env.ABI_CHECKS == 'true' && steps.libabigail-cache.outputs.cache-
> hit != 'true'
> > + run: sudo apt install -y autoconf automake libtool pkg-config libxml2-
> dev
> > + libdw-dev
> Lots of caching stuff. All of it needed?
All these caching stuff is in consistence with the build workflow on GitHub provided runners, so should be needed.
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2021-10-14 7:05 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <20211013080341.29561-1-serena.he@arm.com>
2021-10-13 8:03 ` [dpdk-stable] [PATCH v1 1/1] ci: enable DPDK GHA for arm64 with self-hosted runners Serena He
2021-10-13 11:32 ` [dpdk-stable] [dpdk-dev] " Michael Santana
2021-10-14 7:04 ` Serena He
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).