Since Travis added credits we need to make some changes to the Travis CI. Arm has a credit agreement with Travis for arm jobs, so remove x86 jobs. Also, migrate all arm jobs to the latest environment. Juraj Linkeš (2): ci: remove x86 Travis jobs ci: update arm64 Travis jobs to Graviton2 .travis.yml | 113 ++++++++-------------------------------------------- 1 file changed, 16 insertions(+), 97 deletions(-) -- 2.20.1
Remove x86 jobs for which we don't have Travis credits. Leave arm64 jobs for which we do have the credits. Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech> --- .travis.yml | 87 ----------------------------------------------------- 1 file changed, 87 deletions(-) diff --git a/.travis.yml b/.travis.yml index 6a2181299f..899e0cd605 100644 --- a/.travis.yml +++ b/.travis.yml @@ -17,21 +17,9 @@ addons: - [libnuma-dev, python3-setuptools, python3-wheel, python3-pip, python3-pyelftools, ninja-build] - [libbsd-dev, libpcap-dev, libibverbs-dev, libcrypto++-dev, libfdt-dev, libjansson-dev] -_aarch64_packages: &aarch64_packages - - *required_packages - - [gcc-aarch64-linux-gnu, libc6-dev-arm64-cross, pkg-config-aarch64-linux-gnu] - -_aarch64_clang_packages: &aarch64_clang_packages - - *required_packages - - [libgcc-7-dev-arm64-cross, libatomic1-arm64-cross, libc6-dev-arm64-cross, pkg-config-aarch64-linux-gnu] - _libabigail_build_packages: &libabigail_build_packages - [autoconf, automake, libtool, pkg-config, libxml2-dev, libdw-dev] -_build_32b_packages: &build_32b_packages - - *required_packages - - [gcc-multilib] - _doc_packages: &doc_packages - [doxygen, graphviz, python3-sphinx] @@ -45,81 +33,6 @@ env: jobs: include: - # x86_64 gcc jobs - - env: DEF_LIB="static" - arch: amd64 - compiler: gcc - - env: DEF_LIB="shared" RUN_TESTS=true - arch: amd64 - compiler: gcc - - env: DEF_LIB="shared" BUILD_DOCS=true - arch: amd64 - compiler: gcc - addons: - apt: - packages: - - *required_packages - - *doc_packages - - env: DEF_LIB="shared" ABI_CHECKS=true - arch: amd64 - compiler: gcc - addons: - apt: - packages: - - *required_packages - - *libabigail_build_packages - # x86_64 clang jobs - - env: DEF_LIB="static" - arch: amd64 - compiler: clang - - env: DEF_LIB="shared" RUN_TESTS=true - arch: amd64 - compiler: clang - - env: DEF_LIB="shared" BUILD_DOCS=true - arch: amd64 - compiler: clang - addons: - apt: - packages: - - *required_packages - - *doc_packages - # x86_64 cross-compiling 32-bits jobs - - env: DEF_LIB="static" BUILD_32BIT=true - arch: amd64 - compiler: gcc - addons: - apt: - packages: - - *build_32b_packages - # x86_64 cross-compiling aarch64 jobs - - env: DEF_LIB="static" AARCH64=true - arch: amd64 - compiler: gcc - addons: - apt: - packages: - - *aarch64_packages - - env: DEF_LIB="shared" AARCH64=true - arch: amd64 - compiler: gcc - addons: - apt: - packages: - - *aarch64_packages - - env: DEF_LIB="static" AARCH64=true - arch: amd64 - compiler: clang - addons: - apt: - packages: - - *aarch64_clang_packages - - env: DEF_LIB="shared" AARCH64=true - arch: amd64 - compiler: clang - addons: - apt: - packages: - - *aarch64_clang_packages # aarch64 gcc jobs - env: DEF_LIB="static" arch: arm64 -- 2.20.1
Instead of using the older platform, use only the newer Graviton2 environment in Travis CI. Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech> --- .travis.yml | 26 ++++++++++++++++---------- 1 file changed, 16 insertions(+), 10 deletions(-) diff --git a/.travis.yml b/.travis.yml index 899e0cd605..fb3913ed20 100644 --- a/.travis.yml +++ b/.travis.yml @@ -35,10 +35,10 @@ jobs: include: # aarch64 gcc jobs - env: DEF_LIB="static" - arch: arm64 - compiler: gcc - - env: DEF_LIB="shared" RUN_TESTS=true - arch: arm64 + dist: focal + arch: arm64-graviton2 + virt: vm + group: edge compiler: gcc - env: DEF_LIB="shared" RUN_TESTS=true dist: focal @@ -47,7 +47,10 @@ jobs: group: edge compiler: gcc - env: DEF_LIB="shared" BUILD_DOCS=true - arch: arm64 + dist: focal + arch: arm64-graviton2 + virt: vm + group: edge compiler: gcc addons: apt: @@ -55,7 +58,10 @@ jobs: - *required_packages - *doc_packages - env: DEF_LIB="shared" ABI_CHECKS=true - arch: arm64 + dist: focal + arch: arm64-graviton2 + virt: vm + group: edge compiler: gcc addons: apt: @@ -64,10 +70,10 @@ jobs: - *libabigail_build_packages # aarch64 clang jobs - env: DEF_LIB="static" - arch: arm64 - compiler: clang - - env: DEF_LIB="shared" RUN_TESTS=true - arch: arm64 + dist: focal + arch: arm64-graviton2 + virt: vm + group: edge compiler: clang - env: DEF_LIB="shared" RUN_TESTS=true dist: focal -- 2.20.1
17/02/2021 10:42, Juraj Linkeš:
> Remove x86 jobs for which we don't have Travis credits. Leave arm64 jobs
> for which we do have the credits.
We are missing some references.
On 2/17/2021 9:42 AM, Juraj Linkeš wrote: > Since Travis added credits we need to make some changes to the Travis > CI. Arm has a credit agreement with Travis for arm jobs, so remove x86 > jobs. Also, migrate all arm jobs to the latest environment. > So this is becoming supporting a CI infrastructure within DPDK for a specific vendor, do we want this? And can't this Travis credit/support be expanded for the whole project, so everyone in the community benefit from it? > Juraj Linkeš (2): > ci: remove x86 Travis jobs > ci: update arm64 Travis jobs to Graviton2 > > .travis.yml | 113 ++++++++-------------------------------------------- > 1 file changed, 16 insertions(+), 97 deletions(-) >
Thomas Monjalon <thomas@monjalon.net> writes:
> 17/02/2021 10:42, Juraj Linkeš:
>> Remove x86 jobs for which we don't have Travis credits. Leave arm64 jobs
>> for which we do have the credits.
>
> We are missing some references.
I do see that the travis job is still running, and does include the x86
builds as well... Is there a specific problem to be resolved?
Ferruh Yigit <ferruh.yigit@intel.com> writes: > On 2/17/2021 9:42 AM, Juraj Linkeš wrote: >> Since Travis added credits we need to make some changes to the Travis >> CI. Arm has a credit agreement with Travis for arm jobs, so remove x86 >> jobs. Also, migrate all arm jobs to the latest environment. >> > > So this is becoming supporting a CI infrastructure within DPDK for a > specific vendor, do we want this? > > And can't this Travis credit/support be expanded for the whole > project, so everyone in the community benefit from it? I agree - it would be best to have it working universally. >> Juraj Linkeš (2): >> ci: remove x86 Travis jobs >> ci: update arm64 Travis jobs to Graviton2 >> >> .travis.yml | 113 ++++++++-------------------------------------------- >> 1 file changed, 16 insertions(+), 97 deletions(-) >>
Juraj Linkeš <juraj.linkes@pantheon.tech> writes:
> Remove x86 jobs for which we don't have Travis credits. Leave arm64 jobs
> for which we do have the credits.
>
> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> ---
Individual developers will still derive benefit from those jobs, yes? I
thought it was possible to use travis still for individual developer
accounts. If that's true, then we shouldn't remove these jobs.
<snip> > Subject: Re: [dpdk-dev] [RFC PATCH v1 0/2] Travis CI jobs update after > credits > > Ferruh Yigit <ferruh.yigit@intel.com> writes: > > > On 2/17/2021 9:42 AM, Juraj Linkeš wrote: > >> Since Travis added credits we need to make some changes to the Travis > >> CI. Arm has a credit agreement with Travis for arm jobs, so remove > >> x86 jobs. Also, migrate all arm jobs to the latest environment. > >> > > > > So this is becoming supporting a CI infrastructure within DPDK for a > > specific vendor, do we want this? > > > > And can't this Travis credit/support be expanded for the whole > > project, so everyone in the community benefit from it? > > I agree - it would be best to have it working universally. My understanding is that the credits are not associated with any architecture. > > >> Juraj Linkeš (2): > >> ci: remove x86 Travis jobs > >> ci: update arm64 Travis jobs to Graviton2 > >> > >> .travis.yml | 113 ++++++++-------------------------------------------- > >> 1 file changed, 16 insertions(+), 97 deletions(-) > >>
<snip>
>
> Thomas Monjalon <thomas@monjalon.net> writes:
>
> > 17/02/2021 10:42, Juraj Linkeš:
> >> Remove x86 jobs for which we don't have Travis credits. Leave arm64
> >> jobs for which we do have the credits.
> >
> > We are missing some references.
>
> I do see that the travis job is still running, and does include the x86 builds as
> well... Is there a specific problem to be resolved?
Agree, I do not see a particular problem to be resolved.
<snip>
>
> Juraj Linkeš <juraj.linkes@pantheon.tech> writes:
>
> > Remove x86 jobs for which we don't have Travis credits. Leave arm64
> > jobs for which we do have the credits.
> >
> > Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> > ---
>
> Individual developers will still derive benefit from those jobs, yes? I thought
> it was possible to use travis still for individual developer accounts. If that's
> true, then we shouldn't remove these jobs.
+1
<snip> > Subject: [RFC PATCH v1 0/2] Travis CI jobs update after credits > > Since Travis added credits we need to make some changes to the Travis CI. Arm > has a credit agreement with Travis for arm jobs, so remove x86 jobs. Also, I think there is a mis-understanding here. The credits are for the ovsrobot account and apply to all architectures. > migrate all arm jobs to the latest environment. > > Juraj Linkeš (2): > ci: remove x86 Travis jobs > ci: update arm64 Travis jobs to Graviton2 > > .travis.yml | 113 ++++++++-------------------------------------------- > 1 file changed, 16 insertions(+), 97 deletions(-) > > -- > 2.20.1
> -----Original Message----- > From: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com> > Sent: Thursday, February 18, 2021 3:40 AM > To: Juraj Linkeš <juraj.linkes@pantheon.tech>; thomas@monjalon.net; > david.marchand@redhat.com; aconole@redhat.com; > maicolgabriel@hotmail.com > Cc: dev@dpdk.org; Ruifeng Wang <Ruifeng.Wang@arm.com>; nd > <nd@arm.com>; Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>; nd > <nd@arm.com> > Subject: RE: [RFC PATCH v1 0/2] Travis CI jobs update after credits > > <snip> > > > Subject: [RFC PATCH v1 0/2] Travis CI jobs update after credits > > > > Since Travis added credits we need to make some changes to the Travis > > CI. Arm has a credit agreement with Travis for arm jobs, so remove x86 > > jobs. Also, > I think there is a mis-understanding here. The credits are for the ovsrobot > account and apply to all architectures. > This is likely my mistake. I thought there would only be enough credits to run just arm jobs. It seems there are enough credits to run everything, so I'll drop this patch. > > migrate all arm jobs to the latest environment. > > > > Juraj Linkeš (2): > > ci: remove x86 Travis jobs > > ci: update arm64 Travis jobs to Graviton2 > > > > .travis.yml | 113 > > ++++++++-------------------------------------------- > > 1 file changed, 16 insertions(+), 97 deletions(-) > > > > -- > > 2.20.1
Instead of using the older platform, use only the newer Graviton2 environment in Travis CI. Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech> --- .travis.yml | 26 ++++++++++++++++---------- 1 file changed, 16 insertions(+), 10 deletions(-) diff --git a/.travis.yml b/.travis.yml index 6a2181299f..7ed982ac07 100644 --- a/.travis.yml +++ b/.travis.yml @@ -122,10 +122,10 @@ jobs: - *aarch64_clang_packages # aarch64 gcc jobs - env: DEF_LIB="static" - arch: arm64 - compiler: gcc - - env: DEF_LIB="shared" RUN_TESTS=true - arch: arm64 + dist: focal + arch: arm64-graviton2 + virt: vm + group: edge compiler: gcc - env: DEF_LIB="shared" RUN_TESTS=true dist: focal @@ -134,7 +134,10 @@ jobs: group: edge compiler: gcc - env: DEF_LIB="shared" BUILD_DOCS=true - arch: arm64 + dist: focal + arch: arm64-graviton2 + virt: vm + group: edge compiler: gcc addons: apt: @@ -142,7 +145,10 @@ jobs: - *required_packages - *doc_packages - env: DEF_LIB="shared" ABI_CHECKS=true - arch: arm64 + dist: focal + arch: arm64-graviton2 + virt: vm + group: edge compiler: gcc addons: apt: @@ -151,10 +157,10 @@ jobs: - *libabigail_build_packages # aarch64 clang jobs - env: DEF_LIB="static" - arch: arm64 - compiler: clang - - env: DEF_LIB="shared" RUN_TESTS=true - arch: arm64 + dist: focal + arch: arm64-graviton2 + virt: vm + group: edge compiler: clang - env: DEF_LIB="shared" RUN_TESTS=true dist: focal -- 2.20.1
> -----Original Message----- > From: Juraj Linkeš <juraj.linkes@pantheon.tech> > Sent: Friday, February 19, 2021 3:34 PM > To: thomas@monjalon.net; david.marchand@redhat.com; > aconole@redhat.com; maicolgabriel@hotmail.com > Cc: dev@dpdk.org; Ruifeng Wang <Ruifeng.Wang@arm.com>; Honnappa > Nagarahalli <Honnappa.Nagarahalli@arm.com>; Juraj Linkeš > <juraj.linkes@pantheon.tech> > Subject: [PATCH v2] ci: update arm64 Travis jobs to Graviton2 > > Instead of using the older platform, use only the newer Graviton2 > environment in Travis CI. > > Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech> Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com> > --- > .travis.yml | 26 ++++++++++++++++---------- > 1 file changed, 16 insertions(+), 10 deletions(-)
Juraj Linkeš <juraj.linkes@pantheon.tech> writes: > Instead of using the older platform, use only the newer Graviton2 > environment in Travis CI. Can you expand on this a bit. Why should we need to use the newer environment? What problems will occur by using the older environment? I'm not against the change, but I don't understand it. > Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech> > ---
> -----Original Message----- > From: Aaron Conole <aconole@redhat.com> > Sent: Friday, February 19, 2021 2:27 PM > To: Juraj Linkeš <juraj.linkes@pantheon.tech> > Cc: thomas@monjalon.net; david.marchand@redhat.com; > maicolgabriel@hotmail.com; dev@dpdk.org; Ruifeng.Wang@arm.com; > Honnappa.Nagarahalli@arm.com > Subject: Re: [PATCH v2] ci: update arm64 Travis jobs to Graviton2 > > Juraj Linkeš <juraj.linkes@pantheon.tech> writes: > > > Instead of using the older platform, use only the newer Graviton2 > > environment in Travis CI. > > Can you expand on this a bit. Why should we need to use the newer > environment? What problems will occur by using the older environment? > We're seeing intermittent issues with the environment. An example run https://travis-ci.com/github/ovsrobot/dpdk/builds/217641063 Issue 1: collect2: fatal error: ld terminated with signal 9 [Killed] This suggest an issue with resources. Issue 2: ticketlock_autotest sometimes times out. This could be due variable resource availability in a container environment, but may not be. There could be other issues, I haven't analysed all of the logs, but there are enough to make us want to switch to Graviton2 for all jobs. We're don't see either issue with the Graviton2 environment (which uses VMs). > I'm not against the change, but I don't understand it. > > > Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech> > > --- >
Juraj Linkeš <juraj.linkes@pantheon.tech> writes: >> -----Original Message----- >> From: Aaron Conole <aconole@redhat.com> >> Sent: Friday, February 19, 2021 2:27 PM >> To: Juraj Linkeš <juraj.linkes@pantheon.tech> >> Cc: thomas@monjalon.net; david.marchand@redhat.com; >> maicolgabriel@hotmail.com; dev@dpdk.org; Ruifeng.Wang@arm.com; >> Honnappa.Nagarahalli@arm.com >> Subject: Re: [PATCH v2] ci: update arm64 Travis jobs to Graviton2 >> >> Juraj Linkeš <juraj.linkes@pantheon.tech> writes: >> >> > Instead of using the older platform, use only the newer Graviton2 >> > environment in Travis CI. >> >> Can you expand on this a bit. Why should we need to use the newer >> environment? What problems will occur by using the older environment? >> > > We're seeing intermittent issues with the environment. > > An example run https://travis-ci.com/github/ovsrobot/dpdk/builds/217641063 > > Issue 1: > collect2: fatal error: ld terminated with signal 9 [Killed] > > This suggest an issue with resources. > > Issue 2: > ticketlock_autotest sometimes times out. This could be due variable > resource availability in a container environment, but may not be. > > There could be other issues, I haven't analysed all of the logs, but > there are enough to make us want to switch to Graviton2 for all jobs. > > We're don't see either issue with the Graviton2 environment (which uses VMs). I guess it would be good to mention this motivation in the commit message - otherwise looking at the history we won't understand why we made this change. Please post v2 mentioning resource availability issues with travis-ci using the container infrastructure which don't seem to impact the VM infrastructure. I will ACK. >> I'm not against the change, but I don't understand it. >> >> > Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech> >> > --- >>
Use only the newer Graviton2 environment in Travis CI instead of using the older platform, which has intermittent issues: 1. collect2: fatal error: ld terminated with signal 9 [Killed] 2. ticketlock_autotest sometimes times out These failures hint at resource availability issues in container environments. The Graviton2 environment is using VMs and these failures are not observed in it. Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech> --- .travis.yml | 26 ++++++++++++++++---------- 1 file changed, 16 insertions(+), 10 deletions(-) diff --git a/.travis.yml b/.travis.yml index 6a2181299f..7ed982ac07 100644 --- a/.travis.yml +++ b/.travis.yml @@ -122,10 +122,10 @@ jobs: - *aarch64_clang_packages # aarch64 gcc jobs - env: DEF_LIB="static" - arch: arm64 - compiler: gcc - - env: DEF_LIB="shared" RUN_TESTS=true - arch: arm64 + dist: focal + arch: arm64-graviton2 + virt: vm + group: edge compiler: gcc - env: DEF_LIB="shared" RUN_TESTS=true dist: focal @@ -134,7 +134,10 @@ jobs: group: edge compiler: gcc - env: DEF_LIB="shared" BUILD_DOCS=true - arch: arm64 + dist: focal + arch: arm64-graviton2 + virt: vm + group: edge compiler: gcc addons: apt: @@ -142,7 +145,10 @@ jobs: - *required_packages - *doc_packages - env: DEF_LIB="shared" ABI_CHECKS=true - arch: arm64 + dist: focal + arch: arm64-graviton2 + virt: vm + group: edge compiler: gcc addons: apt: @@ -151,10 +157,10 @@ jobs: - *libabigail_build_packages # aarch64 clang jobs - env: DEF_LIB="static" - arch: arm64 - compiler: clang - - env: DEF_LIB="shared" RUN_TESTS=true - arch: arm64 + dist: focal + arch: arm64-graviton2 + virt: vm + group: edge compiler: clang - env: DEF_LIB="shared" RUN_TESTS=true dist: focal -- 2.20.1
> -----Original Message----- > From: Juraj Linkeš <juraj.linkes@pantheon.tech> > Sent: Tuesday, February 23, 2021 4:28 PM > To: thomas@monjalon.net; david.marchand@redhat.com; > aconole@redhat.com; maicolgabriel@hotmail.com > Cc: dev@dpdk.org; Ruifeng Wang <Ruifeng.Wang@arm.com>; Honnappa > Nagarahalli <Honnappa.Nagarahalli@arm.com>; Juraj Linkeš > <juraj.linkes@pantheon.tech> > Subject: [PATCH v3] ci: update arm64 Travis jobs to Graviton2 > > Use only the newer Graviton2 environment in Travis CI instead of using the > older platform, which has intermittent issues: > 1. collect2: fatal error: ld terminated with signal 9 [Killed] 2. > ticketlock_autotest sometimes times out > > These failures hint at resource availability issues in container environments. > The Graviton2 environment is using VMs and these failures are not observed > in it. > > Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech> Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com> > --- > .travis.yml | 26 ++++++++++++++++---------- > 1 file changed, 16 insertions(+), 10 deletions(-)
Juraj Linkeš <juraj.linkes@pantheon.tech> writes:
> Use only the newer Graviton2 environment in Travis CI instead of using
> the older platform, which has intermittent issues:
> 1. collect2: fatal error: ld terminated with signal 9 [Killed]
> 2. ticketlock_autotest sometimes times out
>
> These failures hint at resource availability issues in container
> environments. The Graviton2 environment is using VMs and these failures
> are not observed in it.
>
> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> ---
Acked-by: Aaron Conole <aconole@redhat.com>
On Tue, Feb 23, 2021 at 9:28 AM Juraj Linkeš <juraj.linkes@pantheon.tech> wrote:
>
> Use only the newer Graviton2 environment in Travis CI instead of using
> the older platform, which has intermittent issues:
> 1. collect2: fatal error: ld terminated with signal 9 [Killed]
> 2. ticketlock_autotest sometimes times out
>
> These failures hint at resource availability issues in container
> environments. The Graviton2 environment is using VMs and these failures
> are not observed in it.
>
> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Acked-by: Aaron Conole <aconole@redhat.com>
Applied, thanks Juraj.
--
David Marchand