Re: [dpdk-dev] [PATCH v1] ci: add test suite run without hugepage

[Ruifeng Wang <Ruifeng.Wang@arm.com>]

> -----Original Message-----
> From: Aaron Conole <aconole@redhat.com>
> Sent: Tuesday, February 25, 2020 23:57
> To: Ruifeng Wang <Ruifeng.Wang@arm.com>
> Cc: David Marchand <david.marchand@redhat.com>; Michael Santana
> <maicolgabriel@hotmail.com>; Bruce Richardson
> <bruce.richardson@intel.com>; dev <dev@dpdk.org>; Gavin Hu
> <Gavin.Hu@arm.com>; Honnappa Nagarahalli
> <Honnappa.Nagarahalli@arm.com>; nd <nd@arm.com>
> Subject: Re: [PATCH v1] ci: add test suite run without hugepage
> 
> Ruifeng Wang <Ruifeng.Wang@arm.com> writes:
> 
> >> -----Original Message-----
> >> From: Aaron Conole <aconole@redhat.com>
> >> Sent: Tuesday, February 25, 2020 22:36
> >> To: David Marchand <david.marchand@redhat.com>
> >> Cc: Ruifeng Wang <Ruifeng.Wang@arm.com>; Michael Santana
> >> <maicolgabriel@hotmail.com>; Bruce Richardson
> >> <bruce.richardson@intel.com>; dev <dev@dpdk.org>; Gavin Hu
> >> <Gavin.Hu@arm.com>; Honnappa Nagarahalli
> >> <Honnappa.Nagarahalli@arm.com>; nd <nd@arm.com>
> >> Subject: Re: [PATCH v1] ci: add test suite run without hugepage
> >>
> >> David Marchand <david.marchand@redhat.com> writes:
> >>
> >> > On Tue, Feb 25, 2020 at 8:33 AM Ruifeng Wang
> <ruifeng.wang@arm.com>
> >> wrote:
> >> >>
> >> >> This test suite is derived from fast-tests suite. Cases in this
> >> >> suite are run with '--no-huge' flag.
> >> >>
> >> >> The suite aims to cover as many as possible test cases out of the
> >> >> fast-tests suites in the environments without huge pages support,
> >> >> like containers.
> >> >>
> >> >> Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> >> >> Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> >> >
> >> > Compilation time is what makes the most of a "tests" job in Travis.
> >> > So I'd prefer we avoid adding more jobs for x86_64 (see below).
> >> >
> >> >
> >> >> ---
> >> >>  .ci/linux-build.sh   |  4 +++
> >> >>  .travis.yml          | 12 +++++++
> >> >>  app/test/meson.build | 75
> >> >> ++++++++++++++++++++++++++++++++++++++++++++
> >> >>  3 files changed, 91 insertions(+)
> >> >>
> >> >> diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh index
> >> >> d500c4c00..39515d915 100755
> >> >> --- a/.ci/linux-build.sh
> >> >> +++ b/.ci/linux-build.sh
> >> >> @@ -92,3 +92,7 @@ fi
> >> >>  if [ "$RUN_TESTS" = "1" ]; then
> >> >>      sudo meson test -C build --suite fast-tests -t 3  fi
> >> >> +
> >> >> +if [ "$RUN_TESTS_NO_HUGE" = "1" ]; then
> >> >> +    sudo meson test -C build --suite nohuge-tests -t 3 fi
> >> >
> >> > You can replace the "boolean" RUN_TESTS with a TESTSUITES variable
> >> > that contains a list of testsuites.
> >> > Then this part becomes:
> >> >
> >> > for testsuite in ${TESTSUITES:-}; do
> >> >    sudo meson test -C build --suite $testsuite -t 3 done
> >> >
> >> > (I wonder if we reaaaally need to be root to run those tests w/ and
> >> > w/o hugepages, Aaron?)
> >>
> >> Last I tested, we didn't need root access when not using hugepages
> >> (at least for core library functionality).  It's possible that the
> >> test suites for PMDs might need that access, but we don't run them in the
> travis environment.
> >>
> >> >
> >> >> diff --git a/.travis.yml b/.travis.yml index b64a81bd0..0e07d52d0
> >> >> 100644
> >> >> --- a/.travis.yml
> >> >> +++ b/.travis.yml
> >> >> @@ -43,6 +43,9 @@ jobs:
> >> >>    - env: DEF_LIB="shared" RUN_TESTS=1
> >> >>      arch: amd64
> >> >>      compiler: gcc
> >> >> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
> >> >> +    arch: amd64
> >> >> +    compiler: gcc
> >> >
> >> > And then we only need to update the existing RUN_TESTS jobs for
> x86_64.
> >>
> >> I like the suggestion.  But I guess we'd just make RUN_TESTS=1 set
> >> TESTSUITES="..." and otherwise, we can have an ARM64 test to do all
> >> the test suites.
> > It should be OK to do all applicable test suites in a single job?
> 
> Yes, I think so.
> 
> >>
> >> Actually, the ideal would be for the tests to automatically SKIP when
> >> no hugepages are enabled.
> > In this way, case will run either w/ hugepage or w/o hugepage but not
> > both.
> 
> Hrrm?  I don't understand.  If there are hugepages configured, all the tests
> will run.  If there are no hugepages, only the tests that can pass without
> hugepages will run.
> 
I was wondering whether it makes sense to do tests with "--no-huge" in environment where hugepages are enabled.

> > It will run faster, but with less coverage?
> 
> Sortof - if there aren't hugepages configured for an environment reason (like
> unsupported) why get failure reports?  Better to either fail when we try to
> setup the hugepages, or skip when none are available because they are two
> separate concerns.
> 
> >>
> >> >
> >> >>    - env: DEF_LIB="shared" BUILD_DOCS=1
> >> >>      arch: amd64
> >> >>      compiler: gcc
> >> >> @@ -66,6 +69,9 @@ jobs:
> >> >>    - env: DEF_LIB="shared" RUN_TESTS=1
> >> >>      arch: amd64
> >> >>      compiler: clang
> >> >> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
> >> >> +    arch: amd64
> >> >> +    compiler: clang
> >> >>    - env: DEF_LIB="shared" BUILD_DOCS=1
> >> >>      arch: amd64
> >> >>      compiler: clang
> >> >> @@ -101,6 +107,9 @@ jobs:
> >> >>    - env: DEF_LIB="static"
> >> >>      arch: arm64
> >> >>      compiler: gcc
> >> >> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
> >> >> +    arch: arm64
> >> >> +    compiler: gcc
> >> >>    - env: DEF_LIB="shared" BUILD_DOCS=1
> >> >>      arch: arm64
> >> >>      compiler: gcc
> >> >> @@ -124,3 +133,6 @@ jobs:
> >> >>    - env: DEF_LIB="shared"
> >> >>      arch: arm64
> >> >>      compiler: clang
> >> >> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
> >> >> +    arch: arm64
> >> >> +    compiler: clang
> >> >
> >> >
> >> > --
> >> > David Marchand