[dpdk-dev] [PATCH v1] ci: add test suite run without hugepage

[dpdk-dev] [PATCH v1] ci: add test suite run without hugepage

[Ruifeng Wang]

This test suite is derived from fast-tests suite. Cases in this
suite are run with '--no-huge' flag.

The suite aims to cover as many as possible test cases out of the
fast-tests suites in the environments without huge pages support,
like containers.

Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>
---
 .ci/linux-build.sh   |  4 +++
 .travis.yml          | 12 +++++++
 app/test/meson.build | 75 ++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 91 insertions(+)

diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh
index d500c4c00..39515d915 100755
--- a/.ci/linux-build.sh
+++ b/.ci/linux-build.sh
@@ -92,3 +92,7 @@ fi
 if [ "$RUN_TESTS" = "1" ]; then
     sudo meson test -C build --suite fast-tests -t 3
 fi
+
+if [ "$RUN_TESTS_NO_HUGE" = "1" ]; then
+    sudo meson test -C build --suite nohuge-tests -t 3
+fi
diff --git a/.travis.yml b/.travis.yml
index b64a81bd0..0e07d52d0 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -43,6 +43,9 @@ jobs:
   - env: DEF_LIB="shared" RUN_TESTS=1
     arch: amd64
     compiler: gcc
+  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
+    arch: amd64
+    compiler: gcc
   - env: DEF_LIB="shared" BUILD_DOCS=1
     arch: amd64
     compiler: gcc
@@ -66,6 +69,9 @@ jobs:
   - env: DEF_LIB="shared" RUN_TESTS=1
     arch: amd64
     compiler: clang
+  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
+    arch: amd64
+    compiler: clang
   - env: DEF_LIB="shared" BUILD_DOCS=1
     arch: amd64
     compiler: clang
@@ -101,6 +107,9 @@ jobs:
   - env: DEF_LIB="static"
     arch: arm64
     compiler: gcc
+  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
+    arch: arm64
+    compiler: gcc
   - env: DEF_LIB="shared" BUILD_DOCS=1
     arch: arm64
     compiler: gcc
@@ -124,3 +133,6 @@ jobs:
   - env: DEF_LIB="shared"
     arch: arm64
     compiler: clang
+  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
+    arch: arm64
+    compiler: clang
diff --git a/app/test/meson.build b/app/test/meson.build
index 0a2ce710f..43a23d351 100644
--- a/app/test/meson.build
+++ b/app/test/meson.build
@@ -237,6 +237,64 @@ fast_test_names = [
         'thash_autotest',
 ]
 
+nohuge_test_names = [
+        'alarm_autotest',
+        'byteorder_autotest',
+        'cmdline_autotest',
+        'common_autotest',
+        'cpuflags_autotest',
+        'cycles_autotest',
+        'debug_autotest',
+        'eal_flags_n_opt_autotest',
+        'eal_flags_no_huge_autotest',
+        'eal_flags_vdev_opt_autotest',
+        'eal_fs_autotest',
+        'errno_autotest',
+        'event_ring_autotest',
+        'fib_autotest',
+        'fib6_autotest',
+        'interrupt_autotest',
+        'logs_autotest',
+        'lpm_autotest',
+        'lpm6_autotest',
+        'mbuf_autotest',
+        'memcpy_autotest',
+        'memory_autotest',
+        'meter_autotest',
+        'multiprocess_autotest',
+        'per_lcore_autotest',
+        'prefetch_autotest',
+        'rcu_qsbr_autotest',
+        'red_autotest',
+        'rib_autotest',
+        'rib6_autotest',
+        'ring_autotest',
+        'rwlock_rda_autotest',
+        'rwlock_rds_wrm_autotest',
+        'rwlock_rde_wro_autotest',
+        'sched_autotest',
+        'spinlock_autotest',
+        'string_autotest',
+        'tailq_autotest',
+        'user_delay_us',
+        'version_autotest',
+        'crc_autotest',
+        'delay_us_sleep_autotest',
+        'eventdev_common_autotest',
+        'fbarray_autotest',
+        'ipsec_autotest',
+        'kni_autotest',
+        'kvargs_autotest',
+        'member_autotest',
+        'metrics_autotest',
+        'power_cpufreq_autotest',
+        'power_autotest',
+        'power_kvm_vm_autotest',
+        'reorder_autotest',
+        'service_autotest',
+        'thash_autotest',
+]
+
 perf_test_names = [
         'ring_perf_autotest',
         'mempool_perf_autotest',
@@ -341,6 +399,10 @@ if dpdk_conf.has('RTE_LIBRTE_RING_PMD')
 	fast_test_names += 'latencystats_autotest'
 	driver_test_names += 'link_bonding_mode4_autotest'
 	fast_test_names += 'pdump_autotest'
+	nohuge_test_names += 'ring_pmd_autotest'
+	nohuge_test_names += 'bitratestats_autotest'
+	nohuge_test_names += 'latencystats_autotest'
+	nohuge_test_names += 'pdump_autotest'
 endif
 
 if dpdk_conf.has('RTE_LIBRTE_POWER')
@@ -430,6 +492,19 @@ foreach arg : fast_test_names
 	endif
 endforeach
 
+foreach arg : nohuge_test_names
+	if host_machine.system() == 'linux'
+		test(arg, dpdk_test,
+			  env : ['DPDK_TEST=' + arg],
+			  args : test_args +
+				 ['--no-huge'] + ['-m 1024'] +
+				 ['--file-prefix=@0@'.format(arg)],
+		timeout : timeout_seconds_fast,
+		is_parallel : false,
+		suite : 'nohuge-tests')
+	endif
+endforeach
+
 foreach arg : perf_test_names
 	test(arg, dpdk_test,
 	env : ['DPDK_TEST=' + arg],
-- 
2.17.1

Re: [dpdk-dev] [PATCH v1] ci: add test suite run without hugepage

[David Marchand]

On Tue, Feb 25, 2020 at 8:33 AM Ruifeng Wang <ruifeng.wang@arm.com> wrote:
>
> This test suite is derived from fast-tests suite. Cases in this
> suite are run with '--no-huge' flag.
>
> The suite aims to cover as many as possible test cases out of the
> fast-tests suites in the environments without huge pages support,
> like containers.
>
> Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> Reviewed-by: Gavin Hu <gavin.hu@arm.com>

Compilation time is what makes the most of a "tests" job in Travis.
So I'd prefer we avoid adding more jobs for x86_64 (see below).


> ---
>  .ci/linux-build.sh   |  4 +++
>  .travis.yml          | 12 +++++++
>  app/test/meson.build | 75 ++++++++++++++++++++++++++++++++++++++++++++
>  3 files changed, 91 insertions(+)
>
> diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh
> index d500c4c00..39515d915 100755
> --- a/.ci/linux-build.sh
> +++ b/.ci/linux-build.sh
> @@ -92,3 +92,7 @@ fi
>  if [ "$RUN_TESTS" = "1" ]; then
>      sudo meson test -C build --suite fast-tests -t 3
>  fi
> +
> +if [ "$RUN_TESTS_NO_HUGE" = "1" ]; then
> +    sudo meson test -C build --suite nohuge-tests -t 3
> +fi

You can replace the "boolean" RUN_TESTS with a TESTSUITES variable
that contains a list of testsuites.
Then this part becomes:

for testsuite in ${TESTSUITES:-}; do
   sudo meson test -C build --suite $testsuite -t 3
done

(I wonder if we reaaaally need to be root to run those tests w/ and
w/o hugepages, Aaron?)


> diff --git a/.travis.yml b/.travis.yml
> index b64a81bd0..0e07d52d0 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -43,6 +43,9 @@ jobs:
>    - env: DEF_LIB="shared" RUN_TESTS=1
>      arch: amd64
>      compiler: gcc
> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
> +    arch: amd64
> +    compiler: gcc

And then we only need to update the existing RUN_TESTS jobs for x86_64.


>    - env: DEF_LIB="shared" BUILD_DOCS=1
>      arch: amd64
>      compiler: gcc
> @@ -66,6 +69,9 @@ jobs:
>    - env: DEF_LIB="shared" RUN_TESTS=1
>      arch: amd64
>      compiler: clang
> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
> +    arch: amd64
> +    compiler: clang
>    - env: DEF_LIB="shared" BUILD_DOCS=1
>      arch: amd64
>      compiler: clang
> @@ -101,6 +107,9 @@ jobs:
>    - env: DEF_LIB="static"
>      arch: arm64
>      compiler: gcc
> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
> +    arch: arm64
> +    compiler: gcc
>    - env: DEF_LIB="shared" BUILD_DOCS=1
>      arch: arm64
>      compiler: gcc
> @@ -124,3 +133,6 @@ jobs:
>    - env: DEF_LIB="shared"
>      arch: arm64
>      compiler: clang
> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
> +    arch: arm64
> +    compiler: clang


--
David Marchand

Re: [dpdk-dev] [PATCH v1] ci: add test suite run without hugepage

[Aaron Conole]

David Marchand <david.marchand@redhat.com> writes:

> On Tue, Feb 25, 2020 at 8:33 AM Ruifeng Wang <ruifeng.wang@arm.com> wrote:
>>
>> This test suite is derived from fast-tests suite. Cases in this
>> suite are run with '--no-huge' flag.
>>
>> The suite aims to cover as many as possible test cases out of the
>> fast-tests suites in the environments without huge pages support,
>> like containers.
>>
>> Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
>> Reviewed-by: Gavin Hu <gavin.hu@arm.com>
>
> Compilation time is what makes the most of a "tests" job in Travis.
> So I'd prefer we avoid adding more jobs for x86_64 (see below).
>
>
>> ---
>>  .ci/linux-build.sh   |  4 +++
>>  .travis.yml          | 12 +++++++
>>  app/test/meson.build | 75 ++++++++++++++++++++++++++++++++++++++++++++
>>  3 files changed, 91 insertions(+)
>>
>> diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh
>> index d500c4c00..39515d915 100755
>> --- a/.ci/linux-build.sh
>> +++ b/.ci/linux-build.sh
>> @@ -92,3 +92,7 @@ fi
>>  if [ "$RUN_TESTS" = "1" ]; then
>>      sudo meson test -C build --suite fast-tests -t 3
>>  fi
>> +
>> +if [ "$RUN_TESTS_NO_HUGE" = "1" ]; then
>> +    sudo meson test -C build --suite nohuge-tests -t 3
>> +fi
>
> You can replace the "boolean" RUN_TESTS with a TESTSUITES variable
> that contains a list of testsuites.
> Then this part becomes:
>
> for testsuite in ${TESTSUITES:-}; do
>    sudo meson test -C build --suite $testsuite -t 3
> done
>
> (I wonder if we reaaaally need to be root to run those tests w/ and
> w/o hugepages, Aaron?)

Last I tested, we didn't need root access when not using hugepages (at
least for core library functionality).  It's possible that the test
suites for PMDs might need that access, but we don't run them in the
travis environment.

>
>> diff --git a/.travis.yml b/.travis.yml
>> index b64a81bd0..0e07d52d0 100644
>> --- a/.travis.yml
>> +++ b/.travis.yml
>> @@ -43,6 +43,9 @@ jobs:
>>    - env: DEF_LIB="shared" RUN_TESTS=1
>>      arch: amd64
>>      compiler: gcc
>> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
>> +    arch: amd64
>> +    compiler: gcc
>
> And then we only need to update the existing RUN_TESTS jobs for x86_64.

I like the suggestion.  But I guess we'd just make RUN_TESTS=1 set
TESTSUITES="..." and otherwise, we can have an ARM64 test to do all the
test suites.

Actually, the ideal would be for the tests to automatically SKIP when no
hugepages are enabled.

>
>>    - env: DEF_LIB="shared" BUILD_DOCS=1
>>      arch: amd64
>>      compiler: gcc
>> @@ -66,6 +69,9 @@ jobs:
>>    - env: DEF_LIB="shared" RUN_TESTS=1
>>      arch: amd64
>>      compiler: clang
>> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
>> +    arch: amd64
>> +    compiler: clang
>>    - env: DEF_LIB="shared" BUILD_DOCS=1
>>      arch: amd64
>>      compiler: clang
>> @@ -101,6 +107,9 @@ jobs:
>>    - env: DEF_LIB="static"
>>      arch: arm64
>>      compiler: gcc
>> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
>> +    arch: arm64
>> +    compiler: gcc
>>    - env: DEF_LIB="shared" BUILD_DOCS=1
>>      arch: arm64
>>      compiler: gcc
>> @@ -124,3 +133,6 @@ jobs:
>>    - env: DEF_LIB="shared"
>>      arch: arm64
>>      compiler: clang
>> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
>> +    arch: arm64
>> +    compiler: clang
>
>
> --
> David Marchand

Re: [dpdk-dev] [PATCH v1] ci: add test suite run without hugepage

[Ruifeng Wang]

> -----Original Message-----
> From: Aaron Conole <aconole@redhat.com>
> Sent: Tuesday, February 25, 2020 22:36
> To: David Marchand <david.marchand@redhat.com>
> Cc: Ruifeng Wang <Ruifeng.Wang@arm.com>; Michael Santana
> <maicolgabriel@hotmail.com>; Bruce Richardson
> <bruce.richardson@intel.com>; dev <dev@dpdk.org>; Gavin Hu
> <Gavin.Hu@arm.com>; Honnappa Nagarahalli
> <Honnappa.Nagarahalli@arm.com>; nd <nd@arm.com>
> Subject: Re: [PATCH v1] ci: add test suite run without hugepage
> 
> David Marchand <david.marchand@redhat.com> writes:
> 
> > On Tue, Feb 25, 2020 at 8:33 AM Ruifeng Wang <ruifeng.wang@arm.com>
> wrote:
> >>
> >> This test suite is derived from fast-tests suite. Cases in this suite
> >> are run with '--no-huge' flag.
> >>
> >> The suite aims to cover as many as possible test cases out of the
> >> fast-tests suites in the environments without huge pages support,
> >> like containers.
> >>
> >> Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> >> Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> >
> > Compilation time is what makes the most of a "tests" job in Travis.
> > So I'd prefer we avoid adding more jobs for x86_64 (see below).
> >
> >
> >> ---
> >>  .ci/linux-build.sh   |  4 +++
> >>  .travis.yml          | 12 +++++++
> >>  app/test/meson.build | 75
> >> ++++++++++++++++++++++++++++++++++++++++++++
> >>  3 files changed, 91 insertions(+)
> >>
> >> diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh index
> >> d500c4c00..39515d915 100755
> >> --- a/.ci/linux-build.sh
> >> +++ b/.ci/linux-build.sh
> >> @@ -92,3 +92,7 @@ fi
> >>  if [ "$RUN_TESTS" = "1" ]; then
> >>      sudo meson test -C build --suite fast-tests -t 3  fi
> >> +
> >> +if [ "$RUN_TESTS_NO_HUGE" = "1" ]; then
> >> +    sudo meson test -C build --suite nohuge-tests -t 3 fi
> >
> > You can replace the "boolean" RUN_TESTS with a TESTSUITES variable
> > that contains a list of testsuites.
> > Then this part becomes:
> >
> > for testsuite in ${TESTSUITES:-}; do
> >    sudo meson test -C build --suite $testsuite -t 3 done
> >
> > (I wonder if we reaaaally need to be root to run those tests w/ and
> > w/o hugepages, Aaron?)
> 
> Last I tested, we didn't need root access when not using hugepages (at least
> for core library functionality).  It's possible that the test suites for PMDs might
> need that access, but we don't run them in the travis environment.
> 
> >
> >> diff --git a/.travis.yml b/.travis.yml index b64a81bd0..0e07d52d0
> >> 100644
> >> --- a/.travis.yml
> >> +++ b/.travis.yml
> >> @@ -43,6 +43,9 @@ jobs:
> >>    - env: DEF_LIB="shared" RUN_TESTS=1
> >>      arch: amd64
> >>      compiler: gcc
> >> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
> >> +    arch: amd64
> >> +    compiler: gcc
> >
> > And then we only need to update the existing RUN_TESTS jobs for x86_64.
> 
> I like the suggestion.  But I guess we'd just make RUN_TESTS=1 set
> TESTSUITES="..." and otherwise, we can have an ARM64 test to do all the test
> suites.
It should be OK to do all applicable test suites in a single job?

> 
> Actually, the ideal would be for the tests to automatically SKIP when no
> hugepages are enabled.
In this way, case will run either w/ hugepage or w/o hugepage but not both.
It will run faster, but with less coverage?

> 
> >
> >>    - env: DEF_LIB="shared" BUILD_DOCS=1
> >>      arch: amd64
> >>      compiler: gcc
> >> @@ -66,6 +69,9 @@ jobs:
> >>    - env: DEF_LIB="shared" RUN_TESTS=1
> >>      arch: amd64
> >>      compiler: clang
> >> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
> >> +    arch: amd64
> >> +    compiler: clang
> >>    - env: DEF_LIB="shared" BUILD_DOCS=1
> >>      arch: amd64
> >>      compiler: clang
> >> @@ -101,6 +107,9 @@ jobs:
> >>    - env: DEF_LIB="static"
> >>      arch: arm64
> >>      compiler: gcc
> >> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
> >> +    arch: arm64
> >> +    compiler: gcc
> >>    - env: DEF_LIB="shared" BUILD_DOCS=1
> >>      arch: arm64
> >>      compiler: gcc
> >> @@ -124,3 +133,6 @@ jobs:
> >>    - env: DEF_LIB="shared"
> >>      arch: arm64
> >>      compiler: clang
> >> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
> >> +    arch: arm64
> >> +    compiler: clang
> >
> >
> > --
> > David Marchand

Re: [dpdk-dev] [PATCH v1] ci: add test suite run without hugepage

[Aaron Conole]

Ruifeng Wang <Ruifeng.Wang@arm.com> writes:

>> -----Original Message-----
>> From: Aaron Conole <aconole@redhat.com>
>> Sent: Tuesday, February 25, 2020 22:36
>> To: David Marchand <david.marchand@redhat.com>
>> Cc: Ruifeng Wang <Ruifeng.Wang@arm.com>; Michael Santana
>> <maicolgabriel@hotmail.com>; Bruce Richardson
>> <bruce.richardson@intel.com>; dev <dev@dpdk.org>; Gavin Hu
>> <Gavin.Hu@arm.com>; Honnappa Nagarahalli
>> <Honnappa.Nagarahalli@arm.com>; nd <nd@arm.com>
>> Subject: Re: [PATCH v1] ci: add test suite run without hugepage
>> 
>> David Marchand <david.marchand@redhat.com> writes:
>> 
>> > On Tue, Feb 25, 2020 at 8:33 AM Ruifeng Wang <ruifeng.wang@arm.com>
>> wrote:
>> >>
>> >> This test suite is derived from fast-tests suite. Cases in this suite
>> >> are run with '--no-huge' flag.
>> >>
>> >> The suite aims to cover as many as possible test cases out of the
>> >> fast-tests suites in the environments without huge pages support,
>> >> like containers.
>> >>
>> >> Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
>> >> Reviewed-by: Gavin Hu <gavin.hu@arm.com>
>> >
>> > Compilation time is what makes the most of a "tests" job in Travis.
>> > So I'd prefer we avoid adding more jobs for x86_64 (see below).
>> >
>> >
>> >> ---
>> >>  .ci/linux-build.sh   |  4 +++
>> >>  .travis.yml          | 12 +++++++
>> >>  app/test/meson.build | 75
>> >> ++++++++++++++++++++++++++++++++++++++++++++
>> >>  3 files changed, 91 insertions(+)
>> >>
>> >> diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh index
>> >> d500c4c00..39515d915 100755
>> >> --- a/.ci/linux-build.sh
>> >> +++ b/.ci/linux-build.sh
>> >> @@ -92,3 +92,7 @@ fi
>> >>  if [ "$RUN_TESTS" = "1" ]; then
>> >>      sudo meson test -C build --suite fast-tests -t 3  fi
>> >> +
>> >> +if [ "$RUN_TESTS_NO_HUGE" = "1" ]; then
>> >> +    sudo meson test -C build --suite nohuge-tests -t 3 fi
>> >
>> > You can replace the "boolean" RUN_TESTS with a TESTSUITES variable
>> > that contains a list of testsuites.
>> > Then this part becomes:
>> >
>> > for testsuite in ${TESTSUITES:-}; do
>> >    sudo meson test -C build --suite $testsuite -t 3 done
>> >
>> > (I wonder if we reaaaally need to be root to run those tests w/ and
>> > w/o hugepages, Aaron?)
>> 
>> Last I tested, we didn't need root access when not using hugepages (at least
>> for core library functionality).  It's possible that the test suites for PMDs might
>> need that access, but we don't run them in the travis environment.
>> 
>> >
>> >> diff --git a/.travis.yml b/.travis.yml index b64a81bd0..0e07d52d0
>> >> 100644
>> >> --- a/.travis.yml
>> >> +++ b/.travis.yml
>> >> @@ -43,6 +43,9 @@ jobs:
>> >>    - env: DEF_LIB="shared" RUN_TESTS=1
>> >>      arch: amd64
>> >>      compiler: gcc
>> >> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
>> >> +    arch: amd64
>> >> +    compiler: gcc
>> >
>> > And then we only need to update the existing RUN_TESTS jobs for x86_64.
>> 
>> I like the suggestion.  But I guess we'd just make RUN_TESTS=1 set
>> TESTSUITES="..." and otherwise, we can have an ARM64 test to do all the test
>> suites.
> It should be OK to do all applicable test suites in a single job?

Yes, I think so.

>> 
>> Actually, the ideal would be for the tests to automatically SKIP when no
>> hugepages are enabled.
> In this way, case will run either w/ hugepage or w/o hugepage but not
> both.

Hrrm?  I don't understand.  If there are hugepages configured, all the
tests will run.  If there are no hugepages, only the tests that can pass
without hugepages will run.

> It will run faster, but with less coverage?

Sortof - if there aren't hugepages configured for an environment
reason (like unsupported) why get failure reports?  Better to either
fail when we try to setup the hugepages, or skip when none are available
because they are two separate concerns.

>> 
>> >
>> >>    - env: DEF_LIB="shared" BUILD_DOCS=1
>> >>      arch: amd64
>> >>      compiler: gcc
>> >> @@ -66,6 +69,9 @@ jobs:
>> >>    - env: DEF_LIB="shared" RUN_TESTS=1
>> >>      arch: amd64
>> >>      compiler: clang
>> >> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
>> >> +    arch: amd64
>> >> +    compiler: clang
>> >>    - env: DEF_LIB="shared" BUILD_DOCS=1
>> >>      arch: amd64
>> >>      compiler: clang
>> >> @@ -101,6 +107,9 @@ jobs:
>> >>    - env: DEF_LIB="static"
>> >>      arch: arm64
>> >>      compiler: gcc
>> >> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
>> >> +    arch: arm64
>> >> +    compiler: gcc
>> >>    - env: DEF_LIB="shared" BUILD_DOCS=1
>> >>      arch: arm64
>> >>      compiler: gcc
>> >> @@ -124,3 +133,6 @@ jobs:
>> >>    - env: DEF_LIB="shared"
>> >>      arch: arm64
>> >>      compiler: clang
>> >> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
>> >> +    arch: arm64
>> >> +    compiler: clang
>> >
>> >
>> > --
>> > David Marchand

Re: [dpdk-dev] [PATCH v1] ci: add test suite run without hugepage

[Ruifeng Wang]

> -----Original Message-----
> From: Aaron Conole <aconole@redhat.com>
> Sent: Tuesday, February 25, 2020 23:57
> To: Ruifeng Wang <Ruifeng.Wang@arm.com>
> Cc: David Marchand <david.marchand@redhat.com>; Michael Santana
> <maicolgabriel@hotmail.com>; Bruce Richardson
> <bruce.richardson@intel.com>; dev <dev@dpdk.org>; Gavin Hu
> <Gavin.Hu@arm.com>; Honnappa Nagarahalli
> <Honnappa.Nagarahalli@arm.com>; nd <nd@arm.com>
> Subject: Re: [PATCH v1] ci: add test suite run without hugepage
> 
> Ruifeng Wang <Ruifeng.Wang@arm.com> writes:
> 
> >> -----Original Message-----
> >> From: Aaron Conole <aconole@redhat.com>
> >> Sent: Tuesday, February 25, 2020 22:36
> >> To: David Marchand <david.marchand@redhat.com>
> >> Cc: Ruifeng Wang <Ruifeng.Wang@arm.com>; Michael Santana
> >> <maicolgabriel@hotmail.com>; Bruce Richardson
> >> <bruce.richardson@intel.com>; dev <dev@dpdk.org>; Gavin Hu
> >> <Gavin.Hu@arm.com>; Honnappa Nagarahalli
> >> <Honnappa.Nagarahalli@arm.com>; nd <nd@arm.com>
> >> Subject: Re: [PATCH v1] ci: add test suite run without hugepage
> >>
> >> David Marchand <david.marchand@redhat.com> writes:
> >>
> >> > On Tue, Feb 25, 2020 at 8:33 AM Ruifeng Wang
> <ruifeng.wang@arm.com>
> >> wrote:
> >> >>
> >> >> This test suite is derived from fast-tests suite. Cases in this
> >> >> suite are run with '--no-huge' flag.
> >> >>
> >> >> The suite aims to cover as many as possible test cases out of the
> >> >> fast-tests suites in the environments without huge pages support,
> >> >> like containers.
> >> >>
> >> >> Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> >> >> Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> >> >
> >> > Compilation time is what makes the most of a "tests" job in Travis.
> >> > So I'd prefer we avoid adding more jobs for x86_64 (see below).
> >> >
> >> >
> >> >> ---
> >> >>  .ci/linux-build.sh   |  4 +++
> >> >>  .travis.yml          | 12 +++++++
> >> >>  app/test/meson.build | 75
> >> >> ++++++++++++++++++++++++++++++++++++++++++++
> >> >>  3 files changed, 91 insertions(+)
> >> >>
> >> >> diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh index
> >> >> d500c4c00..39515d915 100755
> >> >> --- a/.ci/linux-build.sh
> >> >> +++ b/.ci/linux-build.sh
> >> >> @@ -92,3 +92,7 @@ fi
> >> >>  if [ "$RUN_TESTS" = "1" ]; then
> >> >>      sudo meson test -C build --suite fast-tests -t 3  fi
> >> >> +
> >> >> +if [ "$RUN_TESTS_NO_HUGE" = "1" ]; then
> >> >> +    sudo meson test -C build --suite nohuge-tests -t 3 fi
> >> >
> >> > You can replace the "boolean" RUN_TESTS with a TESTSUITES variable
> >> > that contains a list of testsuites.
> >> > Then this part becomes:
> >> >
> >> > for testsuite in ${TESTSUITES:-}; do
> >> >    sudo meson test -C build --suite $testsuite -t 3 done
> >> >
> >> > (I wonder if we reaaaally need to be root to run those tests w/ and
> >> > w/o hugepages, Aaron?)
> >>
> >> Last I tested, we didn't need root access when not using hugepages
> >> (at least for core library functionality).  It's possible that the
> >> test suites for PMDs might need that access, but we don't run them in the
> travis environment.
> >>
> >> >
> >> >> diff --git a/.travis.yml b/.travis.yml index b64a81bd0..0e07d52d0
> >> >> 100644
> >> >> --- a/.travis.yml
> >> >> +++ b/.travis.yml
> >> >> @@ -43,6 +43,9 @@ jobs:
> >> >>    - env: DEF_LIB="shared" RUN_TESTS=1
> >> >>      arch: amd64
> >> >>      compiler: gcc
> >> >> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
> >> >> +    arch: amd64
> >> >> +    compiler: gcc
> >> >
> >> > And then we only need to update the existing RUN_TESTS jobs for
> x86_64.
> >>
> >> I like the suggestion.  But I guess we'd just make RUN_TESTS=1 set
> >> TESTSUITES="..." and otherwise, we can have an ARM64 test to do all
> >> the test suites.
> > It should be OK to do all applicable test suites in a single job?
> 
> Yes, I think so.
> 
> >>
> >> Actually, the ideal would be for the tests to automatically SKIP when
> >> no hugepages are enabled.
> > In this way, case will run either w/ hugepage or w/o hugepage but not
> > both.
> 
> Hrrm?  I don't understand.  If there are hugepages configured, all the tests
> will run.  If there are no hugepages, only the tests that can pass without
> hugepages will run.
> 
I was wondering whether it makes sense to do tests with "--no-huge" in environment where hugepages are enabled.

> > It will run faster, but with less coverage?
> 
> Sortof - if there aren't hugepages configured for an environment reason (like
> unsupported) why get failure reports?  Better to either fail when we try to
> setup the hugepages, or skip when none are available because they are two
> separate concerns.
> 
> >>
> >> >
> >> >>    - env: DEF_LIB="shared" BUILD_DOCS=1
> >> >>      arch: amd64
> >> >>      compiler: gcc
> >> >> @@ -66,6 +69,9 @@ jobs:
> >> >>    - env: DEF_LIB="shared" RUN_TESTS=1
> >> >>      arch: amd64
> >> >>      compiler: clang
> >> >> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
> >> >> +    arch: amd64
> >> >> +    compiler: clang
> >> >>    - env: DEF_LIB="shared" BUILD_DOCS=1
> >> >>      arch: amd64
> >> >>      compiler: clang
> >> >> @@ -101,6 +107,9 @@ jobs:
> >> >>    - env: DEF_LIB="static"
> >> >>      arch: arm64
> >> >>      compiler: gcc
> >> >> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
> >> >> +    arch: arm64
> >> >> +    compiler: gcc
> >> >>    - env: DEF_LIB="shared" BUILD_DOCS=1
> >> >>      arch: arm64
> >> >>      compiler: gcc
> >> >> @@ -124,3 +133,6 @@ jobs:
> >> >>    - env: DEF_LIB="shared"
> >> >>      arch: arm64
> >> >>      compiler: clang
> >> >> +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
> >> >> +    arch: arm64
> >> >> +    compiler: clang
> >> >
> >> >
> >> > --
> >> > David Marchand

Re: [dpdk-dev] [PATCH v1] ci: add test suite run without hugepage

[Ruifeng Wang]

> -----Original Message-----
> From: David Marchand <david.marchand@redhat.com>
> Sent: Tuesday, February 25, 2020 17:15
> To: Ruifeng Wang <Ruifeng.Wang@arm.com>
> Cc: Aaron Conole <aconole@redhat.com>; Michael Santana
> <maicolgabriel@hotmail.com>; Bruce Richardson
> <bruce.richardson@intel.com>; dev <dev@dpdk.org>; Gavin Hu
> <Gavin.Hu@arm.com>; Honnappa Nagarahalli
> <Honnappa.Nagarahalli@arm.com>; nd <nd@arm.com>
> Subject: Re: [PATCH v1] ci: add test suite run without hugepage
> 
> On Tue, Feb 25, 2020 at 8:33 AM Ruifeng Wang <ruifeng.wang@arm.com>
> wrote:
> >
> > This test suite is derived from fast-tests suite. Cases in this suite
> > are run with '--no-huge' flag.
> >
> > The suite aims to cover as many as possible test cases out of the
> > fast-tests suites in the environments without huge pages support, like
> > containers.
> >
> > Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> > Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> 
> Compilation time is what makes the most of a "tests" job in Travis.
> So I'd prefer we avoid adding more jobs for x86_64 (see below).
> 
> 
> > ---
> >  .ci/linux-build.sh   |  4 +++
> >  .travis.yml          | 12 +++++++
> >  app/test/meson.build | 75
> > ++++++++++++++++++++++++++++++++++++++++++++
> >  3 files changed, 91 insertions(+)
> >
> > diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh index
> > d500c4c00..39515d915 100755
> > --- a/.ci/linux-build.sh
> > +++ b/.ci/linux-build.sh
> > @@ -92,3 +92,7 @@ fi
> >  if [ "$RUN_TESTS" = "1" ]; then
> >      sudo meson test -C build --suite fast-tests -t 3  fi
> > +
> > +if [ "$RUN_TESTS_NO_HUGE" = "1" ]; then
> > +    sudo meson test -C build --suite nohuge-tests -t 3 fi
> 
> You can replace the "boolean" RUN_TESTS with a TESTSUITES variable that
> contains a list of testsuites.
> Then this part becomes:
> 
> for testsuite in ${TESTSUITES:-}; do
>    sudo meson test -C build --suite $testsuite -t 3 done
> 
Thanks for the suggestion. 
A good way to consolidate test jobs.
I will follow this way in new version.

> (I wonder if we reaaaally need to be root to run those tests w/ and w/o
> hugepages, Aaron?)
> 
> 
> > diff --git a/.travis.yml b/.travis.yml index b64a81bd0..0e07d52d0
> > 100644
> > --- a/.travis.yml
> > +++ b/.travis.yml
> > @@ -43,6 +43,9 @@ jobs:
> >    - env: DEF_LIB="shared" RUN_TESTS=1
> >      arch: amd64
> >      compiler: gcc
> > +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
> > +    arch: amd64
> > +    compiler: gcc
> 
> And then we only need to update the existing RUN_TESTS jobs for x86_64.
> 
> 
> >    - env: DEF_LIB="shared" BUILD_DOCS=1
> >      arch: amd64
> >      compiler: gcc
> > @@ -66,6 +69,9 @@ jobs:
> >    - env: DEF_LIB="shared" RUN_TESTS=1
> >      arch: amd64
> >      compiler: clang
> > +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
> > +    arch: amd64
> > +    compiler: clang
> >    - env: DEF_LIB="shared" BUILD_DOCS=1
> >      arch: amd64
> >      compiler: clang
> > @@ -101,6 +107,9 @@ jobs:
> >    - env: DEF_LIB="static"
> >      arch: arm64
> >      compiler: gcc
> > +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
> > +    arch: arm64
> > +    compiler: gcc
> >    - env: DEF_LIB="shared" BUILD_DOCS=1
> >      arch: arm64
> >      compiler: gcc
> > @@ -124,3 +133,6 @@ jobs:
> >    - env: DEF_LIB="shared"
> >      arch: arm64
> >      compiler: clang
> > +  - env: DEF_LIB="shared" RUN_TESTS_NO_HUGE=1
> > +    arch: arm64
> > +    compiler: clang
> 
> 
> --
> David Marchand

[dpdk-dev] [PATCH v2 0/2] no-huge test suite

[Ruifeng Wang]

For environments (such as containers) where hugetlbfs are not available,
some unit tests can be run with 'no-huge' option.

A suite is added. It includes cases that can run without hugetlbfs support.

To avoid Travis CI job bloating, multiple test suites are allowed to run
in one job.


v2:
Add a patch to enable running multiple suites in a job. (David)

Ruifeng Wang (2):
  ci: allow multiple test suites to run in one job
  ci: add test suite run without hugepage

 .ci/linux-build.sh   |  4 ++-
 .ci/linux-setup.sh   | 16 ++++++----
 .travis.yml          | 10 +++++--
 app/test/meson.build | 71 ++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 93 insertions(+), 8 deletions(-)

-- 
2.17.1

[dpdk-dev] [PATCH v2 1/2] ci: allow multiple test suites to run in one job

[Ruifeng Wang]

Run multiple test suites in a single Travis CI job helps to avoid
job bloating and save build time.

When RUN_TESTS is set, TEST_SUITES includes a list of suites that
need to run in the job, otherwise fast-tests is run by default.

Suggested-by: David Marchand <david.marchand@redhat.com>
Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>
---
 .ci/linux-build.sh |  4 +++-
 .ci/linux-setup.sh | 16 +++++++++++-----
 2 files changed, 14 insertions(+), 6 deletions(-)

diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh
index d500c4c00..ddb925418 100755
--- a/.ci/linux-build.sh
+++ b/.ci/linux-build.sh
@@ -90,5 +90,7 @@ if [ "$ABI_CHECKS" = "1" ]; then
 fi
 
 if [ "$RUN_TESTS" = "1" ]; then
-    sudo meson test -C build --suite fast-tests -t 3
+    for testsuite in ${TEST_SUITES:-fast-tests}; do
+        sudo meson test -C build --suite $testsuite -t 3
+    done
 fi
diff --git a/.ci/linux-setup.sh b/.ci/linux-setup.sh
index 2b07d1e0b..4c5ddeb96 100755
--- a/.ci/linux-setup.sh
+++ b/.ci/linux-setup.sh
@@ -3,10 +3,16 @@
 # need to install as 'root' since some of the unit tests won't run without it
 sudo python3 -m pip install --upgrade 'meson==0.47.1'
 
-# skip hugepage settings if tests will not run
+# set up hugepage if fast-tests suite (default) will run
 if [ "$RUN_TESTS" = "1" ]; then
-    # setup hugepages
-    cat /proc/meminfo
-    sudo sh -c 'echo 1024 > /proc/sys/vm/nr_hugepages'
-    cat /proc/meminfo
+    for testsuite in ${TEST_SUITES:-fast-tests}; do
+        if [ "$testsuite" = "fast-tests" ]; then
+            # setup hugepages
+            cat /proc/meminfo
+            sudo sh -c 'echo 1024 > /proc/sys/vm/nr_hugepages'
+            cat /proc/meminfo
+
+            break
+        fi
+    done
 fi
-- 
2.17.1

[dpdk-dev] [PATCH v2 2/2] ci: add test suite run without hugepage

[Ruifeng Wang]

This test suite is derived from fast-tests suite. Cases in this
suite are run with '--no-huge' flag.

The suite aims to cover as many as possible test cases out of the
fast-tests suites in the environments without huge pages support,
like containers.

Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>
---
 .travis.yml          | 10 +++++--
 app/test/meson.build | 71 ++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 79 insertions(+), 2 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index b64a81bd0..eed1d96db 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -40,7 +40,7 @@ jobs:
   - env: DEF_LIB="static"
     arch: amd64
     compiler: gcc
-  - env: DEF_LIB="shared" RUN_TESTS=1
+  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="fast-tests nohuge-tests"
     arch: amd64
     compiler: gcc
   - env: DEF_LIB="shared" BUILD_DOCS=1
@@ -63,7 +63,7 @@ jobs:
   - env: DEF_LIB="static"
     arch: amd64
     compiler: clang
-  - env: DEF_LIB="shared" RUN_TESTS=1
+  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="fast-tests nohuge-tests"
     arch: amd64
     compiler: clang
   - env: DEF_LIB="shared" BUILD_DOCS=1
@@ -101,6 +101,9 @@ jobs:
   - env: DEF_LIB="static"
     arch: arm64
     compiler: gcc
+  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="nohuge-tests"
+    arch: arm64
+    compiler: gcc
   - env: DEF_LIB="shared" BUILD_DOCS=1
     arch: arm64
     compiler: gcc
@@ -124,3 +127,6 @@ jobs:
   - env: DEF_LIB="shared"
     arch: arm64
     compiler: clang
+  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="nohuge-tests"
+    arch: arm64
+    compiler: clang
diff --git a/app/test/meson.build b/app/test/meson.build
index 0a2ce710f..162a1a76f 100644
--- a/app/test/meson.build
+++ b/app/test/meson.build
@@ -237,6 +237,60 @@ fast_test_names = [
         'thash_autotest',
 ]
 
+nohuge_test_names = [
+        'byteorder_autotest',
+        'cmdline_autotest',
+        'common_autotest',
+        'cpuflags_autotest',
+        'cycles_autotest',
+        'debug_autotest',
+        'eal_flags_n_opt_autotest',
+        'eal_flags_no_huge_autotest',
+        'eal_flags_vdev_opt_autotest',
+        'eal_fs_autotest',
+        'errno_autotest',
+        'event_ring_autotest',
+        'fib_autotest',
+        'fib6_autotest',
+        'interrupt_autotest',
+        'logs_autotest',
+        'lpm_autotest',
+        'lpm6_autotest',
+        'memcpy_autotest',
+        'meter_autotest',
+        'per_lcore_autotest',
+        'prefetch_autotest',
+        'rcu_qsbr_autotest',
+        'red_autotest',
+        'rib_autotest',
+        'rib6_autotest',
+        'ring_autotest',
+        'rwlock_rda_autotest',
+        'rwlock_rds_wrm_autotest',
+        'rwlock_rde_wro_autotest',
+        'sched_autotest',
+        'spinlock_autotest',
+        'string_autotest',
+        'tailq_autotest',
+        'user_delay_us',
+        'version_autotest',
+        'crc_autotest',
+        'delay_us_sleep_autotest',
+        'eventdev_common_autotest',
+        'fbarray_autotest',
+        'ipsec_autotest',
+        'kni_autotest',
+        'kvargs_autotest',
+        'member_autotest',
+        'metrics_autotest',
+        'power_cpufreq_autotest',
+        'power_autotest',
+        'power_kvm_vm_autotest',
+        'reorder_autotest',
+        'service_autotest',
+        'thash_autotest',
+]
+
 perf_test_names = [
         'ring_perf_autotest',
         'mempool_perf_autotest',
@@ -341,6 +395,10 @@ if dpdk_conf.has('RTE_LIBRTE_RING_PMD')
 	fast_test_names += 'latencystats_autotest'
 	driver_test_names += 'link_bonding_mode4_autotest'
 	fast_test_names += 'pdump_autotest'
+	nohuge_test_names += 'ring_pmd_autotest'
+	nohuge_test_names += 'bitratestats_autotest'
+	nohuge_test_names += 'latencystats_autotest'
+	nohuge_test_names += 'pdump_autotest'
 endif
 
 if dpdk_conf.has('RTE_LIBRTE_POWER')
@@ -430,6 +488,19 @@ foreach arg : fast_test_names
 	endif
 endforeach
 
+foreach arg : nohuge_test_names
+	if host_machine.system() == 'linux'
+		test(arg, dpdk_test,
+			  env : ['DPDK_TEST=' + arg],
+			  args : test_args +
+				 ['--no-huge'] + ['-m 1024'] +
+				 ['--file-prefix=@0@'.format(arg)],
+		timeout : timeout_seconds_fast,
+		is_parallel : false,
+		suite : 'nohuge-tests')
+	endif
+endforeach
+
 foreach arg : perf_test_names
 	test(arg, dpdk_test,
 	env : ['DPDK_TEST=' + arg],
-- 
2.17.1

Re: [dpdk-dev] [PATCH v2 2/2] ci: add test suite run without hugepage

[Aaron Conole]

Ruifeng Wang <ruifeng.wang@arm.com> writes:

> This test suite is derived from fast-tests suite. Cases in this
> suite are run with '--no-huge' flag.
>
> The suite aims to cover as many as possible test cases out of the
> fast-tests suites in the environments without huge pages support,
> like containers.
>
> Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> ---

I like this much more.  Few comments.

>  .travis.yml          | 10 +++++--
>  app/test/meson.build | 71
>  ++++++++++++++++++++++++++++++++++++++++++++

You should update doc/guides/prog_guide/meson_ut.rst to include some
detail about the new tests suite.

>  2 files changed, 79 insertions(+), 2 deletions(-)
>
> diff --git a/.travis.yml b/.travis.yml
> index b64a81bd0..eed1d96db 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -40,7 +40,7 @@ jobs:
>    - env: DEF_LIB="static"
>      arch: amd64
>      compiler: gcc
> -  - env: DEF_LIB="shared" RUN_TESTS=1
> +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="fast-tests nohuge-tests"
>      arch: amd64
>      compiler: gcc
>    - env: DEF_LIB="shared" BUILD_DOCS=1
> @@ -63,7 +63,7 @@ jobs:
>    - env: DEF_LIB="static"
>      arch: amd64
>      compiler: clang
> -  - env: DEF_LIB="shared" RUN_TESTS=1
> +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="fast-tests nohuge-tests"
>      arch: amd64
>      compiler: clang
>    - env: DEF_LIB="shared" BUILD_DOCS=1
> @@ -101,6 +101,9 @@ jobs:
>    - env: DEF_LIB="static"
>      arch: arm64
>      compiler: gcc
> +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="nohuge-tests"
> +    arch: arm64
> +    compiler: gcc
>    - env: DEF_LIB="shared" BUILD_DOCS=1
>      arch: arm64
>      compiler: gcc
> @@ -124,3 +127,6 @@ jobs:
>    - env: DEF_LIB="shared"
>      arch: arm64
>      compiler: clang
> +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="nohuge-tests"
> +    arch: arm64
> +    compiler: clang
> diff --git a/app/test/meson.build b/app/test/meson.build
> index 0a2ce710f..162a1a76f 100644
> --- a/app/test/meson.build
> +++ b/app/test/meson.build
> @@ -237,6 +237,60 @@ fast_test_names = [
>          'thash_autotest',
>  ]

Shouldn't we also trim the list of fast-tests?  Otherwise, these tests
will run twice.

ex: https://travis-ci.com/ovsrobot/dpdk/jobs/292037684

> +nohuge_test_names = [
> +        'byteorder_autotest',
> +        'cmdline_autotest',
> +        'common_autotest',
> +        'cpuflags_autotest',
> +        'cycles_autotest',
> +        'debug_autotest',
> +        'eal_flags_n_opt_autotest',
> +        'eal_flags_no_huge_autotest',
> +        'eal_flags_vdev_opt_autotest',
> +        'eal_fs_autotest',
> +        'errno_autotest',
> +        'event_ring_autotest',
> +        'fib_autotest',
> +        'fib6_autotest',
> +        'interrupt_autotest',
> +        'logs_autotest',
> +        'lpm_autotest',
> +        'lpm6_autotest',
> +        'memcpy_autotest',
> +        'meter_autotest',
> +        'per_lcore_autotest',
> +        'prefetch_autotest',
> +        'rcu_qsbr_autotest',
> +        'red_autotest',
> +        'rib_autotest',
> +        'rib6_autotest',
> +        'ring_autotest',
> +        'rwlock_rda_autotest',
> +        'rwlock_rds_wrm_autotest',
> +        'rwlock_rde_wro_autotest',
> +        'sched_autotest',
> +        'spinlock_autotest',
> +        'string_autotest',
> +        'tailq_autotest',
> +        'user_delay_us',
> +        'version_autotest',
> +        'crc_autotest',
> +        'delay_us_sleep_autotest',
> +        'eventdev_common_autotest',
> +        'fbarray_autotest',
> +        'ipsec_autotest',
> +        'kni_autotest',
> +        'kvargs_autotest',
> +        'member_autotest',
> +        'metrics_autotest',
> +        'power_cpufreq_autotest',
> +        'power_autotest',
> +        'power_kvm_vm_autotest',
> +        'reorder_autotest',
> +        'service_autotest',
> +        'thash_autotest',
> +]
> +
>  perf_test_names = [
>          'ring_perf_autotest',
>          'mempool_perf_autotest',
> @@ -341,6 +395,10 @@ if dpdk_conf.has('RTE_LIBRTE_RING_PMD')
>  	fast_test_names += 'latencystats_autotest'
>  	driver_test_names += 'link_bonding_mode4_autotest'
>  	fast_test_names += 'pdump_autotest'
> +	nohuge_test_names += 'ring_pmd_autotest'
> +	nohuge_test_names += 'bitratestats_autotest'
> +	nohuge_test_names += 'latencystats_autotest'
> +	nohuge_test_names += 'pdump_autotest'
>  endif
>  
>  if dpdk_conf.has('RTE_LIBRTE_POWER')
> @@ -430,6 +488,19 @@ foreach arg : fast_test_names
>  	endif
>  endforeach
>  
> +foreach arg : nohuge_test_names
> +	if host_machine.system() == 'linux'
> +		test(arg, dpdk_test,
> +			  env : ['DPDK_TEST=' + arg],
> +			  args : test_args +
> +				 ['--no-huge'] + ['-m 1024'] +
> +				 ['--file-prefix=@0@'.format(arg)],
> +		timeout : timeout_seconds_fast,
> +		is_parallel : false,
> +		suite : 'nohuge-tests')
> +	endif
> +endforeach
> +
>  foreach arg : perf_test_names
>  	test(arg, dpdk_test,
>  	env : ['DPDK_TEST=' + arg],

Re: [dpdk-dev] [PATCH v2 2/2] ci: add test suite run without hugepage

[Ruifeng Wang]

> -----Original Message-----
> From: Aaron Conole <aconole@redhat.com>
> Sent: Thursday, March 5, 2020 01:31
> To: Ruifeng Wang <Ruifeng.Wang@arm.com>
> Cc: maicolgabriel@hotmail.com; bruce.richardson@intel.com; dev@dpdk.org;
> david.marchand@redhat.com; Gavin Hu <Gavin.Hu@arm.com>; Honnappa
> Nagarahalli <Honnappa.Nagarahalli@arm.com>; juraj.linkes@pantheon.tech;
> nd <nd@arm.com>
> Subject: Re: [PATCH v2 2/2] ci: add test suite run without hugepage
> 
> Ruifeng Wang <ruifeng.wang@arm.com> writes:
> 
> > This test suite is derived from fast-tests suite. Cases in this suite
> > are run with '--no-huge' flag.
> >
> > The suite aims to cover as many as possible test cases out of the
> > fast-tests suites in the environments without huge pages support, like
> > containers.
> >
> > Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> > Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> > ---
> 
> I like this much more.  Few comments.
> 
> >  .travis.yml          | 10 +++++--
> >  app/test/meson.build | 71
> >  ++++++++++++++++++++++++++++++++++++++++++++
> 
> You should update doc/guides/prog_guide/meson_ut.rst to include some
> detail about the new tests suite.
> 
Thanks. Will update document in next version.

> >  2 files changed, 79 insertions(+), 2 deletions(-)
> >
> > diff --git a/.travis.yml b/.travis.yml index b64a81bd0..eed1d96db
> > 100644
> > --- a/.travis.yml
> > +++ b/.travis.yml
> > @@ -40,7 +40,7 @@ jobs:
> >    - env: DEF_LIB="static"
> >      arch: amd64
> >      compiler: gcc
> > -  - env: DEF_LIB="shared" RUN_TESTS=1
> > +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="fast-tests
> nohuge-tests"
> >      arch: amd64
> >      compiler: gcc
> >    - env: DEF_LIB="shared" BUILD_DOCS=1 @@ -63,7 +63,7 @@ jobs:
> >    - env: DEF_LIB="static"
> >      arch: amd64
> >      compiler: clang
> > -  - env: DEF_LIB="shared" RUN_TESTS=1
> > +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="fast-tests
> nohuge-tests"
> >      arch: amd64
> >      compiler: clang
> >    - env: DEF_LIB="shared" BUILD_DOCS=1 @@ -101,6 +101,9 @@ jobs:
> >    - env: DEF_LIB="static"
> >      arch: arm64
> >      compiler: gcc
> > +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="nohuge-tests"
> > +    arch: arm64
> > +    compiler: gcc
> >    - env: DEF_LIB="shared" BUILD_DOCS=1
> >      arch: arm64
> >      compiler: gcc
> > @@ -124,3 +127,6 @@ jobs:
> >    - env: DEF_LIB="shared"
> >      arch: arm64
> >      compiler: clang
> > +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="nohuge-tests"
> > +    arch: arm64
> > +    compiler: clang
> > diff --git a/app/test/meson.build b/app/test/meson.build index
> > 0a2ce710f..162a1a76f 100644
> > --- a/app/test/meson.build
> > +++ b/app/test/meson.build
> > @@ -237,6 +237,60 @@ fast_test_names = [
> >          'thash_autotest',
> >  ]
> 
> Shouldn't we also trim the list of fast-tests?  Otherwise, these tests will run
> twice.
> 
I think you mean to have exclusive lists for fast-tests and nohuge-tests.
Overlapped cases will run twice if both test suites are opted in.
But the two runs are not the same, one runs with hugepage and 
the other runs in no-huge mode.

If fast-tests list is splited,  we will need to always run multiple suites
to cover all fast tests.
We can keep x86 to run only fast-tests suite to avoid extra test runs if
they are not necessary. Thoughts?

> ex: https://travis-ci.com/ovsrobot/dpdk/jobs/292037684
> 
> > +nohuge_test_names = [
> > +        'byteorder_autotest',
> > +        'cmdline_autotest',
> > +        'common_autotest',
> > +        'cpuflags_autotest',
> > +        'cycles_autotest',
> > +        'debug_autotest',
> > +        'eal_flags_n_opt_autotest',
> > +        'eal_flags_no_huge_autotest',
> > +        'eal_flags_vdev_opt_autotest',
> > +        'eal_fs_autotest',
> > +        'errno_autotest',
> > +        'event_ring_autotest',
> > +        'fib_autotest',
> > +        'fib6_autotest',
> > +        'interrupt_autotest',
> > +        'logs_autotest',
> > +        'lpm_autotest',
> > +        'lpm6_autotest',
> > +        'memcpy_autotest',
> > +        'meter_autotest',
> > +        'per_lcore_autotest',
> > +        'prefetch_autotest',
> > +        'rcu_qsbr_autotest',
> > +        'red_autotest',
> > +        'rib_autotest',
> > +        'rib6_autotest',
> > +        'ring_autotest',
> > +        'rwlock_rda_autotest',
> > +        'rwlock_rds_wrm_autotest',
> > +        'rwlock_rde_wro_autotest',
> > +        'sched_autotest',
> > +        'spinlock_autotest',
> > +        'string_autotest',
> > +        'tailq_autotest',
> > +        'user_delay_us',
> > +        'version_autotest',
> > +        'crc_autotest',
> > +        'delay_us_sleep_autotest',
> > +        'eventdev_common_autotest',
> > +        'fbarray_autotest',
> > +        'ipsec_autotest',
> > +        'kni_autotest',
> > +        'kvargs_autotest',
> > +        'member_autotest',
> > +        'metrics_autotest',
> > +        'power_cpufreq_autotest',
> > +        'power_autotest',
> > +        'power_kvm_vm_autotest',
> > +        'reorder_autotest',
> > +        'service_autotest',
> > +        'thash_autotest',
> > +]
> > +
> >  perf_test_names = [
> >          'ring_perf_autotest',
> >          'mempool_perf_autotest',
> > @@ -341,6 +395,10 @@ if dpdk_conf.has('RTE_LIBRTE_RING_PMD')
> >  	fast_test_names += 'latencystats_autotest'
> >  	driver_test_names += 'link_bonding_mode4_autotest'
> >  	fast_test_names += 'pdump_autotest'
> > +	nohuge_test_names += 'ring_pmd_autotest'
> > +	nohuge_test_names += 'bitratestats_autotest'
> > +	nohuge_test_names += 'latencystats_autotest'
> > +	nohuge_test_names += 'pdump_autotest'
> >  endif
> >
> >  if dpdk_conf.has('RTE_LIBRTE_POWER')
> > @@ -430,6 +488,19 @@ foreach arg : fast_test_names
> >  	endif
> >  endforeach
> >
> > +foreach arg : nohuge_test_names
> > +	if host_machine.system() == 'linux'
> > +		test(arg, dpdk_test,
> > +			  env : ['DPDK_TEST=' + arg],
> > +			  args : test_args +
> > +				 ['--no-huge'] + ['-m 1024'] +
> > +				 ['--file-prefix=@0@'.format(arg)],
> > +		timeout : timeout_seconds_fast,
> > +		is_parallel : false,
> > +		suite : 'nohuge-tests')
> > +	endif
> > +endforeach
> > +
> >  foreach arg : perf_test_names
> >  	test(arg, dpdk_test,
> >  	env : ['DPDK_TEST=' + arg],

Re: [dpdk-dev] [PATCH v2 2/2] ci: add test suite run without hugepage

[Aaron Conole]

Ruifeng Wang <Ruifeng.Wang@arm.com> writes:

>> -----Original Message-----
>> From: Aaron Conole <aconole@redhat.com>
>> Sent: Thursday, March 5, 2020 01:31
>> To: Ruifeng Wang <Ruifeng.Wang@arm.com>
>> Cc: maicolgabriel@hotmail.com; bruce.richardson@intel.com; dev@dpdk.org;
>> david.marchand@redhat.com; Gavin Hu <Gavin.Hu@arm.com>; Honnappa
>> Nagarahalli <Honnappa.Nagarahalli@arm.com>; juraj.linkes@pantheon.tech;
>> nd <nd@arm.com>
>> Subject: Re: [PATCH v2 2/2] ci: add test suite run without hugepage
>> 
>> Ruifeng Wang <ruifeng.wang@arm.com> writes:
>> 
>> > This test suite is derived from fast-tests suite. Cases in this suite
>> > are run with '--no-huge' flag.
>> >
>> > The suite aims to cover as many as possible test cases out of the
>> > fast-tests suites in the environments without huge pages support, like
>> > containers.
>> >
>> > Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
>> > Reviewed-by: Gavin Hu <gavin.hu@arm.com>
>> > ---
>> 
>> I like this much more.  Few comments.
>> 
>> >  .travis.yml          | 10 +++++--
>> >  app/test/meson.build | 71
>> >  ++++++++++++++++++++++++++++++++++++++++++++
>> 
>> You should update doc/guides/prog_guide/meson_ut.rst to include some
>> detail about the new tests suite.
>> 
> Thanks. Will update document in next version.
>
>> >  2 files changed, 79 insertions(+), 2 deletions(-)
>> >
>> > diff --git a/.travis.yml b/.travis.yml index b64a81bd0..eed1d96db
>> > 100644
>> > --- a/.travis.yml
>> > +++ b/.travis.yml
>> > @@ -40,7 +40,7 @@ jobs:
>> >    - env: DEF_LIB="static"
>> >      arch: amd64
>> >      compiler: gcc
>> > -  - env: DEF_LIB="shared" RUN_TESTS=1
>> > +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="fast-tests
>> nohuge-tests"
>> >      arch: amd64
>> >      compiler: gcc
>> >    - env: DEF_LIB="shared" BUILD_DOCS=1 @@ -63,7 +63,7 @@ jobs:
>> >    - env: DEF_LIB="static"
>> >      arch: amd64
>> >      compiler: clang
>> > -  - env: DEF_LIB="shared" RUN_TESTS=1
>> > +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="fast-tests
>> nohuge-tests"
>> >      arch: amd64
>> >      compiler: clang
>> >    - env: DEF_LIB="shared" BUILD_DOCS=1 @@ -101,6 +101,9 @@ jobs:
>> >    - env: DEF_LIB="static"
>> >      arch: arm64
>> >      compiler: gcc
>> > +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="nohuge-tests"
>> > +    arch: arm64
>> > +    compiler: gcc
>> >    - env: DEF_LIB="shared" BUILD_DOCS=1
>> >      arch: arm64
>> >      compiler: gcc
>> > @@ -124,3 +127,6 @@ jobs:
>> >    - env: DEF_LIB="shared"
>> >      arch: arm64
>> >      compiler: clang
>> > +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="nohuge-tests"
>> > +    arch: arm64
>> > +    compiler: clang
>> > diff --git a/app/test/meson.build b/app/test/meson.build index
>> > 0a2ce710f..162a1a76f 100644
>> > --- a/app/test/meson.build
>> > +++ b/app/test/meson.build
>> > @@ -237,6 +237,60 @@ fast_test_names = [
>> >          'thash_autotest',
>> >  ]
>> 
>> Shouldn't we also trim the list of fast-tests?  Otherwise, these tests will run
>> twice.
>> 
> I think you mean to have exclusive lists for fast-tests and nohuge-tests.

That's what I was thinking.

> Overlapped cases will run twice if both test suites are opted in.
> But the two runs are not the same, one runs with hugepage and 
> the other runs in no-huge mode.

Is it really so different between huge and no-huge?  Most of the
libraries won't care - they call the rte_**alloc functions, and it
returns blocks of memory.  Maybe I am simplifying it too much.

> If fast-tests list is splited,  we will need to always run multiple suites
> to cover all fast tests.
> We can keep x86 to run only fast-tests suite to avoid extra test runs if
> they are not necessary. Thoughts?

I guess since most DPDK usage will be with hugepages, we should prefer
to test with it.  I don't care too much about the color of this
particular shed.  If you want to do it that way, it's okay by me - it
gives us the coverage, and doesn't duplicate tests between those
environments.

BUT it means when we add a new test to the suite, we need to remember to
add it in two places - fast_test and nohuge_test.  That almost
guarantees we will miss tests (because we accidentally don't add it to
nohuge).  Maybe there's another way, like keep a list of all the tests
and some information on whether the test needs hugepages to run.  Then
if there are no hugepages available, we can write that we SKIP the tests
that don't support huge pages.  In that way, we don't need two different
lists - and if there are hugepages, we will run all the test cases.
WDYT?

>> ex: https://travis-ci.com/ovsrobot/dpdk/jobs/292037684
>> 
>> > +nohuge_test_names = [
>> > +        'byteorder_autotest',
>> > +        'cmdline_autotest',
>> > +        'common_autotest',
>> > +        'cpuflags_autotest',
>> > +        'cycles_autotest',
>> > +        'debug_autotest',
>> > +        'eal_flags_n_opt_autotest',
>> > +        'eal_flags_no_huge_autotest',
>> > +        'eal_flags_vdev_opt_autotest',
>> > +        'eal_fs_autotest',
>> > +        'errno_autotest',
>> > +        'event_ring_autotest',
>> > +        'fib_autotest',
>> > +        'fib6_autotest',
>> > +        'interrupt_autotest',
>> > +        'logs_autotest',
>> > +        'lpm_autotest',
>> > +        'lpm6_autotest',
>> > +        'memcpy_autotest',
>> > +        'meter_autotest',
>> > +        'per_lcore_autotest',
>> > +        'prefetch_autotest',
>> > +        'rcu_qsbr_autotest',
>> > +        'red_autotest',
>> > +        'rib_autotest',
>> > +        'rib6_autotest',
>> > +        'ring_autotest',
>> > +        'rwlock_rda_autotest',
>> > +        'rwlock_rds_wrm_autotest',
>> > +        'rwlock_rde_wro_autotest',
>> > +        'sched_autotest',
>> > +        'spinlock_autotest',
>> > +        'string_autotest',
>> > +        'tailq_autotest',
>> > +        'user_delay_us',
>> > +        'version_autotest',
>> > +        'crc_autotest',
>> > +        'delay_us_sleep_autotest',
>> > +        'eventdev_common_autotest',
>> > +        'fbarray_autotest',
>> > +        'ipsec_autotest',
>> > +        'kni_autotest',
>> > +        'kvargs_autotest',
>> > +        'member_autotest',
>> > +        'metrics_autotest',
>> > +        'power_cpufreq_autotest',
>> > +        'power_autotest',
>> > +        'power_kvm_vm_autotest',
>> > +        'reorder_autotest',
>> > +        'service_autotest',
>> > +        'thash_autotest',
>> > +]
>> > +
>> >  perf_test_names = [
>> >          'ring_perf_autotest',
>> >          'mempool_perf_autotest',
>> > @@ -341,6 +395,10 @@ if dpdk_conf.has('RTE_LIBRTE_RING_PMD')
>> >  	fast_test_names += 'latencystats_autotest'
>> >  	driver_test_names += 'link_bonding_mode4_autotest'
>> >  	fast_test_names += 'pdump_autotest'
>> > +	nohuge_test_names += 'ring_pmd_autotest'
>> > +	nohuge_test_names += 'bitratestats_autotest'
>> > +	nohuge_test_names += 'latencystats_autotest'
>> > +	nohuge_test_names += 'pdump_autotest'
>> >  endif
>> >
>> >  if dpdk_conf.has('RTE_LIBRTE_POWER')
>> > @@ -430,6 +488,19 @@ foreach arg : fast_test_names
>> >  	endif
>> >  endforeach
>> >
>> > +foreach arg : nohuge_test_names
>> > +	if host_machine.system() == 'linux'
>> > +		test(arg, dpdk_test,
>> > +			  env : ['DPDK_TEST=' + arg],
>> > +			  args : test_args +
>> > +				 ['--no-huge'] + ['-m 1024'] +
>> > +				 ['--file-prefix=@0@'.format(arg)],
>> > +		timeout : timeout_seconds_fast,
>> > +		is_parallel : false,
>> > +		suite : 'nohuge-tests')
>> > +	endif
>> > +endforeach
>> > +
>> >  foreach arg : perf_test_names
>> >  	test(arg, dpdk_test,
>> >  	env : ['DPDK_TEST=' + arg],

Re: [dpdk-dev] [PATCH v2 2/2] ci: add test suite run without hugepage

[Ruifeng Wang]

> -----Original Message-----
> From: Aaron Conole <aconole@redhat.com>
> Sent: Thursday, March 5, 2020 22:37
> To: Ruifeng Wang <Ruifeng.Wang@arm.com>
> Cc: maicolgabriel@hotmail.com; bruce.richardson@intel.com; dev@dpdk.org;
> david.marchand@redhat.com; Gavin Hu <Gavin.Hu@arm.com>; Honnappa
> Nagarahalli <Honnappa.Nagarahalli@arm.com>; juraj.linkes@pantheon.tech;
> nd <nd@arm.com>
> Subject: Re: [PATCH v2 2/2] ci: add test suite run without hugepage
> 
> Ruifeng Wang <Ruifeng.Wang@arm.com> writes:
> 
> >> -----Original Message-----
> >> From: Aaron Conole <aconole@redhat.com>
> >> Sent: Thursday, March 5, 2020 01:31
> >> To: Ruifeng Wang <Ruifeng.Wang@arm.com>
> >> Cc: maicolgabriel@hotmail.com; bruce.richardson@intel.com;
> >> dev@dpdk.org; david.marchand@redhat.com; Gavin Hu
> <Gavin.Hu@arm.com>;
> >> Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>;
> >> juraj.linkes@pantheon.tech; nd <nd@arm.com>
> >> Subject: Re: [PATCH v2 2/2] ci: add test suite run without hugepage
> >>
> >> Ruifeng Wang <ruifeng.wang@arm.com> writes:
> >>
> >> > This test suite is derived from fast-tests suite. Cases in this
> >> > suite are run with '--no-huge' flag.
> >> >
> >> > The suite aims to cover as many as possible test cases out of the
> >> > fast-tests suites in the environments without huge pages support,
> >> > like containers.
> >> >
> >> > Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> >> > Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> >> > ---
> >>
> >> I like this much more.  Few comments.
> >>
> >> >  .travis.yml          | 10 +++++--
> >> >  app/test/meson.build | 71
> >> >  ++++++++++++++++++++++++++++++++++++++++++++
> >>
> >> You should update doc/guides/prog_guide/meson_ut.rst to include some
> >> detail about the new tests suite.
> >>
> > Thanks. Will update document in next version.
> >
> >> >  2 files changed, 79 insertions(+), 2 deletions(-)
> >> >
> >> > diff --git a/.travis.yml b/.travis.yml index b64a81bd0..eed1d96db
> >> > 100644
> >> > --- a/.travis.yml
> >> > +++ b/.travis.yml
> >> > @@ -40,7 +40,7 @@ jobs:
> >> >    - env: DEF_LIB="static"
> >> >      arch: amd64
> >> >      compiler: gcc
> >> > -  - env: DEF_LIB="shared" RUN_TESTS=1
> >> > +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="fast-tests
> >> nohuge-tests"
> >> >      arch: amd64
> >> >      compiler: gcc
> >> >    - env: DEF_LIB="shared" BUILD_DOCS=1 @@ -63,7 +63,7 @@ jobs:
> >> >    - env: DEF_LIB="static"
> >> >      arch: amd64
> >> >      compiler: clang
> >> > -  - env: DEF_LIB="shared" RUN_TESTS=1
> >> > +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="fast-tests
> >> nohuge-tests"
> >> >      arch: amd64
> >> >      compiler: clang
> >> >    - env: DEF_LIB="shared" BUILD_DOCS=1 @@ -101,6 +101,9 @@ jobs:
> >> >    - env: DEF_LIB="static"
> >> >      arch: arm64
> >> >      compiler: gcc
> >> > +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="nohuge-tests"
> >> > +    arch: arm64
> >> > +    compiler: gcc
> >> >    - env: DEF_LIB="shared" BUILD_DOCS=1
> >> >      arch: arm64
> >> >      compiler: gcc
> >> > @@ -124,3 +127,6 @@ jobs:
> >> >    - env: DEF_LIB="shared"
> >> >      arch: arm64
> >> >      compiler: clang
> >> > +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="nohuge-tests"
> >> > +    arch: arm64
> >> > +    compiler: clang
> >> > diff --git a/app/test/meson.build b/app/test/meson.build index
> >> > 0a2ce710f..162a1a76f 100644
> >> > --- a/app/test/meson.build
> >> > +++ b/app/test/meson.build
> >> > @@ -237,6 +237,60 @@ fast_test_names = [
> >> >          'thash_autotest',
> >> >  ]
> >>
> >> Shouldn't we also trim the list of fast-tests?  Otherwise, these
> >> tests will run twice.
> >>
> > I think you mean to have exclusive lists for fast-tests and nohuge-tests.
> 
> That's what I was thinking.
> 
> > Overlapped cases will run twice if both test suites are opted in.
> > But the two runs are not the same, one runs with hugepage and the
> > other runs in no-huge mode.
> 
> Is it really so different between huge and no-huge?  Most of the libraries
> won't care - they call the rte_**alloc functions, and it returns blocks of
> memory.  Maybe I am simplifying it too much.
> 
> > If fast-tests list is splited,  we will need to always run multiple
> > suites to cover all fast tests.
> > We can keep x86 to run only fast-tests suite to avoid extra test runs
> > if they are not necessary. Thoughts?
> 
> I guess since most DPDK usage will be with hugepages, we should prefer to
> test with it.  I don't care too much about the color of this particular shed.  If
> you want to do it that way, it's okay by me - it gives us the coverage, and
> doesn't duplicate tests between those environments.
> 
> BUT it means when we add a new test to the suite, we need to remember to
> add it in two places - fast_test and nohuge_test.  That almost guarantees we
> will miss tests (because we accidentally don't add it to nohuge).  Maybe
> there's another way, like keep a list of all the tests and some information on
> whether the test needs hugepages to run.  Then if there are no hugepages
> available, we can write that we SKIP the tests that don't support huge pages.
> In that way, we don't need two different lists - and if there are hugepages,
> we will run all the test cases.
> WDYT?
> 
Yes. Agree with you that having duplicate tests in suites is error prone.

IIUC, cases in a suite is determined at build time, as well as command options to run cases.
This implies hugepage availability needs to be detected at build time if we want to run only 
suitable cases in suite in an environment. It could be something we don't want. 

I'll trim fast-tests in next version to remove duplicated cases.
Thank you.

> >> ex: https://travis-ci.com/ovsrobot/dpdk/jobs/292037684
> >>
> >> > +nohuge_test_names = [
> >> > +        'byteorder_autotest',
> >> > +        'cmdline_autotest',
> >> > +        'common_autotest',
> >> > +        'cpuflags_autotest',
> >> > +        'cycles_autotest',
> >> > +        'debug_autotest',
> >> > +        'eal_flags_n_opt_autotest',
> >> > +        'eal_flags_no_huge_autotest',
> >> > +        'eal_flags_vdev_opt_autotest',
> >> > +        'eal_fs_autotest',
> >> > +        'errno_autotest',
> >> > +        'event_ring_autotest',
> >> > +        'fib_autotest',
> >> > +        'fib6_autotest',
> >> > +        'interrupt_autotest',
> >> > +        'logs_autotest',
> >> > +        'lpm_autotest',
> >> > +        'lpm6_autotest',
> >> > +        'memcpy_autotest',
> >> > +        'meter_autotest',
> >> > +        'per_lcore_autotest',
> >> > +        'prefetch_autotest',
> >> > +        'rcu_qsbr_autotest',
> >> > +        'red_autotest',
> >> > +        'rib_autotest',
> >> > +        'rib6_autotest',
> >> > +        'ring_autotest',
> >> > +        'rwlock_rda_autotest',
> >> > +        'rwlock_rds_wrm_autotest',
> >> > +        'rwlock_rde_wro_autotest',
> >> > +        'sched_autotest',
> >> > +        'spinlock_autotest',
> >> > +        'string_autotest',
> >> > +        'tailq_autotest',
> >> > +        'user_delay_us',
> >> > +        'version_autotest',
> >> > +        'crc_autotest',
> >> > +        'delay_us_sleep_autotest',
> >> > +        'eventdev_common_autotest',
> >> > +        'fbarray_autotest',
> >> > +        'ipsec_autotest',
> >> > +        'kni_autotest',
> >> > +        'kvargs_autotest',
> >> > +        'member_autotest',
> >> > +        'metrics_autotest',
> >> > +        'power_cpufreq_autotest',
> >> > +        'power_autotest',
> >> > +        'power_kvm_vm_autotest',
> >> > +        'reorder_autotest',
> >> > +        'service_autotest',
> >> > +        'thash_autotest',
> >> > +]
> >> > +
> >> >  perf_test_names = [
> >> >          'ring_perf_autotest',
> >> >          'mempool_perf_autotest',
> >> > @@ -341,6 +395,10 @@ if dpdk_conf.has('RTE_LIBRTE_RING_PMD')
> >> >  	fast_test_names += 'latencystats_autotest'
> >> >  	driver_test_names += 'link_bonding_mode4_autotest'
> >> >  	fast_test_names += 'pdump_autotest'
> >> > +	nohuge_test_names += 'ring_pmd_autotest'
> >> > +	nohuge_test_names += 'bitratestats_autotest'
> >> > +	nohuge_test_names += 'latencystats_autotest'
> >> > +	nohuge_test_names += 'pdump_autotest'
> >> >  endif
> >> >
> >> >  if dpdk_conf.has('RTE_LIBRTE_POWER') @@ -430,6 +488,19 @@
> foreach
> >> > arg : fast_test_names
> >> >  	endif
> >> >  endforeach
> >> >
> >> > +foreach arg : nohuge_test_names
> >> > +	if host_machine.system() == 'linux'
> >> > +		test(arg, dpdk_test,
> >> > +			  env : ['DPDK_TEST=' + arg],
> >> > +			  args : test_args +
> >> > +				 ['--no-huge'] + ['-m 1024'] +
> >> > +				 ['--file-prefix=@0@'.format(arg)],
> >> > +		timeout : timeout_seconds_fast,
> >> > +		is_parallel : false,
> >> > +		suite : 'nohuge-tests')
> >> > +	endif
> >> > +endforeach
> >> > +
> >> >  foreach arg : perf_test_names
> >> >  	test(arg, dpdk_test,
> >> >  	env : ['DPDK_TEST=' + arg],

Re: [dpdk-dev] [PATCH v2 2/2] ci: add test suite run without hugepage

[Aaron Conole]

Ruifeng Wang <Ruifeng.Wang@arm.com> writes:

>> -----Original Message-----
>> From: Aaron Conole <aconole@redhat.com>
>> Sent: Thursday, March 5, 2020 22:37
>> To: Ruifeng Wang <Ruifeng.Wang@arm.com>
>> Cc: maicolgabriel@hotmail.com; bruce.richardson@intel.com; dev@dpdk.org;
>> david.marchand@redhat.com; Gavin Hu <Gavin.Hu@arm.com>; Honnappa
>> Nagarahalli <Honnappa.Nagarahalli@arm.com>; juraj.linkes@pantheon.tech;
>> nd <nd@arm.com>
>> Subject: Re: [PATCH v2 2/2] ci: add test suite run without hugepage
>> 
>> Ruifeng Wang <Ruifeng.Wang@arm.com> writes:
>> 
>> >> -----Original Message-----
>> >> From: Aaron Conole <aconole@redhat.com>
>> >> Sent: Thursday, March 5, 2020 01:31
>> >> To: Ruifeng Wang <Ruifeng.Wang@arm.com>
>> >> Cc: maicolgabriel@hotmail.com; bruce.richardson@intel.com;
>> >> dev@dpdk.org; david.marchand@redhat.com; Gavin Hu
>> <Gavin.Hu@arm.com>;
>> >> Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>;
>> >> juraj.linkes@pantheon.tech; nd <nd@arm.com>
>> >> Subject: Re: [PATCH v2 2/2] ci: add test suite run without hugepage
>> >>
>> >> Ruifeng Wang <ruifeng.wang@arm.com> writes:
>> >>
>> >> > This test suite is derived from fast-tests suite. Cases in this
>> >> > suite are run with '--no-huge' flag.
>> >> >
>> >> > The suite aims to cover as many as possible test cases out of the
>> >> > fast-tests suites in the environments without huge pages support,
>> >> > like containers.
>> >> >
>> >> > Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
>> >> > Reviewed-by: Gavin Hu <gavin.hu@arm.com>
>> >> > ---
>> >>
>> >> I like this much more.  Few comments.
>> >>
>> >> >  .travis.yml          | 10 +++++--
>> >> >  app/test/meson.build | 71
>> >> >  ++++++++++++++++++++++++++++++++++++++++++++
>> >>
>> >> You should update doc/guides/prog_guide/meson_ut.rst to include some
>> >> detail about the new tests suite.
>> >>
>> > Thanks. Will update document in next version.
>> >
>> >> >  2 files changed, 79 insertions(+), 2 deletions(-)
>> >> >
>> >> > diff --git a/.travis.yml b/.travis.yml index b64a81bd0..eed1d96db
>> >> > 100644
>> >> > --- a/.travis.yml
>> >> > +++ b/.travis.yml
>> >> > @@ -40,7 +40,7 @@ jobs:
>> >> >    - env: DEF_LIB="static"
>> >> >      arch: amd64
>> >> >      compiler: gcc
>> >> > -  - env: DEF_LIB="shared" RUN_TESTS=1
>> >> > +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="fast-tests
>> >> nohuge-tests"
>> >> >      arch: amd64
>> >> >      compiler: gcc
>> >> >    - env: DEF_LIB="shared" BUILD_DOCS=1 @@ -63,7 +63,7 @@ jobs:
>> >> >    - env: DEF_LIB="static"
>> >> >      arch: amd64
>> >> >      compiler: clang
>> >> > -  - env: DEF_LIB="shared" RUN_TESTS=1
>> >> > +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="fast-tests
>> >> nohuge-tests"
>> >> >      arch: amd64
>> >> >      compiler: clang
>> >> >    - env: DEF_LIB="shared" BUILD_DOCS=1 @@ -101,6 +101,9 @@ jobs:
>> >> >    - env: DEF_LIB="static"
>> >> >      arch: arm64
>> >> >      compiler: gcc
>> >> > +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="nohuge-tests"
>> >> > +    arch: arm64
>> >> > +    compiler: gcc
>> >> >    - env: DEF_LIB="shared" BUILD_DOCS=1
>> >> >      arch: arm64
>> >> >      compiler: gcc
>> >> > @@ -124,3 +127,6 @@ jobs:
>> >> >    - env: DEF_LIB="shared"
>> >> >      arch: arm64
>> >> >      compiler: clang
>> >> > +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="nohuge-tests"
>> >> > +    arch: arm64
>> >> > +    compiler: clang
>> >> > diff --git a/app/test/meson.build b/app/test/meson.build index
>> >> > 0a2ce710f..162a1a76f 100644
>> >> > --- a/app/test/meson.build
>> >> > +++ b/app/test/meson.build
>> >> > @@ -237,6 +237,60 @@ fast_test_names = [
>> >> >          'thash_autotest',
>> >> >  ]
>> >>
>> >> Shouldn't we also trim the list of fast-tests?  Otherwise, these
>> >> tests will run twice.
>> >>
>> > I think you mean to have exclusive lists for fast-tests and nohuge-tests.
>> 
>> That's what I was thinking.
>> 
>> > Overlapped cases will run twice if both test suites are opted in.
>> > But the two runs are not the same, one runs with hugepage and the
>> > other runs in no-huge mode.
>> 
>> Is it really so different between huge and no-huge?  Most of the libraries
>> won't care - they call the rte_**alloc functions, and it returns blocks of
>> memory.  Maybe I am simplifying it too much.
>> 
>> > If fast-tests list is splited,  we will need to always run multiple
>> > suites to cover all fast tests.
>> > We can keep x86 to run only fast-tests suite to avoid extra test runs
>> > if they are not necessary. Thoughts?
>> 
>> I guess since most DPDK usage will be with hugepages, we should prefer to
>> test with it.  I don't care too much about the color of this particular shed.  If
>> you want to do it that way, it's okay by me - it gives us the coverage, and
>> doesn't duplicate tests between those environments.
>> 
>> BUT it means when we add a new test to the suite, we need to remember to
>> add it in two places - fast_test and nohuge_test.  That almost guarantees we
>> will miss tests (because we accidentally don't add it to nohuge).  Maybe
>> there's another way, like keep a list of all the tests and some information on
>> whether the test needs hugepages to run.  Then if there are no hugepages
>> available, we can write that we SKIP the tests that don't support huge pages.
>> In that way, we don't need two different lists - and if there are hugepages,
>> we will run all the test cases.
>> WDYT?
>> 
> Yes. Agree with you that having duplicate tests in suites is error prone.

Cool!

> IIUC, cases in a suite is determined at build time, as well as command options to run cases.
> This implies hugepage availability needs to be detected at build time if we want to run only 
> suitable cases in suite in an environment. It could be something we don't want. 
>
> I'll trim fast-tests in next version to remove duplicated cases.

I think it might be better to make the array something like (just a
psuedo-code example):

   # psuedo-code to check for hugepages
   has_hugepages = check_for_hugepages()

   ...

   fast_test_names = [
     ['acl_autotest', true],
     ['alarm_autotest', true],
     ['atomic_autotest', true],
     ...

Then in the code:

   foreach arg : fast_test_names
      ....
      if not arg[1]
         test(arg[0], ...)
      if has_hugepages and arg[1]
         test(arg[0], )

Does it make sense?  Do you see a problem?

> Thank you.
>
>> >> ex: https://travis-ci.com/ovsrobot/dpdk/jobs/292037684
>> >>
>> >> > +nohuge_test_names = [
>> >> > +        'byteorder_autotest',
>> >> > +        'cmdline_autotest',
>> >> > +        'common_autotest',
>> >> > +        'cpuflags_autotest',
>> >> > +        'cycles_autotest',
>> >> > +        'debug_autotest',
>> >> > +        'eal_flags_n_opt_autotest',
>> >> > +        'eal_flags_no_huge_autotest',
>> >> > +        'eal_flags_vdev_opt_autotest',
>> >> > +        'eal_fs_autotest',
>> >> > +        'errno_autotest',
>> >> > +        'event_ring_autotest',
>> >> > +        'fib_autotest',
>> >> > +        'fib6_autotest',
>> >> > +        'interrupt_autotest',
>> >> > +        'logs_autotest',
>> >> > +        'lpm_autotest',
>> >> > +        'lpm6_autotest',
>> >> > +        'memcpy_autotest',
>> >> > +        'meter_autotest',
>> >> > +        'per_lcore_autotest',
>> >> > +        'prefetch_autotest',
>> >> > +        'rcu_qsbr_autotest',
>> >> > +        'red_autotest',
>> >> > +        'rib_autotest',
>> >> > +        'rib6_autotest',
>> >> > +        'ring_autotest',
>> >> > +        'rwlock_rda_autotest',
>> >> > +        'rwlock_rds_wrm_autotest',
>> >> > +        'rwlock_rde_wro_autotest',
>> >> > +        'sched_autotest',
>> >> > +        'spinlock_autotest',
>> >> > +        'string_autotest',
>> >> > +        'tailq_autotest',
>> >> > +        'user_delay_us',
>> >> > +        'version_autotest',
>> >> > +        'crc_autotest',
>> >> > +        'delay_us_sleep_autotest',
>> >> > +        'eventdev_common_autotest',
>> >> > +        'fbarray_autotest',
>> >> > +        'ipsec_autotest',
>> >> > +        'kni_autotest',
>> >> > +        'kvargs_autotest',
>> >> > +        'member_autotest',
>> >> > +        'metrics_autotest',
>> >> > +        'power_cpufreq_autotest',
>> >> > +        'power_autotest',
>> >> > +        'power_kvm_vm_autotest',
>> >> > +        'reorder_autotest',
>> >> > +        'service_autotest',
>> >> > +        'thash_autotest',
>> >> > +]
>> >> > +
>> >> >  perf_test_names = [
>> >> >          'ring_perf_autotest',
>> >> >          'mempool_perf_autotest',
>> >> > @@ -341,6 +395,10 @@ if dpdk_conf.has('RTE_LIBRTE_RING_PMD')
>> >> >  	fast_test_names += 'latencystats_autotest'
>> >> >  	driver_test_names += 'link_bonding_mode4_autotest'
>> >> >  	fast_test_names += 'pdump_autotest'
>> >> > +	nohuge_test_names += 'ring_pmd_autotest'
>> >> > +	nohuge_test_names += 'bitratestats_autotest'
>> >> > +	nohuge_test_names += 'latencystats_autotest'
>> >> > +	nohuge_test_names += 'pdump_autotest'
>> >> >  endif
>> >> >
>> >> >  if dpdk_conf.has('RTE_LIBRTE_POWER') @@ -430,6 +488,19 @@
>> foreach
>> >> > arg : fast_test_names
>> >> >  	endif
>> >> >  endforeach
>> >> >
>> >> > +foreach arg : nohuge_test_names
>> >> > +	if host_machine.system() == 'linux'
>> >> > +		test(arg, dpdk_test,
>> >> > +			  env : ['DPDK_TEST=' + arg],
>> >> > +			  args : test_args +
>> >> > +				 ['--no-huge'] + ['-m 1024'] +
>> >> > +				 ['--file-prefix=@0@'.format(arg)],
>> >> > +		timeout : timeout_seconds_fast,
>> >> > +		is_parallel : false,
>> >> > +		suite : 'nohuge-tests')
>> >> > +	endif
>> >> > +endforeach
>> >> > +
>> >> >  foreach arg : perf_test_names
>> >> >  	test(arg, dpdk_test,
>> >> >  	env : ['DPDK_TEST=' + arg],

Re: [dpdk-dev] [PATCH v2 2/2] ci: add test suite run without hugepage

[David Marchand]

On Fri, Mar 6, 2020 at 4:57 PM Aaron Conole <aconole@redhat.com> wrote:
>
> Ruifeng Wang <Ruifeng.Wang@arm.com> writes:
>
> >> -----Original Message-----
> >> From: Aaron Conole <aconole@redhat.com>
> >> Sent: Thursday, March 5, 2020 22:37
> >> To: Ruifeng Wang <Ruifeng.Wang@arm.com>
> >> Cc: maicolgabriel@hotmail.com; bruce.richardson@intel.com; dev@dpdk.org;
> >> david.marchand@redhat.com; Gavin Hu <Gavin.Hu@arm.com>; Honnappa
> >> Nagarahalli <Honnappa.Nagarahalli@arm.com>; juraj.linkes@pantheon.tech;
> >> nd <nd@arm.com>
> >> Subject: Re: [PATCH v2 2/2] ci: add test suite run without hugepage
> >>
> >> Ruifeng Wang <Ruifeng.Wang@arm.com> writes:
> >>
> >> >> -----Original Message-----
> >> >> From: Aaron Conole <aconole@redhat.com>
> >> >> Sent: Thursday, March 5, 2020 01:31
> >> >> To: Ruifeng Wang <Ruifeng.Wang@arm.com>
> >> >> Cc: maicolgabriel@hotmail.com; bruce.richardson@intel.com;
> >> >> dev@dpdk.org; david.marchand@redhat.com; Gavin Hu
> >> <Gavin.Hu@arm.com>;
> >> >> Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>;
> >> >> juraj.linkes@pantheon.tech; nd <nd@arm.com>
> >> >> Subject: Re: [PATCH v2 2/2] ci: add test suite run without hugepage
> >> >>
> >> >> Ruifeng Wang <ruifeng.wang@arm.com> writes:
> >> >>
> >> >> > This test suite is derived from fast-tests suite. Cases in this
> >> >> > suite are run with '--no-huge' flag.
> >> >> >
> >> >> > The suite aims to cover as many as possible test cases out of the
> >> >> > fast-tests suites in the environments without huge pages support,
> >> >> > like containers.
> >> >> >
> >> >> > Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> >> >> > Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> >> >> > ---
> >> >>
> >> >> I like this much more.  Few comments.
> >> >>
> >> >> >  .travis.yml          | 10 +++++--
> >> >> >  app/test/meson.build | 71
> >> >> >  ++++++++++++++++++++++++++++++++++++++++++++
> >> >>
> >> >> You should update doc/guides/prog_guide/meson_ut.rst to include some
> >> >> detail about the new tests suite.
> >> >>
> >> > Thanks. Will update document in next version.
> >> >
> >> >> >  2 files changed, 79 insertions(+), 2 deletions(-)
> >> >> >
> >> >> > diff --git a/.travis.yml b/.travis.yml index b64a81bd0..eed1d96db
> >> >> > 100644
> >> >> > --- a/.travis.yml
> >> >> > +++ b/.travis.yml
> >> >> > @@ -40,7 +40,7 @@ jobs:
> >> >> >    - env: DEF_LIB="static"
> >> >> >      arch: amd64
> >> >> >      compiler: gcc
> >> >> > -  - env: DEF_LIB="shared" RUN_TESTS=1
> >> >> > +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="fast-tests
> >> >> nohuge-tests"
> >> >> >      arch: amd64
> >> >> >      compiler: gcc
> >> >> >    - env: DEF_LIB="shared" BUILD_DOCS=1 @@ -63,7 +63,7 @@ jobs:
> >> >> >    - env: DEF_LIB="static"
> >> >> >      arch: amd64
> >> >> >      compiler: clang
> >> >> > -  - env: DEF_LIB="shared" RUN_TESTS=1
> >> >> > +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="fast-tests
> >> >> nohuge-tests"
> >> >> >      arch: amd64
> >> >> >      compiler: clang
> >> >> >    - env: DEF_LIB="shared" BUILD_DOCS=1 @@ -101,6 +101,9 @@ jobs:
> >> >> >    - env: DEF_LIB="static"
> >> >> >      arch: arm64
> >> >> >      compiler: gcc
> >> >> > +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="nohuge-tests"
> >> >> > +    arch: arm64
> >> >> > +    compiler: gcc
> >> >> >    - env: DEF_LIB="shared" BUILD_DOCS=1
> >> >> >      arch: arm64
> >> >> >      compiler: gcc
> >> >> > @@ -124,3 +127,6 @@ jobs:
> >> >> >    - env: DEF_LIB="shared"
> >> >> >      arch: arm64
> >> >> >      compiler: clang
> >> >> > +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="nohuge-tests"
> >> >> > +    arch: arm64
> >> >> > +    compiler: clang
> >> >> > diff --git a/app/test/meson.build b/app/test/meson.build index
> >> >> > 0a2ce710f..162a1a76f 100644
> >> >> > --- a/app/test/meson.build
> >> >> > +++ b/app/test/meson.build
> >> >> > @@ -237,6 +237,60 @@ fast_test_names = [
> >> >> >          'thash_autotest',
> >> >> >  ]
> >> >>
> >> >> Shouldn't we also trim the list of fast-tests?  Otherwise, these
> >> >> tests will run twice.
> >> >>
> >> > I think you mean to have exclusive lists for fast-tests and nohuge-tests.
> >>
> >> That's what I was thinking.
> >>
> >> > Overlapped cases will run twice if both test suites are opted in.
> >> > But the two runs are not the same, one runs with hugepage and the
> >> > other runs in no-huge mode.
> >>
> >> Is it really so different between huge and no-huge?  Most of the libraries
> >> won't care - they call the rte_**alloc functions, and it returns blocks of
> >> memory.  Maybe I am simplifying it too much.
> >>
> >> > If fast-tests list is splited,  we will need to always run multiple
> >> > suites to cover all fast tests.
> >> > We can keep x86 to run only fast-tests suite to avoid extra test runs
> >> > if they are not necessary. Thoughts?
> >>
> >> I guess since most DPDK usage will be with hugepages, we should prefer to
> >> test with it.  I don't care too much about the color of this particular shed.  If
> >> you want to do it that way, it's okay by me - it gives us the coverage, and
> >> doesn't duplicate tests between those environments.
> >>
> >> BUT it means when we add a new test to the suite, we need to remember to
> >> add it in two places - fast_test and nohuge_test.  That almost guarantees we
> >> will miss tests (because we accidentally don't add it to nohuge).  Maybe
> >> there's another way, like keep a list of all the tests and some information on
> >> whether the test needs hugepages to run.  Then if there are no hugepages
> >> available, we can write that we SKIP the tests that don't support huge pages.
> >> In that way, we don't need two different lists - and if there are hugepages,
> >> we will run all the test cases.
> >> WDYT?
> >>
> > Yes. Agree with you that having duplicate tests in suites is error prone.
>
> Cool!
>
> > IIUC, cases in a suite is determined at build time, as well as command options to run cases.
> > This implies hugepage availability needs to be detected at build time if we want to run only
> > suitable cases in suite in an environment. It could be something we don't want.
> >
> > I'll trim fast-tests in next version to remove duplicated cases.
>
> I think it might be better to make the array something like (just a
> psuedo-code example):
>
>    # psuedo-code to check for hugepages
>    has_hugepages = check_for_hugepages()
>
>    ...
>
>    fast_test_names = [
>      ['acl_autotest', true],
>      ['alarm_autotest', true],
>      ['atomic_autotest', true],
>      ...
>
> Then in the code:
>
>    foreach arg : fast_test_names
>       ....
>       if not arg[1]
>          test(arg[0], ...)
>       if has_hugepages and arg[1]
>          test(arg[0], )
>
> Does it make sense?  Do you see a problem?

I just hope meson won't create a dependency on hugepage availability...
Thinking about stuff like:
https://git.dpdk.org/dpdk/commit/?id=599d67b6a4bf96096352cc5fbc8fc28e54a1ca62


-- 
David Marchand

Re: [dpdk-dev] [PATCH v2 2/2] ci: add test suite run without hugepage

[Aaron Conole]

David Marchand <david.marchand@redhat.com> writes:

> On Fri, Mar 6, 2020 at 4:57 PM Aaron Conole <aconole@redhat.com> wrote:
>>
>> Ruifeng Wang <Ruifeng.Wang@arm.com> writes:
>>
>> >> -----Original Message-----
>> >> From: Aaron Conole <aconole@redhat.com>
>> >> Sent: Thursday, March 5, 2020 22:37
>> >> To: Ruifeng Wang <Ruifeng.Wang@arm.com>
>> >> Cc: maicolgabriel@hotmail.com; bruce.richardson@intel.com; dev@dpdk.org;
>> >> david.marchand@redhat.com; Gavin Hu <Gavin.Hu@arm.com>; Honnappa
>> >> Nagarahalli <Honnappa.Nagarahalli@arm.com>; juraj.linkes@pantheon.tech;
>> >> nd <nd@arm.com>
>> >> Subject: Re: [PATCH v2 2/2] ci: add test suite run without hugepage
>> >>
>> >> Ruifeng Wang <Ruifeng.Wang@arm.com> writes:
>> >>
>> >> >> -----Original Message-----
>> >> >> From: Aaron Conole <aconole@redhat.com>
>> >> >> Sent: Thursday, March 5, 2020 01:31
>> >> >> To: Ruifeng Wang <Ruifeng.Wang@arm.com>
>> >> >> Cc: maicolgabriel@hotmail.com; bruce.richardson@intel.com;
>> >> >> dev@dpdk.org; david.marchand@redhat.com; Gavin Hu
>> >> <Gavin.Hu@arm.com>;
>> >> >> Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>;
>> >> >> juraj.linkes@pantheon.tech; nd <nd@arm.com>
>> >> >> Subject: Re: [PATCH v2 2/2] ci: add test suite run without hugepage
>> >> >>
>> >> >> Ruifeng Wang <ruifeng.wang@arm.com> writes:
>> >> >>
>> >> >> > This test suite is derived from fast-tests suite. Cases in this
>> >> >> > suite are run with '--no-huge' flag.
>> >> >> >
>> >> >> > The suite aims to cover as many as possible test cases out of the
>> >> >> > fast-tests suites in the environments without huge pages support,
>> >> >> > like containers.
>> >> >> >
>> >> >> > Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
>> >> >> > Reviewed-by: Gavin Hu <gavin.hu@arm.com>
>> >> >> > ---
>> >> >>
>> >> >> I like this much more.  Few comments.
>> >> >>
>> >> >> >  .travis.yml          | 10 +++++--
>> >> >> >  app/test/meson.build | 71
>> >> >> >  ++++++++++++++++++++++++++++++++++++++++++++
>> >> >>
>> >> >> You should update doc/guides/prog_guide/meson_ut.rst to include some
>> >> >> detail about the new tests suite.
>> >> >>
>> >> > Thanks. Will update document in next version.
>> >> >
>> >> >> >  2 files changed, 79 insertions(+), 2 deletions(-)
>> >> >> >
>> >> >> > diff --git a/.travis.yml b/.travis.yml index b64a81bd0..eed1d96db
>> >> >> > 100644
>> >> >> > --- a/.travis.yml
>> >> >> > +++ b/.travis.yml
>> >> >> > @@ -40,7 +40,7 @@ jobs:
>> >> >> >    - env: DEF_LIB="static"
>> >> >> >      arch: amd64
>> >> >> >      compiler: gcc
>> >> >> > -  - env: DEF_LIB="shared" RUN_TESTS=1
>> >> >> > +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="fast-tests
>> >> >> nohuge-tests"
>> >> >> >      arch: amd64
>> >> >> >      compiler: gcc
>> >> >> >    - env: DEF_LIB="shared" BUILD_DOCS=1 @@ -63,7 +63,7 @@ jobs:
>> >> >> >    - env: DEF_LIB="static"
>> >> >> >      arch: amd64
>> >> >> >      compiler: clang
>> >> >> > -  - env: DEF_LIB="shared" RUN_TESTS=1
>> >> >> > +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="fast-tests
>> >> >> nohuge-tests"
>> >> >> >      arch: amd64
>> >> >> >      compiler: clang
>> >> >> >    - env: DEF_LIB="shared" BUILD_DOCS=1 @@ -101,6 +101,9 @@ jobs:
>> >> >> >    - env: DEF_LIB="static"
>> >> >> >      arch: arm64
>> >> >> >      compiler: gcc
>> >> >> > +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="nohuge-tests"
>> >> >> > +    arch: arm64
>> >> >> > +    compiler: gcc
>> >> >> >    - env: DEF_LIB="shared" BUILD_DOCS=1
>> >> >> >      arch: arm64
>> >> >> >      compiler: gcc
>> >> >> > @@ -124,3 +127,6 @@ jobs:
>> >> >> >    - env: DEF_LIB="shared"
>> >> >> >      arch: arm64
>> >> >> >      compiler: clang
>> >> >> > +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="nohuge-tests"
>> >> >> > +    arch: arm64
>> >> >> > +    compiler: clang
>> >> >> > diff --git a/app/test/meson.build b/app/test/meson.build index
>> >> >> > 0a2ce710f..162a1a76f 100644
>> >> >> > --- a/app/test/meson.build
>> >> >> > +++ b/app/test/meson.build
>> >> >> > @@ -237,6 +237,60 @@ fast_test_names = [
>> >> >> >          'thash_autotest',
>> >> >> >  ]
>> >> >>
>> >> >> Shouldn't we also trim the list of fast-tests?  Otherwise, these
>> >> >> tests will run twice.
>> >> >>
>> >> > I think you mean to have exclusive lists for fast-tests and nohuge-tests.
>> >>
>> >> That's what I was thinking.
>> >>
>> >> > Overlapped cases will run twice if both test suites are opted in.
>> >> > But the two runs are not the same, one runs with hugepage and the
>> >> > other runs in no-huge mode.
>> >>
>> >> Is it really so different between huge and no-huge?  Most of the libraries
>> >> won't care - they call the rte_**alloc functions, and it returns blocks of
>> >> memory.  Maybe I am simplifying it too much.
>> >>
>> >> > If fast-tests list is splited,  we will need to always run multiple
>> >> > suites to cover all fast tests.
>> >> > We can keep x86 to run only fast-tests suite to avoid extra test runs
>> >> > if they are not necessary. Thoughts?
>> >>
>> >> I guess since most DPDK usage will be with hugepages, we should prefer to
>> >> test with it.  I don't care too much about the color of this particular shed.  If
>> >> you want to do it that way, it's okay by me - it gives us the coverage, and
>> >> doesn't duplicate tests between those environments.
>> >>
>> >> BUT it means when we add a new test to the suite, we need to remember to
>> >> add it in two places - fast_test and nohuge_test.  That almost guarantees we
>> >> will miss tests (because we accidentally don't add it to nohuge).  Maybe
>> >> there's another way, like keep a list of all the tests and some information on
>> >> whether the test needs hugepages to run.  Then if there are no hugepages
>> >> available, we can write that we SKIP the tests that don't support huge pages.
>> >> In that way, we don't need two different lists - and if there are hugepages,
>> >> we will run all the test cases.
>> >> WDYT?
>> >>
>> > Yes. Agree with you that having duplicate tests in suites is error prone.
>>
>> Cool!
>>
>> > IIUC, cases in a suite is determined at build time, as well as command options to run cases.
>> > This implies hugepage availability needs to be detected at build time if we want to run only
>> > suitable cases in suite in an environment. It could be something we don't want.
>> >
>> > I'll trim fast-tests in next version to remove duplicated cases.
>>
>> I think it might be better to make the array something like (just a
>> psuedo-code example):
>>
>>    # psuedo-code to check for hugepages
>>    has_hugepages = check_for_hugepages()
>>
>>    ...
>>
>>    fast_test_names = [
>>      ['acl_autotest', true],
>>      ['alarm_autotest', true],
>>      ['atomic_autotest', true],
>>      ...
>>
>> Then in the code:
>>
>>    foreach arg : fast_test_names
>>       ....
>>       if not arg[1]
>>          test(arg[0], ...)
>>       if has_hugepages and arg[1]
>>          test(arg[0], )
>>
>> Does it make sense?  Do you see a problem?
>
> I just hope meson won't create a dependency on hugepage availability...
> Thinking about stuff like:
> https://git.dpdk.org/dpdk/commit/?id=599d67b6a4bf96096352cc5fbc8fc28e54a1ca62

Yes, I agree.  We probably need to do some kind of similar abstraction.

Re: [dpdk-dev] [PATCH v2 2/2] ci: add test suite run without hugepage

[Bruce Richardson]

On Fri, Mar 06, 2020 at 11:16:24AM -0500, Aaron Conole wrote:
> David Marchand <david.marchand@redhat.com> writes:
> 
> > On Fri, Mar 6, 2020 at 4:57 PM Aaron Conole <aconole@redhat.com> wrote:
> >>
> >> Ruifeng Wang <Ruifeng.Wang@arm.com> writes:
> >>
> >> >> -----Original Message-----
> >> >> From: Aaron Conole <aconole@redhat.com>
> >> >> Sent: Thursday, March 5, 2020 22:37
> >> >> To: Ruifeng Wang <Ruifeng.Wang@arm.com>
> >> >> Cc: maicolgabriel@hotmail.com; bruce.richardson@intel.com; dev@dpdk.org;
> >> >> david.marchand@redhat.com; Gavin Hu <Gavin.Hu@arm.com>; Honnappa
> >> >> Nagarahalli <Honnappa.Nagarahalli@arm.com>; juraj.linkes@pantheon.tech;
> >> >> nd <nd@arm.com>
> >> >> Subject: Re: [PATCH v2 2/2] ci: add test suite run without hugepage
> >> >>
> >> >> Ruifeng Wang <Ruifeng.Wang@arm.com> writes:
> >> >>
> >> >> >> -----Original Message-----
> >> >> >> From: Aaron Conole <aconole@redhat.com>
> >> >> >> Sent: Thursday, March 5, 2020 01:31
> >> >> >> To: Ruifeng Wang <Ruifeng.Wang@arm.com>
> >> >> >> Cc: maicolgabriel@hotmail.com; bruce.richardson@intel.com;
> >> >> >> dev@dpdk.org; david.marchand@redhat.com; Gavin Hu
> >> >> <Gavin.Hu@arm.com>;
> >> >> >> Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>;
> >> >> >> juraj.linkes@pantheon.tech; nd <nd@arm.com>
> >> >> >> Subject: Re: [PATCH v2 2/2] ci: add test suite run without hugepage
> >> >> >>
> >> >> >> Ruifeng Wang <ruifeng.wang@arm.com> writes:
> >> >> >>
> >> >> >> > This test suite is derived from fast-tests suite. Cases in this
> >> >> >> > suite are run with '--no-huge' flag.
> >> >> >> >
> >> >> >> > The suite aims to cover as many as possible test cases out of the
> >> >> >> > fast-tests suites in the environments without huge pages support,
> >> >> >> > like containers.
> >> >> >> >
> >> >> >> > Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> >> >> >> > Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> >> >> >> > ---
> >> >> >>
> >> >> >> I like this much more.  Few comments.
> >> >> >>
> >> >> >> >  .travis.yml          | 10 +++++--
> >> >> >> >  app/test/meson.build | 71
> >> >> >> >  ++++++++++++++++++++++++++++++++++++++++++++
> >> >> >>
> >> >> >> You should update doc/guides/prog_guide/meson_ut.rst to include some
> >> >> >> detail about the new tests suite.
> >> >> >>
> >> >> > Thanks. Will update document in next version.
> >> >> >
> >> >> >> >  2 files changed, 79 insertions(+), 2 deletions(-)
> >> >> >> >
> >> >> >> > diff --git a/.travis.yml b/.travis.yml index b64a81bd0..eed1d96db
> >> >> >> > 100644
> >> >> >> > --- a/.travis.yml
> >> >> >> > +++ b/.travis.yml
> >> >> >> > @@ -40,7 +40,7 @@ jobs:
> >> >> >> >    - env: DEF_LIB="static"
> >> >> >> >      arch: amd64
> >> >> >> >      compiler: gcc
> >> >> >> > -  - env: DEF_LIB="shared" RUN_TESTS=1
> >> >> >> > +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="fast-tests
> >> >> >> nohuge-tests"
> >> >> >> >      arch: amd64
> >> >> >> >      compiler: gcc
> >> >> >> >    - env: DEF_LIB="shared" BUILD_DOCS=1 @@ -63,7 +63,7 @@ jobs:
> >> >> >> >    - env: DEF_LIB="static"
> >> >> >> >      arch: amd64
> >> >> >> >      compiler: clang
> >> >> >> > -  - env: DEF_LIB="shared" RUN_TESTS=1
> >> >> >> > +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="fast-tests
> >> >> >> nohuge-tests"
> >> >> >> >      arch: amd64
> >> >> >> >      compiler: clang
> >> >> >> >    - env: DEF_LIB="shared" BUILD_DOCS=1 @@ -101,6 +101,9 @@ jobs:
> >> >> >> >    - env: DEF_LIB="static"
> >> >> >> >      arch: arm64
> >> >> >> >      compiler: gcc
> >> >> >> > +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="nohuge-tests"
> >> >> >> > +    arch: arm64
> >> >> >> > +    compiler: gcc
> >> >> >> >    - env: DEF_LIB="shared" BUILD_DOCS=1
> >> >> >> >      arch: arm64
> >> >> >> >      compiler: gcc
> >> >> >> > @@ -124,3 +127,6 @@ jobs:
> >> >> >> >    - env: DEF_LIB="shared"
> >> >> >> >      arch: arm64
> >> >> >> >      compiler: clang
> >> >> >> > +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="nohuge-tests"
> >> >> >> > +    arch: arm64
> >> >> >> > +    compiler: clang
> >> >> >> > diff --git a/app/test/meson.build b/app/test/meson.build index
> >> >> >> > 0a2ce710f..162a1a76f 100644
> >> >> >> > --- a/app/test/meson.build
> >> >> >> > +++ b/app/test/meson.build
> >> >> >> > @@ -237,6 +237,60 @@ fast_test_names = [
> >> >> >> >          'thash_autotest',
> >> >> >> >  ]
> >> >> >>
> >> >> >> Shouldn't we also trim the list of fast-tests?  Otherwise, these
> >> >> >> tests will run twice.
> >> >> >>
> >> >> > I think you mean to have exclusive lists for fast-tests and nohuge-tests.
> >> >>
> >> >> That's what I was thinking.
> >> >>
> >> >> > Overlapped cases will run twice if both test suites are opted in.
> >> >> > But the two runs are not the same, one runs with hugepage and the
> >> >> > other runs in no-huge mode.
> >> >>
> >> >> Is it really so different between huge and no-huge?  Most of the libraries
> >> >> won't care - they call the rte_**alloc functions, and it returns blocks of
> >> >> memory.  Maybe I am simplifying it too much.
> >> >>
> >> >> > If fast-tests list is splited,  we will need to always run multiple
> >> >> > suites to cover all fast tests.
> >> >> > We can keep x86 to run only fast-tests suite to avoid extra test runs
> >> >> > if they are not necessary. Thoughts?
> >> >>
> >> >> I guess since most DPDK usage will be with hugepages, we should prefer to
> >> >> test with it.  I don't care too much about the color of this particular shed.  If
> >> >> you want to do it that way, it's okay by me - it gives us the coverage, and
> >> >> doesn't duplicate tests between those environments.
> >> >>
> >> >> BUT it means when we add a new test to the suite, we need to remember to
> >> >> add it in two places - fast_test and nohuge_test.  That almost guarantees we
> >> >> will miss tests (because we accidentally don't add it to nohuge).  Maybe
> >> >> there's another way, like keep a list of all the tests and some information on
> >> >> whether the test needs hugepages to run.  Then if there are no hugepages
> >> >> available, we can write that we SKIP the tests that don't support huge pages.
> >> >> In that way, we don't need two different lists - and if there are hugepages,
> >> >> we will run all the test cases.
> >> >> WDYT?
> >> >>
> >> > Yes. Agree with you that having duplicate tests in suites is error prone.
> >>
> >> Cool!
> >>
> >> > IIUC, cases in a suite is determined at build time, as well as command options to run cases.
> >> > This implies hugepage availability needs to be detected at build time if we want to run only
> >> > suitable cases in suite in an environment. It could be something we don't want.
> >> >
> >> > I'll trim fast-tests in next version to remove duplicated cases.
> >>
> >> I think it might be better to make the array something like (just a
> >> psuedo-code example):
> >>
> >>    # psuedo-code to check for hugepages
> >>    has_hugepages = check_for_hugepages()
> >>
> >>    ...
> >>
> >>    fast_test_names = [
> >>      ['acl_autotest', true],
> >>      ['alarm_autotest', true],
> >>      ['atomic_autotest', true],
> >>      ...
> >>
> >> Then in the code:
> >>
> >>    foreach arg : fast_test_names
> >>       ....
> >>       if not arg[1]
> >>          test(arg[0], ...)
> >>       if has_hugepages and arg[1]
> >>          test(arg[0], )
> >>
> >> Does it make sense?  Do you see a problem?
> >
> > I just hope meson won't create a dependency on hugepage availability...
> > Thinking about stuff like:
> > https://git.dpdk.org/dpdk/commit/?id=599d67b6a4bf96096352cc5fbc8fc28e54a1ca62
> 
> Yes, I agree.  We probably need to do some kind of similar abstraction.
>
Hopefully not, since this is for run/test targets rather than for rebuilds.
Try the simple approach first, and only add abstractions if it causes
problems, I think.

Re: [dpdk-dev] [PATCH v2 2/2] ci: add test suite run without hugepage

[Ruifeng Wang]

> -----Original Message-----
> From: Aaron Conole <aconole@redhat.com>
> Sent: Friday, March 6, 2020 23:57
> To: Ruifeng Wang <Ruifeng.Wang@arm.com>
> Cc: maicolgabriel@hotmail.com; bruce.richardson@intel.com; dev@dpdk.org;
> david.marchand@redhat.com; Gavin Hu <Gavin.Hu@arm.com>; Honnappa
> Nagarahalli <Honnappa.Nagarahalli@arm.com>; juraj.linkes@pantheon.tech;
> nd <nd@arm.com>
> Subject: Re: [PATCH v2 2/2] ci: add test suite run without hugepage
> 
> Ruifeng Wang <Ruifeng.Wang@arm.com> writes:
> 
> >> -----Original Message-----
> >> From: Aaron Conole <aconole@redhat.com>
> >> Sent: Thursday, March 5, 2020 22:37
> >> To: Ruifeng Wang <Ruifeng.Wang@arm.com>
> >> Cc: maicolgabriel@hotmail.com; bruce.richardson@intel.com;
> >> dev@dpdk.org; david.marchand@redhat.com; Gavin Hu
> <Gavin.Hu@arm.com>;
> >> Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>;
> >> juraj.linkes@pantheon.tech; nd <nd@arm.com>
> >> Subject: Re: [PATCH v2 2/2] ci: add test suite run without hugepage
> >>
> >> Ruifeng Wang <Ruifeng.Wang@arm.com> writes:
> >>
> >> >> -----Original Message-----
> >> >> From: Aaron Conole <aconole@redhat.com>
> >> >> Sent: Thursday, March 5, 2020 01:31
> >> >> To: Ruifeng Wang <Ruifeng.Wang@arm.com>
> >> >> Cc: maicolgabriel@hotmail.com; bruce.richardson@intel.com;
> >> >> dev@dpdk.org; david.marchand@redhat.com; Gavin Hu
> >> <Gavin.Hu@arm.com>;
> >> >> Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>;
> >> >> juraj.linkes@pantheon.tech; nd <nd@arm.com>
> >> >> Subject: Re: [PATCH v2 2/2] ci: add test suite run without
> >> >> hugepage
> >> >>
> >> >> Ruifeng Wang <ruifeng.wang@arm.com> writes:
> >> >>
> >> >> > This test suite is derived from fast-tests suite. Cases in this
> >> >> > suite are run with '--no-huge' flag.
> >> >> >
> >> >> > The suite aims to cover as many as possible test cases out of
> >> >> > the fast-tests suites in the environments without huge pages
> >> >> > support, like containers.
> >> >> >
> >> >> > Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> >> >> > Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> >> >> > ---
> >> >>
> >> >> I like this much more.  Few comments.
> >> >>
> >> >> >  .travis.yml          | 10 +++++--
> >> >> >  app/test/meson.build | 71
> >> >> >  ++++++++++++++++++++++++++++++++++++++++++++
> >> >>
> >> >> You should update doc/guides/prog_guide/meson_ut.rst to include
> >> >> some detail about the new tests suite.
> >> >>
> >> > Thanks. Will update document in next version.
> >> >
> >> >> >  2 files changed, 79 insertions(+), 2 deletions(-)
> >> >> >
> >> >> > diff --git a/.travis.yml b/.travis.yml index
> >> >> > b64a81bd0..eed1d96db
> >> >> > 100644
> >> >> > --- a/.travis.yml
> >> >> > +++ b/.travis.yml
> >> >> > @@ -40,7 +40,7 @@ jobs:
> >> >> >    - env: DEF_LIB="static"
> >> >> >      arch: amd64
> >> >> >      compiler: gcc
> >> >> > -  - env: DEF_LIB="shared" RUN_TESTS=1
> >> >> > +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="fast-tests
> >> >> nohuge-tests"
> >> >> >      arch: amd64
> >> >> >      compiler: gcc
> >> >> >    - env: DEF_LIB="shared" BUILD_DOCS=1 @@ -63,7 +63,7 @@ jobs:
> >> >> >    - env: DEF_LIB="static"
> >> >> >      arch: amd64
> >> >> >      compiler: clang
> >> >> > -  - env: DEF_LIB="shared" RUN_TESTS=1
> >> >> > +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="fast-tests
> >> >> nohuge-tests"
> >> >> >      arch: amd64
> >> >> >      compiler: clang
> >> >> >    - env: DEF_LIB="shared" BUILD_DOCS=1 @@ -101,6 +101,9 @@
> jobs:
> >> >> >    - env: DEF_LIB="static"
> >> >> >      arch: arm64
> >> >> >      compiler: gcc
> >> >> > +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="nohuge-
> tests"
> >> >> > +    arch: arm64
> >> >> > +    compiler: gcc
> >> >> >    - env: DEF_LIB="shared" BUILD_DOCS=1
> >> >> >      arch: arm64
> >> >> >      compiler: gcc
> >> >> > @@ -124,3 +127,6 @@ jobs:
> >> >> >    - env: DEF_LIB="shared"
> >> >> >      arch: arm64
> >> >> >      compiler: clang
> >> >> > +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="nohuge-
> tests"
> >> >> > +    arch: arm64
> >> >> > +    compiler: clang
> >> >> > diff --git a/app/test/meson.build b/app/test/meson.build index
> >> >> > 0a2ce710f..162a1a76f 100644
> >> >> > --- a/app/test/meson.build
> >> >> > +++ b/app/test/meson.build
> >> >> > @@ -237,6 +237,60 @@ fast_test_names = [
> >> >> >          'thash_autotest',
> >> >> >  ]
> >> >>
> >> >> Shouldn't we also trim the list of fast-tests?  Otherwise, these
> >> >> tests will run twice.
> >> >>
> >> > I think you mean to have exclusive lists for fast-tests and nohuge-tests.
> >>
> >> That's what I was thinking.
> >>
> >> > Overlapped cases will run twice if both test suites are opted in.
> >> > But the two runs are not the same, one runs with hugepage and the
> >> > other runs in no-huge mode.
> >>
> >> Is it really so different between huge and no-huge?  Most of the
> >> libraries won't care - they call the rte_**alloc functions, and it
> >> returns blocks of memory.  Maybe I am simplifying it too much.
> >>
> >> > If fast-tests list is splited,  we will need to always run multiple
> >> > suites to cover all fast tests.
> >> > We can keep x86 to run only fast-tests suite to avoid extra test
> >> > runs if they are not necessary. Thoughts?
> >>
> >> I guess since most DPDK usage will be with hugepages, we should
> >> prefer to test with it.  I don't care too much about the color of
> >> this particular shed.  If you want to do it that way, it's okay by me
> >> - it gives us the coverage, and doesn't duplicate tests between those
> environments.
> >>
> >> BUT it means when we add a new test to the suite, we need to
> remember
> >> to add it in two places - fast_test and nohuge_test.  That almost
> >> guarantees we will miss tests (because we accidentally don't add it
> >> to nohuge).  Maybe there's another way, like keep a list of all the
> >> tests and some information on whether the test needs hugepages to
> >> run.  Then if there are no hugepages available, we can write that we SKIP
> the tests that don't support huge pages.
> >> In that way, we don't need two different lists - and if there are
> >> hugepages, we will run all the test cases.
> >> WDYT?
> >>
> > Yes. Agree with you that having duplicate tests in suites is error prone.
> 
> Cool!
> 
> > IIUC, cases in a suite is determined at build time, as well as command
> options to run cases.
> > This implies hugepage availability needs to be detected at build time
> > if we want to run only suitable cases in suite in an environment. It could be
> something we don't want.
> >
> > I'll trim fast-tests in next version to remove duplicated cases.
> 
> I think it might be better to make the array something like (just a psuedo-
> code example):
> 
>    # psuedo-code to check for hugepages
>    has_hugepages = check_for_hugepages()
> 
>    ...
> 
>    fast_test_names = [
>      ['acl_autotest', true],
>      ['alarm_autotest', true],
>      ['atomic_autotest', true],
>      ...
> 
> Then in the code:
> 
>    foreach arg : fast_test_names
>       ....
>       if not arg[1]
>          test(arg[0], ...)
>       if has_hugepages and arg[1]
>          test(arg[0], )
> 
> Does it make sense?  Do you see a problem?
> 
Yes, this will keep tests in a single suite.

Actually, I thought about this approach, but had no idea on check_for_hugepages().
Value of "/proc/sys/vm/nr_hugepages" may be not reliable. Hugepage could be allocated at run time after project building.
I can try hugepage allocating to detect, but it looks inelegant.
Any suggestions?

Thank you.
/Ruifeng
> > Thank you.
> >
> >> >> ex: https://travis-ci.com/ovsrobot/dpdk/jobs/292037684
> >> >>
> >> >> > +nohuge_test_names = [
> >> >> > +        'byteorder_autotest',
> >> >> > +        'cmdline_autotest',
> >> >> > +        'common_autotest',
> >> >> > +        'cpuflags_autotest',
> >> >> > +        'cycles_autotest',
> >> >> > +        'debug_autotest',
> >> >> > +        'eal_flags_n_opt_autotest',
> >> >> > +        'eal_flags_no_huge_autotest',
> >> >> > +        'eal_flags_vdev_opt_autotest',
> >> >> > +        'eal_fs_autotest',
> >> >> > +        'errno_autotest',
> >> >> > +        'event_ring_autotest',
> >> >> > +        'fib_autotest',
> >> >> > +        'fib6_autotest',
> >> >> > +        'interrupt_autotest',
> >> >> > +        'logs_autotest',
> >> >> > +        'lpm_autotest',
> >> >> > +        'lpm6_autotest',
> >> >> > +        'memcpy_autotest',
> >> >> > +        'meter_autotest',
> >> >> > +        'per_lcore_autotest',
> >> >> > +        'prefetch_autotest',
> >> >> > +        'rcu_qsbr_autotest',
> >> >> > +        'red_autotest',
> >> >> > +        'rib_autotest',
> >> >> > +        'rib6_autotest',
> >> >> > +        'ring_autotest',
> >> >> > +        'rwlock_rda_autotest',
> >> >> > +        'rwlock_rds_wrm_autotest',
> >> >> > +        'rwlock_rde_wro_autotest',
> >> >> > +        'sched_autotest',
> >> >> > +        'spinlock_autotest',
> >> >> > +        'string_autotest',
> >> >> > +        'tailq_autotest',
> >> >> > +        'user_delay_us',
> >> >> > +        'version_autotest',
> >> >> > +        'crc_autotest',
> >> >> > +        'delay_us_sleep_autotest',
> >> >> > +        'eventdev_common_autotest',
> >> >> > +        'fbarray_autotest',
> >> >> > +        'ipsec_autotest',
> >> >> > +        'kni_autotest',
> >> >> > +        'kvargs_autotest',
> >> >> > +        'member_autotest',
> >> >> > +        'metrics_autotest',
> >> >> > +        'power_cpufreq_autotest',
> >> >> > +        'power_autotest',
> >> >> > +        'power_kvm_vm_autotest',
> >> >> > +        'reorder_autotest',
> >> >> > +        'service_autotest',
> >> >> > +        'thash_autotest',
> >> >> > +]
> >> >> > +
> >> >> >  perf_test_names = [
> >> >> >          'ring_perf_autotest',
> >> >> >          'mempool_perf_autotest', @@ -341,6 +395,10 @@ if
> >> >> > dpdk_conf.has('RTE_LIBRTE_RING_PMD')
> >> >> >  	fast_test_names += 'latencystats_autotest'
> >> >> >  	driver_test_names += 'link_bonding_mode4_autotest'
> >> >> >  	fast_test_names += 'pdump_autotest'
> >> >> > +	nohuge_test_names += 'ring_pmd_autotest'
> >> >> > +	nohuge_test_names += 'bitratestats_autotest'
> >> >> > +	nohuge_test_names += 'latencystats_autotest'
> >> >> > +	nohuge_test_names += 'pdump_autotest'
> >> >> >  endif
> >> >> >
> >> >> >  if dpdk_conf.has('RTE_LIBRTE_POWER') @@ -430,6 +488,19 @@
> >> foreach
> >> >> > arg : fast_test_names
> >> >> >  	endif
> >> >> >  endforeach
> >> >> >
> >> >> > +foreach arg : nohuge_test_names
> >> >> > +	if host_machine.system() == 'linux'
> >> >> > +		test(arg, dpdk_test,
> >> >> > +			  env : ['DPDK_TEST=' + arg],
> >> >> > +			  args : test_args +
> >> >> > +				 ['--no-huge'] + ['-m 1024'] +
> >> >> > +				 ['--file-prefix=@0@'.format(arg)],
> >> >> > +		timeout : timeout_seconds_fast,
> >> >> > +		is_parallel : false,
> >> >> > +		suite : 'nohuge-tests')
> >> >> > +	endif
> >> >> > +endforeach
> >> >> > +
> >> >> >  foreach arg : perf_test_names
> >> >> >  	test(arg, dpdk_test,
> >> >> >  	env : ['DPDK_TEST=' + arg],

Re: [dpdk-dev] [PATCH v2 2/2] ci: add test suite run without hugepage

[Juraj Linkeš]

Hi,

What if we just split the test names into fast_hugepage_test_name and fast_nohuge_test_names and create two suites (fast-suite and fast-nohuge-suite) from that? That seems like a simpler version of one list with booleans indicating whether a test is a huge/nohuge test, though it would require dev to add tests into the proper list. We could then run the full suite in x86 jobs and the nogue suite in arm jobs (and users could use either suite when they have/don't have hugepages configured).

The other option would be to detect hugepages at runtime and skip the tests in the full suite when there aren't hugepages on the system. Possibly the best option, but where should the check be? Before testcase setup (i.e. if there aren't hugepages, skip the testcase right away)?

Thoughts?
Thanks,
Juraj

-----Original Message-----
From: Ruifeng Wang <Ruifeng.Wang@arm.com> 
Sent: Saturday, March 7, 2020 3:36 PM
To: Aaron Conole <aconole@redhat.com>
Cc: maicolgabriel@hotmail.com; bruce.richardson@intel.com; dev@dpdk.org; david.marchand@redhat.com; Gavin Hu <Gavin.Hu@arm.com>; Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>; Juraj Linkeš <juraj.linkes@pantheon.tech>; nd <nd@arm.com>; nd <nd@arm.com>
Subject: RE: [PATCH v2 2/2] ci: add test suite run without hugepage


> -----Original Message-----
> From: Aaron Conole <aconole@redhat.com>
> Sent: Friday, March 6, 2020 23:57
> To: Ruifeng Wang <Ruifeng.Wang@arm.com>
> Cc: maicolgabriel@hotmail.com; bruce.richardson@intel.com; 
> dev@dpdk.org; david.marchand@redhat.com; Gavin Hu <Gavin.Hu@arm.com>; 
> Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>; 
> juraj.linkes@pantheon.tech; nd <nd@arm.com>
> Subject: Re: [PATCH v2 2/2] ci: add test suite run without hugepage
> 
> Ruifeng Wang <Ruifeng.Wang@arm.com> writes:
> 
> >> -----Original Message-----
> >> From: Aaron Conole <aconole@redhat.com>
> >> Sent: Thursday, March 5, 2020 22:37
> >> To: Ruifeng Wang <Ruifeng.Wang@arm.com>
> >> Cc: maicolgabriel@hotmail.com; bruce.richardson@intel.com; 
> >> dev@dpdk.org; david.marchand@redhat.com; Gavin Hu
> <Gavin.Hu@arm.com>;
> >> Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>; 
> >> juraj.linkes@pantheon.tech; nd <nd@arm.com>
> >> Subject: Re: [PATCH v2 2/2] ci: add test suite run without hugepage
> >>
> >> Ruifeng Wang <Ruifeng.Wang@arm.com> writes:
> >>
> >> >> -----Original Message-----
> >> >> From: Aaron Conole <aconole@redhat.com>
> >> >> Sent: Thursday, March 5, 2020 01:31
> >> >> To: Ruifeng Wang <Ruifeng.Wang@arm.com>
> >> >> Cc: maicolgabriel@hotmail.com; bruce.richardson@intel.com; 
> >> >> dev@dpdk.org; david.marchand@redhat.com; Gavin Hu
> >> <Gavin.Hu@arm.com>;
> >> >> Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>; 
> >> >> juraj.linkes@pantheon.tech; nd <nd@arm.com>
> >> >> Subject: Re: [PATCH v2 2/2] ci: add test suite run without 
> >> >> hugepage
> >> >>
> >> >> Ruifeng Wang <ruifeng.wang@arm.com> writes:
> >> >>
> >> >> > This test suite is derived from fast-tests suite. Cases in 
> >> >> > this suite are run with '--no-huge' flag.
> >> >> >
> >> >> > The suite aims to cover as many as possible test cases out of 
> >> >> > the fast-tests suites in the environments without huge pages 
> >> >> > support, like containers.
> >> >> >
> >> >> > Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> >> >> > Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> >> >> > ---
> >> >>
> >> >> I like this much more.  Few comments.
> >> >>
> >> >> >  .travis.yml          | 10 +++++--
> >> >> >  app/test/meson.build | 71
> >> >> >  ++++++++++++++++++++++++++++++++++++++++++++
> >> >>
> >> >> You should update doc/guides/prog_guide/meson_ut.rst to include 
> >> >> some detail about the new tests suite.
> >> >>
> >> > Thanks. Will update document in next version.
> >> >
> >> >> >  2 files changed, 79 insertions(+), 2 deletions(-)
> >> >> >
> >> >> > diff --git a/.travis.yml b/.travis.yml index 
> >> >> > b64a81bd0..eed1d96db
> >> >> > 100644
> >> >> > --- a/.travis.yml
> >> >> > +++ b/.travis.yml
> >> >> > @@ -40,7 +40,7 @@ jobs:
> >> >> >    - env: DEF_LIB="static"
> >> >> >      arch: amd64
> >> >> >      compiler: gcc
> >> >> > -  - env: DEF_LIB="shared" RUN_TESTS=1
> >> >> > +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="fast-tests
> >> >> nohuge-tests"
> >> >> >      arch: amd64
> >> >> >      compiler: gcc
> >> >> >    - env: DEF_LIB="shared" BUILD_DOCS=1 @@ -63,7 +63,7 @@ jobs:
> >> >> >    - env: DEF_LIB="static"
> >> >> >      arch: amd64
> >> >> >      compiler: clang
> >> >> > -  - env: DEF_LIB="shared" RUN_TESTS=1
> >> >> > +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="fast-tests
> >> >> nohuge-tests"
> >> >> >      arch: amd64
> >> >> >      compiler: clang
> >> >> >    - env: DEF_LIB="shared" BUILD_DOCS=1 @@ -101,6 +101,9 @@
> jobs:
> >> >> >    - env: DEF_LIB="static"
> >> >> >      arch: arm64
> >> >> >      compiler: gcc
> >> >> > +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="nohuge-
> tests"
> >> >> > +    arch: arm64
> >> >> > +    compiler: gcc
> >> >> >    - env: DEF_LIB="shared" BUILD_DOCS=1
> >> >> >      arch: arm64
> >> >> >      compiler: gcc
> >> >> > @@ -124,3 +127,6 @@ jobs:
> >> >> >    - env: DEF_LIB="shared"
> >> >> >      arch: arm64
> >> >> >      compiler: clang
> >> >> > +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="nohuge-
> tests"
> >> >> > +    arch: arm64
> >> >> > +    compiler: clang
> >> >> > diff --git a/app/test/meson.build b/app/test/meson.build index 
> >> >> > 0a2ce710f..162a1a76f 100644
> >> >> > --- a/app/test/meson.build
> >> >> > +++ b/app/test/meson.build
> >> >> > @@ -237,6 +237,60 @@ fast_test_names = [
> >> >> >          'thash_autotest',
> >> >> >  ]
> >> >>
> >> >> Shouldn't we also trim the list of fast-tests?  Otherwise, these 
> >> >> tests will run twice.
> >> >>
> >> > I think you mean to have exclusive lists for fast-tests and nohuge-tests.
> >>
> >> That's what I was thinking.
> >>
> >> > Overlapped cases will run twice if both test suites are opted in.
> >> > But the two runs are not the same, one runs with hugepage and the 
> >> > other runs in no-huge mode.
> >>
> >> Is it really so different between huge and no-huge?  Most of the 
> >> libraries won't care - they call the rte_**alloc functions, and it 
> >> returns blocks of memory.  Maybe I am simplifying it too much.
> >>
> >> > If fast-tests list is splited,  we will need to always run 
> >> > multiple suites to cover all fast tests.
> >> > We can keep x86 to run only fast-tests suite to avoid extra test 
> >> > runs if they are not necessary. Thoughts?
> >>
> >> I guess since most DPDK usage will be with hugepages, we should 
> >> prefer to test with it.  I don't care too much about the color of 
> >> this particular shed.  If you want to do it that way, it's okay by 
> >> me
> >> - it gives us the coverage, and doesn't duplicate tests between 
> >> those
> environments.
> >>
> >> BUT it means when we add a new test to the suite, we need to
> remember
> >> to add it in two places - fast_test and nohuge_test.  That almost 
> >> guarantees we will miss tests (because we accidentally don't add it 
> >> to nohuge).  Maybe there's another way, like keep a list of all the 
> >> tests and some information on whether the test needs hugepages to 
> >> run.  Then if there are no hugepages available, we can write that 
> >> we SKIP
> the tests that don't support huge pages.
> >> In that way, we don't need two different lists - and if there are 
> >> hugepages, we will run all the test cases.
> >> WDYT?
> >>
> > Yes. Agree with you that having duplicate tests in suites is error prone.
> 
> Cool!
> 
> > IIUC, cases in a suite is determined at build time, as well as 
> > command
> options to run cases.
> > This implies hugepage availability needs to be detected at build 
> > time if we want to run only suitable cases in suite in an 
> > environment. It could be
> something we don't want.
> >
> > I'll trim fast-tests in next version to remove duplicated cases.
> 
> I think it might be better to make the array something like (just a 
> psuedo- code example):
> 
>    # psuedo-code to check for hugepages
>    has_hugepages = check_for_hugepages()
> 
>    ...
> 
>    fast_test_names = [
>      ['acl_autotest', true],
>      ['alarm_autotest', true],
>      ['atomic_autotest', true],
>      ...
> 
> Then in the code:
> 
>    foreach arg : fast_test_names
>       ....
>       if not arg[1]
>          test(arg[0], ...)
>       if has_hugepages and arg[1]
>          test(arg[0], )
> 
> Does it make sense?  Do you see a problem?
> 
Yes, this will keep tests in a single suite.

Actually, I thought about this approach, but had no idea on check_for_hugepages().
Value of "/proc/sys/vm/nr_hugepages" may be not reliable. Hugepage could be allocated at run time after project building.
I can try hugepage allocating to detect, but it looks inelegant.
Any suggestions?

Thank you.
/Ruifeng
> > Thank you.
> >
> >> >> ex: https://travis-ci.com/ovsrobot/dpdk/jobs/292037684
> >> >>
> >> >> > +nohuge_test_names = [
> >> >> > +        'byteorder_autotest',
> >> >> > +        'cmdline_autotest',
> >> >> > +        'common_autotest',
> >> >> > +        'cpuflags_autotest',
> >> >> > +        'cycles_autotest',
> >> >> > +        'debug_autotest',
> >> >> > +        'eal_flags_n_opt_autotest',
> >> >> > +        'eal_flags_no_huge_autotest',
> >> >> > +        'eal_flags_vdev_opt_autotest',
> >> >> > +        'eal_fs_autotest',
> >> >> > +        'errno_autotest',
> >> >> > +        'event_ring_autotest',
> >> >> > +        'fib_autotest',
> >> >> > +        'fib6_autotest',
> >> >> > +        'interrupt_autotest',
> >> >> > +        'logs_autotest',
> >> >> > +        'lpm_autotest',
> >> >> > +        'lpm6_autotest',
> >> >> > +        'memcpy_autotest',
> >> >> > +        'meter_autotest',
> >> >> > +        'per_lcore_autotest',
> >> >> > +        'prefetch_autotest',
> >> >> > +        'rcu_qsbr_autotest',
> >> >> > +        'red_autotest',
> >> >> > +        'rib_autotest',
> >> >> > +        'rib6_autotest',
> >> >> > +        'ring_autotest',
> >> >> > +        'rwlock_rda_autotest',
> >> >> > +        'rwlock_rds_wrm_autotest',
> >> >> > +        'rwlock_rde_wro_autotest',
> >> >> > +        'sched_autotest',
> >> >> > +        'spinlock_autotest',
> >> >> > +        'string_autotest',
> >> >> > +        'tailq_autotest',
> >> >> > +        'user_delay_us',
> >> >> > +        'version_autotest',
> >> >> > +        'crc_autotest',
> >> >> > +        'delay_us_sleep_autotest',
> >> >> > +        'eventdev_common_autotest',
> >> >> > +        'fbarray_autotest',
> >> >> > +        'ipsec_autotest',
> >> >> > +        'kni_autotest',
> >> >> > +        'kvargs_autotest',
> >> >> > +        'member_autotest',
> >> >> > +        'metrics_autotest',
> >> >> > +        'power_cpufreq_autotest',
> >> >> > +        'power_autotest',
> >> >> > +        'power_kvm_vm_autotest',
> >> >> > +        'reorder_autotest',
> >> >> > +        'service_autotest',
> >> >> > +        'thash_autotest',
> >> >> > +]
> >> >> > +
> >> >> >  perf_test_names = [
> >> >> >          'ring_perf_autotest',
> >> >> >          'mempool_perf_autotest', @@ -341,6 +395,10 @@ if
> >> >> > dpdk_conf.has('RTE_LIBRTE_RING_PMD')
> >> >> >  	fast_test_names += 'latencystats_autotest'
> >> >> >  	driver_test_names += 'link_bonding_mode4_autotest'
> >> >> >  	fast_test_names += 'pdump_autotest'
> >> >> > +	nohuge_test_names += 'ring_pmd_autotest'
> >> >> > +	nohuge_test_names += 'bitratestats_autotest'
> >> >> > +	nohuge_test_names += 'latencystats_autotest'
> >> >> > +	nohuge_test_names += 'pdump_autotest'
> >> >> >  endif
> >> >> >
> >> >> >  if dpdk_conf.has('RTE_LIBRTE_POWER') @@ -430,6 +488,19 @@
> >> foreach
> >> >> > arg : fast_test_names
> >> >> >  	endif
> >> >> >  endforeach
> >> >> >
> >> >> > +foreach arg : nohuge_test_names
> >> >> > +	if host_machine.system() == 'linux'
> >> >> > +		test(arg, dpdk_test,
> >> >> > +			  env : ['DPDK_TEST=' + arg],
> >> >> > +			  args : test_args +
> >> >> > +				 ['--no-huge'] + ['-m 1024'] +
> >> >> > +				 ['--file-prefix=@0@'.format(arg)],
> >> >> > +		timeout : timeout_seconds_fast,
> >> >> > +		is_parallel : false,
> >> >> > +		suite : 'nohuge-tests')
> >> >> > +	endif
> >> >> > +endforeach
> >> >> > +
> >> >> >  foreach arg : perf_test_names
> >> >> >  	test(arg, dpdk_test,
> >> >> >  	env : ['DPDK_TEST=' + arg],

Re: [dpdk-dev] [PATCH v2 2/2] ci: add test suite run without hugepage

[Ruifeng Wang]

> -----Original Message-----
> From: Juraj Linkeš <juraj.linkes@pantheon.tech>
> Sent: Thursday, March 12, 2020 15:13
> To: Ruifeng Wang <Ruifeng.Wang@arm.com>; Aaron Conole
> <aconole@redhat.com>
> Cc: maicolgabriel@hotmail.com; bruce.richardson@intel.com; dev@dpdk.org;
> david.marchand@redhat.com; Gavin Hu <Gavin.Hu@arm.com>; Honnappa
> Nagarahalli <Honnappa.Nagarahalli@arm.com>; nd <nd@arm.com>; nd
> <nd@arm.com>
> Subject: RE: [PATCH v2 2/2] ci: add test suite run without hugepage
> 
> Hi,
> 
> What if we just split the test names into fast_hugepage_test_name and
> fast_nohuge_test_names and create two suites (fast-suite and fast-nohuge-
> suite) from that? That seems like a simpler version of one list with booleans
> indicating whether a test is a huge/nohuge test, though it would require dev
> to add tests into the proper list. We could then run the full suite in x86 jobs
> and the nogue suite in arm jobs (and users could use either suite when they
> have/don't have hugepages configured).
> 
> The other option would be to detect hugepages at runtime and skip the tests
> in the full suite when there aren't hugepages on the system. Possibly the
> best option, but where should the check be? Before testcase setup (i.e. if
> there aren't hugepages, skip the testcase right away)?
> 
I'm on this direction. Detect hugepages at build time and generate fast_tests suite with applicable cases.
nr_hugepages is checked before testcase setup.

Will send out patches for review soon.

Thanks.
/Ruifeng

> Thoughts?
> Thanks,
> Juraj
> 
> -----Original Message-----
> From: Ruifeng Wang <Ruifeng.Wang@arm.com>
> Sent: Saturday, March 7, 2020 3:36 PM
> To: Aaron Conole <aconole@redhat.com>
> Cc: maicolgabriel@hotmail.com; bruce.richardson@intel.com; dev@dpdk.org;
> david.marchand@redhat.com; Gavin Hu <Gavin.Hu@arm.com>; Honnappa
> Nagarahalli <Honnappa.Nagarahalli@arm.com>; Juraj Linkeš
> <juraj.linkes@pantheon.tech>; nd <nd@arm.com>; nd <nd@arm.com>
> Subject: RE: [PATCH v2 2/2] ci: add test suite run without hugepage
> 
> 
> > -----Original Message-----
> > From: Aaron Conole <aconole@redhat.com>
> > Sent: Friday, March 6, 2020 23:57
> > To: Ruifeng Wang <Ruifeng.Wang@arm.com>
> > Cc: maicolgabriel@hotmail.com; bruce.richardson@intel.com;
> > dev@dpdk.org; david.marchand@redhat.com; Gavin Hu
> <Gavin.Hu@arm.com>;
> > Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>;
> > juraj.linkes@pantheon.tech; nd <nd@arm.com>
> > Subject: Re: [PATCH v2 2/2] ci: add test suite run without hugepage
> >
> > Ruifeng Wang <Ruifeng.Wang@arm.com> writes:
> >
> > >> -----Original Message-----
> > >> From: Aaron Conole <aconole@redhat.com>
> > >> Sent: Thursday, March 5, 2020 22:37
> > >> To: Ruifeng Wang <Ruifeng.Wang@arm.com>
> > >> Cc: maicolgabriel@hotmail.com; bruce.richardson@intel.com;
> > >> dev@dpdk.org; david.marchand@redhat.com; Gavin Hu
> > <Gavin.Hu@arm.com>;
> > >> Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>;
> > >> juraj.linkes@pantheon.tech; nd <nd@arm.com>
> > >> Subject: Re: [PATCH v2 2/2] ci: add test suite run without hugepage
> > >>
> > >> Ruifeng Wang <Ruifeng.Wang@arm.com> writes:
> > >>
> > >> >> -----Original Message-----
> > >> >> From: Aaron Conole <aconole@redhat.com>
> > >> >> Sent: Thursday, March 5, 2020 01:31
> > >> >> To: Ruifeng Wang <Ruifeng.Wang@arm.com>
> > >> >> Cc: maicolgabriel@hotmail.com; bruce.richardson@intel.com;
> > >> >> dev@dpdk.org; david.marchand@redhat.com; Gavin Hu
> > >> <Gavin.Hu@arm.com>;
> > >> >> Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>;
> > >> >> juraj.linkes@pantheon.tech; nd <nd@arm.com>
> > >> >> Subject: Re: [PATCH v2 2/2] ci: add test suite run without
> > >> >> hugepage
> > >> >>
> > >> >> Ruifeng Wang <ruifeng.wang@arm.com> writes:
> > >> >>
> > >> >> > This test suite is derived from fast-tests suite. Cases in
> > >> >> > this suite are run with '--no-huge' flag.
> > >> >> >
> > >> >> > The suite aims to cover as many as possible test cases out of
> > >> >> > the fast-tests suites in the environments without huge pages
> > >> >> > support, like containers.
> > >> >> >
> > >> >> > Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> > >> >> > Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> > >> >> > ---
> > >> >>
> > >> >> I like this much more.  Few comments.
> > >> >>
> > >> >> >  .travis.yml          | 10 +++++--
> > >> >> >  app/test/meson.build | 71
> > >> >> >  ++++++++++++++++++++++++++++++++++++++++++++
> > >> >>
> > >> >> You should update doc/guides/prog_guide/meson_ut.rst to include
> > >> >> some detail about the new tests suite.
> > >> >>
> > >> > Thanks. Will update document in next version.
> > >> >
> > >> >> >  2 files changed, 79 insertions(+), 2 deletions(-)
> > >> >> >
> > >> >> > diff --git a/.travis.yml b/.travis.yml index
> > >> >> > b64a81bd0..eed1d96db
> > >> >> > 100644
> > >> >> > --- a/.travis.yml
> > >> >> > +++ b/.travis.yml
> > >> >> > @@ -40,7 +40,7 @@ jobs:
> > >> >> >    - env: DEF_LIB="static"
> > >> >> >      arch: amd64
> > >> >> >      compiler: gcc
> > >> >> > -  - env: DEF_LIB="shared" RUN_TESTS=1
> > >> >> > +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="fast-tests
> > >> >> nohuge-tests"
> > >> >> >      arch: amd64
> > >> >> >      compiler: gcc
> > >> >> >    - env: DEF_LIB="shared" BUILD_DOCS=1 @@ -63,7 +63,7 @@
> jobs:
> > >> >> >    - env: DEF_LIB="static"
> > >> >> >      arch: amd64
> > >> >> >      compiler: clang
> > >> >> > -  - env: DEF_LIB="shared" RUN_TESTS=1
> > >> >> > +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="fast-tests
> > >> >> nohuge-tests"
> > >> >> >      arch: amd64
> > >> >> >      compiler: clang
> > >> >> >    - env: DEF_LIB="shared" BUILD_DOCS=1 @@ -101,6 +101,9 @@
> > jobs:
> > >> >> >    - env: DEF_LIB="static"
> > >> >> >      arch: arm64
> > >> >> >      compiler: gcc
> > >> >> > +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="nohuge-
> > tests"
> > >> >> > +    arch: arm64
> > >> >> > +    compiler: gcc
> > >> >> >    - env: DEF_LIB="shared" BUILD_DOCS=1
> > >> >> >      arch: arm64
> > >> >> >      compiler: gcc
> > >> >> > @@ -124,3 +127,6 @@ jobs:
> > >> >> >    - env: DEF_LIB="shared"
> > >> >> >      arch: arm64
> > >> >> >      compiler: clang
> > >> >> > +  - env: DEF_LIB="shared" RUN_TESTS=1 TEST_SUITES="nohuge-
> > tests"
> > >> >> > +    arch: arm64
> > >> >> > +    compiler: clang
> > >> >> > diff --git a/app/test/meson.build b/app/test/meson.build index
> > >> >> > 0a2ce710f..162a1a76f 100644
> > >> >> > --- a/app/test/meson.build
> > >> >> > +++ b/app/test/meson.build
> > >> >> > @@ -237,6 +237,60 @@ fast_test_names = [
> > >> >> >          'thash_autotest',
> > >> >> >  ]
> > >> >>
> > >> >> Shouldn't we also trim the list of fast-tests?  Otherwise, these
> > >> >> tests will run twice.
> > >> >>
> > >> > I think you mean to have exclusive lists for fast-tests and nohuge-tests.
> > >>
> > >> That's what I was thinking.
> > >>
> > >> > Overlapped cases will run twice if both test suites are opted in.
> > >> > But the two runs are not the same, one runs with hugepage and the
> > >> > other runs in no-huge mode.
> > >>
> > >> Is it really so different between huge and no-huge?  Most of the
> > >> libraries won't care - they call the rte_**alloc functions, and it
> > >> returns blocks of memory.  Maybe I am simplifying it too much.
> > >>
> > >> > If fast-tests list is splited,  we will need to always run
> > >> > multiple suites to cover all fast tests.
> > >> > We can keep x86 to run only fast-tests suite to avoid extra test
> > >> > runs if they are not necessary. Thoughts?
> > >>
> > >> I guess since most DPDK usage will be with hugepages, we should
> > >> prefer to test with it.  I don't care too much about the color of
> > >> this particular shed.  If you want to do it that way, it's okay by
> > >> me
> > >> - it gives us the coverage, and doesn't duplicate tests between
> > >> those
> > environments.
> > >>
> > >> BUT it means when we add a new test to the suite, we need to
> > remember
> > >> to add it in two places - fast_test and nohuge_test.  That almost
> > >> guarantees we will miss tests (because we accidentally don't add it
> > >> to nohuge).  Maybe there's another way, like keep a list of all the
> > >> tests and some information on whether the test needs hugepages to
> > >> run.  Then if there are no hugepages available, we can write that
> > >> we SKIP
> > the tests that don't support huge pages.
> > >> In that way, we don't need two different lists - and if there are
> > >> hugepages, we will run all the test cases.
> > >> WDYT?
> > >>
> > > Yes. Agree with you that having duplicate tests in suites is error prone.
> >
> > Cool!
> >
> > > IIUC, cases in a suite is determined at build time, as well as
> > > command
> > options to run cases.
> > > This implies hugepage availability needs to be detected at build
> > > time if we want to run only suitable cases in suite in an
> > > environment. It could be
> > something we don't want.
> > >
> > > I'll trim fast-tests in next version to remove duplicated cases.
> >
> > I think it might be better to make the array something like (just a
> > psuedo- code example):
> >
> >    # psuedo-code to check for hugepages
> >    has_hugepages = check_for_hugepages()
> >
> >    ...
> >
> >    fast_test_names = [
> >      ['acl_autotest', true],
> >      ['alarm_autotest', true],
> >      ['atomic_autotest', true],
> >      ...
> >
> > Then in the code:
> >
> >    foreach arg : fast_test_names
> >       ....
> >       if not arg[1]
> >          test(arg[0], ...)
> >       if has_hugepages and arg[1]
> >          test(arg[0], )
> >
> > Does it make sense?  Do you see a problem?
> >
> Yes, this will keep tests in a single suite.
> 
> Actually, I thought about this approach, but had no idea on
> check_for_hugepages().
> Value of "/proc/sys/vm/nr_hugepages" may be not reliable. Hugepage could
> be allocated at run time after project building.
> I can try hugepage allocating to detect, but it looks inelegant.
> Any suggestions?
> 
> Thank you.
> /Ruifeng
> > > Thank you.
> > >
> > >> >> ex: https://travis-ci.com/ovsrobot/dpdk/jobs/292037684
> > >> >>
> > >> >> > +nohuge_test_names = [
> > >> >> > +        'byteorder_autotest',
> > >> >> > +        'cmdline_autotest',
> > >> >> > +        'common_autotest',
> > >> >> > +        'cpuflags_autotest',
> > >> >> > +        'cycles_autotest',
> > >> >> > +        'debug_autotest',
> > >> >> > +        'eal_flags_n_opt_autotest',
> > >> >> > +        'eal_flags_no_huge_autotest',
> > >> >> > +        'eal_flags_vdev_opt_autotest',
> > >> >> > +        'eal_fs_autotest',
> > >> >> > +        'errno_autotest',
> > >> >> > +        'event_ring_autotest',
> > >> >> > +        'fib_autotest',
> > >> >> > +        'fib6_autotest',
> > >> >> > +        'interrupt_autotest',
> > >> >> > +        'logs_autotest',
> > >> >> > +        'lpm_autotest',
> > >> >> > +        'lpm6_autotest',
> > >> >> > +        'memcpy_autotest',
> > >> >> > +        'meter_autotest',
> > >> >> > +        'per_lcore_autotest',
> > >> >> > +        'prefetch_autotest',
> > >> >> > +        'rcu_qsbr_autotest',
> > >> >> > +        'red_autotest',
> > >> >> > +        'rib_autotest',
> > >> >> > +        'rib6_autotest',
> > >> >> > +        'ring_autotest',
> > >> >> > +        'rwlock_rda_autotest',
> > >> >> > +        'rwlock_rds_wrm_autotest',
> > >> >> > +        'rwlock_rde_wro_autotest',
> > >> >> > +        'sched_autotest',
> > >> >> > +        'spinlock_autotest',
> > >> >> > +        'string_autotest',
> > >> >> > +        'tailq_autotest',
> > >> >> > +        'user_delay_us',
> > >> >> > +        'version_autotest',
> > >> >> > +        'crc_autotest',
> > >> >> > +        'delay_us_sleep_autotest',
> > >> >> > +        'eventdev_common_autotest',
> > >> >> > +        'fbarray_autotest',
> > >> >> > +        'ipsec_autotest',
> > >> >> > +        'kni_autotest',
> > >> >> > +        'kvargs_autotest',
> > >> >> > +        'member_autotest',
> > >> >> > +        'metrics_autotest',
> > >> >> > +        'power_cpufreq_autotest',
> > >> >> > +        'power_autotest',
> > >> >> > +        'power_kvm_vm_autotest',
> > >> >> > +        'reorder_autotest',
> > >> >> > +        'service_autotest',
> > >> >> > +        'thash_autotest',
> > >> >> > +]
> > >> >> > +
> > >> >> >  perf_test_names = [
> > >> >> >          'ring_perf_autotest',
> > >> >> >          'mempool_perf_autotest', @@ -341,6 +395,10 @@ if
> > >> >> > dpdk_conf.has('RTE_LIBRTE_RING_PMD')
> > >> >> >  	fast_test_names += 'latencystats_autotest'
> > >> >> >  	driver_test_names += 'link_bonding_mode4_autotest'
> > >> >> >  	fast_test_names += 'pdump_autotest'
> > >> >> > +	nohuge_test_names += 'ring_pmd_autotest'
> > >> >> > +	nohuge_test_names += 'bitratestats_autotest'
> > >> >> > +	nohuge_test_names += 'latencystats_autotest'
> > >> >> > +	nohuge_test_names += 'pdump_autotest'
> > >> >> >  endif
> > >> >> >
> > >> >> >  if dpdk_conf.has('RTE_LIBRTE_POWER') @@ -430,6 +488,19 @@
> > >> foreach
> > >> >> > arg : fast_test_names
> > >> >> >  	endif
> > >> >> >  endforeach
> > >> >> >
> > >> >> > +foreach arg : nohuge_test_names
> > >> >> > +	if host_machine.system() == 'linux'
> > >> >> > +		test(arg, dpdk_test,
> > >> >> > +			  env : ['DPDK_TEST=' + arg],
> > >> >> > +			  args : test_args +
> > >> >> > +				 ['--no-huge'] + ['-m 1024'] +
> > >> >> > +				 ['--file-prefix=@0@'.format(arg)],
> > >> >> > +		timeout : timeout_seconds_fast,
> > >> >> > +		is_parallel : false,
> > >> >> > +		suite : 'nohuge-tests')
> > >> >> > +	endif
> > >> >> > +endforeach
> > >> >> > +
> > >> >> >  foreach arg : perf_test_names
> > >> >> >  	test(arg, dpdk_test,
> > >> >> >  	env : ['DPDK_TEST=' + arg],

[dpdk-dev] [PATCH v3 0/4] no-huge unit test

[Ruifeng Wang]

For environments (such as containers) where hugetlbfs are not available,
some unit tests can be run with 'no-huge' option.

fast-tests suites is generated dynamically according to hugetlbfs
availability in building environment. This allows unit test to run
in different environments using the same suite name.

Several test cases are fixed to be able to run in no-huge mode.

v3:
Use a single suite instead of create additional one for no-huge. (Aaron)
Fix several test cases in no-huge mode.

v2:
Add a patch to enable running multiple suites in a job. (David)


Ruifeng Wang (4):
  test: enable tests to run in no-huge mode
  ci: generate fast-tests suite base on hugepage availability
  ci: proceed with verification without hugepage
  ci: enable unit test for aarch64

 .ci/linux-setup.sh             |  11 +-
 .travis.yml                    |   5 +-
 app/test/meson.build           | 216 ++++++++++++++++++---------------
 app/test/test_acl.c            |  22 ++--
 app/test/test_hash.c           |   7 +-
 app/test/test_table_pipeline.c |  12 +-
 6 files changed, 152 insertions(+), 121 deletions(-)

-- 
2.17.1

[dpdk-dev] [PATCH v3 1/4] test: enable tests to run in no-huge mode

[Ruifeng Wang]

When running with '--no-huge' flag, tests failed with messages as:
    ACL context creation with invalid NUMA should have failed!
    fbk hash creation should have failed
    test_table_pipeline: Check pipeline invalid params failed.

These cases test against invalid socket ID as input parameter, and
expect error return. But function calls return success because
invalid sock ID is overwritten to SOCKET_ID_ANY when in no-huge mode.

The tests against invalid socket ID are skipped in no-huge mode.

Fixes: 5640171c528a ("malloc: fix external heap allocation in no-huge mode")
Cc: stable@dpdk.org

Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>
---
 app/test/test_acl.c            | 22 ++++++++++++----------
 app/test/test_hash.c           |  7 +++++--
 app/test/test_table_pipeline.c | 12 +++++++-----
 3 files changed, 24 insertions(+), 17 deletions(-)

diff --git a/app/test/test_acl.c b/app/test/test_acl.c
index 501de35da..316bf4d06 100644
--- a/app/test/test_acl.c
+++ b/app/test/test_acl.c
@@ -1397,16 +1397,18 @@ test_invalid_parameters(void)
 	} else
 		rte_acl_free(acx);
 
-	/* invalid NUMA node */
-	memcpy(&param, &acl_param, sizeof(param));
-	param.socket_id = RTE_MAX_NUMA_NODES + 1;
-
-	acx = rte_acl_create(&param);
-	if (acx != NULL) {
-		printf("Line %i: ACL context creation with invalid NUMA "
-				"should have failed!\n", __LINE__);
-		rte_acl_free(acx);
-		return -1;
+	if (rte_eal_has_hugepages()) {
+		/* invalid NUMA node */
+		memcpy(&param, &acl_param, sizeof(param));
+		param.socket_id = RTE_MAX_NUMA_NODES + 1;
+
+		acx = rte_acl_create(&param);
+		if (acx != NULL) {
+			printf("Line %i: ACL context creation with invalid "
+					"NUMA should have failed!\n", __LINE__);
+			rte_acl_free(acx);
+			return -1;
+		}
 	}
 
 	/* NULL name */
diff --git a/app/test/test_hash.c b/app/test/test_hash.c
index fbd5725c6..ab978ea68 100644
--- a/app/test/test_hash.c
+++ b/app/test/test_hash.c
@@ -1136,8 +1136,11 @@ fbk_hash_unit_test(void)
 	handle = rte_fbk_hash_create(&invalid_params_7);
 	RETURN_IF_ERROR_FBK(handle != NULL, "fbk hash creation should have failed");
 
-	handle = rte_fbk_hash_create(&invalid_params_8);
-	RETURN_IF_ERROR_FBK(handle != NULL, "fbk hash creation should have failed");
+	if (rte_eal_has_hugepages()) {
+		handle = rte_fbk_hash_create(&invalid_params_8);
+		RETURN_IF_ERROR_FBK(handle != NULL,
+					"fbk hash creation should have failed");
+	}
 
 	handle = rte_fbk_hash_create(&invalid_params_same_name_1);
 	RETURN_IF_ERROR_FBK(handle == NULL, "fbk hash creation should have succeeded");
diff --git a/app/test/test_table_pipeline.c b/app/test/test_table_pipeline.c
index 441338ac0..bc412c308 100644
--- a/app/test/test_table_pipeline.c
+++ b/app/test/test_table_pipeline.c
@@ -190,11 +190,13 @@ check_pipeline_invalid_params(void)
 		goto fail;
 	}
 
-	p = rte_pipeline_create(&pipeline_params_3);
-	if (p != NULL) {
-		RTE_LOG(INFO, PIPELINE, "%s: Configure pipeline with invalid "
-			"socket\n", __func__);
-		goto fail;
+	if (rte_eal_has_hugepages()) {
+		p = rte_pipeline_create(&pipeline_params_3);
+		if (p != NULL) {
+			RTE_LOG(INFO, PIPELINE, "%s: Configure pipeline with "
+				"invalid socket\n", __func__);
+			goto fail;
+		}
 	}
 
 	/* Check pipeline consistency */
-- 
2.17.1

Re: [dpdk-dev] [PATCH v3 1/4] test: enable tests to run in no-huge mode

[Aaron Conole]

Ruifeng Wang <ruifeng.wang@arm.com> writes:

> When running with '--no-huge' flag, tests failed with messages as:
>     ACL context creation with invalid NUMA should have failed!
>     fbk hash creation should have failed
>     test_table_pipeline: Check pipeline invalid params failed.
>
> These cases test against invalid socket ID as input parameter, and
> expect error return. But function calls return success because
> invalid sock ID is overwritten to SOCKET_ID_ANY when in no-huge mode.
>
> The tests against invalid socket ID are skipped in no-huge mode.
>
> Fixes: 5640171c528a ("malloc: fix external heap allocation in no-huge mode")
> Cc: stable@dpdk.org
>
> Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> ---

Acked-by: Aaron Conole <aconole@redhat.com>

>  app/test/test_acl.c            | 22 ++++++++++++----------
>  app/test/test_hash.c           |  7 +++++--
>  app/test/test_table_pipeline.c | 12 +++++++-----
>  3 files changed, 24 insertions(+), 17 deletions(-)
>
> diff --git a/app/test/test_acl.c b/app/test/test_acl.c
> index 501de35da..316bf4d06 100644
> --- a/app/test/test_acl.c
> +++ b/app/test/test_acl.c
> @@ -1397,16 +1397,18 @@ test_invalid_parameters(void)
>  	} else
>  		rte_acl_free(acx);
>  
> -	/* invalid NUMA node */
> -	memcpy(&param, &acl_param, sizeof(param));
> -	param.socket_id = RTE_MAX_NUMA_NODES + 1;
> -
> -	acx = rte_acl_create(&param);
> -	if (acx != NULL) {
> -		printf("Line %i: ACL context creation with invalid NUMA "
> -				"should have failed!\n", __LINE__);
> -		rte_acl_free(acx);
> -		return -1;
> +	if (rte_eal_has_hugepages()) {
> +		/* invalid NUMA node */
> +		memcpy(&param, &acl_param, sizeof(param));
> +		param.socket_id = RTE_MAX_NUMA_NODES + 1;
> +
> +		acx = rte_acl_create(&param);
> +		if (acx != NULL) {
> +			printf("Line %i: ACL context creation with invalid "
> +					"NUMA should have failed!\n", __LINE__);
> +			rte_acl_free(acx);
> +			return -1;
> +		}
>  	}
>  
>  	/* NULL name */
> diff --git a/app/test/test_hash.c b/app/test/test_hash.c
> index fbd5725c6..ab978ea68 100644
> --- a/app/test/test_hash.c
> +++ b/app/test/test_hash.c
> @@ -1136,8 +1136,11 @@ fbk_hash_unit_test(void)
>  	handle = rte_fbk_hash_create(&invalid_params_7);
>  	RETURN_IF_ERROR_FBK(handle != NULL, "fbk hash creation should have failed");
>  
> -	handle = rte_fbk_hash_create(&invalid_params_8);
> -	RETURN_IF_ERROR_FBK(handle != NULL, "fbk hash creation should have failed");
> +	if (rte_eal_has_hugepages()) {
> +		handle = rte_fbk_hash_create(&invalid_params_8);
> +		RETURN_IF_ERROR_FBK(handle != NULL,
> +					"fbk hash creation should have failed");
> +	}
>  
>  	handle = rte_fbk_hash_create(&invalid_params_same_name_1);
>  	RETURN_IF_ERROR_FBK(handle == NULL, "fbk hash creation should have succeeded");
> diff --git a/app/test/test_table_pipeline.c b/app/test/test_table_pipeline.c
> index 441338ac0..bc412c308 100644
> --- a/app/test/test_table_pipeline.c
> +++ b/app/test/test_table_pipeline.c
> @@ -190,11 +190,13 @@ check_pipeline_invalid_params(void)
>  		goto fail;
>  	}
>  
> -	p = rte_pipeline_create(&pipeline_params_3);
> -	if (p != NULL) {
> -		RTE_LOG(INFO, PIPELINE, "%s: Configure pipeline with invalid "
> -			"socket\n", __func__);
> -		goto fail;
> +	if (rte_eal_has_hugepages()) {
> +		p = rte_pipeline_create(&pipeline_params_3);
> +		if (p != NULL) {
> +			RTE_LOG(INFO, PIPELINE, "%s: Configure pipeline with "
> +				"invalid socket\n", __func__);
> +			goto fail;
> +		}
>  	}
>  
>  	/* Check pipeline consistency */

Re: [dpdk-dev] [PATCH v3 1/4] test: enable tests to run in no-huge mode

[Wang, Yipeng1]

> -----Original Message-----
> From: Ruifeng Wang <ruifeng.wang@arm.com>
> Sent: Friday, March 13, 2020 1:16 AM
> To: aconole@redhat.com; maicolgabriel@hotmail.com; Richardson, Bruce
> <bruce.richardson@intel.com>; Ananyev, Konstantin
> <konstantin.ananyev@intel.com>; Dumitrescu, Cristian
> <cristian.dumitrescu@intel.com>; Wang, Yipeng1
> <yipeng1.wang@intel.com>; Gobriel, Sameh <sameh.gobriel@intel.com>
> Cc: dev@dpdk.org; david.marchand@redhat.com; Burakov, Anatoly
> <anatoly.burakov@intel.com>; gavin.hu@arm.com;
> honnappa.nagarahalli@arm.com; juraj.linkes@pantheon.tech; nd@arm.com;
> Ruifeng Wang <ruifeng.wang@arm.com>; stable@dpdk.org
> Subject: [PATCH v3 1/4] test: enable tests to run in no-huge mode
> 
> When running with '--no-huge' flag, tests failed with messages as:
>     ACL context creation with invalid NUMA should have failed!
>     fbk hash creation should have failed
>     test_table_pipeline: Check pipeline invalid params failed.
> 
> These cases test against invalid socket ID as input parameter, and expect
> error return. But function calls return success because invalid sock ID is
> overwritten to SOCKET_ID_ANY when in no-huge mode.
> 
> The tests against invalid socket ID are skipped in no-huge mode.
> 
> Fixes: 5640171c528a ("malloc: fix external heap allocation in no-huge mode")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> ---
>  app/test/test_acl.c            | 22 ++++++++++++----------
>  app/test/test_hash.c           |  7 +++++--
>  app/test/test_table_pipeline.c | 12 +++++++-----
>  3 files changed, 24 insertions(+), 17 deletions(-)
> 
> diff --git a/app/test/test_acl.c b/app/test/test_acl.c index
> 501de35da..316bf4d06 100644
> --- a/app/test/test_acl.c
> +++ b/app/test/test_acl.c
> @@ -1397,16 +1397,18 @@ test_invalid_parameters(void)
>  	} else
>  		rte_acl_free(acx);
> 
> -	/* invalid NUMA node */
> -	memcpy(&param, &acl_param, sizeof(param));
> -	param.socket_id = RTE_MAX_NUMA_NODES + 1;
> -
> -	acx = rte_acl_create(&param);
> -	if (acx != NULL) {
> -		printf("Line %i: ACL context creation with invalid NUMA "
> -				"should have failed!\n", __LINE__);
> -		rte_acl_free(acx);
> -		return -1;
> +	if (rte_eal_has_hugepages()) {
> +		/* invalid NUMA node */
> +		memcpy(&param, &acl_param, sizeof(param));
> +		param.socket_id = RTE_MAX_NUMA_NODES + 1;
> +
> +		acx = rte_acl_create(&param);
> +		if (acx != NULL) {
> +			printf("Line %i: ACL context creation with invalid "
> +					"NUMA should have failed!\n",
> __LINE__);
> +			rte_acl_free(acx);
> +			return -1;
> +		}
>  	}
> 
>  	/* NULL name */
> diff --git a/app/test/test_hash.c b/app/test/test_hash.c index
> fbd5725c6..ab978ea68 100644
> --- a/app/test/test_hash.c
> +++ b/app/test/test_hash.c
> @@ -1136,8 +1136,11 @@ fbk_hash_unit_test(void)
>  	handle = rte_fbk_hash_create(&invalid_params_7);
>  	RETURN_IF_ERROR_FBK(handle != NULL, "fbk hash creation should
> have failed");
> 
> -	handle = rte_fbk_hash_create(&invalid_params_8);
> -	RETURN_IF_ERROR_FBK(handle != NULL, "fbk hash creation should
> have failed");
> +	if (rte_eal_has_hugepages()) {
> +		handle = rte_fbk_hash_create(&invalid_params_8);
> +		RETURN_IF_ERROR_FBK(handle != NULL,
> +					"fbk hash creation should have
> failed");
> +	}
> 
>  	handle = rte_fbk_hash_create(&invalid_params_same_name_1);
>  	RETURN_IF_ERROR_FBK(handle == NULL, "fbk hash creation should

[Wang, Yipeng] 
For the hash table part:
Acked-by: Yipeng Wang <yipeng1.wang@intel.com>

Just a side-note, not related to this patch,
should we add such info in programmer's guide for rte_malloc saying that if no-huge, then socket-id
Is not taken?

[dpdk-dev] [PATCH v3 2/4] ci: generate fast-tests suite base on hugepage availability

[Ruifeng Wang]

In environments where hugepage are not available, such as
containers, many cases in fast-tests suite should also run
if no-huge EAL option is used.

Flag is appended to each case in fast-tests suite to indicate
whether it lives with no-huge mode.
With the flag, fast-tests suite can be generated based on
detected hugepage availability of building environment.
All cases will be valid if hugepage is available, whereas
only applicable cases will be added if environment has no
hugepage support.

Suggested-by: Aaron Conole <aconole@redhat.com>
Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>
---
 app/test/meson.build | 216 ++++++++++++++++++++++++-------------------
 1 file changed, 120 insertions(+), 96 deletions(-)

diff --git a/app/test/meson.build b/app/test/meson.build
index 0a2ce710f..dd121a297 100644
--- a/app/test/meson.build
+++ b/app/test/meson.build
@@ -154,87 +154,89 @@ test_deps = ['acl',
 	'timer'
 ]
 
+# Each test is marked with flag true/false
+# to indicate whether it can run in no-huge mode.
 fast_test_names = [
-        'acl_autotest',
-        'alarm_autotest',
-        'atomic_autotest',
-        'byteorder_autotest',
-        'cmdline_autotest',
-        'common_autotest',
-        'cpuflags_autotest',
-        'cycles_autotest',
-        'debug_autotest',
-        'eal_flags_c_opt_autotest',
-        'eal_flags_master_opt_autotest',
-        'eal_flags_n_opt_autotest',
-        'eal_flags_hpet_autotest',
-        'eal_flags_no_huge_autotest',
-        'eal_flags_w_opt_autotest',
-        'eal_flags_b_opt_autotest',
-        'eal_flags_vdev_opt_autotest',
-        'eal_flags_r_opt_autotest',
-        'eal_flags_mem_autotest',
-        'eal_flags_file_prefix_autotest',
-        'eal_flags_misc_autotest',
-        'eal_fs_autotest',
-        'errno_autotest',
-        'event_ring_autotest',
-        'fib_autotest',
-        'fib6_autotest',
-        'func_reentrancy_autotest',
-        'flow_classify_autotest',
-        'hash_autotest',
-        'interrupt_autotest',
-        'logs_autotest',
-        'lpm_autotest',
-        'lpm6_autotest',
-        'malloc_autotest',
-        'mbuf_autotest',
-        'mcslock_autotest',
-        'memcpy_autotest',
-        'memory_autotest',
-        'mempool_autotest',
-        'memzone_autotest',
-        'meter_autotest',
-        'multiprocess_autotest',
-        'per_lcore_autotest',
-        'prefetch_autotest',
-        'rcu_qsbr_autotest',
-        'red_autotest',
-        'rib_autotest',
-        'rib6_autotest',
-        'ring_autotest',
-        'rwlock_test1_autotest',
-        'rwlock_rda_autotest',
-        'rwlock_rds_wrm_autotest',
-        'rwlock_rde_wro_autotest',
-        'sched_autotest',
-        'spinlock_autotest',
-        'stack_autotest',
-        'stack_lf_autotest',
-        'string_autotest',
-        'table_autotest',
-        'tailq_autotest',
-        'timer_autotest',
-        'user_delay_us',
-        'version_autotest',
-        'crc_autotest',
-        'delay_us_sleep_autotest',
-        'distributor_autotest',
-        'eventdev_common_autotest',
-        'fbarray_autotest',
-        'hash_readwrite_func_autotest',
-        'ipsec_autotest',
-        'kni_autotest',
-        'kvargs_autotest',
-        'member_autotest',
-        'metrics_autotest',
-        'power_cpufreq_autotest',
-        'power_autotest',
-        'power_kvm_vm_autotest',
-        'reorder_autotest',
-        'service_autotest',
-        'thash_autotest',
+        ['acl_autotest', true],
+        ['alarm_autotest', false],
+        ['atomic_autotest', false],
+        ['byteorder_autotest', true],
+        ['cmdline_autotest', true],
+        ['common_autotest', true],
+        ['cpuflags_autotest', true],
+        ['cycles_autotest', true],
+        ['debug_autotest', true],
+        ['eal_flags_c_opt_autotest', false],
+        ['eal_flags_master_opt_autotest', false],
+        ['eal_flags_n_opt_autotest', false],
+        ['eal_flags_hpet_autotest', false],
+        ['eal_flags_no_huge_autotest', false],
+        ['eal_flags_w_opt_autotest', false],
+        ['eal_flags_b_opt_autotest', false],
+        ['eal_flags_vdev_opt_autotest', false],
+        ['eal_flags_r_opt_autotest', false],
+        ['eal_flags_mem_autotest', false],
+        ['eal_flags_file_prefix_autotest', false],
+        ['eal_flags_misc_autotest', false],
+        ['eal_fs_autotest', true],
+        ['errno_autotest', true],
+        ['event_ring_autotest', true],
+        ['fib_autotest', true],
+        ['fib6_autotest', true],
+        ['func_reentrancy_autotest', false],
+        ['flow_classify_autotest', false],
+        ['hash_autotest', true],
+        ['interrupt_autotest', true],
+        ['logs_autotest', true],
+        ['lpm_autotest', true],
+        ['lpm6_autotest', true],
+        ['malloc_autotest', false],
+        ['mbuf_autotest', false],
+        ['mcslock_autotest', false],
+        ['memcpy_autotest', true],
+        ['memory_autotest', false],
+        ['mempool_autotest', false],
+        ['memzone_autotest', false],
+        ['meter_autotest', true],
+        ['multiprocess_autotest', false],
+        ['per_lcore_autotest', true],
+        ['prefetch_autotest', true],
+        ['rcu_qsbr_autotest', true],
+        ['red_autotest', true],
+        ['rib_autotest', true],
+        ['rib6_autotest', true],
+        ['ring_autotest', true],
+        ['rwlock_test1_autotest', true],
+        ['rwlock_rda_autotest', true],
+        ['rwlock_rds_wrm_autotest', true],
+        ['rwlock_rde_wro_autotest', true],
+        ['sched_autotest', true],
+        ['spinlock_autotest', true],
+        ['stack_autotest', false],
+        ['stack_lf_autotest', false],
+        ['string_autotest', true],
+        ['table_autotest', true],
+        ['tailq_autotest', true],
+        ['timer_autotest', false],
+        ['user_delay_us', true],
+        ['version_autotest', true],
+        ['crc_autotest', true],
+        ['delay_us_sleep_autotest', true],
+        ['distributor_autotest', false],
+        ['eventdev_common_autotest', true],
+        ['fbarray_autotest', true],
+        ['hash_readwrite_func_autotest', false],
+        ['ipsec_autotest', true],
+        ['kni_autotest', false],
+        ['kvargs_autotest', true],
+        ['member_autotest', true],
+        ['metrics_autotest', true],
+        ['power_cpufreq_autotest', false],
+        ['power_autotest', true],
+        ['power_kvm_vm_autotest', false],
+        ['reorder_autotest', true],
+        ['service_autotest', true],
+        ['thash_autotest', true],
 ]
 
 perf_test_names = [
@@ -334,13 +336,13 @@ if dpdk_conf.has('RTE_LIBRTE_RING_PMD')
 	test_sources += 'test_link_bonding_mode4.c'
 	test_sources += 'sample_packet_forward.c'
 	test_sources += 'test_pdump.c'
-	fast_test_names += 'ring_pmd_autotest'
+	fast_test_names += [['ring_pmd_autotest', true]]
 	perf_test_names += 'ring_pmd_perf_autotest'
-	fast_test_names += 'event_eth_tx_adapter_autotest'
-	fast_test_names += 'bitratestats_autotest'
-	fast_test_names += 'latencystats_autotest'
+	fast_test_names += [['event_eth_tx_adapter_autotest', false]]
+	fast_test_names += [['bitratestats_autotest', true]]
+	fast_test_names += [['latencystats_autotest', true]]
 	driver_test_names += 'link_bonding_mode4_autotest'
-	fast_test_names += 'pdump_autotest'
+	fast_test_names += [['pdump_autotest', true]]
 endif
 
 if dpdk_conf.has('RTE_LIBRTE_POWER')
@@ -368,7 +370,7 @@ if dpdk_conf.has('RTE_LIBRTE_COMPRESSDEV')
 		test_dep_objs += compress_test_dep
 		test_sources += 'test_compressdev.c'
 		test_deps += 'compressdev'
-		fast_test_names += 'compressdev_autotest'
+		fast_test_names += [['compressdev_autotest', false]]
 	endif
 endif
 
@@ -395,6 +397,17 @@ dpdk_test = executable('dpdk-test',
 	install_rpath: driver_install_path,
 	install: true)
 
+has_hugepage = true
+if host_machine.system() == 'linux'
+	check_hugepage = run_command('cat',
+				     '/proc/sys/vm/nr_hugepages')
+	if (check_hugepage.returncode() != 0 or
+	    check_hugepage.stdout().strip() == '0')
+		has_hugepage = false
+	endif
+endif
+message('hugepage availbility: @0@'.format(has_hugepage))
+
 # some perf tests (eg: memcpy perf autotest)take very long
 # to complete, so timeout to 10 minutes
 timeout_seconds = 600
@@ -407,22 +420,33 @@ test_args = [num_cores_arg]
 
 foreach arg : fast_test_names
 	if (get_option('default_library') == 'shared' and
-		arg == 'event_eth_tx_adapter_autotest')
+		arg[0] == 'event_eth_tx_adapter_autotest')
 		foreach drv:dpdk_drivers
 			test_args += ['-d', drv.full_path().split('.a')[0] + '.so']
 		endforeach
 	endif
 	if host_machine.system() == 'linux'
-		test(arg, dpdk_test,
-			  env : ['DPDK_TEST=' + arg],
-			  args : test_args +
-				 ['--file-prefix=@0@'.format(arg)],
-		timeout : timeout_seconds_fast,
-		is_parallel : false,
-		suite : 'fast-tests')
+		if has_hugepage
+			test(arg[0], dpdk_test,
+				  env : ['DPDK_TEST=' + arg[0]],
+				  args : test_args +
+					 ['--file-prefix=@0@'.format(arg[0])],
+			timeout : timeout_seconds_fast,
+			is_parallel : false,
+			suite : 'fast-tests')
+		elif arg[1]
+			test(arg[0], dpdk_test,
+				  env : ['DPDK_TEST=' + arg[0]],
+				  args : test_args +
+					 ['--no-huge'] + ['-m 2048'] +
+					 ['--file-prefix=@0@'.format(arg[0])],
+			timeout : timeout_seconds_fast,
+			is_parallel : false,
+			suite : 'fast-tests')
+		endif
 	else
-		test(arg, dpdk_test,
-			env : ['DPDK_TEST=' + arg],
+		test(arg[0], dpdk_test,
+			env : ['DPDK_TEST=' + arg[0]],
 			args : test_args,
 		timeout : timeout_seconds_fast,
 		is_parallel : false,
-- 
2.17.1

Re: [dpdk-dev] [PATCH v3 2/4] ci: generate fast-tests suite base on hugepage availability

[Aaron Conole]

Ruifeng Wang <ruifeng.wang@arm.com> writes:

> In environments where hugepage are not available, such as
> containers, many cases in fast-tests suite should also run
> if no-huge EAL option is used.
>
> Flag is appended to each case in fast-tests suite to indicate
> whether it lives with no-huge mode.
> With the flag, fast-tests suite can be generated based on
> detected hugepage availability of building environment.
> All cases will be valid if hugepage is available, whereas
> only applicable cases will be added if environment has no
> hugepage support.
>
> Suggested-by: Aaron Conole <aconole@redhat.com>
> Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> ---

Acked-by: Aaron Conole <aconole@redhat.com>

Re: [dpdk-dev] [PATCH v3 2/4] ci: generate fast-tests suite base on hugepage availability

[David Marchand]

On Fri, Mar 13, 2020 at 9:17 AM Ruifeng Wang <ruifeng.wang@arm.com> wrote:
>
> In environments where hugepage are not available, such as
> containers, many cases in fast-tests suite should also run
> if no-huge EAL option is used.
>
> Flag is appended to each case in fast-tests suite to indicate
> whether it lives with no-huge mode.
> With the flag, fast-tests suite can be generated based on
> detected hugepage availability of building environment.
> All cases will be valid if hugepage is available, whereas
> only applicable cases will be added if environment has no
> hugepage support.
>
> Suggested-by: Aaron Conole <aconole@redhat.com>
> Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> ---
>  app/test/meson.build | 216 ++++++++++++++++++++++++-------------------
>  1 file changed, 120 insertions(+), 96 deletions(-)
>
> diff --git a/app/test/meson.build b/app/test/meson.build
> index 0a2ce710f..dd121a297 100644
> --- a/app/test/meson.build
> +++ b/app/test/meson.build
> @@ -154,87 +154,89 @@ test_deps = ['acl',
>         'timer'
>  ]
>
> +# Each test is marked with flag true/false
> +# to indicate whether it can run in no-huge mode.
>  fast_test_names = [

Nit: this variable does not contain a list of names anymore, how about
"fast_tests".

> -        'acl_autotest',
> -        'alarm_autotest',
> -        'atomic_autotest',
[snip]
> +        ['acl_autotest', true],
> +        ['alarm_autotest', false],
> +        ['atomic_autotest', false],

[snip]

> @@ -395,6 +397,17 @@ dpdk_test = executable('dpdk-test',
>         install_rpath: driver_install_path,
>         install: true)
>
> +has_hugepage = true
> +if host_machine.system() == 'linux'

We have is_OS helpers, here it should be if is_linux.


> +       check_hugepage = run_command('cat',
> +                                    '/proc/sys/vm/nr_hugepages')
> +       if (check_hugepage.returncode() != 0 or
> +           check_hugepage.stdout().strip() == '0')
> +               has_hugepage = false
> +       endif
> +endif
> +message('hugepage availbility: @0@'.format(has_hugepage))

availability*

> +
>  # some perf tests (eg: memcpy perf autotest)take very long
>  # to complete, so timeout to 10 minutes
>  timeout_seconds = 600
> @@ -407,22 +420,33 @@ test_args = [num_cores_arg]
>
>  foreach arg : fast_test_names
>         if (get_option('default_library') == 'shared' and
> -               arg == 'event_eth_tx_adapter_autotest')
> +               arg[0] == 'event_eth_tx_adapter_autotest')
>                 foreach drv:dpdk_drivers
>                         test_args += ['-d', drv.full_path().split('.a')[0] + '.so']
>                 endforeach
>         endif
>         if host_machine.system() == 'linux'
> -               test(arg, dpdk_test,
> -                         env : ['DPDK_TEST=' + arg],
> -                         args : test_args +
> -                                ['--file-prefix=@0@'.format(arg)],
> -               timeout : timeout_seconds_fast,
> -               is_parallel : false,
> -               suite : 'fast-tests')
> +               if has_hugepage
> +                       test(arg[0], dpdk_test,
> +                                 env : ['DPDK_TEST=' + arg[0]],
> +                                 args : test_args +
> +                                        ['--file-prefix=@0@'.format(arg[0])],
> +                       timeout : timeout_seconds_fast,
> +                       is_parallel : false,
> +                       suite : 'fast-tests')
> +               elif arg[1]
> +                       test(arg[0], dpdk_test,
> +                                 env : ['DPDK_TEST=' + arg[0]],
> +                                 args : test_args +
> +                                        ['--no-huge'] + ['-m 2048'] +
> +                                        ['--file-prefix=@0@'.format(arg[0])],
> +                       timeout : timeout_seconds_fast,
> +                       is_parallel : false,
> +                       suite : 'fast-tests')
> +               endif
>         else
> -               test(arg, dpdk_test,
> -                       env : ['DPDK_TEST=' + arg],
> +               test(arg[0], dpdk_test,
> +                       env : ['DPDK_TEST=' + arg[0]],
>                         args : test_args,
>                 timeout : timeout_seconds_fast,
>                 is_parallel : false,


Not a fan of these conditionals.

I sent a little patch fixing an issue I spotted on test_args:
http://patchwork.dpdk.org/patch/67026/

I rebased this series on it, see:
https://github.com/david-marchand/dpdk/commits/ci

This makes the code easier to read from my pov:
https://github.com/david-marchand/dpdk/blob/ci/app/test/meson.build#L421

If you are fine with it, I can post a v4 series.


-- 
David Marchand

Re: [dpdk-dev] [PATCH v3 2/4] ci: generate fast-tests suite base on hugepage availability

[David Marchand]

On Mon, Mar 23, 2020 at 9:56 AM David Marchand
<david.marchand@redhat.com> wrote:
> I rebased this series on it, see:
> https://github.com/david-marchand/dpdk/commits/ci
>
> This makes the code easier to read from my pov:
> https://github.com/david-marchand/dpdk/blob/ci/app/test/meson.build#L421

First time I see this error:
https://travis-ci.com/github/david-marchand/dpdk/jobs/301063161#L8410

Did you encounter it?


-- 
David Marchand

Re: [dpdk-dev] [PATCH v3 2/4] ci: generate fast-tests suite base on hugepage availability

[Ruifeng Wang]

> -----Original Message-----
> From: David Marchand <david.marchand@redhat.com>
> Sent: Monday, March 23, 2020 17:33
> To: Ruifeng Wang <Ruifeng.Wang@arm.com>; Aaron Conole
> <aconole@redhat.com>
> Cc: Michael Santana <maicolgabriel@hotmail.com>; Bruce Richardson
> <bruce.richardson@intel.com>; Ananyev, Konstantin
> <konstantin.ananyev@intel.com>; Cristian Dumitrescu
> <cristian.dumitrescu@intel.com>; Wang, Yipeng1
> <yipeng1.wang@intel.com>; Gobriel, Sameh <sameh.gobriel@intel.com>;
> dev <dev@dpdk.org>; Burakov, Anatoly <anatoly.burakov@intel.com>;
> Gavin Hu <Gavin.Hu@arm.com>; Honnappa Nagarahalli
> <Honnappa.Nagarahalli@arm.com>; juraj.linkes@pantheon.tech; nd
> <nd@arm.com>
> Subject: Re: [PATCH v3 2/4] ci: generate fast-tests suite base on hugepage
> availability
> 
> On Mon, Mar 23, 2020 at 9:56 AM David Marchand
> <david.marchand@redhat.com> wrote:
> > I rebased this series on it, see:
> > https://github.com/david-marchand/dpdk/commits/ci
> >
> > This makes the code easier to read from my pov:
> > https://github.com/david-
> marchand/dpdk/blob/ci/app/test/meson.build#L4
> > 21
> 
> First time I see this error:
> https://travis-ci.com/github/david-marchand/dpdk/jobs/301063161#L8410
> 
> Did you encounter it?

First time for me to see this as well.
Not sure how often it will happen. Looks like clock was not stable?
I'll need time to check this part.

Thanks.
/Ruifeng
> 
> 
> --
> David Marchand

Re: [dpdk-dev] [PATCH v3 2/4] ci: generate fast-tests suite base on hugepage availability

[David Marchand]

Hello Ruifeng,

On Mon, Mar 23, 2020 at 4:50 PM Ruifeng Wang <Ruifeng.Wang@arm.com> wrote:
> > First time I see this error:
> > https://travis-ci.com/github/david-marchand/dpdk/jobs/301063161#L8410
> >
> > Did you encounter it?
>
> First time for me to see this as well.
> Not sure how often it will happen. Looks like clock was not stable?
> I'll need time to check this part.

Fyi, caught on master:
https://travis-ci.com/github/DPDK/dpdk/jobs/305880147#L8685



-- 
David Marchand

Re: [dpdk-dev] [PATCH v3 2/4] ci: generate fast-tests suite base on hugepage availability

[Ruifeng Wang]

> -----Original Message-----
> From: David Marchand <david.marchand@redhat.com>
> Sent: Sunday, March 29, 2020 00:18
> To: Ruifeng Wang <Ruifeng.Wang@arm.com>
> Cc: Aaron Conole <aconole@redhat.com>; Michael Santana
> <maicolgabriel@hotmail.com>; Bruce Richardson
> <bruce.richardson@intel.com>; Ananyev, Konstantin
> <konstantin.ananyev@intel.com>; dev <dev@dpdk.org>; Burakov, Anatoly
> <anatoly.burakov@intel.com>; Gavin Hu <Gavin.Hu@arm.com>; Honnappa
> Nagarahalli <Honnappa.Nagarahalli@arm.com>; nd <nd@arm.com>
> Subject: Re: [PATCH v3 2/4] ci: generate fast-tests suite base on hugepage
> availability
> 
> Hello Ruifeng,
> 
> On Mon, Mar 23, 2020 at 4:50 PM Ruifeng Wang <Ruifeng.Wang@arm.com>
> wrote:
> > > First time I see this error:
> > > https://travis-ci.com/github/david-
> marchand/dpdk/jobs/301063161#L8410
> > >
> > > Did you encounter it?
> >
> > First time for me to see this as well.
> > Not sure how often it will happen. Looks like clock was not stable?
> > I'll need time to check this part.
> 
> Fyi, caught on master:
> https://travis-ci.com/github/DPDK/dpdk/jobs/305880147#L8685
> 
Thanks for the notice.
I tried to reproduce this locally, but haven't duplicated the issue.
I didn't see issue from DPDK API perspective. 
Maybe we will remove this case from no-huge run to avoid Travis failure.

/Ruifeng
> 
> 
> --
> David Marchand

Re: [dpdk-dev] [PATCH v3 2/4] ci: generate fast-tests suite base on hugepage availability

[David Marchand]

On Mon, Mar 30, 2020 at 6:03 PM Ruifeng Wang <Ruifeng.Wang@arm.com> wrote:
>
>
> > -----Original Message-----
> > From: David Marchand <david.marchand@redhat.com>
> > Sent: Sunday, March 29, 2020 00:18
> > To: Ruifeng Wang <Ruifeng.Wang@arm.com>
> > Cc: Aaron Conole <aconole@redhat.com>; Michael Santana
> > <maicolgabriel@hotmail.com>; Bruce Richardson
> > <bruce.richardson@intel.com>; Ananyev, Konstantin
> > <konstantin.ananyev@intel.com>; dev <dev@dpdk.org>; Burakov, Anatoly
> > <anatoly.burakov@intel.com>; Gavin Hu <Gavin.Hu@arm.com>; Honnappa
> > Nagarahalli <Honnappa.Nagarahalli@arm.com>; nd <nd@arm.com>
> > Subject: Re: [PATCH v3 2/4] ci: generate fast-tests suite base on hugepage
> > availability
> >
> > Hello Ruifeng,
> >
> > On Mon, Mar 23, 2020 at 4:50 PM Ruifeng Wang <Ruifeng.Wang@arm.com>
> > wrote:
> > > > First time I see this error:
> > > > https://travis-ci.com/github/david-
> > marchand/dpdk/jobs/301063161#L8410
> > > >
> > > > Did you encounter it?
> > >
> > > First time for me to see this as well.
> > > Not sure how often it will happen. Looks like clock was not stable?
> > > I'll need time to check this part.
> >
> > Fyi, caught on master:
> > https://travis-ci.com/github/DPDK/dpdk/jobs/305880147#L8685
> >
> Thanks for the notice.
> I tried to reproduce this locally, but haven't duplicated the issue.

Running in containers is likely to have an effect.
Maybe try to run concurrently a busy loop on the cores running the test?

Btw, I noticed a new kind of timeout:
https://travis-ci.com/github/ovsrobot/dpdk/jobs/318096284#L3231


> I didn't see issue from DPDK API perspective.
> Maybe we will remove this case from no-huge run to avoid Travis failure.

I prefer we fix the tests.


-- 
David Marchand

Re: [dpdk-dev] [PATCH v3 2/4] ci: generate fast-tests suite base on hugepage availability

[Ruifeng Wang]

> -----Original Message-----
> From: David Marchand <david.marchand@redhat.com>
> Sent: Monday, March 23, 2020 16:57
> To: Ruifeng Wang <Ruifeng.Wang@arm.com>; Aaron Conole
> <aconole@redhat.com>
> Cc: Michael Santana <maicolgabriel@hotmail.com>; Bruce Richardson
> <bruce.richardson@intel.com>; Ananyev, Konstantin
> <konstantin.ananyev@intel.com>; Cristian Dumitrescu
> <cristian.dumitrescu@intel.com>; Wang, Yipeng1
> <yipeng1.wang@intel.com>; Gobriel, Sameh <sameh.gobriel@intel.com>;
> dev <dev@dpdk.org>; Burakov, Anatoly <anatoly.burakov@intel.com>;
> Gavin Hu <Gavin.Hu@arm.com>; Honnappa Nagarahalli
> <Honnappa.Nagarahalli@arm.com>; juraj.linkes@pantheon.tech; nd
> <nd@arm.com>
> Subject: Re: [PATCH v3 2/4] ci: generate fast-tests suite base on hugepage
> availability
> 
> On Fri, Mar 13, 2020 at 9:17 AM Ruifeng Wang <ruifeng.wang@arm.com>
> wrote:
> >
> > In environments where hugepage are not available, such as containers,
> > many cases in fast-tests suite should also run if no-huge EAL option
> > is used.
> >
> > Flag is appended to each case in fast-tests suite to indicate whether
> > it lives with no-huge mode.
> > With the flag, fast-tests suite can be generated based on detected
> > hugepage availability of building environment.
> > All cases will be valid if hugepage is available, whereas only
> > applicable cases will be added if environment has no hugepage support.
> >
> > Suggested-by: Aaron Conole <aconole@redhat.com>
> > Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> > Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> > ---
> >  app/test/meson.build | 216
> > ++++++++++++++++++++++++-------------------
> >  1 file changed, 120 insertions(+), 96 deletions(-)
> >
> > diff --git a/app/test/meson.build b/app/test/meson.build index
> > 0a2ce710f..dd121a297 100644
> > --- a/app/test/meson.build
> > +++ b/app/test/meson.build
> > @@ -154,87 +154,89 @@ test_deps = ['acl',
> >         'timer'
> >  ]
> >
> > +# Each test is marked with flag true/false # to indicate whether it
> > +can run in no-huge mode.
> >  fast_test_names = [
> 
> Nit: this variable does not contain a list of names anymore, how about
> "fast_tests".

I'm OK with it.

> 
> > -        'acl_autotest',
> > -        'alarm_autotest',
> > -        'atomic_autotest',
> [snip]
> > +        ['acl_autotest', true],
> > +        ['alarm_autotest', false],
> > +        ['atomic_autotest', false],
> 
> [snip]
> 
> > @@ -395,6 +397,17 @@ dpdk_test = executable('dpdk-test',
> >         install_rpath: driver_install_path,
> >         install: true)
> >
> > +has_hugepage = true
> > +if host_machine.system() == 'linux'
> 
> We have is_OS helpers, here it should be if is_linux.

OK.

> 
> 
> > +       check_hugepage = run_command('cat',
> > +                                    '/proc/sys/vm/nr_hugepages')
> > +       if (check_hugepage.returncode() != 0 or
> > +           check_hugepage.stdout().strip() == '0')
> > +               has_hugepage = false
> > +       endif
> > +endif
> > +message('hugepage availbility: @0@'.format(has_hugepage))
> 
> availability*
> 
> > +
> >  # some perf tests (eg: memcpy perf autotest)take very long  # to
> > complete, so timeout to 10 minutes  timeout_seconds = 600 @@ -407,22
> > +420,33 @@ test_args = [num_cores_arg]
> >
> >  foreach arg : fast_test_names
> >         if (get_option('default_library') == 'shared' and
> > -               arg == 'event_eth_tx_adapter_autotest')
> > +               arg[0] == 'event_eth_tx_adapter_autotest')
> >                 foreach drv:dpdk_drivers
> >                         test_args += ['-d', drv.full_path().split('.a')[0] + '.so']
> >                 endforeach
> >         endif
> >         if host_machine.system() == 'linux'
> > -               test(arg, dpdk_test,
> > -                         env : ['DPDK_TEST=' + arg],
> > -                         args : test_args +
> > -                                ['--file-prefix=@0@'.format(arg)],
> > -               timeout : timeout_seconds_fast,
> > -               is_parallel : false,
> > -               suite : 'fast-tests')
> > +               if has_hugepage
> > +                       test(arg[0], dpdk_test,
> > +                                 env : ['DPDK_TEST=' + arg[0]],
> > +                                 args : test_args +
> > +                                        ['--file-prefix=@0@'.format(arg[0])],
> > +                       timeout : timeout_seconds_fast,
> > +                       is_parallel : false,
> > +                       suite : 'fast-tests')
> > +               elif arg[1]
> > +                       test(arg[0], dpdk_test,
> > +                                 env : ['DPDK_TEST=' + arg[0]],
> > +                                 args : test_args +
> > +                                        ['--no-huge'] + ['-m 2048'] +
> > +                                        ['--file-prefix=@0@'.format(arg[0])],
> > +                       timeout : timeout_seconds_fast,
> > +                       is_parallel : false,
> > +                       suite : 'fast-tests')
> > +               endif
> >         else
> > -               test(arg, dpdk_test,
> > -                       env : ['DPDK_TEST=' + arg],
> > +               test(arg[0], dpdk_test,
> > +                       env : ['DPDK_TEST=' + arg[0]],
> >                         args : test_args,
> >                 timeout : timeout_seconds_fast,
> >                 is_parallel : false,
> 
> 
> Not a fan of these conditionals.
> 
> I sent a little patch fixing an issue I spotted on test_args:
> http://patchwork.dpdk.org/patch/67026/
> 
> I rebased this series on it, see:
> https://github.com/david-marchand/dpdk/commits/ci
> 
> This makes the code easier to read from my pov:
> https://github.com/david-
> marchand/dpdk/blob/ci/app/test/meson.build#L421
> 
> If you are fine with it, I can post a v4 series.

Sure, I'm fine with it.
Thanks.
/Ruifeng
> 
> 
> --
> David Marchand

[dpdk-dev] [PATCH v3 3/4] ci: proceed with verification without hugepage

[Ruifeng Wang]

As fast-tests suite generated with only applicable cases included,
hugepage is not a mandatory to run the test.
Ignore the result of hugepage set up, so that validation in environment
without hugepage can proceed.

Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>
---
 .ci/linux-setup.sh | 11 ++++-------
 1 file changed, 4 insertions(+), 7 deletions(-)

diff --git a/.ci/linux-setup.sh b/.ci/linux-setup.sh
index 2b07d1e0b..5d3089cad 100755
--- a/.ci/linux-setup.sh
+++ b/.ci/linux-setup.sh
@@ -3,10 +3,7 @@
 # need to install as 'root' since some of the unit tests won't run without it
 sudo python3 -m pip install --upgrade 'meson==0.47.1'
 
-# skip hugepage settings if tests will not run
-if [ "$RUN_TESTS" = "1" ]; then
-    # setup hugepages
-    cat /proc/meminfo
-    sudo sh -c 'echo 1024 > /proc/sys/vm/nr_hugepages'
-    cat /proc/meminfo
-fi
+# setup hugepages. error ignored because having hugepage is not mandatory.
+cat /proc/meminfo
+sudo sh -c 'echo 1024 > /proc/sys/vm/nr_hugepages' || true
+cat /proc/meminfo
-- 
2.17.1

Re: [dpdk-dev] [PATCH v3 3/4] ci: proceed with verification without hugepage

[Aaron Conole]

Ruifeng Wang <ruifeng.wang@arm.com> writes:

> As fast-tests suite generated with only applicable cases included,
> hugepage is not a mandatory to run the test.
> Ignore the result of hugepage set up, so that validation in environment
> without hugepage can proceed.
>
> Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> ---

Acked-by: Aaron Conole <aconole@redhat.com>

[dpdk-dev] [PATCH v3 4/4] ci: enable unit test for aarch64

[Ruifeng Wang]

Add Travis CI jobs to run unit tests on aarch64 platform.

Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>
---
 .travis.yml | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/.travis.yml b/.travis.yml
index b64a81bd0..26d6bcb6d 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -101,6 +101,9 @@ jobs:
   - env: DEF_LIB="static"
     arch: arm64
     compiler: gcc
+  - env: DEF_LIB="shared" RUN_TESTS=1
+    arch: arm64
+    compiler: gcc
   - env: DEF_LIB="shared" BUILD_DOCS=1
     arch: arm64
     compiler: gcc
@@ -121,6 +124,6 @@ jobs:
   - env: DEF_LIB="static"
     arch: arm64
     compiler: clang
-  - env: DEF_LIB="shared"
+  - env: DEF_LIB="shared" RUN_TESTS=1
     arch: arm64
     compiler: clang
-- 
2.17.1

Re: [dpdk-dev] [PATCH v3 4/4] ci: enable unit test for aarch64

[Aaron Conole]

Ruifeng Wang <ruifeng.wang@arm.com> writes:

> Add Travis CI jobs to run unit tests on aarch64 platform.
>
> Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> ---

Acked-by: Aaron Conole <aconole@redhat.com>

Re: [dpdk-dev] [PATCH v3 0/4] no-huge unit test

[Aaron Conole]

Ruifeng Wang <ruifeng.wang@arm.com> writes:

> For environments (such as containers) where hugetlbfs are not available,
> some unit tests can be run with 'no-huge' option.
>
> fast-tests suites is generated dynamically according to hugetlbfs
> availability in building environment. This allows unit test to run
> in different environments using the same suite name.
>
> Several test cases are fixed to be able to run in no-huge mode.

This looks great!  Thanks, Ruifeng.

I'm going to ack it once I see it run under the robot :)

> v3:
> Use a single suite instead of create additional one for no-huge. (Aaron)
> Fix several test cases in no-huge mode.
>
> v2:
> Add a patch to enable running multiple suites in a job. (David)
>
>
> Ruifeng Wang (4):
>   test: enable tests to run in no-huge mode
>   ci: generate fast-tests suite base on hugepage availability
>   ci: proceed with verification without hugepage
>   ci: enable unit test for aarch64
>
>  .ci/linux-setup.sh             |  11 +-
>  .travis.yml                    |   5 +-
>  app/test/meson.build           | 216 ++++++++++++++++++---------------
>  app/test/test_acl.c            |  22 ++--
>  app/test/test_hash.c           |   7 +-
>  app/test/test_table_pipeline.c |  12 +-
>  6 files changed, 152 insertions(+), 121 deletions(-)

Re: [dpdk-dev] [PATCH v3 0/4] no-huge unit test

[Aaron Conole]

Aaron Conole <aconole@redhat.com> writes:

> Ruifeng Wang <ruifeng.wang@arm.com> writes:
>
>> For environments (such as containers) where hugetlbfs are not available,
>> some unit tests can be run with 'no-huge' option.
>>
>> fast-tests suites is generated dynamically according to hugetlbfs
>> availability in building environment. This allows unit test to run
>> in different environments using the same suite name.
>>
>> Several test cases are fixed to be able to run in no-huge mode.
>
> This looks great!  Thanks, Ruifeng.
>
> I'm going to ack it once I see it run under the robot :)

Just looking through the robot's run, it seems that on the statically
linked Arm64 build, the disk quota is getting exceeded.  Do we need to
request some more disk quota for this somehow?  Is the build getting too
large?

>> v3:
>> Use a single suite instead of create additional one for no-huge. (Aaron)
>> Fix several test cases in no-huge mode.
>>
>> v2:
>> Add a patch to enable running multiple suites in a job. (David)
>>
>>
>> Ruifeng Wang (4):
>>   test: enable tests to run in no-huge mode
>>   ci: generate fast-tests suite base on hugepage availability
>>   ci: proceed with verification without hugepage
>>   ci: enable unit test for aarch64
>>
>>  .ci/linux-setup.sh             |  11 +-
>>  .travis.yml                    |   5 +-
>>  app/test/meson.build           | 216 ++++++++++++++++++---------------
>>  app/test/test_acl.c            |  22 ++--
>>  app/test/test_hash.c           |   7 +-
>>  app/test/test_table_pipeline.c |  12 +-
>>  6 files changed, 152 insertions(+), 121 deletions(-)

Re: [dpdk-dev] [PATCH v3 0/4] no-huge unit test

[David Marchand]

On Fri, Mar 13, 2020 at 2:04 PM Aaron Conole <aconole@redhat.com> wrote:
>
> Aaron Conole <aconole@redhat.com> writes:
>
> > Ruifeng Wang <ruifeng.wang@arm.com> writes:
> >
> >> For environments (such as containers) where hugetlbfs are not available,
> >> some unit tests can be run with 'no-huge' option.
> >>
> >> fast-tests suites is generated dynamically according to hugetlbfs
> >> availability in building environment. This allows unit test to run
> >> in different environments using the same suite name.
> >>
> >> Several test cases are fixed to be able to run in no-huge mode.
> >
> > This looks great!  Thanks, Ruifeng.
> >
> > I'm going to ack it once I see it run under the robot :)
>
> Just looking through the robot's run, it seems that on the statically
> linked Arm64 build, the disk quota is getting exceeded.  Do we need to
> request some more disk quota for this somehow?  Is the build getting too
> large?

It seems to repeat.
https://travis-ci.com/github/ovsrobot/dpdk/jobs/297840285#L2975

Do you know how much space we have in travis?
Is the (c?)cache getting too big?
You can find out the per job cache size via the travis cli.


-- 
David Marchand

Re: [dpdk-dev] [PATCH v3 0/4] no-huge unit test

[Ruifeng Wang]

> -----Original Message-----
> From: David Marchand <david.marchand@redhat.com>
> Sent: Friday, March 13, 2020 23:54
> To: Aaron Conole <aconole@redhat.com>
> Cc: Ruifeng Wang <Ruifeng.Wang@arm.com>; Michael Santana
> <maicolgabriel@hotmail.com>; Bruce Richardson
> <bruce.richardson@intel.com>; Ananyev, Konstantin
> <konstantin.ananyev@intel.com>; Cristian Dumitrescu
> <cristian.dumitrescu@intel.com>; Wang, Yipeng1
> <yipeng1.wang@intel.com>; Gobriel, Sameh <sameh.gobriel@intel.com>;
> dev <dev@dpdk.org>; Burakov, Anatoly <anatoly.burakov@intel.com>;
> Gavin Hu <Gavin.Hu@arm.com>; Honnappa Nagarahalli
> <Honnappa.Nagarahalli@arm.com>; juraj.linkes@pantheon.tech; nd
> <nd@arm.com>
> Subject: Re: [PATCH v3 0/4] no-huge unit test
> 
> On Fri, Mar 13, 2020 at 2:04 PM Aaron Conole <aconole@redhat.com> wrote:
> >
> > Aaron Conole <aconole@redhat.com> writes:
> >
> > > Ruifeng Wang <ruifeng.wang@arm.com> writes:
> > >
> > >> For environments (such as containers) where hugetlbfs are not
> > >> available, some unit tests can be run with 'no-huge' option.
> > >>
> > >> fast-tests suites is generated dynamically according to hugetlbfs
> > >> availability in building environment. This allows unit test to run
> > >> in different environments using the same suite name.
> > >>
> > >> Several test cases are fixed to be able to run in no-huge mode.
> > >
> > > This looks great!  Thanks, Ruifeng.
> > >
> > > I'm going to ack it once I see it run under the robot :)
> >
> > Just looking through the robot's run, it seems that on the statically
> > linked Arm64 build, the disk quota is getting exceeded.  Do we need to
> > request some more disk quota for this somehow?  Is the build getting
> > too large?
> 
> It seems to repeat.
> https://travis-ci.com/github/ovsrobot/dpdk/jobs/297840285#L2975
> 
> Do you know how much space we have in travis?
> Is the (c?)cache getting too big?

Yes, it is probably caused by cache.
I hit the disk quota issue as well when running Travis against the latest Master code.
After deleting caches, the issue was gone. Then Travis run with this series of patches also got a pass.

Hi Aaron,
Is is OK to clear cache of robot and re-run the build?

Thanks.
/Ruifeng

> You can find out the per job cache size via the travis cli.
> 
> 
> --
> David Marchand

Re: [dpdk-dev] [PATCH v3 0/4] no-huge unit test

[Aaron Conole]

Ruifeng Wang <Ruifeng.Wang@arm.com> writes:

>> -----Original Message-----
>> From: David Marchand <david.marchand@redhat.com>
>> Sent: Friday, March 13, 2020 23:54
>> To: Aaron Conole <aconole@redhat.com>
>> Cc: Ruifeng Wang <Ruifeng.Wang@arm.com>; Michael Santana
>> <maicolgabriel@hotmail.com>; Bruce Richardson
>> <bruce.richardson@intel.com>; Ananyev, Konstantin
>> <konstantin.ananyev@intel.com>; Cristian Dumitrescu
>> <cristian.dumitrescu@intel.com>; Wang, Yipeng1
>> <yipeng1.wang@intel.com>; Gobriel, Sameh <sameh.gobriel@intel.com>;
>> dev <dev@dpdk.org>; Burakov, Anatoly <anatoly.burakov@intel.com>;
>> Gavin Hu <Gavin.Hu@arm.com>; Honnappa Nagarahalli
>> <Honnappa.Nagarahalli@arm.com>; juraj.linkes@pantheon.tech; nd
>> <nd@arm.com>
>> Subject: Re: [PATCH v3 0/4] no-huge unit test
>> 
>> On Fri, Mar 13, 2020 at 2:04 PM Aaron Conole <aconole@redhat.com> wrote:
>> >
>> > Aaron Conole <aconole@redhat.com> writes:
>> >
>> > > Ruifeng Wang <ruifeng.wang@arm.com> writes:
>> > >
>> > >> For environments (such as containers) where hugetlbfs are not
>> > >> available, some unit tests can be run with 'no-huge' option.
>> > >>
>> > >> fast-tests suites is generated dynamically according to hugetlbfs
>> > >> availability in building environment. This allows unit test to run
>> > >> in different environments using the same suite name.
>> > >>
>> > >> Several test cases are fixed to be able to run in no-huge mode.
>> > >
>> > > This looks great!  Thanks, Ruifeng.
>> > >
>> > > I'm going to ack it once I see it run under the robot :)
>> >
>> > Just looking through the robot's run, it seems that on the statically
>> > linked Arm64 build, the disk quota is getting exceeded.  Do we need to
>> > request some more disk quota for this somehow?  Is the build getting
>> > too large?
>> 
>> It seems to repeat.
>> https://travis-ci.com/github/ovsrobot/dpdk/jobs/297840285#L2975
>> 
>> Do you know how much space we have in travis?
>> Is the (c?)cache getting too big?
>
> Yes, it is probably caused by cache.
> I hit the disk quota issue as well when running Travis against the latest Master code.
> After deleting caches, the issue was gone. Then Travis run with this
> series of patches also got a pass.
>
> Hi Aaron,
> Is is OK to clear cache of robot and re-run the build?

I can do this, but I am concerned that we will need to disable the cache
to avoid this error on a consistent basis.

> Thanks.
> /Ruifeng
>
>> You can find out the per job cache size via the travis cli.
>> 
>> 
>> --
>> David Marchand

Re: [dpdk-dev] [PATCH v3 0/4] no-huge unit test

[Aaron Conole]

David Marchand <david.marchand@redhat.com> writes:

> On Fri, Mar 13, 2020 at 2:04 PM Aaron Conole <aconole@redhat.com> wrote:
>>
>> Aaron Conole <aconole@redhat.com> writes:
>>
>> > Ruifeng Wang <ruifeng.wang@arm.com> writes:
>> >
>> >> For environments (such as containers) where hugetlbfs are not available,
>> >> some unit tests can be run with 'no-huge' option.
>> >>
>> >> fast-tests suites is generated dynamically according to hugetlbfs
>> >> availability in building environment. This allows unit test to run
>> >> in different environments using the same suite name.
>> >>
>> >> Several test cases are fixed to be able to run in no-huge mode.
>> >
>> > This looks great!  Thanks, Ruifeng.
>> >
>> > I'm going to ack it once I see it run under the robot :)
>>
>> Just looking through the robot's run, it seems that on the statically
>> linked Arm64 build, the disk quota is getting exceeded.  Do we need to
>> request some more disk quota for this somehow?  Is the build getting too
>> large?
>
> It seems to repeat.
> https://travis-ci.com/github/ovsrobot/dpdk/jobs/297840285#L2975

Yes.

> Do you know how much space we have in travis?

Suppposedly we have 18G on that container... :-/

> Is the (c?)cache getting too big?
> You can find out the per job cache size via the travis cli.

When using the travis CLI:

  $ travis cache
  no caches found

I know this must be untrue, but it seems to not want to send me details
on my system.

I tried going through the API, but the largest cache file I see is 200M,
so I must be misunderstanding something.

Re: [dpdk-dev] [PATCH v3 0/4] no-huge unit test

[Bruce Richardson]

On Mon, Mar 16, 2020 at 10:13:23AM -0400, Aaron Conole wrote:
> David Marchand <david.marchand@redhat.com> writes:
> 
> > On Fri, Mar 13, 2020 at 2:04 PM Aaron Conole <aconole@redhat.com> wrote:
> >>
> >> Aaron Conole <aconole@redhat.com> writes:
> >>
> >> > Ruifeng Wang <ruifeng.wang@arm.com> writes:
> >> >
> >> >> For environments (such as containers) where hugetlbfs are not available,
> >> >> some unit tests can be run with 'no-huge' option.
> >> >>
> >> >> fast-tests suites is generated dynamically according to hugetlbfs
> >> >> availability in building environment. This allows unit test to run
> >> >> in different environments using the same suite name.
> >> >>
> >> >> Several test cases are fixed to be able to run in no-huge mode.
> >> >
> >> > This looks great!  Thanks, Ruifeng.
> >> >
> >> > I'm going to ack it once I see it run under the robot :)
> >>
> >> Just looking through the robot's run, it seems that on the statically
> >> linked Arm64 build, the disk quota is getting exceeded.  Do we need to
> >> request some more disk quota for this somehow?  Is the build getting too
> >> large?
> >
I think as a general rule we need to limit the number of static builds we
do, and possibly skip building all the examples for the static builds.
Given we have almost 50 examples, that's a lot of linking of libraries into
binaries. For meson builds, perhaps we only pass -Dexamples=all for the
shared builds, and maybe do -Dexamples=l3fwd or similar for the static
ones. One or two examplse to check that the linking works with examples
should be enough if we test the build of the examples themselves in the
shared build jobs.

Thoughts?

/Bruce

Re: [dpdk-dev] [PATCH v3 0/4] no-huge unit test

[dwilder]

On 2020-03-13 01:16, Ruifeng Wang wrote:
> For environments (such as containers) where hugetlbfs are not 
> available,
> some unit tests can be run with 'no-huge' option.
> 
> fast-tests suites is generated dynamically according to hugetlbfs
> availability in building environment. This allows unit test to run
> in different environments using the same suite name.
> 
> Several test cases are fixed to be able to run in no-huge mode.
> 
> v3:
> Use a single suite instead of create additional one for no-huge. 
> (Aaron)
> Fix several test cases in no-huge mode.
> 
> v2:
> Add a patch to enable running multiple suites in a job. (David)
> 
> 
> Ruifeng Wang (4):
>   test: enable tests to run in no-huge mode
>   ci: generate fast-tests suite base on hugepage availability
>   ci: proceed with verification without hugepage
>   ci: enable unit test for aarch64
> 
>  .ci/linux-setup.sh             |  11 +-
>  .travis.yml                    |   5 +-
>  app/test/meson.build           | 216 ++++++++++++++++++---------------
>  app/test/test_acl.c            |  22 ++--
>  app/test/test_hash.c           |   7 +-
>  app/test/test_table_pipeline.c |  12 +-
>  6 files changed, 152 insertions(+), 121 deletions(-)


I added Ruifeng's patches on top of my ppc64le ci patches and updated 
the matrix.
Ci ran fine on ppc64le.  
https://travis-ci.org/github/djlwilder/dpdk/jobs/663573687

Tested-by: David Wilder <dwilder@us.ibm.com>