[dpdk-dev] [RFC 0/3] ci: enable unit tests for non-aarch64 platforms

[dpdk-dev] [RFC 0/3] ci: enable unit tests for non-aarch64 platforms

[Aaron Conole]

This series is submitted as an RFC because a number of the unit tests are
not successful in the travis environment.  If all of them were passing,
this would be submitted as PATCH instead.  It could be accepted as-is but I
would prefer to see all the tests passing first.

The first patch fixes up the tests to auto-detect the number of cores on
a machine.  This helps on lower-end systems (such as i3 laptops or something)
where someone wants to verify the functionality.  The number of available
cores on the running system will be picked based on the running system
parameters.

The second patch moves some tests out - these tests don't produce output or
complete in any reasonable amount of time (10m+ for a single unit test is
a little strange - they should be investigated to see if the run time can
be reduced).  I prefer to see these separated out since travis will completely
bail if the test takes longer than 10m to produce output.

The third actually enables the testing, and runs each test leg independently.
This version populates the hugepages mapping.  However, it might be useful
to have the option of running without hugepages enabled (and I have a
separate series that can do this).  However, the --no-huge flag seems to cause
most of the unit tests to break since they either spawn a new instance of
the EAL without passing the hugepage flags, or check against the hugepage API
and use that to determine whether memory can be allocated.

Aaron Conole (3):
  test/meson: auto detect number of cores
  meson-tests: separate slower tests
  ci: enable tests on non-arm platforms

 .ci/linux-build.sh   |  7 +++++++
 .ci/linux-setup.sh   |  6 +++++-
 app/test/meson.build | 43 +++++++++++++++++++++++++++++++++----------
 3 files changed, 45 insertions(+), 11 deletions(-)

-- 
2.19.1

[dpdk-dev] [RFC 0/3] ci: enable unit tests for non-aarch64 platforms

[Aaron Conole]

This series is submitted as an RFC because a number of the unit tests are
not successful in the travis environment.  If all of them were passing,
this would be submitted as PATCH instead.  It could be accepted as-is but I
would prefer to see all the tests passing first.

The first patch fixes up the tests to auto-detect the number of cores on
a machine.  This helps on lower-end systems (such as i3 laptops or something)
where someone wants to verify the functionality.  The number of available
cores on the running system will be picked based on the running system
parameters.

The second patch moves some tests out - these tests don't produce output or
complete in any reasonable amount of time (10m+ for a single unit test is
a little strange - they should be investigated to see if the run time can
be reduced).  I prefer to see these separated out since travis will completely
bail if the test takes longer than 10m to produce output.

The third actually enables the testing, and runs each test leg independently.
This version populates the hugepages mapping.  However, it might be useful
to have the option of running without hugepages enabled (and I have a
separate series that can do this).  However, the --no-huge flag seems to cause
most of the unit tests to break since they either spawn a new instance of
the EAL without passing the hugepage flags, or check against the hugepage API
and use that to determine whether memory can be allocated.

Aaron Conole (3):
  test/meson: auto detect number of cores
  meson-tests: separate slower tests
  ci: enable tests on non-arm platforms

 .ci/linux-build.sh   |  7 +++++++
 .ci/linux-setup.sh   |  6 +++++-
 app/test/meson.build | 43 +++++++++++++++++++++++++++++++++----------
 3 files changed, 45 insertions(+), 11 deletions(-)

-- 
2.19.1

[dpdk-dev] [RFC 1/3] test/meson: auto detect number of cores

[Aaron Conole]

Some environments do not provide a minimum 4 cores for running tests.  This
allows those environments to still execute 'ninja test' without causing
multiple failures.

Signed-off-by: Aaron Conole <aconole@redhat.com>
---
 app/test/meson.build | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/app/test/meson.build b/app/test/meson.build
index ddb4d09ae..975b38daa 100644
--- a/app/test/meson.build
+++ b/app/test/meson.build
@@ -337,10 +337,15 @@ if get_option('tests')
 	timeout_seconds = 600
 	timeout_seconds_fast = 10
 
+	# Retreive the number of CPU cores
+	num_cores = run_command('lscpu', '-p=cpu').stdout().strip().split('\n')[-1]
+	num_cores_arg = '-l 0-' + num_cores
+
+	test_args = [num_cores_arg, '-n 4']
 	foreach arg : fast_parallel_test_names
 		test(arg, dpdk_test,
 			env : ['DPDK_TEST=' + arg],
-			args : ['-c f','-n 4', '--file-prefix=@0@'.format(arg)],
+			args : test_args + ['--file-prefix=@0@'.format(arg)],
 			timeout : timeout_seconds_fast,
 			suite : 'fast-tests')
 	endforeach
@@ -348,6 +353,7 @@ if get_option('tests')
 	foreach arg : fast_non_parallel_test_names
 		test(arg, dpdk_test,
 			env : ['DPDK_TEST=' + arg],
+			args : test_args + ['--file-prefix=@0@'.format(arg)],
 			timeout : timeout_seconds_fast,
 			is_parallel : false,
 			suite : 'fast-tests')
@@ -356,6 +362,7 @@ if get_option('tests')
 	foreach arg : perf_test_names
 		test(arg, dpdk_test,
 		env : ['DPDK_TEST=' + arg],
+		args : test_args + ['--file-prefix=@0@'.format(arg)],
 		timeout : timeout_seconds,
 		is_parallel : false,
 		suite : 'perf-tests')
@@ -364,6 +371,7 @@ if get_option('tests')
 	foreach arg : driver_test_names
 		test(arg, dpdk_test,
 			env : ['DPDK_TEST=' + arg],
+			args : test_args + ['--file-prefix=@0@'.format(arg)],
 			timeout : timeout_seconds,
 			is_parallel : false,
 			suite : 'driver-tests')
@@ -372,6 +380,7 @@ if get_option('tests')
 	foreach arg : dump_test_names
 		test(arg, dpdk_test,
 			env : ['DPDK_TEST=' + arg],
+			args : test_args + ['--file-prefix=@0@'.format(arg)],
 			timeout : timeout_seconds,
 			is_parallel : false,
 			suite : 'debug-tests')
-- 
2.19.1

[dpdk-dev] [RFC 1/3] test/meson: auto detect number of cores

[Aaron Conole]

Some environments do not provide a minimum 4 cores for running tests.  This
allows those environments to still execute 'ninja test' without causing
multiple failures.

Signed-off-by: Aaron Conole <aconole@redhat.com>
---
 app/test/meson.build | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/app/test/meson.build b/app/test/meson.build
index ddb4d09ae..975b38daa 100644
--- a/app/test/meson.build
+++ b/app/test/meson.build
@@ -337,10 +337,15 @@ if get_option('tests')
 	timeout_seconds = 600
 	timeout_seconds_fast = 10
 
+	# Retreive the number of CPU cores
+	num_cores = run_command('lscpu', '-p=cpu').stdout().strip().split('\n')[-1]
+	num_cores_arg = '-l 0-' + num_cores
+
+	test_args = [num_cores_arg, '-n 4']
 	foreach arg : fast_parallel_test_names
 		test(arg, dpdk_test,
 			env : ['DPDK_TEST=' + arg],
-			args : ['-c f','-n 4', '--file-prefix=@0@'.format(arg)],
+			args : test_args + ['--file-prefix=@0@'.format(arg)],
 			timeout : timeout_seconds_fast,
 			suite : 'fast-tests')
 	endforeach
@@ -348,6 +353,7 @@ if get_option('tests')
 	foreach arg : fast_non_parallel_test_names
 		test(arg, dpdk_test,
 			env : ['DPDK_TEST=' + arg],
+			args : test_args + ['--file-prefix=@0@'.format(arg)],
 			timeout : timeout_seconds_fast,
 			is_parallel : false,
 			suite : 'fast-tests')
@@ -356,6 +362,7 @@ if get_option('tests')
 	foreach arg : perf_test_names
 		test(arg, dpdk_test,
 		env : ['DPDK_TEST=' + arg],
+		args : test_args + ['--file-prefix=@0@'.format(arg)],
 		timeout : timeout_seconds,
 		is_parallel : false,
 		suite : 'perf-tests')
@@ -364,6 +371,7 @@ if get_option('tests')
 	foreach arg : driver_test_names
 		test(arg, dpdk_test,
 			env : ['DPDK_TEST=' + arg],
+			args : test_args + ['--file-prefix=@0@'.format(arg)],
 			timeout : timeout_seconds,
 			is_parallel : false,
 			suite : 'driver-tests')
@@ -372,6 +380,7 @@ if get_option('tests')
 	foreach arg : dump_test_names
 		test(arg, dpdk_test,
 			env : ['DPDK_TEST=' + arg],
+			args : test_args + ['--file-prefix=@0@'.format(arg)],
 			timeout : timeout_seconds,
 			is_parallel : false,
 			suite : 'debug-tests')
-- 
2.19.1

Re: [dpdk-dev] [RFC 1/3] test/meson: auto detect number of cores

[David Marchand]

On Fri, Mar 29, 2019 at 6:23 PM Aaron Conole <aconole@redhat.com> wrote:

> Some environments do not provide a minimum 4 cores for running tests.  This
> allows those environments to still execute 'ninja test' without causing
> multiple failures.
>
> Signed-off-by: Aaron Conole <aconole@redhat.com>
> ---
>  app/test/meson.build | 11 ++++++++++-
>  1 file changed, 10 insertions(+), 1 deletion(-)
>
> diff --git a/app/test/meson.build b/app/test/meson.build
> index ddb4d09ae..975b38daa 100644
> --- a/app/test/meson.build
> +++ b/app/test/meson.build
> @@ -337,10 +337,15 @@ if get_option('tests')
>         timeout_seconds = 600
>         timeout_seconds_fast = 10
>
> +       # Retreive the number of CPU cores
> +       num_cores = run_command('lscpu',
> '-p=cpu').stdout().strip().split('\n')[-1]
> +       num_cores_arg = '-l 0-' + num_cores
>

Seeing how we can reduce the core number, we can at least cap it to 4 and
not use all the cores on bigger systems.
But, thinking again, do you know if there is a need for more than 2 cores
in the existing tests ?


+
> +       test_args = [num_cores_arg, '-n 4']
>         foreach arg : fast_parallel_test_names
>                 test(arg, dpdk_test,
>                         env : ['DPDK_TEST=' + arg],
> -                       args : ['-c f','-n 4', '--file-prefix=@0@
> '.format(arg)],
> +                       args : test_args + ['--file-prefix=@0@
> '.format(arg)],
>                         timeout : timeout_seconds_fast,
>                         suite : 'fast-tests')
>         endforeach
> @@ -348,6 +353,7 @@ if get_option('tests')
>         foreach arg : fast_non_parallel_test_names
>                 test(arg, dpdk_test,
>                         env : ['DPDK_TEST=' + arg],
> +                       args : test_args + ['--file-prefix=@0@
> '.format(arg)],
>

I don't understand this part.

(How|) was it working before ?
Is there some default args ? I could not find it.

                        timeout : timeout_seconds_fast,
>                         is_parallel : false,
>                         suite : 'fast-tests')
>


-- 
David Marchand

Re: [dpdk-dev] [RFC 1/3] test/meson: auto detect number of cores

[David Marchand]

On Fri, Mar 29, 2019 at 6:23 PM Aaron Conole <aconole@redhat.com> wrote:

> Some environments do not provide a minimum 4 cores for running tests.  This
> allows those environments to still execute 'ninja test' without causing
> multiple failures.
>
> Signed-off-by: Aaron Conole <aconole@redhat.com>
> ---
>  app/test/meson.build | 11 ++++++++++-
>  1 file changed, 10 insertions(+), 1 deletion(-)
>
> diff --git a/app/test/meson.build b/app/test/meson.build
> index ddb4d09ae..975b38daa 100644
> --- a/app/test/meson.build
> +++ b/app/test/meson.build
> @@ -337,10 +337,15 @@ if get_option('tests')
>         timeout_seconds = 600
>         timeout_seconds_fast = 10
>
> +       # Retreive the number of CPU cores
> +       num_cores = run_command('lscpu',
> '-p=cpu').stdout().strip().split('\n')[-1]
> +       num_cores_arg = '-l 0-' + num_cores
>

Seeing how we can reduce the core number, we can at least cap it to 4 and
not use all the cores on bigger systems.
But, thinking again, do you know if there is a need for more than 2 cores
in the existing tests ?


+
> +       test_args = [num_cores_arg, '-n 4']
>         foreach arg : fast_parallel_test_names
>                 test(arg, dpdk_test,
>                         env : ['DPDK_TEST=' + arg],
> -                       args : ['-c f','-n 4', '--file-prefix=@0@
> '.format(arg)],
> +                       args : test_args + ['--file-prefix=@0@
> '.format(arg)],
>                         timeout : timeout_seconds_fast,
>                         suite : 'fast-tests')
>         endforeach
> @@ -348,6 +353,7 @@ if get_option('tests')
>         foreach arg : fast_non_parallel_test_names
>                 test(arg, dpdk_test,
>                         env : ['DPDK_TEST=' + arg],
> +                       args : test_args + ['--file-prefix=@0@
> '.format(arg)],
>

I don't understand this part.

(How|) was it working before ?
Is there some default args ? I could not find it.

                        timeout : timeout_seconds_fast,
>                         is_parallel : false,
>                         suite : 'fast-tests')
>


-- 
David Marchand

Re: [dpdk-dev] [RFC 1/3] test/meson: auto detect number of cores

[Aaron Conole]

David Marchand <david.marchand@redhat.com> writes:

Thanks for the review, David!

> On Fri, Mar 29, 2019 at 6:23 PM Aaron Conole <aconole@redhat.com> wrote:
>
>  Some environments do not provide a minimum 4 cores for running tests.  This
>  allows those environments to still execute 'ninja test' without causing
>  multiple failures.
>
>  Signed-off-by: Aaron Conole <aconole@redhat.com>
>  ---
>   app/test/meson.build | 11 ++++++++++-
>   1 file changed, 10 insertions(+), 1 deletion(-)
>
>  diff --git a/app/test/meson.build b/app/test/meson.build
>  index ddb4d09ae..975b38daa 100644
>  --- a/app/test/meson.build
>  +++ b/app/test/meson.build
>  @@ -337,10 +337,15 @@ if get_option('tests')
>          timeout_seconds = 600
>          timeout_seconds_fast = 10
>
>  +       # Retreive the number of CPU cores
>  +       num_cores = run_command('lscpu', '-p=cpu').stdout().strip().split('\n')[-1]
>  +       num_cores_arg = '-l 0-' + num_cores
>
> Seeing how we can reduce the core number, we can at least cap it to 4 and not use all the cores on bigger
> systems.
> But, thinking again, do you know if there is a need for more than 2 cores in the existing tests ?

Probably not.  At least, the systems in the travis environment only
provide 2 cores and most of the tests pass.  OTOH, some still fail and I
need to investigate them a bit more.  Some of the test case failures are
ex. eal flags failures and maybe they do pass some '-c f' as part of
the test case, which would fail on systems without 4 cores.

>  +
>  +       test_args = [num_cores_arg, '-n 4']

Whoops!  I think, this '-n 4' option can probably be removed as well.  I
think the memory channel config is optional, and I believe there's no
need to try and tune it.

>          foreach arg : fast_parallel_test_names
>                  test(arg, dpdk_test,
>                          env : ['DPDK_TEST=' + arg],
>  -                       args : ['-c f','-n 4', '--file-prefix=@0@'.format(arg)],
>  +                       args : test_args + ['--file-prefix=@0@'.format(arg)],
>                          timeout : timeout_seconds_fast,
>                          suite : 'fast-tests')
>          endforeach
>  @@ -348,6 +353,7 @@ if get_option('tests')
>          foreach arg : fast_non_parallel_test_names
>                  test(arg, dpdk_test,
>                          env : ['DPDK_TEST=' + arg],
>  +                       args : test_args + ['--file-prefix=@0@'.format(arg)],
>
> I don't understand this part.
>
> (How|) was it working before ?

It did work before.

> Is there some default args ? I could not find it.

I think it does try to do some auto-detection.  I did extend this
because it seemed to make sense, and I have some future work to allow
passing '--no-huge' (in case we have a CI system that doesn't allow
allocating hugepages).  Maybe it doesn't make sense to add that right
now, though.  I'll try without it.

>                          timeout : timeout_seconds_fast,
>                          is_parallel : false,
>                          suite : 'fast-tests')

Re: [dpdk-dev] [RFC 1/3] test/meson: auto detect number of cores

[Aaron Conole]

David Marchand <david.marchand@redhat.com> writes:

Thanks for the review, David!

> On Fri, Mar 29, 2019 at 6:23 PM Aaron Conole <aconole@redhat.com> wrote:
>
>  Some environments do not provide a minimum 4 cores for running tests.  This
>  allows those environments to still execute 'ninja test' without causing
>  multiple failures.
>
>  Signed-off-by: Aaron Conole <aconole@redhat.com>
>  ---
>   app/test/meson.build | 11 ++++++++++-
>   1 file changed, 10 insertions(+), 1 deletion(-)
>
>  diff --git a/app/test/meson.build b/app/test/meson.build
>  index ddb4d09ae..975b38daa 100644
>  --- a/app/test/meson.build
>  +++ b/app/test/meson.build
>  @@ -337,10 +337,15 @@ if get_option('tests')
>          timeout_seconds = 600
>          timeout_seconds_fast = 10
>
>  +       # Retreive the number of CPU cores
>  +       num_cores = run_command('lscpu', '-p=cpu').stdout().strip().split('\n')[-1]
>  +       num_cores_arg = '-l 0-' + num_cores
>
> Seeing how we can reduce the core number, we can at least cap it to 4 and not use all the cores on bigger
> systems.
> But, thinking again, do you know if there is a need for more than 2 cores in the existing tests ?

Probably not.  At least, the systems in the travis environment only
provide 2 cores and most of the tests pass.  OTOH, some still fail and I
need to investigate them a bit more.  Some of the test case failures are
ex. eal flags failures and maybe they do pass some '-c f' as part of
the test case, which would fail on systems without 4 cores.

>  +
>  +       test_args = [num_cores_arg, '-n 4']

Whoops!  I think, this '-n 4' option can probably be removed as well.  I
think the memory channel config is optional, and I believe there's no
need to try and tune it.

>          foreach arg : fast_parallel_test_names
>                  test(arg, dpdk_test,
>                          env : ['DPDK_TEST=' + arg],
>  -                       args : ['-c f','-n 4', '--file-prefix=@0@'.format(arg)],
>  +                       args : test_args + ['--file-prefix=@0@'.format(arg)],
>                          timeout : timeout_seconds_fast,
>                          suite : 'fast-tests')
>          endforeach
>  @@ -348,6 +353,7 @@ if get_option('tests')
>          foreach arg : fast_non_parallel_test_names
>                  test(arg, dpdk_test,
>                          env : ['DPDK_TEST=' + arg],
>  +                       args : test_args + ['--file-prefix=@0@'.format(arg)],
>
> I don't understand this part.
>
> (How|) was it working before ?

It did work before.

> Is there some default args ? I could not find it.

I think it does try to do some auto-detection.  I did extend this
because it seemed to make sense, and I have some future work to allow
passing '--no-huge' (in case we have a CI system that doesn't allow
allocating hugepages).  Maybe it doesn't make sense to add that right
now, though.  I'll try without it.

>                          timeout : timeout_seconds_fast,
>                          is_parallel : false,
>                          suite : 'fast-tests')

Re: [dpdk-dev] [RFC 1/3] test/meson: auto detect number of cores

[Pattan, Reshma]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Aaron Conole
> Sent: Friday, March 29, 2019 5:23 PM
> To: dev@dpdk.org
> Subject: [dpdk-dev] [RFC 1/3] test/meson: auto detect number of cores
> 
> Some environments do not provide a minimum 4 cores for running tests.  This
> allows those environments to still execute 'ninja test' without causing multiple
> failures.
> 

FYI, there is a patch http://patches.dpdk.org/patch/50850/ doing something similar.

--file-prefix is not supported on FreeBSD, so how about you also include the related fix from above patch link into your patch series?


Thanks,
Reshma

Re: [dpdk-dev] [RFC 1/3] test/meson: auto detect number of cores

[Pattan, Reshma]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Aaron Conole
> Sent: Friday, March 29, 2019 5:23 PM
> To: dev@dpdk.org
> Subject: [dpdk-dev] [RFC 1/3] test/meson: auto detect number of cores
> 
> Some environments do not provide a minimum 4 cores for running tests.  This
> allows those environments to still execute 'ninja test' without causing multiple
> failures.
> 

FYI, there is a patch http://patches.dpdk.org/patch/50850/ doing something similar.

--file-prefix is not supported on FreeBSD, so how about you also include the related fix from above patch link into your patch series?


Thanks,
Reshma

Re: [dpdk-dev] [RFC 1/3] test/meson: auto detect number of cores

[Aaron Conole]

"Pattan, Reshma" <reshma.pattan@intel.com> writes:

>> -----Original Message-----
>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Aaron Conole
>> Sent: Friday, March 29, 2019 5:23 PM
>> To: dev@dpdk.org
>> Subject: [dpdk-dev] [RFC 1/3] test/meson: auto detect number of cores
>> 
>> Some environments do not provide a minimum 4 cores for running tests.  This
>> allows those environments to still execute 'ninja test' without causing multiple
>> failures.
>> 
>
> FYI, there is a patch http://patches.dpdk.org/patch/50850/ doing something similar.
>
> --file-prefix is not supported on FreeBSD, so how about you also
> include the related fix from above patch link into your patch series?

Is that other patch going to be accepted?  Then I can drop this patch
from my series.  Otherwise, I'll fold in such a change with my next
version.

> Thanks,
> Reshma

Re: [dpdk-dev] [RFC 1/3] test/meson: auto detect number of cores

[Aaron Conole]

"Pattan, Reshma" <reshma.pattan@intel.com> writes:

>> -----Original Message-----
>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Aaron Conole
>> Sent: Friday, March 29, 2019 5:23 PM
>> To: dev@dpdk.org
>> Subject: [dpdk-dev] [RFC 1/3] test/meson: auto detect number of cores
>> 
>> Some environments do not provide a minimum 4 cores for running tests.  This
>> allows those environments to still execute 'ninja test' without causing multiple
>> failures.
>> 
>
> FYI, there is a patch http://patches.dpdk.org/patch/50850/ doing something similar.
>
> --file-prefix is not supported on FreeBSD, so how about you also
> include the related fix from above patch link into your patch series?

Is that other patch going to be accepted?  Then I can drop this patch
from my series.  Otherwise, I'll fold in such a change with my next
version.

> Thanks,
> Reshma

Re: [dpdk-dev] [RFC 1/3] test/meson: auto detect number of cores

[Thomas Monjalon]

01/04/2019 19:48, Aaron Conole:
> "Pattan, Reshma" <reshma.pattan@intel.com> writes:
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Aaron Conole
> >> 
> >> Some environments do not provide a minimum 4 cores for running tests.  This
> >> allows those environments to still execute 'ninja test' without causing multiple
> >> failures.
> >> 
> >
> > FYI, there is a patch http://patches.dpdk.org/patch/50850/ doing something similar.
> >
> > --file-prefix is not supported on FreeBSD, so how about you also
> > include the related fix from above patch link into your patch series?
> 
> Is that other patch going to be accepted?  Then I can drop this patch
> from my series.  Otherwise, I'll fold in such a change with my next
> version.

Sorry, I'm late on applying patches for unit tests.
As I am not sure which one will get the best review first,
you should keep it with a note that it conflicts with the other one.

Re: [dpdk-dev] [RFC 1/3] test/meson: auto detect number of cores

[Thomas Monjalon]

01/04/2019 19:48, Aaron Conole:
> "Pattan, Reshma" <reshma.pattan@intel.com> writes:
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Aaron Conole
> >> 
> >> Some environments do not provide a minimum 4 cores for running tests.  This
> >> allows those environments to still execute 'ninja test' without causing multiple
> >> failures.
> >> 
> >
> > FYI, there is a patch http://patches.dpdk.org/patch/50850/ doing something similar.
> >
> > --file-prefix is not supported on FreeBSD, so how about you also
> > include the related fix from above patch link into your patch series?
> 
> Is that other patch going to be accepted?  Then I can drop this patch
> from my series.  Otherwise, I'll fold in such a change with my next
> version.

Sorry, I'm late on applying patches for unit tests.
As I am not sure which one will get the best review first,
you should keep it with a note that it conflicts with the other one.

Re: [dpdk-dev] [RFC 1/3] test/meson: auto detect number of cores

[Aaron Conole]

Thomas Monjalon <thomas@monjalon.net> writes:

> 01/04/2019 19:48, Aaron Conole:
>> "Pattan, Reshma" <reshma.pattan@intel.com> writes:
>> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Aaron Conole
>> >> 
>> >> Some environments do not provide a minimum 4 cores for running tests.  This
>> >> allows those environments to still execute 'ninja test' without causing multiple
>> >> failures.
>> >> 
>> >
>> > FYI, there is a patch http://patches.dpdk.org/patch/50850/ doing something similar.
>> >
>> > --file-prefix is not supported on FreeBSD, so how about you also
>> > include the related fix from above patch link into your patch series?
>> 
>> Is that other patch going to be accepted?  Then I can drop this patch
>> from my series.  Otherwise, I'll fold in such a change with my next
>> version.
>
> Sorry, I'm late on applying patches for unit tests.
> As I am not sure which one will get the best review first,
> you should keep it with a note that it conflicts with the other one.

Okay - thanks, Thomas!

Re: [dpdk-dev] [RFC 1/3] test/meson: auto detect number of cores

[Aaron Conole]

Thomas Monjalon <thomas@monjalon.net> writes:

> 01/04/2019 19:48, Aaron Conole:
>> "Pattan, Reshma" <reshma.pattan@intel.com> writes:
>> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Aaron Conole
>> >> 
>> >> Some environments do not provide a minimum 4 cores for running tests.  This
>> >> allows those environments to still execute 'ninja test' without causing multiple
>> >> failures.
>> >> 
>> >
>> > FYI, there is a patch http://patches.dpdk.org/patch/50850/ doing something similar.
>> >
>> > --file-prefix is not supported on FreeBSD, so how about you also
>> > include the related fix from above patch link into your patch series?
>> 
>> Is that other patch going to be accepted?  Then I can drop this patch
>> from my series.  Otherwise, I'll fold in such a change with my next
>> version.
>
> Sorry, I'm late on applying patches for unit tests.
> As I am not sure which one will get the best review first,
> you should keep it with a note that it conflicts with the other one.

Okay - thanks, Thomas!

[dpdk-dev] [RFC 2/3] meson-tests: separate slower tests

[Aaron Conole]

When running some tests in the CI environment, the run time can take
in excess of 10 minutes to produce output.  This causes the entire CI run
not to succeed.  Separate these tests so that they can be run from a
normal environment, and deselected in a CI environment.

There's also a small whitespace fix bundled in for the perf_tests block.

Signed-off-by: Aaron Conole <aconole@redhat.com>
---
 app/test/meson.build | 34 ++++++++++++++++++++++++----------
 1 file changed, 24 insertions(+), 10 deletions(-)

diff --git a/app/test/meson.build b/app/test/meson.build
index 975b38daa..1b0ff0523 100644
--- a/app/test/meson.build
+++ b/app/test/meson.build
@@ -171,7 +171,6 @@ fast_parallel_test_names = [
         'red_autotest',
         'ring_autotest',
         'ring_pmd_autotest',
-        'rwlock_autotest',
         'sched_autotest',
         'spinlock_autotest',
         'string_autotest',
@@ -214,15 +213,21 @@ fast_non_parallel_test_names = [
         'thash_autotest',
 ]
 
+# These test cases are non-parallel and also take > 10m to execute in
+# some environments (such as CI systems), so allow deselecting them
+slow_test_names = [
+        'rwlock_autotest',
+        'mempool_perf_autotest',
+        'reciprocal_division',
+        'reciprocal_division_perf',
+]
+
 # All test cases in perf_test_names list are non-parallel
 perf_test_names = [
         'ring_perf_autotest',
-        'mempool_perf_autotest',
         'memcpy_perf_autotest',
         'hash_perf_autotest',
         'timer_perf_autotest',
-        'reciprocal_division',
-        'reciprocal_division_perf',
         'lpm_perf_autotest',
         'red_all',
         'barrier_autotest',
@@ -332,7 +337,7 @@ if get_option('tests')
 		install_rpath: driver_install_path,
 		install: false)
 
-	# some perf tests (eg: memcpy perf autotest)take very long
+	# some tests (eg: the slow tests) take very long
 	# to complete, so timeout to 10 minutes
 	timeout_seconds = 600
 	timeout_seconds_fast = 10
@@ -359,13 +364,22 @@ if get_option('tests')
 			suite : 'fast-tests')
 	endforeach
 
+	foreach arg : slow_test_names
+		test(arg, dpdk_test,
+			env : ['DPDK_TEST=' + arg],
+			args : test_args + ['--file-prefix=@0@'.format(arg)],
+			timeout : timeout_seconds,
+			is_parallel : false,
+			suite : 'slow-tests')
+	endforeach
+
 	foreach arg : perf_test_names
 		test(arg, dpdk_test,
-		env : ['DPDK_TEST=' + arg],
-		args : test_args + ['--file-prefix=@0@'.format(arg)],
-		timeout : timeout_seconds,
-		is_parallel : false,
-		suite : 'perf-tests')
+			env : ['DPDK_TEST=' + arg],
+			args : test_args + ['--file-prefix=@0@'.format(arg)],
+			timeout : timeout_seconds,
+			is_parallel : false,
+			suite : 'perf-tests')
 	endforeach
 
 	foreach arg : driver_test_names
-- 
2.19.1

[dpdk-dev] [RFC 2/3] meson-tests: separate slower tests

[Aaron Conole]

When running some tests in the CI environment, the run time can take
in excess of 10 minutes to produce output.  This causes the entire CI run
not to succeed.  Separate these tests so that they can be run from a
normal environment, and deselected in a CI environment.

There's also a small whitespace fix bundled in for the perf_tests block.

Signed-off-by: Aaron Conole <aconole@redhat.com>
---
 app/test/meson.build | 34 ++++++++++++++++++++++++----------
 1 file changed, 24 insertions(+), 10 deletions(-)

diff --git a/app/test/meson.build b/app/test/meson.build
index 975b38daa..1b0ff0523 100644
--- a/app/test/meson.build
+++ b/app/test/meson.build
@@ -171,7 +171,6 @@ fast_parallel_test_names = [
         'red_autotest',
         'ring_autotest',
         'ring_pmd_autotest',
-        'rwlock_autotest',
         'sched_autotest',
         'spinlock_autotest',
         'string_autotest',
@@ -214,15 +213,21 @@ fast_non_parallel_test_names = [
         'thash_autotest',
 ]
 
+# These test cases are non-parallel and also take > 10m to execute in
+# some environments (such as CI systems), so allow deselecting them
+slow_test_names = [
+        'rwlock_autotest',
+        'mempool_perf_autotest',
+        'reciprocal_division',
+        'reciprocal_division_perf',
+]
+
 # All test cases in perf_test_names list are non-parallel
 perf_test_names = [
         'ring_perf_autotest',
-        'mempool_perf_autotest',
         'memcpy_perf_autotest',
         'hash_perf_autotest',
         'timer_perf_autotest',
-        'reciprocal_division',
-        'reciprocal_division_perf',
         'lpm_perf_autotest',
         'red_all',
         'barrier_autotest',
@@ -332,7 +337,7 @@ if get_option('tests')
 		install_rpath: driver_install_path,
 		install: false)
 
-	# some perf tests (eg: memcpy perf autotest)take very long
+	# some tests (eg: the slow tests) take very long
 	# to complete, so timeout to 10 minutes
 	timeout_seconds = 600
 	timeout_seconds_fast = 10
@@ -359,13 +364,22 @@ if get_option('tests')
 			suite : 'fast-tests')
 	endforeach
 
+	foreach arg : slow_test_names
+		test(arg, dpdk_test,
+			env : ['DPDK_TEST=' + arg],
+			args : test_args + ['--file-prefix=@0@'.format(arg)],
+			timeout : timeout_seconds,
+			is_parallel : false,
+			suite : 'slow-tests')
+	endforeach
+
 	foreach arg : perf_test_names
 		test(arg, dpdk_test,
-		env : ['DPDK_TEST=' + arg],
-		args : test_args + ['--file-prefix=@0@'.format(arg)],
-		timeout : timeout_seconds,
-		is_parallel : false,
-		suite : 'perf-tests')
+			env : ['DPDK_TEST=' + arg],
+			args : test_args + ['--file-prefix=@0@'.format(arg)],
+			timeout : timeout_seconds,
+			is_parallel : false,
+			suite : 'perf-tests')
 	endforeach
 
 	foreach arg : driver_test_names
-- 
2.19.1

[dpdk-dev] [RFC 3/3] ci: enable tests on non-arm platforms

[Aaron Conole]

When building under Travis (or another linux CI service), enable running the
tests for each build.

Signed-off-by: Aaron Conole <aconole@redhat.com>
---
 .ci/linux-build.sh | 7 +++++++
 .ci/linux-setup.sh | 6 +++++-
 2 files changed, 12 insertions(+), 1 deletion(-)

diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh
index 4eb7c3cf0..72510a433 100755
--- a/.ci/linux-build.sh
+++ b/.ci/linux-build.sh
@@ -22,3 +22,10 @@ fi
 OPTS="$OPTS --default-library=$DEF_LIB"
 meson build --werror -Dexamples=all $OPTS
 ninja -C build
+
+if [ "$AARCH64" != "1" ]; then
+    sudo meson test -C build --suite fast-tests
+    sudo meson test -C build --suite perf-tests
+    sudo meson test -C build --suite driver-tests
+    sudo meson test -C build --suite debug-tests
+fi
diff --git a/.ci/linux-setup.sh b/.ci/linux-setup.sh
index acdf9f370..18d2b22fc 100755
--- a/.ci/linux-setup.sh
+++ b/.ci/linux-setup.sh
@@ -1,3 +1,7 @@
 #!/bin/sh
 
-python3 -m pip install --upgrade meson --user
+sudo python3 -m pip install --upgrade meson
+
+cat /proc/meminfo
+sudo sh -c 'echo 1024 > /proc/sys/vm/nr_hugepages'
+cat /proc/meminfo
-- 
2.19.1

[dpdk-dev] [RFC 3/3] ci: enable tests on non-arm platforms

[Aaron Conole]

When building under Travis (or another linux CI service), enable running the
tests for each build.

Signed-off-by: Aaron Conole <aconole@redhat.com>
---
 .ci/linux-build.sh | 7 +++++++
 .ci/linux-setup.sh | 6 +++++-
 2 files changed, 12 insertions(+), 1 deletion(-)

diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh
index 4eb7c3cf0..72510a433 100755
--- a/.ci/linux-build.sh
+++ b/.ci/linux-build.sh
@@ -22,3 +22,10 @@ fi
 OPTS="$OPTS --default-library=$DEF_LIB"
 meson build --werror -Dexamples=all $OPTS
 ninja -C build
+
+if [ "$AARCH64" != "1" ]; then
+    sudo meson test -C build --suite fast-tests
+    sudo meson test -C build --suite perf-tests
+    sudo meson test -C build --suite driver-tests
+    sudo meson test -C build --suite debug-tests
+fi
diff --git a/.ci/linux-setup.sh b/.ci/linux-setup.sh
index acdf9f370..18d2b22fc 100755
--- a/.ci/linux-setup.sh
+++ b/.ci/linux-setup.sh
@@ -1,3 +1,7 @@
 #!/bin/sh
 
-python3 -m pip install --upgrade meson --user
+sudo python3 -m pip install --upgrade meson
+
+cat /proc/meminfo
+sudo sh -c 'echo 1024 > /proc/sys/vm/nr_hugepages'
+cat /proc/meminfo
-- 
2.19.1

Re: [dpdk-dev] [RFC 3/3] ci: enable tests on non-arm platforms

[Jerin Jacob Kollanukkaran]

> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Aaron Conole
> Sent: Friday, March 29, 2019 10:53 PM
> To: dev@dpdk.org
> Subject: [dpdk-dev] [RFC 3/3] ci: enable tests on non-arm platforms
> 
> When building under Travis (or another linux CI service), enable running the tests
> for each build.
> 
> Signed-off-by: Aaron Conole <aconole@redhat.com>
> ---
>  .ci/linux-build.sh | 7 +++++++
>  .ci/linux-setup.sh | 6 +++++-
>  2 files changed, 12 insertions(+), 1 deletion(-)
> 
> diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh index 4eb7c3cf0..72510a433
> 100755
> --- a/.ci/linux-build.sh
> +++ b/.ci/linux-build.sh
> @@ -22,3 +22,10 @@ fi
>  OPTS="$OPTS --default-library=$DEF_LIB"
>  meson build --werror -Dexamples=all $OPTS  ninja -C build
> +
> +if [ "$AARCH64" != "1" ]; then

If the indention is run only on X86 then, X86 == 1 would be the correct logic as we do
have powerpc and arm32 support in dpdk.

Any reason why we are opting out arm64?
There are some cloud hosting services with arm64 as well.

https://www.packet.com/cloud/servers/c1-large-arm/

Re: [dpdk-dev] [RFC 3/3] ci: enable tests on non-arm platforms

[Jerin Jacob Kollanukkaran]

> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Aaron Conole
> Sent: Friday, March 29, 2019 10:53 PM
> To: dev@dpdk.org
> Subject: [dpdk-dev] [RFC 3/3] ci: enable tests on non-arm platforms
> 
> When building under Travis (or another linux CI service), enable running the tests
> for each build.
> 
> Signed-off-by: Aaron Conole <aconole@redhat.com>
> ---
>  .ci/linux-build.sh | 7 +++++++
>  .ci/linux-setup.sh | 6 +++++-
>  2 files changed, 12 insertions(+), 1 deletion(-)
> 
> diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh index 4eb7c3cf0..72510a433
> 100755
> --- a/.ci/linux-build.sh
> +++ b/.ci/linux-build.sh
> @@ -22,3 +22,10 @@ fi
>  OPTS="$OPTS --default-library=$DEF_LIB"
>  meson build --werror -Dexamples=all $OPTS  ninja -C build
> +
> +if [ "$AARCH64" != "1" ]; then

If the indention is run only on X86 then, X86 == 1 would be the correct logic as we do
have powerpc and arm32 support in dpdk.

Any reason why we are opting out arm64?
There are some cloud hosting services with arm64 as well.

https://www.packet.com/cloud/servers/c1-large-arm/

Re: [dpdk-dev] [RFC 3/3] ci: enable tests on non-arm platforms

[Aaron Conole]

Jerin Jacob Kollanukkaran <jerinj@marvell.com> writes:

>> -----Original Message-----
>> From: dev <dev-bounces@dpdk.org> On Behalf Of Aaron Conole
>> Sent: Friday, March 29, 2019 10:53 PM
>> To: dev@dpdk.org
>> Subject: [dpdk-dev] [RFC 3/3] ci: enable tests on non-arm platforms
>> 
>> When building under Travis (or another linux CI service), enable running the tests
>> for each build.
>> 
>> Signed-off-by: Aaron Conole <aconole@redhat.com>
>> ---
>>  .ci/linux-build.sh | 7 +++++++
>>  .ci/linux-setup.sh | 6 +++++-
>>  2 files changed, 12 insertions(+), 1 deletion(-)
>> 
>> diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh index 4eb7c3cf0..72510a433
>> 100755
>> --- a/.ci/linux-build.sh
>> +++ b/.ci/linux-build.sh
>> @@ -22,3 +22,10 @@ fi
>>  OPTS="$OPTS --default-library=$DEF_LIB"
>>  meson build --werror -Dexamples=all $OPTS  ninja -C build
>> +
>> +if [ "$AARCH64" != "1" ]; then
>
> If the indention is run only on X86 then, X86 == 1 would be the correct logic as we do
> have powerpc and arm32 support in dpdk.

Correct - I should have used a more robust and targeted test.

> Any reason why we are opting out arm64?
> There are some cloud hosting services with arm64 as well.
>
> https://www.packet.com/cloud/servers/c1-large-arm/

There is  At the moment, we aren't hooked up to any - do you know of one
that we can easily integrate with right now?  AFAICT, there are hosting
services, but we would still need all the other things that go with CI
service (dashboards, scheduling, tracking, etc.) and it's a big project
to set up, maintain, etc.

Re: [dpdk-dev] [RFC 3/3] ci: enable tests on non-arm platforms

[Aaron Conole]

Jerin Jacob Kollanukkaran <jerinj@marvell.com> writes:

>> -----Original Message-----
>> From: dev <dev-bounces@dpdk.org> On Behalf Of Aaron Conole
>> Sent: Friday, March 29, 2019 10:53 PM
>> To: dev@dpdk.org
>> Subject: [dpdk-dev] [RFC 3/3] ci: enable tests on non-arm platforms
>> 
>> When building under Travis (or another linux CI service), enable running the tests
>> for each build.
>> 
>> Signed-off-by: Aaron Conole <aconole@redhat.com>
>> ---
>>  .ci/linux-build.sh | 7 +++++++
>>  .ci/linux-setup.sh | 6 +++++-
>>  2 files changed, 12 insertions(+), 1 deletion(-)
>> 
>> diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh index 4eb7c3cf0..72510a433
>> 100755
>> --- a/.ci/linux-build.sh
>> +++ b/.ci/linux-build.sh
>> @@ -22,3 +22,10 @@ fi
>>  OPTS="$OPTS --default-library=$DEF_LIB"
>>  meson build --werror -Dexamples=all $OPTS  ninja -C build
>> +
>> +if [ "$AARCH64" != "1" ]; then
>
> If the indention is run only on X86 then, X86 == 1 would be the correct logic as we do
> have powerpc and arm32 support in dpdk.

Correct - I should have used a more robust and targeted test.

> Any reason why we are opting out arm64?
> There are some cloud hosting services with arm64 as well.
>
> https://www.packet.com/cloud/servers/c1-large-arm/

There is  At the moment, we aren't hooked up to any - do you know of one
that we can easily integrate with right now?  AFAICT, there are hosting
services, but we would still need all the other things that go with CI
service (dashboards, scheduling, tracking, etc.) and it's a big project
to set up, maintain, etc.

Re: [dpdk-dev] [EXT] Re: [RFC 3/3] ci: enable tests on non-arm platforms

[Jerin Jacob Kollanukkaran]

> -----Original Message-----
> From: Aaron Conole <aconole@redhat.com>
> Sent: Wednesday, April 17, 2019 6:10 PM
> To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>
> Cc: dev@dpdk.org
> Subject: [EXT] Re: [dpdk-dev] [RFC 3/3] ci: enable tests on non-arm platforms
> 
> 
----------------------------------------------------------------------
> Jerin Jacob Kollanukkaran <jerinj@marvell.com> writes:
> 
> >> -----Original Message-----
> >> From: dev <dev-bounces@dpdk.org> On Behalf Of Aaron Conole
> >> Sent: Friday, March 29, 2019 10:53 PM
> >> To: dev@dpdk.org
> >> Subject: [dpdk-dev] [RFC 3/3] ci: enable tests on non-arm platforms
> >>
> >> When building under Travis (or another linux CI service), enable
> >> running the tests for each build.
> >>
> >> Signed-off-by: Aaron Conole <aconole@redhat.com>
> >> ---
> >>  .ci/linux-build.sh | 7 +++++++
> >>  .ci/linux-setup.sh | 6 +++++-
> >>  2 files changed, 12 insertions(+), 1 deletion(-)
> >>
> >> diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh index
> >> 4eb7c3cf0..72510a433
> >> 100755
> >> --- a/.ci/linux-build.sh
> >> +++ b/.ci/linux-build.sh
> >> @@ -22,3 +22,10 @@ fi
> >>  OPTS="$OPTS --default-library=$DEF_LIB"
> >>  meson build --werror -Dexamples=all $OPTS  ninja -C build
> >> +
> >> +if [ "$AARCH64" != "1" ]; then
> >
> > If the indention is run only on X86 then, X86 == 1 would be the
> > correct logic as we do have powerpc and arm32 support in dpdk.
> 
> Correct - I should have used a more robust and targeted test.
> 
> > Any reason why we are opting out arm64?
> > There are some cloud hosting services with arm64 as well.
> >
> > https://www.packet.com/cloud/servers/c1-large-arm/
> 
> There is  At the moment, we aren't hooked up to any - do you know of one that
> we can easily integrate with right now?  AFAICT, there are hosting services, but

I know, Following cloud hosting providers has arm64 support.
But, not sure about the feature set for CI work.
(2) and (3) are based on ThunderX servers.

https://www.linaro.cloud/
https://www.packet.com/cloud/servers/c1-large-arm/
www.scaleway.com



> we would still need all the other things that go with CI service (dashboards,
> scheduling, tracking, etc.) and it's a big project to set up, maintain, etc.

Re: [dpdk-dev] [EXT] Re: [RFC 3/3] ci: enable tests on non-arm platforms

[Jerin Jacob Kollanukkaran]

> -----Original Message-----
> From: Aaron Conole <aconole@redhat.com>
> Sent: Wednesday, April 17, 2019 6:10 PM
> To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>
> Cc: dev@dpdk.org
> Subject: [EXT] Re: [dpdk-dev] [RFC 3/3] ci: enable tests on non-arm platforms
> 
> 
----------------------------------------------------------------------
> Jerin Jacob Kollanukkaran <jerinj@marvell.com> writes:
> 
> >> -----Original Message-----
> >> From: dev <dev-bounces@dpdk.org> On Behalf Of Aaron Conole
> >> Sent: Friday, March 29, 2019 10:53 PM
> >> To: dev@dpdk.org
> >> Subject: [dpdk-dev] [RFC 3/3] ci: enable tests on non-arm platforms
> >>
> >> When building under Travis (or another linux CI service), enable
> >> running the tests for each build.
> >>
> >> Signed-off-by: Aaron Conole <aconole@redhat.com>
> >> ---
> >>  .ci/linux-build.sh | 7 +++++++
> >>  .ci/linux-setup.sh | 6 +++++-
> >>  2 files changed, 12 insertions(+), 1 deletion(-)
> >>
> >> diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh index
> >> 4eb7c3cf0..72510a433
> >> 100755
> >> --- a/.ci/linux-build.sh
> >> +++ b/.ci/linux-build.sh
> >> @@ -22,3 +22,10 @@ fi
> >>  OPTS="$OPTS --default-library=$DEF_LIB"
> >>  meson build --werror -Dexamples=all $OPTS  ninja -C build
> >> +
> >> +if [ "$AARCH64" != "1" ]; then
> >
> > If the indention is run only on X86 then, X86 == 1 would be the
> > correct logic as we do have powerpc and arm32 support in dpdk.
> 
> Correct - I should have used a more robust and targeted test.
> 
> > Any reason why we are opting out arm64?
> > There are some cloud hosting services with arm64 as well.
> >
> > https://www.packet.com/cloud/servers/c1-large-arm/
> 
> There is  At the moment, we aren't hooked up to any - do you know of one that
> we can easily integrate with right now?  AFAICT, there are hosting services, but

I know, Following cloud hosting providers has arm64 support.
But, not sure about the feature set for CI work.
(2) and (3) are based on ThunderX servers.

https://www.linaro.cloud/
https://www.packet.com/cloud/servers/c1-large-arm/
www.scaleway.com



> we would still need all the other things that go with CI service (dashboards,
> scheduling, tracking, etc.) and it's a big project to set up, maintain, etc.

Re: [dpdk-dev] [RFC 0/3] ci: enable unit tests for non-aarch64 platforms

[David Marchand]

On Fri, Mar 29, 2019 at 6:23 PM Aaron Conole <aconole@redhat.com> wrote:

> This series is submitted as an RFC because a number of the unit tests are
> not successful in the travis environment.  If all of them were passing,
> this would be submitted as PATCH instead.  It could be accepted as-is but I
> would prefer to see all the tests passing first.
>
> The first patch fixes up the tests to auto-detect the number of cores on
> a machine.  This helps on lower-end systems (such as i3 laptops or
> something)
> where someone wants to verify the functionality.  The number of available
> cores on the running system will be picked based on the running system
> parameters.
>
> The second patch moves some tests out - these tests don't produce output or
> complete in any reasonable amount of time (10m+ for a single unit test is
> a little strange - they should be investigated to see if the run time can
> be reduced).  I prefer to see these separated out since travis will
> completely
> bail if the test takes longer than 10m to produce output.
>
> The third actually enables the testing, and runs each test leg
> independently.
> This version populates the hugepages mapping.  However, it might be useful
> to have the option of running without hugepages enabled (and I have a
> separate series that can do this).  However, the --no-huge flag seems to
> cause
> most of the unit tests to break since they either spawn a new instance of
> the EAL without passing the hugepage flags, or check against the hugepage
> API
> and use that to determine whether memory can be allocated.
>
> Aaron Conole (3):
>   test/meson: auto detect number of cores
>   meson-tests: separate slower tests
>   ci: enable tests on non-arm platforms
>
>  .ci/linux-build.sh   |  7 +++++++
>  .ci/linux-setup.sh   |  6 +++++-
>  app/test/meson.build | 43 +++++++++++++++++++++++++++++++++----------
>  3 files changed, 45 insertions(+), 11 deletions(-)
>


I tried using meson/ninja for the tests, something that bothered me is that
I can't interrupt the tests.
I had to kill manually, meson, ninja and I had some leftover dpdk-test
processes (maybe due to some ^Z I hit...).
Is this expected ?

This is quite frustrating when testing "before" and "after" each patch.


-- 
David Marchand

Re: [dpdk-dev] [RFC 0/3] ci: enable unit tests for non-aarch64 platforms

[David Marchand]

On Fri, Mar 29, 2019 at 6:23 PM Aaron Conole <aconole@redhat.com> wrote:

> This series is submitted as an RFC because a number of the unit tests are
> not successful in the travis environment.  If all of them were passing,
> this would be submitted as PATCH instead.  It could be accepted as-is but I
> would prefer to see all the tests passing first.
>
> The first patch fixes up the tests to auto-detect the number of cores on
> a machine.  This helps on lower-end systems (such as i3 laptops or
> something)
> where someone wants to verify the functionality.  The number of available
> cores on the running system will be picked based on the running system
> parameters.
>
> The second patch moves some tests out - these tests don't produce output or
> complete in any reasonable amount of time (10m+ for a single unit test is
> a little strange - they should be investigated to see if the run time can
> be reduced).  I prefer to see these separated out since travis will
> completely
> bail if the test takes longer than 10m to produce output.
>
> The third actually enables the testing, and runs each test leg
> independently.
> This version populates the hugepages mapping.  However, it might be useful
> to have the option of running without hugepages enabled (and I have a
> separate series that can do this).  However, the --no-huge flag seems to
> cause
> most of the unit tests to break since they either spawn a new instance of
> the EAL without passing the hugepage flags, or check against the hugepage
> API
> and use that to determine whether memory can be allocated.
>
> Aaron Conole (3):
>   test/meson: auto detect number of cores
>   meson-tests: separate slower tests
>   ci: enable tests on non-arm platforms
>
>  .ci/linux-build.sh   |  7 +++++++
>  .ci/linux-setup.sh   |  6 +++++-
>  app/test/meson.build | 43 +++++++++++++++++++++++++++++++++----------
>  3 files changed, 45 insertions(+), 11 deletions(-)
>


I tried using meson/ninja for the tests, something that bothered me is that
I can't interrupt the tests.
I had to kill manually, meson, ninja and I had some leftover dpdk-test
processes (maybe due to some ^Z I hit...).
Is this expected ?

This is quite frustrating when testing "before" and "after" each patch.


-- 
David Marchand

Re: [dpdk-dev] [RFC 0/3] ci: enable unit tests for non-aarch64 platforms

[Aaron Conole]

David Marchand <david.marchand@redhat.com> writes:

> On Fri, Mar 29, 2019 at 6:23 PM Aaron Conole <aconole@redhat.com> wrote:
>
>  This series is submitted as an RFC because a number of the unit tests are
>  not successful in the travis environment.  If all of them were passing,
>  this would be submitted as PATCH instead.  It could be accepted as-is but I
>  would prefer to see all the tests passing first.
>
>  The first patch fixes up the tests to auto-detect the number of cores on
>  a machine.  This helps on lower-end systems (such as i3 laptops or something)
>  where someone wants to verify the functionality.  The number of available
>  cores on the running system will be picked based on the running system
>  parameters.
>
>  The second patch moves some tests out - these tests don't produce output or
>  complete in any reasonable amount of time (10m+ for a single unit test is
>  a little strange - they should be investigated to see if the run time can
>  be reduced).  I prefer to see these separated out since travis will completely
>  bail if the test takes longer than 10m to produce output.
>
>  The third actually enables the testing, and runs each test leg independently.
>  This version populates the hugepages mapping.  However, it might be useful
>  to have the option of running without hugepages enabled (and I have a
>  separate series that can do this).  However, the --no-huge flag seems to cause
>  most of the unit tests to break since they either spawn a new instance of
>  the EAL without passing the hugepage flags, or check against the hugepage API
>  and use that to determine whether memory can be allocated.
>
>  Aaron Conole (3):
>    test/meson: auto detect number of cores
>    meson-tests: separate slower tests
>    ci: enable tests on non-arm platforms
>
>   .ci/linux-build.sh   |  7 +++++++
>   .ci/linux-setup.sh   |  6 +++++-
>   app/test/meson.build | 43 +++++++++++++++++++++++++++++++++----------
>   3 files changed, 45 insertions(+), 11 deletions(-)
>
> I tried using meson/ninja for the tests, something that bothered me is that I can't interrupt the tests.
> I had to kill manually, meson, ninja and I had some leftover dpdk-test processes (maybe due to some ^Z I
> hit...).
> Is this expected ?

Certainly not by me.  I usually let everything complete, though (which
takes a looong time if I run the full suite).

> This is quite frustrating when testing "before" and "after" each patch.

Agreed.  :-/

I'll have to try it out to see what's happening.  Does it only happen
with this series?  I'd be surprised, but possibly I introduced some error.

Re: [dpdk-dev] [RFC 0/3] ci: enable unit tests for non-aarch64 platforms

[Aaron Conole]

David Marchand <david.marchand@redhat.com> writes:

> On Fri, Mar 29, 2019 at 6:23 PM Aaron Conole <aconole@redhat.com> wrote:
>
>  This series is submitted as an RFC because a number of the unit tests are
>  not successful in the travis environment.  If all of them were passing,
>  this would be submitted as PATCH instead.  It could be accepted as-is but I
>  would prefer to see all the tests passing first.
>
>  The first patch fixes up the tests to auto-detect the number of cores on
>  a machine.  This helps on lower-end systems (such as i3 laptops or something)
>  where someone wants to verify the functionality.  The number of available
>  cores on the running system will be picked based on the running system
>  parameters.
>
>  The second patch moves some tests out - these tests don't produce output or
>  complete in any reasonable amount of time (10m+ for a single unit test is
>  a little strange - they should be investigated to see if the run time can
>  be reduced).  I prefer to see these separated out since travis will completely
>  bail if the test takes longer than 10m to produce output.
>
>  The third actually enables the testing, and runs each test leg independently.
>  This version populates the hugepages mapping.  However, it might be useful
>  to have the option of running without hugepages enabled (and I have a
>  separate series that can do this).  However, the --no-huge flag seems to cause
>  most of the unit tests to break since they either spawn a new instance of
>  the EAL without passing the hugepage flags, or check against the hugepage API
>  and use that to determine whether memory can be allocated.
>
>  Aaron Conole (3):
>    test/meson: auto detect number of cores
>    meson-tests: separate slower tests
>    ci: enable tests on non-arm platforms
>
>   .ci/linux-build.sh   |  7 +++++++
>   .ci/linux-setup.sh   |  6 +++++-
>   app/test/meson.build | 43 +++++++++++++++++++++++++++++++++----------
>   3 files changed, 45 insertions(+), 11 deletions(-)
>
> I tried using meson/ninja for the tests, something that bothered me is that I can't interrupt the tests.
> I had to kill manually, meson, ninja and I had some leftover dpdk-test processes (maybe due to some ^Z I
> hit...).
> Is this expected ?

Certainly not by me.  I usually let everything complete, though (which
takes a looong time if I run the full suite).

> This is quite frustrating when testing "before" and "after" each patch.

Agreed.  :-/

I'll have to try it out to see what's happening.  Does it only happen
with this series?  I'd be surprised, but possibly I introduced some error.

Re: [dpdk-dev] [RFC 0/3] ci: enable unit tests for non-aarch64 platforms

[David Marchand]

On Mon, Apr 1, 2019 at 9:28 PM Aaron Conole <aconole@redhat.com> wrote:

> David Marchand <david.marchand@redhat.com> writes:
> > I tried using meson/ninja for the tests, something that bothered me is
> that I can't interrupt the tests.
> > I had to kill manually, meson, ninja and I had some leftover dpdk-test
> processes (maybe due to some ^Z I
> > hit...).
> > Is this expected ?
>
> Certainly not by me.  I usually let everything complete, though (which
> takes a looong time if I run the full suite).
>
> > This is quite frustrating when testing "before" and "after" each patch.
>
> Agreed.  :-/
>
> I'll have to try it out to see what's happening.  Does it only happen
> with this series?  I'd be surprised, but possibly I introduced some error.
>

Nop, I got this even before your first patch.


-- 
David Marchand

Re: [dpdk-dev] [RFC 0/3] ci: enable unit tests for non-aarch64 platforms

[David Marchand]

On Mon, Apr 1, 2019 at 9:28 PM Aaron Conole <aconole@redhat.com> wrote:

> David Marchand <david.marchand@redhat.com> writes:
> > I tried using meson/ninja for the tests, something that bothered me is
> that I can't interrupt the tests.
> > I had to kill manually, meson, ninja and I had some leftover dpdk-test
> processes (maybe due to some ^Z I
> > hit...).
> > Is this expected ?
>
> Certainly not by me.  I usually let everything complete, though (which
> takes a looong time if I run the full suite).
>
> > This is quite frustrating when testing "before" and "after" each patch.
>
> Agreed.  :-/
>
> I'll have to try it out to see what's happening.  Does it only happen
> with this series?  I'd be surprised, but possibly I introduced some error.
>

Nop, I got this even before your first patch.


-- 
David Marchand

Re: [dpdk-dev] [RFC 0/3] ci: enable unit tests for non-aarch64 platforms

[Bruce Richardson]

On Mon, Apr 01, 2019 at 09:29:51PM +0200, David Marchand wrote:
> On Mon, Apr 1, 2019 at 9:28 PM Aaron Conole <aconole@redhat.com> wrote:
> 
> > David Marchand <david.marchand@redhat.com> writes:
> > > I tried using meson/ninja for the tests, something that bothered me is
> > that I can't interrupt the tests.
> > > I had to kill manually, meson, ninja and I had some leftover dpdk-test
> > processes (maybe due to some ^Z I
> > > hit...).
> > > Is this expected ?
> >
> > Certainly not by me.  I usually let everything complete, though (which
> > takes a looong time if I run the full suite).
> >
> > > This is quite frustrating when testing "before" and "after" each patch.
> >
> > Agreed.  :-/
> >
> > I'll have to try it out to see what's happening.  Does it only happen
> > with this series?  I'd be surprised, but possibly I introduced some error.
> >
> 
> Nop, I got this even before your first patch.
> 

Is this meson related or related to the auto test binary in DPDK. I know
traditionally I've found the test binary rather difficult to kill, but I'd
like to be sure that the meson infrastructure itself isn't making it worse.

/Bruce

Re: [dpdk-dev] [RFC 0/3] ci: enable unit tests for non-aarch64 platforms

[Bruce Richardson]

On Mon, Apr 01, 2019 at 09:29:51PM +0200, David Marchand wrote:
> On Mon, Apr 1, 2019 at 9:28 PM Aaron Conole <aconole@redhat.com> wrote:
> 
> > David Marchand <david.marchand@redhat.com> writes:
> > > I tried using meson/ninja for the tests, something that bothered me is
> > that I can't interrupt the tests.
> > > I had to kill manually, meson, ninja and I had some leftover dpdk-test
> > processes (maybe due to some ^Z I
> > > hit...).
> > > Is this expected ?
> >
> > Certainly not by me.  I usually let everything complete, though (which
> > takes a looong time if I run the full suite).
> >
> > > This is quite frustrating when testing "before" and "after" each patch.
> >
> > Agreed.  :-/
> >
> > I'll have to try it out to see what's happening.  Does it only happen
> > with this series?  I'd be surprised, but possibly I introduced some error.
> >
> 
> Nop, I got this even before your first patch.
> 

Is this meson related or related to the auto test binary in DPDK. I know
traditionally I've found the test binary rather difficult to kill, but I'd
like to be sure that the meson infrastructure itself isn't making it worse.

/Bruce

Re: [dpdk-dev] [RFC 0/3] ci: enable unit tests for non-aarch64 platforms

[David Marchand]

On Tue, Apr 2, 2019 at 11:37 AM Bruce Richardson <bruce.richardson@intel.com>
wrote:

> On Mon, Apr 01, 2019 at 09:29:51PM +0200, David Marchand wrote:
> > On Mon, Apr 1, 2019 at 9:28 PM Aaron Conole <aconole@redhat.com> wrote:
> >
> > > David Marchand <david.marchand@redhat.com> writes:
> > > > I tried using meson/ninja for the tests, something that bothered me
> is
> > > that I can't interrupt the tests.
> > > > I had to kill manually, meson, ninja and I had some leftover
> dpdk-test
> > > processes (maybe due to some ^Z I
> > > > hit...).
> > > > Is this expected ?
> > >
> > > Certainly not by me.  I usually let everything complete, though (which
> > > takes a looong time if I run the full suite).
> > >
> > > > This is quite frustrating when testing "before" and "after" each
> patch.
> > >
> > > Agreed.  :-/
> > >
> > > I'll have to try it out to see what's happening.  Does it only happen
> > > with this series?  I'd be surprised, but possibly I introduced some
> error.
> > >
> >
> > Nop, I got this even before your first patch.
> >
>
> Is this meson related or related to the auto test binary in DPDK. I know
> traditionally I've found the test binary rather difficult to kill, but I'd
> like to be sure that the meson infrastructure itself isn't making it worse.
>

Hard to tell, I would have to retest and investigate, unless Aaron went
further than me.


-- 
David Marchand

Re: [dpdk-dev] [RFC 0/3] ci: enable unit tests for non-aarch64 platforms

[David Marchand]

On Tue, Apr 2, 2019 at 11:37 AM Bruce Richardson <bruce.richardson@intel.com>
wrote:

> On Mon, Apr 01, 2019 at 09:29:51PM +0200, David Marchand wrote:
> > On Mon, Apr 1, 2019 at 9:28 PM Aaron Conole <aconole@redhat.com> wrote:
> >
> > > David Marchand <david.marchand@redhat.com> writes:
> > > > I tried using meson/ninja for the tests, something that bothered me
> is
> > > that I can't interrupt the tests.
> > > > I had to kill manually, meson, ninja and I had some leftover
> dpdk-test
> > > processes (maybe due to some ^Z I
> > > > hit...).
> > > > Is this expected ?
> > >
> > > Certainly not by me.  I usually let everything complete, though (which
> > > takes a looong time if I run the full suite).
> > >
> > > > This is quite frustrating when testing "before" and "after" each
> patch.
> > >
> > > Agreed.  :-/
> > >
> > > I'll have to try it out to see what's happening.  Does it only happen
> > > with this series?  I'd be surprised, but possibly I introduced some
> error.
> > >
> >
> > Nop, I got this even before your first patch.
> >
>
> Is this meson related or related to the auto test binary in DPDK. I know
> traditionally I've found the test binary rather difficult to kill, but I'd
> like to be sure that the meson infrastructure itself isn't making it worse.
>

Hard to tell, I would have to retest and investigate, unless Aaron went
further than me.


-- 
David Marchand

Re: [dpdk-dev] [RFC 0/3] ci: enable unit tests for non-aarch64 platforms

[Aaron Conole]

David Marchand <david.marchand@redhat.com> writes:

> On Tue, Apr 2, 2019 at 11:37 AM Bruce Richardson <bruce.richardson@intel.com> wrote:
>
>  On Mon, Apr 01, 2019 at 09:29:51PM +0200, David Marchand wrote:
>  > On Mon, Apr 1, 2019 at 9:28 PM Aaron Conole <aconole@redhat.com> wrote:
>  > 
>  > > David Marchand <david.marchand@redhat.com> writes:
>  > > > I tried using meson/ninja for the tests, something that bothered me is
>  > > that I can't interrupt the tests.
>  > > > I had to kill manually, meson, ninja and I had some leftover dpdk-test
>  > > processes (maybe due to some ^Z I
>  > > > hit...).
>  > > > Is this expected ?
>  > >
>  > > Certainly not by me.  I usually let everything complete, though (which
>  > > takes a looong time if I run the full suite).
>  > >
>  > > > This is quite frustrating when testing "before" and "after" each patch.
>  > >
>  > > Agreed.  :-/
>  > >
>  > > I'll have to try it out to see what's happening.  Does it only happen
>  > > with this series?  I'd be surprised, but possibly I introduced some error.
>  > >
>  > 
>  > Nop, I got this even before your first patch.
>  > 
>
>  Is this meson related or related to the auto test binary in DPDK. I know
>  traditionally I've found the test binary rather difficult to kill, but I'd
>  like to be sure that the meson infrastructure itself isn't making it worse.
>
> Hard to tell, I would have to retest and investigate, unless Aaron went further than me.

I did some investigation with this.  At least I don't see any lingering
process from meson, but it does take time for the tests to die when I
hit CTRL-C, and I get a warning - I found this related bug:
   https://github.com/mesonbuild/meson/issues/2281

I guess it could be some kind of interplay between the way meson kills
tests?  Looking at the commit that eventually 'closes' the bug, they
merely set a flag rather than pass a kill signal down, so I guess it's
probably never going to be immediate.

Re: [dpdk-dev] [RFC 0/3] ci: enable unit tests for non-aarch64 platforms

[Aaron Conole]

David Marchand <david.marchand@redhat.com> writes:

> On Tue, Apr 2, 2019 at 11:37 AM Bruce Richardson <bruce.richardson@intel.com> wrote:
>
>  On Mon, Apr 01, 2019 at 09:29:51PM +0200, David Marchand wrote:
>  > On Mon, Apr 1, 2019 at 9:28 PM Aaron Conole <aconole@redhat.com> wrote:
>  > 
>  > > David Marchand <david.marchand@redhat.com> writes:
>  > > > I tried using meson/ninja for the tests, something that bothered me is
>  > > that I can't interrupt the tests.
>  > > > I had to kill manually, meson, ninja and I had some leftover dpdk-test
>  > > processes (maybe due to some ^Z I
>  > > > hit...).
>  > > > Is this expected ?
>  > >
>  > > Certainly not by me.  I usually let everything complete, though (which
>  > > takes a looong time if I run the full suite).
>  > >
>  > > > This is quite frustrating when testing "before" and "after" each patch.
>  > >
>  > > Agreed.  :-/
>  > >
>  > > I'll have to try it out to see what's happening.  Does it only happen
>  > > with this series?  I'd be surprised, but possibly I introduced some error.
>  > >
>  > 
>  > Nop, I got this even before your first patch.
>  > 
>
>  Is this meson related or related to the auto test binary in DPDK. I know
>  traditionally I've found the test binary rather difficult to kill, but I'd
>  like to be sure that the meson infrastructure itself isn't making it worse.
>
> Hard to tell, I would have to retest and investigate, unless Aaron went further than me.

I did some investigation with this.  At least I don't see any lingering
process from meson, but it does take time for the tests to die when I
hit CTRL-C, and I get a warning - I found this related bug:
   https://github.com/mesonbuild/meson/issues/2281

I guess it could be some kind of interplay between the way meson kills
tests?  Looking at the commit that eventually 'closes' the bug, they
merely set a flag rather than pass a kill signal down, so I guess it's
probably never going to be immediate.