Re: [dpdk-dev] [PATCH v3 2/4] ci: generate fast-tests suite base on hugepage availability

[Ruifeng Wang <Ruifeng.Wang@arm.com>]

> -----Original Message-----
> From: David Marchand <david.marchand@redhat.com>
> Sent: Monday, March 23, 2020 16:57
> To: Ruifeng Wang <Ruifeng.Wang@arm.com>; Aaron Conole
> <aconole@redhat.com>
> Cc: Michael Santana <maicolgabriel@hotmail.com>; Bruce Richardson
> <bruce.richardson@intel.com>; Ananyev, Konstantin
> <konstantin.ananyev@intel.com>; Cristian Dumitrescu
> <cristian.dumitrescu@intel.com>; Wang, Yipeng1
> <yipeng1.wang@intel.com>; Gobriel, Sameh <sameh.gobriel@intel.com>;
> dev <dev@dpdk.org>; Burakov, Anatoly <anatoly.burakov@intel.com>;
> Gavin Hu <Gavin.Hu@arm.com>; Honnappa Nagarahalli
> <Honnappa.Nagarahalli@arm.com>; juraj.linkes@pantheon.tech; nd
> <nd@arm.com>
> Subject: Re: [PATCH v3 2/4] ci: generate fast-tests suite base on hugepage
> availability
> 
> On Fri, Mar 13, 2020 at 9:17 AM Ruifeng Wang <ruifeng.wang@arm.com>
> wrote:
> >
> > In environments where hugepage are not available, such as containers,
> > many cases in fast-tests suite should also run if no-huge EAL option
> > is used.
> >
> > Flag is appended to each case in fast-tests suite to indicate whether
> > it lives with no-huge mode.
> > With the flag, fast-tests suite can be generated based on detected
> > hugepage availability of building environment.
> > All cases will be valid if hugepage is available, whereas only
> > applicable cases will be added if environment has no hugepage support.
> >
> > Suggested-by: Aaron Conole <aconole@redhat.com>
> > Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> > Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> > ---
> >  app/test/meson.build | 216
> > ++++++++++++++++++++++++-------------------
> >  1 file changed, 120 insertions(+), 96 deletions(-)
> >
> > diff --git a/app/test/meson.build b/app/test/meson.build index
> > 0a2ce710f..dd121a297 100644
> > --- a/app/test/meson.build
> > +++ b/app/test/meson.build
> > @@ -154,87 +154,89 @@ test_deps = ['acl',
> >         'timer'
> >  ]
> >
> > +# Each test is marked with flag true/false # to indicate whether it
> > +can run in no-huge mode.
> >  fast_test_names = [
> 
> Nit: this variable does not contain a list of names anymore, how about
> "fast_tests".

I'm OK with it.

> 
> > -        'acl_autotest',
> > -        'alarm_autotest',
> > -        'atomic_autotest',
> [snip]
> > +        ['acl_autotest', true],
> > +        ['alarm_autotest', false],
> > +        ['atomic_autotest', false],
> 
> [snip]
> 
> > @@ -395,6 +397,17 @@ dpdk_test = executable('dpdk-test',
> >         install_rpath: driver_install_path,
> >         install: true)
> >
> > +has_hugepage = true
> > +if host_machine.system() == 'linux'
> 
> We have is_OS helpers, here it should be if is_linux.

OK.

> 
> 
> > +       check_hugepage = run_command('cat',
> > +                                    '/proc/sys/vm/nr_hugepages')
> > +       if (check_hugepage.returncode() != 0 or
> > +           check_hugepage.stdout().strip() == '0')
> > +               has_hugepage = false
> > +       endif
> > +endif
> > +message('hugepage availbility: @0@'.format(has_hugepage))
> 
> availability*
> 
> > +
> >  # some perf tests (eg: memcpy perf autotest)take very long  # to
> > complete, so timeout to 10 minutes  timeout_seconds = 600 @@ -407,22
> > +420,33 @@ test_args = [num_cores_arg]
> >
> >  foreach arg : fast_test_names
> >         if (get_option('default_library') == 'shared' and
> > -               arg == 'event_eth_tx_adapter_autotest')
> > +               arg[0] == 'event_eth_tx_adapter_autotest')
> >                 foreach drv:dpdk_drivers
> >                         test_args += ['-d', drv.full_path().split('.a')[0] + '.so']
> >                 endforeach
> >         endif
> >         if host_machine.system() == 'linux'
> > -               test(arg, dpdk_test,
> > -                         env : ['DPDK_TEST=' + arg],
> > -                         args : test_args +
> > -                                ['--file-prefix=@0@'.format(arg)],
> > -               timeout : timeout_seconds_fast,
> > -               is_parallel : false,
> > -               suite : 'fast-tests')
> > +               if has_hugepage
> > +                       test(arg[0], dpdk_test,
> > +                                 env : ['DPDK_TEST=' + arg[0]],
> > +                                 args : test_args +
> > +                                        ['--file-prefix=@0@'.format(arg[0])],
> > +                       timeout : timeout_seconds_fast,
> > +                       is_parallel : false,
> > +                       suite : 'fast-tests')
> > +               elif arg[1]
> > +                       test(arg[0], dpdk_test,
> > +                                 env : ['DPDK_TEST=' + arg[0]],
> > +                                 args : test_args +
> > +                                        ['--no-huge'] + ['-m 2048'] +
> > +                                        ['--file-prefix=@0@'.format(arg[0])],
> > +                       timeout : timeout_seconds_fast,
> > +                       is_parallel : false,
> > +                       suite : 'fast-tests')
> > +               endif
> >         else
> > -               test(arg, dpdk_test,
> > -                       env : ['DPDK_TEST=' + arg],
> > +               test(arg[0], dpdk_test,
> > +                       env : ['DPDK_TEST=' + arg[0]],
> >                         args : test_args,
> >                 timeout : timeout_seconds_fast,
> >                 is_parallel : false,
> 
> 
> Not a fan of these conditionals.
> 
> I sent a little patch fixing an issue I spotted on test_args:
> http://patchwork.dpdk.org/patch/67026/
> 
> I rebased this series on it, see:
> https://github.com/david-marchand/dpdk/commits/ci
> 
> This makes the code easier to read from my pov:
> https://github.com/david-
> marchand/dpdk/blob/ci/app/test/meson.build#L421
> 
> If you are fine with it, I can post a v4 series.

Sure, I'm fine with it.
Thanks.
/Ruifeng
> 
> 
> --
> David Marchand