From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id EA614A0569; Thu, 12 Mar 2020 08:13:31 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 868102BE6; Thu, 12 Mar 2020 08:13:30 +0100 (CET) Received: from lb.pantheon.sk (lb.pantheon.sk [46.229.239.20]) by dpdk.org (Postfix) with ESMTP id 099FA2BAA for ; Thu, 12 Mar 2020 08:13:28 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by lb.pantheon.sk (Postfix) with ESMTP id 33C771A39DC; Thu, 12 Mar 2020 08:13:27 +0100 (CET) X-Virus-Scanned: amavisd-new at siecit.sk Received: from lb.pantheon.sk ([127.0.0.1]) by localhost (lb.pantheon.sk [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fkMlXh3nUCqY; Thu, 12 Mar 2020 08:13:25 +0100 (CET) Received: from mail.pantheon.sk (srvw-ptex1.pantheon.local [10.101.4.5]) by lb.pantheon.sk (Postfix) with ESMTPS id 0AE3519C422; Thu, 12 Mar 2020 08:13:24 +0100 (CET) Received: from srvw-ptex1.pantheon.local (10.101.4.5) by srvw-ptex1.pantheon.local (10.101.4.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1779.2; Thu, 12 Mar 2020 08:13:24 +0100 Received: from srvw-ptex1.pantheon.local ([::1]) by srvw-ptex1.pantheon.local ([fe80::b583:f9c5:55e8:f949%7]) with mapi id 15.01.1779.002; Thu, 12 Mar 2020 08:13:24 +0100 From: =?iso-8859-2?Q?Juraj_Linke=B9?= To: Ruifeng Wang , Aaron Conole CC: "maicolgabriel@hotmail.com" , "bruce.richardson@intel.com" , "dev@dpdk.org" , "david.marchand@redhat.com" , Gavin Hu , Honnappa Nagarahalli , nd , nd Thread-Topic: [PATCH v2 2/2] ci: add test suite run without hugepage Thread-Index: AQHV7e5JbAhmiM46FEmRvzdpTfej8Kg4uZpGgACZjYCAAMgAwYABFVeAgACTcpeAAWr1gIAHbUQg Date: Thu, 12 Mar 2020 07:13:24 +0000 Message-ID: References: <20200225073236.135581-1-ruifeng.wang@arm.com> <20200228041904.195597-1-ruifeng.wang@arm.com> <20200228041904.195597-3-ruifeng.wang@arm.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.101.4.10] Content-Type: text/plain; charset="iso-8859-2" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH v2 2/2] ci: add test suite run without hugepage X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi, What if we just split the test names into fast_hugepage_test_name and fast_= nohuge_test_names and create two suites (fast-suite and fast-nohuge-suite) = from that? That seems like a simpler version of one list with booleans indi= cating whether a test is a huge/nohuge test, though it would require dev to= add tests into the proper list. We could then run the full suite in x86 jo= bs and the nogue suite in arm jobs (and users could use either suite when t= hey have/don't have hugepages configured). The other option would be to detect hugepages at runtime and skip the tests= in the full suite when there aren't hugepages on the system. Possibly the = best option, but where should the check be? Before testcase setup (i.e. if = there aren't hugepages, skip the testcase right away)? Thoughts? Thanks, Juraj -----Original Message----- From: Ruifeng Wang =20 Sent: Saturday, March 7, 2020 3:36 PM To: Aaron Conole Cc: maicolgabriel@hotmail.com; bruce.richardson@intel.com; dev@dpdk.org; da= vid.marchand@redhat.com; Gavin Hu ; Honnappa Nagarahalli = ; Juraj Linke=B9 = ; nd ; nd Subject: RE: [PATCH v2 2/2] ci: add test suite run without hugepage > -----Original Message----- > From: Aaron Conole > Sent: Friday, March 6, 2020 23:57 > To: Ruifeng Wang > Cc: maicolgabriel@hotmail.com; bruce.richardson@intel.com;=20 > dev@dpdk.org; david.marchand@redhat.com; Gavin Hu ;=20 > Honnappa Nagarahalli ;=20 > juraj.linkes@pantheon.tech; nd > Subject: Re: [PATCH v2 2/2] ci: add test suite run without hugepage >=20 > Ruifeng Wang writes: >=20 > >> -----Original Message----- > >> From: Aaron Conole > >> Sent: Thursday, March 5, 2020 22:37 > >> To: Ruifeng Wang > >> Cc: maicolgabriel@hotmail.com; bruce.richardson@intel.com;=20 > >> dev@dpdk.org; david.marchand@redhat.com; Gavin Hu > ; > >> Honnappa Nagarahalli ;=20 > >> juraj.linkes@pantheon.tech; nd > >> Subject: Re: [PATCH v2 2/2] ci: add test suite run without hugepage > >> > >> Ruifeng Wang writes: > >> > >> >> -----Original Message----- > >> >> From: Aaron Conole > >> >> Sent: Thursday, March 5, 2020 01:31 > >> >> To: Ruifeng Wang > >> >> Cc: maicolgabriel@hotmail.com; bruce.richardson@intel.com;=20 > >> >> dev@dpdk.org; david.marchand@redhat.com; Gavin Hu > >> ; > >> >> Honnappa Nagarahalli ;=20 > >> >> juraj.linkes@pantheon.tech; nd > >> >> Subject: Re: [PATCH v2 2/2] ci: add test suite run without=20 > >> >> hugepage > >> >> > >> >> Ruifeng Wang writes: > >> >> > >> >> > This test suite is derived from fast-tests suite. Cases in=20 > >> >> > this suite are run with '--no-huge' flag. > >> >> > > >> >> > The suite aims to cover as many as possible test cases out of=20 > >> >> > the fast-tests suites in the environments without huge pages=20 > >> >> > support, like containers. > >> >> > > >> >> > Signed-off-by: Ruifeng Wang > >> >> > Reviewed-by: Gavin Hu > >> >> > --- > >> >> > >> >> I like this much more. Few comments. > >> >> > >> >> > .travis.yml | 10 +++++-- > >> >> > app/test/meson.build | 71 > >> >> > ++++++++++++++++++++++++++++++++++++++++++++ > >> >> > >> >> You should update doc/guides/prog_guide/meson_ut.rst to include=20 > >> >> some detail about the new tests suite. > >> >> > >> > Thanks. Will update document in next version. > >> > > >> >> > 2 files changed, 79 insertions(+), 2 deletions(-) > >> >> > > >> >> > diff --git a/.travis.yml b/.travis.yml index=20 > >> >> > b64a81bd0..eed1d96db > >> >> > 100644 > >> >> > --- a/.travis.yml > >> >> > +++ b/.travis.yml > >> >> > @@ -40,7 +40,7 @@ jobs: > >> >> > - env: DEF_LIB=3D"static" > >> >> > arch: amd64 > >> >> > compiler: gcc > >> >> > - - env: DEF_LIB=3D"shared" RUN_TESTS=3D1 > >> >> > + - env: DEF_LIB=3D"shared" RUN_TESTS=3D1 TEST_SUITES=3D"fast-te= sts > >> >> nohuge-tests" > >> >> > arch: amd64 > >> >> > compiler: gcc > >> >> > - env: DEF_LIB=3D"shared" BUILD_DOCS=3D1 @@ -63,7 +63,7 @@ job= s: > >> >> > - env: DEF_LIB=3D"static" > >> >> > arch: amd64 > >> >> > compiler: clang > >> >> > - - env: DEF_LIB=3D"shared" RUN_TESTS=3D1 > >> >> > + - env: DEF_LIB=3D"shared" RUN_TESTS=3D1 TEST_SUITES=3D"fast-te= sts > >> >> nohuge-tests" > >> >> > arch: amd64 > >> >> > compiler: clang > >> >> > - env: DEF_LIB=3D"shared" BUILD_DOCS=3D1 @@ -101,6 +101,9 @@ > jobs: > >> >> > - env: DEF_LIB=3D"static" > >> >> > arch: arm64 > >> >> > compiler: gcc > >> >> > + - env: DEF_LIB=3D"shared" RUN_TESTS=3D1 TEST_SUITES=3D"nohuge- > tests" > >> >> > + arch: arm64 > >> >> > + compiler: gcc > >> >> > - env: DEF_LIB=3D"shared" BUILD_DOCS=3D1 > >> >> > arch: arm64 > >> >> > compiler: gcc > >> >> > @@ -124,3 +127,6 @@ jobs: > >> >> > - env: DEF_LIB=3D"shared" > >> >> > arch: arm64 > >> >> > compiler: clang > >> >> > + - env: DEF_LIB=3D"shared" RUN_TESTS=3D1 TEST_SUITES=3D"nohuge- > tests" > >> >> > + arch: arm64 > >> >> > + compiler: clang > >> >> > diff --git a/app/test/meson.build b/app/test/meson.build index=20 > >> >> > 0a2ce710f..162a1a76f 100644 > >> >> > --- a/app/test/meson.build > >> >> > +++ b/app/test/meson.build > >> >> > @@ -237,6 +237,60 @@ fast_test_names =3D [ > >> >> > 'thash_autotest', > >> >> > ] > >> >> > >> >> Shouldn't we also trim the list of fast-tests? Otherwise, these=20 > >> >> tests will run twice. > >> >> > >> > I think you mean to have exclusive lists for fast-tests and nohuge-t= ests. > >> > >> That's what I was thinking. > >> > >> > Overlapped cases will run twice if both test suites are opted in. > >> > But the two runs are not the same, one runs with hugepage and the=20 > >> > other runs in no-huge mode. > >> > >> Is it really so different between huge and no-huge? Most of the=20 > >> libraries won't care - they call the rte_**alloc functions, and it=20 > >> returns blocks of memory. Maybe I am simplifying it too much. > >> > >> > If fast-tests list is splited, we will need to always run=20 > >> > multiple suites to cover all fast tests. > >> > We can keep x86 to run only fast-tests suite to avoid extra test=20 > >> > runs if they are not necessary. Thoughts? > >> > >> I guess since most DPDK usage will be with hugepages, we should=20 > >> prefer to test with it. I don't care too much about the color of=20 > >> this particular shed. If you want to do it that way, it's okay by=20 > >> me > >> - it gives us the coverage, and doesn't duplicate tests between=20 > >> those > environments. > >> > >> BUT it means when we add a new test to the suite, we need to > remember > >> to add it in two places - fast_test and nohuge_test. That almost=20 > >> guarantees we will miss tests (because we accidentally don't add it=20 > >> to nohuge). Maybe there's another way, like keep a list of all the=20 > >> tests and some information on whether the test needs hugepages to=20 > >> run. Then if there are no hugepages available, we can write that=20 > >> we SKIP > the tests that don't support huge pages. > >> In that way, we don't need two different lists - and if there are=20 > >> hugepages, we will run all the test cases. > >> WDYT? > >> > > Yes. Agree with you that having duplicate tests in suites is error pron= e. >=20 > Cool! >=20 > > IIUC, cases in a suite is determined at build time, as well as=20 > > command > options to run cases. > > This implies hugepage availability needs to be detected at build=20 > > time if we want to run only suitable cases in suite in an=20 > > environment. It could be > something we don't want. > > > > I'll trim fast-tests in next version to remove duplicated cases. >=20 > I think it might be better to make the array something like (just a=20 > psuedo- code example): >=20 > # psuedo-code to check for hugepages > has_hugepages =3D check_for_hugepages() >=20 > ... >=20 > fast_test_names =3D [ > ['acl_autotest', true], > ['alarm_autotest', true], > ['atomic_autotest', true], > ... >=20 > Then in the code: >=20 > foreach arg : fast_test_names > .... > if not arg[1] > test(arg[0], ...) > if has_hugepages and arg[1] > test(arg[0], ) >=20 > Does it make sense? Do you see a problem? >=20 Yes, this will keep tests in a single suite. Actually, I thought about this approach, but had no idea on check_for_hugep= ages(). Value of "/proc/sys/vm/nr_hugepages" may be not reliable. Hugepage could be= allocated at run time after project building. I can try hugepage allocating to detect, but it looks inelegant. Any suggestions? Thank you. /Ruifeng > > Thank you. > > > >> >> ex: https://travis-ci.com/ovsrobot/dpdk/jobs/292037684 > >> >> > >> >> > +nohuge_test_names =3D [ > >> >> > + 'byteorder_autotest', > >> >> > + 'cmdline_autotest', > >> >> > + 'common_autotest', > >> >> > + 'cpuflags_autotest', > >> >> > + 'cycles_autotest', > >> >> > + 'debug_autotest', > >> >> > + 'eal_flags_n_opt_autotest', > >> >> > + 'eal_flags_no_huge_autotest', > >> >> > + 'eal_flags_vdev_opt_autotest', > >> >> > + 'eal_fs_autotest', > >> >> > + 'errno_autotest', > >> >> > + 'event_ring_autotest', > >> >> > + 'fib_autotest', > >> >> > + 'fib6_autotest', > >> >> > + 'interrupt_autotest', > >> >> > + 'logs_autotest', > >> >> > + 'lpm_autotest', > >> >> > + 'lpm6_autotest', > >> >> > + 'memcpy_autotest', > >> >> > + 'meter_autotest', > >> >> > + 'per_lcore_autotest', > >> >> > + 'prefetch_autotest', > >> >> > + 'rcu_qsbr_autotest', > >> >> > + 'red_autotest', > >> >> > + 'rib_autotest', > >> >> > + 'rib6_autotest', > >> >> > + 'ring_autotest', > >> >> > + 'rwlock_rda_autotest', > >> >> > + 'rwlock_rds_wrm_autotest', > >> >> > + 'rwlock_rde_wro_autotest', > >> >> > + 'sched_autotest', > >> >> > + 'spinlock_autotest', > >> >> > + 'string_autotest', > >> >> > + 'tailq_autotest', > >> >> > + 'user_delay_us', > >> >> > + 'version_autotest', > >> >> > + 'crc_autotest', > >> >> > + 'delay_us_sleep_autotest', > >> >> > + 'eventdev_common_autotest', > >> >> > + 'fbarray_autotest', > >> >> > + 'ipsec_autotest', > >> >> > + 'kni_autotest', > >> >> > + 'kvargs_autotest', > >> >> > + 'member_autotest', > >> >> > + 'metrics_autotest', > >> >> > + 'power_cpufreq_autotest', > >> >> > + 'power_autotest', > >> >> > + 'power_kvm_vm_autotest', > >> >> > + 'reorder_autotest', > >> >> > + 'service_autotest', > >> >> > + 'thash_autotest', > >> >> > +] > >> >> > + > >> >> > perf_test_names =3D [ > >> >> > 'ring_perf_autotest', > >> >> > 'mempool_perf_autotest', @@ -341,6 +395,10 @@ if > >> >> > dpdk_conf.has('RTE_LIBRTE_RING_PMD') > >> >> > fast_test_names +=3D 'latencystats_autotest' > >> >> > driver_test_names +=3D 'link_bonding_mode4_autotest' > >> >> > fast_test_names +=3D 'pdump_autotest' > >> >> > + nohuge_test_names +=3D 'ring_pmd_autotest' > >> >> > + nohuge_test_names +=3D 'bitratestats_autotest' > >> >> > + nohuge_test_names +=3D 'latencystats_autotest' > >> >> > + nohuge_test_names +=3D 'pdump_autotest' > >> >> > endif > >> >> > > >> >> > if dpdk_conf.has('RTE_LIBRTE_POWER') @@ -430,6 +488,19 @@ > >> foreach > >> >> > arg : fast_test_names > >> >> > endif > >> >> > endforeach > >> >> > > >> >> > +foreach arg : nohuge_test_names > >> >> > + if host_machine.system() =3D=3D 'linux' > >> >> > + test(arg, dpdk_test, > >> >> > + env : ['DPDK_TEST=3D' + arg], > >> >> > + args : test_args + > >> >> > + ['--no-huge'] + ['-m 1024'] + > >> >> > + ['--file-prefix=3D@0@'.format(arg)], > >> >> > + timeout : timeout_seconds_fast, > >> >> > + is_parallel : false, > >> >> > + suite : 'nohuge-tests') > >> >> > + endif > >> >> > +endforeach > >> >> > + > >> >> > foreach arg : perf_test_names > >> >> > test(arg, dpdk_test, > >> >> > env : ['DPDK_TEST=3D' + arg],