From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7A1E2A056A; Fri, 6 Mar 2020 16:57:08 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 932712BA8; Fri, 6 Mar 2020 16:57:07 +0100 (CET) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) by dpdk.org (Postfix) with ESMTP id E4C0F23D for ; Fri, 6 Mar 2020 16:57:06 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1583510226; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1iAxPIG5UOeWjFOxje1U/Zz64BqoSjgpeqxGNmU+b5A=; b=U91sbK9vNY/X6ZdLkF9yw/KJmQBKsI84R+myePS0ZNTY/sKk/ENqBkX1qGDqSggwpHoJOD eu4vH5qFc0xp40L0nZIqt+hkgyXyy5GovSyYUlZRkx2CecbKwxuTjOBlApEBHAGwX0QpkM ylUF5F5NAKQu0XmPi46lWpCm2HtgEH8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-79-BjfsnZfqMbyme_xjCRNiSw-1; Fri, 06 Mar 2020 10:57:03 -0500 X-MC-Unique: BjfsnZfqMbyme_xjCRNiSw-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 73C8B801E5C; Fri, 6 Mar 2020 15:57:01 +0000 (UTC) Received: from dhcp-25.97.bos.redhat.com (unknown [10.18.25.84]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 162E05C28D; Fri, 6 Mar 2020 15:56:56 +0000 (UTC) From: Aaron Conole To: Ruifeng Wang Cc: "maicolgabriel\@hotmail.com" , "bruce.richardson\@intel.com" , "dev\@dpdk.org" , "david.marchand\@redhat.com" , Gavin Hu , Honnappa Nagarahalli , "juraj.linkes\@pantheon.tech" , nd References: <20200225073236.135581-1-ruifeng.wang@arm.com> <20200228041904.195597-1-ruifeng.wang@arm.com> <20200228041904.195597-3-ruifeng.wang@arm.com> Date: Fri, 06 Mar 2020 10:56:56 -0500 In-Reply-To: (Ruifeng Wang's message of "Fri, 6 Mar 2020 08:09:29 +0000") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Subject: Re: [dpdk-dev] [PATCH v2 2/2] ci: add test suite run without hugepage X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Ruifeng Wang writes: >> -----Original Message----- >> From: Aaron Conole >> Sent: Thursday, March 5, 2020 22:37 >> To: Ruifeng Wang >> Cc: maicolgabriel@hotmail.com; bruce.richardson@intel.com; dev@dpdk.org; >> david.marchand@redhat.com; Gavin Hu ; Honnappa >> Nagarahalli ; juraj.linkes@pantheon.tech; >> nd >> Subject: Re: [PATCH v2 2/2] ci: add test suite run without hugepage >>=20 >> Ruifeng Wang writes: >>=20 >> >> -----Original Message----- >> >> From: Aaron Conole >> >> Sent: Thursday, March 5, 2020 01:31 >> >> To: Ruifeng Wang >> >> Cc: maicolgabriel@hotmail.com; bruce.richardson@intel.com; >> >> dev@dpdk.org; david.marchand@redhat.com; Gavin Hu >> ; >> >> Honnappa Nagarahalli ; >> >> juraj.linkes@pantheon.tech; nd >> >> Subject: Re: [PATCH v2 2/2] ci: add test suite run without hugepage >> >> >> >> Ruifeng Wang writes: >> >> >> >> > This test suite is derived from fast-tests suite. Cases in this >> >> > suite are run with '--no-huge' flag. >> >> > >> >> > The suite aims to cover as many as possible test cases out of the >> >> > fast-tests suites in the environments without huge pages support, >> >> > like containers. >> >> > >> >> > Signed-off-by: Ruifeng Wang >> >> > Reviewed-by: Gavin Hu >> >> > --- >> >> >> >> I like this much more. Few comments. >> >> >> >> > .travis.yml | 10 +++++-- >> >> > app/test/meson.build | 71 >> >> > ++++++++++++++++++++++++++++++++++++++++++++ >> >> >> >> You should update doc/guides/prog_guide/meson_ut.rst to include some >> >> detail about the new tests suite. >> >> >> > Thanks. Will update document in next version. >> > >> >> > 2 files changed, 79 insertions(+), 2 deletions(-) >> >> > >> >> > diff --git a/.travis.yml b/.travis.yml index b64a81bd0..eed1d96db >> >> > 100644 >> >> > --- a/.travis.yml >> >> > +++ b/.travis.yml >> >> > @@ -40,7 +40,7 @@ jobs: >> >> > - env: DEF_LIB=3D"static" >> >> > arch: amd64 >> >> > compiler: gcc >> >> > - - env: DEF_LIB=3D"shared" RUN_TESTS=3D1 >> >> > + - env: DEF_LIB=3D"shared" RUN_TESTS=3D1 TEST_SUITES=3D"fast-test= s >> >> nohuge-tests" >> >> > arch: amd64 >> >> > compiler: gcc >> >> > - env: DEF_LIB=3D"shared" BUILD_DOCS=3D1 @@ -63,7 +63,7 @@ jobs: >> >> > - env: DEF_LIB=3D"static" >> >> > arch: amd64 >> >> > compiler: clang >> >> > - - env: DEF_LIB=3D"shared" RUN_TESTS=3D1 >> >> > + - env: DEF_LIB=3D"shared" RUN_TESTS=3D1 TEST_SUITES=3D"fast-test= s >> >> nohuge-tests" >> >> > arch: amd64 >> >> > compiler: clang >> >> > - env: DEF_LIB=3D"shared" BUILD_DOCS=3D1 @@ -101,6 +101,9 @@ job= s: >> >> > - env: DEF_LIB=3D"static" >> >> > arch: arm64 >> >> > compiler: gcc >> >> > + - env: DEF_LIB=3D"shared" RUN_TESTS=3D1 TEST_SUITES=3D"nohuge-te= sts" >> >> > + arch: arm64 >> >> > + compiler: gcc >> >> > - env: DEF_LIB=3D"shared" BUILD_DOCS=3D1 >> >> > arch: arm64 >> >> > compiler: gcc >> >> > @@ -124,3 +127,6 @@ jobs: >> >> > - env: DEF_LIB=3D"shared" >> >> > arch: arm64 >> >> > compiler: clang >> >> > + - env: DEF_LIB=3D"shared" RUN_TESTS=3D1 TEST_SUITES=3D"nohuge-te= sts" >> >> > + arch: arm64 >> >> > + compiler: clang >> >> > diff --git a/app/test/meson.build b/app/test/meson.build index >> >> > 0a2ce710f..162a1a76f 100644 >> >> > --- a/app/test/meson.build >> >> > +++ b/app/test/meson.build >> >> > @@ -237,6 +237,60 @@ fast_test_names =3D [ >> >> > 'thash_autotest', >> >> > ] >> >> >> >> Shouldn't we also trim the list of fast-tests? Otherwise, these >> >> tests will run twice. >> >> >> > I think you mean to have exclusive lists for fast-tests and nohuge-tes= ts. >>=20 >> That's what I was thinking. >>=20 >> > Overlapped cases will run twice if both test suites are opted in. >> > But the two runs are not the same, one runs with hugepage and the >> > other runs in no-huge mode. >>=20 >> Is it really so different between huge and no-huge? Most of the librari= es >> won't care - they call the rte_**alloc functions, and it returns blocks = of >> memory. Maybe I am simplifying it too much. >>=20 >> > If fast-tests list is splited, we will need to always run multiple >> > suites to cover all fast tests. >> > We can keep x86 to run only fast-tests suite to avoid extra test runs >> > if they are not necessary. Thoughts? >>=20 >> I guess since most DPDK usage will be with hugepages, we should prefer t= o >> test with it. I don't care too much about the color of this particular = shed. If >> you want to do it that way, it's okay by me - it gives us the coverage, = and >> doesn't duplicate tests between those environments. >>=20 >> BUT it means when we add a new test to the suite, we need to remember to >> add it in two places - fast_test and nohuge_test. That almost guarantee= s we >> will miss tests (because we accidentally don't add it to nohuge). Maybe >> there's another way, like keep a list of all the tests and some informat= ion on >> whether the test needs hugepages to run. Then if there are no hugepages >> available, we can write that we SKIP the tests that don't support huge p= ages. >> In that way, we don't need two different lists - and if there are hugepa= ges, >> we will run all the test cases. >> WDYT? >>=20 > Yes. Agree with you that having duplicate tests in suites is error prone. Cool! > IIUC, cases in a suite is determined at build time, as well as command op= tions to run cases. > This implies hugepage availability needs to be detected at build time if = we want to run only=20 > suitable cases in suite in an environment. It could be something we don't= want.=20 > > I'll trim fast-tests in next version to remove duplicated cases. I think it might be better to make the array something like (just a psuedo-code example): # psuedo-code to check for hugepages has_hugepages =3D check_for_hugepages() ... fast_test_names =3D [ ['acl_autotest', true], ['alarm_autotest', true], ['atomic_autotest', true], ... Then in the code: foreach arg : fast_test_names .... if not arg[1] test(arg[0], ...) if has_hugepages and arg[1] test(arg[0], ) Does it make sense? Do you see a problem? > Thank you. > >> >> ex: https://travis-ci.com/ovsrobot/dpdk/jobs/292037684 >> >> >> >> > +nohuge_test_names =3D [ >> >> > + 'byteorder_autotest', >> >> > + 'cmdline_autotest', >> >> > + 'common_autotest', >> >> > + 'cpuflags_autotest', >> >> > + 'cycles_autotest', >> >> > + 'debug_autotest', >> >> > + 'eal_flags_n_opt_autotest', >> >> > + 'eal_flags_no_huge_autotest', >> >> > + 'eal_flags_vdev_opt_autotest', >> >> > + 'eal_fs_autotest', >> >> > + 'errno_autotest', >> >> > + 'event_ring_autotest', >> >> > + 'fib_autotest', >> >> > + 'fib6_autotest', >> >> > + 'interrupt_autotest', >> >> > + 'logs_autotest', >> >> > + 'lpm_autotest', >> >> > + 'lpm6_autotest', >> >> > + 'memcpy_autotest', >> >> > + 'meter_autotest', >> >> > + 'per_lcore_autotest', >> >> > + 'prefetch_autotest', >> >> > + 'rcu_qsbr_autotest', >> >> > + 'red_autotest', >> >> > + 'rib_autotest', >> >> > + 'rib6_autotest', >> >> > + 'ring_autotest', >> >> > + 'rwlock_rda_autotest', >> >> > + 'rwlock_rds_wrm_autotest', >> >> > + 'rwlock_rde_wro_autotest', >> >> > + 'sched_autotest', >> >> > + 'spinlock_autotest', >> >> > + 'string_autotest', >> >> > + 'tailq_autotest', >> >> > + 'user_delay_us', >> >> > + 'version_autotest', >> >> > + 'crc_autotest', >> >> > + 'delay_us_sleep_autotest', >> >> > + 'eventdev_common_autotest', >> >> > + 'fbarray_autotest', >> >> > + 'ipsec_autotest', >> >> > + 'kni_autotest', >> >> > + 'kvargs_autotest', >> >> > + 'member_autotest', >> >> > + 'metrics_autotest', >> >> > + 'power_cpufreq_autotest', >> >> > + 'power_autotest', >> >> > + 'power_kvm_vm_autotest', >> >> > + 'reorder_autotest', >> >> > + 'service_autotest', >> >> > + 'thash_autotest', >> >> > +] >> >> > + >> >> > perf_test_names =3D [ >> >> > 'ring_perf_autotest', >> >> > 'mempool_perf_autotest', >> >> > @@ -341,6 +395,10 @@ if dpdk_conf.has('RTE_LIBRTE_RING_PMD') >> >> > =09fast_test_names +=3D 'latencystats_autotest' >> >> > =09driver_test_names +=3D 'link_bonding_mode4_autotest' >> >> > =09fast_test_names +=3D 'pdump_autotest' >> >> > +=09nohuge_test_names +=3D 'ring_pmd_autotest' >> >> > +=09nohuge_test_names +=3D 'bitratestats_autotest' >> >> > +=09nohuge_test_names +=3D 'latencystats_autotest' >> >> > +=09nohuge_test_names +=3D 'pdump_autotest' >> >> > endif >> >> > >> >> > if dpdk_conf.has('RTE_LIBRTE_POWER') @@ -430,6 +488,19 @@ >> foreach >> >> > arg : fast_test_names >> >> > =09endif >> >> > endforeach >> >> > >> >> > +foreach arg : nohuge_test_names >> >> > +=09if host_machine.system() =3D=3D 'linux' >> >> > +=09=09test(arg, dpdk_test, >> >> > +=09=09=09 env : ['DPDK_TEST=3D' + arg], >> >> > +=09=09=09 args : test_args + >> >> > +=09=09=09=09 ['--no-huge'] + ['-m 1024'] + >> >> > +=09=09=09=09 ['--file-prefix=3D@0@'.format(arg)], >> >> > +=09=09timeout : timeout_seconds_fast, >> >> > +=09=09is_parallel : false, >> >> > +=09=09suite : 'nohuge-tests') >> >> > +=09endif >> >> > +endforeach >> >> > + >> >> > foreach arg : perf_test_names >> >> > =09test(arg, dpdk_test, >> >> > =09env : ['DPDK_TEST=3D' + arg],