From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id B6C00A056A; Fri, 6 Mar 2020 17:06:17 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 1FB741BFCD; Fri, 6 Mar 2020 17:06:17 +0100 (CET) Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [205.139.110.61]) by dpdk.org (Postfix) with ESMTP id 303B22BA8 for ; Fri, 6 Mar 2020 17:06:15 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1583510774; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5GmETMxgU4lJ0Yx4AshZaEGQUr6RLKMVRAVoErY/5Bs=; b=VuJvcTwXZgRXDX6FYWxZFaBHyZlwzNV1bpHn/ZhQhLIWWKrp7nBU9GkNFaPPx7JXrU1+8M mClQhfOkz6dzbqrtX7TWuBtVRW/9+CeYHTJWjJTL7cEfNZKWuOOPKIugr5lFr66TaAZSrA EXoLBu6L6UgbE31Lt0v6/8I9FsHk+kE= Received: from mail-vs1-f69.google.com (mail-vs1-f69.google.com [209.85.217.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-456-7VjS68RmNP2Ii_acLe1pew-1; Fri, 06 Mar 2020 11:06:10 -0500 X-MC-Unique: 7VjS68RmNP2Ii_acLe1pew-1 Received: by mail-vs1-f69.google.com with SMTP id x200so407137vsx.13 for ; Fri, 06 Mar 2020 08:06:10 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=gx0GEWcOQeLW5wVOP78NH/mBQK6HG9frU0E2VNdDvnM=; b=AK1es4aSmwW6BD8Rym3yVVgOaEDgbt0DhWNV5rD5ZXojXVjlSb3KhvSlZYN3e4Hm58 5rSDAL7PC6eOK6XVTS/dTWNWVY1v3fANVLG4tVxiqivPWIE/AJUMw6KfNXsNfXn+bn+5 AmMl065oZ98NOgO/Rb3BNBYw36b2UW6y+wND1FKRee5+HL2AAUeN3ghnmiL5Ois2ptJ/ Ul9AhOgbv+rBl8xfD/Gam7yVpMo05+kAdYGmqXs9HicdkITQgyLU5fJRFD7c27MzCRLp wuWNvHxRSSKWLGiPeaMfXMR4QIVtzJFr+ItnOQUqByspx0ESvuu+rEY+pgt8Tp0mspE3 1anw== X-Gm-Message-State: ANhLgQ241G7tAwIMMhMMMWhbK3zJ5l5MK8PwMxK9gZbGrsZt9brgDUI4 6DQ7XJYj/A70mnAL/Vt3Gnb8cwQmeSwri37+zQPDirQA+KExpNUtXHOeG3yF0xcFDype9j51eQP RKmn0H7IlJyGx/4obkM0= X-Received: by 2002:a67:643:: with SMTP id 64mr2689507vsg.180.1583510769448; Fri, 06 Mar 2020 08:06:09 -0800 (PST) X-Google-Smtp-Source: ADFU+vuNqUYgxbkvUVGjTUWqsjdQuKgU9H/CdIYQsd0576LsF9BWOwvlBw/rDnf5TY7IIBpQPXyuubE7ukrhtGKK0DY= X-Received: by 2002:a67:643:: with SMTP id 64mr2689470vsg.180.1583510769018; Fri, 06 Mar 2020 08:06:09 -0800 (PST) MIME-Version: 1.0 References: <20200225073236.135581-1-ruifeng.wang@arm.com> <20200228041904.195597-1-ruifeng.wang@arm.com> <20200228041904.195597-3-ruifeng.wang@arm.com> In-Reply-To: From: David Marchand Date: Fri, 6 Mar 2020 17:05:57 +0100 Message-ID: To: Aaron Conole Cc: Ruifeng Wang , "maicolgabriel@hotmail.com" , "bruce.richardson@intel.com" , "dev@dpdk.org" , Gavin Hu , Honnappa Nagarahalli , "juraj.linkes@pantheon.tech" , nd X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [dpdk-dev] [PATCH v2 2/2] ci: add test suite run without hugepage X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Fri, Mar 6, 2020 at 4:57 PM Aaron Conole wrote: > > Ruifeng Wang writes: > > >> -----Original Message----- > >> From: Aaron Conole > >> Sent: Thursday, March 5, 2020 22:37 > >> To: Ruifeng Wang > >> Cc: maicolgabriel@hotmail.com; bruce.richardson@intel.com; dev@dpdk.or= g; > >> david.marchand@redhat.com; Gavin Hu ; Honnappa > >> Nagarahalli ; juraj.linkes@pantheon.tech= ; > >> nd > >> Subject: Re: [PATCH v2 2/2] ci: add test suite run without hugepage > >> > >> Ruifeng Wang writes: > >> > >> >> -----Original Message----- > >> >> From: Aaron Conole > >> >> Sent: Thursday, March 5, 2020 01:31 > >> >> To: Ruifeng Wang > >> >> Cc: maicolgabriel@hotmail.com; bruce.richardson@intel.com; > >> >> dev@dpdk.org; david.marchand@redhat.com; Gavin Hu > >> ; > >> >> Honnappa Nagarahalli ; > >> >> juraj.linkes@pantheon.tech; nd > >> >> Subject: Re: [PATCH v2 2/2] ci: add test suite run without hugepage > >> >> > >> >> Ruifeng Wang writes: > >> >> > >> >> > This test suite is derived from fast-tests suite. Cases in this > >> >> > suite are run with '--no-huge' flag. > >> >> > > >> >> > The suite aims to cover as many as possible test cases out of the > >> >> > fast-tests suites in the environments without huge pages support, > >> >> > like containers. > >> >> > > >> >> > Signed-off-by: Ruifeng Wang > >> >> > Reviewed-by: Gavin Hu > >> >> > --- > >> >> > >> >> I like this much more. Few comments. > >> >> > >> >> > .travis.yml | 10 +++++-- > >> >> > app/test/meson.build | 71 > >> >> > ++++++++++++++++++++++++++++++++++++++++++++ > >> >> > >> >> You should update doc/guides/prog_guide/meson_ut.rst to include som= e > >> >> detail about the new tests suite. > >> >> > >> > Thanks. Will update document in next version. > >> > > >> >> > 2 files changed, 79 insertions(+), 2 deletions(-) > >> >> > > >> >> > diff --git a/.travis.yml b/.travis.yml index b64a81bd0..eed1d96db > >> >> > 100644 > >> >> > --- a/.travis.yml > >> >> > +++ b/.travis.yml > >> >> > @@ -40,7 +40,7 @@ jobs: > >> >> > - env: DEF_LIB=3D"static" > >> >> > arch: amd64 > >> >> > compiler: gcc > >> >> > - - env: DEF_LIB=3D"shared" RUN_TESTS=3D1 > >> >> > + - env: DEF_LIB=3D"shared" RUN_TESTS=3D1 TEST_SUITES=3D"fast-te= sts > >> >> nohuge-tests" > >> >> > arch: amd64 > >> >> > compiler: gcc > >> >> > - env: DEF_LIB=3D"shared" BUILD_DOCS=3D1 @@ -63,7 +63,7 @@ job= s: > >> >> > - env: DEF_LIB=3D"static" > >> >> > arch: amd64 > >> >> > compiler: clang > >> >> > - - env: DEF_LIB=3D"shared" RUN_TESTS=3D1 > >> >> > + - env: DEF_LIB=3D"shared" RUN_TESTS=3D1 TEST_SUITES=3D"fast-te= sts > >> >> nohuge-tests" > >> >> > arch: amd64 > >> >> > compiler: clang > >> >> > - env: DEF_LIB=3D"shared" BUILD_DOCS=3D1 @@ -101,6 +101,9 @@ j= obs: > >> >> > - env: DEF_LIB=3D"static" > >> >> > arch: arm64 > >> >> > compiler: gcc > >> >> > + - env: DEF_LIB=3D"shared" RUN_TESTS=3D1 TEST_SUITES=3D"nohuge-= tests" > >> >> > + arch: arm64 > >> >> > + compiler: gcc > >> >> > - env: DEF_LIB=3D"shared" BUILD_DOCS=3D1 > >> >> > arch: arm64 > >> >> > compiler: gcc > >> >> > @@ -124,3 +127,6 @@ jobs: > >> >> > - env: DEF_LIB=3D"shared" > >> >> > arch: arm64 > >> >> > compiler: clang > >> >> > + - env: DEF_LIB=3D"shared" RUN_TESTS=3D1 TEST_SUITES=3D"nohuge-= tests" > >> >> > + arch: arm64 > >> >> > + compiler: clang > >> >> > diff --git a/app/test/meson.build b/app/test/meson.build index > >> >> > 0a2ce710f..162a1a76f 100644 > >> >> > --- a/app/test/meson.build > >> >> > +++ b/app/test/meson.build > >> >> > @@ -237,6 +237,60 @@ fast_test_names =3D [ > >> >> > 'thash_autotest', > >> >> > ] > >> >> > >> >> Shouldn't we also trim the list of fast-tests? Otherwise, these > >> >> tests will run twice. > >> >> > >> > I think you mean to have exclusive lists for fast-tests and nohuge-t= ests. > >> > >> That's what I was thinking. > >> > >> > Overlapped cases will run twice if both test suites are opted in. > >> > But the two runs are not the same, one runs with hugepage and the > >> > other runs in no-huge mode. > >> > >> Is it really so different between huge and no-huge? Most of the libra= ries > >> won't care - they call the rte_**alloc functions, and it returns block= s of > >> memory. Maybe I am simplifying it too much. > >> > >> > If fast-tests list is splited, we will need to always run multiple > >> > suites to cover all fast tests. > >> > We can keep x86 to run only fast-tests suite to avoid extra test run= s > >> > if they are not necessary. Thoughts? > >> > >> I guess since most DPDK usage will be with hugepages, we should prefer= to > >> test with it. I don't care too much about the color of this particula= r shed. If > >> you want to do it that way, it's okay by me - it gives us the coverage= , and > >> doesn't duplicate tests between those environments. > >> > >> BUT it means when we add a new test to the suite, we need to remember = to > >> add it in two places - fast_test and nohuge_test. That almost guarant= ees we > >> will miss tests (because we accidentally don't add it to nohuge). May= be > >> there's another way, like keep a list of all the tests and some inform= ation on > >> whether the test needs hugepages to run. Then if there are no hugepag= es > >> available, we can write that we SKIP the tests that don't support huge= pages. > >> In that way, we don't need two different lists - and if there are huge= pages, > >> we will run all the test cases. > >> WDYT? > >> > > Yes. Agree with you that having duplicate tests in suites is error pron= e. > > Cool! > > > IIUC, cases in a suite is determined at build time, as well as command = options to run cases. > > This implies hugepage availability needs to be detected at build time i= f we want to run only > > suitable cases in suite in an environment. It could be something we don= 't want. > > > > I'll trim fast-tests in next version to remove duplicated cases. > > I think it might be better to make the array something like (just a > psuedo-code example): > > # psuedo-code to check for hugepages > has_hugepages =3D check_for_hugepages() > > ... > > fast_test_names =3D [ > ['acl_autotest', true], > ['alarm_autotest', true], > ['atomic_autotest', true], > ... > > Then in the code: > > foreach arg : fast_test_names > .... > if not arg[1] > test(arg[0], ...) > if has_hugepages and arg[1] > test(arg[0], ) > > Does it make sense? Do you see a problem? I just hope meson won't create a dependency on hugepage availability... Thinking about stuff like: https://git.dpdk.org/dpdk/commit/?id=3D599d67b6a4bf96096352cc5fbc8fc28e54a1= ca62 --=20 David Marchand