From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3EE3043081; Wed, 16 Aug 2023 16:57:25 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 3817043257; Wed, 16 Aug 2023 16:57:25 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id 27CE240693 for ; Wed, 16 Aug 2023 16:57:22 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1692197841; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=glF8j5eDurCs/oQqXtX9k7Awhl3gwAZqhD15I8Ad++Q=; b=Uy7k58yqe11EDj+pjfzZafxDW2Ab79PbIto3jOaHwXHxcyL+vUQQdyyymU3r0A9xIiJ9u6 W/HdBm/3kg4ky3/WMnPuu+iEwnY4AII5bDthD6UnKpt2xt6/RGQqsG5D2U2RS+IQZnRCJc k+E7fuwuSlChZ8eKboQWLGl3pWuvLew= Received: from mail-lj1-f200.google.com (mail-lj1-f200.google.com [209.85.208.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-147-MGShNptFMEyOgAOP8oHUNA-1; Wed, 16 Aug 2023 10:57:20 -0400 X-MC-Unique: MGShNptFMEyOgAOP8oHUNA-1 Received: by mail-lj1-f200.google.com with SMTP id 38308e7fff4ca-2b9dc1bfdd2so67724431fa.1 for ; Wed, 16 Aug 2023 07:57:19 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692197838; x=1692802638; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=glF8j5eDurCs/oQqXtX9k7Awhl3gwAZqhD15I8Ad++Q=; b=e23Py58CWett1eK4ATxMznFFA8Y6gnWy1BytXScoc5vPEiDYsm03BBZRjJMmJVcqUz vtpL7GKdZAAmVoeJupEu1kOtWFRIcrjbD2bJZCOJ7TZ6QiCtgMAYcPNx1p2rFBIjxkuC bmV8xYyRbThZWr7z1S/B2dhEcuxHnYFZ96ap93jebsAf+/c4yt9wrmBCXs+JKRd2c8u+ 5UYY52BZVucI6vJx76rVO5Zgm3ba2kVHg2T9qUA9E5vPPnVLeDD+Bkxq6BycBsPuF2Ix O08vP9cZlRQT76/EPinOog9eahPvDXAXU3VutD+/aA823qeoY7NYRH5204aIpxZpPX4Q QyqQ== X-Gm-Message-State: AOJu0Yxs/lv5RWAc9bGQSeY3F/ZD5WLQMZMu15lHkldPQDBpQRUFOAZS 68t/5XXOm6VwbtHuvSwromBUtr450vMs99Kc0nxK9e4mCtlgMVv2aoEyhpidzWbHMkIxb/OaBXJ XKh35iFKGqlCDGW4O/w== X-Received: by 2002:ac2:5f6e:0:b0:4f9:596d:c803 with SMTP id c14-20020ac25f6e000000b004f9596dc803mr1557273lfc.53.1692197838696; Wed, 16 Aug 2023 07:57:18 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEPxv8d6YN2ytvRh85IjRRLehjvAXvlOw6MieqZN1VGykv28dIF00g4QTnl1LejgsUJWQlm871j9KGYT25HLpE= X-Received: by 2002:ac2:5f6e:0:b0:4f9:596d:c803 with SMTP id c14-20020ac25f6e000000b004f9596dc803mr1557265lfc.53.1692197838453; Wed, 16 Aug 2023 07:57:18 -0700 (PDT) MIME-Version: 1.0 References: <20230721115125.55137-1-bruce.richardson@intel.com> <20230815151053.996469-1-bruce.richardson@intel.com> <20230815151053.996469-6-bruce.richardson@intel.com> In-Reply-To: <20230815151053.996469-6-bruce.richardson@intel.com> From: David Marchand Date: Wed, 16 Aug 2023 16:57:07 +0200 Message-ID: Subject: Re: [PATCH v5 05/10] app/test: define unit tests suites based on test macros To: Bruce Richardson Cc: dev@dpdk.org, ci@dpdk.org, =?UTF-8?Q?Morten_Br=C3=B8rup?= X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ci-bounces@dpdk.org On Tue, Aug 15, 2023 at 5:24=E2=80=AFPM Bruce Richardson wrote: > > Rather than having the test suites listed out in the meson.build files > and having to have them enabled/disabled selectively based on what libs > are being built, pull the tests to run from the source files which were > added to the build. > > Most test suites require no additional info other than the list of test > names in the suite. However the fast-test are special that they have > additional parameters associated with them. This requires some > additional work in the test extraction script and in the meson.build > file for processing the output. > > Signed-off-by: Bruce Richardson > Acked-by: Morten Br=C3=B8rup > --- > app/meson.build | 7 +- > app/test/suites/meson.build | 74 +++++++++++++++++++ > buildtools/get-test-suites.py | 33 +++++++++ > .../has-hugepages.py | 0 > buildtools/meson.build | 2 + > 5 files changed, 115 insertions(+), 1 deletion(-) > create mode 100644 app/test/suites/meson.build > create mode 100644 buildtools/get-test-suites.py > rename app/test/has_hugepage.py =3D> buildtools/has-hugepages.py (100%) > > diff --git a/app/meson.build b/app/meson.build > index 0d8b618e7f..c14dc80892 100644 > --- a/app/meson.build > +++ b/app/meson.build > @@ -101,7 +101,7 @@ foreach app:apps > link_libs =3D dpdk_static_libraries + dpdk_drivers > endif > > - executable('dpdk-' + name, > + exec =3D executable('dpdk-' + name, > sources, > c_args: cflags, > link_args: ldflags, > @@ -110,4 +110,9 @@ foreach app:apps > include_directories: includes, > install_rpath: join_paths(get_option('prefix'), driver_insta= ll_path), > install: true) > + if name =3D=3D 'test' > + dpdk_test =3D exec > + autotest_sources =3D sources > + subdir('test/suites') # define the pre-canned test suites > + endif > endforeach > diff --git a/app/test/suites/meson.build b/app/test/suites/meson.build > new file mode 100644 > index 0000000000..ec74d8adf2 > --- /dev/null > +++ b/app/test/suites/meson.build > @@ -0,0 +1,74 @@ > +# SPDX-License-Identifier: BSD-3-Clause > +# Copyright(c) 2023 Intel Corporation > + > +# some perf tests (eg: memcpy perf autotest)take very long s/)take/) take/ > +# to complete, so timeout to 10 minutes > +timeout_seconds =3D 600 > +timeout_seconds_fast =3D 10 > + > +test_no_huge_args =3D ['--no-huge', '-m', '2048'] > +has_hugepage =3D run_command(has_hugepages_cmd, check: true).stdout().st= rip() !=3D '0' > +message('hugepage availability: @0@'.format(has_hugepage)) > + > +# process source files to determine the different unit test suites > +# - fast_tests > +# - perf_tests > +# - driver_tests > +test_suites =3D run_command(get_test_suites_cmd, autotest_sources, > + check: true).stdout().strip().split() > +foreach suite:test_suites > + # simple cases - tests without parameters or special handling > + suite =3D suite.split('=3D') > + suite_name =3D suite[0] > + suite_tests =3D suite[1].split(',') > + if suite_name !=3D 'fast-tests' > + # simple cases - tests without parameters or special handling > + foreach t: suite_tests > + test(t, dpdk_test, > + env: ['DPDK_TEST=3D' + t], > + timeout: timeout_seconds, > + is_parallel: false, > + suite: suite_name) > + endforeach > + else > + # special fast-test handling here Indentation of this comment is off. > + foreach t: suite_tests > + params =3D t.split(':') > + test_name =3D params[0] > + nohuge =3D params[1] =3D=3D 'true' > + asan =3D params[2] =3D=3D 'true' > + > + test_args =3D [] > + if nohuge > + test_args +=3D test_no_huge_args > + elif not has_hugepage > + continue #skip this tests > + endif > + if not asan and (get_option('b_sanitize') =3D=3D 'address' > + or get_option('b_sanitize') =3D=3D 'address,undefine= d') > + continue # skip this test > + endif > + > + if get_option('default_library') =3D=3D 'shared' > + test_args +=3D ['-d', dpdk_drivers_build_dir] > + endif > + > + test(test_name, dpdk_test, > + args : test_args, > + env: ['DPDK_TEST=3D' + test_name], > + timeout : timeout_seconds_fast, > + is_parallel : false, > + suite : 'fast-tests') > + if not is_windows and test_name =3D=3D 'trace_autotest' > + test_args +=3D ['--trace=3D.*'] > + test_args +=3D ['--trace-dir=3D@0@'.format(meson.current= _build_dir())] > + test(test_name + '_with_traces', dpdk_test, > + args : test_args, > + env: ['DPDK_TEST=3D' + test_name], > + timeout : timeout_seconds_fast, > + is_parallel : false, > + suite : 'fast-tests') > + endif > + endforeach > + endif > +endforeach > diff --git a/buildtools/get-test-suites.py b/buildtools/get-test-suites.p= y > new file mode 100644 > index 0000000000..95a9cad4c8 > --- /dev/null > +++ b/buildtools/get-test-suites.py > @@ -0,0 +1,33 @@ > +#! /usr/bin/env python3 > +# SPDX-License-Identifier: BSD-3-Clause > +# Copyright(c) 2023 Intel Corporation > + > +import sys > +import re > + > +input_list =3D sys.argv[1:] > +test_def_regex =3D re.compile("REGISTER_([A-Z]+)_TEST\s*\(\s*([a-z0-9_]+= )") > +test_suites =3D {} > + > +def get_fast_test_params(test_name, ln): > + "Extract the extra fast-test parameters from the line" > + #print(f"ln: {ln.rstrip()}, test_name: {test_name}, split: {ln.split= (test_name, 1)}") > + (_, rest_of_line) =3D ln.split(test_name, 1) > + (_, nohuge, asan, _func) =3D rest_of_line.split(',', 3) > + return f":{nohuge.strip().lower()}:{asan.strip().lower()}" > + > +for fname in input_list: > + with open(fname) as f: > + contents =3D [ln for ln in f.readlines() if test_def_regex.match= (ln.strip())] > + for ln in contents: > + (test_suite, test_name) =3D test_def_regex.match(ln).group(1, 2) > + suite_name =3D f"{test_suite.lower()}-tests" > + if suite_name in test_suites: > + test_suites[suite_name].append(test_name) > + else: > + test_suites[suite_name] =3D [test_name] > + if suite_name =3D=3D "fast-tests": > + test_suites["fast-tests"][-1] +=3D get_fast_test_params(test= _name, ln) > + > +for suite in test_suites.keys(): > + print(f"{suite}=3D{','.join(test_suites[suite])}") > diff --git a/app/test/has_hugepage.py b/buildtools/has-hugepages.py > similarity index 100% > rename from app/test/has_hugepage.py > rename to buildtools/has-hugepages.py app/test/has_hugepage.py is still referenced in MAINTAINERS. And new files (in this patch, and later patches) are not affiliated to some= one. > diff --git a/buildtools/meson.build b/buildtools/meson.build > index e1c600e40f..ac5e4dcf08 100644 > --- a/buildtools/meson.build > +++ b/buildtools/meson.build > @@ -18,6 +18,8 @@ map_to_win_cmd =3D py3 + files('map_to_win.py') > sphinx_wrapper =3D py3 + files('call-sphinx-build.py') > get_cpu_count_cmd =3D py3 + files('get-cpu-count.py') > get_numa_count_cmd =3D py3 + files('get-numa-count.py') > +get_test_suites_cmd =3D py3 + files('get-test-suites.py') > +has_hugepages_cmd =3D py3 + files('has-hugepages.py') > binutils_avx512_check =3D (py3 + files('binutils-avx512-check.py') + > [objdump] + cc.cmd_array()) > > -- > 2.39.2 > --=20 David Marchand