From: Bruce Richardson <bruce.richardson@intel.com>
To: David Marchand <david.marchand@redhat.com>
Cc: dev@dpdk.org, ci@dpdk.org, "Morten Brørup" <mb@smartsharesystems.com>
Subject: Re: [PATCH v5 05/10] app/test: define unit tests suites based on test macros
Date: Wed, 16 Aug 2023 16:06:51 +0100 [thread overview]
Message-ID: <ZNzmC+W1z0jt9mtQ@bricha3-MOBL.ger.corp.intel.com> (raw)
In-Reply-To: <CAJFAV8xJ49WNNzgnp7G67gp-mE+3BcpjTSp8b8=OMdX6sFDGpg@mail.gmail.com>
On Wed, Aug 16, 2023 at 04:57:07PM +0200, David Marchand wrote:
> On Tue, Aug 15, 2023 at 5:24 PM Bruce Richardson
> <bruce.richardson@intel.com> wrote:
> >
> > Rather than having the test suites listed out in the meson.build files
> > and having to have them enabled/disabled selectively based on what libs
> > are being built, pull the tests to run from the source files which were
> > added to the build.
> >
> > Most test suites require no additional info other than the list of test
> > names in the suite. However the fast-test are special that they have
> > additional parameters associated with them. This requires some
> > additional work in the test extraction script and in the meson.build
> > file for processing the output.
> >
> > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> > Acked-by: Morten Brørup <mb@smartsharesystems.com>
> > ---
> > app/meson.build | 7 +-
> > app/test/suites/meson.build | 74 +++++++++++++++++++
> > buildtools/get-test-suites.py | 33 +++++++++
> > .../has-hugepages.py | 0
> > buildtools/meson.build | 2 +
> > 5 files changed, 115 insertions(+), 1 deletion(-)
> > create mode 100644 app/test/suites/meson.build
> > create mode 100644 buildtools/get-test-suites.py
> > rename app/test/has_hugepage.py => buildtools/has-hugepages.py (100%)
> >
> > diff --git a/app/meson.build b/app/meson.build
> > index 0d8b618e7f..c14dc80892 100644
> > --- a/app/meson.build
> > +++ b/app/meson.build
> > @@ -101,7 +101,7 @@ foreach app:apps
> > link_libs = dpdk_static_libraries + dpdk_drivers
> > endif
> >
> > - executable('dpdk-' + name,
> > + exec = executable('dpdk-' + name,
> > sources,
> > c_args: cflags,
> > link_args: ldflags,
> > @@ -110,4 +110,9 @@ foreach app:apps
> > include_directories: includes,
> > install_rpath: join_paths(get_option('prefix'), driver_install_path),
> > install: true)
> > + if name == 'test'
> > + dpdk_test = exec
> > + autotest_sources = sources
> > + subdir('test/suites') # define the pre-canned test suites
> > + endif
> > endforeach
> > diff --git a/app/test/suites/meson.build b/app/test/suites/meson.build
> > new file mode 100644
> > index 0000000000..ec74d8adf2
> > --- /dev/null
> > +++ b/app/test/suites/meson.build
> > @@ -0,0 +1,74 @@
> > +# SPDX-License-Identifier: BSD-3-Clause
> > +# Copyright(c) 2023 Intel Corporation
> > +
> > +# some perf tests (eg: memcpy perf autotest)take very long
>
> s/)take/) take/
>
Ack
> > +# to complete, so timeout to 10 minutes
> > +timeout_seconds = 600
> > +timeout_seconds_fast = 10
> > +
> > +test_no_huge_args = ['--no-huge', '-m', '2048']
> > +has_hugepage = run_command(has_hugepages_cmd, check: true).stdout().strip() != '0'
> > +message('hugepage availability: @0@'.format(has_hugepage))
> > +
> > +# process source files to determine the different unit test suites
> > +# - fast_tests
> > +# - perf_tests
> > +# - driver_tests
> > +test_suites = run_command(get_test_suites_cmd, autotest_sources,
> > + check: true).stdout().strip().split()
> > +foreach suite:test_suites
> > + # simple cases - tests without parameters or special handling
> > + suite = suite.split('=')
> > + suite_name = suite[0]
> > + suite_tests = suite[1].split(',')
> > + if suite_name != 'fast-tests'
> > + # simple cases - tests without parameters or special handling
> > + foreach t: suite_tests
> > + test(t, dpdk_test,
> > + env: ['DPDK_TEST=' + t],
> > + timeout: timeout_seconds,
> > + is_parallel: false,
> > + suite: suite_name)
> > + endforeach
> > + else
> > + # special fast-test handling here
>
> Indentation of this comment is off.
>
Ack
> > + foreach t: suite_tests
> > + params = t.split(':')
> > + test_name = params[0]
> > + nohuge = params[1] == 'true'
> > + asan = params[2] == 'true'
> > +
> > + test_args = []
> > + if nohuge
> > + test_args += test_no_huge_args
> > + elif not has_hugepage
> > + continue #skip this tests
> > + endif
> > + if not asan and (get_option('b_sanitize') == 'address'
> > + or get_option('b_sanitize') == 'address,undefined')
> > + continue # skip this test
> > + endif
> > +
> > + if get_option('default_library') == 'shared'
> > + test_args += ['-d', dpdk_drivers_build_dir]
> > + endif
> > +
> > + test(test_name, dpdk_test,
> > + args : test_args,
> > + env: ['DPDK_TEST=' + test_name],
> > + timeout : timeout_seconds_fast,
> > + is_parallel : false,
> > + suite : 'fast-tests')
> > + if not is_windows and test_name == 'trace_autotest'
> > + test_args += ['--trace=.*']
> > + test_args += ['--trace-dir=@0@'.format(meson.current_build_dir())]
> > + test(test_name + '_with_traces', dpdk_test,
> > + args : test_args,
> > + env: ['DPDK_TEST=' + test_name],
> > + timeout : timeout_seconds_fast,
> > + is_parallel : false,
> > + suite : 'fast-tests')
> > + endif
> > + endforeach
> > + endif
> > +endforeach
> > diff --git a/buildtools/get-test-suites.py b/buildtools/get-test-suites.py
> > new file mode 100644
> > index 0000000000..95a9cad4c8
> > --- /dev/null
> > +++ b/buildtools/get-test-suites.py
> > @@ -0,0 +1,33 @@
> > +#! /usr/bin/env python3
> > +# SPDX-License-Identifier: BSD-3-Clause
> > +# Copyright(c) 2023 Intel Corporation
> > +
> > +import sys
> > +import re
> > +
> > +input_list = sys.argv[1:]
> > +test_def_regex = re.compile("REGISTER_([A-Z]+)_TEST\s*\(\s*([a-z0-9_]+)")
> > +test_suites = {}
> > +
> > +def get_fast_test_params(test_name, ln):
> > + "Extract the extra fast-test parameters from the line"
> > + #print(f"ln: {ln.rstrip()}, test_name: {test_name}, split: {ln.split(test_name, 1)}")
> > + (_, rest_of_line) = ln.split(test_name, 1)
> > + (_, nohuge, asan, _func) = rest_of_line.split(',', 3)
> > + return f":{nohuge.strip().lower()}:{asan.strip().lower()}"
> > +
> > +for fname in input_list:
> > + with open(fname) as f:
> > + contents = [ln for ln in f.readlines() if test_def_regex.match(ln.strip())]
> > + for ln in contents:
> > + (test_suite, test_name) = test_def_regex.match(ln).group(1, 2)
> > + suite_name = f"{test_suite.lower()}-tests"
> > + if suite_name in test_suites:
> > + test_suites[suite_name].append(test_name)
> > + else:
> > + test_suites[suite_name] = [test_name]
> > + if suite_name == "fast-tests":
> > + test_suites["fast-tests"][-1] += get_fast_test_params(test_name, ln)
> > +
> > +for suite in test_suites.keys():
> > + print(f"{suite}={','.join(test_suites[suite])}")
> > diff --git a/app/test/has_hugepage.py b/buildtools/has-hugepages.py
> > similarity index 100%
> > rename from app/test/has_hugepage.py
> > rename to buildtools/has-hugepages.py
>
> app/test/has_hugepage.py is still referenced in MAINTAINERS.
> And new files (in this patch, and later patches) are not affiliated to someone.
>
Will look to fix in V6.
next prev parent reply other threads:[~2023-08-16 15:07 UTC|newest]
Thread overview: 72+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <20230721115125.55137-1-bruce.richardson@intel.com>
2023-08-08 8:46 ` [RFC PATCH 0/5] replace build code for unit tests David Marchand
2023-08-09 14:53 ` Patrick Robb
2023-08-09 15:05 ` Bruce Richardson
2023-08-09 15:39 ` Patrick Robb
2023-08-14 15:16 ` [PATCH v2 0/8] expand list of optional libraries Bruce Richardson
2023-08-14 15:16 ` [PATCH v2 1/8] app/test: add new macros for various test types Bruce Richardson
2023-08-14 15:16 ` [PATCH v2 2/8] app/test: tag tests with the test type Bruce Richardson
2023-08-14 15:16 ` [PATCH v2 3/8] app/test: make telemetry data test buildable on windows Bruce Richardson
2023-08-14 15:16 ` [PATCH v2 4/8] app/test: build using per-file dependency matrix Bruce Richardson
2023-08-14 15:16 ` [PATCH v2 5/8] app/test: define unit tests suites based on test macros Bruce Richardson
2023-08-14 15:16 ` [PATCH v2 6/8] examples/l3fwd: make eventdev an optional dependency Bruce Richardson
2023-08-14 15:16 ` [PATCH v2 7/8] build: make most device classes optional Bruce Richardson
2023-08-14 15:16 ` [PATCH v2 8/8] build: expand list of optional libraries Bruce Richardson
2023-08-14 15:22 ` [PATCH v2 0/8] " Bruce Richardson
2023-08-14 18:20 ` [PATCH v3 " Bruce Richardson
2023-08-14 18:20 ` [PATCH v3 1/8] app/test: add new macros for various test types Bruce Richardson
2023-08-14 18:20 ` [PATCH v3 2/8] app/test: tag tests with the test type Bruce Richardson
2023-08-14 18:20 ` [PATCH v3 3/8] app/test: make telemetry data test buildable on windows Bruce Richardson
2023-08-14 18:21 ` [PATCH v3 4/8] app/test: build using per-file dependency matrix Bruce Richardson
2023-08-14 18:21 ` [PATCH v3 5/8] app/test: define unit tests suites based on test macros Bruce Richardson
2023-08-14 18:21 ` [PATCH v3 6/8] examples/l3fwd: make eventdev an optional dependency Bruce Richardson
2023-08-14 18:21 ` [PATCH v3 7/8] build: make most device classes optional Bruce Richardson
2023-08-14 18:21 ` [PATCH v3 8/8] build: expand list of optional libraries Bruce Richardson
2023-08-14 18:33 ` [PATCH v3 0/8] " Morten Brørup
2023-08-15 13:13 ` [PATCH v4 " Bruce Richardson
2023-08-15 13:13 ` [PATCH v4 1/8] app/test: add new macros for various test types Bruce Richardson
2023-08-15 13:13 ` [PATCH v4 2/8] app/test: tag tests with the test type Bruce Richardson
2023-08-15 13:13 ` [PATCH v4 3/8] app/test: make telemetry data test buildable on windows Bruce Richardson
2023-08-15 13:13 ` [PATCH v4 4/8] app/test: build using per-file dependency matrix Bruce Richardson
2023-08-15 13:13 ` [PATCH v4 5/8] app/test: define unit tests suites based on test macros Bruce Richardson
2023-08-15 13:13 ` [PATCH v4 6/8] examples/l3fwd: make eventdev an optional dependency Bruce Richardson
2023-08-15 13:13 ` [PATCH v4 7/8] build: make most device classes optional Bruce Richardson
2023-08-15 13:13 ` [PATCH v4 8/8] build: expand list of optional libraries Bruce Richardson
2023-08-15 13:15 ` [PATCH v4 0/8] " Bruce Richardson
2023-08-15 15:10 ` [PATCH v5 00/10] " Bruce Richardson
2023-08-15 15:10 ` [PATCH v5 01/10] app/test: add new macros for various test types Bruce Richardson
2023-08-15 15:10 ` [PATCH v5 02/10] app/test: tag tests with the test type Bruce Richardson
2023-08-16 14:56 ` David Marchand
2023-08-16 15:05 ` Bruce Richardson
2023-08-15 15:10 ` [PATCH v5 03/10] app/test: make telemetry data test buildable on windows Bruce Richardson
2023-08-21 15:50 ` Tyler Retzlaff
2023-08-15 15:10 ` [PATCH v5 04/10] app/test: build using per-file dependency matrix Bruce Richardson
2023-08-15 19:05 ` Patrick Robb
2023-08-16 10:56 ` Bruce Richardson
2023-08-16 12:55 ` Bruce Richardson
2023-08-16 14:40 ` David Marchand
2023-08-16 18:29 ` Patrick Robb
2023-08-16 19:26 ` David Marchand
2023-08-16 20:38 ` Patrick Robb
2023-08-18 7:07 ` David Marchand
2023-08-18 22:33 ` Patrick Robb
2023-08-18 23:26 ` Patrick Robb
2023-08-21 7:12 ` David Marchand
2023-08-21 14:32 ` Patrick Robb
2023-08-17 7:27 ` David Marchand
2023-08-17 8:46 ` Bruce Richardson
2023-08-17 18:30 ` Patrick Robb
2023-08-15 15:10 ` [PATCH v5 05/10] app/test: define unit tests suites based on test macros Bruce Richardson
2023-08-16 11:02 ` Bruce Richardson
2023-08-16 11:15 ` David Marchand
2023-08-16 11:40 ` David Marchand
2023-08-16 12:33 ` Bruce Richardson
2023-08-16 13:16 ` Olivier Matz
2023-08-16 13:35 ` Morten Brørup
2023-08-16 13:44 ` Bruce Richardson
2023-08-16 14:57 ` David Marchand
2023-08-16 15:06 ` Bruce Richardson [this message]
2023-08-15 15:10 ` [PATCH v5 06/10] app/test: add test case for scripted telemetry commands Bruce Richardson
2023-08-15 15:10 ` [PATCH v5 07/10] app/test: add debug test suite Bruce Richardson
2023-08-15 15:10 ` [PATCH v5 08/10] examples/l3fwd: make eventdev an optional dependency Bruce Richardson
2023-08-15 15:10 ` [PATCH v5 09/10] build: make most device classes optional Bruce Richardson
2023-08-15 15:12 ` [PATCH v5 10/10] build: expand list of optional libraries Bruce Richardson
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ZNzmC+W1z0jt9mtQ@bricha3-MOBL.ger.corp.intel.com \
--to=bruce.richardson@intel.com \
--cc=ci@dpdk.org \
--cc=david.marchand@redhat.com \
--cc=dev@dpdk.org \
--cc=mb@smartsharesystems.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).