From: Bruce Richardson <bruce.richardson@intel.com>
To: dev@dpdk.org
Cc: david.marchand@redhat.com, Bruce Richardson <bruce.richardson@intel.com>
Subject: [PATCH 1/2] app/test: emit warning for tests not in a test suite
Date: Fri, 15 Sep 2023 12:52:05 +0100 [thread overview]
Message-ID: <20230915115206.132198-2-bruce.richardson@intel.com> (raw)
In-Reply-To: <20230915115206.132198-1-bruce.richardson@intel.com>
When doing a scan for tests to add to test suites, we can also detect
the tests which have not been added to any suite. By doing so:
a) we can emit a warning informing the developer that the test is not in
a suite (if developer mode is enabled)
b) we can still register a test for that command making these tests
callable through "meson test"
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
app/test/suites/meson.build | 13 ++++++++++++-
buildtools/get-test-suites.py | 12 +++++++++---
2 files changed, 21 insertions(+), 4 deletions(-)
diff --git a/app/test/suites/meson.build b/app/test/suites/meson.build
index 19a6b902fa..478f245a54 100644
--- a/app/test/suites/meson.build
+++ b/app/test/suites/meson.build
@@ -21,7 +21,18 @@ foreach suite:test_suites
suite = suite.split('=')
suite_name = suite[0]
suite_tests = suite[1].split(',')
- if suite_name != 'fast-tests'
+ if suite_name == 'non_suite_tests'
+ # tests not in any suite
+ foreach t: suite_tests
+ if developer_mode
+ warning('Test "@0@" is not defined in any test suite'.format(t))
+ endif
+ test(t, dpdk_test,
+ env: ['DPDK_TEST=' + t],
+ timeout: timeout_seconds,
+ is_parallel: false)
+ endforeach
+ elif suite_name != 'fast-tests'
# simple cases - tests without parameters or special handling
foreach t: suite_tests
test(t, dpdk_test,
diff --git a/buildtools/get-test-suites.py b/buildtools/get-test-suites.py
index 95a9cad4c8..574c233aa8 100644
--- a/buildtools/get-test-suites.py
+++ b/buildtools/get-test-suites.py
@@ -8,18 +8,23 @@
input_list = sys.argv[1:]
test_def_regex = re.compile("REGISTER_([A-Z]+)_TEST\s*\(\s*([a-z0-9_]+)")
test_suites = {}
+# track tests not in any test suite.
+non_suite_regex = re.compile("REGISTER_TEST_COMMAND\s*\(\s*([a-z0-9_]+)")
+non_suite_tests = []
def get_fast_test_params(test_name, ln):
"Extract the extra fast-test parameters from the line"
- #print(f"ln: {ln.rstrip()}, test_name: {test_name}, split: {ln.split(test_name, 1)}")
(_, rest_of_line) = ln.split(test_name, 1)
(_, nohuge, asan, _func) = rest_of_line.split(',', 3)
return f":{nohuge.strip().lower()}:{asan.strip().lower()}"
for fname in input_list:
with open(fname) as f:
- contents = [ln for ln in f.readlines() if test_def_regex.match(ln.strip())]
- for ln in contents:
+ contents = [ln.strip() for ln in f.readlines()]
+ test_lines = [ln for ln in contents if test_def_regex.match(ln)]
+ non_suite_tests.extend([non_suite_regex.match(ln).group(1)
+ for ln in contents if non_suite_regex.match(ln)])
+ for ln in test_lines:
(test_suite, test_name) = test_def_regex.match(ln).group(1, 2)
suite_name = f"{test_suite.lower()}-tests"
if suite_name in test_suites:
@@ -31,3 +36,4 @@ def get_fast_test_params(test_name, ln):
for suite in test_suites.keys():
print(f"{suite}={','.join(test_suites[suite])}")
+print(f"non_suite_tests={','.join(non_suite_tests)}")
--
2.39.2
next prev parent reply other threads:[~2023-09-15 11:52 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-09-15 11:52 [PATCH 0/2] add checks for tests not in a suite Bruce Richardson
2023-09-15 11:52 ` Bruce Richardson [this message]
2023-09-15 11:52 ` [PATCH 2/2] devtools: check for tests added without a test suite Bruce Richardson
2023-09-15 11:55 ` [PATCH 0/2] add checks for tests not in a suite Bruce Richardson
2023-09-19 8:29 ` David Marchand
2023-09-19 8:36 ` Bruce Richardson
2023-09-19 9:07 ` David Marchand
2023-09-26 15:01 ` Aaron Conole
2023-09-27 6:30 ` David Marchand
2023-09-27 8:26 ` Bruce Richardson
2023-09-27 9:31 ` David Marchand
2023-09-27 9:56 ` Bruce Richardson
2023-09-27 14:00 ` Aaron Conole
2023-10-02 9:35 ` David Marchand
2023-10-02 9:35 ` David Marchand
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230915115206.132198-2-bruce.richardson@intel.com \
--to=bruce.richardson@intel.com \
--cc=david.marchand@redhat.com \
--cc=dev@dpdk.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).