[PATCH] buildtools/get-test-suites.py: muti-line macros

[PATCH] buildtools/get-test-suites.py: muti-line macros

[Marat Khalili]

Test list is currently generated by scanning all files for macros
starting with `REGISTER_` and ending with `_TEST`. Unfortunately, this
was done line-by-line, and macros split into several lines were silently
ignored resulting in tests being excluded from test suites without any
warning.

Make regular expression multiline, capturing everything until the
closing parenthesis. (There should be no nested parentheses due to the
nature of the arguments these macros accept.)

The rest of the functionality stays the same. The result was manually
compared to be identical to the previous version.

Signed-off-by: Marat Khalili <marat.khalili@huawei.com>
---

MAINTAINERS file does not list any maintainers for the Unit tests
framework, so addressing Bruce Richardson who maintains other build
tools. Apologies in advance, feel free to ignore or redirect.

 buildtools/get-test-suites.py | 20 +++++++++++---------
 1 file changed, 11 insertions(+), 9 deletions(-)

diff --git a/buildtools/get-test-suites.py b/buildtools/get-test-suites.py
index fd22d25f36..c3a99a862e 100644
--- a/buildtools/get-test-suites.py
+++ b/buildtools/get-test-suites.py
@@ -6,10 +6,12 @@
 import re
 
 input_list = sys.argv[1:]
-test_def_regex = re.compile(r"REGISTER_([A-Z]+)_TEST\s*\(\s*([a-z0-9_]+)")
+test_def_regex = re.compile(
+        r"^\s*REGISTER_([A-Z]+)_TEST\s*\(\s*([a-z0-9_]+)[^)]*\)", re.MULTILINE)
 test_suites = {}
 # track tests not in any test suite.
-non_suite_regex = re.compile(r"REGISTER_TEST_COMMAND\s*\(\s*([a-z0-9_]+)")
+non_suite_regex = re.compile(
+        r"^\s*REGISTER_TEST_COMMAND\s*\(\s*([a-z0-9_]+)[^)]*\)", re.MULTILINE)
 non_suite_tests = []
 
 def get_fast_test_params(test_name, ln):
@@ -20,19 +22,19 @@ def get_fast_test_params(test_name, ln):
 
 for fname in input_list:
     with open(fname, "r", encoding="utf-8") as f:
-        contents = [ln.strip() for ln in f.readlines()]
-        test_lines = [ln for ln in contents if test_def_regex.match(ln)]
-        non_suite_tests.extend([non_suite_regex.match(ln).group(1)
-                for ln in contents if non_suite_regex.match(ln)])
-    for ln in test_lines:
-        (test_suite, test_name) = test_def_regex.match(ln).group(1, 2)
+        contents = f.read()
+    non_suite_tests.extend(
+            match.group(1) for match in non_suite_regex.finditer(contents))
+    for match in test_def_regex.finditer(contents):
+        (test_suite, test_name) = match.group(1, 2)
         suite_name = f"{test_suite.lower()}-tests"
         if suite_name in test_suites:
             test_suites[suite_name].append(test_name)
         else:
             test_suites[suite_name] = [test_name]
         if suite_name == "fast-tests":
-            test_suites["fast-tests"][-1] += get_fast_test_params(test_name, ln)
+            test_suites["fast-tests"][-1] += get_fast_test_params(
+                    test_name, match.group(0))
 
 for suite in test_suites.keys():
     print(f"{suite}={','.join(test_suites[suite])}")
-- 
2.43.0

Re: [PATCH] buildtools/get-test-suites.py: muti-line macros

[Thomas Monjalon]

18/06/2025 14:39, Marat Khalili:
> Test list is currently generated by scanning all files for macros
> starting with `REGISTER_` and ending with `_TEST`. Unfortunately, this
> was done line-by-line, and macros split into several lines were silently
> ignored resulting in tests being excluded from test suites without any
> warning.
> 
> Make regular expression multiline, capturing everything until the
> closing parenthesis. (There should be no nested parentheses due to the
> nature of the arguments these macros accept.)
> 
> The rest of the functionality stays the same. The result was manually
> compared to be identical to the previous version.
> 
> Signed-off-by: Marat Khalili <marat.khalili@huawei.com>

Applied, thanks.