DPDK CI discussions
 help / color / mirror / Atom feed
From: Bruce Richardson <bruce.richardson@intel.com>
To: dev@dpdk.org
Cc: ci@dpdk.org, Bruce Richardson <bruce.richardson@intel.com>
Subject: [PATCH v3 5/8] app/test: define unit tests suites based on test macros
Date: Mon, 14 Aug 2023 19:21:01 +0100	[thread overview]
Message-ID: <20230814182104.470270-6-bruce.richardson@intel.com> (raw)
In-Reply-To: <20230814182104.470270-1-bruce.richardson@intel.com>

Rather than having the test suites listed out in the meson.build files
and having to have them enabled/disabled selectively based on what libs
are being built, pull the tests to run from the source files which were
added to the build.

Most test suites require no additional info other than the list of test
names in the suite. However the fast-test are special that they have
additional parameters associated with them. This requires some
additional work in the test extraction script and in the meson.build
file for processing the output.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 app/meson.build                               |  7 +-
 app/test/suites/meson.build                   | 76 +++++++++++++++++++
 buildtools/get-test-suites.py                 | 33 ++++++++
 .../has-hugepages.py                          |  0
 buildtools/meson.build                        |  2 +
 5 files changed, 117 insertions(+), 1 deletion(-)
 create mode 100644 app/test/suites/meson.build
 create mode 100644 buildtools/get-test-suites.py
 rename app/test/has_hugepage.py => buildtools/has-hugepages.py (100%)

diff --git a/app/meson.build b/app/meson.build
index 0d8b618e7f..c14dc80892 100644
--- a/app/meson.build
+++ b/app/meson.build
@@ -101,7 +101,7 @@ foreach app:apps
         link_libs = dpdk_static_libraries + dpdk_drivers
     endif
 
-    executable('dpdk-' + name,
+    exec = executable('dpdk-' + name,
             sources,
             c_args: cflags,
             link_args: ldflags,
@@ -110,4 +110,9 @@ foreach app:apps
             include_directories: includes,
             install_rpath: join_paths(get_option('prefix'), driver_install_path),
             install: true)
+    if name == 'test'
+        dpdk_test = exec
+        autotest_sources = sources
+        subdir('test/suites')  # define the pre-canned test suites
+    endif
 endforeach
diff --git a/app/test/suites/meson.build b/app/test/suites/meson.build
new file mode 100644
index 0000000000..df0c66ff0d
--- /dev/null
+++ b/app/test/suites/meson.build
@@ -0,0 +1,76 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2023 Intel Corporation
+
+# some perf tests (eg: memcpy perf autotest)take very long
+# to complete, so timeout to 10 minutes
+timeout_seconds = 600
+timeout_seconds_fast = 10
+
+test_no_huge_args = ['--no-huge', '-m', '2048']
+has_hugepage = run_command(has_hugepages_cmd, check: true).stdout().strip() != '0'
+message('hugepage availability: @0@'.format(has_hugepage))
+
+# process source files to determine the different unit test suites
+# - fast_tests
+# - perf_tests
+# - driver_tests
+test_suites = run_command(get_test_suites_cmd, autotest_sources,
+         check: true).stdout().strip().split()
+foreach suite:test_suites
+    # simple cases - tests without parameters or special handling
+    suite = suite.split('=')
+    suite_name = suite[0]
+    suite_tests = suite[1].split(',')
+    if suite_name != 'fast-tests'
+        # simple cases - tests without parameters or special handling
+        foreach t: suite_tests
+            test(t, dpdk_test,
+                    args: ['--', t],
+                    timeout: timeout_seconds,
+                    is_parallel: false,
+                    suite: suite_name)
+        endforeach
+    else
+    # special fast-test handling here
+        foreach t: suite_tests
+            params = t.split(':')
+            test_name = params[0]
+            nohuge = params[1] == 'true'
+            asan = params[2] == 'true'
+
+            test_args = []
+            if nohuge
+                test_args += test_no_huge_args
+            elif not has_hugepage
+                continue  #skip this tests
+            endif
+            if not asan and (get_option('b_sanitize') == 'address'
+                    or get_option('b_sanitize') == 'address,undefined')
+                continue  # skip this test
+            endif
+
+            if (get_option('default_library') == 'shared'
+                    and test_name == 'event_eth_tx_adapter_autotest')
+                test_args += ['-d', dpdk_drivers_build_dir]
+            endif
+            if is_linux
+                test_args += ['--file-prefix=@0@'.format(arg[0])]
+            endif
+
+            test(test_name, dpdk_test,
+                args : test_args + ['--', test_name],
+                timeout : timeout_seconds_fast,
+                is_parallel : false,
+                suite : 'fast-tests')
+            if not is_windows and test_name == 'trace_autotest'
+                test_args += ['--trace=.*']
+                test_args += ['--trace-dir=@0@'.format(meson.current_build_dir())]
+                test(test_name + '_with_traces', dpdk_test,
+                    args : test_args + ['--', test_name],
+                    timeout : timeout_seconds_fast,
+                    is_parallel : false,
+                    suite : 'fast-tests')
+            endif
+        endforeach
+    endif
+endforeach
diff --git a/buildtools/get-test-suites.py b/buildtools/get-test-suites.py
new file mode 100644
index 0000000000..95a9cad4c8
--- /dev/null
+++ b/buildtools/get-test-suites.py
@@ -0,0 +1,33 @@
+#! /usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2023 Intel Corporation
+
+import sys
+import re
+
+input_list = sys.argv[1:]
+test_def_regex = re.compile("REGISTER_([A-Z]+)_TEST\s*\(\s*([a-z0-9_]+)")
+test_suites = {}
+
+def get_fast_test_params(test_name, ln):
+    "Extract the extra fast-test parameters from the line"
+    #print(f"ln: {ln.rstrip()}, test_name: {test_name}, split: {ln.split(test_name, 1)}")
+    (_, rest_of_line) = ln.split(test_name, 1)
+    (_, nohuge, asan, _func) = rest_of_line.split(',', 3)
+    return f":{nohuge.strip().lower()}:{asan.strip().lower()}"
+
+for fname in input_list:
+    with open(fname) as f:
+        contents = [ln for ln in f.readlines() if test_def_regex.match(ln.strip())]
+    for ln in contents:
+        (test_suite, test_name) = test_def_regex.match(ln).group(1, 2)
+        suite_name = f"{test_suite.lower()}-tests"
+        if suite_name in test_suites:
+            test_suites[suite_name].append(test_name)
+        else:
+            test_suites[suite_name] = [test_name]
+        if suite_name == "fast-tests":
+            test_suites["fast-tests"][-1] += get_fast_test_params(test_name, ln)
+
+for suite in test_suites.keys():
+    print(f"{suite}={','.join(test_suites[suite])}")
diff --git a/app/test/has_hugepage.py b/buildtools/has-hugepages.py
similarity index 100%
rename from app/test/has_hugepage.py
rename to buildtools/has-hugepages.py
diff --git a/buildtools/meson.build b/buildtools/meson.build
index e1c600e40f..ac5e4dcf08 100644
--- a/buildtools/meson.build
+++ b/buildtools/meson.build
@@ -18,6 +18,8 @@ map_to_win_cmd = py3 + files('map_to_win.py')
 sphinx_wrapper = py3 + files('call-sphinx-build.py')
 get_cpu_count_cmd = py3 + files('get-cpu-count.py')
 get_numa_count_cmd = py3 + files('get-numa-count.py')
+get_test_suites_cmd = py3 + files('get-test-suites.py')
+has_hugepages_cmd = py3 + files('has-hugepages.py')
 binutils_avx512_check = (py3 + files('binutils-avx512-check.py') +
                         [objdump] + cc.cmd_array())
 
-- 
2.39.2


  parent reply	other threads:[~2023-08-14 18:21 UTC|newest]

Thread overview: 72+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <20230721115125.55137-1-bruce.richardson@intel.com>
2023-08-08  8:46 ` [RFC PATCH 0/5] replace build code for unit tests David Marchand
2023-08-09 14:53   ` Patrick Robb
2023-08-09 15:05     ` Bruce Richardson
2023-08-09 15:39       ` Patrick Robb
2023-08-14 15:16 ` [PATCH v2 0/8] expand list of optional libraries Bruce Richardson
2023-08-14 15:16   ` [PATCH v2 1/8] app/test: add new macros for various test types Bruce Richardson
2023-08-14 15:16   ` [PATCH v2 2/8] app/test: tag tests with the test type Bruce Richardson
2023-08-14 15:16   ` [PATCH v2 3/8] app/test: make telemetry data test buildable on windows Bruce Richardson
2023-08-14 15:16   ` [PATCH v2 4/8] app/test: build using per-file dependency matrix Bruce Richardson
2023-08-14 15:16   ` [PATCH v2 5/8] app/test: define unit tests suites based on test macros Bruce Richardson
2023-08-14 15:16   ` [PATCH v2 6/8] examples/l3fwd: make eventdev an optional dependency Bruce Richardson
2023-08-14 15:16   ` [PATCH v2 7/8] build: make most device classes optional Bruce Richardson
2023-08-14 15:16   ` [PATCH v2 8/8] build: expand list of optional libraries Bruce Richardson
2023-08-14 15:22   ` [PATCH v2 0/8] " Bruce Richardson
2023-08-14 18:20 ` [PATCH v3 " Bruce Richardson
2023-08-14 18:20   ` [PATCH v3 1/8] app/test: add new macros for various test types Bruce Richardson
2023-08-14 18:20   ` [PATCH v3 2/8] app/test: tag tests with the test type Bruce Richardson
2023-08-14 18:20   ` [PATCH v3 3/8] app/test: make telemetry data test buildable on windows Bruce Richardson
2023-08-14 18:21   ` [PATCH v3 4/8] app/test: build using per-file dependency matrix Bruce Richardson
2023-08-14 18:21   ` Bruce Richardson [this message]
2023-08-14 18:21   ` [PATCH v3 6/8] examples/l3fwd: make eventdev an optional dependency Bruce Richardson
2023-08-14 18:21   ` [PATCH v3 7/8] build: make most device classes optional Bruce Richardson
2023-08-14 18:21   ` [PATCH v3 8/8] build: expand list of optional libraries Bruce Richardson
2023-08-14 18:33   ` [PATCH v3 0/8] " Morten Brørup
2023-08-15 13:13 ` [PATCH v4 " Bruce Richardson
2023-08-15 13:13   ` [PATCH v4 1/8] app/test: add new macros for various test types Bruce Richardson
2023-08-15 13:13   ` [PATCH v4 2/8] app/test: tag tests with the test type Bruce Richardson
2023-08-15 13:13   ` [PATCH v4 3/8] app/test: make telemetry data test buildable on windows Bruce Richardson
2023-08-15 13:13   ` [PATCH v4 4/8] app/test: build using per-file dependency matrix Bruce Richardson
2023-08-15 13:13   ` [PATCH v4 5/8] app/test: define unit tests suites based on test macros Bruce Richardson
2023-08-15 13:13   ` [PATCH v4 6/8] examples/l3fwd: make eventdev an optional dependency Bruce Richardson
2023-08-15 13:13   ` [PATCH v4 7/8] build: make most device classes optional Bruce Richardson
2023-08-15 13:13   ` [PATCH v4 8/8] build: expand list of optional libraries Bruce Richardson
2023-08-15 13:15   ` [PATCH v4 0/8] " Bruce Richardson
2023-08-15 15:10 ` [PATCH v5 00/10] " Bruce Richardson
2023-08-15 15:10   ` [PATCH v5 01/10] app/test: add new macros for various test types Bruce Richardson
2023-08-15 15:10   ` [PATCH v5 02/10] app/test: tag tests with the test type Bruce Richardson
2023-08-16 14:56     ` David Marchand
2023-08-16 15:05       ` Bruce Richardson
2023-08-15 15:10   ` [PATCH v5 03/10] app/test: make telemetry data test buildable on windows Bruce Richardson
2023-08-21 15:50     ` Tyler Retzlaff
2023-08-15 15:10   ` [PATCH v5 04/10] app/test: build using per-file dependency matrix Bruce Richardson
2023-08-15 19:05     ` Patrick Robb
2023-08-16 10:56       ` Bruce Richardson
2023-08-16 12:55       ` Bruce Richardson
2023-08-16 14:40         ` David Marchand
2023-08-16 18:29           ` Patrick Robb
2023-08-16 19:26             ` David Marchand
2023-08-16 20:38               ` Patrick Robb
2023-08-18  7:07               ` David Marchand
2023-08-18 22:33                 ` Patrick Robb
2023-08-18 23:26                   ` Patrick Robb
2023-08-21  7:12                     ` David Marchand
2023-08-21 14:32                       ` Patrick Robb
2023-08-17  7:27             ` David Marchand
2023-08-17  8:46               ` Bruce Richardson
2023-08-17 18:30               ` Patrick Robb
2023-08-15 15:10   ` [PATCH v5 05/10] app/test: define unit tests suites based on test macros Bruce Richardson
2023-08-16 11:02     ` Bruce Richardson
2023-08-16 11:15       ` David Marchand
2023-08-16 11:40         ` David Marchand
2023-08-16 12:33           ` Bruce Richardson
2023-08-16 13:16             ` Olivier Matz
2023-08-16 13:35               ` Morten Brørup
2023-08-16 13:44                 ` Bruce Richardson
2023-08-16 14:57     ` David Marchand
2023-08-16 15:06       ` Bruce Richardson
2023-08-15 15:10   ` [PATCH v5 06/10] app/test: add test case for scripted telemetry commands Bruce Richardson
2023-08-15 15:10   ` [PATCH v5 07/10] app/test: add debug test suite Bruce Richardson
2023-08-15 15:10   ` [PATCH v5 08/10] examples/l3fwd: make eventdev an optional dependency Bruce Richardson
2023-08-15 15:10   ` [PATCH v5 09/10] build: make most device classes optional Bruce Richardson
2023-08-15 15:12   ` [PATCH v5 10/10] build: expand list of optional libraries Bruce Richardson

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20230814182104.470270-6-bruce.richardson@intel.com \
    --to=bruce.richardson@intel.com \
    --cc=ci@dpdk.org \
    --cc=dev@dpdk.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).