DPDK patches and discussions
 help / color / mirror / Atom feed
* [RFC PATCH 0/5] replace build code for unit tests
@ 2023-07-21 11:51 Bruce Richardson
  2023-07-21 11:51 ` [RFC PATCH 1/5] app/test: add new macros for various test types Bruce Richardson
                   ` (10 more replies)
  0 siblings, 11 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-07-21 11:51 UTC (permalink / raw)
  To: dev; +Cc: Bruce Richardson

While working on series such as [1] to make DPDK builds more
configurable, the majority of the complexity involved centered on the
unit tests, and ensuring that the individual files were added/removed
from the build as the components were enabled/disabled.

This complexity was exacerbated by:
* C files having multiple dependencies, specifically,
* Some libraries having one set of independent tests, while other
  library tests requiring additional components
* Having to maintain lists of tests in various suites which also needed
  to be updated, depending on component availability.
* Everything having to be checked via if-else clauses, since this was
  not data-driven

This RFC attempts to address these issues by doing the following:
* Creating a per-file dependency list for the autotests, allowing easy
  checks of when a file could be built
* Tagging the various fast-tests/perf-tests/drivers-tests suites via
  macros in the C code. Then scan the C-files we are building to
  construct the test suites based on what is available


This RFC is just an early prototype, and probably needs cleanup and
validation of the dependency lists. Windows support is probably also
broken, given the fact that our current build file maintains custom
lists there. However, I would hope it should not be too big a job to
enable the unit tests for the windows components that are already
available. (Tests for any components not being build on windows are
already removed via the dependency lists).

Another (hopefully less-impactful) side-effect of this change is that
the "test" binary has moved in the build folder. It is now put
along-side the other binaries in "build/app" rather than being in the
subdirectory "build/app/test".

Future work could also be done to scan the C files that are not being
build for tests. Those tests could then be stubbed out in an
auto-genereated file to always return "SKIPPED", meaning that all test
commands are always valid, irrespective of what components are present
or not.

[1] http://patches.dpdk.org/project/dpdk/list/?series=28628&state=10

Bruce Richardson (5):
  app/test: add new macros for various test types
  app/test: tag tests with the test type
  app/test: build using per-file dependency matrix
  app/test: define unit tests suites based on test macros
  app/test: add fast test suite to new build infrastructure

 app/meson.build                               |  15 +-
 app/test/meson.build                          | 796 +++++-------------
 app/test/suites/meson.build                   |  75 ++
 app/test/test.h                               |   9 +-
 app/test/test_acl.c                           |   2 +-
 app/test/test_atomic.c                        |   2 +-
 app/test/test_barrier.c                       |   2 +-
 app/test/test_bitmap.c                        |   2 +-
 app/test/test_bitops.c                        |   2 +-
 app/test/test_bpf.c                           |   4 +-
 app/test/test_byteorder.c                     |   2 +-
 app/test/test_cksum.c                         |   2 +-
 app/test/test_cmdline.c                       |   2 +-
 app/test/test_common.c                        |   2 +-
 app/test/test_cpuflags.c                      |   2 +-
 app/test/test_crc.c                           |   2 +-
 app/test/test_cryptodev.c                     |  38 +-
 app/test/test_cryptodev_asym.c                |   2 +-
 app/test/test_cycles.c                        |   2 +-
 app/test/test_debug.c                         |   2 +-
 app/test/test_devargs.c                       |   2 +-
 app/test/test_distributor.c                   |   2 +-
 app/test/test_distributor_perf.c              |   2 +-
 app/test/test_dmadev.c                        |   2 +-
 app/test/test_eal_flags.c                     |  24 +-
 app/test/test_eal_fs.c                        |   2 +-
 app/test/test_efd.c                           |   2 +-
 app/test/test_efd_perf.c                      |   2 +-
 app/test/test_errno.c                         |   2 +-
 app/test/test_ethdev_link.c                   |   2 +-
 app/test/test_event_ring.c                    |   2 +-
 app/test/test_eventdev.c                      |   2 +-
 app/test/test_fbarray.c                       |   2 +-
 app/test/test_fib.c                           |   4 +-
 app/test/test_fib6.c                          |   4 +-
 app/test/test_fib6_perf.c                     |   2 +-
 app/test/test_fib_perf.c                      |   2 +-
 app/test/test_func_reentrancy.c               |   2 +-
 app/test/test_hash.c                          |   2 +-
 app/test/test_hash_functions.c                |   2 +-
 app/test/test_hash_multiwriter.c              |   2 +-
 app/test/test_hash_perf.c                     |   2 +-
 app/test/test_hash_readwrite.c                |   4 +-
 app/test/test_hash_readwrite_lf_perf.c        |   2 +-
 app/test/test_interrupts.c                    |   2 +-
 app/test/test_ipfrag.c                        |   2 +-
 app/test/test_ipsec.c                         |   2 +-
 app/test/test_ipsec_perf.c                    |   2 +-
 app/test/test_kni.c                           |   2 +-
 app/test/test_kvargs.c                        |   2 +-
 app/test/test_lcores.c                        |   2 +-
 app/test/test_logs.c                          |   2 +-
 app/test/test_lpm.c                           |   2 +-
 app/test/test_lpm6.c                          |   2 +-
 app/test/test_lpm6_perf.c                     |   2 +-
 app/test/test_lpm_perf.c                      |   2 +-
 app/test/test_malloc.c                        |   2 +-
 app/test/test_malloc_perf.c                   |   2 +-
 app/test/test_mbuf.c                          |   2 +-
 app/test/test_mcslock.c                       |   2 +-
 app/test/test_member.c                        |   2 +-
 app/test/test_member_perf.c                   |   2 +-
 app/test/test_memcpy.c                        |   2 +-
 app/test/test_memcpy_perf.c                   |   2 +-
 app/test/test_memory.c                        |   2 +-
 app/test/test_mempool.c                       |   2 +-
 app/test/test_mempool_perf.c                  |   2 +-
 app/test/test_memzone.c                       |   2 +-
 app/test/test_meter.c                         |   2 +-
 app/test/test_mp_secondary.c                  |   2 +-
 app/test/test_per_lcore.c                     |   2 +-
 app/test/test_pflock.c                        |   2 +-
 app/test/test_pie.c                           |   6 +-
 app/test/test_pmd_perf.c                      |   2 +-
 app/test/test_power.c                         |   2 +-
 app/test/test_power_cpufreq.c                 |   2 +-
 app/test/test_power_intel_uncore.c            |   2 +-
 app/test/test_power_kvm_vm.c                  |   2 +-
 app/test/test_prefetch.c                      |   2 +-
 app/test/test_rand_perf.c                     |   2 +-
 app/test/test_rcu_qsbr.c                      |   2 +-
 app/test/test_rcu_qsbr_perf.c                 |   2 +-
 app/test/test_reassembly_perf.c               |   2 +-
 app/test/test_reciprocal_division.c           |   2 +-
 app/test/test_reciprocal_division_perf.c      |   2 +-
 app/test/test_red.c                           |   4 +-
 app/test/test_reorder.c                       |   2 +-
 app/test/test_rib.c                           |   4 +-
 app/test/test_rib6.c                          |   4 +-
 app/test/test_ring.c                          |   2 +-
 app/test/test_ring_perf.c                     |   2 +-
 app/test/test_rwlock.c                        |   8 +-
 app/test/test_sched.c                         |   2 +-
 app/test/test_security.c                      |   2 +-
 app/test/test_seqlock.c                       |   2 +-
 app/test/test_service_cores.c                 |   4 +-
 app/test/test_spinlock.c                      |   2 +-
 app/test/test_stack.c                         |   4 +-
 app/test/test_stack_perf.c                    |   4 +-
 app/test/test_string_fns.c                    |   2 +-
 app/test/test_tailq.c                         |   2 +-
 app/test/test_thash.c                         |   2 +-
 app/test/test_thash_perf.c                    |   2 +-
 app/test/test_threads.c                       |   2 +-
 app/test/test_ticketlock.c                    |   2 +-
 app/test/test_timer.c                         |   2 +-
 app/test/test_timer_perf.c                    |   2 +-
 app/test/test_timer_racecond.c                |   2 +-
 app/test/test_trace.c                         |   2 +-
 app/test/test_trace_perf.c                    |   2 +-
 app/test/test_version.c                       |   2 +-
 buildtools/get-test-suites.py                 |  33 +
 .../has-hugepages.py                          |   0
 buildtools/meson.build                        |   2 +
 114 files changed, 478 insertions(+), 754 deletions(-)
 create mode 100644 app/test/suites/meson.build
 create mode 100644 buildtools/get-test-suites.py
 rename app/test/has_hugepage.py => buildtools/has-hugepages.py (100%)

--
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [RFC PATCH 1/5] app/test: add new macros for various test types
  2023-07-21 11:51 [RFC PATCH 0/5] replace build code for unit tests Bruce Richardson
@ 2023-07-21 11:51 ` Bruce Richardson
  2023-07-21 11:51 ` [RFC PATCH 2/5] app/test: tag tests with the test type Bruce Richardson
                   ` (9 subsequent siblings)
  10 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-07-21 11:51 UTC (permalink / raw)
  To: dev; +Cc: Bruce Richardson

Rather than just registering all tests using a single generic macro,
add macros which identify the test as being of a particular type.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 app/test/test.h | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/app/test/test.h b/app/test/test.h
index 85f57efbc6..a91ded76af 100644
--- a/app/test/test.h
+++ b/app/test/test.h
@@ -190,7 +190,7 @@ struct test_command {
 
 void add_test_command(struct test_command *t);
 
-/* Register a test function with its command string */
+/* Register a test function with its command string. Should not be used directly */
 #define REGISTER_TEST_COMMAND(cmd, func) \
 	static struct test_command test_struct_##cmd = { \
 		.command = RTE_STR(cmd), \
@@ -201,4 +201,11 @@ void add_test_command(struct test_command *t);
 		add_test_command(&test_struct_##cmd); \
 	}
 
+/* Register a test function as a particular type.
+ * These can be used to build up test suites automatically
+ */
+#define REGISTER_FAST_TEST(cmd, no_huge, ASan, func)  REGISTER_TEST_COMMAND(cmd, func)
+#define REGISTER_PERF_TEST REGISTER_TEST_COMMAND
+#define REGISTER_DRIVER_TEST REGISTER_TEST_COMMAND
+
 #endif
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [RFC PATCH 2/5] app/test: tag tests with the test type
  2023-07-21 11:51 [RFC PATCH 0/5] replace build code for unit tests Bruce Richardson
  2023-07-21 11:51 ` [RFC PATCH 1/5] app/test: add new macros for various test types Bruce Richardson
@ 2023-07-21 11:51 ` Bruce Richardson
  2023-07-21 11:51 ` [RFC PATCH 3/5] app/test: build using per-file dependency matrix Bruce Richardson
                   ` (8 subsequent siblings)
  10 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-07-21 11:51 UTC (permalink / raw)
  To: dev; +Cc: Bruce Richardson

Rather than having the test types called out in the meson.build file, we
can use macros to identify the test type in the C file itself and then
dynamically build up the tests lists at config time.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
NOTE:
This replacement was automatically done using the list of tests given
in the meson.build file. As can be seen by checking the repo after this,
there were a number of tests not called out in any test suite. The
addition of these tests to suites is a separate issue not covered by
this patchset
---
 app/test/test_acl.c                      |  2 +-
 app/test/test_atomic.c                   |  2 +-
 app/test/test_barrier.c                  |  2 +-
 app/test/test_bitmap.c                   |  2 +-
 app/test/test_bitops.c                   |  2 +-
 app/test/test_bpf.c                      |  4 +--
 app/test/test_byteorder.c                |  2 +-
 app/test/test_cksum.c                    |  2 +-
 app/test/test_cmdline.c                  |  2 +-
 app/test/test_common.c                   |  2 +-
 app/test/test_cpuflags.c                 |  2 +-
 app/test/test_crc.c                      |  2 +-
 app/test/test_cryptodev.c                | 38 ++++++++++++------------
 app/test/test_cryptodev_asym.c           |  2 +-
 app/test/test_cycles.c                   |  2 +-
 app/test/test_debug.c                    |  2 +-
 app/test/test_devargs.c                  |  2 +-
 app/test/test_distributor.c              |  2 +-
 app/test/test_distributor_perf.c         |  2 +-
 app/test/test_dmadev.c                   |  2 +-
 app/test/test_eal_flags.c                | 24 +++++++--------
 app/test/test_eal_fs.c                   |  2 +-
 app/test/test_efd.c                      |  2 +-
 app/test/test_efd_perf.c                 |  2 +-
 app/test/test_errno.c                    |  2 +-
 app/test/test_ethdev_link.c              |  2 +-
 app/test/test_event_ring.c               |  2 +-
 app/test/test_eventdev.c                 |  2 +-
 app/test/test_fbarray.c                  |  2 +-
 app/test/test_fib.c                      |  4 +--
 app/test/test_fib6.c                     |  4 +--
 app/test/test_fib6_perf.c                |  2 +-
 app/test/test_fib_perf.c                 |  2 +-
 app/test/test_func_reentrancy.c          |  2 +-
 app/test/test_hash.c                     |  2 +-
 app/test/test_hash_functions.c           |  2 +-
 app/test/test_hash_multiwriter.c         |  2 +-
 app/test/test_hash_perf.c                |  2 +-
 app/test/test_hash_readwrite.c           |  4 +--
 app/test/test_hash_readwrite_lf_perf.c   |  2 +-
 app/test/test_interrupts.c               |  2 +-
 app/test/test_ipfrag.c                   |  2 +-
 app/test/test_ipsec.c                    |  2 +-
 app/test/test_ipsec_perf.c               |  2 +-
 app/test/test_kni.c                      |  2 +-
 app/test/test_kvargs.c                   |  2 +-
 app/test/test_lcores.c                   |  2 +-
 app/test/test_logs.c                     |  2 +-
 app/test/test_lpm.c                      |  2 +-
 app/test/test_lpm6.c                     |  2 +-
 app/test/test_lpm6_perf.c                |  2 +-
 app/test/test_lpm_perf.c                 |  2 +-
 app/test/test_malloc.c                   |  2 +-
 app/test/test_malloc_perf.c              |  2 +-
 app/test/test_mbuf.c                     |  2 +-
 app/test/test_mcslock.c                  |  2 +-
 app/test/test_member.c                   |  2 +-
 app/test/test_member_perf.c              |  2 +-
 app/test/test_memcpy.c                   |  2 +-
 app/test/test_memcpy_perf.c              |  2 +-
 app/test/test_memory.c                   |  2 +-
 app/test/test_mempool.c                  |  2 +-
 app/test/test_mempool_perf.c             |  2 +-
 app/test/test_memzone.c                  |  2 +-
 app/test/test_meter.c                    |  2 +-
 app/test/test_mp_secondary.c             |  2 +-
 app/test/test_per_lcore.c                |  2 +-
 app/test/test_pflock.c                   |  2 +-
 app/test/test_pie.c                      |  6 ++--
 app/test/test_pmd_perf.c                 |  2 +-
 app/test/test_power.c                    |  2 +-
 app/test/test_power_cpufreq.c            |  2 +-
 app/test/test_power_intel_uncore.c       |  2 +-
 app/test/test_power_kvm_vm.c             |  2 +-
 app/test/test_prefetch.c                 |  2 +-
 app/test/test_rand_perf.c                |  2 +-
 app/test/test_rcu_qsbr.c                 |  2 +-
 app/test/test_rcu_qsbr_perf.c            |  2 +-
 app/test/test_reassembly_perf.c          |  2 +-
 app/test/test_reciprocal_division.c      |  2 +-
 app/test/test_reciprocal_division_perf.c |  2 +-
 app/test/test_red.c                      |  4 +--
 app/test/test_reorder.c                  |  2 +-
 app/test/test_rib.c                      |  4 +--
 app/test/test_rib6.c                     |  4 +--
 app/test/test_ring.c                     |  2 +-
 app/test/test_ring_perf.c                |  2 +-
 app/test/test_rwlock.c                   |  8 ++---
 app/test/test_sched.c                    |  2 +-
 app/test/test_security.c                 |  2 +-
 app/test/test_seqlock.c                  |  2 +-
 app/test/test_service_cores.c            |  4 +--
 app/test/test_spinlock.c                 |  2 +-
 app/test/test_stack.c                    |  4 +--
 app/test/test_stack_perf.c               |  4 +--
 app/test/test_string_fns.c               |  2 +-
 app/test/test_tailq.c                    |  2 +-
 app/test/test_thash.c                    |  2 +-
 app/test/test_thash_perf.c               |  2 +-
 app/test/test_threads.c                  |  2 +-
 app/test/test_ticketlock.c               |  2 +-
 app/test/test_timer.c                    |  2 +-
 app/test/test_timer_perf.c               |  2 +-
 app/test/test_timer_racecond.c           |  2 +-
 app/test/test_trace.c                    |  2 +-
 app/test/test_trace_perf.c               |  2 +-
 app/test/test_version.c                  |  2 +-
 107 files changed, 151 insertions(+), 151 deletions(-)

diff --git a/app/test/test_acl.c b/app/test/test_acl.c
index 623f34682e..bf1466fe11 100644
--- a/app/test/test_acl.c
+++ b/app/test/test_acl.c
@@ -1749,4 +1749,4 @@ test_acl(void)

 #endif /* !RTE_EXEC_ENV_WINDOWS */

-REGISTER_TEST_COMMAND(acl_autotest, test_acl);
+REGISTER_FAST_TEST(acl_autotest, True, True, test_acl);
diff --git a/app/test/test_atomic.c b/app/test/test_atomic.c
index e4b997827e..b52420c10b 100644
--- a/app/test/test_atomic.c
+++ b/app/test/test_atomic.c
@@ -631,4 +631,4 @@ test_atomic(void)

 	return 0;
 }
-REGISTER_TEST_COMMAND(atomic_autotest, test_atomic);
+REGISTER_FAST_TEST(atomic_autotest, False, True, test_atomic);
diff --git a/app/test/test_barrier.c b/app/test/test_barrier.c
index ec69af25bf..925a88b68a 100644
--- a/app/test/test_barrier.c
+++ b/app/test/test_barrier.c
@@ -285,4 +285,4 @@ test_barrier(void)
 	return ret;
 }

-REGISTER_TEST_COMMAND(barrier_autotest, test_barrier);
+REGISTER_PERF_TEST(barrier_autotest, test_barrier);
diff --git a/app/test/test_bitmap.c b/app/test/test_bitmap.c
index e9c61590ae..2014d2682c 100644
--- a/app/test/test_bitmap.c
+++ b/app/test/test_bitmap.c
@@ -269,4 +269,4 @@ test_bitmap(void)
 	return test_bitmap_all_set();
 }

-REGISTER_TEST_COMMAND(bitmap_autotest, test_bitmap);
+REGISTER_FAST_TEST(bitmap_autotest, True, True, test_bitmap);
diff --git a/app/test/test_bitops.c b/app/test/test_bitops.c
index c21426bf2f..dac466602b 100644
--- a/app/test/test_bitops.c
+++ b/app/test/test_bitops.c
@@ -135,4 +135,4 @@ test_bitops(void)
 	return TEST_SUCCESS;
 }

-REGISTER_TEST_COMMAND(bitops_autotest, test_bitops);
+REGISTER_FAST_TEST(bitops_autotest, True, True, test_bitops);
diff --git a/app/test/test_bpf.c b/app/test/test_bpf.c
index f5af5e8a3f..5fef17fbc2 100644
--- a/app/test/test_bpf.c
+++ b/app/test/test_bpf.c
@@ -3262,7 +3262,7 @@ test_bpf(void)

 #endif /* !RTE_LIB_BPF */

-REGISTER_TEST_COMMAND(bpf_autotest, test_bpf);
+REGISTER_FAST_TEST(bpf_autotest, True, True, test_bpf);

 #ifndef RTE_HAS_LIBPCAP

@@ -3473,4 +3473,4 @@ test_bpf_convert(void)

 #endif /* RTE_HAS_LIBPCAP */

-REGISTER_TEST_COMMAND(bpf_convert_autotest, test_bpf_convert);
+REGISTER_FAST_TEST(bpf_convert_autotest, True, True, test_bpf_convert);
diff --git a/app/test/test_byteorder.c b/app/test/test_byteorder.c
index de14ed539e..2189f94744 100644
--- a/app/test/test_byteorder.c
+++ b/app/test/test_byteorder.c
@@ -63,4 +63,4 @@ test_byteorder(void)
 	return 0;
 }

-REGISTER_TEST_COMMAND(byteorder_autotest, test_byteorder);
+REGISTER_FAST_TEST(byteorder_autotest, True, True, test_byteorder);
diff --git a/app/test/test_cksum.c b/app/test/test_cksum.c
index 6c15de9a93..c04fae800e 100644
--- a/app/test/test_cksum.c
+++ b/app/test/test_cksum.c
@@ -267,4 +267,4 @@ test_cksum(void)
 }
 #undef GOTO_FAIL

-REGISTER_TEST_COMMAND(cksum_autotest, test_cksum);
+REGISTER_FAST_TEST(cksum_autotest, True, True, test_cksum);
diff --git a/app/test/test_cmdline.c b/app/test/test_cmdline.c
index 115bee966d..c2a375ac1c 100644
--- a/app/test/test_cmdline.c
+++ b/app/test/test_cmdline.c
@@ -60,4 +60,4 @@ test_cmdline(void)
 	return 0;
 }

-REGISTER_TEST_COMMAND(cmdline_autotest, test_cmdline);
+REGISTER_FAST_TEST(cmdline_autotest, True, True, test_cmdline);
diff --git a/app/test/test_common.c b/app/test/test_common.c
index f89e1eb7ee..c7b41a3f0e 100644
--- a/app/test/test_common.c
+++ b/app/test/test_common.c
@@ -350,4 +350,4 @@ test_common(void)
 	return ret;
 }

-REGISTER_TEST_COMMAND(common_autotest, test_common);
+REGISTER_FAST_TEST(common_autotest, True, True, test_common);
diff --git a/app/test/test_cpuflags.c b/app/test/test_cpuflags.c
index a0e342ae48..c3417dda24 100644
--- a/app/test/test_cpuflags.c
+++ b/app/test/test_cpuflags.c
@@ -334,4 +334,4 @@ test_cpuflags(void)
 	return 0;
 }

-REGISTER_TEST_COMMAND(cpuflags_autotest, test_cpuflags);
+REGISTER_FAST_TEST(cpuflags_autotest, True, True, test_cpuflags);
diff --git a/app/test/test_crc.c b/app/test/test_crc.c
index 5edc8fb13b..b267bdde9f 100644
--- a/app/test/test_crc.c
+++ b/app/test/test_crc.c
@@ -171,4 +171,4 @@ test_crc(void)
 	return 0;
 }

-REGISTER_TEST_COMMAND(crc_autotest, test_crc);
+REGISTER_FAST_TEST(crc_autotest, True, True, test_crc);
diff --git a/app/test/test_cryptodev.c b/app/test/test_cryptodev.c
index fb2af40b99..956268bfcd 100644
--- a/app/test/test_cryptodev.c
+++ b/app/test/test_cryptodev.c
@@ -17535,33 +17535,33 @@ REGISTER_TEST_COMMAND(cryptodev_dpaa2_sec_raw_api_autotest,
 		test_cryptodev_dpaa2_sec_raw_api);
 REGISTER_TEST_COMMAND(cryptodev_dpaa_sec_raw_api_autotest,
 		test_cryptodev_dpaa_sec_raw_api);
-REGISTER_TEST_COMMAND(cryptodev_qat_raw_api_autotest,
+REGISTER_DRIVER_TEST(cryptodev_qat_raw_api_autotest,
 		test_cryptodev_qat_raw_api);
-REGISTER_TEST_COMMAND(cryptodev_qat_autotest, test_cryptodev_qat);
-REGISTER_TEST_COMMAND(cryptodev_aesni_mb_autotest, test_cryptodev_aesni_mb);
-REGISTER_TEST_COMMAND(cryptodev_cpu_aesni_mb_autotest,
+REGISTER_DRIVER_TEST(cryptodev_qat_autotest, test_cryptodev_qat);
+REGISTER_DRIVER_TEST(cryptodev_aesni_mb_autotest, test_cryptodev_aesni_mb);
+REGISTER_DRIVER_TEST(cryptodev_cpu_aesni_mb_autotest,
 	test_cryptodev_cpu_aesni_mb);
-REGISTER_TEST_COMMAND(cryptodev_chacha_poly_mb_autotest,
+REGISTER_DRIVER_TEST(cryptodev_chacha_poly_mb_autotest,
 	test_cryptodev_chacha_poly_mb);
-REGISTER_TEST_COMMAND(cryptodev_openssl_autotest, test_cryptodev_openssl);
-REGISTER_TEST_COMMAND(cryptodev_aesni_gcm_autotest, test_cryptodev_aesni_gcm);
-REGISTER_TEST_COMMAND(cryptodev_cpu_aesni_gcm_autotest,
+REGISTER_DRIVER_TEST(cryptodev_openssl_autotest, test_cryptodev_openssl);
+REGISTER_DRIVER_TEST(cryptodev_aesni_gcm_autotest, test_cryptodev_aesni_gcm);
+REGISTER_DRIVER_TEST(cryptodev_cpu_aesni_gcm_autotest,
 	test_cryptodev_cpu_aesni_gcm);
 REGISTER_TEST_COMMAND(cryptodev_mlx5_autotest, test_cryptodev_mlx5);
-REGISTER_TEST_COMMAND(cryptodev_null_autotest, test_cryptodev_null);
-REGISTER_TEST_COMMAND(cryptodev_sw_snow3g_autotest, test_cryptodev_sw_snow3g);
-REGISTER_TEST_COMMAND(cryptodev_sw_kasumi_autotest, test_cryptodev_sw_kasumi);
-REGISTER_TEST_COMMAND(cryptodev_sw_zuc_autotest, test_cryptodev_sw_zuc);
-REGISTER_TEST_COMMAND(cryptodev_sw_armv8_autotest, test_cryptodev_armv8);
-REGISTER_TEST_COMMAND(cryptodev_sw_mvsam_autotest, test_cryptodev_mrvl);
-REGISTER_TEST_COMMAND(cryptodev_dpaa2_sec_autotest, test_cryptodev_dpaa2_sec);
-REGISTER_TEST_COMMAND(cryptodev_dpaa_sec_autotest, test_cryptodev_dpaa_sec);
+REGISTER_DRIVER_TEST(cryptodev_null_autotest, test_cryptodev_null);
+REGISTER_DRIVER_TEST(cryptodev_sw_snow3g_autotest, test_cryptodev_sw_snow3g);
+REGISTER_DRIVER_TEST(cryptodev_sw_kasumi_autotest, test_cryptodev_sw_kasumi);
+REGISTER_DRIVER_TEST(cryptodev_sw_zuc_autotest, test_cryptodev_sw_zuc);
+REGISTER_DRIVER_TEST(cryptodev_sw_armv8_autotest, test_cryptodev_armv8);
+REGISTER_DRIVER_TEST(cryptodev_sw_mvsam_autotest, test_cryptodev_mrvl);
+REGISTER_DRIVER_TEST(cryptodev_dpaa2_sec_autotest, test_cryptodev_dpaa2_sec);
+REGISTER_DRIVER_TEST(cryptodev_dpaa_sec_autotest, test_cryptodev_dpaa_sec);
 REGISTER_TEST_COMMAND(cryptodev_ccp_autotest, test_cryptodev_ccp);
-REGISTER_TEST_COMMAND(cryptodev_uadk_autotest, test_cryptodev_uadk);
+REGISTER_DRIVER_TEST(cryptodev_uadk_autotest, test_cryptodev_uadk);
 REGISTER_TEST_COMMAND(cryptodev_virtio_autotest, test_cryptodev_virtio);
 REGISTER_TEST_COMMAND(cryptodev_octeontx_autotest, test_cryptodev_octeontx);
 REGISTER_TEST_COMMAND(cryptodev_caam_jr_autotest, test_cryptodev_caam_jr);
 REGISTER_TEST_COMMAND(cryptodev_nitrox_autotest, test_cryptodev_nitrox);
 REGISTER_TEST_COMMAND(cryptodev_bcmfs_autotest, test_cryptodev_bcmfs);
-REGISTER_TEST_COMMAND(cryptodev_cn9k_autotest, test_cryptodev_cn9k);
-REGISTER_TEST_COMMAND(cryptodev_cn10k_autotest, test_cryptodev_cn10k);
+REGISTER_DRIVER_TEST(cryptodev_cn9k_autotest, test_cryptodev_cn9k);
+REGISTER_DRIVER_TEST(cryptodev_cn10k_autotest, test_cryptodev_cn10k);
diff --git a/app/test/test_cryptodev_asym.c b/app/test/test_cryptodev_asym.c
index 0ef2642fdd..3d5a73bf89 100644
--- a/app/test/test_cryptodev_asym.c
+++ b/app/test/test_cryptodev_asym.c
@@ -2828,7 +2828,7 @@ test_cryptodev_cn10k_asym(void)
 REGISTER_TEST_COMMAND(cryptodev_openssl_asym_autotest,
 					  test_cryptodev_openssl_asym);

-REGISTER_TEST_COMMAND(cryptodev_qat_asym_autotest, test_cryptodev_qat_asym);
+REGISTER_DRIVER_TEST(cryptodev_qat_asym_autotest, test_cryptodev_qat_asym);

 REGISTER_TEST_COMMAND(cryptodev_octeontx_asym_autotest,
 					  test_cryptodev_octeontx_asym);
diff --git a/app/test/test_cycles.c b/app/test/test_cycles.c
index 66d11e6db8..08c91cef78 100644
--- a/app/test/test_cycles.c
+++ b/app/test/test_cycles.c
@@ -53,4 +53,4 @@ test_user_delay_us(void)
 	return 0;
 }

-REGISTER_TEST_COMMAND(user_delay_us, test_user_delay_us);
+REGISTER_FAST_TEST(user_delay_us, True, True, test_user_delay_us);
diff --git a/app/test/test_debug.c b/app/test/test_debug.c
index 2704f5b927..acd9542257 100644
--- a/app/test/test_debug.c
+++ b/app/test/test_debug.c
@@ -140,4 +140,4 @@ test_debug(void)

 #endif /* !RTE_EXEC_ENV_WINDOWS */

-REGISTER_TEST_COMMAND(debug_autotest, test_debug);
+REGISTER_FAST_TEST(debug_autotest, True, True, test_debug);
diff --git a/app/test/test_devargs.c b/app/test/test_devargs.c
index 0a4c34a1ad..59ec04957e 100644
--- a/app/test/test_devargs.c
+++ b/app/test/test_devargs.c
@@ -213,4 +213,4 @@ test_devargs(void)
 	return 0;
 }

-REGISTER_TEST_COMMAND(devargs_autotest, test_devargs);
+REGISTER_FAST_TEST(devargs_autotest, True, True, test_devargs);
diff --git a/app/test/test_distributor.c b/app/test/test_distributor.c
index 3efa4af104..ee263a7fee 100644
--- a/app/test/test_distributor.c
+++ b/app/test/test_distributor.c
@@ -952,4 +952,4 @@ test_distributor(void)

 #endif /* !RTE_EXEC_ENV_WINDOWS */

-REGISTER_TEST_COMMAND(distributor_autotest, test_distributor);
+REGISTER_FAST_TEST(distributor_autotest, False, True, test_distributor);
diff --git a/app/test/test_distributor_perf.c b/app/test/test_distributor_perf.c
index ee4321486d..ca868451d7 100644
--- a/app/test/test_distributor_perf.c
+++ b/app/test/test_distributor_perf.c
@@ -277,4 +277,4 @@ test_distributor_perf(void)

 #endif /* !RTE_EXEC_ENV_WINDOWS */

-REGISTER_TEST_COMMAND(distributor_perf_autotest, test_distributor_perf);
+REGISTER_PERF_TEST(distributor_perf_autotest, test_distributor_perf);
diff --git a/app/test/test_dmadev.c b/app/test/test_dmadev.c
index 0736ff2a18..6ef875e545 100644
--- a/app/test/test_dmadev.c
+++ b/app/test/test_dmadev.c
@@ -941,4 +941,4 @@ test_dma(void)
 	return 0;
 }

-REGISTER_TEST_COMMAND(dmadev_autotest, test_dma);
+REGISTER_DRIVER_TEST(dmadev_autotest, test_dma);
diff --git a/app/test/test_eal_flags.c b/app/test/test_eal_flags.c
index d2b91e2075..e016881191 100644
--- a/app/test/test_eal_flags.c
+++ b/app/test/test_eal_flags.c
@@ -1644,15 +1644,15 @@ test_memory_flags(void)

 #endif /* !RTE_EXEC_ENV_WINDOWS */

-REGISTER_TEST_COMMAND(eal_flags_c_opt_autotest, test_missing_c_flag);
-REGISTER_TEST_COMMAND(eal_flags_main_opt_autotest, test_main_lcore_flag);
-REGISTER_TEST_COMMAND(eal_flags_n_opt_autotest, test_invalid_n_flag);
-REGISTER_TEST_COMMAND(eal_flags_hpet_autotest, test_no_hpet_flag);
-REGISTER_TEST_COMMAND(eal_flags_no_huge_autotest, test_no_huge_flag);
-REGISTER_TEST_COMMAND(eal_flags_a_opt_autotest, test_allow_flag);
-REGISTER_TEST_COMMAND(eal_flags_b_opt_autotest, test_invalid_b_flag);
-REGISTER_TEST_COMMAND(eal_flags_vdev_opt_autotest, test_invalid_vdev_flag);
-REGISTER_TEST_COMMAND(eal_flags_r_opt_autotest, test_invalid_r_flag);
-REGISTER_TEST_COMMAND(eal_flags_mem_autotest, test_memory_flags);
-REGISTER_TEST_COMMAND(eal_flags_file_prefix_autotest, test_file_prefix);
-REGISTER_TEST_COMMAND(eal_flags_misc_autotest, test_misc_flags);
+REGISTER_FAST_TEST(eal_flags_c_opt_autotest, False, False, test_missing_c_flag);
+REGISTER_FAST_TEST(eal_flags_main_opt_autotest, False, False, test_main_lcore_flag);
+REGISTER_FAST_TEST(eal_flags_n_opt_autotest, False, False, test_invalid_n_flag);
+REGISTER_FAST_TEST(eal_flags_hpet_autotest, False, False, test_no_hpet_flag);
+REGISTER_FAST_TEST(eal_flags_no_huge_autotest, False, False, test_no_huge_flag);
+REGISTER_FAST_TEST(eal_flags_a_opt_autotest, False, False, test_allow_flag);
+REGISTER_FAST_TEST(eal_flags_b_opt_autotest, False, False, test_invalid_b_flag);
+REGISTER_FAST_TEST(eal_flags_vdev_opt_autotest, False, False, test_invalid_vdev_flag);
+REGISTER_FAST_TEST(eal_flags_r_opt_autotest, False, False, test_invalid_r_flag);
+REGISTER_FAST_TEST(eal_flags_mem_autotest, False, False, test_memory_flags);
+REGISTER_FAST_TEST(eal_flags_file_prefix_autotest, False, False, test_file_prefix);
+REGISTER_FAST_TEST(eal_flags_misc_autotest, False, False, test_misc_flags);
diff --git a/app/test/test_eal_fs.c b/app/test/test_eal_fs.c
index b3686edcb4..a4bdc1236d 100644
--- a/app/test/test_eal_fs.c
+++ b/app/test/test_eal_fs.c
@@ -185,4 +185,4 @@ test_eal_fs(void)

 #endif /* !RTE_EXEC_ENV_WINDOWS */

-REGISTER_TEST_COMMAND(eal_fs_autotest, test_eal_fs);
+REGISTER_FAST_TEST(eal_fs_autotest, True, True, test_eal_fs);
diff --git a/app/test/test_efd.c b/app/test/test_efd.c
index fa29e8f97a..1c0986b9bc 100644
--- a/app/test/test_efd.c
+++ b/app/test/test_efd.c
@@ -473,4 +473,4 @@ test_efd(void)

 #endif /* !RTE_EXEC_ENV_WINDOWS */

-REGISTER_TEST_COMMAND(efd_autotest, test_efd);
+REGISTER_PERF_TEST(efd_autotest, test_efd);
diff --git a/app/test/test_efd_perf.c b/app/test/test_efd_perf.c
index 4d04ed93e3..b212e96767 100644
--- a/app/test/test_efd_perf.c
+++ b/app/test/test_efd_perf.c
@@ -393,4 +393,4 @@ test_efd_perf(void)

 #endif /* !RTE_EXEC_ENV_WINDOWS */

-REGISTER_TEST_COMMAND(efd_perf_autotest, test_efd_perf);
+REGISTER_PERF_TEST(efd_perf_autotest, test_efd_perf);
diff --git a/app/test/test_errno.c b/app/test/test_errno.c
index 0db4fbc8b3..1082299665 100644
--- a/app/test/test_errno.c
+++ b/app/test/test_errno.c
@@ -94,4 +94,4 @@ test_errno(void)
 	return 0;
 }

-REGISTER_TEST_COMMAND(errno_autotest, test_errno);
+REGISTER_FAST_TEST(errno_autotest, True, True, test_errno);
diff --git a/app/test/test_ethdev_link.c b/app/test/test_ethdev_link.c
index ab52385a12..d73e18065d 100644
--- a/app/test/test_ethdev_link.c
+++ b/app/test/test_ethdev_link.c
@@ -167,4 +167,4 @@ test_link_status(void)
 	return unit_test_suite_runner(&link_status_testsuite);
 }

-REGISTER_TEST_COMMAND(ethdev_link_status, test_link_status);
+REGISTER_FAST_TEST(ethdev_link_status, True, True, test_link_status);
diff --git a/app/test/test_event_ring.c b/app/test/test_event_ring.c
index bc4a6e73f5..d5c8499cfd 100644
--- a/app/test/test_event_ring.c
+++ b/app/test/test_event_ring.c
@@ -256,4 +256,4 @@ test_event_ring(void)

 #endif /* !RTE_EXEC_ENV_WINDOWS */

-REGISTER_TEST_COMMAND(event_ring_autotest, test_event_ring);
+REGISTER_FAST_TEST(event_ring_autotest, True, True, test_event_ring);
diff --git a/app/test/test_eventdev.c b/app/test/test_eventdev.c
index 336529038e..e65d893781 100644
--- a/app/test/test_eventdev.c
+++ b/app/test/test_eventdev.c
@@ -1256,7 +1256,7 @@ test_eventdev_selftest_cn10k(void)

 #endif /* !RTE_EXEC_ENV_WINDOWS */

-REGISTER_TEST_COMMAND(eventdev_common_autotest, test_eventdev_common);
+REGISTER_FAST_TEST(eventdev_common_autotest, True, True, test_eventdev_common);

 #ifndef RTE_EXEC_ENV_WINDOWS
 REGISTER_TEST_COMMAND(eventdev_selftest_sw, test_eventdev_selftest_sw);
diff --git a/app/test/test_fbarray.c b/app/test/test_fbarray.c
index a691bf4458..03428a45fe 100644
--- a/app/test/test_fbarray.c
+++ b/app/test/test_fbarray.c
@@ -733,4 +733,4 @@ test_fbarray(void)
 	return unit_test_suite_runner(&fbarray_test_suite);
 }

-REGISTER_TEST_COMMAND(fbarray_autotest, test_fbarray);
+REGISTER_FAST_TEST(fbarray_autotest, True, True, test_fbarray);
diff --git a/app/test/test_fib.c b/app/test/test_fib.c
index eb69d6e2fd..2a8404db61 100644
--- a/app/test/test_fib.c
+++ b/app/test/test_fib.c
@@ -415,5 +415,5 @@ test_slow_fib(void)
 	return unit_test_suite_runner(&fib_slow_tests);
 }

-REGISTER_TEST_COMMAND(fib_autotest, test_fib);
-REGISTER_TEST_COMMAND(fib_slow_autotest, test_slow_fib);
+REGISTER_FAST_TEST(fib_autotest, True, True, test_fib);
+REGISTER_PERF_TEST(fib_slow_autotest, test_slow_fib);
diff --git a/app/test/test_fib6.c b/app/test/test_fib6.c
index 15ad09178a..9b1c0ac8c7 100644
--- a/app/test/test_fib6.c
+++ b/app/test/test_fib6.c
@@ -424,5 +424,5 @@ test_slow_fib6(void)
 	return unit_test_suite_runner(&fib6_slow_tests);
 }

-REGISTER_TEST_COMMAND(fib6_autotest, test_fib6);
-REGISTER_TEST_COMMAND(fib6_slow_autotest, test_slow_fib6);
+REGISTER_FAST_TEST(fib6_autotest, True, True, test_fib6);
+REGISTER_PERF_TEST(fib6_slow_autotest, test_slow_fib6);
diff --git a/app/test/test_fib6_perf.c b/app/test/test_fib6_perf.c
index add20c2331..a7abc46af9 100644
--- a/app/test/test_fib6_perf.c
+++ b/app/test/test_fib6_perf.c
@@ -156,4 +156,4 @@ test_fib6_perf(void)
 	return 0;
 }

-REGISTER_TEST_COMMAND(fib6_perf_autotest, test_fib6_perf);
+REGISTER_PERF_TEST(fib6_perf_autotest, test_fib6_perf);
diff --git a/app/test/test_fib_perf.c b/app/test/test_fib_perf.c
index b56293e64f..a9119c1bb0 100644
--- a/app/test/test_fib_perf.c
+++ b/app/test/test_fib_perf.c
@@ -409,4 +409,4 @@ test_fib_perf(void)
 	return 0;
 }

-REGISTER_TEST_COMMAND(fib_perf_autotest, test_fib_perf);
+REGISTER_PERF_TEST(fib_perf_autotest, test_fib_perf);
diff --git a/app/test/test_func_reentrancy.c b/app/test/test_func_reentrancy.c
index ae9de6f93d..63b00009b5 100644
--- a/app/test/test_func_reentrancy.c
+++ b/app/test/test_func_reentrancy.c
@@ -507,4 +507,4 @@ test_func_reentrancy(void)
 	return 0;
 }

-REGISTER_TEST_COMMAND(func_reentrancy_autotest, test_func_reentrancy);
+REGISTER_FAST_TEST(func_reentrancy_autotest, False, True, test_func_reentrancy);
diff --git a/app/test/test_hash.c b/app/test/test_hash.c
index 3e45afaa67..e160b27b4c 100644
--- a/app/test/test_hash.c
+++ b/app/test/test_hash.c
@@ -2264,4 +2264,4 @@ test_hash(void)
 	return 0;
 }

-REGISTER_TEST_COMMAND(hash_autotest, test_hash);
+REGISTER_FAST_TEST(hash_autotest, True, True, test_hash);
diff --git a/app/test/test_hash_functions.c b/app/test/test_hash_functions.c
index 76d51b6e71..70820d1f19 100644
--- a/app/test/test_hash_functions.c
+++ b/app/test/test_hash_functions.c
@@ -290,4 +290,4 @@ test_hash_functions(void)
 	return 0;
 }

-REGISTER_TEST_COMMAND(hash_functions_autotest, test_hash_functions);
+REGISTER_PERF_TEST(hash_functions_autotest, test_hash_functions);
diff --git a/app/test/test_hash_multiwriter.c b/app/test/test_hash_multiwriter.c
index 0c5a8ca186..dd5ca677b9 100644
--- a/app/test/test_hash_multiwriter.c
+++ b/app/test/test_hash_multiwriter.c
@@ -287,4 +287,4 @@ test_hash_multiwriter_main(void)
 	return 0;
 }

-REGISTER_TEST_COMMAND(hash_multiwriter_autotest, test_hash_multiwriter_main);
+REGISTER_PERF_TEST(hash_multiwriter_autotest, test_hash_multiwriter_main);
diff --git a/app/test/test_hash_perf.c b/app/test/test_hash_perf.c
index 14a1283aba..d66b96e5ce 100644
--- a/app/test/test_hash_perf.c
+++ b/app/test/test_hash_perf.c
@@ -757,4 +757,4 @@ test_hash_perf(void)
 	return 0;
 }

-REGISTER_TEST_COMMAND(hash_perf_autotest, test_hash_perf);
+REGISTER_PERF_TEST(hash_perf_autotest, test_hash_perf);
diff --git a/app/test/test_hash_readwrite.c b/app/test/test_hash_readwrite.c
index 6373e62d33..9c33b49878 100644
--- a/app/test/test_hash_readwrite.c
+++ b/app/test/test_hash_readwrite.c
@@ -760,5 +760,5 @@ test_hash_rw_func_main(void)
 	return 0;
 }

-REGISTER_TEST_COMMAND(hash_readwrite_func_autotest, test_hash_rw_func_main);
-REGISTER_TEST_COMMAND(hash_readwrite_perf_autotest, test_hash_rw_perf_main);
+REGISTER_FAST_TEST(hash_readwrite_func_autotest, False, True, test_hash_rw_func_main);
+REGISTER_PERF_TEST(hash_readwrite_perf_autotest, test_hash_rw_perf_main);
diff --git a/app/test/test_hash_readwrite_lf_perf.c b/app/test/test_hash_readwrite_lf_perf.c
index cf86046a2f..5d18850e19 100644
--- a/app/test/test_hash_readwrite_lf_perf.c
+++ b/app/test/test_hash_readwrite_lf_perf.c
@@ -1579,5 +1579,5 @@ test_hash_readwrite_lf_perf_main(void)
 	return 0;
 }

-REGISTER_TEST_COMMAND(hash_readwrite_lf_perf_autotest,
+REGISTER_PERF_TEST(hash_readwrite_lf_perf_autotest,
 	test_hash_readwrite_lf_perf_main);
diff --git a/app/test/test_interrupts.c b/app/test/test_interrupts.c
index b59ab42699..cf6b5a6a8e 100644
--- a/app/test/test_interrupts.c
+++ b/app/test/test_interrupts.c
@@ -594,4 +594,4 @@ test_interrupt(void)
 	return ret;
 }

-REGISTER_TEST_COMMAND(interrupt_autotest, test_interrupt);
+REGISTER_FAST_TEST(interrupt_autotest, True, True, test_interrupt);
diff --git a/app/test/test_ipfrag.c b/app/test/test_ipfrag.c
index 402ce361c1..264825634d 100644
--- a/app/test/test_ipfrag.c
+++ b/app/test/test_ipfrag.c
@@ -510,4 +510,4 @@ test_ipfrag(void)
 }


-REGISTER_TEST_COMMAND(ipfrag_autotest, test_ipfrag);
+REGISTER_FAST_TEST(ipfrag_autotest, False, True, test_ipfrag);
diff --git a/app/test/test_ipsec.c b/app/test/test_ipsec.c
index c2a52ec305..629580bc21 100644
--- a/app/test/test_ipsec.c
+++ b/app/test/test_ipsec.c
@@ -2532,4 +2532,4 @@ test_ipsec(void)

 #endif /* !RTE_EXEC_ENV_WINDOWS */

-REGISTER_TEST_COMMAND(ipsec_autotest, test_ipsec);
+REGISTER_FAST_TEST(ipsec_autotest, True, True, test_ipsec);
diff --git a/app/test/test_ipsec_perf.c b/app/test/test_ipsec_perf.c
index b221b7fc32..a32a2086e9 100644
--- a/app/test/test_ipsec_perf.c
+++ b/app/test/test_ipsec_perf.c
@@ -631,4 +631,4 @@ test_libipsec_perf(void)

 #endif /* !RTE_EXEC_ENV_WINDOWS */

-REGISTER_TEST_COMMAND(ipsec_perf_autotest, test_libipsec_perf);
+REGISTER_PERF_TEST(ipsec_perf_autotest, test_libipsec_perf);
diff --git a/app/test/test_kni.c b/app/test/test_kni.c
index 4039da0b08..70b1bd6a05 100644
--- a/app/test/test_kni.c
+++ b/app/test/test_kni.c
@@ -737,4 +737,4 @@ test_kni(void)

 #endif

-REGISTER_TEST_COMMAND(kni_autotest, test_kni);
+REGISTER_FAST_TEST(kni_autotest, False, True, test_kni);
diff --git a/app/test/test_kvargs.c b/app/test/test_kvargs.c
index b7b97a0dd9..d0f05a55c7 100644
--- a/app/test/test_kvargs.c
+++ b/app/test/test_kvargs.c
@@ -292,4 +292,4 @@ test_kvargs(void)
 	return 0;
 }

-REGISTER_TEST_COMMAND(kvargs_autotest, test_kvargs);
+REGISTER_FAST_TEST(kvargs_autotest, True, True, test_kvargs);
diff --git a/app/test/test_lcores.c b/app/test/test_lcores.c
index 2c945b0136..b86629c0db 100644
--- a/app/test/test_lcores.c
+++ b/app/test/test_lcores.c
@@ -412,4 +412,4 @@ test_lcores(void)
 	return TEST_SUCCESS;
 }

-REGISTER_TEST_COMMAND(lcores_autotest, test_lcores);
+REGISTER_FAST_TEST(lcores_autotest, True, True, test_lcores);
diff --git a/app/test/test_logs.c b/app/test/test_logs.c
index 8da8824bee..38052910e9 100644
--- a/app/test/test_logs.c
+++ b/app/test/test_logs.c
@@ -158,4 +158,4 @@ test_logs(void)
 	return 0;
 }

-REGISTER_TEST_COMMAND(logs_autotest, test_logs);
+REGISTER_FAST_TEST(logs_autotest, True, True, test_logs);
diff --git a/app/test/test_lpm.c b/app/test/test_lpm.c
index 37b460af3a..85a75ccec0 100644
--- a/app/test/test_lpm.c
+++ b/app/test/test_lpm.c
@@ -1584,4 +1584,4 @@ test_lpm(void)
 	return global_status;
 }

-REGISTER_TEST_COMMAND(lpm_autotest, test_lpm);
+REGISTER_FAST_TEST(lpm_autotest, True, True, test_lpm);
diff --git a/app/test/test_lpm6.c b/app/test/test_lpm6.c
index b6b6f8615e..4a9eddf276 100644
--- a/app/test/test_lpm6.c
+++ b/app/test/test_lpm6.c
@@ -1793,4 +1793,4 @@ test_lpm6(void)
 	return global_status;
 }

-REGISTER_TEST_COMMAND(lpm6_autotest, test_lpm6);
+REGISTER_FAST_TEST(lpm6_autotest, True, True, test_lpm6);
diff --git a/app/test/test_lpm6_perf.c b/app/test/test_lpm6_perf.c
index 5b684686a6..8a49f74c84 100644
--- a/app/test/test_lpm6_perf.c
+++ b/app/test/test_lpm6_perf.c
@@ -161,4 +161,4 @@ test_lpm6_perf(void)
 	return 0;
 }

-REGISTER_TEST_COMMAND(lpm6_perf_autotest, test_lpm6_perf);
+REGISTER_PERF_TEST(lpm6_perf_autotest, test_lpm6_perf);
diff --git a/app/test/test_lpm_perf.c b/app/test/test_lpm_perf.c
index e72437ba38..15ff396dd0 100644
--- a/app/test/test_lpm_perf.c
+++ b/app/test/test_lpm_perf.c
@@ -760,4 +760,4 @@ test_lpm_perf(void)
 	return 0;
 }

-REGISTER_TEST_COMMAND(lpm_perf_autotest, test_lpm_perf);
+REGISTER_PERF_TEST(lpm_perf_autotest, test_lpm_perf);
diff --git a/app/test/test_malloc.c b/app/test/test_malloc.c
index ff081dd931..6057623473 100644
--- a/app/test/test_malloc.c
+++ b/app/test/test_malloc.c
@@ -1091,4 +1091,4 @@ test_malloc(void)
 	return 0;
 }

-REGISTER_TEST_COMMAND(malloc_autotest, test_malloc);
+REGISTER_FAST_TEST(malloc_autotest, False, True, test_malloc);
diff --git a/app/test/test_malloc_perf.c b/app/test/test_malloc_perf.c
index 9bd1662981..a99bfd8531 100644
--- a/app/test/test_malloc_perf.c
+++ b/app/test/test_malloc_perf.c
@@ -171,4 +171,4 @@ test_malloc_perf(void)
 	return 0;
 }

-REGISTER_TEST_COMMAND(malloc_perf_autotest, test_malloc_perf);
+REGISTER_PERF_TEST(malloc_perf_autotest, test_malloc_perf);
diff --git a/app/test/test_mbuf.c b/app/test/test_mbuf.c
index efac01806b..2eeed268af 100644
--- a/app/test/test_mbuf.c
+++ b/app/test/test_mbuf.c
@@ -2955,4 +2955,4 @@ test_mbuf(void)
 }
 #undef GOTO_FAIL

-REGISTER_TEST_COMMAND(mbuf_autotest, test_mbuf);
+REGISTER_FAST_TEST(mbuf_autotest, False, True, test_mbuf);
diff --git a/app/test/test_mcslock.c b/app/test/test_mcslock.c
index 52e45e7e2a..a3753a4397 100644
--- a/app/test/test_mcslock.c
+++ b/app/test/test_mcslock.c
@@ -241,4 +241,4 @@ test_mcslock(void)
 	return ret;
 }

-REGISTER_TEST_COMMAND(mcslock_autotest, test_mcslock);
+REGISTER_FAST_TEST(mcslock_autotest, False, True, test_mcslock);
diff --git a/app/test/test_member.c b/app/test/test_member.c
index 4a93f8bff4..e17a8f3190 100644
--- a/app/test/test_member.c
+++ b/app/test/test_member.c
@@ -996,4 +996,4 @@ test_member(void)

 #endif /* !RTE_EXEC_ENV_WINDOWS */

-REGISTER_TEST_COMMAND(member_autotest, test_member);
+REGISTER_FAST_TEST(member_autotest, True, True, test_member);
diff --git a/app/test/test_member_perf.c b/app/test/test_member_perf.c
index 2f79888fbd..db6b8a18ef 100644
--- a/app/test/test_member_perf.c
+++ b/app/test/test_member_perf.c
@@ -780,4 +780,4 @@ test_member_perf(void)

 #endif /* !RTE_EXEC_ENV_WINDOWS */

-REGISTER_TEST_COMMAND(member_perf_autotest, test_member_perf);
+REGISTER_PERF_TEST(member_perf_autotest, test_member_perf);
diff --git a/app/test/test_memcpy.c b/app/test/test_memcpy.c
index 1ab86f4967..c81efb85c2 100644
--- a/app/test/test_memcpy.c
+++ b/app/test/test_memcpy.c
@@ -129,4 +129,4 @@ test_memcpy(void)
 	return 0;
 }

-REGISTER_TEST_COMMAND(memcpy_autotest, test_memcpy);
+REGISTER_FAST_TEST(memcpy_autotest, True, True, test_memcpy);
diff --git a/app/test/test_memcpy_perf.c b/app/test/test_memcpy_perf.c
index 3727c160e6..5c05a84619 100644
--- a/app/test/test_memcpy_perf.c
+++ b/app/test/test_memcpy_perf.c
@@ -348,4 +348,4 @@ test_memcpy_perf(void)
 	return 0;
 }

-REGISTER_TEST_COMMAND(memcpy_perf_autotest, test_memcpy_perf);
+REGISTER_PERF_TEST(memcpy_perf_autotest, test_memcpy_perf);
diff --git a/app/test/test_memory.c b/app/test/test_memory.c
index 440e5ef838..13ae9569c2 100644
--- a/app/test/test_memory.c
+++ b/app/test/test_memory.c
@@ -110,4 +110,4 @@ test_memory(void)
 	return 0;
 }

-REGISTER_TEST_COMMAND(memory_autotest, test_memory);
+REGISTER_FAST_TEST(memory_autotest, False, True, test_memory);
diff --git a/app/test/test_mempool.c b/app/test/test_mempool.c
index 8e493eda47..03047e5b14 100644
--- a/app/test/test_mempool.c
+++ b/app/test/test_mempool.c
@@ -1043,4 +1043,4 @@ test_mempool(void)
 	return ret;
 }

-REGISTER_TEST_COMMAND(mempool_autotest, test_mempool);
+REGISTER_FAST_TEST(mempool_autotest, False, True, test_mempool);
diff --git a/app/test/test_mempool_perf.c b/app/test/test_mempool_perf.c
index ce7c6241ab..96de347f04 100644
--- a/app/test/test_mempool_perf.c
+++ b/app/test/test_mempool_perf.c
@@ -437,4 +437,4 @@ test_mempool_perf(void)
 	return ret;
 }

-REGISTER_TEST_COMMAND(mempool_perf_autotest, test_mempool_perf);
+REGISTER_PERF_TEST(mempool_perf_autotest, test_mempool_perf);
diff --git a/app/test/test_memzone.c b/app/test/test_memzone.c
index f10f4fd9cd..85b5839034 100644
--- a/app/test/test_memzone.c
+++ b/app/test/test_memzone.c
@@ -1163,4 +1163,4 @@ test_memzone(void)
 	return 0;
 }

-REGISTER_TEST_COMMAND(memzone_autotest, test_memzone);
+REGISTER_FAST_TEST(memzone_autotest, False, True, test_memzone);
diff --git a/app/test/test_meter.c b/app/test/test_meter.c
index 15d5a4839b..3016681bbc 100644
--- a/app/test/test_meter.c
+++ b/app/test/test_meter.c
@@ -713,4 +713,4 @@ test_meter(void)

 }

-REGISTER_TEST_COMMAND(meter_autotest, test_meter);
+REGISTER_FAST_TEST(meter_autotest, True, True, test_meter);
diff --git a/app/test/test_mp_secondary.c b/app/test/test_mp_secondary.c
index ad47d578f2..502b0235a7 100644
--- a/app/test/test_mp_secondary.c
+++ b/app/test/test_mp_secondary.c
@@ -223,4 +223,4 @@ test_mp_secondary(void)

 #endif /* !RTE_EXEC_ENV_WINDOWS */

-REGISTER_TEST_COMMAND(multiprocess_autotest, test_mp_secondary);
+REGISTER_FAST_TEST(multiprocess_autotest, False, False, test_mp_secondary);
diff --git a/app/test/test_per_lcore.c b/app/test/test_per_lcore.c
index 129578d1a3..50011366a7 100644
--- a/app/test/test_per_lcore.c
+++ b/app/test/test_per_lcore.c
@@ -105,4 +105,4 @@ test_per_lcore(void)
 	return 0;
 }

-REGISTER_TEST_COMMAND(per_lcore_autotest, test_per_lcore);
+REGISTER_FAST_TEST(per_lcore_autotest, True, True, test_per_lcore);
diff --git a/app/test/test_pflock.c b/app/test/test_pflock.c
index 38da6bce27..07e4a68c2c 100644
--- a/app/test/test_pflock.c
+++ b/app/test/test_pflock.c
@@ -193,4 +193,4 @@ test_pflock(void)
 	return 0;
 }

-REGISTER_TEST_COMMAND(pflock_autotest, test_pflock);
+REGISTER_FAST_TEST(pflock_autotest, True, True, test_pflock);
diff --git a/app/test/test_pie.c b/app/test/test_pie.c
index a3c0f97c9d..5674602b69 100644
--- a/app/test/test_pie.c
+++ b/app/test/test_pie.c
@@ -1087,6 +1087,6 @@ test_pie_all(void)

 #endif /* !RTE_EXEC_ENV_WINDOWS */

-REGISTER_TEST_COMMAND(pie_autotest, test_pie);
-REGISTER_TEST_COMMAND(pie_perf, test_pie_perf);
-REGISTER_TEST_COMMAND(pie_all, test_pie_all);
+REGISTER_FAST_TEST(pie_autotest, True, True, test_pie);
+REGISTER_PERF_TEST(pie_perf, test_pie_perf);
+REGISTER_PERF_TEST(pie_all, test_pie_all);
diff --git a/app/test/test_pmd_perf.c b/app/test/test_pmd_perf.c
index 3ef590cb51..f6d97f21c9 100644
--- a/app/test/test_pmd_perf.c
+++ b/app/test/test_pmd_perf.c
@@ -899,4 +899,4 @@ test_set_rxtx_sc(cmdline_fixed_string_t type)
 	return -1;
 }

-REGISTER_TEST_COMMAND(pmd_perf_autotest, test_pmd_perf);
+REGISTER_PERF_TEST(pmd_perf_autotest, test_pmd_perf);
diff --git a/app/test/test_power.c b/app/test/test_power.c
index b7b5561348..d4f622ff2f 100644
--- a/app/test/test_power.c
+++ b/app/test/test_power.c
@@ -172,4 +172,4 @@ test_power(void)
 }
 #endif

-REGISTER_TEST_COMMAND(power_autotest, test_power);
+REGISTER_FAST_TEST(power_autotest, True, True, test_power);
diff --git a/app/test/test_power_cpufreq.c b/app/test/test_power_cpufreq.c
index 4d013cd7bb..f56053ac10 100644
--- a/app/test/test_power_cpufreq.c
+++ b/app/test/test_power_cpufreq.c
@@ -703,5 +703,5 @@ test_power_caps(void)

 #endif

-REGISTER_TEST_COMMAND(power_cpufreq_autotest, test_power_cpufreq);
+REGISTER_FAST_TEST(power_cpufreq_autotest, False, True, test_power_cpufreq);
 REGISTER_TEST_COMMAND(power_caps_autotest, test_power_caps);
diff --git a/app/test/test_power_intel_uncore.c b/app/test/test_power_intel_uncore.c
index 31163af84e..c8a8621e44 100644
--- a/app/test/test_power_intel_uncore.c
+++ b/app/test/test_power_intel_uncore.c
@@ -298,4 +298,4 @@ test_power_intel_uncore(void)
 }
 #endif

-REGISTER_TEST_COMMAND(power_intel_uncore_autotest, test_power_intel_uncore);
+REGISTER_FAST_TEST(power_intel_uncore_autotest, True, True, test_power_intel_uncore);
diff --git a/app/test/test_power_kvm_vm.c b/app/test/test_power_kvm_vm.c
index cc66b7a8a0..4919df8b62 100644
--- a/app/test/test_power_kvm_vm.c
+++ b/app/test/test_power_kvm_vm.c
@@ -299,4 +299,4 @@ test_power_kvm_vm(void)
 }
 #endif

-REGISTER_TEST_COMMAND(power_kvm_vm_autotest, test_power_kvm_vm);
+REGISTER_FAST_TEST(power_kvm_vm_autotest, False, True, test_power_kvm_vm);
diff --git a/app/test/test_prefetch.c b/app/test/test_prefetch.c
index 7b4a8e4144..a72ae80557 100644
--- a/app/test/test_prefetch.c
+++ b/app/test/test_prefetch.c
@@ -35,4 +35,4 @@ test_prefetch(void)
 	return 0;
 }

-REGISTER_TEST_COMMAND(prefetch_autotest, test_prefetch);
+REGISTER_FAST_TEST(prefetch_autotest, True, True, test_prefetch);
diff --git a/app/test/test_rand_perf.c b/app/test/test_rand_perf.c
index 26fb1d9a58..30204e12c0 100644
--- a/app/test/test_rand_perf.c
+++ b/app/test/test_rand_perf.c
@@ -96,4 +96,4 @@ test_rand_perf(void)
 	return 0;
 }

-REGISTER_TEST_COMMAND(rand_perf_autotest, test_rand_perf);
+REGISTER_PERF_TEST(rand_perf_autotest, test_rand_perf);
diff --git a/app/test/test_rcu_qsbr.c b/app/test/test_rcu_qsbr.c
index 70404e89e6..16c24f58e7 100644
--- a/app/test/test_rcu_qsbr.c
+++ b/app/test/test_rcu_qsbr.c
@@ -1418,4 +1418,4 @@ test_rcu_qsbr_main(void)
 	return -1;
 }

-REGISTER_TEST_COMMAND(rcu_qsbr_autotest, test_rcu_qsbr_main);
+REGISTER_FAST_TEST(rcu_qsbr_autotest, True, True, test_rcu_qsbr_main);
diff --git a/app/test/test_rcu_qsbr_perf.c b/app/test/test_rcu_qsbr_perf.c
index b15e5cef88..ce88a7333c 100644
--- a/app/test/test_rcu_qsbr_perf.c
+++ b/app/test/test_rcu_qsbr_perf.c
@@ -690,4 +690,4 @@ test_rcu_qsbr_main(void)
 	return -1;
 }

-REGISTER_TEST_COMMAND(rcu_qsbr_perf_autotest, test_rcu_qsbr_main);
+REGISTER_PERF_TEST(rcu_qsbr_perf_autotest, test_rcu_qsbr_main);
diff --git a/app/test/test_reassembly_perf.c b/app/test/test_reassembly_perf.c
index c11b65291f..4b4929d777 100644
--- a/app/test/test_reassembly_perf.c
+++ b/app/test/test_reassembly_perf.c
@@ -1000,4 +1000,4 @@ test_reassembly_perf(void)
 	return TEST_SUCCESS;
 }

-REGISTER_TEST_COMMAND(reassembly_perf_autotest, test_reassembly_perf);
+REGISTER_PERF_TEST(reassembly_perf_autotest, test_reassembly_perf);
diff --git a/app/test/test_reciprocal_division.c b/app/test/test_reciprocal_division.c
index 8ea9b1d24d..fb52b2d5a1 100644
--- a/app/test/test_reciprocal_division.c
+++ b/app/test/test_reciprocal_division.c
@@ -164,4 +164,4 @@ test_reciprocal(void)
 	return result;
 }

-REGISTER_TEST_COMMAND(reciprocal_division, test_reciprocal);
+REGISTER_PERF_TEST(reciprocal_division, test_reciprocal);
diff --git a/app/test/test_reciprocal_division_perf.c b/app/test/test_reciprocal_division_perf.c
index 4f625873e5..cf96d46a22 100644
--- a/app/test/test_reciprocal_division_perf.c
+++ b/app/test/test_reciprocal_division_perf.c
@@ -205,4 +205,4 @@ test_reciprocal_division_perf(void)
 	return result;
 }

-REGISTER_TEST_COMMAND(reciprocal_division_perf, test_reciprocal_division_perf);
+REGISTER_PERF_TEST(reciprocal_division_perf, test_reciprocal_division_perf);
diff --git a/app/test/test_red.c b/app/test/test_red.c
index 84c292f8d8..aa7538d51a 100644
--- a/app/test/test_red.c
+++ b/app/test/test_red.c
@@ -1878,5 +1878,5 @@ test_red_all(void)
 #endif /* !RTE_EXEC_ENV_WINDOWS */

 REGISTER_TEST_COMMAND(red_autotest, test_red);
-REGISTER_TEST_COMMAND(red_perf, test_red_perf);
-REGISTER_TEST_COMMAND(red_all, test_red_all);
+REGISTER_PERF_TEST(red_perf, test_red_perf);
+REGISTER_PERF_TEST(red_all, test_red_all);
diff --git a/app/test/test_reorder.c b/app/test/test_reorder.c
index c188f6ce67..e66b4b05d4 100644
--- a/app/test/test_reorder.c
+++ b/app/test/test_reorder.c
@@ -548,4 +548,4 @@ test_reorder(void)
 }


-REGISTER_TEST_COMMAND(reorder_autotest, test_reorder);
+REGISTER_FAST_TEST(reorder_autotest, True, True, test_reorder);
diff --git a/app/test/test_rib.c b/app/test/test_rib.c
index 65b6856410..a8d60acc3a 100644
--- a/app/test/test_rib.c
+++ b/app/test/test_rib.c
@@ -363,5 +363,5 @@ test_slow_rib(void)
 	return unit_test_suite_runner(&rib_slow_tests);
 }

-REGISTER_TEST_COMMAND(rib_autotest, test_rib);
-REGISTER_TEST_COMMAND(rib_slow_autotest, test_slow_rib);
+REGISTER_FAST_TEST(rib_autotest, True, True, test_rib);
+REGISTER_PERF_TEST(rib_slow_autotest, test_slow_rib);
diff --git a/app/test/test_rib6.c b/app/test/test_rib6.c
index 336b779d2e..9bbd0f406d 100644
--- a/app/test/test_rib6.c
+++ b/app/test/test_rib6.c
@@ -367,5 +367,5 @@ test_slow_rib6(void)
 	return unit_test_suite_runner(&rib6_slow_tests);
 }

-REGISTER_TEST_COMMAND(rib6_autotest, test_rib6);
-REGISTER_TEST_COMMAND(rib6_slow_autotest, test_slow_rib6);
+REGISTER_FAST_TEST(rib6_autotest, True, True, test_rib6);
+REGISTER_PERF_TEST(rib6_slow_autotest, test_slow_rib6);
diff --git a/app/test/test_ring.c b/app/test/test_ring.c
index bde33ab4a1..d2accbabfa 100644
--- a/app/test/test_ring.c
+++ b/app/test/test_ring.c
@@ -1241,4 +1241,4 @@ test_ring(void)
 	return -1;
 }

-REGISTER_TEST_COMMAND(ring_autotest, test_ring);
+REGISTER_FAST_TEST(ring_autotest, True, True, test_ring);
diff --git a/app/test/test_ring_perf.c b/app/test/test_ring_perf.c
index 3972fd9db3..d7c5a4c30b 100644
--- a/app/test/test_ring_perf.c
+++ b/app/test/test_ring_perf.c
@@ -579,4 +579,4 @@ test_ring_perf(void)
 	return 0;
 }

-REGISTER_TEST_COMMAND(ring_perf_autotest, test_ring_perf);
+REGISTER_PERF_TEST(ring_perf_autotest, test_ring_perf);
diff --git a/app/test/test_rwlock.c b/app/test/test_rwlock.c
index 4ae0bf8deb..e8086888fe 100644
--- a/app/test/test_rwlock.c
+++ b/app/test/test_rwlock.c
@@ -506,7 +506,7 @@ try_rwlock_test_rde_wro(void)
 	return process_try_lcore_stats();
 }

-REGISTER_TEST_COMMAND(rwlock_test1_autotest, rwlock_test1);
-REGISTER_TEST_COMMAND(rwlock_rda_autotest, try_rwlock_test_rda);
-REGISTER_TEST_COMMAND(rwlock_rds_wrm_autotest, try_rwlock_test_rds_wrm);
-REGISTER_TEST_COMMAND(rwlock_rde_wro_autotest, try_rwlock_test_rde_wro);
+REGISTER_FAST_TEST(rwlock_test1_autotest, True, True, rwlock_test1);
+REGISTER_FAST_TEST(rwlock_rda_autotest, True, True, try_rwlock_test_rda);
+REGISTER_FAST_TEST(rwlock_rds_wrm_autotest, True, True, try_rwlock_test_rds_wrm);
+REGISTER_FAST_TEST(rwlock_rde_wro_autotest, True, True, try_rwlock_test_rde_wro);
diff --git a/app/test/test_sched.c b/app/test/test_sched.c
index ddec572447..385ca45136 100644
--- a/app/test/test_sched.c
+++ b/app/test/test_sched.c
@@ -215,4 +215,4 @@ test_sched(void)

 #endif /* !RTE_EXEC_ENV_WINDOWS */

-REGISTER_TEST_COMMAND(sched_autotest, test_sched);
+REGISTER_FAST_TEST(sched_autotest, True, True, test_sched);
diff --git a/app/test/test_security.c b/app/test/test_security.c
index 4783cd0663..a3b962750a 100644
--- a/app/test/test_security.c
+++ b/app/test/test_security.c
@@ -2314,4 +2314,4 @@ test_security(void)
 	return unit_test_suite_runner(&security_testsuite);
 }

-REGISTER_TEST_COMMAND(security_autotest, test_security);
+REGISTER_FAST_TEST(security_autotest, False, True, test_security);
diff --git a/app/test/test_seqlock.c b/app/test/test_seqlock.c
index d26d2c010e..325af38070 100644
--- a/app/test/test_seqlock.c
+++ b/app/test/test_seqlock.c
@@ -187,4 +187,4 @@ test_seqlock(void)
 	return rc;
 }

-REGISTER_TEST_COMMAND(seqlock_autotest, test_seqlock);
+REGISTER_FAST_TEST(seqlock_autotest, True, True, test_seqlock);
diff --git a/app/test/test_service_cores.c b/app/test/test_service_cores.c
index 422d2a83e8..6c000417d2 100644
--- a/app/test/test_service_cores.c
+++ b/app/test/test_service_cores.c
@@ -1040,7 +1040,7 @@ test_service_common(void)
 	return unit_test_suite_runner(&service_tests);
 }

-REGISTER_TEST_COMMAND(service_autotest, test_service_common);
+REGISTER_FAST_TEST(service_autotest, True, True, test_service_common);

 static struct unit_test_suite service_perf_tests  = {
 	.suite_name = "service core performance test suite",
@@ -1062,4 +1062,4 @@ test_service_perf(void)
 	return unit_test_suite_runner(&service_perf_tests);
 }

-REGISTER_TEST_COMMAND(service_perf_autotest, test_service_perf);
+REGISTER_PERF_TEST(service_perf_autotest, test_service_perf);
diff --git a/app/test/test_spinlock.c b/app/test/test_spinlock.c
index 3f59372300..d655b606f7 100644
--- a/app/test/test_spinlock.c
+++ b/app/test/test_spinlock.c
@@ -302,4 +302,4 @@ test_spinlock(void)
 	return ret;
 }

-REGISTER_TEST_COMMAND(spinlock_autotest, test_spinlock);
+REGISTER_FAST_TEST(spinlock_autotest, True, True, test_spinlock);
diff --git a/app/test/test_stack.c b/app/test/test_stack.c
index bc38961433..93bba18469 100644
--- a/app/test/test_stack.c
+++ b/app/test/test_stack.c
@@ -379,5 +379,5 @@ test_lf_stack(void)
 #endif
 }

-REGISTER_TEST_COMMAND(stack_autotest, test_stack);
-REGISTER_TEST_COMMAND(stack_lf_autotest, test_lf_stack);
+REGISTER_FAST_TEST(stack_autotest, False, True, test_stack);
+REGISTER_FAST_TEST(stack_lf_autotest, False, True, test_lf_stack);
diff --git a/app/test/test_stack_perf.c b/app/test/test_stack_perf.c
index 1eae00a334..c5e1caa036 100644
--- a/app/test/test_stack_perf.c
+++ b/app/test/test_stack_perf.c
@@ -354,5 +354,5 @@ test_lf_stack_perf(void)
 #endif
 }

-REGISTER_TEST_COMMAND(stack_perf_autotest, test_stack_perf);
-REGISTER_TEST_COMMAND(stack_lf_perf_autotest, test_lf_stack_perf);
+REGISTER_PERF_TEST(stack_perf_autotest, test_stack_perf);
+REGISTER_PERF_TEST(stack_lf_perf_autotest, test_lf_stack_perf);
diff --git a/app/test/test_string_fns.c b/app/test/test_string_fns.c
index 5e105d2bb9..07e72bf680 100644
--- a/app/test/test_string_fns.c
+++ b/app/test/test_string_fns.c
@@ -182,4 +182,4 @@ test_string_fns(void)
 	return 0;
 }

-REGISTER_TEST_COMMAND(string_autotest, test_string_fns);
+REGISTER_FAST_TEST(string_autotest, True, True, test_string_fns);
diff --git a/app/test/test_tailq.c b/app/test/test_tailq.c
index 9520219b0a..5598bcd12d 100644
--- a/app/test/test_tailq.c
+++ b/app/test/test_tailq.c
@@ -125,4 +125,4 @@ test_tailq(void)
 	return ret;
 }

-REGISTER_TEST_COMMAND(tailq_autotest, test_tailq);
+REGISTER_FAST_TEST(tailq_autotest, True, True, test_tailq);
diff --git a/app/test/test_thash.c b/app/test/test_thash.c
index 53d9611e18..dd6468eb56 100644
--- a/app/test/test_thash.c
+++ b/app/test/test_thash.c
@@ -966,4 +966,4 @@ test_thash(void)
 	return unit_test_suite_runner(&thash_tests);
 }

-REGISTER_TEST_COMMAND(thash_autotest, test_thash);
+REGISTER_FAST_TEST(thash_autotest, True, True, test_thash);
diff --git a/app/test/test_thash_perf.c b/app/test/test_thash_perf.c
index 687582aa32..9dfd5d3c21 100644
--- a/app/test/test_thash_perf.c
+++ b/app/test/test_thash_perf.c
@@ -135,4 +135,4 @@ test_thash_perf(void)
 	return 0;
 }

-REGISTER_TEST_COMMAND(thash_perf_autotest, test_thash_perf);
+REGISTER_PERF_TEST(thash_perf_autotest, test_thash_perf);
diff --git a/app/test/test_threads.c b/app/test/test_threads.c
index a4c4f651a4..6ed6b168d4 100644
--- a/app/test/test_threads.c
+++ b/app/test/test_threads.c
@@ -279,4 +279,4 @@ test_threads(void)
 	return unit_test_suite_runner(&threads_test_suite);
 }

-REGISTER_TEST_COMMAND(threads_autotest, test_threads);
+REGISTER_FAST_TEST(threads_autotest, True, True, test_threads);
diff --git a/app/test/test_ticketlock.c b/app/test/test_ticketlock.c
index 242c136478..25af697c0a 100644
--- a/app/test/test_ticketlock.c
+++ b/app/test/test_ticketlock.c
@@ -314,4 +314,4 @@ test_ticketlock(void)
 	return ret;
 }

-REGISTER_TEST_COMMAND(ticketlock_autotest, test_ticketlock);
+REGISTER_FAST_TEST(ticketlock_autotest, True, True, test_ticketlock);
diff --git a/app/test/test_timer.c b/app/test/test_timer.c
index 0c36dc9010..09608a1845 100644
--- a/app/test/test_timer.c
+++ b/app/test/test_timer.c
@@ -594,4 +594,4 @@ test_timer(void)
 	return TEST_SUCCESS;
 }

-REGISTER_TEST_COMMAND(timer_autotest, test_timer);
+REGISTER_FAST_TEST(timer_autotest, False, True, test_timer);
diff --git a/app/test/test_timer_perf.c b/app/test/test_timer_perf.c
index 0ede4b3e40..d2d74ebbc6 100644
--- a/app/test/test_timer_perf.c
+++ b/app/test/test_timer_perf.c
@@ -131,4 +131,4 @@ test_timer_perf(void)
 	return 0;
 }

-REGISTER_TEST_COMMAND(timer_perf_autotest, test_timer_perf);
+REGISTER_PERF_TEST(timer_perf_autotest, test_timer_perf);
diff --git a/app/test/test_timer_racecond.c b/app/test/test_timer_racecond.c
index bb56ae8324..6f8b448ff8 100644
--- a/app/test/test_timer_racecond.c
+++ b/app/test/test_timer_racecond.c
@@ -172,4 +172,4 @@ test_timer_racecond(void)
 	return TEST_SUCCESS;
 }

-REGISTER_TEST_COMMAND(timer_racecond_autotest, test_timer_racecond);
+REGISTER_PERF_TEST(timer_racecond_autotest, test_timer_racecond);
diff --git a/app/test/test_trace.c b/app/test/test_trace.c
index ad4a394a29..96829caed5 100644
--- a/app/test/test_trace.c
+++ b/app/test/test_trace.c
@@ -250,4 +250,4 @@ test_trace(void)

 #endif /* !RTE_EXEC_ENV_WINDOWS */

-REGISTER_TEST_COMMAND(trace_autotest, test_trace);
+REGISTER_FAST_TEST(trace_autotest, True, True, test_trace);
diff --git a/app/test/test_trace_perf.c b/app/test/test_trace_perf.c
index 46ae7d8074..a6dd075722 100644
--- a/app/test/test_trace_perf.c
+++ b/app/test/test_trace_perf.c
@@ -179,4 +179,4 @@ test_trace_perf(void)
 	return TEST_SUCCESS;
 }

-REGISTER_TEST_COMMAND(trace_perf_autotest, test_trace_perf);
+REGISTER_PERF_TEST(trace_perf_autotest, test_trace_perf);
diff --git a/app/test/test_version.c b/app/test/test_version.c
index 1e1ff18656..21016237bc 100644
--- a/app/test/test_version.c
+++ b/app/test/test_version.c
@@ -25,4 +25,4 @@ test_version(void)
 	return 0;
 }

-REGISTER_TEST_COMMAND(version_autotest, test_version);
+REGISTER_FAST_TEST(version_autotest, True, True, test_version);
--
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [RFC PATCH 3/5] app/test: build using per-file dependency matrix
  2023-07-21 11:51 [RFC PATCH 0/5] replace build code for unit tests Bruce Richardson
  2023-07-21 11:51 ` [RFC PATCH 1/5] app/test: add new macros for various test types Bruce Richardson
  2023-07-21 11:51 ` [RFC PATCH 2/5] app/test: tag tests with the test type Bruce Richardson
@ 2023-07-21 11:51 ` Bruce Richardson
  2023-07-21 11:51 ` [RFC PATCH 4/5] app/test: define unit tests suites based on test macros Bruce Richardson
                   ` (7 subsequent siblings)
  10 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-07-21 11:51 UTC (permalink / raw)
  To: dev; +Cc: Bruce Richardson

Rather than using if-else constructs to selectively add or remove files
from the UT build, switch to a table-based approach where each file
lists out what libs or drivers it depends upon.

Initial version of this table was generated via analysis of the header
files included in each C file.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 app/meson.build      |   8 +-
 app/test/meson.build | 796 +++++++++++--------------------------------
 2 files changed, 203 insertions(+), 601 deletions(-)

diff --git a/app/meson.build b/app/meson.build
index 4fc1a83eba..0d8b618e7f 100644
--- a/app/meson.build
+++ b/app/meson.build
@@ -32,6 +32,11 @@ apps = [
         'test-security-perf',
 ]
 
+if get_option('tests')
+# build the auto test app if enabled.
+    apps += 'test'
+endif
+
 default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API']
 default_ldflags = []
 if get_option('default_library') == 'static' and not is_windows
@@ -106,6 +111,3 @@ foreach app:apps
             install_rpath: join_paths(get_option('prefix'), driver_install_path),
             install: true)
 endforeach
-
-# special case the autotests
-subdir('test')
diff --git a/app/test/meson.build b/app/test/meson.build
index b89cf0368f..475d87860e 100644
--- a/app/test/meson.build
+++ b/app/test/meson.build
@@ -1,452 +1,202 @@
 # SPDX-License-Identifier: BSD-3-Clause
-# Copyright(c) 2017 Intel Corporation
-
-if not get_option('tests')
-    subdir_done()
-endif
-
-test_sources = files(
-        'commands.c',
-        'packet_burst_generator.c',
-        'test.c',
-        'test_acl.c',
-        'test_alarm.c',
-        'test_atomic.c',
-        'test_barrier.c',
-        'test_bitops.c',
-        'test_bitmap.c',
-        'test_bpf.c',
-        'test_byteorder.c',
-        'test_cksum.c',
-        'test_cksum_perf.c',
-        'test_cmdline.c',
-        'test_cmdline_cirbuf.c',
-        'test_cmdline_etheraddr.c',
-        'test_cmdline_ipaddr.c',
-        'test_cmdline_lib.c',
-        'test_cmdline_num.c',
-        'test_cmdline_portlist.c',
-        'test_cmdline_string.c',
-        'test_common.c',
-        'test_cpuflags.c',
-        'test_crc.c',
-        'test_cryptodev.c',
-        'test_cryptodev_asym.c',
-        'test_cryptodev_blockcipher.c',
-        'test_cryptodev_crosscheck.c',
-        'test_cryptodev_security_ipsec.c',
-        'test_cryptodev_security_pdcp.c',
-        'test_cycles.c',
-        'test_debug.c',
-        'test_devargs.c',
-        'test_distributor.c',
-        'test_distributor_perf.c',
-        'test_dmadev.c',
-        'test_dmadev_api.c',
-        'test_eal_flags.c',
-        'test_eal_fs.c',
-        'test_efd.c',
-        'test_efd_perf.c',
-        'test_errno.c',
-        'test_ethdev_link.c',
-        'test_event_crypto_adapter.c',
-        'test_event_eth_rx_adapter.c',
-        'test_event_ring.c',
-        'test_event_timer_adapter.c',
-        'test_eventdev.c',
-        'test_external_mem.c',
-        'test_fbarray.c',
-        'test_fib.c',
-        'test_fib_perf.c',
-        'test_fib6.c',
-        'test_fib6_perf.c',
-        'test_func_reentrancy.c',
-        'test_hash.c',
-        'test_hash_functions.c',
-        'test_hash_multiwriter.c',
-        'test_hash_readwrite.c',
-        'test_hash_perf.c',
-        'test_hash_readwrite_lf_perf.c',
-        'test_interrupts.c',
-        'test_ipfrag.c',
-        'test_ipsec.c',
-        'test_ipsec_sad.c',
-        'test_ipsec_perf.c',
-        'test_kni.c',
-        'test_kvargs.c',
-        'test_lcores.c',
-        'test_logs.c',
-        'test_lpm.c',
-        'test_lpm6.c',
-        'test_lpm6_perf.c',
-        'test_lpm_perf.c',
-        'test_malloc.c',
-        'test_malloc_perf.c',
-        'test_mbuf.c',
-        'test_member.c',
-        'test_member_perf.c',
-        'test_memcpy.c',
-        'test_memcpy_perf.c',
-        'test_memory.c',
-        'test_mempool.c',
-        'test_mempool_perf.c',
-        'test_memzone.c',
-        'test_meter.c',
-        'test_mcslock.c',
-        'test_mp_secondary.c',
-        'test_per_lcore.c',
-        'test_pflock.c',
-        'test_pmd_perf.c',
-        'test_power.c',
-        'test_power_cpufreq.c',
-        'test_power_kvm_vm.c',
-        'test_power_intel_uncore.c',
-        'test_prefetch.c',
-        'test_rand_perf.c',
-        'test_rawdev.c',
-        'test_rcu_qsbr.c',
-        'test_rcu_qsbr_perf.c',
-        'test_reassembly_perf.c',
-        'test_reciprocal_division.c',
-        'test_reciprocal_division_perf.c',
-        'test_red.c',
-        'test_pie.c',
-        'test_reorder.c',
-        'test_rib.c',
-        'test_rib6.c',
-        'test_ring.c',
-        'test_ring_mpmc_stress.c',
-        'test_ring_hts_stress.c',
-        'test_ring_mt_peek_stress.c',
-        'test_ring_mt_peek_stress_zc.c',
-        'test_ring_perf.c',
-        'test_ring_rts_stress.c',
-        'test_ring_st_peek_stress.c',
-        'test_ring_st_peek_stress_zc.c',
-        'test_ring_stress.c',
-        'test_rwlock.c',
-        'test_sched.c',
-        'test_security.c',
-        'test_security_inline_macsec.c',
-        'test_security_inline_proto.c',
-        'test_seqlock.c',
-        'test_service_cores.c',
-        'test_spinlock.c',
-        'test_stack.c',
-        'test_stack_perf.c',
-        'test_string_fns.c',
-        'test_tailq.c',
-        'test_thash.c',
-        'test_thash_perf.c',
-        'test_threads.c',
-        'test_timer.c',
-        'test_timer_perf.c',
-        'test_timer_racecond.c',
-        'test_timer_secondary.c',
-        'test_ticketlock.c',
-        'test_trace.c',
-        'test_trace_register.c',
-        'test_trace_perf.c',
-        'test_version.c',
-        'virtual_pmd.c',
-)
-
-test_deps = dpdk_libs_enabled
-# as well as libs, the pci and vdev bus drivers are needed for a lot of tests
-test_deps += ['bus_pci', 'bus_vdev']
-
-# Each test is marked with flags:
-# - the first flag indicates whether the test can run in no-huge mode,
-# - the second flag indicates whether the test can run with ASan enabled,
-fast_tests = [
-        ['acl_autotest', true, true],
-        ['atomic_autotest', false, true],
-        ['bitmap_autotest', true, true],
-        ['bpf_autotest', true, true],
-        ['bpf_convert_autotest', true, true],
-        ['bitops_autotest', true, true],
-        ['byteorder_autotest', true, true],
-        ['cksum_autotest', true, true],
-        ['cmdline_autotest', true, true],
-        ['common_autotest', true, true],
-        ['cpuflags_autotest', true, true],
-        ['debug_autotest', true, true],
-        ['devargs_autotest', true, true],
-        ['eal_flags_c_opt_autotest', false, false],
-        ['eal_flags_main_opt_autotest', false, false],
-        ['eal_flags_n_opt_autotest', false, false],
-        ['eal_flags_hpet_autotest', false, false],
-        ['eal_flags_no_huge_autotest', false, false],
-        ['eal_flags_a_opt_autotest', false, false],
-        ['eal_flags_b_opt_autotest', false, false],
-        ['eal_flags_vdev_opt_autotest', false, false],
-        ['eal_flags_r_opt_autotest', false, false],
-        ['eal_flags_mem_autotest', false, false],
-        ['eal_flags_file_prefix_autotest', false, false],
-        ['eal_flags_misc_autotest', false, false],
-        ['eal_fs_autotest', true, true],
-        ['errno_autotest', true, true],
-        ['ethdev_link_status', true, true],
-        ['event_ring_autotest', true, true],
-        ['fib_autotest', true, true],
-        ['fib6_autotest', true, true],
-        ['func_reentrancy_autotest', false, true],
-        ['hash_autotest', true, true],
-        ['interrupt_autotest', true, true],
-        ['ipfrag_autotest', false, true],
-        ['lcores_autotest', true, true],
-        ['logs_autotest', true, true],
-        ['lpm_autotest', true, true],
-        ['lpm6_autotest', true, true],
-        ['malloc_autotest', false, true],
-        ['mbuf_autotest', false, true],
-        ['mcslock_autotest', false, true],
-        ['memcpy_autotest', true, true],
-        ['memory_autotest', false, true],
-        ['mempool_autotest', false, true],
-        ['memzone_autotest', false, true],
-        ['meter_autotest', true, true],
-        ['multiprocess_autotest', false, false],
-        ['per_lcore_autotest', true, true],
-        ['pflock_autotest', true, true],
-        ['prefetch_autotest', true, true],
-        ['rcu_qsbr_autotest', true, true],
-        ['pie_autotest', true, true],
-        ['rib_autotest', true, true],
-        ['rib6_autotest', true, true],
-        ['ring_autotest', true, true],
-        ['rwlock_test1_autotest', true, true],
-        ['rwlock_rda_autotest', true, true],
-        ['rwlock_rds_wrm_autotest', true, true],
-        ['rwlock_rde_wro_autotest', true, true],
-        ['sched_autotest', true, true],
-        ['security_autotest', false, true],
-        ['seqlock_autotest', true, true],
-        ['spinlock_autotest', true, true],
-        ['stack_autotest', false, true],
-        ['stack_lf_autotest', false, true],
-        ['string_autotest', true, true],
-        ['tailq_autotest', true, true],
-        ['ticketlock_autotest', true, true],
-        ['timer_autotest', false, true],
-        ['user_delay_us', true, true],
-        ['version_autotest', true, true],
-        ['crc_autotest', true, true],
-        ['distributor_autotest', false, true],
-        ['eventdev_common_autotest', true, true],
-        ['fbarray_autotest', true, true],
-        ['hash_readwrite_func_autotest', false, true],
-        ['ipsec_autotest', true, true],
-        ['kni_autotest', false, true],
-        ['kvargs_autotest', true, true],
-        ['member_autotest', true, true],
-        ['power_cpufreq_autotest', false, true],
-        ['power_autotest', true, true],
-        ['power_kvm_vm_autotest', false, true],
-        ['power_intel_uncore_autotest', true, true],
-        ['reorder_autotest', true, true],
-        ['service_autotest', true, true],
-        ['thash_autotest', true, true],
-        ['threads_autotest', true, true],
-        ['trace_autotest', true, true],
-]
-
-# Tests known to have issues or which don't belong in other tests lists.
-extra_test_names = [
-        'alarm_autotest', # ee00af60170b ("test: remove strict timing requirements some tests")
-        'red_autotest', # https://bugs.dpdk.org/show_bug.cgi?id=826
-]
-
-perf_test_names = [
-        'ring_perf_autotest',
-        'malloc_perf_autotest',
-        'mempool_perf_autotest',
-        'memcpy_perf_autotest',
-        'hash_perf_autotest',
-        'timer_perf_autotest',
-        'reciprocal_division',
-        'reciprocal_division_perf',
-        'lpm_perf_autotest',
-        'rib_slow_autotest',
-        'fib_slow_autotest',
-        'fib_perf_autotest',
-        'red_all',
-        'pie_all',
-        'barrier_autotest',
-        'hash_multiwriter_autotest',
-        'timer_racecond_autotest',
-        'efd_autotest',
-        'hash_functions_autotest',
-        'member_perf_autotest',
-        'efd_perf_autotest',
-        'lpm6_perf_autotest',
-        'rib6_slow_autotest',
-        'fib6_slow_autotest',
-        'fib6_perf_autotest',
-        'rcu_qsbr_perf_autotest',
-        'red_perf',
-        'pie_perf',
-        'distributor_perf_autotest',
-        'pmd_perf_autotest',
-        'service_perf_autotest',
-        'stack_perf_autotest',
-        'stack_lf_perf_autotest',
-        'rand_perf_autotest',
-        'hash_readwrite_perf_autotest',
-        'hash_readwrite_lf_perf_autotest',
-        'trace_perf_autotest',
-        'ipsec_perf_autotest',
-        'thash_perf_autotest',
-        'reassembly_perf_autotest',
-]
-
-driver_test_names = [
-        'cryptodev_aesni_gcm_autotest',
-        'cryptodev_aesni_mb_autotest',
-        'cryptodev_chacha_poly_mb_autotest',
-        'cryptodev_cn10k_autotest',
-        'cryptodev_cn9k_autotest',
-        'cryptodev_cpu_aesni_mb_autotest',
-        'cryptodev_cpu_aesni_gcm_autotest',
-        'cryptodev_dpaa2_sec_autotest',
-        'cryptodev_dpaa_sec_autotest',
-        'cryptodev_null_autotest',
-        'cryptodev_openssl_autotest',
-        'cryptodev_qat_autotest',
-        'cryptodev_qat_asym_autotest',
-        'cryptodev_qat_raw_api_autotest',
-        'cryptodev_sw_armv8_autotest',
-        'cryptodev_sw_kasumi_autotest',
-        'cryptodev_sw_mvsam_autotest',
-        'cryptodev_sw_snow3g_autotest',
-        'cryptodev_sw_zuc_autotest',
-        'cryptodev_uadk_autotest',
-        'dmadev_autotest',
-]
-
-dump_test_names = []
-
-if not is_windows
-    driver_test_names += [
-            'cryptodev_openssl_asym_autotest',
-            'eventdev_selftest_octeontx',
-            'eventdev_selftest_sw',
-    ]
-
-    dump_test_names += [
-            'dump_struct_sizes',
-            'dump_mempool',
-            'dump_malloc_stats',
-            'dump_devargs',
-            'dump_log_types',
-            'dump_ring',
-            'dump_physmem',
-            'dump_memzone',
-    ]
-endif
-
-# The following linkages are an exception to allow running the
-# unit tests without requiring that the developer install the
-# DPDK libraries.  Explicit linkage of drivers (plugin libraries)
-# in applications should not be used.
-if dpdk_conf.has('RTE_MEMPOOL_RING')
-    test_deps += 'mempool_ring'
-endif
-if dpdk_conf.has('RTE_MEMPOOL_STACK')
-    test_deps += 'mempool_stack'
-endif
-if dpdk_conf.has('RTE_EVENT_SKELETON')
-    test_deps += 'event_skeleton'
-endif
-
-if dpdk_conf.has('RTE_LIB_FLOW_CLASSIFY')
-    test_sources += 'test_flow_classify.c'
-    fast_tests += [['flow_classify_autotest', false, true]]
-endif
-if dpdk_conf.has('RTE_LIB_GRAPH')
-    test_sources += 'test_graph.c'
-    fast_tests += [['graph_autotest', true, true]]
-    fast_tests += [['node_list_dump', true, true]]
-    test_sources += 'test_graph_perf.c'
-    perf_test_names += 'graph_perf_autotest'
-endif
-if dpdk_conf.has('RTE_LIB_METRICS')
-    test_sources += ['test_metrics.c']
-    fast_tests += [['metrics_autotest', true, true]]
-endif
-if not is_windows and dpdk_conf.has('RTE_LIB_TELEMETRY')
-    test_sources += ['test_telemetry_json.c', 'test_telemetry_data.c']
-    fast_tests += [['telemetry_json_autotest', true, true]]
-    fast_tests += [['telemetry_data_autotest', true, true]]
-endif
-if dpdk_conf.has('RTE_LIB_PIPELINE')
-# pipeline lib depends on port and table libs, so those must be present
-# if pipeline library is.
-    test_sources += [
-            'test_table.c',
-            'test_table_acl.c',
-            'test_table_combined.c',
-            'test_table_pipeline.c',
-            'test_table_ports.c',
-            'test_table_tables.c',
-    ]
-    fast_tests += [['table_autotest', true, true]]
-endif
-
-# The following linkages of drivers are required because
-# they are used via a driver-specific API.
-if dpdk_conf.has('RTE_NET_BOND')
-    test_deps += 'net_bond'
-    test_sources += ['test_link_bonding.c', 'test_link_bonding_rssconf.c']
-    driver_test_names += ['link_bonding_autotest', 'link_bonding_rssconf_autotest']
-    if dpdk_conf.has('RTE_NET_RING')
-        test_sources += 'test_link_bonding_mode4.c'
-        driver_test_names += 'link_bonding_mode4_autotest'
-    endif
-endif
-if dpdk_conf.has('RTE_LIB_EVENTDEV') and dpdk_conf.has('RTE_NET_RING')
-    test_deps += 'net_ring'
-    test_sources += 'test_pmd_ring_perf.c'
-    test_sources += 'test_pmd_ring.c'
-    test_sources += 'test_event_eth_tx_adapter.c'
-    test_sources += 'sample_packet_forward.c'
-    fast_tests += [['ring_pmd_autotest', true, true]]
-    perf_test_names += 'ring_pmd_perf_autotest'
-    fast_tests += [['event_eth_tx_adapter_autotest', false, true]]
-    if dpdk_conf.has('RTE_LIB_BITRATESTATS')
-        test_sources += 'test_bitratestats.c'
-        fast_tests += [['bitratestats_autotest', true, true]]
-    endif
-    if dpdk_conf.has('RTE_LIB_LATENCYSTATS')
-        test_sources += 'test_latencystats.c'
-        fast_tests += [['latencystats_autotest', true, true]]
-    endif
-    if dpdk_conf.has('RTE_LIB_PDUMP')
-        test_sources += 'test_pdump.c'
-        fast_tests += [['pdump_autotest', true, false]]
-    endif
-endif
-if dpdk_conf.has('RTE_NET_NULL')
-    test_deps += 'net_null'
-    test_sources += 'test_vdev.c'
-    fast_tests += [['vdev_autotest', true, true]]
-endif
-if dpdk_conf.has('RTE_RAW_SKELETON')
-    test_deps += 'raw_skeleton'
-    fast_tests += [['rawdev_autotest', true, true]]
-endif
-
-if dpdk_conf.has('RTE_HAS_LIBPCAP')
-    ext_deps += pcap_dep
-    if dpdk_conf.has('RTE_LIB_PCAPNG')
-        test_sources += 'test_pcapng.c'
+# Copyright(c) 2017-2023 Intel Corporation
+
+source_file_deps = {
+    'commands.c': ['ring', 'mempool', 'mbuf'],
+    'packet_burst_generator.c': ['mbuf', 'net'],
+    'resource.c': [],
+    'sample_packet_forward.c': ['net_ring', 'ethdev', 'mbuf', 'bus_vdev', 'mempool', 'ring'],
+    'test.c': ['timer'],
+    'test_acl.c': ['mbuf', 'net', 'acl'],
+    'test_alarm.c': [],
+    'test_atomic.c': ['hash'],
+    'test_barrier.c': [],
+    'test_bitmap.c': [],
+    'test_bitops.c': [],
+    'test_bitratestats.c': ['metrics', 'bitratestats', 'ethdev'],
+    'test_bpf.c': ['bpf', 'net'],
+    'test_byteorder.c': [],
+#    'test_cfgfile.c': ['cfgfile'],
+    'test_cksum.c': ['net', 'mbuf'],
+    'test_cksum_perf.c': ['net'],
+    'test_cmdline.c': [],
+    'test_cmdline_cirbuf.c': [],
+    'test_cmdline_etheraddr.c': ['net'],
+    'test_cmdline_ipaddr.c': [],
+    'test_cmdline_lib.c': [],
+    'test_cmdline_num.c': [],
+    'test_cmdline_portlist.c': [],
+    'test_cmdline_string.c': [],
+    'test_common.c': [],
+    'test_compressdev.c': ['mempool', 'mbuf', 'compressdev'],
+    'test_cpuflags.c': [],
+    'test_crc.c': ['net'],
+    'test_cryptodev.c': ['mbuf', 'bus_vdev', 'net', 'cryptodev', 'crypto_scheduler', 'security'],
+    'test_cryptodev_asym.c': ['bus_vdev', 'mbuf', 'cryptodev'],
+    'test_cryptodev_blockcipher.c': ['mbuf', 'cryptodev'],
+    'test_cryptodev_crosscheck.c': ['cryptodev'],
+    'test_cryptodev_security_ipsec.c': ['cryptodev', 'net', 'security'],
+    'test_cryptodev_security_pdcp.c': ['mbuf', 'bus_vdev', 'cryptodev', 'security'],
+    'test_cycles.c': [],
+    'test_debug.c': [],
+    'test_devargs.c': ['kvargs'],
+    'test_distributor.c': ['mempool', 'mbuf', 'distributor'],
+    'test_distributor_perf.c': ['mempool', 'mbuf', 'distributor'],
+    'test_dmadev.c': ['dmadev', 'mbuf', 'bus_vdev'],
+    'test_dmadev_api.c': ['dmadev'],
+    'test_eal_flags.c': [],
+    'test_eal_fs.c': [],
+    'test_efd.c': ['efd', 'net'],
+    'test_efd_perf.c': ['efd', 'hash'],
+    'test_errno.c': [],
+    'test_ethdev_link.c': ['ethdev'],
+    'test_event_crypto_adapter.c': ['mempool', 'mbuf', 'cryptodev', 'eventdev', 'bus_vdev'],
+    'test_event_eth_rx_adapter.c': ['mempool', 'mbuf', 'ethdev', 'eventdev', 'bus_vdev'],
+    'test_event_eth_tx_adapter.c': ['bus_vdev', 'ethdev', 'net_ring', 'eventdev', 'mbuf', 'mempool'],
+    'test_event_ring.c': ['eventdev'],
+    'test_event_timer_adapter.c': ['ethdev', 'eventdev', 'mempool', 'bus_vdev'],
+    'test_eventdev.c': ['mbuf', 'eventdev', 'bus_vdev'],
+    'test_external_mem.c': ['ring'],
+    'test_fbarray.c': [],
+    'test_fib.c': ['net', 'fib'],
+    'test_fib6.c': ['rib', 'fib'],
+    'test_fib6_perf.c': ['fib'],
+    'test_fib_perf.c': ['net', 'fib'],
+    'test_flow_classify.c': ['mbuf', 'net', 'acl', 'table', 'ethdev', 'flow_classify'],
+    'test_func_reentrancy.c': ['ring', 'mempool', 'hash', 'lpm'],
+    'test_graph.c': ['graph', 'mbuf'],
+    'test_graph_perf.c': ['graph', 'mbuf'],
+    'test_hash.c': ['net', 'hash'],
+    'test_hash_functions.c': ['hash'],
+    'test_hash_multiwriter.c': ['hash'],
+    'test_hash_perf.c': ['hash'],
+    'test_hash_readwrite.c': ['hash'],
+    'test_hash_readwrite_lf_perf.c': ['hash'],
+    'test_interrupts.c': [],
+    'test_ipfrag.c': ['net', 'ip_frag', 'mbuf'],
+    'test_ipsec.c': ['mbuf', 'bus_vdev', 'net', 'cryptodev', 'ipsec', 'security'],
+    'test_ipsec_perf.c': ['net', 'ring', 'mbuf', 'ipsec'],
+    'test_ipsec_sad.c': ['ipsec'],
+    'test_kni.c': ['mempool', 'ethdev', 'kni'],
+    'test_kvargs.c': ['kvargs'],
+    'test_latencystats.c': ['ethdev', 'latencystats', 'metrics'],
+    'test_lcores.c': [],
+    'test_link_bonding.c': ['ethdev', 'net_bond', 'net', 'mbuf'],
+    'test_link_bonding_mode4.c': ['ethdev', 'net_ring', 'net_bond', 'mbuf', 'net'],
+    'test_link_bonding_rssconf.c': ['ethdev', 'bus_vdev', 'net_bond'],
+    'test_logs.c': [],
+    'test_lpm.c': ['net', 'lpm'],
+    'test_lpm6.c': ['lpm'],
+    'test_lpm6_perf.c': ['lpm'],
+    'test_lpm_perf.c': ['net', 'lpm'],
+    'test_malloc.c': [],
+    'test_malloc_perf.c': [],
+    'test_mbuf.c': ['ring', 'mempool', 'mbuf', 'net'],
+    'test_mcslock.c': [],
+    'test_member.c': ['member', 'net'],
+    'test_member_perf.c': ['hash', 'member'],
+    'test_memcpy.c': [],
+    'test_memcpy_perf.c': [],
+    'test_memory.c': [],
+    'test_mempool.c': ['mempool', 'mbuf'],
+    'test_mempool_perf.c': ['mempool', 'mbuf'],
+    'test_memzone.c': [],
+    'test_meter.c': ['meter'],
+    'test_metrics.c': ['metrics'],
+    'test_mp_secondary.c': ['ring', 'mempool', 'hash', 'lpm'],
+    'test_pcapng.c': ['ethdev', 'net', 'mbuf', 'mempool', 'pcapng'],
+    'test_pdcp.c': ['eventdev', 'pdcp', 'net', 'timer', 'security'],
+    'test_pdump.c': ['pdump', 'mempool', 'ring'],
+    'test_per_lcore.c': [],
+    'test_pflock.c': [],
+    'test_pie.c': ['sched'],
+    'test_pmd_perf.c': ['ethdev', 'mbuf', 'net'],
+    'test_pmd_ring.c': ['net_ring', 'ethdev', 'bus_vdev'],
+    'test_pmd_ring_perf.c': ['ring', 'ethdev', 'net_ring', 'bus_vdev'],
+    'test_power.c': ['power'],
+    'test_power_cpufreq.c': ['power'],
+    'test_power_intel_uncore.c': ['power'],
+    'test_power_kvm_vm.c': ['power'],
+    'test_prefetch.c': [],
+    'test_rand_perf.c': [],
+    'test_rawdev.c': ['mbuf', 'rawdev', 'bus_vdev'],
+    'test_rcu_qsbr.c': ['rcu', 'hash'],
+    'test_rcu_qsbr_perf.c': ['rcu', 'hash'],
+    'test_reassembly_perf.c': ['net', 'ip_frag', 'mbuf'],
+    'test_reciprocal_division.c': [],
+    'test_reciprocal_division_perf.c': [],
+    'test_red.c': ['sched'],
+    'test_reorder.c': ['mbuf', 'reorder'],
+#    'test_resource.c': [],
+    'test_rib.c': ['net', 'rib'],
+    'test_rib6.c': ['net', 'rib'],
+    'test_ring.c': ['ring'],
+    'test_ring_hts_stress.c': [],
+    'test_ring_mpmc_stress.c': [],
+    'test_ring_mt_peek_stress.c': ['ring'],
+    'test_ring_mt_peek_stress_zc.c': ['ring'],
+    'test_ring_perf.c': ['ring'],
+    'test_ring_rts_stress.c': [],
+    'test_ring_st_peek_stress.c': ['ring'],
+    'test_ring_st_peek_stress_zc.c': ['ring'],
+    'test_ring_stress.c': ['ring'],
+    'test_rwlock.c': [],
+    'test_sched.c': ['net', 'sched'],
+    'test_security.c': ['mempool', 'net', 'security'],
+    'test_security_inline_macsec.c': ['ethdev', 'security'],
+    'test_security_inline_proto.c': ['ethdev', 'security', 'eventdev'],
+    'test_seqlock.c': [],
+    'test_service_cores.c': ['mbuf'],
+    'test_spinlock.c': [],
+    'test_stack.c': ['stack'],
+    'test_stack_perf.c': ['stack'],
+    'test_string_fns.c': [],
+    'test_table.c': ['table', 'pipeline', 'port'],
+    'test_table_acl.c': ['net', 'table', 'pipeline', 'port'],
+    'test_table_combined.c': ['table', 'pipeline', 'port'],
+    'test_table_pipeline.c': ['pipeline', 'table', 'port'],
+    'test_table_ports.c': ['table', 'pipeline', 'port'],
+    'test_table_tables.c': ['table', 'pipeline', 'port'],
+    'test_tailq.c': [],
+    'test_telemetry_data.c': ['telemetry'],
+    'test_telemetry_json.c': ['telemetry'],
+    'test_thash.c': ['net', 'hash'],
+    'test_thash_perf.c': ['hash'],
+    'test_threads.c': [],
+    'test_ticketlock.c': [],
+    'test_timer.c': ['timer'],
+    'test_timer_perf.c': ['timer'],
+    'test_timer_racecond.c': ['timer'],
+    'test_timer_secondary.c': ['timer', 'mempool'],
+    'test_trace.c': [],
+    'test_trace_perf.c': [],
+    'test_trace_register.c': [],
+    'test_vdev.c': ['kvargs', 'bus_vdev'],
+    'test_version.c': [],
+    'virtual_pmd.c': ['mbuf', 'ethdev', 'pci', 'ring', 'net'],
+}
+
+# autotests always need cmdline library
+deps += 'cmdline'
+
+def_lib = get_option('default_library')
+foreach f, f_deps : source_file_deps
+    has_deps = true
+    foreach d : f_deps
+        if not is_variable(def_lib + '_rte_' + d)
+            has_deps = false
+            break
+        else
+            # technically we might not need this dep, but adding it is harmless
+            if d not in deps
+                deps += d
+            endif
+        endif
+    endforeach
+    if has_deps
+        sources += files(f)
     endif
-endif
-
-if dpdk_conf.has('RTE_LIB_PDCP')
-    test_sources += 'test_pdcp.c'
-    fast_tests += [['pdcp_autotest', false, true]]
-endif
+endforeach
 
 if cc.has_argument('-Wno-format-truncation')
     cflags += '-Wno-format-truncation'
@@ -456,154 +206,4 @@ endif
 cflags += '-fno-strict-aliasing'
 
 # Enable using internal APIs in unit tests
-cflags += ['-DALLOW_INTERNAL_API']
-
-test_dep_objs = []
-if dpdk_conf.has('RTE_LIB_COMPRESSDEV')
-    compress_test_dep = dependency('zlib', required: false, method: 'pkg-config')
-    if compress_test_dep.found()
-        test_dep_objs += compress_test_dep
-        test_sources += 'test_compressdev.c'
-        fast_tests += [['compressdev_autotest', false, true]]
-    endif
-endif
-
-if dpdk_conf.has('RTE_CRYPTO_SCHEDULER')
-    driver_test_names += 'cryptodev_scheduler_autotest'
-    test_deps += 'crypto_scheduler'
-endif
-
-foreach d:test_deps
-    def_lib = get_option('default_library')
-    test_dep_objs += get_variable(def_lib + '_rte_' + d)
-endforeach
-
-link_libs = []
-if get_option('default_library') == 'static'
-    link_libs = dpdk_static_libraries + dpdk_drivers
-endif
-
-dpdk_test = executable('dpdk-test',
-        test_sources,
-        link_whole: link_libs,
-        dependencies: test_dep_objs + ext_deps,
-        c_args: cflags,
-        install_rpath: join_paths(get_option('prefix'),
-             driver_install_path),
-        install: true)
-
-has_hugepage = run_command(py3, files('has_hugepage.py'), check: true).stdout().strip() != '0'
-message('hugepage availability: @0@'.format(has_hugepage))
-
-# some perf tests (eg: memcpy perf autotest)take very long
-# to complete, so timeout to 10 minutes
-timeout_seconds = 600
-timeout_seconds_fast = 10
-
-test_no_huge_args = ['--no-huge', '-m', '2048']
-
-foreach arg : fast_tests
-    test_args = []
-    run_test = true
-    if not has_hugepage
-        if arg[1]
-            test_args += test_no_huge_args
-        else
-            run_test = false
-        endif
-    endif
-
-    if get_option('b_sanitize') == 'address' or get_option('b_sanitize') == 'address,undefined'
-        if not arg[2]
-            run_test = false
-        endif
-    endif
-
-    if (get_option('default_library') == 'shared' and
-        arg[0] == 'event_eth_tx_adapter_autotest')
-        test_args += ['-d', dpdk_drivers_build_dir]
-    endif
-    if is_linux
-        test_args += ['--file-prefix=@0@'.format(arg[0])]
-    endif
-
-    if run_test
-        test(arg[0], dpdk_test,
-                env : ['DPDK_TEST=' + arg[0]],
-                args : test_args,
-                timeout : timeout_seconds_fast,
-                is_parallel : false,
-                suite : 'fast-tests')
-        if not is_windows and arg[0] == 'trace_autotest'
-            test_args += ['--trace=.*']
-            test_args += ['--trace-dir=@0@'.format(meson.current_build_dir())]
-            test(arg[0] + '_with_traces', dpdk_test,
-                    env : ['DPDK_TEST=' + arg[0]],
-                    args : test_args,
-                    timeout : timeout_seconds_fast,
-                    is_parallel : false,
-                    suite : 'fast-tests')
-        endif
-    endif
-endforeach
-
-if not is_windows and dpdk_conf.has('RTE_LIB_TELEMETRY')
-    test_args = [dpdk_test]
-    test_args += test_no_huge_args
-    if get_option('default_library') == 'shared'
-        test_args += ['-d', dpdk_drivers_build_dir]
-    endif
-    if dpdk_conf.has('RTE_CRYPTO_NULL')
-        test_args += ['--vdev=crypto_null0']
-    endif
-    if dpdk_conf.has('RTE_DMA_SKELETON')
-        test_args += ['--vdev=dma_skeleton0']
-    endif
-    if dpdk_conf.has('RTE_EVENT_SKELETON')
-        test_args += ['--vdev=event_skeleton0']
-    endif
-    if dpdk_conf.has('RTE_NET_NULL')
-        test_args += ['--vdev=net_null0']
-    endif
-    if dpdk_conf.has('RTE_RAW_SKELETON')
-        test_args += ['--vdev=rawdev_skeleton0']
-    endif
-    test_args += ['-a', '0000:00:00.0']
-    test('telemetry_all', find_program('test_telemetry.sh'),
-            args: test_args,
-            timeout : timeout_seconds_fast,
-            is_parallel : false,
-            suite : 'fast-tests')
-endif
-
-foreach arg : perf_test_names
-    test(arg, dpdk_test,
-            env : ['DPDK_TEST=' + arg],
-            timeout : timeout_seconds,
-            is_parallel : false,
-            suite : 'perf-tests')
-endforeach
-
-foreach arg : driver_test_names
-    test(arg, dpdk_test,
-            env : ['DPDK_TEST=' + arg],
-            timeout : timeout_seconds,
-            is_parallel : false,
-            suite : 'driver-tests')
-endforeach
-
-foreach arg : dump_test_names
-    test(arg, dpdk_test,
-            env : ['DPDK_TEST=' + arg],
-            timeout : timeout_seconds,
-            is_parallel : false,
-            suite : 'debug-tests')
-endforeach
-
-foreach arg : extra_test_names
-    test(arg, dpdk_test,
-            env : ['DPDK_TEST=' + arg],
-            timeout : timeout_seconds,
-            is_parallel : false,
-            suite : 'extra-tests')
-endforeach
+cflags += '-DALLOW_INTERNAL_API'
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [RFC PATCH 4/5] app/test: define unit tests suites based on test macros
  2023-07-21 11:51 [RFC PATCH 0/5] replace build code for unit tests Bruce Richardson
                   ` (2 preceding siblings ...)
  2023-07-21 11:51 ` [RFC PATCH 3/5] app/test: build using per-file dependency matrix Bruce Richardson
@ 2023-07-21 11:51 ` Bruce Richardson
  2023-07-21 11:51 ` [RFC PATCH 5/5] app/test: add fast test suite to new build infrastructure Bruce Richardson
                   ` (6 subsequent siblings)
  10 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-07-21 11:51 UTC (permalink / raw)
  To: dev; +Cc: Bruce Richardson

Rather than having the test suites listed out in the meson.build files
and having to have them enabled/disabled selectively based on what libs
are being built, pull the tests to run from the source files which were
added to the build.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 app/meson.build               |  7 ++++++-
 app/test/suites/meson.build   | 29 +++++++++++++++++++++++++++++
 buildtools/get-test-suites.py | 24 ++++++++++++++++++++++++
 buildtools/meson.build        |  1 +
 4 files changed, 60 insertions(+), 1 deletion(-)
 create mode 100644 app/test/suites/meson.build
 create mode 100644 buildtools/get-test-suites.py

diff --git a/app/meson.build b/app/meson.build
index 0d8b618e7f..c14dc80892 100644
--- a/app/meson.build
+++ b/app/meson.build
@@ -101,7 +101,7 @@ foreach app:apps
         link_libs = dpdk_static_libraries + dpdk_drivers
     endif
 
-    executable('dpdk-' + name,
+    exec = executable('dpdk-' + name,
             sources,
             c_args: cflags,
             link_args: ldflags,
@@ -110,4 +110,9 @@ foreach app:apps
             include_directories: includes,
             install_rpath: join_paths(get_option('prefix'), driver_install_path),
             install: true)
+    if name == 'test'
+        dpdk_test = exec
+        autotest_sources = sources
+        subdir('test/suites')  # define the pre-canned test suites
+    endif
 endforeach
diff --git a/app/test/suites/meson.build b/app/test/suites/meson.build
new file mode 100644
index 0000000000..135620578c
--- /dev/null
+++ b/app/test/suites/meson.build
@@ -0,0 +1,29 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2023 Intel Corporation
+
+# some perf tests (eg: memcpy perf autotest)take very long
+# to complete, so timeout to 10 minutes
+timeout_seconds = 600
+
+# process source files to determine the different unit test suites
+# - fast_tests
+# - perf_tests
+# - driver_tests
+test_suites = run_command(get_test_suites_cmd, autotest_sources).stdout().strip().split()
+foreach suite:test_suites
+    suite = suite.split('=')
+    suite_name = suite[0]
+    suite_tests = suite[1].split(',')
+    if suite_name == 'fast-tests'
+    # special fast-test handling here
+
+    else
+        foreach t: suite_tests
+            test(t, dpdk_test,
+                    args: ['--', t],
+                    timeout: timeout_seconds,
+                    is_parallel: false,
+                    suite: suite_name)
+        endforeach
+    endif
+endforeach
diff --git a/buildtools/get-test-suites.py b/buildtools/get-test-suites.py
new file mode 100644
index 0000000000..43cde80970
--- /dev/null
+++ b/buildtools/get-test-suites.py
@@ -0,0 +1,24 @@
+#! /usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2023 Intel Corporation
+
+import sys
+import re
+
+input_list = sys.argv[1:]
+test_def_regex = re.compile("REGISTER_([A-Z]+)_TEST\s*\(\s*([a-z0-9_]+)")
+test_suites = {}
+
+for fname in input_list:
+    with open(fname) as f:
+        contents = [ln for ln in f.readlines() if test_def_regex.match(ln.strip())]
+    for ln in contents:
+        (test_suite, test_name) = test_def_regex.match(ln).group(1, 2)
+        suite_name = f"{test_suite.lower()}-tests"
+        if suite_name in test_suites:
+            test_suites[suite_name].append(test_name)
+        else:
+            test_suites[suite_name] = [test_name]
+
+for suite in test_suites.keys():
+    print(f"{suite}={','.join(test_suites[suite])}")
diff --git a/buildtools/meson.build b/buildtools/meson.build
index e1c600e40f..12e4b36165 100644
--- a/buildtools/meson.build
+++ b/buildtools/meson.build
@@ -18,6 +18,7 @@ map_to_win_cmd = py3 + files('map_to_win.py')
 sphinx_wrapper = py3 + files('call-sphinx-build.py')
 get_cpu_count_cmd = py3 + files('get-cpu-count.py')
 get_numa_count_cmd = py3 + files('get-numa-count.py')
+get_test_suites_cmd = py3 + files('get-test-suites.py')
 binutils_avx512_check = (py3 + files('binutils-avx512-check.py') +
                         [objdump] + cc.cmd_array())
 
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [RFC PATCH 5/5] app/test: add fast test suite to new build infrastructure
  2023-07-21 11:51 [RFC PATCH 0/5] replace build code for unit tests Bruce Richardson
                   ` (3 preceding siblings ...)
  2023-07-21 11:51 ` [RFC PATCH 4/5] app/test: define unit tests suites based on test macros Bruce Richardson
@ 2023-07-21 11:51 ` Bruce Richardson
  2023-08-08  8:46 ` [RFC PATCH 0/5] replace build code for unit tests David Marchand
                   ` (5 subsequent siblings)
  10 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-07-21 11:51 UTC (permalink / raw)
  To: dev; +Cc: Bruce Richardson

The fast-tests are special in that they have additional parameters
associated with them. This requires script changes and meson.build
changes to take account of these parameters.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 app/test/suites/meson.build                   | 56 +++++++++++++++++--
 buildtools/get-test-suites.py                 |  9 +++
 .../has-hugepages.py                          |  0
 buildtools/meson.build                        |  1 +
 4 files changed, 61 insertions(+), 5 deletions(-)
 rename app/test/has_hugepage.py => buildtools/has-hugepages.py (100%)

diff --git a/app/test/suites/meson.build b/app/test/suites/meson.build
index 135620578c..720b43eb62 100644
--- a/app/test/suites/meson.build
+++ b/app/test/suites/meson.build
@@ -4,20 +4,24 @@
 # some perf tests (eg: memcpy perf autotest)take very long
 # to complete, so timeout to 10 minutes
 timeout_seconds = 600
+timeout_seconds_fast = 10
+
+test_no_huge_args = ['--no-huge', '-m', '2048']
+has_hugepage = run_command(has_hugepages_cmd, check: true).stdout().strip() != '0'
+message('hugepage availability: @0@'.format(has_hugepage))
 
 # process source files to determine the different unit test suites
 # - fast_tests
 # - perf_tests
 # - driver_tests
-test_suites = run_command(get_test_suites_cmd, autotest_sources).stdout().strip().split()
+test_suites = run_command(get_test_suites_cmd, autotest_sources,
+         check: true).stdout().strip().split()
 foreach suite:test_suites
     suite = suite.split('=')
     suite_name = suite[0]
     suite_tests = suite[1].split(',')
-    if suite_name == 'fast-tests'
-    # special fast-test handling here
-
-    else
+    if suite_name != 'fast-tests'
+        # simple cases - tests without parameters or special handling
         foreach t: suite_tests
             test(t, dpdk_test,
                     args: ['--', t],
@@ -25,5 +29,47 @@ foreach suite:test_suites
                     is_parallel: false,
                     suite: suite_name)
         endforeach
+    else
+    # special fast-test handling here
+        foreach t: suite_tests
+            params = t.split(':')
+            test_name = params[0]
+            nohuge = params[1] == 'true'
+            asan = params[2] == 'true'
+
+            test_args = []
+            if nohuge
+                test_args += test_no_huge_args
+            elif not has_hugepage
+                continue  #skip this tests
+            endif
+            if not asan and (get_option('b_sanitize') == 'address'
+                    or get_option('b_sanitize') == 'address,undefined')
+                continue  # skip this test
+            endif
+
+            if (get_option('default_library') == 'shared' 
+                    and test_name == 'event_eth_tx_adapter_autotest')
+                test_args += ['-d', dpdk_drivers_build_dir]
+            endif
+            if is_linux
+                test_args += ['--file-prefix=@0@'.format(arg[0])]
+            endif
+
+            test(test_name, dpdk_test,
+                args : test_args + ['--', test_name],
+                timeout : timeout_seconds_fast,
+                is_parallel : false,
+                suite : 'fast-tests')
+            if not is_windows and test_name == 'trace_autotest'
+                test_args += ['--trace=.*']
+                test_args += ['--trace-dir=@0@'.format(meson.current_build_dir())]
+                test(test_name + '_with_traces', dpdk_test,
+                    args : test_args + ['--', test_name],
+                    timeout : timeout_seconds_fast,
+                    is_parallel : false,
+                    suite : 'fast-tests')
+            endif
+        endforeach
     endif
 endforeach
diff --git a/buildtools/get-test-suites.py b/buildtools/get-test-suites.py
index 43cde80970..95a9cad4c8 100644
--- a/buildtools/get-test-suites.py
+++ b/buildtools/get-test-suites.py
@@ -9,6 +9,13 @@
 test_def_regex = re.compile("REGISTER_([A-Z]+)_TEST\s*\(\s*([a-z0-9_]+)")
 test_suites = {}
 
+def get_fast_test_params(test_name, ln):
+    "Extract the extra fast-test parameters from the line"
+    #print(f"ln: {ln.rstrip()}, test_name: {test_name}, split: {ln.split(test_name, 1)}")
+    (_, rest_of_line) = ln.split(test_name, 1)
+    (_, nohuge, asan, _func) = rest_of_line.split(',', 3)
+    return f":{nohuge.strip().lower()}:{asan.strip().lower()}"
+
 for fname in input_list:
     with open(fname) as f:
         contents = [ln for ln in f.readlines() if test_def_regex.match(ln.strip())]
@@ -19,6 +26,8 @@
             test_suites[suite_name].append(test_name)
         else:
             test_suites[suite_name] = [test_name]
+        if suite_name == "fast-tests":
+            test_suites["fast-tests"][-1] += get_fast_test_params(test_name, ln)
 
 for suite in test_suites.keys():
     print(f"{suite}={','.join(test_suites[suite])}")
diff --git a/app/test/has_hugepage.py b/buildtools/has-hugepages.py
similarity index 100%
rename from app/test/has_hugepage.py
rename to buildtools/has-hugepages.py
diff --git a/buildtools/meson.build b/buildtools/meson.build
index 12e4b36165..ac5e4dcf08 100644
--- a/buildtools/meson.build
+++ b/buildtools/meson.build
@@ -19,6 +19,7 @@ sphinx_wrapper = py3 + files('call-sphinx-build.py')
 get_cpu_count_cmd = py3 + files('get-cpu-count.py')
 get_numa_count_cmd = py3 + files('get-numa-count.py')
 get_test_suites_cmd = py3 + files('get-test-suites.py')
+has_hugepages_cmd = py3 + files('has-hugepages.py')
 binutils_avx512_check = (py3 + files('binutils-avx512-check.py') +
                         [objdump] + cc.cmd_array())
 
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* Re: [RFC PATCH 0/5] replace build code for unit tests
  2023-07-21 11:51 [RFC PATCH 0/5] replace build code for unit tests Bruce Richardson
                   ` (4 preceding siblings ...)
  2023-07-21 11:51 ` [RFC PATCH 5/5] app/test: add fast test suite to new build infrastructure Bruce Richardson
@ 2023-08-08  8:46 ` David Marchand
  2023-08-09 14:53   ` Patrick Robb
  2023-08-14 15:16 ` [PATCH v2 0/8] expand list of optional libraries Bruce Richardson
                   ` (4 subsequent siblings)
  10 siblings, 1 reply; 93+ messages in thread
From: David Marchand @ 2023-08-08  8:46 UTC (permalink / raw)
  To: Bruce Richardson
  Cc: dev, Tyler Retzlaff, Thomas Monjalon, Morten Brørup,
	Aaron Conole, ci

Hello Bruce,

On Fri, Jul 21, 2023 at 1:51 PM Bruce Richardson
<bruce.richardson@intel.com> wrote:
>
> While working on series such as [1] to make DPDK builds more
> configurable, the majority of the complexity involved centered on the
> unit tests, and ensuring that the individual files were added/removed
> from the build as the components were enabled/disabled.
>
> This complexity was exacerbated by:
> * C files having multiple dependencies, specifically,
> * Some libraries having one set of independent tests, while other
>   library tests requiring additional components
> * Having to maintain lists of tests in various suites which also needed
>   to be updated, depending on component availability.
> * Everything having to be checked via if-else clauses, since this was
>   not data-driven
>
> This RFC attempts to address these issues by doing the following:
> * Creating a per-file dependency list for the autotests, allowing easy
>   checks of when a file could be built
> * Tagging the various fast-tests/perf-tests/drivers-tests suites via
>   macros in the C code. Then scan the C-files we are building to
>   construct the test suites based on what is available

I did not look into the implementation details too much but I like the idea.

>
>
> This RFC is just an early prototype, and probably needs cleanup and
> validation of the dependency lists. Windows support is probably also
> broken, given the fact that our current build file maintains custom
> lists there. However, I would hope it should not be too big a job to
> enable the unit tests for the windows components that are already
> available. (Tests for any components not being build on windows are
> already removed via the dependency lists).
>
> Another (hopefully less-impactful) side-effect of this change is that
> the "test" binary has moved in the build folder. It is now put
> along-side the other binaries in "build/app" rather than being in the
> subdirectory "build/app/test".

Cc: ci people.

It may break some CI scripts but it is hard to tell which one.
As far as testsuites we run in GHA or UNH are concerned, it is done
through meson test so we are good on this side.


>
> Future work could also be done to scan the C files that are not being
> build for tests. Those tests could then be stubbed out in an
> auto-genereated file to always return "SKIPPED", meaning that all test
> commands are always valid, irrespective of what components are present
> or not.
>
> [1] http://patches.dpdk.org/project/dpdk/list/?series=28628&state=10
>
> Bruce Richardson (5):
>   app/test: add new macros for various test types
>   app/test: tag tests with the test type
>   app/test: build using per-file dependency matrix
>   app/test: define unit tests suites based on test macros
>   app/test: add fast test suite to new build infrastructure
>
>  app/meson.build                               |  15 +-
>  app/test/meson.build                          | 796 +++++-------------
>  app/test/suites/meson.build                   |  75 ++
>  app/test/test.h                               |   9 +-
>  app/test/test_acl.c                           |   2 +-
>  app/test/test_atomic.c                        |   2 +-
>  app/test/test_barrier.c                       |   2 +-
>  app/test/test_bitmap.c                        |   2 +-
>  app/test/test_bitops.c                        |   2 +-
>  app/test/test_bpf.c                           |   4 +-
>  app/test/test_byteorder.c                     |   2 +-
>  app/test/test_cksum.c                         |   2 +-
>  app/test/test_cmdline.c                       |   2 +-
>  app/test/test_common.c                        |   2 +-
>  app/test/test_cpuflags.c                      |   2 +-
>  app/test/test_crc.c                           |   2 +-
>  app/test/test_cryptodev.c                     |  38 +-
>  app/test/test_cryptodev_asym.c                |   2 +-
>  app/test/test_cycles.c                        |   2 +-
>  app/test/test_debug.c                         |   2 +-
>  app/test/test_devargs.c                       |   2 +-
>  app/test/test_distributor.c                   |   2 +-
>  app/test/test_distributor_perf.c              |   2 +-
>  app/test/test_dmadev.c                        |   2 +-
>  app/test/test_eal_flags.c                     |  24 +-
>  app/test/test_eal_fs.c                        |   2 +-
>  app/test/test_efd.c                           |   2 +-
>  app/test/test_efd_perf.c                      |   2 +-
>  app/test/test_errno.c                         |   2 +-
>  app/test/test_ethdev_link.c                   |   2 +-
>  app/test/test_event_ring.c                    |   2 +-
>  app/test/test_eventdev.c                      |   2 +-
>  app/test/test_fbarray.c                       |   2 +-
>  app/test/test_fib.c                           |   4 +-
>  app/test/test_fib6.c                          |   4 +-
>  app/test/test_fib6_perf.c                     |   2 +-
>  app/test/test_fib_perf.c                      |   2 +-
>  app/test/test_func_reentrancy.c               |   2 +-
>  app/test/test_hash.c                          |   2 +-
>  app/test/test_hash_functions.c                |   2 +-
>  app/test/test_hash_multiwriter.c              |   2 +-
>  app/test/test_hash_perf.c                     |   2 +-
>  app/test/test_hash_readwrite.c                |   4 +-
>  app/test/test_hash_readwrite_lf_perf.c        |   2 +-
>  app/test/test_interrupts.c                    |   2 +-
>  app/test/test_ipfrag.c                        |   2 +-
>  app/test/test_ipsec.c                         |   2 +-
>  app/test/test_ipsec_perf.c                    |   2 +-
>  app/test/test_kni.c                           |   2 +-
>  app/test/test_kvargs.c                        |   2 +-
>  app/test/test_lcores.c                        |   2 +-
>  app/test/test_logs.c                          |   2 +-
>  app/test/test_lpm.c                           |   2 +-
>  app/test/test_lpm6.c                          |   2 +-
>  app/test/test_lpm6_perf.c                     |   2 +-
>  app/test/test_lpm_perf.c                      |   2 +-
>  app/test/test_malloc.c                        |   2 +-
>  app/test/test_malloc_perf.c                   |   2 +-
>  app/test/test_mbuf.c                          |   2 +-
>  app/test/test_mcslock.c                       |   2 +-
>  app/test/test_member.c                        |   2 +-
>  app/test/test_member_perf.c                   |   2 +-
>  app/test/test_memcpy.c                        |   2 +-
>  app/test/test_memcpy_perf.c                   |   2 +-
>  app/test/test_memory.c                        |   2 +-
>  app/test/test_mempool.c                       |   2 +-
>  app/test/test_mempool_perf.c                  |   2 +-
>  app/test/test_memzone.c                       |   2 +-
>  app/test/test_meter.c                         |   2 +-
>  app/test/test_mp_secondary.c                  |   2 +-
>  app/test/test_per_lcore.c                     |   2 +-
>  app/test/test_pflock.c                        |   2 +-
>  app/test/test_pie.c                           |   6 +-
>  app/test/test_pmd_perf.c                      |   2 +-
>  app/test/test_power.c                         |   2 +-
>  app/test/test_power_cpufreq.c                 |   2 +-
>  app/test/test_power_intel_uncore.c            |   2 +-
>  app/test/test_power_kvm_vm.c                  |   2 +-
>  app/test/test_prefetch.c                      |   2 +-
>  app/test/test_rand_perf.c                     |   2 +-
>  app/test/test_rcu_qsbr.c                      |   2 +-
>  app/test/test_rcu_qsbr_perf.c                 |   2 +-
>  app/test/test_reassembly_perf.c               |   2 +-
>  app/test/test_reciprocal_division.c           |   2 +-
>  app/test/test_reciprocal_division_perf.c      |   2 +-
>  app/test/test_red.c                           |   4 +-
>  app/test/test_reorder.c                       |   2 +-
>  app/test/test_rib.c                           |   4 +-
>  app/test/test_rib6.c                          |   4 +-
>  app/test/test_ring.c                          |   2 +-
>  app/test/test_ring_perf.c                     |   2 +-
>  app/test/test_rwlock.c                        |   8 +-
>  app/test/test_sched.c                         |   2 +-
>  app/test/test_security.c                      |   2 +-
>  app/test/test_seqlock.c                       |   2 +-
>  app/test/test_service_cores.c                 |   4 +-
>  app/test/test_spinlock.c                      |   2 +-
>  app/test/test_stack.c                         |   4 +-
>  app/test/test_stack_perf.c                    |   4 +-
>  app/test/test_string_fns.c                    |   2 +-
>  app/test/test_tailq.c                         |   2 +-
>  app/test/test_thash.c                         |   2 +-
>  app/test/test_thash_perf.c                    |   2 +-
>  app/test/test_threads.c                       |   2 +-
>  app/test/test_ticketlock.c                    |   2 +-
>  app/test/test_timer.c                         |   2 +-
>  app/test/test_timer_perf.c                    |   2 +-
>  app/test/test_timer_racecond.c                |   2 +-
>  app/test/test_trace.c                         |   2 +-
>  app/test/test_trace_perf.c                    |   2 +-
>  app/test/test_version.c                       |   2 +-
>  buildtools/get-test-suites.py                 |  33 +
>  .../has-hugepages.py                          |   0
>  buildtools/meson.build                        |   2 +
>  114 files changed, 478 insertions(+), 754 deletions(-)
>  create mode 100644 app/test/suites/meson.build
>  create mode 100644 buildtools/get-test-suites.py
>  rename app/test/has_hugepage.py => buildtools/has-hugepages.py (100%)
>
> --
> 2.39.2
>


-- 
David Marchand


^ permalink raw reply	[flat|nested] 93+ messages in thread

* Re: [RFC PATCH 0/5] replace build code for unit tests
  2023-08-08  8:46 ` [RFC PATCH 0/5] replace build code for unit tests David Marchand
@ 2023-08-09 14:53   ` Patrick Robb
  2023-08-09 15:05     ` Bruce Richardson
  0 siblings, 1 reply; 93+ messages in thread
From: Patrick Robb @ 2023-08-09 14:53 UTC (permalink / raw)
  To: David Marchand
  Cc: Bruce Richardson, dev, Tyler Retzlaff, Thomas Monjalon,
	Morten Brørup, Aaron Conole, ci

[-- Attachment #1: Type: text/plain, Size: 334 bytes --]

It will break our 32 bit arm testing because we are unable to use meson
test after building for that SoC, so we run through unit tests with a
custom list using just dpdk test. It's not a problem, I just need to know
when this is going to reach main so I can make the corresponding change to
the 32 bit unit testing script beforehand.

[-- Attachment #2: Type: text/html, Size: 383 bytes --]

^ permalink raw reply	[flat|nested] 93+ messages in thread

* Re: [RFC PATCH 0/5] replace build code for unit tests
  2023-08-09 14:53   ` Patrick Robb
@ 2023-08-09 15:05     ` Bruce Richardson
  2023-08-09 15:39       ` Patrick Robb
  0 siblings, 1 reply; 93+ messages in thread
From: Bruce Richardson @ 2023-08-09 15:05 UTC (permalink / raw)
  To: Patrick Robb
  Cc: David Marchand, dev, Tyler Retzlaff, Thomas Monjalon,
	Morten Brørup, Aaron Conole, ci

On Wed, Aug 09, 2023 at 10:53:21AM -0400, Patrick Robb wrote:
>    It will break our 32 bit arm testing because we are unable to use meson
>    test after building for that SoC, so we run through unit tests with a
>    custom list using just dpdk test. It's not a problem, I just need to
>    know when this is going to reach main so I can make the corresponding
>    change to the 32 bit unit testing script beforehand.

The change is necessary because the test binary has moved? Just want to
check there are no other issues. I might see if I can provide a symlink or
something for compatibility if it helps.

^ permalink raw reply	[flat|nested] 93+ messages in thread

* Re: [RFC PATCH 0/5] replace build code for unit tests
  2023-08-09 15:05     ` Bruce Richardson
@ 2023-08-09 15:39       ` Patrick Robb
  0 siblings, 0 replies; 93+ messages in thread
From: Patrick Robb @ 2023-08-09 15:39 UTC (permalink / raw)
  To: Bruce Richardson
  Cc: David Marchand, dev, Tyler Retzlaff, Thomas Monjalon,
	Morten Brørup, Aaron Conole, ci

[-- Attachment #1: Type: text/plain, Size: 898 bytes --]

On Wed, Aug 9, 2023 at 11:05 AM Bruce Richardson <bruce.richardson@intel.com>
wrote:

> On Wed, Aug 09, 2023 at 10:53:21AM -0400, Patrick Robb wrote:
> >    It will break our 32 bit arm testing because we are unable to use
> meson
> >    test after building for that SoC, so we run through unit tests with a
> >    custom list using just dpdk test. It's not a problem, I just need to
> >    know when this is going to reach main so I can make the corresponding
> >    change to the 32 bit unit testing script beforehand.
>
> The change is necessary because the test binary has moved? Just want to
> check there are no other issues. I might see if I can provide a symlink or
> something for compatibility if it helps.
>

That's correct, there are no other issues, so it really is simple to
correct the patch in our script. A symlink should work, but I guess it
becomes clutter.

[-- Attachment #2: Type: text/html, Size: 1290 bytes --]

^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v2 0/8] expand list of optional libraries
  2023-07-21 11:51 [RFC PATCH 0/5] replace build code for unit tests Bruce Richardson
                   ` (5 preceding siblings ...)
  2023-08-08  8:46 ` [RFC PATCH 0/5] replace build code for unit tests David Marchand
@ 2023-08-14 15:16 ` Bruce Richardson
  2023-08-14 15:16   ` [PATCH v2 1/8] app/test: add new macros for various test types Bruce Richardson
                     ` (8 more replies)
  2023-08-14 18:20 ` [PATCH v3 " Bruce Richardson
                   ` (3 subsequent siblings)
  10 siblings, 9 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-14 15:16 UTC (permalink / raw)
  To: dev; +Cc: ci, Bruce Richardson

This patchset contains changes to DPDK to make the build more configurable
overall. The changes can be grouped into two areas:

* Firstly, there are changes to make the build of the unit tests more
  flexible and maintainable. These 5 patches switch the unit tests from
  having separate hard-coded lists of files and tests, with lots of
  conditional logic to add/remove items from the list, to instead being
  a single list of files and dependencies per-file. The actual lists of
  test-suites are built up dynamically from the list of files whose
  dependencies are met and therefore will be part of the build.
  This rework enables...
* The second set of patches which expands the list of libraries which
  can be disabled at build-time, while still giving a successful build
  of DPDK. Overall, after this set is applied, the list of optional
  libraries in DPDK is approx doubled in size. [15 additions giving
  31 optional libs]

NOTES:
* as stated in the RFC, the test binary moves in the build as a result
  of this set - from app/test/dpdk-test to app/dpdk-test.
* running the suites of tests no longer works after patch 4 until patch
  5 is applied. If this is an issue, the two patches can be merged on
  apply. However, the individual tests are still runnable through the
  test binary - it's only through "meson test" that there are issues, so
  I'm keeping the patches separate since each is relatively complex in its
  own right.

V2:
- Changes since RFC:
   -  merged two previous sets - reworking test build, and disabling
      libs - into one, since they depend on each other
   -  Reworked the list of dependencies for each file in the tests,
      so that files that depend on other files have those deps called
      out.
   -  Added list of external dependencies to the test builds
   -  For disabling libs, merged patches disabling individual libs
      together into one patch, since all unit test build changes were
      unnecessary. This made each patch a single-line addition to
      the optional list in lib/meson.build.

Bruce Richardson (8):
  app/test: add new macros for various test types
  app/test: tag tests with the test type
  app/test: make telemetry data test buildable on windows
  app/test: build using per-file dependency matrix
  app/test: define unit tests suites based on test macros
  examples/l3fwd: make eventdev an optional dependency
  build: make most device classes optional
  build: expand list of optional libraries

 app/meson.build                               |  15 +-
 app/test/meson.build                          | 818 +++++-------------
 app/test/suites/meson.build                   |  76 ++
 app/test/test.h                               |   9 +-
 app/test/test_acl.c                           |   2 +-
 app/test/test_atomic.c                        |   2 +-
 app/test/test_barrier.c                       |   2 +-
 app/test/test_bitmap.c                        |   2 +-
 app/test/test_bitops.c                        |   2 +-
 app/test/test_bpf.c                           |   4 +-
 app/test/test_byteorder.c                     |   2 +-
 app/test/test_cksum.c                         |   2 +-
 app/test/test_cmdline.c                       |   2 +-
 app/test/test_common.c                        |   2 +-
 app/test/test_cpuflags.c                      |   2 +-
 app/test/test_crc.c                           |   2 +-
 app/test/test_cryptodev.c                     |  38 +-
 app/test/test_cryptodev_asym.c                |   2 +-
 app/test/test_cycles.c                        |   2 +-
 app/test/test_debug.c                         |   2 +-
 app/test/test_devargs.c                       |   2 +-
 app/test/test_distributor.c                   |   2 +-
 app/test/test_distributor_perf.c              |   2 +-
 app/test/test_dmadev.c                        |   2 +-
 app/test/test_eal_flags.c                     |  24 +-
 app/test/test_eal_fs.c                        |   2 +-
 app/test/test_efd.c                           |   2 +-
 app/test/test_efd_perf.c                      |   2 +-
 app/test/test_errno.c                         |   2 +-
 app/test/test_ethdev_link.c                   |   2 +-
 app/test/test_event_ring.c                    |   2 +-
 app/test/test_eventdev.c                      |   2 +-
 app/test/test_fbarray.c                       |   2 +-
 app/test/test_fib.c                           |   4 +-
 app/test/test_fib6.c                          |   4 +-
 app/test/test_fib6_perf.c                     |   2 +-
 app/test/test_fib_perf.c                      |   2 +-
 app/test/test_func_reentrancy.c               |   2 +-
 app/test/test_hash.c                          |   2 +-
 app/test/test_hash_functions.c                |   2 +-
 app/test/test_hash_multiwriter.c              |   2 +-
 app/test/test_hash_perf.c                     |   2 +-
 app/test/test_hash_readwrite.c                |   4 +-
 app/test/test_hash_readwrite_lf_perf.c        |   2 +-
 app/test/test_interrupts.c                    |   2 +-
 app/test/test_ipfrag.c                        |   2 +-
 app/test/test_ipsec.c                         |   2 +-
 app/test/test_ipsec_perf.c                    |   2 +-
 app/test/test_kvargs.c                        |   2 +-
 app/test/test_lcores.c                        |   2 +-
 app/test/test_logs.c                          |   2 +-
 app/test/test_lpm.c                           |   2 +-
 app/test/test_lpm6.c                          |   2 +-
 app/test/test_lpm6_perf.c                     |   2 +-
 app/test/test_lpm_perf.c                      |   2 +-
 app/test/test_malloc.c                        |   2 +-
 app/test/test_malloc_perf.c                   |   2 +-
 app/test/test_mbuf.c                          |   2 +-
 app/test/test_mcslock.c                       |   2 +-
 app/test/test_member.c                        |   2 +-
 app/test/test_member_perf.c                   |   2 +-
 app/test/test_memcpy.c                        |   2 +-
 app/test/test_memcpy_perf.c                   |   2 +-
 app/test/test_memory.c                        |   2 +-
 app/test/test_mempool.c                       |   2 +-
 app/test/test_mempool_perf.c                  |   2 +-
 app/test/test_memzone.c                       |   2 +-
 app/test/test_meter.c                         |   2 +-
 app/test/test_mp_secondary.c                  |   2 +-
 app/test/test_per_lcore.c                     |   2 +-
 app/test/test_pflock.c                        |   2 +-
 app/test/test_pie.c                           |   6 +-
 app/test/test_pmd_perf.c                      |   2 +-
 app/test/test_power.c                         |   2 +-
 app/test/test_power_cpufreq.c                 |   2 +-
 app/test/test_power_intel_uncore.c            |   2 +-
 app/test/test_power_kvm_vm.c                  |   2 +-
 app/test/test_prefetch.c                      |   2 +-
 app/test/test_rand_perf.c                     |   2 +-
 app/test/test_rcu_qsbr.c                      |   2 +-
 app/test/test_rcu_qsbr_perf.c                 |   2 +-
 app/test/test_reassembly_perf.c               |   2 +-
 app/test/test_reciprocal_division.c           |   2 +-
 app/test/test_reciprocal_division_perf.c      |   2 +-
 app/test/test_red.c                           |   4 +-
 app/test/test_reorder.c                       |   2 +-
 app/test/test_rib.c                           |   4 +-
 app/test/test_rib6.c                          |   4 +-
 app/test/test_ring.c                          |   2 +-
 app/test/test_ring_perf.c                     |   2 +-
 app/test/test_rwlock.c                        |   8 +-
 app/test/test_sched.c                         |   2 +-
 app/test/test_security.c                      |   2 +-
 app/test/test_seqlock.c                       |   2 +-
 app/test/test_service_cores.c                 |   4 +-
 app/test/test_spinlock.c                      |   2 +-
 app/test/test_stack.c                         |   4 +-
 app/test/test_stack_perf.c                    |   4 +-
 app/test/test_string_fns.c                    |   2 +-
 app/test/test_tailq.c                         |   2 +-
 app/test/test_telemetry_data.c                |  14 +-
 app/test/test_thash.c                         |   2 +-
 app/test/test_thash_perf.c                    |   2 +-
 app/test/test_threads.c                       |   2 +-
 app/test/test_ticketlock.c                    |   2 +-
 app/test/test_timer.c                         |   2 +-
 app/test/test_timer_perf.c                    |   2 +-
 app/test/test_timer_racecond.c                |   2 +-
 app/test/test_trace.c                         |   2 +-
 app/test/test_trace_perf.c                    |   2 +-
 app/test/test_version.c                       |   2 +-
 buildtools/get-test-suites.py                 |  33 +
 .../has-hugepages.py                          |   0
 buildtools/meson.build                        |   2 +
 examples/l3fwd/l3fwd_em.c                     |   2 +
 examples/l3fwd/l3fwd_em_hlm.h                 |   2 +
 examples/l3fwd/l3fwd_event.c                  |   2 +
 examples/l3fwd/l3fwd_event.h                  |   7 +-
 examples/l3fwd/l3fwd_event_generic.c          |   2 +
 examples/l3fwd/l3fwd_event_internal_port.c    |   2 +
 examples/l3fwd/l3fwd_fib.c                    |   2 +
 examples/l3fwd/l3fwd_lpm.c                    |   2 +
 examples/l3fwd/main.c                         |  65 +-
 examples/l3fwd/meson.build                    |   5 +-
 lib/meson.build                               |  15 +
 125 files changed, 607 insertions(+), 766 deletions(-)
 create mode 100644 app/test/suites/meson.build
 create mode 100644 buildtools/get-test-suites.py
 rename app/test/has_hugepage.py => buildtools/has-hugepages.py (100%)

--
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v2 1/8] app/test: add new macros for various test types
  2023-08-14 15:16 ` [PATCH v2 0/8] expand list of optional libraries Bruce Richardson
@ 2023-08-14 15:16   ` Bruce Richardson
  2023-08-14 15:16   ` [PATCH v2 2/8] app/test: tag tests with the test type Bruce Richardson
                     ` (7 subsequent siblings)
  8 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-14 15:16 UTC (permalink / raw)
  To: dev; +Cc: ci, Bruce Richardson

Rather than just registering all tests using a single generic macro,
add macros which identify the test as being of a particular type.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 app/test/test.h | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/app/test/test.h b/app/test/test.h
index 85f57efbc6..a91ded76af 100644
--- a/app/test/test.h
+++ b/app/test/test.h
@@ -190,7 +190,7 @@ struct test_command {
 
 void add_test_command(struct test_command *t);
 
-/* Register a test function with its command string */
+/* Register a test function with its command string. Should not be used directly */
 #define REGISTER_TEST_COMMAND(cmd, func) \
 	static struct test_command test_struct_##cmd = { \
 		.command = RTE_STR(cmd), \
@@ -201,4 +201,11 @@ void add_test_command(struct test_command *t);
 		add_test_command(&test_struct_##cmd); \
 	}
 
+/* Register a test function as a particular type.
+ * These can be used to build up test suites automatically
+ */
+#define REGISTER_FAST_TEST(cmd, no_huge, ASan, func)  REGISTER_TEST_COMMAND(cmd, func)
+#define REGISTER_PERF_TEST REGISTER_TEST_COMMAND
+#define REGISTER_DRIVER_TEST REGISTER_TEST_COMMAND
+
 #endif
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v2 2/8] app/test: tag tests with the test type
  2023-08-14 15:16 ` [PATCH v2 0/8] expand list of optional libraries Bruce Richardson
  2023-08-14 15:16   ` [PATCH v2 1/8] app/test: add new macros for various test types Bruce Richardson
@ 2023-08-14 15:16   ` Bruce Richardson
  2023-08-14 15:16   ` [PATCH v2 3/8] app/test: make telemetry data test buildable on windows Bruce Richardson
                     ` (6 subsequent siblings)
  8 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-14 15:16 UTC (permalink / raw)
  To: dev; +Cc: ci, Bruce Richardson

Rather than having the test types called out in the meson.build file, we
can use macros to identify the test type in the C file itself and then
dynamically build up the tests lists at config time.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 app/test/test_acl.c                      |  2 +-
 app/test/test_atomic.c                   |  2 +-
 app/test/test_barrier.c                  |  2 +-
 app/test/test_bitmap.c                   |  2 +-
 app/test/test_bitops.c                   |  2 +-
 app/test/test_bpf.c                      |  4 +--
 app/test/test_byteorder.c                |  2 +-
 app/test/test_cksum.c                    |  2 +-
 app/test/test_cmdline.c                  |  2 +-
 app/test/test_common.c                   |  2 +-
 app/test/test_cpuflags.c                 |  2 +-
 app/test/test_crc.c                      |  2 +-
 app/test/test_cryptodev.c                | 38 ++++++++++++------------
 app/test/test_cryptodev_asym.c           |  2 +-
 app/test/test_cycles.c                   |  2 +-
 app/test/test_debug.c                    |  2 +-
 app/test/test_devargs.c                  |  2 +-
 app/test/test_distributor.c              |  2 +-
 app/test/test_distributor_perf.c         |  2 +-
 app/test/test_dmadev.c                   |  2 +-
 app/test/test_eal_flags.c                | 24 +++++++--------
 app/test/test_eal_fs.c                   |  2 +-
 app/test/test_efd.c                      |  2 +-
 app/test/test_efd_perf.c                 |  2 +-
 app/test/test_errno.c                    |  2 +-
 app/test/test_ethdev_link.c              |  2 +-
 app/test/test_event_ring.c               |  2 +-
 app/test/test_eventdev.c                 |  2 +-
 app/test/test_fbarray.c                  |  2 +-
 app/test/test_fib.c                      |  4 +--
 app/test/test_fib6.c                     |  4 +--
 app/test/test_fib6_perf.c                |  2 +-
 app/test/test_fib_perf.c                 |  2 +-
 app/test/test_func_reentrancy.c          |  2 +-
 app/test/test_hash.c                     |  2 +-
 app/test/test_hash_functions.c           |  2 +-
 app/test/test_hash_multiwriter.c         |  2 +-
 app/test/test_hash_perf.c                |  2 +-
 app/test/test_hash_readwrite.c           |  4 +--
 app/test/test_hash_readwrite_lf_perf.c   |  2 +-
 app/test/test_interrupts.c               |  2 +-
 app/test/test_ipfrag.c                   |  2 +-
 app/test/test_ipsec.c                    |  2 +-
 app/test/test_ipsec_perf.c               |  2 +-
 app/test/test_kvargs.c                   |  2 +-
 app/test/test_lcores.c                   |  2 +-
 app/test/test_logs.c                     |  2 +-
 app/test/test_lpm.c                      |  2 +-
 app/test/test_lpm6.c                     |  2 +-
 app/test/test_lpm6_perf.c                |  2 +-
 app/test/test_lpm_perf.c                 |  2 +-
 app/test/test_malloc.c                   |  2 +-
 app/test/test_malloc_perf.c              |  2 +-
 app/test/test_mbuf.c                     |  2 +-
 app/test/test_mcslock.c                  |  2 +-
 app/test/test_member.c                   |  2 +-
 app/test/test_member_perf.c              |  2 +-
 app/test/test_memcpy.c                   |  2 +-
 app/test/test_memcpy_perf.c              |  2 +-
 app/test/test_memory.c                   |  2 +-
 app/test/test_mempool.c                  |  2 +-
 app/test/test_mempool_perf.c             |  2 +-
 app/test/test_memzone.c                  |  2 +-
 app/test/test_meter.c                    |  2 +-
 app/test/test_mp_secondary.c             |  2 +-
 app/test/test_per_lcore.c                |  2 +-
 app/test/test_pflock.c                   |  2 +-
 app/test/test_pie.c                      |  6 ++--
 app/test/test_pmd_perf.c                 |  2 +-
 app/test/test_power.c                    |  2 +-
 app/test/test_power_cpufreq.c            |  2 +-
 app/test/test_power_intel_uncore.c       |  2 +-
 app/test/test_power_kvm_vm.c             |  2 +-
 app/test/test_prefetch.c                 |  2 +-
 app/test/test_rand_perf.c                |  2 +-
 app/test/test_rcu_qsbr.c                 |  2 +-
 app/test/test_rcu_qsbr_perf.c            |  2 +-
 app/test/test_reassembly_perf.c          |  2 +-
 app/test/test_reciprocal_division.c      |  2 +-
 app/test/test_reciprocal_division_perf.c |  2 +-
 app/test/test_red.c                      |  4 +--
 app/test/test_reorder.c                  |  2 +-
 app/test/test_rib.c                      |  4 +--
 app/test/test_rib6.c                     |  4 +--
 app/test/test_ring.c                     |  2 +-
 app/test/test_ring_perf.c                |  2 +-
 app/test/test_rwlock.c                   |  8 ++---
 app/test/test_sched.c                    |  2 +-
 app/test/test_security.c                 |  2 +-
 app/test/test_seqlock.c                  |  2 +-
 app/test/test_service_cores.c            |  4 +--
 app/test/test_spinlock.c                 |  2 +-
 app/test/test_stack.c                    |  4 +--
 app/test/test_stack_perf.c               |  4 +--
 app/test/test_string_fns.c               |  2 +-
 app/test/test_tailq.c                    |  2 +-
 app/test/test_thash.c                    |  2 +-
 app/test/test_thash_perf.c               |  2 +-
 app/test/test_threads.c                  |  2 +-
 app/test/test_ticketlock.c               |  2 +-
 app/test/test_timer.c                    |  2 +-
 app/test/test_timer_perf.c               |  2 +-
 app/test/test_timer_racecond.c           |  2 +-
 app/test/test_trace.c                    |  2 +-
 app/test/test_trace_perf.c               |  2 +-
 app/test/test_version.c                  |  2 +-
 106 files changed, 150 insertions(+), 150 deletions(-)

diff --git a/app/test/test_acl.c b/app/test/test_acl.c
index 623f34682e..bf1466fe11 100644
--- a/app/test/test_acl.c
+++ b/app/test/test_acl.c
@@ -1749,4 +1749,4 @@ test_acl(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(acl_autotest, test_acl);
+REGISTER_FAST_TEST(acl_autotest, True, True, test_acl);
diff --git a/app/test/test_atomic.c b/app/test/test_atomic.c
index e4b997827e..b52420c10b 100644
--- a/app/test/test_atomic.c
+++ b/app/test/test_atomic.c
@@ -631,4 +631,4 @@ test_atomic(void)
 
 	return 0;
 }
-REGISTER_TEST_COMMAND(atomic_autotest, test_atomic);
+REGISTER_FAST_TEST(atomic_autotest, False, True, test_atomic);
diff --git a/app/test/test_barrier.c b/app/test/test_barrier.c
index ec69af25bf..925a88b68a 100644
--- a/app/test/test_barrier.c
+++ b/app/test/test_barrier.c
@@ -285,4 +285,4 @@ test_barrier(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(barrier_autotest, test_barrier);
+REGISTER_PERF_TEST(barrier_autotest, test_barrier);
diff --git a/app/test/test_bitmap.c b/app/test/test_bitmap.c
index e9c61590ae..2014d2682c 100644
--- a/app/test/test_bitmap.c
+++ b/app/test/test_bitmap.c
@@ -269,4 +269,4 @@ test_bitmap(void)
 	return test_bitmap_all_set();
 }
 
-REGISTER_TEST_COMMAND(bitmap_autotest, test_bitmap);
+REGISTER_FAST_TEST(bitmap_autotest, True, True, test_bitmap);
diff --git a/app/test/test_bitops.c b/app/test/test_bitops.c
index c21426bf2f..dac466602b 100644
--- a/app/test/test_bitops.c
+++ b/app/test/test_bitops.c
@@ -135,4 +135,4 @@ test_bitops(void)
 	return TEST_SUCCESS;
 }
 
-REGISTER_TEST_COMMAND(bitops_autotest, test_bitops);
+REGISTER_FAST_TEST(bitops_autotest, True, True, test_bitops);
diff --git a/app/test/test_bpf.c b/app/test/test_bpf.c
index f5af5e8a3f..5fef17fbc2 100644
--- a/app/test/test_bpf.c
+++ b/app/test/test_bpf.c
@@ -3262,7 +3262,7 @@ test_bpf(void)
 
 #endif /* !RTE_LIB_BPF */
 
-REGISTER_TEST_COMMAND(bpf_autotest, test_bpf);
+REGISTER_FAST_TEST(bpf_autotest, True, True, test_bpf);
 
 #ifndef RTE_HAS_LIBPCAP
 
@@ -3473,4 +3473,4 @@ test_bpf_convert(void)
 
 #endif /* RTE_HAS_LIBPCAP */
 
-REGISTER_TEST_COMMAND(bpf_convert_autotest, test_bpf_convert);
+REGISTER_FAST_TEST(bpf_convert_autotest, True, True, test_bpf_convert);
diff --git a/app/test/test_byteorder.c b/app/test/test_byteorder.c
index de14ed539e..2189f94744 100644
--- a/app/test/test_byteorder.c
+++ b/app/test/test_byteorder.c
@@ -63,4 +63,4 @@ test_byteorder(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(byteorder_autotest, test_byteorder);
+REGISTER_FAST_TEST(byteorder_autotest, True, True, test_byteorder);
diff --git a/app/test/test_cksum.c b/app/test/test_cksum.c
index 6c15de9a93..c04fae800e 100644
--- a/app/test/test_cksum.c
+++ b/app/test/test_cksum.c
@@ -267,4 +267,4 @@ test_cksum(void)
 }
 #undef GOTO_FAIL
 
-REGISTER_TEST_COMMAND(cksum_autotest, test_cksum);
+REGISTER_FAST_TEST(cksum_autotest, True, True, test_cksum);
diff --git a/app/test/test_cmdline.c b/app/test/test_cmdline.c
index 115bee966d..c2a375ac1c 100644
--- a/app/test/test_cmdline.c
+++ b/app/test/test_cmdline.c
@@ -60,4 +60,4 @@ test_cmdline(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(cmdline_autotest, test_cmdline);
+REGISTER_FAST_TEST(cmdline_autotest, True, True, test_cmdline);
diff --git a/app/test/test_common.c b/app/test/test_common.c
index f89e1eb7ee..c7b41a3f0e 100644
--- a/app/test/test_common.c
+++ b/app/test/test_common.c
@@ -350,4 +350,4 @@ test_common(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(common_autotest, test_common);
+REGISTER_FAST_TEST(common_autotest, True, True, test_common);
diff --git a/app/test/test_cpuflags.c b/app/test/test_cpuflags.c
index a0e342ae48..c3417dda24 100644
--- a/app/test/test_cpuflags.c
+++ b/app/test/test_cpuflags.c
@@ -334,4 +334,4 @@ test_cpuflags(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(cpuflags_autotest, test_cpuflags);
+REGISTER_FAST_TEST(cpuflags_autotest, True, True, test_cpuflags);
diff --git a/app/test/test_crc.c b/app/test/test_crc.c
index 5edc8fb13b..b267bdde9f 100644
--- a/app/test/test_crc.c
+++ b/app/test/test_crc.c
@@ -171,4 +171,4 @@ test_crc(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(crc_autotest, test_crc);
+REGISTER_FAST_TEST(crc_autotest, True, True, test_crc);
diff --git a/app/test/test_cryptodev.c b/app/test/test_cryptodev.c
index fb2af40b99..956268bfcd 100644
--- a/app/test/test_cryptodev.c
+++ b/app/test/test_cryptodev.c
@@ -17535,33 +17535,33 @@ REGISTER_TEST_COMMAND(cryptodev_dpaa2_sec_raw_api_autotest,
 		test_cryptodev_dpaa2_sec_raw_api);
 REGISTER_TEST_COMMAND(cryptodev_dpaa_sec_raw_api_autotest,
 		test_cryptodev_dpaa_sec_raw_api);
-REGISTER_TEST_COMMAND(cryptodev_qat_raw_api_autotest,
+REGISTER_DRIVER_TEST(cryptodev_qat_raw_api_autotest,
 		test_cryptodev_qat_raw_api);
-REGISTER_TEST_COMMAND(cryptodev_qat_autotest, test_cryptodev_qat);
-REGISTER_TEST_COMMAND(cryptodev_aesni_mb_autotest, test_cryptodev_aesni_mb);
-REGISTER_TEST_COMMAND(cryptodev_cpu_aesni_mb_autotest,
+REGISTER_DRIVER_TEST(cryptodev_qat_autotest, test_cryptodev_qat);
+REGISTER_DRIVER_TEST(cryptodev_aesni_mb_autotest, test_cryptodev_aesni_mb);
+REGISTER_DRIVER_TEST(cryptodev_cpu_aesni_mb_autotest,
 	test_cryptodev_cpu_aesni_mb);
-REGISTER_TEST_COMMAND(cryptodev_chacha_poly_mb_autotest,
+REGISTER_DRIVER_TEST(cryptodev_chacha_poly_mb_autotest,
 	test_cryptodev_chacha_poly_mb);
-REGISTER_TEST_COMMAND(cryptodev_openssl_autotest, test_cryptodev_openssl);
-REGISTER_TEST_COMMAND(cryptodev_aesni_gcm_autotest, test_cryptodev_aesni_gcm);
-REGISTER_TEST_COMMAND(cryptodev_cpu_aesni_gcm_autotest,
+REGISTER_DRIVER_TEST(cryptodev_openssl_autotest, test_cryptodev_openssl);
+REGISTER_DRIVER_TEST(cryptodev_aesni_gcm_autotest, test_cryptodev_aesni_gcm);
+REGISTER_DRIVER_TEST(cryptodev_cpu_aesni_gcm_autotest,
 	test_cryptodev_cpu_aesni_gcm);
 REGISTER_TEST_COMMAND(cryptodev_mlx5_autotest, test_cryptodev_mlx5);
-REGISTER_TEST_COMMAND(cryptodev_null_autotest, test_cryptodev_null);
-REGISTER_TEST_COMMAND(cryptodev_sw_snow3g_autotest, test_cryptodev_sw_snow3g);
-REGISTER_TEST_COMMAND(cryptodev_sw_kasumi_autotest, test_cryptodev_sw_kasumi);
-REGISTER_TEST_COMMAND(cryptodev_sw_zuc_autotest, test_cryptodev_sw_zuc);
-REGISTER_TEST_COMMAND(cryptodev_sw_armv8_autotest, test_cryptodev_armv8);
-REGISTER_TEST_COMMAND(cryptodev_sw_mvsam_autotest, test_cryptodev_mrvl);
-REGISTER_TEST_COMMAND(cryptodev_dpaa2_sec_autotest, test_cryptodev_dpaa2_sec);
-REGISTER_TEST_COMMAND(cryptodev_dpaa_sec_autotest, test_cryptodev_dpaa_sec);
+REGISTER_DRIVER_TEST(cryptodev_null_autotest, test_cryptodev_null);
+REGISTER_DRIVER_TEST(cryptodev_sw_snow3g_autotest, test_cryptodev_sw_snow3g);
+REGISTER_DRIVER_TEST(cryptodev_sw_kasumi_autotest, test_cryptodev_sw_kasumi);
+REGISTER_DRIVER_TEST(cryptodev_sw_zuc_autotest, test_cryptodev_sw_zuc);
+REGISTER_DRIVER_TEST(cryptodev_sw_armv8_autotest, test_cryptodev_armv8);
+REGISTER_DRIVER_TEST(cryptodev_sw_mvsam_autotest, test_cryptodev_mrvl);
+REGISTER_DRIVER_TEST(cryptodev_dpaa2_sec_autotest, test_cryptodev_dpaa2_sec);
+REGISTER_DRIVER_TEST(cryptodev_dpaa_sec_autotest, test_cryptodev_dpaa_sec);
 REGISTER_TEST_COMMAND(cryptodev_ccp_autotest, test_cryptodev_ccp);
-REGISTER_TEST_COMMAND(cryptodev_uadk_autotest, test_cryptodev_uadk);
+REGISTER_DRIVER_TEST(cryptodev_uadk_autotest, test_cryptodev_uadk);
 REGISTER_TEST_COMMAND(cryptodev_virtio_autotest, test_cryptodev_virtio);
 REGISTER_TEST_COMMAND(cryptodev_octeontx_autotest, test_cryptodev_octeontx);
 REGISTER_TEST_COMMAND(cryptodev_caam_jr_autotest, test_cryptodev_caam_jr);
 REGISTER_TEST_COMMAND(cryptodev_nitrox_autotest, test_cryptodev_nitrox);
 REGISTER_TEST_COMMAND(cryptodev_bcmfs_autotest, test_cryptodev_bcmfs);
-REGISTER_TEST_COMMAND(cryptodev_cn9k_autotest, test_cryptodev_cn9k);
-REGISTER_TEST_COMMAND(cryptodev_cn10k_autotest, test_cryptodev_cn10k);
+REGISTER_DRIVER_TEST(cryptodev_cn9k_autotest, test_cryptodev_cn9k);
+REGISTER_DRIVER_TEST(cryptodev_cn10k_autotest, test_cryptodev_cn10k);
diff --git a/app/test/test_cryptodev_asym.c b/app/test/test_cryptodev_asym.c
index 0ef2642fdd..3d5a73bf89 100644
--- a/app/test/test_cryptodev_asym.c
+++ b/app/test/test_cryptodev_asym.c
@@ -2828,7 +2828,7 @@ test_cryptodev_cn10k_asym(void)
 REGISTER_TEST_COMMAND(cryptodev_openssl_asym_autotest,
 					  test_cryptodev_openssl_asym);
 
-REGISTER_TEST_COMMAND(cryptodev_qat_asym_autotest, test_cryptodev_qat_asym);
+REGISTER_DRIVER_TEST(cryptodev_qat_asym_autotest, test_cryptodev_qat_asym);
 
 REGISTER_TEST_COMMAND(cryptodev_octeontx_asym_autotest,
 					  test_cryptodev_octeontx_asym);
diff --git a/app/test/test_cycles.c b/app/test/test_cycles.c
index 66d11e6db8..08c91cef78 100644
--- a/app/test/test_cycles.c
+++ b/app/test/test_cycles.c
@@ -53,4 +53,4 @@ test_user_delay_us(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(user_delay_us, test_user_delay_us);
+REGISTER_FAST_TEST(user_delay_us, True, True, test_user_delay_us);
diff --git a/app/test/test_debug.c b/app/test/test_debug.c
index 2704f5b927..acd9542257 100644
--- a/app/test/test_debug.c
+++ b/app/test/test_debug.c
@@ -140,4 +140,4 @@ test_debug(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(debug_autotest, test_debug);
+REGISTER_FAST_TEST(debug_autotest, True, True, test_debug);
diff --git a/app/test/test_devargs.c b/app/test/test_devargs.c
index 0a4c34a1ad..59ec04957e 100644
--- a/app/test/test_devargs.c
+++ b/app/test/test_devargs.c
@@ -213,4 +213,4 @@ test_devargs(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(devargs_autotest, test_devargs);
+REGISTER_FAST_TEST(devargs_autotest, True, True, test_devargs);
diff --git a/app/test/test_distributor.c b/app/test/test_distributor.c
index 3efa4af104..ee263a7fee 100644
--- a/app/test/test_distributor.c
+++ b/app/test/test_distributor.c
@@ -952,4 +952,4 @@ test_distributor(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(distributor_autotest, test_distributor);
+REGISTER_FAST_TEST(distributor_autotest, False, True, test_distributor);
diff --git a/app/test/test_distributor_perf.c b/app/test/test_distributor_perf.c
index ee4321486d..ca868451d7 100644
--- a/app/test/test_distributor_perf.c
+++ b/app/test/test_distributor_perf.c
@@ -277,4 +277,4 @@ test_distributor_perf(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(distributor_perf_autotest, test_distributor_perf);
+REGISTER_PERF_TEST(distributor_perf_autotest, test_distributor_perf);
diff --git a/app/test/test_dmadev.c b/app/test/test_dmadev.c
index 0736ff2a18..6ef875e545 100644
--- a/app/test/test_dmadev.c
+++ b/app/test/test_dmadev.c
@@ -941,4 +941,4 @@ test_dma(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(dmadev_autotest, test_dma);
+REGISTER_DRIVER_TEST(dmadev_autotest, test_dma);
diff --git a/app/test/test_eal_flags.c b/app/test/test_eal_flags.c
index 148e9098e8..3b7af7405d 100644
--- a/app/test/test_eal_flags.c
+++ b/app/test/test_eal_flags.c
@@ -1644,15 +1644,15 @@ test_memory_flags(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(eal_flags_c_opt_autotest, test_missing_c_flag);
-REGISTER_TEST_COMMAND(eal_flags_main_opt_autotest, test_main_lcore_flag);
-REGISTER_TEST_COMMAND(eal_flags_n_opt_autotest, test_invalid_n_flag);
-REGISTER_TEST_COMMAND(eal_flags_hpet_autotest, test_no_hpet_flag);
-REGISTER_TEST_COMMAND(eal_flags_no_huge_autotest, test_no_huge_flag);
-REGISTER_TEST_COMMAND(eal_flags_a_opt_autotest, test_allow_flag);
-REGISTER_TEST_COMMAND(eal_flags_b_opt_autotest, test_invalid_b_flag);
-REGISTER_TEST_COMMAND(eal_flags_vdev_opt_autotest, test_invalid_vdev_flag);
-REGISTER_TEST_COMMAND(eal_flags_r_opt_autotest, test_invalid_r_flag);
-REGISTER_TEST_COMMAND(eal_flags_mem_autotest, test_memory_flags);
-REGISTER_TEST_COMMAND(eal_flags_file_prefix_autotest, test_file_prefix);
-REGISTER_TEST_COMMAND(eal_flags_misc_autotest, test_misc_flags);
+REGISTER_FAST_TEST(eal_flags_c_opt_autotest, False, False, test_missing_c_flag);
+REGISTER_FAST_TEST(eal_flags_main_opt_autotest, False, False, test_main_lcore_flag);
+REGISTER_FAST_TEST(eal_flags_n_opt_autotest, False, False, test_invalid_n_flag);
+REGISTER_FAST_TEST(eal_flags_hpet_autotest, False, False, test_no_hpet_flag);
+REGISTER_FAST_TEST(eal_flags_no_huge_autotest, False, False, test_no_huge_flag);
+REGISTER_FAST_TEST(eal_flags_a_opt_autotest, False, False, test_allow_flag);
+REGISTER_FAST_TEST(eal_flags_b_opt_autotest, False, False, test_invalid_b_flag);
+REGISTER_FAST_TEST(eal_flags_vdev_opt_autotest, False, False, test_invalid_vdev_flag);
+REGISTER_FAST_TEST(eal_flags_r_opt_autotest, False, False, test_invalid_r_flag);
+REGISTER_FAST_TEST(eal_flags_mem_autotest, False, False, test_memory_flags);
+REGISTER_FAST_TEST(eal_flags_file_prefix_autotest, False, False, test_file_prefix);
+REGISTER_FAST_TEST(eal_flags_misc_autotest, False, False, test_misc_flags);
diff --git a/app/test/test_eal_fs.c b/app/test/test_eal_fs.c
index b3686edcb4..a4bdc1236d 100644
--- a/app/test/test_eal_fs.c
+++ b/app/test/test_eal_fs.c
@@ -185,4 +185,4 @@ test_eal_fs(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(eal_fs_autotest, test_eal_fs);
+REGISTER_FAST_TEST(eal_fs_autotest, True, True, test_eal_fs);
diff --git a/app/test/test_efd.c b/app/test/test_efd.c
index fa29e8f97a..1c0986b9bc 100644
--- a/app/test/test_efd.c
+++ b/app/test/test_efd.c
@@ -473,4 +473,4 @@ test_efd(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(efd_autotest, test_efd);
+REGISTER_PERF_TEST(efd_autotest, test_efd);
diff --git a/app/test/test_efd_perf.c b/app/test/test_efd_perf.c
index 4d04ed93e3..b212e96767 100644
--- a/app/test/test_efd_perf.c
+++ b/app/test/test_efd_perf.c
@@ -393,4 +393,4 @@ test_efd_perf(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(efd_perf_autotest, test_efd_perf);
+REGISTER_PERF_TEST(efd_perf_autotest, test_efd_perf);
diff --git a/app/test/test_errno.c b/app/test/test_errno.c
index 0db4fbc8b3..1082299665 100644
--- a/app/test/test_errno.c
+++ b/app/test/test_errno.c
@@ -94,4 +94,4 @@ test_errno(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(errno_autotest, test_errno);
+REGISTER_FAST_TEST(errno_autotest, True, True, test_errno);
diff --git a/app/test/test_ethdev_link.c b/app/test/test_ethdev_link.c
index ab52385a12..d73e18065d 100644
--- a/app/test/test_ethdev_link.c
+++ b/app/test/test_ethdev_link.c
@@ -167,4 +167,4 @@ test_link_status(void)
 	return unit_test_suite_runner(&link_status_testsuite);
 }
 
-REGISTER_TEST_COMMAND(ethdev_link_status, test_link_status);
+REGISTER_FAST_TEST(ethdev_link_status, True, True, test_link_status);
diff --git a/app/test/test_event_ring.c b/app/test/test_event_ring.c
index bc4a6e73f5..d5c8499cfd 100644
--- a/app/test/test_event_ring.c
+++ b/app/test/test_event_ring.c
@@ -256,4 +256,4 @@ test_event_ring(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(event_ring_autotest, test_event_ring);
+REGISTER_FAST_TEST(event_ring_autotest, True, True, test_event_ring);
diff --git a/app/test/test_eventdev.c b/app/test/test_eventdev.c
index 336529038e..e65d893781 100644
--- a/app/test/test_eventdev.c
+++ b/app/test/test_eventdev.c
@@ -1256,7 +1256,7 @@ test_eventdev_selftest_cn10k(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(eventdev_common_autotest, test_eventdev_common);
+REGISTER_FAST_TEST(eventdev_common_autotest, True, True, test_eventdev_common);
 
 #ifndef RTE_EXEC_ENV_WINDOWS
 REGISTER_TEST_COMMAND(eventdev_selftest_sw, test_eventdev_selftest_sw);
diff --git a/app/test/test_fbarray.c b/app/test/test_fbarray.c
index a691bf4458..03428a45fe 100644
--- a/app/test/test_fbarray.c
+++ b/app/test/test_fbarray.c
@@ -733,4 +733,4 @@ test_fbarray(void)
 	return unit_test_suite_runner(&fbarray_test_suite);
 }
 
-REGISTER_TEST_COMMAND(fbarray_autotest, test_fbarray);
+REGISTER_FAST_TEST(fbarray_autotest, True, True, test_fbarray);
diff --git a/app/test/test_fib.c b/app/test/test_fib.c
index eb69d6e2fd..2a8404db61 100644
--- a/app/test/test_fib.c
+++ b/app/test/test_fib.c
@@ -415,5 +415,5 @@ test_slow_fib(void)
 	return unit_test_suite_runner(&fib_slow_tests);
 }
 
-REGISTER_TEST_COMMAND(fib_autotest, test_fib);
-REGISTER_TEST_COMMAND(fib_slow_autotest, test_slow_fib);
+REGISTER_FAST_TEST(fib_autotest, True, True, test_fib);
+REGISTER_PERF_TEST(fib_slow_autotest, test_slow_fib);
diff --git a/app/test/test_fib6.c b/app/test/test_fib6.c
index 15ad09178a..9b1c0ac8c7 100644
--- a/app/test/test_fib6.c
+++ b/app/test/test_fib6.c
@@ -424,5 +424,5 @@ test_slow_fib6(void)
 	return unit_test_suite_runner(&fib6_slow_tests);
 }
 
-REGISTER_TEST_COMMAND(fib6_autotest, test_fib6);
-REGISTER_TEST_COMMAND(fib6_slow_autotest, test_slow_fib6);
+REGISTER_FAST_TEST(fib6_autotest, True, True, test_fib6);
+REGISTER_PERF_TEST(fib6_slow_autotest, test_slow_fib6);
diff --git a/app/test/test_fib6_perf.c b/app/test/test_fib6_perf.c
index add20c2331..a7abc46af9 100644
--- a/app/test/test_fib6_perf.c
+++ b/app/test/test_fib6_perf.c
@@ -156,4 +156,4 @@ test_fib6_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(fib6_perf_autotest, test_fib6_perf);
+REGISTER_PERF_TEST(fib6_perf_autotest, test_fib6_perf);
diff --git a/app/test/test_fib_perf.c b/app/test/test_fib_perf.c
index b56293e64f..a9119c1bb0 100644
--- a/app/test/test_fib_perf.c
+++ b/app/test/test_fib_perf.c
@@ -409,4 +409,4 @@ test_fib_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(fib_perf_autotest, test_fib_perf);
+REGISTER_PERF_TEST(fib_perf_autotest, test_fib_perf);
diff --git a/app/test/test_func_reentrancy.c b/app/test/test_func_reentrancy.c
index ae9de6f93d..63b00009b5 100644
--- a/app/test/test_func_reentrancy.c
+++ b/app/test/test_func_reentrancy.c
@@ -507,4 +507,4 @@ test_func_reentrancy(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(func_reentrancy_autotest, test_func_reentrancy);
+REGISTER_FAST_TEST(func_reentrancy_autotest, False, True, test_func_reentrancy);
diff --git a/app/test/test_hash.c b/app/test/test_hash.c
index 3e45afaa67..e160b27b4c 100644
--- a/app/test/test_hash.c
+++ b/app/test/test_hash.c
@@ -2264,4 +2264,4 @@ test_hash(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(hash_autotest, test_hash);
+REGISTER_FAST_TEST(hash_autotest, True, True, test_hash);
diff --git a/app/test/test_hash_functions.c b/app/test/test_hash_functions.c
index 76d51b6e71..70820d1f19 100644
--- a/app/test/test_hash_functions.c
+++ b/app/test/test_hash_functions.c
@@ -290,4 +290,4 @@ test_hash_functions(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(hash_functions_autotest, test_hash_functions);
+REGISTER_PERF_TEST(hash_functions_autotest, test_hash_functions);
diff --git a/app/test/test_hash_multiwriter.c b/app/test/test_hash_multiwriter.c
index 0c5a8ca186..dd5ca677b9 100644
--- a/app/test/test_hash_multiwriter.c
+++ b/app/test/test_hash_multiwriter.c
@@ -287,4 +287,4 @@ test_hash_multiwriter_main(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(hash_multiwriter_autotest, test_hash_multiwriter_main);
+REGISTER_PERF_TEST(hash_multiwriter_autotest, test_hash_multiwriter_main);
diff --git a/app/test/test_hash_perf.c b/app/test/test_hash_perf.c
index 14a1283aba..d66b96e5ce 100644
--- a/app/test/test_hash_perf.c
+++ b/app/test/test_hash_perf.c
@@ -757,4 +757,4 @@ test_hash_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(hash_perf_autotest, test_hash_perf);
+REGISTER_PERF_TEST(hash_perf_autotest, test_hash_perf);
diff --git a/app/test/test_hash_readwrite.c b/app/test/test_hash_readwrite.c
index 6373e62d33..9c33b49878 100644
--- a/app/test/test_hash_readwrite.c
+++ b/app/test/test_hash_readwrite.c
@@ -760,5 +760,5 @@ test_hash_rw_func_main(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(hash_readwrite_func_autotest, test_hash_rw_func_main);
-REGISTER_TEST_COMMAND(hash_readwrite_perf_autotest, test_hash_rw_perf_main);
+REGISTER_FAST_TEST(hash_readwrite_func_autotest, False, True, test_hash_rw_func_main);
+REGISTER_PERF_TEST(hash_readwrite_perf_autotest, test_hash_rw_perf_main);
diff --git a/app/test/test_hash_readwrite_lf_perf.c b/app/test/test_hash_readwrite_lf_perf.c
index cf86046a2f..5d18850e19 100644
--- a/app/test/test_hash_readwrite_lf_perf.c
+++ b/app/test/test_hash_readwrite_lf_perf.c
@@ -1579,5 +1579,5 @@ test_hash_readwrite_lf_perf_main(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(hash_readwrite_lf_perf_autotest,
+REGISTER_PERF_TEST(hash_readwrite_lf_perf_autotest,
 	test_hash_readwrite_lf_perf_main);
diff --git a/app/test/test_interrupts.c b/app/test/test_interrupts.c
index b59ab42699..cf6b5a6a8e 100644
--- a/app/test/test_interrupts.c
+++ b/app/test/test_interrupts.c
@@ -594,4 +594,4 @@ test_interrupt(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(interrupt_autotest, test_interrupt);
+REGISTER_FAST_TEST(interrupt_autotest, True, True, test_interrupt);
diff --git a/app/test/test_ipfrag.c b/app/test/test_ipfrag.c
index 402ce361c1..264825634d 100644
--- a/app/test/test_ipfrag.c
+++ b/app/test/test_ipfrag.c
@@ -510,4 +510,4 @@ test_ipfrag(void)
 }
 
 
-REGISTER_TEST_COMMAND(ipfrag_autotest, test_ipfrag);
+REGISTER_FAST_TEST(ipfrag_autotest, False, True, test_ipfrag);
diff --git a/app/test/test_ipsec.c b/app/test/test_ipsec.c
index c2a52ec305..629580bc21 100644
--- a/app/test/test_ipsec.c
+++ b/app/test/test_ipsec.c
@@ -2532,4 +2532,4 @@ test_ipsec(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(ipsec_autotest, test_ipsec);
+REGISTER_FAST_TEST(ipsec_autotest, True, True, test_ipsec);
diff --git a/app/test/test_ipsec_perf.c b/app/test/test_ipsec_perf.c
index b221b7fc32..a32a2086e9 100644
--- a/app/test/test_ipsec_perf.c
+++ b/app/test/test_ipsec_perf.c
@@ -631,4 +631,4 @@ test_libipsec_perf(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(ipsec_perf_autotest, test_libipsec_perf);
+REGISTER_PERF_TEST(ipsec_perf_autotest, test_libipsec_perf);
diff --git a/app/test/test_kvargs.c b/app/test/test_kvargs.c
index b7b97a0dd9..d0f05a55c7 100644
--- a/app/test/test_kvargs.c
+++ b/app/test/test_kvargs.c
@@ -292,4 +292,4 @@ test_kvargs(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(kvargs_autotest, test_kvargs);
+REGISTER_FAST_TEST(kvargs_autotest, True, True, test_kvargs);
diff --git a/app/test/test_lcores.c b/app/test/test_lcores.c
index 2c945b0136..b86629c0db 100644
--- a/app/test/test_lcores.c
+++ b/app/test/test_lcores.c
@@ -412,4 +412,4 @@ test_lcores(void)
 	return TEST_SUCCESS;
 }
 
-REGISTER_TEST_COMMAND(lcores_autotest, test_lcores);
+REGISTER_FAST_TEST(lcores_autotest, True, True, test_lcores);
diff --git a/app/test/test_logs.c b/app/test/test_logs.c
index 8da8824bee..38052910e9 100644
--- a/app/test/test_logs.c
+++ b/app/test/test_logs.c
@@ -158,4 +158,4 @@ test_logs(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(logs_autotest, test_logs);
+REGISTER_FAST_TEST(logs_autotest, True, True, test_logs);
diff --git a/app/test/test_lpm.c b/app/test/test_lpm.c
index 37b460af3a..85a75ccec0 100644
--- a/app/test/test_lpm.c
+++ b/app/test/test_lpm.c
@@ -1584,4 +1584,4 @@ test_lpm(void)
 	return global_status;
 }
 
-REGISTER_TEST_COMMAND(lpm_autotest, test_lpm);
+REGISTER_FAST_TEST(lpm_autotest, True, True, test_lpm);
diff --git a/app/test/test_lpm6.c b/app/test/test_lpm6.c
index b6b6f8615e..4a9eddf276 100644
--- a/app/test/test_lpm6.c
+++ b/app/test/test_lpm6.c
@@ -1793,4 +1793,4 @@ test_lpm6(void)
 	return global_status;
 }
 
-REGISTER_TEST_COMMAND(lpm6_autotest, test_lpm6);
+REGISTER_FAST_TEST(lpm6_autotest, True, True, test_lpm6);
diff --git a/app/test/test_lpm6_perf.c b/app/test/test_lpm6_perf.c
index 5b684686a6..8a49f74c84 100644
--- a/app/test/test_lpm6_perf.c
+++ b/app/test/test_lpm6_perf.c
@@ -161,4 +161,4 @@ test_lpm6_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(lpm6_perf_autotest, test_lpm6_perf);
+REGISTER_PERF_TEST(lpm6_perf_autotest, test_lpm6_perf);
diff --git a/app/test/test_lpm_perf.c b/app/test/test_lpm_perf.c
index e72437ba38..15ff396dd0 100644
--- a/app/test/test_lpm_perf.c
+++ b/app/test/test_lpm_perf.c
@@ -760,4 +760,4 @@ test_lpm_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(lpm_perf_autotest, test_lpm_perf);
+REGISTER_PERF_TEST(lpm_perf_autotest, test_lpm_perf);
diff --git a/app/test/test_malloc.c b/app/test/test_malloc.c
index ff081dd931..6057623473 100644
--- a/app/test/test_malloc.c
+++ b/app/test/test_malloc.c
@@ -1091,4 +1091,4 @@ test_malloc(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(malloc_autotest, test_malloc);
+REGISTER_FAST_TEST(malloc_autotest, False, True, test_malloc);
diff --git a/app/test/test_malloc_perf.c b/app/test/test_malloc_perf.c
index 9bd1662981..a99bfd8531 100644
--- a/app/test/test_malloc_perf.c
+++ b/app/test/test_malloc_perf.c
@@ -171,4 +171,4 @@ test_malloc_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(malloc_perf_autotest, test_malloc_perf);
+REGISTER_PERF_TEST(malloc_perf_autotest, test_malloc_perf);
diff --git a/app/test/test_mbuf.c b/app/test/test_mbuf.c
index efac01806b..2eeed268af 100644
--- a/app/test/test_mbuf.c
+++ b/app/test/test_mbuf.c
@@ -2955,4 +2955,4 @@ test_mbuf(void)
 }
 #undef GOTO_FAIL
 
-REGISTER_TEST_COMMAND(mbuf_autotest, test_mbuf);
+REGISTER_FAST_TEST(mbuf_autotest, False, True, test_mbuf);
diff --git a/app/test/test_mcslock.c b/app/test/test_mcslock.c
index 52e45e7e2a..a3753a4397 100644
--- a/app/test/test_mcslock.c
+++ b/app/test/test_mcslock.c
@@ -241,4 +241,4 @@ test_mcslock(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(mcslock_autotest, test_mcslock);
+REGISTER_FAST_TEST(mcslock_autotest, False, True, test_mcslock);
diff --git a/app/test/test_member.c b/app/test/test_member.c
index 4a93f8bff4..e17a8f3190 100644
--- a/app/test/test_member.c
+++ b/app/test/test_member.c
@@ -996,4 +996,4 @@ test_member(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(member_autotest, test_member);
+REGISTER_FAST_TEST(member_autotest, True, True, test_member);
diff --git a/app/test/test_member_perf.c b/app/test/test_member_perf.c
index 2f79888fbd..db6b8a18ef 100644
--- a/app/test/test_member_perf.c
+++ b/app/test/test_member_perf.c
@@ -780,4 +780,4 @@ test_member_perf(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(member_perf_autotest, test_member_perf);
+REGISTER_PERF_TEST(member_perf_autotest, test_member_perf);
diff --git a/app/test/test_memcpy.c b/app/test/test_memcpy.c
index 1ab86f4967..c81efb85c2 100644
--- a/app/test/test_memcpy.c
+++ b/app/test/test_memcpy.c
@@ -129,4 +129,4 @@ test_memcpy(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(memcpy_autotest, test_memcpy);
+REGISTER_FAST_TEST(memcpy_autotest, True, True, test_memcpy);
diff --git a/app/test/test_memcpy_perf.c b/app/test/test_memcpy_perf.c
index 3727c160e6..5c05a84619 100644
--- a/app/test/test_memcpy_perf.c
+++ b/app/test/test_memcpy_perf.c
@@ -348,4 +348,4 @@ test_memcpy_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(memcpy_perf_autotest, test_memcpy_perf);
+REGISTER_PERF_TEST(memcpy_perf_autotest, test_memcpy_perf);
diff --git a/app/test/test_memory.c b/app/test/test_memory.c
index 440e5ef838..13ae9569c2 100644
--- a/app/test/test_memory.c
+++ b/app/test/test_memory.c
@@ -110,4 +110,4 @@ test_memory(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(memory_autotest, test_memory);
+REGISTER_FAST_TEST(memory_autotest, False, True, test_memory);
diff --git a/app/test/test_mempool.c b/app/test/test_mempool.c
index 8e493eda47..03047e5b14 100644
--- a/app/test/test_mempool.c
+++ b/app/test/test_mempool.c
@@ -1043,4 +1043,4 @@ test_mempool(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(mempool_autotest, test_mempool);
+REGISTER_FAST_TEST(mempool_autotest, False, True, test_mempool);
diff --git a/app/test/test_mempool_perf.c b/app/test/test_mempool_perf.c
index ce7c6241ab..96de347f04 100644
--- a/app/test/test_mempool_perf.c
+++ b/app/test/test_mempool_perf.c
@@ -437,4 +437,4 @@ test_mempool_perf(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(mempool_perf_autotest, test_mempool_perf);
+REGISTER_PERF_TEST(mempool_perf_autotest, test_mempool_perf);
diff --git a/app/test/test_memzone.c b/app/test/test_memzone.c
index f10f4fd9cd..85b5839034 100644
--- a/app/test/test_memzone.c
+++ b/app/test/test_memzone.c
@@ -1163,4 +1163,4 @@ test_memzone(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(memzone_autotest, test_memzone);
+REGISTER_FAST_TEST(memzone_autotest, False, True, test_memzone);
diff --git a/app/test/test_meter.c b/app/test/test_meter.c
index 15d5a4839b..3016681bbc 100644
--- a/app/test/test_meter.c
+++ b/app/test/test_meter.c
@@ -713,4 +713,4 @@ test_meter(void)
 
 }
 
-REGISTER_TEST_COMMAND(meter_autotest, test_meter);
+REGISTER_FAST_TEST(meter_autotest, True, True, test_meter);
diff --git a/app/test/test_mp_secondary.c b/app/test/test_mp_secondary.c
index ad47d578f2..502b0235a7 100644
--- a/app/test/test_mp_secondary.c
+++ b/app/test/test_mp_secondary.c
@@ -223,4 +223,4 @@ test_mp_secondary(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(multiprocess_autotest, test_mp_secondary);
+REGISTER_FAST_TEST(multiprocess_autotest, False, False, test_mp_secondary);
diff --git a/app/test/test_per_lcore.c b/app/test/test_per_lcore.c
index 129578d1a3..50011366a7 100644
--- a/app/test/test_per_lcore.c
+++ b/app/test/test_per_lcore.c
@@ -105,4 +105,4 @@ test_per_lcore(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(per_lcore_autotest, test_per_lcore);
+REGISTER_FAST_TEST(per_lcore_autotest, True, True, test_per_lcore);
diff --git a/app/test/test_pflock.c b/app/test/test_pflock.c
index 38da6bce27..07e4a68c2c 100644
--- a/app/test/test_pflock.c
+++ b/app/test/test_pflock.c
@@ -193,4 +193,4 @@ test_pflock(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(pflock_autotest, test_pflock);
+REGISTER_FAST_TEST(pflock_autotest, True, True, test_pflock);
diff --git a/app/test/test_pie.c b/app/test/test_pie.c
index a3c0f97c9d..5674602b69 100644
--- a/app/test/test_pie.c
+++ b/app/test/test_pie.c
@@ -1087,6 +1087,6 @@ test_pie_all(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(pie_autotest, test_pie);
-REGISTER_TEST_COMMAND(pie_perf, test_pie_perf);
-REGISTER_TEST_COMMAND(pie_all, test_pie_all);
+REGISTER_FAST_TEST(pie_autotest, True, True, test_pie);
+REGISTER_PERF_TEST(pie_perf, test_pie_perf);
+REGISTER_PERF_TEST(pie_all, test_pie_all);
diff --git a/app/test/test_pmd_perf.c b/app/test/test_pmd_perf.c
index 3ef590cb51..f6d97f21c9 100644
--- a/app/test/test_pmd_perf.c
+++ b/app/test/test_pmd_perf.c
@@ -899,4 +899,4 @@ test_set_rxtx_sc(cmdline_fixed_string_t type)
 	return -1;
 }
 
-REGISTER_TEST_COMMAND(pmd_perf_autotest, test_pmd_perf);
+REGISTER_PERF_TEST(pmd_perf_autotest, test_pmd_perf);
diff --git a/app/test/test_power.c b/app/test/test_power.c
index b7b5561348..d4f622ff2f 100644
--- a/app/test/test_power.c
+++ b/app/test/test_power.c
@@ -172,4 +172,4 @@ test_power(void)
 }
 #endif
 
-REGISTER_TEST_COMMAND(power_autotest, test_power);
+REGISTER_FAST_TEST(power_autotest, True, True, test_power);
diff --git a/app/test/test_power_cpufreq.c b/app/test/test_power_cpufreq.c
index 4d013cd7bb..f56053ac10 100644
--- a/app/test/test_power_cpufreq.c
+++ b/app/test/test_power_cpufreq.c
@@ -703,5 +703,5 @@ test_power_caps(void)
 
 #endif
 
-REGISTER_TEST_COMMAND(power_cpufreq_autotest, test_power_cpufreq);
+REGISTER_FAST_TEST(power_cpufreq_autotest, False, True, test_power_cpufreq);
 REGISTER_TEST_COMMAND(power_caps_autotest, test_power_caps);
diff --git a/app/test/test_power_intel_uncore.c b/app/test/test_power_intel_uncore.c
index 31163af84e..c8a8621e44 100644
--- a/app/test/test_power_intel_uncore.c
+++ b/app/test/test_power_intel_uncore.c
@@ -298,4 +298,4 @@ test_power_intel_uncore(void)
 }
 #endif
 
-REGISTER_TEST_COMMAND(power_intel_uncore_autotest, test_power_intel_uncore);
+REGISTER_FAST_TEST(power_intel_uncore_autotest, True, True, test_power_intel_uncore);
diff --git a/app/test/test_power_kvm_vm.c b/app/test/test_power_kvm_vm.c
index cc66b7a8a0..4919df8b62 100644
--- a/app/test/test_power_kvm_vm.c
+++ b/app/test/test_power_kvm_vm.c
@@ -299,4 +299,4 @@ test_power_kvm_vm(void)
 }
 #endif
 
-REGISTER_TEST_COMMAND(power_kvm_vm_autotest, test_power_kvm_vm);
+REGISTER_FAST_TEST(power_kvm_vm_autotest, False, True, test_power_kvm_vm);
diff --git a/app/test/test_prefetch.c b/app/test/test_prefetch.c
index 7b4a8e4144..a72ae80557 100644
--- a/app/test/test_prefetch.c
+++ b/app/test/test_prefetch.c
@@ -35,4 +35,4 @@ test_prefetch(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(prefetch_autotest, test_prefetch);
+REGISTER_FAST_TEST(prefetch_autotest, True, True, test_prefetch);
diff --git a/app/test/test_rand_perf.c b/app/test/test_rand_perf.c
index 26fb1d9a58..30204e12c0 100644
--- a/app/test/test_rand_perf.c
+++ b/app/test/test_rand_perf.c
@@ -96,4 +96,4 @@ test_rand_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(rand_perf_autotest, test_rand_perf);
+REGISTER_PERF_TEST(rand_perf_autotest, test_rand_perf);
diff --git a/app/test/test_rcu_qsbr.c b/app/test/test_rcu_qsbr.c
index 70404e89e6..16c24f58e7 100644
--- a/app/test/test_rcu_qsbr.c
+++ b/app/test/test_rcu_qsbr.c
@@ -1418,4 +1418,4 @@ test_rcu_qsbr_main(void)
 	return -1;
 }
 
-REGISTER_TEST_COMMAND(rcu_qsbr_autotest, test_rcu_qsbr_main);
+REGISTER_FAST_TEST(rcu_qsbr_autotest, True, True, test_rcu_qsbr_main);
diff --git a/app/test/test_rcu_qsbr_perf.c b/app/test/test_rcu_qsbr_perf.c
index b15e5cef88..ce88a7333c 100644
--- a/app/test/test_rcu_qsbr_perf.c
+++ b/app/test/test_rcu_qsbr_perf.c
@@ -690,4 +690,4 @@ test_rcu_qsbr_main(void)
 	return -1;
 }
 
-REGISTER_TEST_COMMAND(rcu_qsbr_perf_autotest, test_rcu_qsbr_main);
+REGISTER_PERF_TEST(rcu_qsbr_perf_autotest, test_rcu_qsbr_main);
diff --git a/app/test/test_reassembly_perf.c b/app/test/test_reassembly_perf.c
index c11b65291f..4b4929d777 100644
--- a/app/test/test_reassembly_perf.c
+++ b/app/test/test_reassembly_perf.c
@@ -1000,4 +1000,4 @@ test_reassembly_perf(void)
 	return TEST_SUCCESS;
 }
 
-REGISTER_TEST_COMMAND(reassembly_perf_autotest, test_reassembly_perf);
+REGISTER_PERF_TEST(reassembly_perf_autotest, test_reassembly_perf);
diff --git a/app/test/test_reciprocal_division.c b/app/test/test_reciprocal_division.c
index 8ea9b1d24d..fb52b2d5a1 100644
--- a/app/test/test_reciprocal_division.c
+++ b/app/test/test_reciprocal_division.c
@@ -164,4 +164,4 @@ test_reciprocal(void)
 	return result;
 }
 
-REGISTER_TEST_COMMAND(reciprocal_division, test_reciprocal);
+REGISTER_PERF_TEST(reciprocal_division, test_reciprocal);
diff --git a/app/test/test_reciprocal_division_perf.c b/app/test/test_reciprocal_division_perf.c
index 4f625873e5..cf96d46a22 100644
--- a/app/test/test_reciprocal_division_perf.c
+++ b/app/test/test_reciprocal_division_perf.c
@@ -205,4 +205,4 @@ test_reciprocal_division_perf(void)
 	return result;
 }
 
-REGISTER_TEST_COMMAND(reciprocal_division_perf, test_reciprocal_division_perf);
+REGISTER_PERF_TEST(reciprocal_division_perf, test_reciprocal_division_perf);
diff --git a/app/test/test_red.c b/app/test/test_red.c
index 84c292f8d8..aa7538d51a 100644
--- a/app/test/test_red.c
+++ b/app/test/test_red.c
@@ -1878,5 +1878,5 @@ test_red_all(void)
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
 REGISTER_TEST_COMMAND(red_autotest, test_red);
-REGISTER_TEST_COMMAND(red_perf, test_red_perf);
-REGISTER_TEST_COMMAND(red_all, test_red_all);
+REGISTER_PERF_TEST(red_perf, test_red_perf);
+REGISTER_PERF_TEST(red_all, test_red_all);
diff --git a/app/test/test_reorder.c b/app/test/test_reorder.c
index c188f6ce67..e66b4b05d4 100644
--- a/app/test/test_reorder.c
+++ b/app/test/test_reorder.c
@@ -548,4 +548,4 @@ test_reorder(void)
 }
 
 
-REGISTER_TEST_COMMAND(reorder_autotest, test_reorder);
+REGISTER_FAST_TEST(reorder_autotest, True, True, test_reorder);
diff --git a/app/test/test_rib.c b/app/test/test_rib.c
index 65b6856410..a8d60acc3a 100644
--- a/app/test/test_rib.c
+++ b/app/test/test_rib.c
@@ -363,5 +363,5 @@ test_slow_rib(void)
 	return unit_test_suite_runner(&rib_slow_tests);
 }
 
-REGISTER_TEST_COMMAND(rib_autotest, test_rib);
-REGISTER_TEST_COMMAND(rib_slow_autotest, test_slow_rib);
+REGISTER_FAST_TEST(rib_autotest, True, True, test_rib);
+REGISTER_PERF_TEST(rib_slow_autotest, test_slow_rib);
diff --git a/app/test/test_rib6.c b/app/test/test_rib6.c
index 336b779d2e..9bbd0f406d 100644
--- a/app/test/test_rib6.c
+++ b/app/test/test_rib6.c
@@ -367,5 +367,5 @@ test_slow_rib6(void)
 	return unit_test_suite_runner(&rib6_slow_tests);
 }
 
-REGISTER_TEST_COMMAND(rib6_autotest, test_rib6);
-REGISTER_TEST_COMMAND(rib6_slow_autotest, test_slow_rib6);
+REGISTER_FAST_TEST(rib6_autotest, True, True, test_rib6);
+REGISTER_PERF_TEST(rib6_slow_autotest, test_slow_rib6);
diff --git a/app/test/test_ring.c b/app/test/test_ring.c
index bde33ab4a1..d2accbabfa 100644
--- a/app/test/test_ring.c
+++ b/app/test/test_ring.c
@@ -1241,4 +1241,4 @@ test_ring(void)
 	return -1;
 }
 
-REGISTER_TEST_COMMAND(ring_autotest, test_ring);
+REGISTER_FAST_TEST(ring_autotest, True, True, test_ring);
diff --git a/app/test/test_ring_perf.c b/app/test/test_ring_perf.c
index 3972fd9db3..d7c5a4c30b 100644
--- a/app/test/test_ring_perf.c
+++ b/app/test/test_ring_perf.c
@@ -579,4 +579,4 @@ test_ring_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(ring_perf_autotest, test_ring_perf);
+REGISTER_PERF_TEST(ring_perf_autotest, test_ring_perf);
diff --git a/app/test/test_rwlock.c b/app/test/test_rwlock.c
index 4ae0bf8deb..e8086888fe 100644
--- a/app/test/test_rwlock.c
+++ b/app/test/test_rwlock.c
@@ -506,7 +506,7 @@ try_rwlock_test_rde_wro(void)
 	return process_try_lcore_stats();
 }
 
-REGISTER_TEST_COMMAND(rwlock_test1_autotest, rwlock_test1);
-REGISTER_TEST_COMMAND(rwlock_rda_autotest, try_rwlock_test_rda);
-REGISTER_TEST_COMMAND(rwlock_rds_wrm_autotest, try_rwlock_test_rds_wrm);
-REGISTER_TEST_COMMAND(rwlock_rde_wro_autotest, try_rwlock_test_rde_wro);
+REGISTER_FAST_TEST(rwlock_test1_autotest, True, True, rwlock_test1);
+REGISTER_FAST_TEST(rwlock_rda_autotest, True, True, try_rwlock_test_rda);
+REGISTER_FAST_TEST(rwlock_rds_wrm_autotest, True, True, try_rwlock_test_rds_wrm);
+REGISTER_FAST_TEST(rwlock_rde_wro_autotest, True, True, try_rwlock_test_rde_wro);
diff --git a/app/test/test_sched.c b/app/test/test_sched.c
index ddec572447..385ca45136 100644
--- a/app/test/test_sched.c
+++ b/app/test/test_sched.c
@@ -215,4 +215,4 @@ test_sched(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(sched_autotest, test_sched);
+REGISTER_FAST_TEST(sched_autotest, True, True, test_sched);
diff --git a/app/test/test_security.c b/app/test/test_security.c
index 4783cd0663..a3b962750a 100644
--- a/app/test/test_security.c
+++ b/app/test/test_security.c
@@ -2314,4 +2314,4 @@ test_security(void)
 	return unit_test_suite_runner(&security_testsuite);
 }
 
-REGISTER_TEST_COMMAND(security_autotest, test_security);
+REGISTER_FAST_TEST(security_autotest, False, True, test_security);
diff --git a/app/test/test_seqlock.c b/app/test/test_seqlock.c
index d26d2c010e..325af38070 100644
--- a/app/test/test_seqlock.c
+++ b/app/test/test_seqlock.c
@@ -187,4 +187,4 @@ test_seqlock(void)
 	return rc;
 }
 
-REGISTER_TEST_COMMAND(seqlock_autotest, test_seqlock);
+REGISTER_FAST_TEST(seqlock_autotest, True, True, test_seqlock);
diff --git a/app/test/test_service_cores.c b/app/test/test_service_cores.c
index 422d2a83e8..6c000417d2 100644
--- a/app/test/test_service_cores.c
+++ b/app/test/test_service_cores.c
@@ -1040,7 +1040,7 @@ test_service_common(void)
 	return unit_test_suite_runner(&service_tests);
 }
 
-REGISTER_TEST_COMMAND(service_autotest, test_service_common);
+REGISTER_FAST_TEST(service_autotest, True, True, test_service_common);
 
 static struct unit_test_suite service_perf_tests  = {
 	.suite_name = "service core performance test suite",
@@ -1062,4 +1062,4 @@ test_service_perf(void)
 	return unit_test_suite_runner(&service_perf_tests);
 }
 
-REGISTER_TEST_COMMAND(service_perf_autotest, test_service_perf);
+REGISTER_PERF_TEST(service_perf_autotest, test_service_perf);
diff --git a/app/test/test_spinlock.c b/app/test/test_spinlock.c
index 3f59372300..d655b606f7 100644
--- a/app/test/test_spinlock.c
+++ b/app/test/test_spinlock.c
@@ -302,4 +302,4 @@ test_spinlock(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(spinlock_autotest, test_spinlock);
+REGISTER_FAST_TEST(spinlock_autotest, True, True, test_spinlock);
diff --git a/app/test/test_stack.c b/app/test/test_stack.c
index bc38961433..93bba18469 100644
--- a/app/test/test_stack.c
+++ b/app/test/test_stack.c
@@ -379,5 +379,5 @@ test_lf_stack(void)
 #endif
 }
 
-REGISTER_TEST_COMMAND(stack_autotest, test_stack);
-REGISTER_TEST_COMMAND(stack_lf_autotest, test_lf_stack);
+REGISTER_FAST_TEST(stack_autotest, False, True, test_stack);
+REGISTER_FAST_TEST(stack_lf_autotest, False, True, test_lf_stack);
diff --git a/app/test/test_stack_perf.c b/app/test/test_stack_perf.c
index 1eae00a334..c5e1caa036 100644
--- a/app/test/test_stack_perf.c
+++ b/app/test/test_stack_perf.c
@@ -354,5 +354,5 @@ test_lf_stack_perf(void)
 #endif
 }
 
-REGISTER_TEST_COMMAND(stack_perf_autotest, test_stack_perf);
-REGISTER_TEST_COMMAND(stack_lf_perf_autotest, test_lf_stack_perf);
+REGISTER_PERF_TEST(stack_perf_autotest, test_stack_perf);
+REGISTER_PERF_TEST(stack_lf_perf_autotest, test_lf_stack_perf);
diff --git a/app/test/test_string_fns.c b/app/test/test_string_fns.c
index 5e105d2bb9..07e72bf680 100644
--- a/app/test/test_string_fns.c
+++ b/app/test/test_string_fns.c
@@ -182,4 +182,4 @@ test_string_fns(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(string_autotest, test_string_fns);
+REGISTER_FAST_TEST(string_autotest, True, True, test_string_fns);
diff --git a/app/test/test_tailq.c b/app/test/test_tailq.c
index 9520219b0a..5598bcd12d 100644
--- a/app/test/test_tailq.c
+++ b/app/test/test_tailq.c
@@ -125,4 +125,4 @@ test_tailq(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(tailq_autotest, test_tailq);
+REGISTER_FAST_TEST(tailq_autotest, True, True, test_tailq);
diff --git a/app/test/test_thash.c b/app/test/test_thash.c
index 53d9611e18..dd6468eb56 100644
--- a/app/test/test_thash.c
+++ b/app/test/test_thash.c
@@ -966,4 +966,4 @@ test_thash(void)
 	return unit_test_suite_runner(&thash_tests);
 }
 
-REGISTER_TEST_COMMAND(thash_autotest, test_thash);
+REGISTER_FAST_TEST(thash_autotest, True, True, test_thash);
diff --git a/app/test/test_thash_perf.c b/app/test/test_thash_perf.c
index 687582aa32..9dfd5d3c21 100644
--- a/app/test/test_thash_perf.c
+++ b/app/test/test_thash_perf.c
@@ -135,4 +135,4 @@ test_thash_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(thash_perf_autotest, test_thash_perf);
+REGISTER_PERF_TEST(thash_perf_autotest, test_thash_perf);
diff --git a/app/test/test_threads.c b/app/test/test_threads.c
index a4c4f651a4..6ed6b168d4 100644
--- a/app/test/test_threads.c
+++ b/app/test/test_threads.c
@@ -279,4 +279,4 @@ test_threads(void)
 	return unit_test_suite_runner(&threads_test_suite);
 }
 
-REGISTER_TEST_COMMAND(threads_autotest, test_threads);
+REGISTER_FAST_TEST(threads_autotest, True, True, test_threads);
diff --git a/app/test/test_ticketlock.c b/app/test/test_ticketlock.c
index 242c136478..25af697c0a 100644
--- a/app/test/test_ticketlock.c
+++ b/app/test/test_ticketlock.c
@@ -314,4 +314,4 @@ test_ticketlock(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(ticketlock_autotest, test_ticketlock);
+REGISTER_FAST_TEST(ticketlock_autotest, True, True, test_ticketlock);
diff --git a/app/test/test_timer.c b/app/test/test_timer.c
index 0c36dc9010..09608a1845 100644
--- a/app/test/test_timer.c
+++ b/app/test/test_timer.c
@@ -594,4 +594,4 @@ test_timer(void)
 	return TEST_SUCCESS;
 }
 
-REGISTER_TEST_COMMAND(timer_autotest, test_timer);
+REGISTER_FAST_TEST(timer_autotest, False, True, test_timer);
diff --git a/app/test/test_timer_perf.c b/app/test/test_timer_perf.c
index 0ede4b3e40..d2d74ebbc6 100644
--- a/app/test/test_timer_perf.c
+++ b/app/test/test_timer_perf.c
@@ -131,4 +131,4 @@ test_timer_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(timer_perf_autotest, test_timer_perf);
+REGISTER_PERF_TEST(timer_perf_autotest, test_timer_perf);
diff --git a/app/test/test_timer_racecond.c b/app/test/test_timer_racecond.c
index bb56ae8324..6f8b448ff8 100644
--- a/app/test/test_timer_racecond.c
+++ b/app/test/test_timer_racecond.c
@@ -172,4 +172,4 @@ test_timer_racecond(void)
 	return TEST_SUCCESS;
 }
 
-REGISTER_TEST_COMMAND(timer_racecond_autotest, test_timer_racecond);
+REGISTER_PERF_TEST(timer_racecond_autotest, test_timer_racecond);
diff --git a/app/test/test_trace.c b/app/test/test_trace.c
index ad4a394a29..96829caed5 100644
--- a/app/test/test_trace.c
+++ b/app/test/test_trace.c
@@ -250,4 +250,4 @@ test_trace(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(trace_autotest, test_trace);
+REGISTER_FAST_TEST(trace_autotest, True, True, test_trace);
diff --git a/app/test/test_trace_perf.c b/app/test/test_trace_perf.c
index 46ae7d8074..a6dd075722 100644
--- a/app/test/test_trace_perf.c
+++ b/app/test/test_trace_perf.c
@@ -179,4 +179,4 @@ test_trace_perf(void)
 	return TEST_SUCCESS;
 }
 
-REGISTER_TEST_COMMAND(trace_perf_autotest, test_trace_perf);
+REGISTER_PERF_TEST(trace_perf_autotest, test_trace_perf);
diff --git a/app/test/test_version.c b/app/test/test_version.c
index 1e1ff18656..21016237bc 100644
--- a/app/test/test_version.c
+++ b/app/test/test_version.c
@@ -25,4 +25,4 @@ test_version(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(version_autotest, test_version);
+REGISTER_FAST_TEST(version_autotest, True, True, test_version);
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v2 3/8] app/test: make telemetry data test buildable on windows
  2023-08-14 15:16 ` [PATCH v2 0/8] expand list of optional libraries Bruce Richardson
  2023-08-14 15:16   ` [PATCH v2 1/8] app/test: add new macros for various test types Bruce Richardson
  2023-08-14 15:16   ` [PATCH v2 2/8] app/test: tag tests with the test type Bruce Richardson
@ 2023-08-14 15:16   ` Bruce Richardson
  2023-08-14 15:16   ` [PATCH v2 4/8] app/test: build using per-file dependency matrix Bruce Richardson
                     ` (5 subsequent siblings)
  8 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-14 15:16 UTC (permalink / raw)
  To: dev; +Cc: ci, Bruce Richardson

To enable the building of the telemetry data tests file when building on
windows, we need to provide a stub implementation. That way, the test
file is buildable any time the library itself is built.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 app/test/test_telemetry_data.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/app/test/test_telemetry_data.c b/app/test/test_telemetry_data.c
index a960f44c1b..b2dc1d75db 100644
--- a/app/test/test_telemetry_data.c
+++ b/app/test/test_telemetry_data.c
@@ -2,11 +2,20 @@
  * Copyright 2020 Intel Corporation
  */
 
+#ifdef RTE_EXEC_ENV_WINDOWS
+#include "test.h"
+
+static int
+telemetry_data_autotest(void)
+{
+	return TEST_SKIPPED;
+}
+
+#else
+
 #include <string.h>
 #include <sys/socket.h>
-#ifndef RTE_EXEC_ENV_WINDOWS
 #include <sys/un.h>
-#endif
 #include <unistd.h>
 #include <limits.h>
 
@@ -604,5 +613,6 @@ telemetry_data_autotest(void)
 	close(sock);
 	return 0;
 }
+#endif /* windows/non-windows */
 
 REGISTER_TEST_COMMAND(telemetry_data_autotest, telemetry_data_autotest);
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v2 4/8] app/test: build using per-file dependency matrix
  2023-08-14 15:16 ` [PATCH v2 0/8] expand list of optional libraries Bruce Richardson
                     ` (2 preceding siblings ...)
  2023-08-14 15:16   ` [PATCH v2 3/8] app/test: make telemetry data test buildable on windows Bruce Richardson
@ 2023-08-14 15:16   ` Bruce Richardson
  2023-08-14 15:16   ` [PATCH v2 5/8] app/test: define unit tests suites based on test macros Bruce Richardson
                     ` (4 subsequent siblings)
  8 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-14 15:16 UTC (permalink / raw)
  To: dev; +Cc: ci, Bruce Richardson

Rather than using if-else constructs to selectively add or remove files
from the UT build, switch to a table-based approach where each file
lists out what libs or drivers it depends upon.

Initial version of this table was generated via analysis of the header
files included in each C file. The basic dependencies of the test binary
[cmdline, ring, mempool and mbuf] were then removed from the per-file
lists, as there is no point in checking them as the whole app will be
disabled if they are not present.

With the file list, the dependencies for the "utility" C-files are kept
separate, so that other tests which depend on the functions provided by
those files can have that dependency recorded properly. The basic
cryptodev tests also fall into this category as functions from the main
cryptodev test file are used by other crypto tests.

As well as the main table for internal dependencies, some test
files have separate external components too. A second, much smaller
table lists these dependencies.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 app/meson.build      |   8 +-
 app/test/meson.build | 818 ++++++++++++-------------------------------
 2 files changed, 231 insertions(+), 595 deletions(-)

diff --git a/app/meson.build b/app/meson.build
index 4fc1a83eba..0d8b618e7f 100644
--- a/app/meson.build
+++ b/app/meson.build
@@ -32,6 +32,11 @@ apps = [
         'test-security-perf',
 ]
 
+if get_option('tests')
+# build the auto test app if enabled.
+    apps += 'test'
+endif
+
 default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API']
 default_ldflags = []
 if get_option('default_library') == 'static' and not is_windows
@@ -106,6 +111,3 @@ foreach app:apps
             install_rpath: join_paths(get_option('prefix'), driver_install_path),
             install: true)
 endforeach
-
-# special case the autotests
-subdir('test')
diff --git a/app/test/meson.build b/app/test/meson.build
index 66897c14a3..75c2414308 100644
--- a/app/test/meson.build
+++ b/app/test/meson.build
@@ -1,446 +1,230 @@
 # SPDX-License-Identifier: BSD-3-Clause
-# Copyright(c) 2017 Intel Corporation
-
-if not get_option('tests')
-    subdir_done()
-endif
-
-test_sources = files(
-        'commands.c',
-        'packet_burst_generator.c',
-        'test.c',
-        'test_acl.c',
-        'test_alarm.c',
-        'test_atomic.c',
-        'test_barrier.c',
-        'test_bitops.c',
-        'test_bitmap.c',
-        'test_bpf.c',
-        'test_byteorder.c',
-        'test_cksum.c',
-        'test_cksum_perf.c',
-        'test_cmdline.c',
-        'test_cmdline_cirbuf.c',
-        'test_cmdline_etheraddr.c',
-        'test_cmdline_ipaddr.c',
-        'test_cmdline_lib.c',
-        'test_cmdline_num.c',
-        'test_cmdline_portlist.c',
-        'test_cmdline_string.c',
-        'test_common.c',
-        'test_cpuflags.c',
-        'test_crc.c',
-        'test_cryptodev.c',
-        'test_cryptodev_asym.c',
-        'test_cryptodev_blockcipher.c',
-        'test_cryptodev_crosscheck.c',
-        'test_cryptodev_security_ipsec.c',
-        'test_cryptodev_security_pdcp.c',
-        'test_cycles.c',
-        'test_debug.c',
-        'test_devargs.c',
-        'test_distributor.c',
-        'test_distributor_perf.c',
-        'test_dmadev.c',
-        'test_dmadev_api.c',
-        'test_eal_flags.c',
-        'test_eal_fs.c',
-        'test_efd.c',
-        'test_efd_perf.c',
-        'test_errno.c',
-        'test_ethdev_link.c',
-        'test_event_crypto_adapter.c',
-        'test_event_eth_rx_adapter.c',
-        'test_event_ring.c',
-        'test_event_timer_adapter.c',
-        'test_eventdev.c',
-        'test_external_mem.c',
-        'test_fbarray.c',
-        'test_fib.c',
-        'test_fib_perf.c',
-        'test_fib6.c',
-        'test_fib6_perf.c',
-        'test_func_reentrancy.c',
-        'test_hash.c',
-        'test_hash_functions.c',
-        'test_hash_multiwriter.c',
-        'test_hash_readwrite.c',
-        'test_hash_perf.c',
-        'test_hash_readwrite_lf_perf.c',
-        'test_interrupts.c',
-        'test_ipfrag.c',
-        'test_ipsec.c',
-        'test_ipsec_sad.c',
-        'test_ipsec_perf.c',
-        'test_kvargs.c',
-        'test_lcores.c',
-        'test_logs.c',
-        'test_lpm.c',
-        'test_lpm6.c',
-        'test_lpm6_perf.c',
-        'test_lpm_perf.c',
-        'test_malloc.c',
-        'test_malloc_perf.c',
-        'test_mbuf.c',
-        'test_member.c',
-        'test_member_perf.c',
-        'test_memcpy.c',
-        'test_memcpy_perf.c',
-        'test_memory.c',
-        'test_mempool.c',
-        'test_mempool_perf.c',
-        'test_memzone.c',
-        'test_meter.c',
-        'test_mcslock.c',
-        'test_mp_secondary.c',
-        'test_per_lcore.c',
-        'test_pflock.c',
-        'test_pmd_perf.c',
-        'test_power.c',
-        'test_power_cpufreq.c',
-        'test_power_kvm_vm.c',
-        'test_power_intel_uncore.c',
-        'test_prefetch.c',
-        'test_rand_perf.c',
-        'test_rawdev.c',
-        'test_rcu_qsbr.c',
-        'test_rcu_qsbr_perf.c',
-        'test_reassembly_perf.c',
-        'test_reciprocal_division.c',
-        'test_reciprocal_division_perf.c',
-        'test_red.c',
-        'test_pie.c',
-        'test_reorder.c',
-        'test_rib.c',
-        'test_rib6.c',
-        'test_ring.c',
-        'test_ring_mpmc_stress.c',
-        'test_ring_hts_stress.c',
-        'test_ring_mt_peek_stress.c',
-        'test_ring_mt_peek_stress_zc.c',
-        'test_ring_perf.c',
-        'test_ring_rts_stress.c',
-        'test_ring_st_peek_stress.c',
-        'test_ring_st_peek_stress_zc.c',
-        'test_ring_stress.c',
-        'test_rwlock.c',
-        'test_sched.c',
-        'test_security.c',
-        'test_security_inline_macsec.c',
-        'test_security_inline_proto.c',
-        'test_seqlock.c',
-        'test_service_cores.c',
-        'test_spinlock.c',
-        'test_stack.c',
-        'test_stack_perf.c',
-        'test_string_fns.c',
-        'test_tailq.c',
-        'test_thash.c',
-        'test_thash_perf.c',
-        'test_threads.c',
-        'test_timer.c',
-        'test_timer_perf.c',
-        'test_timer_racecond.c',
-        'test_timer_secondary.c',
-        'test_ticketlock.c',
-        'test_trace.c',
-        'test_trace_register.c',
-        'test_trace_perf.c',
-        'test_version.c',
-        'virtual_pmd.c',
-)
-
-test_deps = dpdk_libs_enabled
-# as well as libs, the pci and vdev bus drivers are needed for a lot of tests
-test_deps += ['bus_pci', 'bus_vdev']
-
-# Each test is marked with flags:
-# - the first flag indicates whether the test can run in no-huge mode,
-# - the second flag indicates whether the test can run with ASan enabled,
-fast_tests = [
-        ['acl_autotest', true, true],
-        ['atomic_autotest', false, true],
-        ['bitmap_autotest', true, true],
-        ['bpf_autotest', true, true],
-        ['bpf_convert_autotest', true, true],
-        ['bitops_autotest', true, true],
-        ['byteorder_autotest', true, true],
-        ['cksum_autotest', true, true],
-        ['cmdline_autotest', true, true],
-        ['common_autotest', true, true],
-        ['cpuflags_autotest', true, true],
-        ['debug_autotest', true, true],
-        ['devargs_autotest', true, true],
-        ['eal_flags_c_opt_autotest', false, false],
-        ['eal_flags_main_opt_autotest', false, false],
-        ['eal_flags_n_opt_autotest', false, false],
-        ['eal_flags_hpet_autotest', false, false],
-        ['eal_flags_no_huge_autotest', false, false],
-        ['eal_flags_a_opt_autotest', false, false],
-        ['eal_flags_b_opt_autotest', false, false],
-        ['eal_flags_vdev_opt_autotest', false, false],
-        ['eal_flags_r_opt_autotest', false, false],
-        ['eal_flags_mem_autotest', false, false],
-        ['eal_flags_file_prefix_autotest', false, false],
-        ['eal_flags_misc_autotest', false, false],
-        ['eal_fs_autotest', true, true],
-        ['errno_autotest', true, true],
-        ['ethdev_link_status', true, true],
-        ['event_ring_autotest', true, true],
-        ['fib_autotest', true, true],
-        ['fib6_autotest', true, true],
-        ['func_reentrancy_autotest', false, true],
-        ['hash_autotest', true, true],
-        ['interrupt_autotest', true, true],
-        ['ipfrag_autotest', false, true],
-        ['lcores_autotest', true, true],
-        ['logs_autotest', true, true],
-        ['lpm_autotest', true, true],
-        ['lpm6_autotest', true, true],
-        ['malloc_autotest', false, true],
-        ['mbuf_autotest', false, true],
-        ['mcslock_autotest', false, true],
-        ['memcpy_autotest', true, true],
-        ['memory_autotest', false, true],
-        ['mempool_autotest', false, true],
-        ['memzone_autotest', false, true],
-        ['meter_autotest', true, true],
-        ['multiprocess_autotest', false, false],
-        ['per_lcore_autotest', true, true],
-        ['pflock_autotest', true, true],
-        ['prefetch_autotest', true, true],
-        ['rcu_qsbr_autotest', true, true],
-        ['pie_autotest', true, true],
-        ['rib_autotest', true, true],
-        ['rib6_autotest', true, true],
-        ['ring_autotest', true, true],
-        ['rwlock_test1_autotest', true, true],
-        ['rwlock_rda_autotest', true, true],
-        ['rwlock_rds_wrm_autotest', true, true],
-        ['rwlock_rde_wro_autotest', true, true],
-        ['sched_autotest', true, true],
-        ['security_autotest', false, true],
-        ['seqlock_autotest', true, true],
-        ['spinlock_autotest', true, true],
-        ['stack_autotest', false, true],
-        ['stack_lf_autotest', false, true],
-        ['string_autotest', true, true],
-        ['tailq_autotest', true, true],
-        ['ticketlock_autotest', true, true],
-        ['timer_autotest', false, true],
-        ['user_delay_us', true, true],
-        ['version_autotest', true, true],
-        ['crc_autotest', true, true],
-        ['distributor_autotest', false, true],
-        ['eventdev_common_autotest', true, true],
-        ['fbarray_autotest', true, true],
-        ['hash_readwrite_func_autotest', false, true],
-        ['ipsec_autotest', true, true],
-        ['kvargs_autotest', true, true],
-        ['member_autotest', true, true],
-        ['power_cpufreq_autotest', false, true],
-        ['power_autotest', true, true],
-        ['power_kvm_vm_autotest', false, true],
-        ['power_intel_uncore_autotest', true, true],
-        ['reorder_autotest', true, true],
-        ['service_autotest', true, true],
-        ['thash_autotest', true, true],
-        ['threads_autotest', true, true],
-        ['trace_autotest', true, true],
-]
-
-# Tests known to have issues or which don't belong in other tests lists.
-extra_test_names = [
-        'alarm_autotest', # ee00af60170b ("test: remove strict timing requirements some tests")
-        'red_autotest', # https://bugs.dpdk.org/show_bug.cgi?id=826
-]
-
-perf_test_names = [
-        'ring_perf_autotest',
-        'malloc_perf_autotest',
-        'mempool_perf_autotest',
-        'memcpy_perf_autotest',
-        'hash_perf_autotest',
-        'timer_perf_autotest',
-        'reciprocal_division',
-        'reciprocal_division_perf',
-        'lpm_perf_autotest',
-        'rib_slow_autotest',
-        'fib_slow_autotest',
-        'fib_perf_autotest',
-        'red_all',
-        'pie_all',
-        'barrier_autotest',
-        'hash_multiwriter_autotest',
-        'timer_racecond_autotest',
-        'efd_autotest',
-        'hash_functions_autotest',
-        'member_perf_autotest',
-        'efd_perf_autotest',
-        'lpm6_perf_autotest',
-        'rib6_slow_autotest',
-        'fib6_slow_autotest',
-        'fib6_perf_autotest',
-        'rcu_qsbr_perf_autotest',
-        'red_perf',
-        'pie_perf',
-        'distributor_perf_autotest',
-        'pmd_perf_autotest',
-        'service_perf_autotest',
-        'stack_perf_autotest',
-        'stack_lf_perf_autotest',
-        'rand_perf_autotest',
-        'hash_readwrite_perf_autotest',
-        'hash_readwrite_lf_perf_autotest',
-        'trace_perf_autotest',
-        'ipsec_perf_autotest',
-        'thash_perf_autotest',
-        'reassembly_perf_autotest',
-]
-
-driver_test_names = [
-        'cryptodev_aesni_gcm_autotest',
-        'cryptodev_aesni_mb_autotest',
-        'cryptodev_chacha_poly_mb_autotest',
-        'cryptodev_cn10k_autotest',
-        'cryptodev_cn9k_autotest',
-        'cryptodev_cpu_aesni_mb_autotest',
-        'cryptodev_cpu_aesni_gcm_autotest',
-        'cryptodev_dpaa2_sec_autotest',
-        'cryptodev_dpaa_sec_autotest',
-        'cryptodev_null_autotest',
-        'cryptodev_openssl_autotest',
-        'cryptodev_qat_autotest',
-        'cryptodev_qat_asym_autotest',
-        'cryptodev_qat_raw_api_autotest',
-        'cryptodev_sw_armv8_autotest',
-        'cryptodev_sw_kasumi_autotest',
-        'cryptodev_sw_mvsam_autotest',
-        'cryptodev_sw_snow3g_autotest',
-        'cryptodev_sw_zuc_autotest',
-        'cryptodev_uadk_autotest',
-        'dmadev_autotest',
-]
-
-dump_test_names = []
-
-if not is_windows
-    driver_test_names += [
-            'cryptodev_openssl_asym_autotest',
-            'eventdev_selftest_octeontx',
-            'eventdev_selftest_sw',
-    ]
-
-    dump_test_names += [
-            'dump_struct_sizes',
-            'dump_mempool',
-            'dump_malloc_stats',
-            'dump_devargs',
-            'dump_log_types',
-            'dump_ring',
-            'dump_physmem',
-            'dump_memzone',
-    ]
-endif
-
-# The following linkages are an exception to allow running the
-# unit tests without requiring that the developer install the
-# DPDK libraries.  Explicit linkage of drivers (plugin libraries)
-# in applications should not be used.
-if dpdk_conf.has('RTE_MEMPOOL_RING')
-    test_deps += 'mempool_ring'
-endif
-if dpdk_conf.has('RTE_MEMPOOL_STACK')
-    test_deps += 'mempool_stack'
-endif
-if dpdk_conf.has('RTE_EVENT_SKELETON')
-    test_deps += 'event_skeleton'
-endif
-
-if dpdk_conf.has('RTE_LIB_GRAPH')
-    test_sources += 'test_graph.c'
-    fast_tests += [['graph_autotest', true, true]]
-    fast_tests += [['node_list_dump', true, true]]
-    test_sources += 'test_graph_perf.c'
-    perf_test_names += 'graph_perf_autotest'
-endif
-if dpdk_conf.has('RTE_LIB_METRICS')
-    test_sources += ['test_metrics.c']
-    fast_tests += [['metrics_autotest', true, true]]
-endif
-if not is_windows and dpdk_conf.has('RTE_LIB_TELEMETRY')
-    test_sources += ['test_telemetry_json.c', 'test_telemetry_data.c']
-    fast_tests += [['telemetry_json_autotest', true, true]]
-    fast_tests += [['telemetry_data_autotest', true, true]]
-endif
-if dpdk_conf.has('RTE_LIB_PIPELINE')
-# pipeline lib depends on port and table libs, so those must be present
-# if pipeline library is.
-    test_sources += [
-            'test_table.c',
-            'test_table_acl.c',
-            'test_table_combined.c',
-            'test_table_pipeline.c',
-            'test_table_ports.c',
-            'test_table_tables.c',
-    ]
-    fast_tests += [['table_autotest', true, true]]
-endif
-
-# The following linkages of drivers are required because
-# they are used via a driver-specific API.
-if dpdk_conf.has('RTE_NET_BOND')
-    test_deps += 'net_bond'
-    test_sources += ['test_link_bonding.c', 'test_link_bonding_rssconf.c']
-    driver_test_names += ['link_bonding_autotest', 'link_bonding_rssconf_autotest']
-    if dpdk_conf.has('RTE_NET_RING')
-        test_sources += 'test_link_bonding_mode4.c'
-        driver_test_names += 'link_bonding_mode4_autotest'
-    endif
-endif
-if dpdk_conf.has('RTE_LIB_EVENTDEV') and dpdk_conf.has('RTE_NET_RING')
-    test_deps += 'net_ring'
-    test_sources += 'test_pmd_ring_perf.c'
-    test_sources += 'test_pmd_ring.c'
-    test_sources += 'test_event_eth_tx_adapter.c'
-    test_sources += 'sample_packet_forward.c'
-    fast_tests += [['ring_pmd_autotest', true, true]]
-    perf_test_names += 'ring_pmd_perf_autotest'
-    fast_tests += [['event_eth_tx_adapter_autotest', false, true]]
-    if dpdk_conf.has('RTE_LIB_BITRATESTATS')
-        test_sources += 'test_bitratestats.c'
-        fast_tests += [['bitratestats_autotest', true, true]]
-    endif
-    if dpdk_conf.has('RTE_LIB_LATENCYSTATS')
-        test_sources += 'test_latencystats.c'
-        fast_tests += [['latencystats_autotest', true, true]]
-    endif
-    if dpdk_conf.has('RTE_LIB_PDUMP')
-        test_sources += 'test_pdump.c'
-        fast_tests += [['pdump_autotest', true, false]]
-    endif
-endif
-if dpdk_conf.has('RTE_NET_NULL')
-    test_deps += 'net_null'
-    test_sources += 'test_vdev.c'
-    fast_tests += [['vdev_autotest', true, true]]
-endif
-if dpdk_conf.has('RTE_RAW_SKELETON')
-    test_deps += 'raw_skeleton'
-    fast_tests += [['rawdev_autotest', true, true]]
-endif
-
-if dpdk_conf.has('RTE_HAS_LIBPCAP')
-    ext_deps += pcap_dep
-    if dpdk_conf.has('RTE_LIB_PCAPNG')
-        test_sources += 'test_pcapng.c'
+# Copyright(c) 2017-2023 Intel Corporation
+
+# the main test files [test.c and commands.c] relies on these libraries
+deps += ['cmdline', 'ring', 'mempool', 'mbuf']
+sources += files('commands.c', 'test.c')
+
+# some other utility C files, providing functions used by various tests
+# so we need to include these deps in the dependency list for the files using those fns.
+packet_burst_generator_deps = ['net']
+sample_packet_forward_deps = ['net_ring', 'ethdev', 'bus_vdev']
+virtual_pmd_deps = ['ethdev', 'net', 'bus_pci']
+# test_cryptodev has material that other crypto tests need
+test_cryptodev_deps = ['bus_vdev', 'net', 'cryptodev', 'crypto_scheduler', 'security']
+
+source_file_deps = {
+    # The C files providing functionality to other test cases
+    'packet_burst_generator.c': packet_burst_generator_deps,
+#    'resource.c': [],          # unused currently.
+    'sample_packet_forward.c': sample_packet_forward_deps,
+    'virtual_pmd.c': virtual_pmd_deps,
+
+    # the various test_*.c files
+    'test_acl.c': ['net', 'acl'],
+    'test_alarm.c': [],
+    'test_atomic.c': ['hash'],
+    'test_barrier.c': [],
+    'test_bitmap.c': [],
+    'test_bitops.c': [],
+    'test_bitratestats.c': ['metrics', 'bitratestats', 'ethdev'] + sample_packet_forward_deps,
+    'test_bpf.c': ['bpf', 'net'],
+    'test_byteorder.c': [],
+#    'test_cfgfile.c': ['cfgfile'],
+    'test_cksum.c': ['net'],
+    'test_cksum_perf.c': ['net'],
+    'test_cmdline.c': [],
+    'test_cmdline_cirbuf.c': [],
+    'test_cmdline_etheraddr.c': ['net'],
+    'test_cmdline_ipaddr.c': [],
+    'test_cmdline_lib.c': [],
+    'test_cmdline_num.c': [],
+    'test_cmdline_portlist.c': [],
+    'test_cmdline_string.c': [],
+    'test_common.c': [],
+    'test_compressdev.c': ['compressdev'],
+    'test_cpuflags.c': [],
+    'test_crc.c': ['net'],
+    'test_cryptodev.c': test_cryptodev_deps,
+    'test_cryptodev_asym.c': ['bus_vdev'] + test_cryptodev_deps,
+    'test_cryptodev_blockcipher.c': test_cryptodev_deps,
+    'test_cryptodev_crosscheck.c': test_cryptodev_deps,
+    'test_cryptodev_security_ipsec.c': test_cryptodev_deps,
+    'test_cryptodev_security_pdcp.c': test_cryptodev_deps,
+    'test_cycles.c': [],
+    'test_debug.c': [],
+    'test_devargs.c': ['kvargs'],
+    'test_distributor.c': ['distributor'],
+    'test_distributor_perf.c': ['distributor'],
+    'test_dmadev.c': ['dmadev', 'bus_vdev'],
+    'test_dmadev_api.c': ['dmadev'],
+    'test_eal_flags.c': [],
+    'test_eal_fs.c': [],
+    'test_efd.c': ['efd', 'net'],
+    'test_efd_perf.c': ['efd', 'hash'],
+    'test_errno.c': [],
+    'test_ethdev_link.c': ['ethdev'],
+    'test_event_crypto_adapter.c': ['cryptodev', 'eventdev', 'bus_vdev'],
+    'test_event_eth_rx_adapter.c': ['ethdev', 'eventdev', 'bus_vdev'],
+    'test_event_eth_tx_adapter.c': ['bus_vdev', 'ethdev', 'net_ring', 'eventdev'],
+    'test_event_ring.c': ['eventdev'],
+    'test_event_timer_adapter.c': ['ethdev', 'eventdev', 'bus_vdev'],
+    'test_eventdev.c': ['eventdev', 'bus_vdev'],
+    'test_external_mem.c': [],
+    'test_fbarray.c': [],
+    'test_fib.c': ['net', 'fib'],
+    'test_fib6.c': ['rib', 'fib'],
+    'test_fib6_perf.c': ['fib'],
+    'test_fib_perf.c': ['net', 'fib'],
+    'test_flow_classify.c': ['net', 'acl', 'table', 'ethdev', 'flow_classify'],
+    'test_func_reentrancy.c': ['hash', 'lpm'],
+    'test_graph.c': ['graph'],
+    'test_graph_perf.c': ['graph'],
+    'test_hash.c': ['net', 'hash'],
+    'test_hash_functions.c': ['hash'],
+    'test_hash_multiwriter.c': ['hash'],
+    'test_hash_perf.c': ['hash'],
+    'test_hash_readwrite.c': ['hash'],
+    'test_hash_readwrite_lf_perf.c': ['hash'],
+    'test_interrupts.c': [],
+    'test_ipfrag.c': ['net', 'ip_frag'],
+    'test_ipsec.c': ['bus_vdev', 'net', 'cryptodev', 'ipsec', 'security'],
+    'test_ipsec_perf.c': ['net', 'ipsec'],
+    'test_ipsec_sad.c': ['ipsec'],
+    'test_kvargs.c': ['kvargs'],
+    'test_latencystats.c': ['ethdev', 'latencystats', 'metrics'] + sample_packet_forward_deps,
+    'test_lcores.c': [],
+    'test_link_bonding.c': ['ethdev', 'net_bond',
+        'net'] + packet_burst_generator_deps + virtual_pmd_deps,
+    'test_link_bonding_mode4.c': ['ethdev', 'net_ring', 'net_bond',
+        'net'] + packet_burst_generator_deps,
+    'test_link_bonding_rssconf.c': ['ethdev', 'bus_vdev', 'net_bond'],
+    'test_logs.c': [],
+    'test_lpm.c': ['net', 'lpm'],
+    'test_lpm6.c': ['lpm'],
+    'test_lpm6_perf.c': ['lpm'],
+    'test_lpm_perf.c': ['net', 'lpm'],
+    'test_malloc.c': [],
+    'test_malloc_perf.c': [],
+    'test_mbuf.c': ['net'],
+    'test_mcslock.c': [],
+    'test_member.c': ['member', 'net'],
+    'test_member_perf.c': ['hash', 'member'],
+    'test_memcpy.c': [],
+    'test_memcpy_perf.c': [],
+    'test_memory.c': [],
+    'test_mempool.c': [],
+    'test_mempool_perf.c': [],
+    'test_memzone.c': [],
+    'test_meter.c': ['meter'],
+    'test_metrics.c': ['metrics'],
+    'test_mp_secondary.c': ['hash', 'lpm'],
+    'test_pcapng.c': ['ethdev', 'net', 'pcapng'],
+    'test_pdcp.c': ['eventdev', 'pdcp', 'net', 'timer', 'security'],
+    'test_pdump.c': ['pdump'] + sample_packet_forward_deps,
+    'test_per_lcore.c': [],
+    'test_pflock.c': [],
+    'test_pie.c': ['sched'],
+    'test_pmd_perf.c': ['ethdev', 'net'] + packet_burst_generator_deps,
+    'test_pmd_ring.c': ['net_ring', 'ethdev', 'bus_vdev'],
+    'test_pmd_ring_perf.c': ['ethdev', 'net_ring', 'bus_vdev'],
+    'test_power.c': ['power'],
+    'test_power_cpufreq.c': ['power'],
+    'test_power_intel_uncore.c': ['power'],
+    'test_power_kvm_vm.c': ['power'],
+    'test_prefetch.c': [],
+    'test_rand_perf.c': [],
+    'test_rawdev.c': ['rawdev', 'bus_vdev'],
+    'test_rcu_qsbr.c': ['rcu', 'hash'],
+    'test_rcu_qsbr_perf.c': ['rcu', 'hash'],
+    'test_reassembly_perf.c': ['net', 'ip_frag'],
+    'test_reciprocal_division.c': [],
+    'test_reciprocal_division_perf.c': [],
+    'test_red.c': ['sched'],
+    'test_reorder.c': ['reorder'],
+#    'test_resource.c': [],
+    'test_rib.c': ['net', 'rib'],
+    'test_rib6.c': ['net', 'rib'],
+    'test_ring.c': [],
+    'test_ring_hts_stress.c': [],
+    'test_ring_mpmc_stress.c': [],
+    'test_ring_mt_peek_stress.c': [],
+    'test_ring_mt_peek_stress_zc.c': [],
+    'test_ring_perf.c': [],
+    'test_ring_rts_stress.c': [],
+    'test_ring_st_peek_stress.c': [],
+    'test_ring_st_peek_stress_zc.c': [],
+    'test_ring_stress.c': [],
+    'test_rwlock.c': [],
+    'test_sched.c': ['net', 'sched'],
+    'test_security.c': ['net', 'security'],
+    'test_security_inline_macsec.c': ['ethdev', 'security'],
+    'test_security_inline_proto.c': ['ethdev', 'security', 'eventdev'] + test_cryptodev_deps,
+    'test_seqlock.c': [],
+    'test_service_cores.c': [],
+    'test_spinlock.c': [],
+    'test_stack.c': ['stack'],
+    'test_stack_perf.c': ['stack'],
+    'test_string_fns.c': [],
+    'test_table.c': ['table', 'pipeline', 'port'],
+    'test_table_acl.c': ['net', 'table', 'pipeline', 'port'],
+    'test_table_combined.c': ['table', 'pipeline', 'port'],
+    'test_table_pipeline.c': ['pipeline', 'table', 'port'],
+    'test_table_ports.c': ['table', 'pipeline', 'port'],
+    'test_table_tables.c': ['table', 'pipeline', 'port'],
+    'test_tailq.c': [],
+    'test_telemetry_data.c': ['telemetry'],
+    'test_telemetry_json.c': ['telemetry'],
+    'test_thash.c': ['net', 'hash'],
+    'test_thash_perf.c': ['hash'],
+    'test_threads.c': [],
+    'test_ticketlock.c': [],
+    'test_timer.c': ['timer'],
+    'test_timer_perf.c': ['timer'],
+    'test_timer_racecond.c': ['timer'],
+    'test_timer_secondary.c': ['timer'],
+    'test_trace.c': [],
+    'test_trace_perf.c': [],
+    'test_trace_register.c': [],
+    'test_vdev.c': ['kvargs', 'bus_vdev'],
+    'test_version.c': [],
+}
+
+source_file_ext_deps = {
+    'test_compressdev.c': ['zlib'],
+    'test_pcapng.c': ['pcap'],
+}
+
+def_lib = get_option('default_library')
+foreach f, f_deps : source_file_deps
+    has_deps = true
+    foreach d : f_deps
+        if not is_variable(def_lib + '_rte_' + d)
+            has_deps = false
+            break
+        else
+            # technically we might not need this dep, but adding it is harmless
+            if d not in deps
+                deps += d
+            endif
+        endif
+    endforeach
+    # check for any external dependencies for this file
+    if source_file_ext_deps.has_key(f)
+        foreach d: source_file_ext_deps.get(f)
+            dep = dependency(d, required: false, method: 'pkg-config')
+            if not dep.found()
+                message('Skipping test file @0@ due to missing external dependency @1@'.format(f, d))
+                has_deps = false
+            else
+                ext_deps += dep
+            endif
+        endforeach
+    endif
+    if has_deps
+        sources += files(f)
     endif
-endif
-
-if dpdk_conf.has('RTE_LIB_PDCP')
-    test_sources += 'test_pdcp.c'
-    fast_tests += [['pdcp_autotest', false, true]]
-endif
+endforeach
 
 if cc.has_argument('-Wno-format-truncation')
     cflags += '-Wno-format-truncation'
@@ -450,154 +234,4 @@ endif
 cflags += '-fno-strict-aliasing'
 
 # Enable using internal APIs in unit tests
-cflags += ['-DALLOW_INTERNAL_API']
-
-test_dep_objs = []
-if dpdk_conf.has('RTE_LIB_COMPRESSDEV')
-    compress_test_dep = dependency('zlib', required: false, method: 'pkg-config')
-    if compress_test_dep.found()
-        test_dep_objs += compress_test_dep
-        test_sources += 'test_compressdev.c'
-        fast_tests += [['compressdev_autotest', false, true]]
-    endif
-endif
-
-if dpdk_conf.has('RTE_CRYPTO_SCHEDULER')
-    driver_test_names += 'cryptodev_scheduler_autotest'
-    test_deps += 'crypto_scheduler'
-endif
-
-foreach d:test_deps
-    def_lib = get_option('default_library')
-    test_dep_objs += get_variable(def_lib + '_rte_' + d)
-endforeach
-
-link_libs = []
-if get_option('default_library') == 'static'
-    link_libs = dpdk_static_libraries + dpdk_drivers
-endif
-
-dpdk_test = executable('dpdk-test',
-        test_sources,
-        link_whole: link_libs,
-        dependencies: test_dep_objs + ext_deps,
-        c_args: cflags,
-        install_rpath: join_paths(get_option('prefix'),
-             driver_install_path),
-        install: true)
-
-has_hugepage = run_command(py3, files('has_hugepage.py'), check: true).stdout().strip() != '0'
-message('hugepage availability: @0@'.format(has_hugepage))
-
-# some perf tests (eg: memcpy perf autotest)take very long
-# to complete, so timeout to 10 minutes
-timeout_seconds = 600
-timeout_seconds_fast = 10
-
-test_no_huge_args = ['--no-huge', '-m', '2048']
-
-foreach arg : fast_tests
-    test_args = []
-    run_test = true
-    if not has_hugepage
-        if arg[1]
-            test_args += test_no_huge_args
-        else
-            run_test = false
-        endif
-    endif
-
-    if get_option('b_sanitize') == 'address' or get_option('b_sanitize') == 'address,undefined'
-        if not arg[2]
-            run_test = false
-        endif
-    endif
-
-    if (get_option('default_library') == 'shared' and
-        arg[0] == 'event_eth_tx_adapter_autotest')
-        test_args += ['-d', dpdk_drivers_build_dir]
-    endif
-    if is_linux
-        test_args += ['--file-prefix=@0@'.format(arg[0])]
-    endif
-
-    if run_test
-        test(arg[0], dpdk_test,
-                env : ['DPDK_TEST=' + arg[0]],
-                args : test_args,
-                timeout : timeout_seconds_fast,
-                is_parallel : false,
-                suite : 'fast-tests')
-        if not is_windows and arg[0] == 'trace_autotest'
-            test_args += ['--trace=.*']
-            test_args += ['--trace-dir=@0@'.format(meson.current_build_dir())]
-            test(arg[0] + '_with_traces', dpdk_test,
-                    env : ['DPDK_TEST=' + arg[0]],
-                    args : test_args,
-                    timeout : timeout_seconds_fast,
-                    is_parallel : false,
-                    suite : 'fast-tests')
-        endif
-    endif
-endforeach
-
-if not is_windows and dpdk_conf.has('RTE_LIB_TELEMETRY')
-    test_args = [dpdk_test]
-    test_args += test_no_huge_args
-    if get_option('default_library') == 'shared'
-        test_args += ['-d', dpdk_drivers_build_dir]
-    endif
-    if dpdk_conf.has('RTE_CRYPTO_NULL')
-        test_args += ['--vdev=crypto_null0']
-    endif
-    if dpdk_conf.has('RTE_DMA_SKELETON')
-        test_args += ['--vdev=dma_skeleton0']
-    endif
-    if dpdk_conf.has('RTE_EVENT_SKELETON')
-        test_args += ['--vdev=event_skeleton0']
-    endif
-    if dpdk_conf.has('RTE_NET_NULL')
-        test_args += ['--vdev=net_null0']
-    endif
-    if dpdk_conf.has('RTE_RAW_SKELETON')
-        test_args += ['--vdev=rawdev_skeleton0']
-    endif
-    test_args += ['-a', '0000:00:00.0']
-    test('telemetry_all', find_program('test_telemetry.sh'),
-            args: test_args,
-            timeout : timeout_seconds_fast,
-            is_parallel : false,
-            suite : 'fast-tests')
-endif
-
-foreach arg : perf_test_names
-    test(arg, dpdk_test,
-            env : ['DPDK_TEST=' + arg],
-            timeout : timeout_seconds,
-            is_parallel : false,
-            suite : 'perf-tests')
-endforeach
-
-foreach arg : driver_test_names
-    test(arg, dpdk_test,
-            env : ['DPDK_TEST=' + arg],
-            timeout : timeout_seconds,
-            is_parallel : false,
-            suite : 'driver-tests')
-endforeach
-
-foreach arg : dump_test_names
-    test(arg, dpdk_test,
-            env : ['DPDK_TEST=' + arg],
-            timeout : timeout_seconds,
-            is_parallel : false,
-            suite : 'debug-tests')
-endforeach
-
-foreach arg : extra_test_names
-    test(arg, dpdk_test,
-            env : ['DPDK_TEST=' + arg],
-            timeout : timeout_seconds,
-            is_parallel : false,
-            suite : 'extra-tests')
-endforeach
+cflags += '-DALLOW_INTERNAL_API'
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v2 5/8] app/test: define unit tests suites based on test macros
  2023-08-14 15:16 ` [PATCH v2 0/8] expand list of optional libraries Bruce Richardson
                     ` (3 preceding siblings ...)
  2023-08-14 15:16   ` [PATCH v2 4/8] app/test: build using per-file dependency matrix Bruce Richardson
@ 2023-08-14 15:16   ` Bruce Richardson
  2023-08-14 15:16   ` [PATCH v2 6/8] examples/l3fwd: make eventdev an optional dependency Bruce Richardson
                     ` (3 subsequent siblings)
  8 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-14 15:16 UTC (permalink / raw)
  To: dev; +Cc: ci, Bruce Richardson

Rather than having the test suites listed out in the meson.build files
and having to have them enabled/disabled selectively based on what libs
are being built, pull the tests to run from the source files which were
added to the build.

Most test suites require no additional info other than the list of test
names in the suite. However the fast-test are special that they have
additional parameters associated with them. This requires some
additional work in the test extraction script and in the meson.build
file for processing the output.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 app/meson.build                               |  7 +-
 app/test/suites/meson.build                   | 76 +++++++++++++++++++
 buildtools/get-test-suites.py                 | 33 ++++++++
 .../has-hugepages.py                          |  0
 buildtools/meson.build                        |  2 +
 5 files changed, 117 insertions(+), 1 deletion(-)
 create mode 100644 app/test/suites/meson.build
 create mode 100644 buildtools/get-test-suites.py
 rename app/test/has_hugepage.py => buildtools/has-hugepages.py (100%)

diff --git a/app/meson.build b/app/meson.build
index 0d8b618e7f..c14dc80892 100644
--- a/app/meson.build
+++ b/app/meson.build
@@ -101,7 +101,7 @@ foreach app:apps
         link_libs = dpdk_static_libraries + dpdk_drivers
     endif
 
-    executable('dpdk-' + name,
+    exec = executable('dpdk-' + name,
             sources,
             c_args: cflags,
             link_args: ldflags,
@@ -110,4 +110,9 @@ foreach app:apps
             include_directories: includes,
             install_rpath: join_paths(get_option('prefix'), driver_install_path),
             install: true)
+    if name == 'test'
+        dpdk_test = exec
+        autotest_sources = sources
+        subdir('test/suites')  # define the pre-canned test suites
+    endif
 endforeach
diff --git a/app/test/suites/meson.build b/app/test/suites/meson.build
new file mode 100644
index 0000000000..df0c66ff0d
--- /dev/null
+++ b/app/test/suites/meson.build
@@ -0,0 +1,76 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2023 Intel Corporation
+
+# some perf tests (eg: memcpy perf autotest)take very long
+# to complete, so timeout to 10 minutes
+timeout_seconds = 600
+timeout_seconds_fast = 10
+
+test_no_huge_args = ['--no-huge', '-m', '2048']
+has_hugepage = run_command(has_hugepages_cmd, check: true).stdout().strip() != '0'
+message('hugepage availability: @0@'.format(has_hugepage))
+
+# process source files to determine the different unit test suites
+# - fast_tests
+# - perf_tests
+# - driver_tests
+test_suites = run_command(get_test_suites_cmd, autotest_sources,
+         check: true).stdout().strip().split()
+foreach suite:test_suites
+    # simple cases - tests without parameters or special handling
+    suite = suite.split('=')
+    suite_name = suite[0]
+    suite_tests = suite[1].split(',')
+    if suite_name != 'fast-tests'
+        # simple cases - tests without parameters or special handling
+        foreach t: suite_tests
+            test(t, dpdk_test,
+                    args: ['--', t],
+                    timeout: timeout_seconds,
+                    is_parallel: false,
+                    suite: suite_name)
+        endforeach
+    else
+    # special fast-test handling here
+        foreach t: suite_tests
+            params = t.split(':')
+            test_name = params[0]
+            nohuge = params[1] == 'true'
+            asan = params[2] == 'true'
+
+            test_args = []
+            if nohuge
+                test_args += test_no_huge_args
+            elif not has_hugepage
+                continue  #skip this tests
+            endif
+            if not asan and (get_option('b_sanitize') == 'address'
+                    or get_option('b_sanitize') == 'address,undefined')
+                continue  # skip this test
+            endif
+
+            if (get_option('default_library') == 'shared'
+                    and test_name == 'event_eth_tx_adapter_autotest')
+                test_args += ['-d', dpdk_drivers_build_dir]
+            endif
+            if is_linux
+                test_args += ['--file-prefix=@0@'.format(arg[0])]
+            endif
+
+            test(test_name, dpdk_test,
+                args : test_args + ['--', test_name],
+                timeout : timeout_seconds_fast,
+                is_parallel : false,
+                suite : 'fast-tests')
+            if not is_windows and test_name == 'trace_autotest'
+                test_args += ['--trace=.*']
+                test_args += ['--trace-dir=@0@'.format(meson.current_build_dir())]
+                test(test_name + '_with_traces', dpdk_test,
+                    args : test_args + ['--', test_name],
+                    timeout : timeout_seconds_fast,
+                    is_parallel : false,
+                    suite : 'fast-tests')
+            endif
+        endforeach
+    endif
+endforeach
diff --git a/buildtools/get-test-suites.py b/buildtools/get-test-suites.py
new file mode 100644
index 0000000000..95a9cad4c8
--- /dev/null
+++ b/buildtools/get-test-suites.py
@@ -0,0 +1,33 @@
+#! /usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2023 Intel Corporation
+
+import sys
+import re
+
+input_list = sys.argv[1:]
+test_def_regex = re.compile("REGISTER_([A-Z]+)_TEST\s*\(\s*([a-z0-9_]+)")
+test_suites = {}
+
+def get_fast_test_params(test_name, ln):
+    "Extract the extra fast-test parameters from the line"
+    #print(f"ln: {ln.rstrip()}, test_name: {test_name}, split: {ln.split(test_name, 1)}")
+    (_, rest_of_line) = ln.split(test_name, 1)
+    (_, nohuge, asan, _func) = rest_of_line.split(',', 3)
+    return f":{nohuge.strip().lower()}:{asan.strip().lower()}"
+
+for fname in input_list:
+    with open(fname) as f:
+        contents = [ln for ln in f.readlines() if test_def_regex.match(ln.strip())]
+    for ln in contents:
+        (test_suite, test_name) = test_def_regex.match(ln).group(1, 2)
+        suite_name = f"{test_suite.lower()}-tests"
+        if suite_name in test_suites:
+            test_suites[suite_name].append(test_name)
+        else:
+            test_suites[suite_name] = [test_name]
+        if suite_name == "fast-tests":
+            test_suites["fast-tests"][-1] += get_fast_test_params(test_name, ln)
+
+for suite in test_suites.keys():
+    print(f"{suite}={','.join(test_suites[suite])}")
diff --git a/app/test/has_hugepage.py b/buildtools/has-hugepages.py
similarity index 100%
rename from app/test/has_hugepage.py
rename to buildtools/has-hugepages.py
diff --git a/buildtools/meson.build b/buildtools/meson.build
index e1c600e40f..ac5e4dcf08 100644
--- a/buildtools/meson.build
+++ b/buildtools/meson.build
@@ -18,6 +18,8 @@ map_to_win_cmd = py3 + files('map_to_win.py')
 sphinx_wrapper = py3 + files('call-sphinx-build.py')
 get_cpu_count_cmd = py3 + files('get-cpu-count.py')
 get_numa_count_cmd = py3 + files('get-numa-count.py')
+get_test_suites_cmd = py3 + files('get-test-suites.py')
+has_hugepages_cmd = py3 + files('has-hugepages.py')
 binutils_avx512_check = (py3 + files('binutils-avx512-check.py') +
                         [objdump] + cc.cmd_array())
 
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v2 6/8] examples/l3fwd: make eventdev an optional dependency
  2023-08-14 15:16 ` [PATCH v2 0/8] expand list of optional libraries Bruce Richardson
                     ` (4 preceding siblings ...)
  2023-08-14 15:16   ` [PATCH v2 5/8] app/test: define unit tests suites based on test macros Bruce Richardson
@ 2023-08-14 15:16   ` Bruce Richardson
  2023-08-14 15:16   ` [PATCH v2 7/8] build: make most device classes optional Bruce Richardson
                     ` (2 subsequent siblings)
  8 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-14 15:16 UTC (permalink / raw)
  To: dev; +Cc: ci, Bruce Richardson

With l3fwd being a very commonly used example app, and built as part of
our CI tests, we need to ensure it's buildable with just about all
supported DPDK configurations.  To enable l3fwd application to be built
when the eventdev library is disabled, we need to compile in the
eventdev support conditionally.  Thankfully, the eventdev support is
pretty self-contained, with only the main.c file having more than a
couple of ifdefs.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 examples/l3fwd/l3fwd_em.c                  |  2 +
 examples/l3fwd/l3fwd_em_hlm.h              |  2 +
 examples/l3fwd/l3fwd_event.c               |  2 +
 examples/l3fwd/l3fwd_event.h               |  7 ++-
 examples/l3fwd/l3fwd_event_generic.c       |  2 +
 examples/l3fwd/l3fwd_event_internal_port.c |  2 +
 examples/l3fwd/l3fwd_fib.c                 |  2 +
 examples/l3fwd/l3fwd_lpm.c                 |  2 +
 examples/l3fwd/main.c                      | 65 +++++++++++++++++-----
 examples/l3fwd/meson.build                 |  5 +-
 10 files changed, 74 insertions(+), 17 deletions(-)

diff --git a/examples/l3fwd/l3fwd_em.c b/examples/l3fwd/l3fwd_em.c
index 476ac0c54f..40e102b38a 100644
--- a/examples/l3fwd/l3fwd_em.c
+++ b/examples/l3fwd/l3fwd_em.c
@@ -663,6 +663,7 @@ em_main_loop(__rte_unused void *dummy)
 	return 0;
 }
 
+#ifdef RTE_LIB_EVENTDEV
 static __rte_always_inline void
 em_event_loop_single(struct l3fwd_event_resources *evt_rsrc,
 		const uint8_t flags)
@@ -959,6 +960,7 @@ em_event_main_loop_tx_q_burst_vector(__rte_unused void *dummy)
 	em_event_loop_vector(evt_rsrc, L3FWD_EVENT_TX_ENQ);
 	return 0;
 }
+#endif
 
 /* Initialize exact match (hash) parameters. 8< */
 void
diff --git a/examples/l3fwd/l3fwd_em_hlm.h b/examples/l3fwd/l3fwd_em_hlm.h
index 2e11eefad7..31cda9ddc1 100644
--- a/examples/l3fwd/l3fwd_em_hlm.h
+++ b/examples/l3fwd/l3fwd_em_hlm.h
@@ -255,6 +255,7 @@ l3fwd_em_send_packets(int nb_rx, struct rte_mbuf **pkts_burst, uint16_t portid,
 	send_packets_multi(qconf, pkts_burst, dst_port, nb_rx);
 }
 
+#ifdef RTE_LIB_EVENTDEV
 /*
  * Buffer optimized handling of events, invoked
  * from main_loop.
@@ -347,5 +348,6 @@ l3fwd_em_process_event_vector(struct rte_event_vector *vec,
 
 	process_event_vector(vec, dst_port);
 }
+#endif /* RTE_LIB_EVENTDEV */
 
 #endif /* __L3FWD_EM_HLM_H__ */
diff --git a/examples/l3fwd/l3fwd_event.c b/examples/l3fwd/l3fwd_event.c
index 32906ab08d..d72a4138cc 100644
--- a/examples/l3fwd/l3fwd_event.c
+++ b/examples/l3fwd/l3fwd_event.c
@@ -2,6 +2,7 @@
  * Copyright(C) 2019 Marvell International Ltd.
  */
 
+#ifdef RTE_LIB_EVENTDEV
 #include <stdbool.h>
 #include <getopt.h>
 
@@ -341,3 +342,4 @@ l3fwd_event_worker_cleanup(uint8_t event_d_id, uint8_t event_p_id,
 	rte_event_port_quiesce(event_d_id, event_p_id, l3fwd_event_port_flush,
 			       NULL);
 }
+#endif /* #ifdef RTE_LIB_EVENTDEV */
diff --git a/examples/l3fwd/l3fwd_event.h b/examples/l3fwd/l3fwd_event.h
index e21817c36b..1fd6fe4a78 100644
--- a/examples/l3fwd/l3fwd_event.h
+++ b/examples/l3fwd/l3fwd_event.h
@@ -6,11 +6,13 @@
 #define __L3FWD_EVENTDEV_H__
 
 #include <rte_common.h>
+#include <rte_service.h>
+#include <rte_spinlock.h>
+
+#ifdef RTE_LIB_EVENTDEV
 #include <rte_eventdev.h>
 #include <rte_event_eth_rx_adapter.h>
 #include <rte_event_eth_tx_adapter.h>
-#include <rte_service.h>
-#include <rte_spinlock.h>
 
 #include "l3fwd.h"
 
@@ -164,4 +166,5 @@ void l3fwd_event_worker_cleanup(uint8_t event_d_id, uint8_t event_p_id,
 				struct rte_event events[], uint16_t nb_enq,
 				uint16_t nb_deq, uint8_t is_vector);
 
+#endif /* ifdef RTE_LIB_EVENTDEV */
 #endif /* __L3FWD_EVENTDEV_H__ */
diff --git a/examples/l3fwd/l3fwd_event_generic.c b/examples/l3fwd/l3fwd_event_generic.c
index c80573fc58..ddb6e5c38d 100644
--- a/examples/l3fwd/l3fwd_event_generic.c
+++ b/examples/l3fwd/l3fwd_event_generic.c
@@ -2,6 +2,7 @@
  * Copyright(C) 2019 Marvell International Ltd.
  */
 
+#ifdef RTE_LIB_EVENTDEV
 #include <stdbool.h>
 
 #include "l3fwd.h"
@@ -309,3 +310,4 @@ l3fwd_event_set_generic_ops(struct l3fwd_event_setup_ops *ops)
 	ops->event_port_setup = l3fwd_event_port_setup_generic;
 	ops->adapter_setup = l3fwd_rx_tx_adapter_setup_generic;
 }
+#endif /* RTE_LIB_EVENTDEV */
diff --git a/examples/l3fwd/l3fwd_event_internal_port.c b/examples/l3fwd/l3fwd_event_internal_port.c
index 32cf657148..cb49a8b9fa 100644
--- a/examples/l3fwd/l3fwd_event_internal_port.c
+++ b/examples/l3fwd/l3fwd_event_internal_port.c
@@ -2,6 +2,7 @@
  * Copyright(C) 2019 Marvell International Ltd.
  */
 
+#ifdef RTE_LIB_EVENTDEV
 #include <stdbool.h>
 
 #include "l3fwd.h"
@@ -311,3 +312,4 @@ l3fwd_event_set_internal_port_ops(struct l3fwd_event_setup_ops *ops)
 	ops->event_port_setup = l3fwd_event_port_setup_internal_port;
 	ops->adapter_setup = l3fwd_rx_tx_adapter_setup_internal_port;
 }
+#endif /* RTE_LIB_EVENTDEV */
diff --git a/examples/l3fwd/l3fwd_fib.c b/examples/l3fwd/l3fwd_fib.c
index 18398492ae..6a21984415 100644
--- a/examples/l3fwd/l3fwd_fib.c
+++ b/examples/l3fwd/l3fwd_fib.c
@@ -253,6 +253,7 @@ fib_main_loop(__rte_unused void *dummy)
 	return 0;
 }
 
+#ifdef RTE_LIB_EVENTDEV
 /* One eventdev loop for single and burst using fib. */
 static __rte_always_inline void
 fib_event_loop(struct l3fwd_event_resources *evt_rsrc,
@@ -635,6 +636,7 @@ fib_event_main_loop_tx_q_burst_vector(__rte_unused void *dummy)
 	fib_event_loop_vector(evt_rsrc, L3FWD_EVENT_TX_ENQ);
 	return 0;
 }
+#endif
 
 /* Function to setup fib. 8< */
 void
diff --git a/examples/l3fwd/l3fwd_lpm.c b/examples/l3fwd/l3fwd_lpm.c
index 4ac1925c84..a484a33089 100644
--- a/examples/l3fwd/l3fwd_lpm.c
+++ b/examples/l3fwd/l3fwd_lpm.c
@@ -226,6 +226,7 @@ lpm_main_loop(__rte_unused void *dummy)
 	return 0;
 }
 
+#ifdef RTE_LIB_EVENTDEV
 static __rte_always_inline uint16_t
 lpm_process_event_pkt(const struct lcore_conf *lconf, struct rte_mbuf *mbuf)
 {
@@ -554,6 +555,7 @@ lpm_event_main_loop_tx_q_burst_vector(__rte_unused void *dummy)
 	lpm_event_loop_vector(evt_rsrc, L3FWD_EVENT_TX_ENQ);
 	return 0;
 }
+#endif
 
 void
 setup_lpm(const int socketid)
diff --git a/examples/l3fwd/main.c b/examples/l3fwd/main.c
index a4f061537e..6063eb1399 100644
--- a/examples/l3fwd/main.c
+++ b/examples/l3fwd/main.c
@@ -135,8 +135,10 @@ static struct rte_eth_conf port_conf = {
 
 uint32_t max_pkt_len;
 
-static struct rte_mempool *pktmbuf_pool[RTE_MAX_ETHPORTS][NB_SOCKETS];
+#ifdef RTE_LIB_EVENTDEV
 static struct rte_mempool *vector_pool[RTE_MAX_ETHPORTS];
+#endif
+static struct rte_mempool *pktmbuf_pool[RTE_MAX_ETHPORTS][NB_SOCKETS];
 static uint8_t lkp_per_socket[NB_SOCKETS];
 
 struct l3fwd_lkp_mode {
@@ -398,8 +400,10 @@ print_usage(const char *prgname)
 		" [--parse-ptype]"
 		" [--per-port-pool]"
 		" [--mode]"
+#ifdef RTE_LIB_EVENTDEV
 		" [--eventq-sched]"
 		" [--event-vector [--event-vector-size SIZE] [--event-vector-tmo NS]]"
+#endif
 		" [-E]"
 		" [-L]\n\n"
 
@@ -422,6 +426,7 @@ print_usage(const char *prgname)
 		"  --per-port-pool: Use separate buffer pool per port\n"
 		"  --mode: Packet transfer mode for I/O, poll or eventdev\n"
 		"          Default mode = poll\n"
+#ifdef RTE_LIB_EVENTDEV
 		"  --eventq-sched: Event queue synchronization method\n"
 		"                  ordered, atomic or parallel.\n"
 		"                  Default: atomic\n"
@@ -432,6 +437,7 @@ print_usage(const char *prgname)
 		"  --event-vector:  Enable event vectorization.\n"
 		"  --event-vector-size: Max vector size if event vectorization is enabled.\n"
 		"  --event-vector-tmo: Max timeout to form vector in nanoseconds if event vectorization is enabled\n"
+#endif
 		"  -E : Enable exact match, legacy flag please use --lookup=em instead\n"
 		"  -L : Enable longest prefix match, legacy flag please use --lookup=lpm instead\n"
 		"  --rule_ipv4=FILE: Specify the ipv4 rules entries file.\n"
@@ -559,14 +565,16 @@ parse_eth_dest(const char *optarg)
 }
 
 static void
-parse_mode(const char *optarg)
+parse_mode(const char *optarg __rte_unused)
 {
+#ifdef RTE_LIB_EVENTDEV
 	struct l3fwd_event_resources *evt_rsrc = l3fwd_get_eventdev_rsrc();
 
 	if (!strcmp(optarg, "poll"))
 		evt_rsrc->enabled = false;
 	else if (!strcmp(optarg, "eventdev"))
 		evt_rsrc->enabled = true;
+#endif
 }
 
 static void
@@ -601,6 +609,7 @@ parse_queue_size(const char *queue_size_arg, uint16_t *queue_size, int rx)
 	*queue_size = value;
 }
 
+#ifdef RTE_LIB_EVENTDEV
 static void
 parse_eventq_sched(const char *optarg)
 {
@@ -631,6 +640,7 @@ parse_event_eth_rx_queues(const char *eth_rx_queues)
 
 	evt_rsrc->eth_rx_queues = num_eth_rx_queues;
 }
+#endif
 
 static int
 parse_lookup(const char *optarg)
@@ -756,9 +766,11 @@ parse_args(int argc, char **argv)
 	int option_index;
 	char *prgname = argv[0];
 	uint8_t lcore_params = 0;
+#ifdef RTE_LIB_EVENTDEV
 	uint8_t eventq_sched = 0;
 	uint8_t eth_rx_q = 0;
 	struct l3fwd_event_resources *evt_rsrc = l3fwd_get_eventdev_rsrc();
+#endif
 
 	argvopt = argv;
 
@@ -850,6 +862,7 @@ parse_args(int argc, char **argv)
 			parse_mode(optarg);
 			break;
 
+#ifdef RTE_LIB_EVENTDEV
 		case CMD_LINE_OPT_EVENTQ_SYNC_NUM:
 			parse_eventq_sched(optarg);
 			eventq_sched = 1;
@@ -860,6 +873,20 @@ parse_args(int argc, char **argv)
 			eth_rx_q = 1;
 			break;
 
+		case CMD_LINE_OPT_ENABLE_VECTOR_NUM:
+			printf("event vectorization is enabled\n");
+			evt_rsrc->vector_enabled = 1;
+			break;
+
+		case CMD_LINE_OPT_VECTOR_SIZE_NUM:
+			evt_rsrc->vector_size = strtol(optarg, NULL, 10);
+			break;
+
+		case CMD_LINE_OPT_VECTOR_TMO_NS_NUM:
+			evt_rsrc->vector_tmo_ns = strtoull(optarg, NULL, 10);
+			break;
+#endif
+
 		case CMD_LINE_OPT_LOOKUP_NUM:
 			if (lookup_mode != L3FWD_LOOKUP_DEFAULT) {
 				fprintf(stderr, "Only one lookup mode is allowed at a time!\n");
@@ -875,16 +902,6 @@ parse_args(int argc, char **argv)
 				return -1;
 			break;
 
-		case CMD_LINE_OPT_ENABLE_VECTOR_NUM:
-			printf("event vectorization is enabled\n");
-			evt_rsrc->vector_enabled = 1;
-			break;
-		case CMD_LINE_OPT_VECTOR_SIZE_NUM:
-			evt_rsrc->vector_size = strtol(optarg, NULL, 10);
-			break;
-		case CMD_LINE_OPT_VECTOR_TMO_NS_NUM:
-			evt_rsrc->vector_tmo_ns = strtoull(optarg, NULL, 10);
-			break;
 		case CMD_LINE_OPT_RULE_IPV4_NUM:
 			l3fwd_set_rule_ipv4_name(optarg);
 			break;
@@ -900,6 +917,8 @@ parse_args(int argc, char **argv)
 		}
 	}
 
+	RTE_SET_USED(lcore_params); /* needed if no eventdev block */
+#ifdef RTE_LIB_EVENTDEV
 	if (evt_rsrc->enabled && lcore_params) {
 		fprintf(stderr, "lcore config is not valid when event mode is selected\n");
 		return -1;
@@ -927,6 +946,7 @@ parse_args(int argc, char **argv)
 			"vector timeout set to default (%" PRIu64 " ns)\n",
 			evt_rsrc->vector_tmo_ns);
 	}
+#endif
 
 	/*
 	 * Nothing is selected, pick longest-prefix match
@@ -962,7 +982,9 @@ print_ethaddr(const char *name, const struct rte_ether_addr *eth_addr)
 int
 init_mem(uint16_t portid, unsigned int nb_mbuf)
 {
+#ifdef RTE_LIB_EVENTDEV
 	struct l3fwd_event_resources *evt_rsrc = l3fwd_get_eventdev_rsrc();
+#endif
 	struct lcore_conf *qconf;
 	int socketid;
 	unsigned lcore_id;
@@ -1007,6 +1029,7 @@ init_mem(uint16_t portid, unsigned int nb_mbuf)
 			}
 		}
 
+#ifdef RTE_LIB_EVENTDEV
 		if (evt_rsrc->vector_enabled && vector_pool[portid] == NULL) {
 			unsigned int nb_vec;
 
@@ -1025,6 +1048,7 @@ init_mem(uint16_t portid, unsigned int nb_mbuf)
 				printf("Allocated vector pool for port %d\n",
 				       portid);
 		}
+#endif
 
 		qconf = &lcore_conf[lcore_id];
 		qconf->ipv4_lookup_struct =
@@ -1406,6 +1430,7 @@ l3fwd_service_enable(uint32_t service_id)
 	return 0;
 }
 
+#ifdef RTE_LIB_EVENTDEV
 static void
 l3fwd_event_service_setup(void)
 {
@@ -1458,16 +1483,20 @@ l3fwd_event_service_setup(void)
 		l3fwd_service_enable(service_id);
 	}
 }
+#endif
 
 int
 main(int argc, char **argv)
 {
+#ifdef RTE_LIB_EVENTDEV
 	struct l3fwd_event_resources *evt_rsrc;
+	int i;
+#endif
 	struct lcore_conf *qconf;
 	uint16_t queueid, portid;
 	unsigned int lcore_id;
 	uint8_t queue;
-	int i, ret;
+	int ret;
 
 	/* init EAL */
 	ret = rte_eal_init(argc, argv);
@@ -1487,7 +1516,9 @@ main(int argc, char **argv)
 		*(uint64_t *)(val_eth + portid) = dest_eth_addr[portid];
 	}
 
+#ifdef RTE_LIB_EVENTDEV
 	evt_rsrc = l3fwd_get_eventdev_rsrc();
+#endif
 	/* parse application arguments (after the EAL ones) */
 	ret = parse_args(argc, argv);
 	if (ret < 0)
@@ -1499,6 +1530,7 @@ main(int argc, char **argv)
 	/* Add the config file rules */
 	l3fwd_lkp.read_config_files();
 
+#ifdef RTE_LIB_EVENTDEV
 	evt_rsrc->per_port_pool = per_port_pool;
 	evt_rsrc->pkt_pool = pktmbuf_pool;
 	evt_rsrc->vec_pool = vector_pool;
@@ -1514,6 +1546,7 @@ main(int argc, char **argv)
 			l3fwd_lkp.main_loop = evt_rsrc->ops.lpm_event_loop;
 		l3fwd_event_service_setup();
 	} else
+#endif
 		l3fwd_poll_resource_setup();
 
 	/* start ports */
@@ -1562,6 +1595,8 @@ main(int argc, char **argv)
 	ret = 0;
 	/* launch per-lcore init on every lcore */
 	rte_eal_mp_remote_launch(l3fwd_lkp.main_loop, NULL, CALL_MAIN);
+
+#ifdef RTE_LIB_EVENTDEV
 	if (evt_rsrc->enabled) {
 		for (i = 0; i < evt_rsrc->rx_adptr.nb_rx_adptr; i++)
 			rte_event_eth_rx_adapter_stop(
@@ -1589,7 +1624,9 @@ main(int argc, char **argv)
 		rte_event_dev_stop(evt_rsrc->event_d_id);
 		rte_event_dev_close(evt_rsrc->event_d_id);
 
-	} else {
+	} else
+#endif
+	{
 		rte_eal_mp_wait_lcore();
 
 		RTE_ETH_FOREACH_DEV(portid) {
diff --git a/examples/l3fwd/meson.build b/examples/l3fwd/meson.build
index b40244a941..c25de77bba 100644
--- a/examples/l3fwd/meson.build
+++ b/examples/l3fwd/meson.build
@@ -7,7 +7,7 @@
 # DPDK instance, use 'make'
 
 allow_experimental_apis = true
-deps += ['acl', 'hash', 'lpm', 'fib', 'eventdev']
+deps += ['acl', 'hash', 'lpm', 'fib']
 sources = files(
         'l3fwd_acl.c',
         'l3fwd_em.c',
@@ -18,3 +18,6 @@ sources = files(
         'l3fwd_lpm.c',
         'main.c',
 )
+if dpdk_conf.has('RTE_LIB_EVENTDEV')
+    deps += 'eventdev'
+endif
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v2 7/8] build: make most device classes optional
  2023-08-14 15:16 ` [PATCH v2 0/8] expand list of optional libraries Bruce Richardson
                     ` (5 preceding siblings ...)
  2023-08-14 15:16   ` [PATCH v2 6/8] examples/l3fwd: make eventdev an optional dependency Bruce Richardson
@ 2023-08-14 15:16   ` Bruce Richardson
  2023-08-14 15:16   ` [PATCH v2 8/8] build: expand list of optional libraries Bruce Richardson
  2023-08-14 15:22   ` [PATCH v2 0/8] " Bruce Richardson
  8 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-14 15:16 UTC (permalink / raw)
  To: dev; +Cc: ci, Bruce Richardson, Morten Brørup

Apart from ethdev and cryptodev, which have lots of components and tests
which depend on them, we can make the device class libraries optional
without too much work.

This patch marks:
* bbdev,
* compressdev,
* dmadev,
* eventdev,
* mldev,
* rawdev,
* regexdev
optional, and ensures that DPDK - including tests - can be built with
these components disabled.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
---
 lib/meson.build | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/lib/meson.build b/lib/meson.build
index 92cbd6bb66..1a78c8c984 100644
--- a/lib/meson.build
+++ b/lib/meson.build
@@ -67,8 +67,12 @@ libraries = [
 ]
 
 optional_libs = [
+        'bbdev',
         'bitratestats',
         'cfgfile',
+        'compressdev',
+        'dmadev',
+        'eventdev',
         'gpudev',
         'graph',
         'gro',
@@ -76,11 +80,14 @@ optional_libs = [
         'jobstats',
         'latencystats',
         'metrics',
+        'mldev',
         'node',
         'pdump',
         'pipeline',
         'port',
         'power',
+        'rawdev',
+        'regexdev',
         'table',
         'vhost',
 ]
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v2 8/8] build: expand list of optional libraries
  2023-08-14 15:16 ` [PATCH v2 0/8] expand list of optional libraries Bruce Richardson
                     ` (6 preceding siblings ...)
  2023-08-14 15:16   ` [PATCH v2 7/8] build: make most device classes optional Bruce Richardson
@ 2023-08-14 15:16   ` Bruce Richardson
  2023-08-14 15:22   ` [PATCH v2 0/8] " Bruce Richardson
  8 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-14 15:16 UTC (permalink / raw)
  To: dev; +Cc: ci, Bruce Richardson

With the unit test build now with individual per-file dependencies, we
can more easily expand the list of optional libraries. Add 8 new
libraries to the optional list.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 lib/meson.build | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/lib/meson.build b/lib/meson.build
index 1a78c8c984..0c9dc8b3b6 100644
--- a/lib/meson.build
+++ b/lib/meson.build
@@ -69,25 +69,33 @@ libraries = [
 optional_libs = [
         'bbdev',
         'bitratestats',
+        'bpf',
         'cfgfile',
         'compressdev',
+        'distributor',
         'dmadev',
+        'efd',
         'eventdev',
         'gpudev',
         'graph',
         'gro',
         'gso',
+        'ip_frag',
         'jobstats',
         'latencystats',
+        'member',
         'metrics',
         'mldev',
         'node',
+        'pcapng',
         'pdump',
         'pipeline',
         'port',
         'power',
         'rawdev',
         'regexdev',
+        'reorder',
+        'sched',
         'table',
         'vhost',
 ]
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* Re: [PATCH v2 0/8] expand list of optional libraries
  2023-08-14 15:16 ` [PATCH v2 0/8] expand list of optional libraries Bruce Richardson
                     ` (7 preceding siblings ...)
  2023-08-14 15:16   ` [PATCH v2 8/8] build: expand list of optional libraries Bruce Richardson
@ 2023-08-14 15:22   ` Bruce Richardson
  8 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-14 15:22 UTC (permalink / raw)
  To: dev; +Cc: ci

On Mon, Aug 14, 2023 at 04:16:01PM +0100, Bruce Richardson wrote:
> This patchset contains changes to DPDK to make the build more configurable
> overall. The changes can be grouped into two areas:
> 
> * Firstly, there are changes to make the build of the unit tests more
>   flexible and maintainable. These 5 patches switch the unit tests from
>   having separate hard-coded lists of files and tests, with lots of
>   conditional logic to add/remove items from the list, to instead being
>   a single list of files and dependencies per-file. The actual lists of
>   test-suites are built up dynamically from the list of files whose
>   dependencies are met and therefore will be part of the build.
>   This rework enables...
> * The second set of patches which expands the list of libraries which
>   can be disabled at build-time, while still giving a successful build
>   of DPDK. Overall, after this set is applied, the list of optional
>   libraries in DPDK is approx doubled in size. [15 additions giving
>   31 optional libs]
> 
> NOTES:
> * as stated in the RFC, the test binary moves in the build as a result
>   of this set - from app/test/dpdk-test to app/dpdk-test.
> * running the suites of tests no longer works after patch 4 until patch
>   5 is applied. If this is an issue, the two patches can be merged on
>   apply. However, the individual tests are still runnable through the
>   test binary - it's only through "meson test" that there are issues, so
>   I'm keeping the patches separate since each is relatively complex in its
>   own right.
> 
Apologies, some other things I neglected to mention in the notes above.

This patchset has revealed some issues with the building and running of
unit tests:
* Some files/tests were not getting built, and look to have been missed
  years ago in the move from make to meson. Specifically those tests using
  external resources, e.g. the cfgfile tests, are not buildable and have
  not been for years.
* There are quite a number of tests defined that are not registered in a
  test suite in meson and which cannot be run through meson test. Since the
  test cases were automatically tagged via script when generating this
  patchset, the offending test cases can now be found by grepping the
  sources for use of the old "REGISTER_TEST_COMMAND" macro.

The fixing of these issues I consider outside the scope of this set, and
won't be looking at in subsequent revisions. Hopefully #2 should be fairly
easy to rectify in future, but #1 may be more complex.

/Bruce

^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v3 0/8] expand list of optional libraries
  2023-07-21 11:51 [RFC PATCH 0/5] replace build code for unit tests Bruce Richardson
                   ` (6 preceding siblings ...)
  2023-08-14 15:16 ` [PATCH v2 0/8] expand list of optional libraries Bruce Richardson
@ 2023-08-14 18:20 ` Bruce Richardson
  2023-08-14 18:20   ` [PATCH v3 1/8] app/test: add new macros for various test types Bruce Richardson
                     ` (8 more replies)
  2023-08-15 13:13 ` [PATCH v4 " Bruce Richardson
                   ` (2 subsequent siblings)
  10 siblings, 9 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-14 18:20 UTC (permalink / raw)
  To: dev; +Cc: ci, Bruce Richardson

This patchset contains changes to DPDK to make the build more configurable
overall. The changes can be grouped into two areas:

* Firstly, there are changes to make the build of the unit tests more
  flexible and maintainable. These 5 patches switch the unit tests from
  having separate hard-coded lists of files and tests, with lots of
  conditional logic to add/remove items from the list, to instead being
  a single list of files and dependencies per-file. The actual lists of
  test-suites are built up dynamically from the list of files whose
  dependencies are met and therefore will be part of the build.
  This rework enables...
* The second set of patches which expands the list of libraries which
  can be disabled at build-time, while still giving a successful build
  of DPDK. Overall, after this set is applied, the list of optional
  libraries in DPDK is approx doubled in size. [15 additions giving
  31 optional libs]

NOTES:
* as stated in the RFC, the test binary moves in the build as a result
  of this set - from app/test/dpdk-test to app/dpdk-test.
* running the suites of tests no longer works after patch 4 until patch
  5 is applied. If this is an issue, the two patches can be merged on
  apply. However, the individual tests are still runnable through the
  test binary - it's only through "meson test" that there are issues, so
  I'm keeping the patches separate since each is relatively complex in its
  own right.
* A number of files in the test directory were not being built, and seem
  to have never been built with meson. This requires separate work, outside
  the scope of the patchset to fix.
* Similarly, some unit tests were never added to test suites. These can
  now be found by using grep to find the old REGISTER_TEST_COMMAND macro.
  Again, adding orphan tests to suites is outside the scope of this set.

V3:
- Create symlink to old path to test binary, to ensure all automated CI,
  and any other use of the test binary still works.

V2:
- Changes since RFC:
   -  merged two previous sets - reworking test build, and disabling
      libs - into one, since they depend on each other
   -  Reworked the list of dependencies for each file in the tests,
      so that files that depend on other files have those deps called
      out.
   -  Added list of external dependencies to the test builds
   -  For disabling libs, merged patches disabling individual libs
      together into one patch, since all unit test build changes were
      unnecessary. This made each patch a single-line addition to
      the optional list in lib/meson.build.

Bruce Richardson (8):
  app/test: add new macros for various test types
  app/test: tag tests with the test type
  app/test: make telemetry data test buildable on windows
  app/test: build using per-file dependency matrix
  app/test: define unit tests suites based on test macros
  examples/l3fwd: make eventdev an optional dependency
  build: make most device classes optional
  build: expand list of optional libraries

 app/meson.build                               |  15 +-
 app/test/meson.build                          | 823 +++++-------------
 app/test/suites/meson.build                   |  76 ++
 app/test/test.h                               |   9 +-
 app/test/test_acl.c                           |   2 +-
 app/test/test_atomic.c                        |   2 +-
 app/test/test_barrier.c                       |   2 +-
 app/test/test_bitmap.c                        |   2 +-
 app/test/test_bitops.c                        |   2 +-
 app/test/test_bpf.c                           |   4 +-
 app/test/test_byteorder.c                     |   2 +-
 app/test/test_cksum.c                         |   2 +-
 app/test/test_cmdline.c                       |   2 +-
 app/test/test_common.c                        |   2 +-
 app/test/test_cpuflags.c                      |   2 +-
 app/test/test_crc.c                           |   2 +-
 app/test/test_cryptodev.c                     |  38 +-
 app/test/test_cryptodev_asym.c                |   2 +-
 app/test/test_cycles.c                        |   2 +-
 app/test/test_debug.c                         |   2 +-
 app/test/test_devargs.c                       |   2 +-
 app/test/test_distributor.c                   |   2 +-
 app/test/test_distributor_perf.c              |   2 +-
 app/test/test_dmadev.c                        |   2 +-
 app/test/test_eal_flags.c                     |  24 +-
 app/test/test_eal_fs.c                        |   2 +-
 app/test/test_efd.c                           |   2 +-
 app/test/test_efd_perf.c                      |   2 +-
 app/test/test_errno.c                         |   2 +-
 app/test/test_ethdev_link.c                   |   2 +-
 app/test/test_event_ring.c                    |   2 +-
 app/test/test_eventdev.c                      |   2 +-
 app/test/test_fbarray.c                       |   2 +-
 app/test/test_fib.c                           |   4 +-
 app/test/test_fib6.c                          |   4 +-
 app/test/test_fib6_perf.c                     |   2 +-
 app/test/test_fib_perf.c                      |   2 +-
 app/test/test_func_reentrancy.c               |   2 +-
 app/test/test_hash.c                          |   2 +-
 app/test/test_hash_functions.c                |   2 +-
 app/test/test_hash_multiwriter.c              |   2 +-
 app/test/test_hash_perf.c                     |   2 +-
 app/test/test_hash_readwrite.c                |   4 +-
 app/test/test_hash_readwrite_lf_perf.c        |   2 +-
 app/test/test_interrupts.c                    |   2 +-
 app/test/test_ipfrag.c                        |   2 +-
 app/test/test_ipsec.c                         |   2 +-
 app/test/test_ipsec_perf.c                    |   2 +-
 app/test/test_kvargs.c                        |   2 +-
 app/test/test_lcores.c                        |   2 +-
 app/test/test_logs.c                          |   2 +-
 app/test/test_lpm.c                           |   2 +-
 app/test/test_lpm6.c                          |   2 +-
 app/test/test_lpm6_perf.c                     |   2 +-
 app/test/test_lpm_perf.c                      |   2 +-
 app/test/test_malloc.c                        |   2 +-
 app/test/test_malloc_perf.c                   |   2 +-
 app/test/test_mbuf.c                          |   2 +-
 app/test/test_mcslock.c                       |   2 +-
 app/test/test_member.c                        |   2 +-
 app/test/test_member_perf.c                   |   2 +-
 app/test/test_memcpy.c                        |   2 +-
 app/test/test_memcpy_perf.c                   |   2 +-
 app/test/test_memory.c                        |   2 +-
 app/test/test_mempool.c                       |   2 +-
 app/test/test_mempool_perf.c                  |   2 +-
 app/test/test_memzone.c                       |   2 +-
 app/test/test_meter.c                         |   2 +-
 app/test/test_mp_secondary.c                  |   2 +-
 app/test/test_per_lcore.c                     |   2 +-
 app/test/test_pflock.c                        |   2 +-
 app/test/test_pie.c                           |   6 +-
 app/test/test_pmd_perf.c                      |   2 +-
 app/test/test_power.c                         |   2 +-
 app/test/test_power_cpufreq.c                 |   2 +-
 app/test/test_power_intel_uncore.c            |   2 +-
 app/test/test_power_kvm_vm.c                  |   2 +-
 app/test/test_prefetch.c                      |   2 +-
 app/test/test_rand_perf.c                     |   2 +-
 app/test/test_rcu_qsbr.c                      |   2 +-
 app/test/test_rcu_qsbr_perf.c                 |   2 +-
 app/test/test_reassembly_perf.c               |   2 +-
 app/test/test_reciprocal_division.c           |   2 +-
 app/test/test_reciprocal_division_perf.c      |   2 +-
 app/test/test_red.c                           |   4 +-
 app/test/test_reorder.c                       |   2 +-
 app/test/test_rib.c                           |   4 +-
 app/test/test_rib6.c                          |   4 +-
 app/test/test_ring.c                          |   2 +-
 app/test/test_ring_perf.c                     |   2 +-
 app/test/test_rwlock.c                        |   8 +-
 app/test/test_sched.c                         |   2 +-
 app/test/test_security.c                      |   2 +-
 app/test/test_seqlock.c                       |   2 +-
 app/test/test_service_cores.c                 |   4 +-
 app/test/test_spinlock.c                      |   2 +-
 app/test/test_stack.c                         |   4 +-
 app/test/test_stack_perf.c                    |   4 +-
 app/test/test_string_fns.c                    |   2 +-
 app/test/test_tailq.c                         |   2 +-
 app/test/test_telemetry_data.c                |  14 +-
 app/test/test_thash.c                         |   2 +-
 app/test/test_thash_perf.c                    |   2 +-
 app/test/test_threads.c                       |   2 +-
 app/test/test_ticketlock.c                    |   2 +-
 app/test/test_timer.c                         |   2 +-
 app/test/test_timer_perf.c                    |   2 +-
 app/test/test_timer_racecond.c                |   2 +-
 app/test/test_trace.c                         |   2 +-
 app/test/test_trace_perf.c                    |   2 +-
 app/test/test_version.c                       |   2 +-
 buildtools/get-test-suites.py                 |  33 +
 .../has-hugepages.py                          |   0
 buildtools/meson.build                        |   2 +
 examples/l3fwd/l3fwd_em.c                     |   2 +
 examples/l3fwd/l3fwd_em_hlm.h                 |   2 +
 examples/l3fwd/l3fwd_event.c                  |   2 +
 examples/l3fwd/l3fwd_event.h                  |   7 +-
 examples/l3fwd/l3fwd_event_generic.c          |   2 +
 examples/l3fwd/l3fwd_event_internal_port.c    |   2 +
 examples/l3fwd/l3fwd_fib.c                    |   2 +
 examples/l3fwd/l3fwd_lpm.c                    |   2 +
 examples/l3fwd/main.c                         |  65 +-
 examples/l3fwd/meson.build                    |   5 +-
 lib/meson.build                               |  15 +
 125 files changed, 614 insertions(+), 764 deletions(-)
 create mode 100644 app/test/suites/meson.build
 create mode 100644 buildtools/get-test-suites.py
 rename app/test/has_hugepage.py => buildtools/has-hugepages.py (100%)

--
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v3 1/8] app/test: add new macros for various test types
  2023-08-14 18:20 ` [PATCH v3 " Bruce Richardson
@ 2023-08-14 18:20   ` Bruce Richardson
  2023-08-14 18:20   ` [PATCH v3 2/8] app/test: tag tests with the test type Bruce Richardson
                     ` (7 subsequent siblings)
  8 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-14 18:20 UTC (permalink / raw)
  To: dev; +Cc: ci, Bruce Richardson

Rather than just registering all tests using a single generic macro,
add macros which identify the test as being of a particular type.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 app/test/test.h | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/app/test/test.h b/app/test/test.h
index 85f57efbc6..a91ded76af 100644
--- a/app/test/test.h
+++ b/app/test/test.h
@@ -190,7 +190,7 @@ struct test_command {
 
 void add_test_command(struct test_command *t);
 
-/* Register a test function with its command string */
+/* Register a test function with its command string. Should not be used directly */
 #define REGISTER_TEST_COMMAND(cmd, func) \
 	static struct test_command test_struct_##cmd = { \
 		.command = RTE_STR(cmd), \
@@ -201,4 +201,11 @@ void add_test_command(struct test_command *t);
 		add_test_command(&test_struct_##cmd); \
 	}
 
+/* Register a test function as a particular type.
+ * These can be used to build up test suites automatically
+ */
+#define REGISTER_FAST_TEST(cmd, no_huge, ASan, func)  REGISTER_TEST_COMMAND(cmd, func)
+#define REGISTER_PERF_TEST REGISTER_TEST_COMMAND
+#define REGISTER_DRIVER_TEST REGISTER_TEST_COMMAND
+
 #endif
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v3 2/8] app/test: tag tests with the test type
  2023-08-14 18:20 ` [PATCH v3 " Bruce Richardson
  2023-08-14 18:20   ` [PATCH v3 1/8] app/test: add new macros for various test types Bruce Richardson
@ 2023-08-14 18:20   ` Bruce Richardson
  2023-08-14 18:20   ` [PATCH v3 3/8] app/test: make telemetry data test buildable on windows Bruce Richardson
                     ` (6 subsequent siblings)
  8 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-14 18:20 UTC (permalink / raw)
  To: dev; +Cc: ci, Bruce Richardson

Rather than having the test types called out in the meson.build file, we
can use macros to identify the test type in the C file itself and then
dynamically build up the tests lists at config time.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 app/test/test_acl.c                      |  2 +-
 app/test/test_atomic.c                   |  2 +-
 app/test/test_barrier.c                  |  2 +-
 app/test/test_bitmap.c                   |  2 +-
 app/test/test_bitops.c                   |  2 +-
 app/test/test_bpf.c                      |  4 +--
 app/test/test_byteorder.c                |  2 +-
 app/test/test_cksum.c                    |  2 +-
 app/test/test_cmdline.c                  |  2 +-
 app/test/test_common.c                   |  2 +-
 app/test/test_cpuflags.c                 |  2 +-
 app/test/test_crc.c                      |  2 +-
 app/test/test_cryptodev.c                | 38 ++++++++++++------------
 app/test/test_cryptodev_asym.c           |  2 +-
 app/test/test_cycles.c                   |  2 +-
 app/test/test_debug.c                    |  2 +-
 app/test/test_devargs.c                  |  2 +-
 app/test/test_distributor.c              |  2 +-
 app/test/test_distributor_perf.c         |  2 +-
 app/test/test_dmadev.c                   |  2 +-
 app/test/test_eal_flags.c                | 24 +++++++--------
 app/test/test_eal_fs.c                   |  2 +-
 app/test/test_efd.c                      |  2 +-
 app/test/test_efd_perf.c                 |  2 +-
 app/test/test_errno.c                    |  2 +-
 app/test/test_ethdev_link.c              |  2 +-
 app/test/test_event_ring.c               |  2 +-
 app/test/test_eventdev.c                 |  2 +-
 app/test/test_fbarray.c                  |  2 +-
 app/test/test_fib.c                      |  4 +--
 app/test/test_fib6.c                     |  4 +--
 app/test/test_fib6_perf.c                |  2 +-
 app/test/test_fib_perf.c                 |  2 +-
 app/test/test_func_reentrancy.c          |  2 +-
 app/test/test_hash.c                     |  2 +-
 app/test/test_hash_functions.c           |  2 +-
 app/test/test_hash_multiwriter.c         |  2 +-
 app/test/test_hash_perf.c                |  2 +-
 app/test/test_hash_readwrite.c           |  4 +--
 app/test/test_hash_readwrite_lf_perf.c   |  2 +-
 app/test/test_interrupts.c               |  2 +-
 app/test/test_ipfrag.c                   |  2 +-
 app/test/test_ipsec.c                    |  2 +-
 app/test/test_ipsec_perf.c               |  2 +-
 app/test/test_kvargs.c                   |  2 +-
 app/test/test_lcores.c                   |  2 +-
 app/test/test_logs.c                     |  2 +-
 app/test/test_lpm.c                      |  2 +-
 app/test/test_lpm6.c                     |  2 +-
 app/test/test_lpm6_perf.c                |  2 +-
 app/test/test_lpm_perf.c                 |  2 +-
 app/test/test_malloc.c                   |  2 +-
 app/test/test_malloc_perf.c              |  2 +-
 app/test/test_mbuf.c                     |  2 +-
 app/test/test_mcslock.c                  |  2 +-
 app/test/test_member.c                   |  2 +-
 app/test/test_member_perf.c              |  2 +-
 app/test/test_memcpy.c                   |  2 +-
 app/test/test_memcpy_perf.c              |  2 +-
 app/test/test_memory.c                   |  2 +-
 app/test/test_mempool.c                  |  2 +-
 app/test/test_mempool_perf.c             |  2 +-
 app/test/test_memzone.c                  |  2 +-
 app/test/test_meter.c                    |  2 +-
 app/test/test_mp_secondary.c             |  2 +-
 app/test/test_per_lcore.c                |  2 +-
 app/test/test_pflock.c                   |  2 +-
 app/test/test_pie.c                      |  6 ++--
 app/test/test_pmd_perf.c                 |  2 +-
 app/test/test_power.c                    |  2 +-
 app/test/test_power_cpufreq.c            |  2 +-
 app/test/test_power_intel_uncore.c       |  2 +-
 app/test/test_power_kvm_vm.c             |  2 +-
 app/test/test_prefetch.c                 |  2 +-
 app/test/test_rand_perf.c                |  2 +-
 app/test/test_rcu_qsbr.c                 |  2 +-
 app/test/test_rcu_qsbr_perf.c            |  2 +-
 app/test/test_reassembly_perf.c          |  2 +-
 app/test/test_reciprocal_division.c      |  2 +-
 app/test/test_reciprocal_division_perf.c |  2 +-
 app/test/test_red.c                      |  4 +--
 app/test/test_reorder.c                  |  2 +-
 app/test/test_rib.c                      |  4 +--
 app/test/test_rib6.c                     |  4 +--
 app/test/test_ring.c                     |  2 +-
 app/test/test_ring_perf.c                |  2 +-
 app/test/test_rwlock.c                   |  8 ++---
 app/test/test_sched.c                    |  2 +-
 app/test/test_security.c                 |  2 +-
 app/test/test_seqlock.c                  |  2 +-
 app/test/test_service_cores.c            |  4 +--
 app/test/test_spinlock.c                 |  2 +-
 app/test/test_stack.c                    |  4 +--
 app/test/test_stack_perf.c               |  4 +--
 app/test/test_string_fns.c               |  2 +-
 app/test/test_tailq.c                    |  2 +-
 app/test/test_thash.c                    |  2 +-
 app/test/test_thash_perf.c               |  2 +-
 app/test/test_threads.c                  |  2 +-
 app/test/test_ticketlock.c               |  2 +-
 app/test/test_timer.c                    |  2 +-
 app/test/test_timer_perf.c               |  2 +-
 app/test/test_timer_racecond.c           |  2 +-
 app/test/test_trace.c                    |  2 +-
 app/test/test_trace_perf.c               |  2 +-
 app/test/test_version.c                  |  2 +-
 106 files changed, 150 insertions(+), 150 deletions(-)

diff --git a/app/test/test_acl.c b/app/test/test_acl.c
index 623f34682e..bf1466fe11 100644
--- a/app/test/test_acl.c
+++ b/app/test/test_acl.c
@@ -1749,4 +1749,4 @@ test_acl(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(acl_autotest, test_acl);
+REGISTER_FAST_TEST(acl_autotest, True, True, test_acl);
diff --git a/app/test/test_atomic.c b/app/test/test_atomic.c
index e4b997827e..b52420c10b 100644
--- a/app/test/test_atomic.c
+++ b/app/test/test_atomic.c
@@ -631,4 +631,4 @@ test_atomic(void)
 
 	return 0;
 }
-REGISTER_TEST_COMMAND(atomic_autotest, test_atomic);
+REGISTER_FAST_TEST(atomic_autotest, False, True, test_atomic);
diff --git a/app/test/test_barrier.c b/app/test/test_barrier.c
index ec69af25bf..925a88b68a 100644
--- a/app/test/test_barrier.c
+++ b/app/test/test_barrier.c
@@ -285,4 +285,4 @@ test_barrier(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(barrier_autotest, test_barrier);
+REGISTER_PERF_TEST(barrier_autotest, test_barrier);
diff --git a/app/test/test_bitmap.c b/app/test/test_bitmap.c
index e9c61590ae..2014d2682c 100644
--- a/app/test/test_bitmap.c
+++ b/app/test/test_bitmap.c
@@ -269,4 +269,4 @@ test_bitmap(void)
 	return test_bitmap_all_set();
 }
 
-REGISTER_TEST_COMMAND(bitmap_autotest, test_bitmap);
+REGISTER_FAST_TEST(bitmap_autotest, True, True, test_bitmap);
diff --git a/app/test/test_bitops.c b/app/test/test_bitops.c
index c21426bf2f..dac466602b 100644
--- a/app/test/test_bitops.c
+++ b/app/test/test_bitops.c
@@ -135,4 +135,4 @@ test_bitops(void)
 	return TEST_SUCCESS;
 }
 
-REGISTER_TEST_COMMAND(bitops_autotest, test_bitops);
+REGISTER_FAST_TEST(bitops_autotest, True, True, test_bitops);
diff --git a/app/test/test_bpf.c b/app/test/test_bpf.c
index f5af5e8a3f..5fef17fbc2 100644
--- a/app/test/test_bpf.c
+++ b/app/test/test_bpf.c
@@ -3262,7 +3262,7 @@ test_bpf(void)
 
 #endif /* !RTE_LIB_BPF */
 
-REGISTER_TEST_COMMAND(bpf_autotest, test_bpf);
+REGISTER_FAST_TEST(bpf_autotest, True, True, test_bpf);
 
 #ifndef RTE_HAS_LIBPCAP
 
@@ -3473,4 +3473,4 @@ test_bpf_convert(void)
 
 #endif /* RTE_HAS_LIBPCAP */
 
-REGISTER_TEST_COMMAND(bpf_convert_autotest, test_bpf_convert);
+REGISTER_FAST_TEST(bpf_convert_autotest, True, True, test_bpf_convert);
diff --git a/app/test/test_byteorder.c b/app/test/test_byteorder.c
index de14ed539e..2189f94744 100644
--- a/app/test/test_byteorder.c
+++ b/app/test/test_byteorder.c
@@ -63,4 +63,4 @@ test_byteorder(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(byteorder_autotest, test_byteorder);
+REGISTER_FAST_TEST(byteorder_autotest, True, True, test_byteorder);
diff --git a/app/test/test_cksum.c b/app/test/test_cksum.c
index 6c15de9a93..c04fae800e 100644
--- a/app/test/test_cksum.c
+++ b/app/test/test_cksum.c
@@ -267,4 +267,4 @@ test_cksum(void)
 }
 #undef GOTO_FAIL
 
-REGISTER_TEST_COMMAND(cksum_autotest, test_cksum);
+REGISTER_FAST_TEST(cksum_autotest, True, True, test_cksum);
diff --git a/app/test/test_cmdline.c b/app/test/test_cmdline.c
index 115bee966d..c2a375ac1c 100644
--- a/app/test/test_cmdline.c
+++ b/app/test/test_cmdline.c
@@ -60,4 +60,4 @@ test_cmdline(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(cmdline_autotest, test_cmdline);
+REGISTER_FAST_TEST(cmdline_autotest, True, True, test_cmdline);
diff --git a/app/test/test_common.c b/app/test/test_common.c
index f89e1eb7ee..c7b41a3f0e 100644
--- a/app/test/test_common.c
+++ b/app/test/test_common.c
@@ -350,4 +350,4 @@ test_common(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(common_autotest, test_common);
+REGISTER_FAST_TEST(common_autotest, True, True, test_common);
diff --git a/app/test/test_cpuflags.c b/app/test/test_cpuflags.c
index a0e342ae48..c3417dda24 100644
--- a/app/test/test_cpuflags.c
+++ b/app/test/test_cpuflags.c
@@ -334,4 +334,4 @@ test_cpuflags(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(cpuflags_autotest, test_cpuflags);
+REGISTER_FAST_TEST(cpuflags_autotest, True, True, test_cpuflags);
diff --git a/app/test/test_crc.c b/app/test/test_crc.c
index 5edc8fb13b..b267bdde9f 100644
--- a/app/test/test_crc.c
+++ b/app/test/test_crc.c
@@ -171,4 +171,4 @@ test_crc(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(crc_autotest, test_crc);
+REGISTER_FAST_TEST(crc_autotest, True, True, test_crc);
diff --git a/app/test/test_cryptodev.c b/app/test/test_cryptodev.c
index fb2af40b99..956268bfcd 100644
--- a/app/test/test_cryptodev.c
+++ b/app/test/test_cryptodev.c
@@ -17535,33 +17535,33 @@ REGISTER_TEST_COMMAND(cryptodev_dpaa2_sec_raw_api_autotest,
 		test_cryptodev_dpaa2_sec_raw_api);
 REGISTER_TEST_COMMAND(cryptodev_dpaa_sec_raw_api_autotest,
 		test_cryptodev_dpaa_sec_raw_api);
-REGISTER_TEST_COMMAND(cryptodev_qat_raw_api_autotest,
+REGISTER_DRIVER_TEST(cryptodev_qat_raw_api_autotest,
 		test_cryptodev_qat_raw_api);
-REGISTER_TEST_COMMAND(cryptodev_qat_autotest, test_cryptodev_qat);
-REGISTER_TEST_COMMAND(cryptodev_aesni_mb_autotest, test_cryptodev_aesni_mb);
-REGISTER_TEST_COMMAND(cryptodev_cpu_aesni_mb_autotest,
+REGISTER_DRIVER_TEST(cryptodev_qat_autotest, test_cryptodev_qat);
+REGISTER_DRIVER_TEST(cryptodev_aesni_mb_autotest, test_cryptodev_aesni_mb);
+REGISTER_DRIVER_TEST(cryptodev_cpu_aesni_mb_autotest,
 	test_cryptodev_cpu_aesni_mb);
-REGISTER_TEST_COMMAND(cryptodev_chacha_poly_mb_autotest,
+REGISTER_DRIVER_TEST(cryptodev_chacha_poly_mb_autotest,
 	test_cryptodev_chacha_poly_mb);
-REGISTER_TEST_COMMAND(cryptodev_openssl_autotest, test_cryptodev_openssl);
-REGISTER_TEST_COMMAND(cryptodev_aesni_gcm_autotest, test_cryptodev_aesni_gcm);
-REGISTER_TEST_COMMAND(cryptodev_cpu_aesni_gcm_autotest,
+REGISTER_DRIVER_TEST(cryptodev_openssl_autotest, test_cryptodev_openssl);
+REGISTER_DRIVER_TEST(cryptodev_aesni_gcm_autotest, test_cryptodev_aesni_gcm);
+REGISTER_DRIVER_TEST(cryptodev_cpu_aesni_gcm_autotest,
 	test_cryptodev_cpu_aesni_gcm);
 REGISTER_TEST_COMMAND(cryptodev_mlx5_autotest, test_cryptodev_mlx5);
-REGISTER_TEST_COMMAND(cryptodev_null_autotest, test_cryptodev_null);
-REGISTER_TEST_COMMAND(cryptodev_sw_snow3g_autotest, test_cryptodev_sw_snow3g);
-REGISTER_TEST_COMMAND(cryptodev_sw_kasumi_autotest, test_cryptodev_sw_kasumi);
-REGISTER_TEST_COMMAND(cryptodev_sw_zuc_autotest, test_cryptodev_sw_zuc);
-REGISTER_TEST_COMMAND(cryptodev_sw_armv8_autotest, test_cryptodev_armv8);
-REGISTER_TEST_COMMAND(cryptodev_sw_mvsam_autotest, test_cryptodev_mrvl);
-REGISTER_TEST_COMMAND(cryptodev_dpaa2_sec_autotest, test_cryptodev_dpaa2_sec);
-REGISTER_TEST_COMMAND(cryptodev_dpaa_sec_autotest, test_cryptodev_dpaa_sec);
+REGISTER_DRIVER_TEST(cryptodev_null_autotest, test_cryptodev_null);
+REGISTER_DRIVER_TEST(cryptodev_sw_snow3g_autotest, test_cryptodev_sw_snow3g);
+REGISTER_DRIVER_TEST(cryptodev_sw_kasumi_autotest, test_cryptodev_sw_kasumi);
+REGISTER_DRIVER_TEST(cryptodev_sw_zuc_autotest, test_cryptodev_sw_zuc);
+REGISTER_DRIVER_TEST(cryptodev_sw_armv8_autotest, test_cryptodev_armv8);
+REGISTER_DRIVER_TEST(cryptodev_sw_mvsam_autotest, test_cryptodev_mrvl);
+REGISTER_DRIVER_TEST(cryptodev_dpaa2_sec_autotest, test_cryptodev_dpaa2_sec);
+REGISTER_DRIVER_TEST(cryptodev_dpaa_sec_autotest, test_cryptodev_dpaa_sec);
 REGISTER_TEST_COMMAND(cryptodev_ccp_autotest, test_cryptodev_ccp);
-REGISTER_TEST_COMMAND(cryptodev_uadk_autotest, test_cryptodev_uadk);
+REGISTER_DRIVER_TEST(cryptodev_uadk_autotest, test_cryptodev_uadk);
 REGISTER_TEST_COMMAND(cryptodev_virtio_autotest, test_cryptodev_virtio);
 REGISTER_TEST_COMMAND(cryptodev_octeontx_autotest, test_cryptodev_octeontx);
 REGISTER_TEST_COMMAND(cryptodev_caam_jr_autotest, test_cryptodev_caam_jr);
 REGISTER_TEST_COMMAND(cryptodev_nitrox_autotest, test_cryptodev_nitrox);
 REGISTER_TEST_COMMAND(cryptodev_bcmfs_autotest, test_cryptodev_bcmfs);
-REGISTER_TEST_COMMAND(cryptodev_cn9k_autotest, test_cryptodev_cn9k);
-REGISTER_TEST_COMMAND(cryptodev_cn10k_autotest, test_cryptodev_cn10k);
+REGISTER_DRIVER_TEST(cryptodev_cn9k_autotest, test_cryptodev_cn9k);
+REGISTER_DRIVER_TEST(cryptodev_cn10k_autotest, test_cryptodev_cn10k);
diff --git a/app/test/test_cryptodev_asym.c b/app/test/test_cryptodev_asym.c
index 0ef2642fdd..3d5a73bf89 100644
--- a/app/test/test_cryptodev_asym.c
+++ b/app/test/test_cryptodev_asym.c
@@ -2828,7 +2828,7 @@ test_cryptodev_cn10k_asym(void)
 REGISTER_TEST_COMMAND(cryptodev_openssl_asym_autotest,
 					  test_cryptodev_openssl_asym);
 
-REGISTER_TEST_COMMAND(cryptodev_qat_asym_autotest, test_cryptodev_qat_asym);
+REGISTER_DRIVER_TEST(cryptodev_qat_asym_autotest, test_cryptodev_qat_asym);
 
 REGISTER_TEST_COMMAND(cryptodev_octeontx_asym_autotest,
 					  test_cryptodev_octeontx_asym);
diff --git a/app/test/test_cycles.c b/app/test/test_cycles.c
index 66d11e6db8..08c91cef78 100644
--- a/app/test/test_cycles.c
+++ b/app/test/test_cycles.c
@@ -53,4 +53,4 @@ test_user_delay_us(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(user_delay_us, test_user_delay_us);
+REGISTER_FAST_TEST(user_delay_us, True, True, test_user_delay_us);
diff --git a/app/test/test_debug.c b/app/test/test_debug.c
index 2704f5b927..acd9542257 100644
--- a/app/test/test_debug.c
+++ b/app/test/test_debug.c
@@ -140,4 +140,4 @@ test_debug(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(debug_autotest, test_debug);
+REGISTER_FAST_TEST(debug_autotest, True, True, test_debug);
diff --git a/app/test/test_devargs.c b/app/test/test_devargs.c
index 0a4c34a1ad..59ec04957e 100644
--- a/app/test/test_devargs.c
+++ b/app/test/test_devargs.c
@@ -213,4 +213,4 @@ test_devargs(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(devargs_autotest, test_devargs);
+REGISTER_FAST_TEST(devargs_autotest, True, True, test_devargs);
diff --git a/app/test/test_distributor.c b/app/test/test_distributor.c
index 3efa4af104..ee263a7fee 100644
--- a/app/test/test_distributor.c
+++ b/app/test/test_distributor.c
@@ -952,4 +952,4 @@ test_distributor(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(distributor_autotest, test_distributor);
+REGISTER_FAST_TEST(distributor_autotest, False, True, test_distributor);
diff --git a/app/test/test_distributor_perf.c b/app/test/test_distributor_perf.c
index ee4321486d..ca868451d7 100644
--- a/app/test/test_distributor_perf.c
+++ b/app/test/test_distributor_perf.c
@@ -277,4 +277,4 @@ test_distributor_perf(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(distributor_perf_autotest, test_distributor_perf);
+REGISTER_PERF_TEST(distributor_perf_autotest, test_distributor_perf);
diff --git a/app/test/test_dmadev.c b/app/test/test_dmadev.c
index 0736ff2a18..6ef875e545 100644
--- a/app/test/test_dmadev.c
+++ b/app/test/test_dmadev.c
@@ -941,4 +941,4 @@ test_dma(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(dmadev_autotest, test_dma);
+REGISTER_DRIVER_TEST(dmadev_autotest, test_dma);
diff --git a/app/test/test_eal_flags.c b/app/test/test_eal_flags.c
index 148e9098e8..3b7af7405d 100644
--- a/app/test/test_eal_flags.c
+++ b/app/test/test_eal_flags.c
@@ -1644,15 +1644,15 @@ test_memory_flags(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(eal_flags_c_opt_autotest, test_missing_c_flag);
-REGISTER_TEST_COMMAND(eal_flags_main_opt_autotest, test_main_lcore_flag);
-REGISTER_TEST_COMMAND(eal_flags_n_opt_autotest, test_invalid_n_flag);
-REGISTER_TEST_COMMAND(eal_flags_hpet_autotest, test_no_hpet_flag);
-REGISTER_TEST_COMMAND(eal_flags_no_huge_autotest, test_no_huge_flag);
-REGISTER_TEST_COMMAND(eal_flags_a_opt_autotest, test_allow_flag);
-REGISTER_TEST_COMMAND(eal_flags_b_opt_autotest, test_invalid_b_flag);
-REGISTER_TEST_COMMAND(eal_flags_vdev_opt_autotest, test_invalid_vdev_flag);
-REGISTER_TEST_COMMAND(eal_flags_r_opt_autotest, test_invalid_r_flag);
-REGISTER_TEST_COMMAND(eal_flags_mem_autotest, test_memory_flags);
-REGISTER_TEST_COMMAND(eal_flags_file_prefix_autotest, test_file_prefix);
-REGISTER_TEST_COMMAND(eal_flags_misc_autotest, test_misc_flags);
+REGISTER_FAST_TEST(eal_flags_c_opt_autotest, False, False, test_missing_c_flag);
+REGISTER_FAST_TEST(eal_flags_main_opt_autotest, False, False, test_main_lcore_flag);
+REGISTER_FAST_TEST(eal_flags_n_opt_autotest, False, False, test_invalid_n_flag);
+REGISTER_FAST_TEST(eal_flags_hpet_autotest, False, False, test_no_hpet_flag);
+REGISTER_FAST_TEST(eal_flags_no_huge_autotest, False, False, test_no_huge_flag);
+REGISTER_FAST_TEST(eal_flags_a_opt_autotest, False, False, test_allow_flag);
+REGISTER_FAST_TEST(eal_flags_b_opt_autotest, False, False, test_invalid_b_flag);
+REGISTER_FAST_TEST(eal_flags_vdev_opt_autotest, False, False, test_invalid_vdev_flag);
+REGISTER_FAST_TEST(eal_flags_r_opt_autotest, False, False, test_invalid_r_flag);
+REGISTER_FAST_TEST(eal_flags_mem_autotest, False, False, test_memory_flags);
+REGISTER_FAST_TEST(eal_flags_file_prefix_autotest, False, False, test_file_prefix);
+REGISTER_FAST_TEST(eal_flags_misc_autotest, False, False, test_misc_flags);
diff --git a/app/test/test_eal_fs.c b/app/test/test_eal_fs.c
index b3686edcb4..a4bdc1236d 100644
--- a/app/test/test_eal_fs.c
+++ b/app/test/test_eal_fs.c
@@ -185,4 +185,4 @@ test_eal_fs(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(eal_fs_autotest, test_eal_fs);
+REGISTER_FAST_TEST(eal_fs_autotest, True, True, test_eal_fs);
diff --git a/app/test/test_efd.c b/app/test/test_efd.c
index fa29e8f97a..1c0986b9bc 100644
--- a/app/test/test_efd.c
+++ b/app/test/test_efd.c
@@ -473,4 +473,4 @@ test_efd(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(efd_autotest, test_efd);
+REGISTER_PERF_TEST(efd_autotest, test_efd);
diff --git a/app/test/test_efd_perf.c b/app/test/test_efd_perf.c
index 4d04ed93e3..b212e96767 100644
--- a/app/test/test_efd_perf.c
+++ b/app/test/test_efd_perf.c
@@ -393,4 +393,4 @@ test_efd_perf(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(efd_perf_autotest, test_efd_perf);
+REGISTER_PERF_TEST(efd_perf_autotest, test_efd_perf);
diff --git a/app/test/test_errno.c b/app/test/test_errno.c
index 0db4fbc8b3..1082299665 100644
--- a/app/test/test_errno.c
+++ b/app/test/test_errno.c
@@ -94,4 +94,4 @@ test_errno(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(errno_autotest, test_errno);
+REGISTER_FAST_TEST(errno_autotest, True, True, test_errno);
diff --git a/app/test/test_ethdev_link.c b/app/test/test_ethdev_link.c
index ab52385a12..d73e18065d 100644
--- a/app/test/test_ethdev_link.c
+++ b/app/test/test_ethdev_link.c
@@ -167,4 +167,4 @@ test_link_status(void)
 	return unit_test_suite_runner(&link_status_testsuite);
 }
 
-REGISTER_TEST_COMMAND(ethdev_link_status, test_link_status);
+REGISTER_FAST_TEST(ethdev_link_status, True, True, test_link_status);
diff --git a/app/test/test_event_ring.c b/app/test/test_event_ring.c
index bc4a6e73f5..d5c8499cfd 100644
--- a/app/test/test_event_ring.c
+++ b/app/test/test_event_ring.c
@@ -256,4 +256,4 @@ test_event_ring(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(event_ring_autotest, test_event_ring);
+REGISTER_FAST_TEST(event_ring_autotest, True, True, test_event_ring);
diff --git a/app/test/test_eventdev.c b/app/test/test_eventdev.c
index 336529038e..e65d893781 100644
--- a/app/test/test_eventdev.c
+++ b/app/test/test_eventdev.c
@@ -1256,7 +1256,7 @@ test_eventdev_selftest_cn10k(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(eventdev_common_autotest, test_eventdev_common);
+REGISTER_FAST_TEST(eventdev_common_autotest, True, True, test_eventdev_common);
 
 #ifndef RTE_EXEC_ENV_WINDOWS
 REGISTER_TEST_COMMAND(eventdev_selftest_sw, test_eventdev_selftest_sw);
diff --git a/app/test/test_fbarray.c b/app/test/test_fbarray.c
index a691bf4458..03428a45fe 100644
--- a/app/test/test_fbarray.c
+++ b/app/test/test_fbarray.c
@@ -733,4 +733,4 @@ test_fbarray(void)
 	return unit_test_suite_runner(&fbarray_test_suite);
 }
 
-REGISTER_TEST_COMMAND(fbarray_autotest, test_fbarray);
+REGISTER_FAST_TEST(fbarray_autotest, True, True, test_fbarray);
diff --git a/app/test/test_fib.c b/app/test/test_fib.c
index eb69d6e2fd..2a8404db61 100644
--- a/app/test/test_fib.c
+++ b/app/test/test_fib.c
@@ -415,5 +415,5 @@ test_slow_fib(void)
 	return unit_test_suite_runner(&fib_slow_tests);
 }
 
-REGISTER_TEST_COMMAND(fib_autotest, test_fib);
-REGISTER_TEST_COMMAND(fib_slow_autotest, test_slow_fib);
+REGISTER_FAST_TEST(fib_autotest, True, True, test_fib);
+REGISTER_PERF_TEST(fib_slow_autotest, test_slow_fib);
diff --git a/app/test/test_fib6.c b/app/test/test_fib6.c
index 15ad09178a..9b1c0ac8c7 100644
--- a/app/test/test_fib6.c
+++ b/app/test/test_fib6.c
@@ -424,5 +424,5 @@ test_slow_fib6(void)
 	return unit_test_suite_runner(&fib6_slow_tests);
 }
 
-REGISTER_TEST_COMMAND(fib6_autotest, test_fib6);
-REGISTER_TEST_COMMAND(fib6_slow_autotest, test_slow_fib6);
+REGISTER_FAST_TEST(fib6_autotest, True, True, test_fib6);
+REGISTER_PERF_TEST(fib6_slow_autotest, test_slow_fib6);
diff --git a/app/test/test_fib6_perf.c b/app/test/test_fib6_perf.c
index add20c2331..a7abc46af9 100644
--- a/app/test/test_fib6_perf.c
+++ b/app/test/test_fib6_perf.c
@@ -156,4 +156,4 @@ test_fib6_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(fib6_perf_autotest, test_fib6_perf);
+REGISTER_PERF_TEST(fib6_perf_autotest, test_fib6_perf);
diff --git a/app/test/test_fib_perf.c b/app/test/test_fib_perf.c
index b56293e64f..a9119c1bb0 100644
--- a/app/test/test_fib_perf.c
+++ b/app/test/test_fib_perf.c
@@ -409,4 +409,4 @@ test_fib_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(fib_perf_autotest, test_fib_perf);
+REGISTER_PERF_TEST(fib_perf_autotest, test_fib_perf);
diff --git a/app/test/test_func_reentrancy.c b/app/test/test_func_reentrancy.c
index ae9de6f93d..63b00009b5 100644
--- a/app/test/test_func_reentrancy.c
+++ b/app/test/test_func_reentrancy.c
@@ -507,4 +507,4 @@ test_func_reentrancy(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(func_reentrancy_autotest, test_func_reentrancy);
+REGISTER_FAST_TEST(func_reentrancy_autotest, False, True, test_func_reentrancy);
diff --git a/app/test/test_hash.c b/app/test/test_hash.c
index 3e45afaa67..e160b27b4c 100644
--- a/app/test/test_hash.c
+++ b/app/test/test_hash.c
@@ -2264,4 +2264,4 @@ test_hash(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(hash_autotest, test_hash);
+REGISTER_FAST_TEST(hash_autotest, True, True, test_hash);
diff --git a/app/test/test_hash_functions.c b/app/test/test_hash_functions.c
index 76d51b6e71..70820d1f19 100644
--- a/app/test/test_hash_functions.c
+++ b/app/test/test_hash_functions.c
@@ -290,4 +290,4 @@ test_hash_functions(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(hash_functions_autotest, test_hash_functions);
+REGISTER_PERF_TEST(hash_functions_autotest, test_hash_functions);
diff --git a/app/test/test_hash_multiwriter.c b/app/test/test_hash_multiwriter.c
index 0c5a8ca186..dd5ca677b9 100644
--- a/app/test/test_hash_multiwriter.c
+++ b/app/test/test_hash_multiwriter.c
@@ -287,4 +287,4 @@ test_hash_multiwriter_main(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(hash_multiwriter_autotest, test_hash_multiwriter_main);
+REGISTER_PERF_TEST(hash_multiwriter_autotest, test_hash_multiwriter_main);
diff --git a/app/test/test_hash_perf.c b/app/test/test_hash_perf.c
index 14a1283aba..d66b96e5ce 100644
--- a/app/test/test_hash_perf.c
+++ b/app/test/test_hash_perf.c
@@ -757,4 +757,4 @@ test_hash_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(hash_perf_autotest, test_hash_perf);
+REGISTER_PERF_TEST(hash_perf_autotest, test_hash_perf);
diff --git a/app/test/test_hash_readwrite.c b/app/test/test_hash_readwrite.c
index 6373e62d33..9c33b49878 100644
--- a/app/test/test_hash_readwrite.c
+++ b/app/test/test_hash_readwrite.c
@@ -760,5 +760,5 @@ test_hash_rw_func_main(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(hash_readwrite_func_autotest, test_hash_rw_func_main);
-REGISTER_TEST_COMMAND(hash_readwrite_perf_autotest, test_hash_rw_perf_main);
+REGISTER_FAST_TEST(hash_readwrite_func_autotest, False, True, test_hash_rw_func_main);
+REGISTER_PERF_TEST(hash_readwrite_perf_autotest, test_hash_rw_perf_main);
diff --git a/app/test/test_hash_readwrite_lf_perf.c b/app/test/test_hash_readwrite_lf_perf.c
index cf86046a2f..5d18850e19 100644
--- a/app/test/test_hash_readwrite_lf_perf.c
+++ b/app/test/test_hash_readwrite_lf_perf.c
@@ -1579,5 +1579,5 @@ test_hash_readwrite_lf_perf_main(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(hash_readwrite_lf_perf_autotest,
+REGISTER_PERF_TEST(hash_readwrite_lf_perf_autotest,
 	test_hash_readwrite_lf_perf_main);
diff --git a/app/test/test_interrupts.c b/app/test/test_interrupts.c
index b59ab42699..cf6b5a6a8e 100644
--- a/app/test/test_interrupts.c
+++ b/app/test/test_interrupts.c
@@ -594,4 +594,4 @@ test_interrupt(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(interrupt_autotest, test_interrupt);
+REGISTER_FAST_TEST(interrupt_autotest, True, True, test_interrupt);
diff --git a/app/test/test_ipfrag.c b/app/test/test_ipfrag.c
index 402ce361c1..264825634d 100644
--- a/app/test/test_ipfrag.c
+++ b/app/test/test_ipfrag.c
@@ -510,4 +510,4 @@ test_ipfrag(void)
 }
 
 
-REGISTER_TEST_COMMAND(ipfrag_autotest, test_ipfrag);
+REGISTER_FAST_TEST(ipfrag_autotest, False, True, test_ipfrag);
diff --git a/app/test/test_ipsec.c b/app/test/test_ipsec.c
index c2a52ec305..629580bc21 100644
--- a/app/test/test_ipsec.c
+++ b/app/test/test_ipsec.c
@@ -2532,4 +2532,4 @@ test_ipsec(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(ipsec_autotest, test_ipsec);
+REGISTER_FAST_TEST(ipsec_autotest, True, True, test_ipsec);
diff --git a/app/test/test_ipsec_perf.c b/app/test/test_ipsec_perf.c
index b221b7fc32..a32a2086e9 100644
--- a/app/test/test_ipsec_perf.c
+++ b/app/test/test_ipsec_perf.c
@@ -631,4 +631,4 @@ test_libipsec_perf(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(ipsec_perf_autotest, test_libipsec_perf);
+REGISTER_PERF_TEST(ipsec_perf_autotest, test_libipsec_perf);
diff --git a/app/test/test_kvargs.c b/app/test/test_kvargs.c
index b7b97a0dd9..d0f05a55c7 100644
--- a/app/test/test_kvargs.c
+++ b/app/test/test_kvargs.c
@@ -292,4 +292,4 @@ test_kvargs(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(kvargs_autotest, test_kvargs);
+REGISTER_FAST_TEST(kvargs_autotest, True, True, test_kvargs);
diff --git a/app/test/test_lcores.c b/app/test/test_lcores.c
index 2c945b0136..b86629c0db 100644
--- a/app/test/test_lcores.c
+++ b/app/test/test_lcores.c
@@ -412,4 +412,4 @@ test_lcores(void)
 	return TEST_SUCCESS;
 }
 
-REGISTER_TEST_COMMAND(lcores_autotest, test_lcores);
+REGISTER_FAST_TEST(lcores_autotest, True, True, test_lcores);
diff --git a/app/test/test_logs.c b/app/test/test_logs.c
index 8da8824bee..38052910e9 100644
--- a/app/test/test_logs.c
+++ b/app/test/test_logs.c
@@ -158,4 +158,4 @@ test_logs(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(logs_autotest, test_logs);
+REGISTER_FAST_TEST(logs_autotest, True, True, test_logs);
diff --git a/app/test/test_lpm.c b/app/test/test_lpm.c
index 37b460af3a..85a75ccec0 100644
--- a/app/test/test_lpm.c
+++ b/app/test/test_lpm.c
@@ -1584,4 +1584,4 @@ test_lpm(void)
 	return global_status;
 }
 
-REGISTER_TEST_COMMAND(lpm_autotest, test_lpm);
+REGISTER_FAST_TEST(lpm_autotest, True, True, test_lpm);
diff --git a/app/test/test_lpm6.c b/app/test/test_lpm6.c
index b6b6f8615e..4a9eddf276 100644
--- a/app/test/test_lpm6.c
+++ b/app/test/test_lpm6.c
@@ -1793,4 +1793,4 @@ test_lpm6(void)
 	return global_status;
 }
 
-REGISTER_TEST_COMMAND(lpm6_autotest, test_lpm6);
+REGISTER_FAST_TEST(lpm6_autotest, True, True, test_lpm6);
diff --git a/app/test/test_lpm6_perf.c b/app/test/test_lpm6_perf.c
index 5b684686a6..8a49f74c84 100644
--- a/app/test/test_lpm6_perf.c
+++ b/app/test/test_lpm6_perf.c
@@ -161,4 +161,4 @@ test_lpm6_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(lpm6_perf_autotest, test_lpm6_perf);
+REGISTER_PERF_TEST(lpm6_perf_autotest, test_lpm6_perf);
diff --git a/app/test/test_lpm_perf.c b/app/test/test_lpm_perf.c
index e72437ba38..15ff396dd0 100644
--- a/app/test/test_lpm_perf.c
+++ b/app/test/test_lpm_perf.c
@@ -760,4 +760,4 @@ test_lpm_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(lpm_perf_autotest, test_lpm_perf);
+REGISTER_PERF_TEST(lpm_perf_autotest, test_lpm_perf);
diff --git a/app/test/test_malloc.c b/app/test/test_malloc.c
index ff081dd931..6057623473 100644
--- a/app/test/test_malloc.c
+++ b/app/test/test_malloc.c
@@ -1091,4 +1091,4 @@ test_malloc(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(malloc_autotest, test_malloc);
+REGISTER_FAST_TEST(malloc_autotest, False, True, test_malloc);
diff --git a/app/test/test_malloc_perf.c b/app/test/test_malloc_perf.c
index 9bd1662981..a99bfd8531 100644
--- a/app/test/test_malloc_perf.c
+++ b/app/test/test_malloc_perf.c
@@ -171,4 +171,4 @@ test_malloc_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(malloc_perf_autotest, test_malloc_perf);
+REGISTER_PERF_TEST(malloc_perf_autotest, test_malloc_perf);
diff --git a/app/test/test_mbuf.c b/app/test/test_mbuf.c
index efac01806b..2eeed268af 100644
--- a/app/test/test_mbuf.c
+++ b/app/test/test_mbuf.c
@@ -2955,4 +2955,4 @@ test_mbuf(void)
 }
 #undef GOTO_FAIL
 
-REGISTER_TEST_COMMAND(mbuf_autotest, test_mbuf);
+REGISTER_FAST_TEST(mbuf_autotest, False, True, test_mbuf);
diff --git a/app/test/test_mcslock.c b/app/test/test_mcslock.c
index 52e45e7e2a..a3753a4397 100644
--- a/app/test/test_mcslock.c
+++ b/app/test/test_mcslock.c
@@ -241,4 +241,4 @@ test_mcslock(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(mcslock_autotest, test_mcslock);
+REGISTER_FAST_TEST(mcslock_autotest, False, True, test_mcslock);
diff --git a/app/test/test_member.c b/app/test/test_member.c
index 4a93f8bff4..e17a8f3190 100644
--- a/app/test/test_member.c
+++ b/app/test/test_member.c
@@ -996,4 +996,4 @@ test_member(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(member_autotest, test_member);
+REGISTER_FAST_TEST(member_autotest, True, True, test_member);
diff --git a/app/test/test_member_perf.c b/app/test/test_member_perf.c
index 2f79888fbd..db6b8a18ef 100644
--- a/app/test/test_member_perf.c
+++ b/app/test/test_member_perf.c
@@ -780,4 +780,4 @@ test_member_perf(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(member_perf_autotest, test_member_perf);
+REGISTER_PERF_TEST(member_perf_autotest, test_member_perf);
diff --git a/app/test/test_memcpy.c b/app/test/test_memcpy.c
index 1ab86f4967..c81efb85c2 100644
--- a/app/test/test_memcpy.c
+++ b/app/test/test_memcpy.c
@@ -129,4 +129,4 @@ test_memcpy(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(memcpy_autotest, test_memcpy);
+REGISTER_FAST_TEST(memcpy_autotest, True, True, test_memcpy);
diff --git a/app/test/test_memcpy_perf.c b/app/test/test_memcpy_perf.c
index 3727c160e6..5c05a84619 100644
--- a/app/test/test_memcpy_perf.c
+++ b/app/test/test_memcpy_perf.c
@@ -348,4 +348,4 @@ test_memcpy_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(memcpy_perf_autotest, test_memcpy_perf);
+REGISTER_PERF_TEST(memcpy_perf_autotest, test_memcpy_perf);
diff --git a/app/test/test_memory.c b/app/test/test_memory.c
index 440e5ef838..13ae9569c2 100644
--- a/app/test/test_memory.c
+++ b/app/test/test_memory.c
@@ -110,4 +110,4 @@ test_memory(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(memory_autotest, test_memory);
+REGISTER_FAST_TEST(memory_autotest, False, True, test_memory);
diff --git a/app/test/test_mempool.c b/app/test/test_mempool.c
index 8e493eda47..03047e5b14 100644
--- a/app/test/test_mempool.c
+++ b/app/test/test_mempool.c
@@ -1043,4 +1043,4 @@ test_mempool(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(mempool_autotest, test_mempool);
+REGISTER_FAST_TEST(mempool_autotest, False, True, test_mempool);
diff --git a/app/test/test_mempool_perf.c b/app/test/test_mempool_perf.c
index ce7c6241ab..96de347f04 100644
--- a/app/test/test_mempool_perf.c
+++ b/app/test/test_mempool_perf.c
@@ -437,4 +437,4 @@ test_mempool_perf(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(mempool_perf_autotest, test_mempool_perf);
+REGISTER_PERF_TEST(mempool_perf_autotest, test_mempool_perf);
diff --git a/app/test/test_memzone.c b/app/test/test_memzone.c
index f10f4fd9cd..85b5839034 100644
--- a/app/test/test_memzone.c
+++ b/app/test/test_memzone.c
@@ -1163,4 +1163,4 @@ test_memzone(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(memzone_autotest, test_memzone);
+REGISTER_FAST_TEST(memzone_autotest, False, True, test_memzone);
diff --git a/app/test/test_meter.c b/app/test/test_meter.c
index 15d5a4839b..3016681bbc 100644
--- a/app/test/test_meter.c
+++ b/app/test/test_meter.c
@@ -713,4 +713,4 @@ test_meter(void)
 
 }
 
-REGISTER_TEST_COMMAND(meter_autotest, test_meter);
+REGISTER_FAST_TEST(meter_autotest, True, True, test_meter);
diff --git a/app/test/test_mp_secondary.c b/app/test/test_mp_secondary.c
index ad47d578f2..502b0235a7 100644
--- a/app/test/test_mp_secondary.c
+++ b/app/test/test_mp_secondary.c
@@ -223,4 +223,4 @@ test_mp_secondary(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(multiprocess_autotest, test_mp_secondary);
+REGISTER_FAST_TEST(multiprocess_autotest, False, False, test_mp_secondary);
diff --git a/app/test/test_per_lcore.c b/app/test/test_per_lcore.c
index 129578d1a3..50011366a7 100644
--- a/app/test/test_per_lcore.c
+++ b/app/test/test_per_lcore.c
@@ -105,4 +105,4 @@ test_per_lcore(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(per_lcore_autotest, test_per_lcore);
+REGISTER_FAST_TEST(per_lcore_autotest, True, True, test_per_lcore);
diff --git a/app/test/test_pflock.c b/app/test/test_pflock.c
index 38da6bce27..07e4a68c2c 100644
--- a/app/test/test_pflock.c
+++ b/app/test/test_pflock.c
@@ -193,4 +193,4 @@ test_pflock(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(pflock_autotest, test_pflock);
+REGISTER_FAST_TEST(pflock_autotest, True, True, test_pflock);
diff --git a/app/test/test_pie.c b/app/test/test_pie.c
index a3c0f97c9d..5674602b69 100644
--- a/app/test/test_pie.c
+++ b/app/test/test_pie.c
@@ -1087,6 +1087,6 @@ test_pie_all(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(pie_autotest, test_pie);
-REGISTER_TEST_COMMAND(pie_perf, test_pie_perf);
-REGISTER_TEST_COMMAND(pie_all, test_pie_all);
+REGISTER_FAST_TEST(pie_autotest, True, True, test_pie);
+REGISTER_PERF_TEST(pie_perf, test_pie_perf);
+REGISTER_PERF_TEST(pie_all, test_pie_all);
diff --git a/app/test/test_pmd_perf.c b/app/test/test_pmd_perf.c
index 3ef590cb51..f6d97f21c9 100644
--- a/app/test/test_pmd_perf.c
+++ b/app/test/test_pmd_perf.c
@@ -899,4 +899,4 @@ test_set_rxtx_sc(cmdline_fixed_string_t type)
 	return -1;
 }
 
-REGISTER_TEST_COMMAND(pmd_perf_autotest, test_pmd_perf);
+REGISTER_PERF_TEST(pmd_perf_autotest, test_pmd_perf);
diff --git a/app/test/test_power.c b/app/test/test_power.c
index b7b5561348..d4f622ff2f 100644
--- a/app/test/test_power.c
+++ b/app/test/test_power.c
@@ -172,4 +172,4 @@ test_power(void)
 }
 #endif
 
-REGISTER_TEST_COMMAND(power_autotest, test_power);
+REGISTER_FAST_TEST(power_autotest, True, True, test_power);
diff --git a/app/test/test_power_cpufreq.c b/app/test/test_power_cpufreq.c
index 4d013cd7bb..f56053ac10 100644
--- a/app/test/test_power_cpufreq.c
+++ b/app/test/test_power_cpufreq.c
@@ -703,5 +703,5 @@ test_power_caps(void)
 
 #endif
 
-REGISTER_TEST_COMMAND(power_cpufreq_autotest, test_power_cpufreq);
+REGISTER_FAST_TEST(power_cpufreq_autotest, False, True, test_power_cpufreq);
 REGISTER_TEST_COMMAND(power_caps_autotest, test_power_caps);
diff --git a/app/test/test_power_intel_uncore.c b/app/test/test_power_intel_uncore.c
index 31163af84e..c8a8621e44 100644
--- a/app/test/test_power_intel_uncore.c
+++ b/app/test/test_power_intel_uncore.c
@@ -298,4 +298,4 @@ test_power_intel_uncore(void)
 }
 #endif
 
-REGISTER_TEST_COMMAND(power_intel_uncore_autotest, test_power_intel_uncore);
+REGISTER_FAST_TEST(power_intel_uncore_autotest, True, True, test_power_intel_uncore);
diff --git a/app/test/test_power_kvm_vm.c b/app/test/test_power_kvm_vm.c
index cc66b7a8a0..4919df8b62 100644
--- a/app/test/test_power_kvm_vm.c
+++ b/app/test/test_power_kvm_vm.c
@@ -299,4 +299,4 @@ test_power_kvm_vm(void)
 }
 #endif
 
-REGISTER_TEST_COMMAND(power_kvm_vm_autotest, test_power_kvm_vm);
+REGISTER_FAST_TEST(power_kvm_vm_autotest, False, True, test_power_kvm_vm);
diff --git a/app/test/test_prefetch.c b/app/test/test_prefetch.c
index 7b4a8e4144..a72ae80557 100644
--- a/app/test/test_prefetch.c
+++ b/app/test/test_prefetch.c
@@ -35,4 +35,4 @@ test_prefetch(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(prefetch_autotest, test_prefetch);
+REGISTER_FAST_TEST(prefetch_autotest, True, True, test_prefetch);
diff --git a/app/test/test_rand_perf.c b/app/test/test_rand_perf.c
index 26fb1d9a58..30204e12c0 100644
--- a/app/test/test_rand_perf.c
+++ b/app/test/test_rand_perf.c
@@ -96,4 +96,4 @@ test_rand_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(rand_perf_autotest, test_rand_perf);
+REGISTER_PERF_TEST(rand_perf_autotest, test_rand_perf);
diff --git a/app/test/test_rcu_qsbr.c b/app/test/test_rcu_qsbr.c
index 70404e89e6..16c24f58e7 100644
--- a/app/test/test_rcu_qsbr.c
+++ b/app/test/test_rcu_qsbr.c
@@ -1418,4 +1418,4 @@ test_rcu_qsbr_main(void)
 	return -1;
 }
 
-REGISTER_TEST_COMMAND(rcu_qsbr_autotest, test_rcu_qsbr_main);
+REGISTER_FAST_TEST(rcu_qsbr_autotest, True, True, test_rcu_qsbr_main);
diff --git a/app/test/test_rcu_qsbr_perf.c b/app/test/test_rcu_qsbr_perf.c
index b15e5cef88..ce88a7333c 100644
--- a/app/test/test_rcu_qsbr_perf.c
+++ b/app/test/test_rcu_qsbr_perf.c
@@ -690,4 +690,4 @@ test_rcu_qsbr_main(void)
 	return -1;
 }
 
-REGISTER_TEST_COMMAND(rcu_qsbr_perf_autotest, test_rcu_qsbr_main);
+REGISTER_PERF_TEST(rcu_qsbr_perf_autotest, test_rcu_qsbr_main);
diff --git a/app/test/test_reassembly_perf.c b/app/test/test_reassembly_perf.c
index c11b65291f..4b4929d777 100644
--- a/app/test/test_reassembly_perf.c
+++ b/app/test/test_reassembly_perf.c
@@ -1000,4 +1000,4 @@ test_reassembly_perf(void)
 	return TEST_SUCCESS;
 }
 
-REGISTER_TEST_COMMAND(reassembly_perf_autotest, test_reassembly_perf);
+REGISTER_PERF_TEST(reassembly_perf_autotest, test_reassembly_perf);
diff --git a/app/test/test_reciprocal_division.c b/app/test/test_reciprocal_division.c
index 8ea9b1d24d..fb52b2d5a1 100644
--- a/app/test/test_reciprocal_division.c
+++ b/app/test/test_reciprocal_division.c
@@ -164,4 +164,4 @@ test_reciprocal(void)
 	return result;
 }
 
-REGISTER_TEST_COMMAND(reciprocal_division, test_reciprocal);
+REGISTER_PERF_TEST(reciprocal_division, test_reciprocal);
diff --git a/app/test/test_reciprocal_division_perf.c b/app/test/test_reciprocal_division_perf.c
index 4f625873e5..cf96d46a22 100644
--- a/app/test/test_reciprocal_division_perf.c
+++ b/app/test/test_reciprocal_division_perf.c
@@ -205,4 +205,4 @@ test_reciprocal_division_perf(void)
 	return result;
 }
 
-REGISTER_TEST_COMMAND(reciprocal_division_perf, test_reciprocal_division_perf);
+REGISTER_PERF_TEST(reciprocal_division_perf, test_reciprocal_division_perf);
diff --git a/app/test/test_red.c b/app/test/test_red.c
index 84c292f8d8..aa7538d51a 100644
--- a/app/test/test_red.c
+++ b/app/test/test_red.c
@@ -1878,5 +1878,5 @@ test_red_all(void)
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
 REGISTER_TEST_COMMAND(red_autotest, test_red);
-REGISTER_TEST_COMMAND(red_perf, test_red_perf);
-REGISTER_TEST_COMMAND(red_all, test_red_all);
+REGISTER_PERF_TEST(red_perf, test_red_perf);
+REGISTER_PERF_TEST(red_all, test_red_all);
diff --git a/app/test/test_reorder.c b/app/test/test_reorder.c
index c188f6ce67..e66b4b05d4 100644
--- a/app/test/test_reorder.c
+++ b/app/test/test_reorder.c
@@ -548,4 +548,4 @@ test_reorder(void)
 }
 
 
-REGISTER_TEST_COMMAND(reorder_autotest, test_reorder);
+REGISTER_FAST_TEST(reorder_autotest, True, True, test_reorder);
diff --git a/app/test/test_rib.c b/app/test/test_rib.c
index 65b6856410..a8d60acc3a 100644
--- a/app/test/test_rib.c
+++ b/app/test/test_rib.c
@@ -363,5 +363,5 @@ test_slow_rib(void)
 	return unit_test_suite_runner(&rib_slow_tests);
 }
 
-REGISTER_TEST_COMMAND(rib_autotest, test_rib);
-REGISTER_TEST_COMMAND(rib_slow_autotest, test_slow_rib);
+REGISTER_FAST_TEST(rib_autotest, True, True, test_rib);
+REGISTER_PERF_TEST(rib_slow_autotest, test_slow_rib);
diff --git a/app/test/test_rib6.c b/app/test/test_rib6.c
index 336b779d2e..9bbd0f406d 100644
--- a/app/test/test_rib6.c
+++ b/app/test/test_rib6.c
@@ -367,5 +367,5 @@ test_slow_rib6(void)
 	return unit_test_suite_runner(&rib6_slow_tests);
 }
 
-REGISTER_TEST_COMMAND(rib6_autotest, test_rib6);
-REGISTER_TEST_COMMAND(rib6_slow_autotest, test_slow_rib6);
+REGISTER_FAST_TEST(rib6_autotest, True, True, test_rib6);
+REGISTER_PERF_TEST(rib6_slow_autotest, test_slow_rib6);
diff --git a/app/test/test_ring.c b/app/test/test_ring.c
index bde33ab4a1..d2accbabfa 100644
--- a/app/test/test_ring.c
+++ b/app/test/test_ring.c
@@ -1241,4 +1241,4 @@ test_ring(void)
 	return -1;
 }
 
-REGISTER_TEST_COMMAND(ring_autotest, test_ring);
+REGISTER_FAST_TEST(ring_autotest, True, True, test_ring);
diff --git a/app/test/test_ring_perf.c b/app/test/test_ring_perf.c
index 3972fd9db3..d7c5a4c30b 100644
--- a/app/test/test_ring_perf.c
+++ b/app/test/test_ring_perf.c
@@ -579,4 +579,4 @@ test_ring_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(ring_perf_autotest, test_ring_perf);
+REGISTER_PERF_TEST(ring_perf_autotest, test_ring_perf);
diff --git a/app/test/test_rwlock.c b/app/test/test_rwlock.c
index 4ae0bf8deb..e8086888fe 100644
--- a/app/test/test_rwlock.c
+++ b/app/test/test_rwlock.c
@@ -506,7 +506,7 @@ try_rwlock_test_rde_wro(void)
 	return process_try_lcore_stats();
 }
 
-REGISTER_TEST_COMMAND(rwlock_test1_autotest, rwlock_test1);
-REGISTER_TEST_COMMAND(rwlock_rda_autotest, try_rwlock_test_rda);
-REGISTER_TEST_COMMAND(rwlock_rds_wrm_autotest, try_rwlock_test_rds_wrm);
-REGISTER_TEST_COMMAND(rwlock_rde_wro_autotest, try_rwlock_test_rde_wro);
+REGISTER_FAST_TEST(rwlock_test1_autotest, True, True, rwlock_test1);
+REGISTER_FAST_TEST(rwlock_rda_autotest, True, True, try_rwlock_test_rda);
+REGISTER_FAST_TEST(rwlock_rds_wrm_autotest, True, True, try_rwlock_test_rds_wrm);
+REGISTER_FAST_TEST(rwlock_rde_wro_autotest, True, True, try_rwlock_test_rde_wro);
diff --git a/app/test/test_sched.c b/app/test/test_sched.c
index ddec572447..385ca45136 100644
--- a/app/test/test_sched.c
+++ b/app/test/test_sched.c
@@ -215,4 +215,4 @@ test_sched(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(sched_autotest, test_sched);
+REGISTER_FAST_TEST(sched_autotest, True, True, test_sched);
diff --git a/app/test/test_security.c b/app/test/test_security.c
index 4783cd0663..a3b962750a 100644
--- a/app/test/test_security.c
+++ b/app/test/test_security.c
@@ -2314,4 +2314,4 @@ test_security(void)
 	return unit_test_suite_runner(&security_testsuite);
 }
 
-REGISTER_TEST_COMMAND(security_autotest, test_security);
+REGISTER_FAST_TEST(security_autotest, False, True, test_security);
diff --git a/app/test/test_seqlock.c b/app/test/test_seqlock.c
index d26d2c010e..325af38070 100644
--- a/app/test/test_seqlock.c
+++ b/app/test/test_seqlock.c
@@ -187,4 +187,4 @@ test_seqlock(void)
 	return rc;
 }
 
-REGISTER_TEST_COMMAND(seqlock_autotest, test_seqlock);
+REGISTER_FAST_TEST(seqlock_autotest, True, True, test_seqlock);
diff --git a/app/test/test_service_cores.c b/app/test/test_service_cores.c
index 422d2a83e8..6c000417d2 100644
--- a/app/test/test_service_cores.c
+++ b/app/test/test_service_cores.c
@@ -1040,7 +1040,7 @@ test_service_common(void)
 	return unit_test_suite_runner(&service_tests);
 }
 
-REGISTER_TEST_COMMAND(service_autotest, test_service_common);
+REGISTER_FAST_TEST(service_autotest, True, True, test_service_common);
 
 static struct unit_test_suite service_perf_tests  = {
 	.suite_name = "service core performance test suite",
@@ -1062,4 +1062,4 @@ test_service_perf(void)
 	return unit_test_suite_runner(&service_perf_tests);
 }
 
-REGISTER_TEST_COMMAND(service_perf_autotest, test_service_perf);
+REGISTER_PERF_TEST(service_perf_autotest, test_service_perf);
diff --git a/app/test/test_spinlock.c b/app/test/test_spinlock.c
index 3f59372300..d655b606f7 100644
--- a/app/test/test_spinlock.c
+++ b/app/test/test_spinlock.c
@@ -302,4 +302,4 @@ test_spinlock(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(spinlock_autotest, test_spinlock);
+REGISTER_FAST_TEST(spinlock_autotest, True, True, test_spinlock);
diff --git a/app/test/test_stack.c b/app/test/test_stack.c
index bc38961433..93bba18469 100644
--- a/app/test/test_stack.c
+++ b/app/test/test_stack.c
@@ -379,5 +379,5 @@ test_lf_stack(void)
 #endif
 }
 
-REGISTER_TEST_COMMAND(stack_autotest, test_stack);
-REGISTER_TEST_COMMAND(stack_lf_autotest, test_lf_stack);
+REGISTER_FAST_TEST(stack_autotest, False, True, test_stack);
+REGISTER_FAST_TEST(stack_lf_autotest, False, True, test_lf_stack);
diff --git a/app/test/test_stack_perf.c b/app/test/test_stack_perf.c
index 1eae00a334..c5e1caa036 100644
--- a/app/test/test_stack_perf.c
+++ b/app/test/test_stack_perf.c
@@ -354,5 +354,5 @@ test_lf_stack_perf(void)
 #endif
 }
 
-REGISTER_TEST_COMMAND(stack_perf_autotest, test_stack_perf);
-REGISTER_TEST_COMMAND(stack_lf_perf_autotest, test_lf_stack_perf);
+REGISTER_PERF_TEST(stack_perf_autotest, test_stack_perf);
+REGISTER_PERF_TEST(stack_lf_perf_autotest, test_lf_stack_perf);
diff --git a/app/test/test_string_fns.c b/app/test/test_string_fns.c
index 5e105d2bb9..07e72bf680 100644
--- a/app/test/test_string_fns.c
+++ b/app/test/test_string_fns.c
@@ -182,4 +182,4 @@ test_string_fns(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(string_autotest, test_string_fns);
+REGISTER_FAST_TEST(string_autotest, True, True, test_string_fns);
diff --git a/app/test/test_tailq.c b/app/test/test_tailq.c
index 9520219b0a..5598bcd12d 100644
--- a/app/test/test_tailq.c
+++ b/app/test/test_tailq.c
@@ -125,4 +125,4 @@ test_tailq(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(tailq_autotest, test_tailq);
+REGISTER_FAST_TEST(tailq_autotest, True, True, test_tailq);
diff --git a/app/test/test_thash.c b/app/test/test_thash.c
index 53d9611e18..dd6468eb56 100644
--- a/app/test/test_thash.c
+++ b/app/test/test_thash.c
@@ -966,4 +966,4 @@ test_thash(void)
 	return unit_test_suite_runner(&thash_tests);
 }
 
-REGISTER_TEST_COMMAND(thash_autotest, test_thash);
+REGISTER_FAST_TEST(thash_autotest, True, True, test_thash);
diff --git a/app/test/test_thash_perf.c b/app/test/test_thash_perf.c
index 687582aa32..9dfd5d3c21 100644
--- a/app/test/test_thash_perf.c
+++ b/app/test/test_thash_perf.c
@@ -135,4 +135,4 @@ test_thash_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(thash_perf_autotest, test_thash_perf);
+REGISTER_PERF_TEST(thash_perf_autotest, test_thash_perf);
diff --git a/app/test/test_threads.c b/app/test/test_threads.c
index a4c4f651a4..6ed6b168d4 100644
--- a/app/test/test_threads.c
+++ b/app/test/test_threads.c
@@ -279,4 +279,4 @@ test_threads(void)
 	return unit_test_suite_runner(&threads_test_suite);
 }
 
-REGISTER_TEST_COMMAND(threads_autotest, test_threads);
+REGISTER_FAST_TEST(threads_autotest, True, True, test_threads);
diff --git a/app/test/test_ticketlock.c b/app/test/test_ticketlock.c
index 242c136478..25af697c0a 100644
--- a/app/test/test_ticketlock.c
+++ b/app/test/test_ticketlock.c
@@ -314,4 +314,4 @@ test_ticketlock(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(ticketlock_autotest, test_ticketlock);
+REGISTER_FAST_TEST(ticketlock_autotest, True, True, test_ticketlock);
diff --git a/app/test/test_timer.c b/app/test/test_timer.c
index 0c36dc9010..09608a1845 100644
--- a/app/test/test_timer.c
+++ b/app/test/test_timer.c
@@ -594,4 +594,4 @@ test_timer(void)
 	return TEST_SUCCESS;
 }
 
-REGISTER_TEST_COMMAND(timer_autotest, test_timer);
+REGISTER_FAST_TEST(timer_autotest, False, True, test_timer);
diff --git a/app/test/test_timer_perf.c b/app/test/test_timer_perf.c
index 0ede4b3e40..d2d74ebbc6 100644
--- a/app/test/test_timer_perf.c
+++ b/app/test/test_timer_perf.c
@@ -131,4 +131,4 @@ test_timer_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(timer_perf_autotest, test_timer_perf);
+REGISTER_PERF_TEST(timer_perf_autotest, test_timer_perf);
diff --git a/app/test/test_timer_racecond.c b/app/test/test_timer_racecond.c
index bb56ae8324..6f8b448ff8 100644
--- a/app/test/test_timer_racecond.c
+++ b/app/test/test_timer_racecond.c
@@ -172,4 +172,4 @@ test_timer_racecond(void)
 	return TEST_SUCCESS;
 }
 
-REGISTER_TEST_COMMAND(timer_racecond_autotest, test_timer_racecond);
+REGISTER_PERF_TEST(timer_racecond_autotest, test_timer_racecond);
diff --git a/app/test/test_trace.c b/app/test/test_trace.c
index ad4a394a29..96829caed5 100644
--- a/app/test/test_trace.c
+++ b/app/test/test_trace.c
@@ -250,4 +250,4 @@ test_trace(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(trace_autotest, test_trace);
+REGISTER_FAST_TEST(trace_autotest, True, True, test_trace);
diff --git a/app/test/test_trace_perf.c b/app/test/test_trace_perf.c
index 46ae7d8074..a6dd075722 100644
--- a/app/test/test_trace_perf.c
+++ b/app/test/test_trace_perf.c
@@ -179,4 +179,4 @@ test_trace_perf(void)
 	return TEST_SUCCESS;
 }
 
-REGISTER_TEST_COMMAND(trace_perf_autotest, test_trace_perf);
+REGISTER_PERF_TEST(trace_perf_autotest, test_trace_perf);
diff --git a/app/test/test_version.c b/app/test/test_version.c
index 1e1ff18656..21016237bc 100644
--- a/app/test/test_version.c
+++ b/app/test/test_version.c
@@ -25,4 +25,4 @@ test_version(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(version_autotest, test_version);
+REGISTER_FAST_TEST(version_autotest, True, True, test_version);
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v3 3/8] app/test: make telemetry data test buildable on windows
  2023-08-14 18:20 ` [PATCH v3 " Bruce Richardson
  2023-08-14 18:20   ` [PATCH v3 1/8] app/test: add new macros for various test types Bruce Richardson
  2023-08-14 18:20   ` [PATCH v3 2/8] app/test: tag tests with the test type Bruce Richardson
@ 2023-08-14 18:20   ` Bruce Richardson
  2023-08-14 18:21   ` [PATCH v3 4/8] app/test: build using per-file dependency matrix Bruce Richardson
                     ` (5 subsequent siblings)
  8 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-14 18:20 UTC (permalink / raw)
  To: dev; +Cc: ci, Bruce Richardson

To enable the building of the telemetry data tests file when building on
windows, we need to provide a stub implementation. That way, the test
file is buildable any time the library itself is built.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 app/test/test_telemetry_data.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/app/test/test_telemetry_data.c b/app/test/test_telemetry_data.c
index a960f44c1b..b2dc1d75db 100644
--- a/app/test/test_telemetry_data.c
+++ b/app/test/test_telemetry_data.c
@@ -2,11 +2,20 @@
  * Copyright 2020 Intel Corporation
  */
 
+#ifdef RTE_EXEC_ENV_WINDOWS
+#include "test.h"
+
+static int
+telemetry_data_autotest(void)
+{
+	return TEST_SKIPPED;
+}
+
+#else
+
 #include <string.h>
 #include <sys/socket.h>
-#ifndef RTE_EXEC_ENV_WINDOWS
 #include <sys/un.h>
-#endif
 #include <unistd.h>
 #include <limits.h>
 
@@ -604,5 +613,6 @@ telemetry_data_autotest(void)
 	close(sock);
 	return 0;
 }
+#endif /* windows/non-windows */
 
 REGISTER_TEST_COMMAND(telemetry_data_autotest, telemetry_data_autotest);
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v3 4/8] app/test: build using per-file dependency matrix
  2023-08-14 18:20 ` [PATCH v3 " Bruce Richardson
                     ` (2 preceding siblings ...)
  2023-08-14 18:20   ` [PATCH v3 3/8] app/test: make telemetry data test buildable on windows Bruce Richardson
@ 2023-08-14 18:21   ` Bruce Richardson
  2023-08-14 18:21   ` [PATCH v3 5/8] app/test: define unit tests suites based on test macros Bruce Richardson
                     ` (4 subsequent siblings)
  8 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-14 18:21 UTC (permalink / raw)
  To: dev; +Cc: ci, Bruce Richardson

Rather than using if-else constructs to selectively add or remove files
from the UT build, switch to a table-based approach where each file
lists out what libs or drivers it depends upon.

Initial version of this table was generated via analysis of the header
files included in each C file. The basic dependencies of the test binary
[cmdline, ring, mempool and mbuf] were then removed from the per-file
lists, as there is no point in checking them as the whole app will be
disabled if they are not present.

With the file list, the dependencies for the "utility" C-files are kept
separate, so that other tests which depend on the functions provided by
those files can have that dependency recorded properly. The basic
cryptodev tests also fall into this category as functions from the main
cryptodev test file are used by other crypto tests.

As well as the main table for internal dependencies, some test
files have separate external components too. A second, much smaller
table lists these dependencies.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 app/meson.build      |   8 +-
 app/test/meson.build | 823 ++++++++++++-------------------------------
 2 files changed, 238 insertions(+), 593 deletions(-)

diff --git a/app/meson.build b/app/meson.build
index 4fc1a83eba..0d8b618e7f 100644
--- a/app/meson.build
+++ b/app/meson.build
@@ -32,6 +32,11 @@ apps = [
         'test-security-perf',
 ]
 
+if get_option('tests')
+# build the auto test app if enabled.
+    apps += 'test'
+endif
+
 default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API']
 default_ldflags = []
 if get_option('default_library') == 'static' and not is_windows
@@ -106,6 +111,3 @@ foreach app:apps
             install_rpath: join_paths(get_option('prefix'), driver_install_path),
             install: true)
 endforeach
-
-# special case the autotests
-subdir('test')
diff --git a/app/test/meson.build b/app/test/meson.build
index 66897c14a3..2b885ae273 100644
--- a/app/test/meson.build
+++ b/app/test/meson.build
@@ -1,446 +1,230 @@
 # SPDX-License-Identifier: BSD-3-Clause
-# Copyright(c) 2017 Intel Corporation
-
-if not get_option('tests')
-    subdir_done()
-endif
-
-test_sources = files(
-        'commands.c',
-        'packet_burst_generator.c',
-        'test.c',
-        'test_acl.c',
-        'test_alarm.c',
-        'test_atomic.c',
-        'test_barrier.c',
-        'test_bitops.c',
-        'test_bitmap.c',
-        'test_bpf.c',
-        'test_byteorder.c',
-        'test_cksum.c',
-        'test_cksum_perf.c',
-        'test_cmdline.c',
-        'test_cmdline_cirbuf.c',
-        'test_cmdline_etheraddr.c',
-        'test_cmdline_ipaddr.c',
-        'test_cmdline_lib.c',
-        'test_cmdline_num.c',
-        'test_cmdline_portlist.c',
-        'test_cmdline_string.c',
-        'test_common.c',
-        'test_cpuflags.c',
-        'test_crc.c',
-        'test_cryptodev.c',
-        'test_cryptodev_asym.c',
-        'test_cryptodev_blockcipher.c',
-        'test_cryptodev_crosscheck.c',
-        'test_cryptodev_security_ipsec.c',
-        'test_cryptodev_security_pdcp.c',
-        'test_cycles.c',
-        'test_debug.c',
-        'test_devargs.c',
-        'test_distributor.c',
-        'test_distributor_perf.c',
-        'test_dmadev.c',
-        'test_dmadev_api.c',
-        'test_eal_flags.c',
-        'test_eal_fs.c',
-        'test_efd.c',
-        'test_efd_perf.c',
-        'test_errno.c',
-        'test_ethdev_link.c',
-        'test_event_crypto_adapter.c',
-        'test_event_eth_rx_adapter.c',
-        'test_event_ring.c',
-        'test_event_timer_adapter.c',
-        'test_eventdev.c',
-        'test_external_mem.c',
-        'test_fbarray.c',
-        'test_fib.c',
-        'test_fib_perf.c',
-        'test_fib6.c',
-        'test_fib6_perf.c',
-        'test_func_reentrancy.c',
-        'test_hash.c',
-        'test_hash_functions.c',
-        'test_hash_multiwriter.c',
-        'test_hash_readwrite.c',
-        'test_hash_perf.c',
-        'test_hash_readwrite_lf_perf.c',
-        'test_interrupts.c',
-        'test_ipfrag.c',
-        'test_ipsec.c',
-        'test_ipsec_sad.c',
-        'test_ipsec_perf.c',
-        'test_kvargs.c',
-        'test_lcores.c',
-        'test_logs.c',
-        'test_lpm.c',
-        'test_lpm6.c',
-        'test_lpm6_perf.c',
-        'test_lpm_perf.c',
-        'test_malloc.c',
-        'test_malloc_perf.c',
-        'test_mbuf.c',
-        'test_member.c',
-        'test_member_perf.c',
-        'test_memcpy.c',
-        'test_memcpy_perf.c',
-        'test_memory.c',
-        'test_mempool.c',
-        'test_mempool_perf.c',
-        'test_memzone.c',
-        'test_meter.c',
-        'test_mcslock.c',
-        'test_mp_secondary.c',
-        'test_per_lcore.c',
-        'test_pflock.c',
-        'test_pmd_perf.c',
-        'test_power.c',
-        'test_power_cpufreq.c',
-        'test_power_kvm_vm.c',
-        'test_power_intel_uncore.c',
-        'test_prefetch.c',
-        'test_rand_perf.c',
-        'test_rawdev.c',
-        'test_rcu_qsbr.c',
-        'test_rcu_qsbr_perf.c',
-        'test_reassembly_perf.c',
-        'test_reciprocal_division.c',
-        'test_reciprocal_division_perf.c',
-        'test_red.c',
-        'test_pie.c',
-        'test_reorder.c',
-        'test_rib.c',
-        'test_rib6.c',
-        'test_ring.c',
-        'test_ring_mpmc_stress.c',
-        'test_ring_hts_stress.c',
-        'test_ring_mt_peek_stress.c',
-        'test_ring_mt_peek_stress_zc.c',
-        'test_ring_perf.c',
-        'test_ring_rts_stress.c',
-        'test_ring_st_peek_stress.c',
-        'test_ring_st_peek_stress_zc.c',
-        'test_ring_stress.c',
-        'test_rwlock.c',
-        'test_sched.c',
-        'test_security.c',
-        'test_security_inline_macsec.c',
-        'test_security_inline_proto.c',
-        'test_seqlock.c',
-        'test_service_cores.c',
-        'test_spinlock.c',
-        'test_stack.c',
-        'test_stack_perf.c',
-        'test_string_fns.c',
-        'test_tailq.c',
-        'test_thash.c',
-        'test_thash_perf.c',
-        'test_threads.c',
-        'test_timer.c',
-        'test_timer_perf.c',
-        'test_timer_racecond.c',
-        'test_timer_secondary.c',
-        'test_ticketlock.c',
-        'test_trace.c',
-        'test_trace_register.c',
-        'test_trace_perf.c',
-        'test_version.c',
-        'virtual_pmd.c',
-)
-
-test_deps = dpdk_libs_enabled
-# as well as libs, the pci and vdev bus drivers are needed for a lot of tests
-test_deps += ['bus_pci', 'bus_vdev']
-
-# Each test is marked with flags:
-# - the first flag indicates whether the test can run in no-huge mode,
-# - the second flag indicates whether the test can run with ASan enabled,
-fast_tests = [
-        ['acl_autotest', true, true],
-        ['atomic_autotest', false, true],
-        ['bitmap_autotest', true, true],
-        ['bpf_autotest', true, true],
-        ['bpf_convert_autotest', true, true],
-        ['bitops_autotest', true, true],
-        ['byteorder_autotest', true, true],
-        ['cksum_autotest', true, true],
-        ['cmdline_autotest', true, true],
-        ['common_autotest', true, true],
-        ['cpuflags_autotest', true, true],
-        ['debug_autotest', true, true],
-        ['devargs_autotest', true, true],
-        ['eal_flags_c_opt_autotest', false, false],
-        ['eal_flags_main_opt_autotest', false, false],
-        ['eal_flags_n_opt_autotest', false, false],
-        ['eal_flags_hpet_autotest', false, false],
-        ['eal_flags_no_huge_autotest', false, false],
-        ['eal_flags_a_opt_autotest', false, false],
-        ['eal_flags_b_opt_autotest', false, false],
-        ['eal_flags_vdev_opt_autotest', false, false],
-        ['eal_flags_r_opt_autotest', false, false],
-        ['eal_flags_mem_autotest', false, false],
-        ['eal_flags_file_prefix_autotest', false, false],
-        ['eal_flags_misc_autotest', false, false],
-        ['eal_fs_autotest', true, true],
-        ['errno_autotest', true, true],
-        ['ethdev_link_status', true, true],
-        ['event_ring_autotest', true, true],
-        ['fib_autotest', true, true],
-        ['fib6_autotest', true, true],
-        ['func_reentrancy_autotest', false, true],
-        ['hash_autotest', true, true],
-        ['interrupt_autotest', true, true],
-        ['ipfrag_autotest', false, true],
-        ['lcores_autotest', true, true],
-        ['logs_autotest', true, true],
-        ['lpm_autotest', true, true],
-        ['lpm6_autotest', true, true],
-        ['malloc_autotest', false, true],
-        ['mbuf_autotest', false, true],
-        ['mcslock_autotest', false, true],
-        ['memcpy_autotest', true, true],
-        ['memory_autotest', false, true],
-        ['mempool_autotest', false, true],
-        ['memzone_autotest', false, true],
-        ['meter_autotest', true, true],
-        ['multiprocess_autotest', false, false],
-        ['per_lcore_autotest', true, true],
-        ['pflock_autotest', true, true],
-        ['prefetch_autotest', true, true],
-        ['rcu_qsbr_autotest', true, true],
-        ['pie_autotest', true, true],
-        ['rib_autotest', true, true],
-        ['rib6_autotest', true, true],
-        ['ring_autotest', true, true],
-        ['rwlock_test1_autotest', true, true],
-        ['rwlock_rda_autotest', true, true],
-        ['rwlock_rds_wrm_autotest', true, true],
-        ['rwlock_rde_wro_autotest', true, true],
-        ['sched_autotest', true, true],
-        ['security_autotest', false, true],
-        ['seqlock_autotest', true, true],
-        ['spinlock_autotest', true, true],
-        ['stack_autotest', false, true],
-        ['stack_lf_autotest', false, true],
-        ['string_autotest', true, true],
-        ['tailq_autotest', true, true],
-        ['ticketlock_autotest', true, true],
-        ['timer_autotest', false, true],
-        ['user_delay_us', true, true],
-        ['version_autotest', true, true],
-        ['crc_autotest', true, true],
-        ['distributor_autotest', false, true],
-        ['eventdev_common_autotest', true, true],
-        ['fbarray_autotest', true, true],
-        ['hash_readwrite_func_autotest', false, true],
-        ['ipsec_autotest', true, true],
-        ['kvargs_autotest', true, true],
-        ['member_autotest', true, true],
-        ['power_cpufreq_autotest', false, true],
-        ['power_autotest', true, true],
-        ['power_kvm_vm_autotest', false, true],
-        ['power_intel_uncore_autotest', true, true],
-        ['reorder_autotest', true, true],
-        ['service_autotest', true, true],
-        ['thash_autotest', true, true],
-        ['threads_autotest', true, true],
-        ['trace_autotest', true, true],
-]
-
-# Tests known to have issues or which don't belong in other tests lists.
-extra_test_names = [
-        'alarm_autotest', # ee00af60170b ("test: remove strict timing requirements some tests")
-        'red_autotest', # https://bugs.dpdk.org/show_bug.cgi?id=826
-]
-
-perf_test_names = [
-        'ring_perf_autotest',
-        'malloc_perf_autotest',
-        'mempool_perf_autotest',
-        'memcpy_perf_autotest',
-        'hash_perf_autotest',
-        'timer_perf_autotest',
-        'reciprocal_division',
-        'reciprocal_division_perf',
-        'lpm_perf_autotest',
-        'rib_slow_autotest',
-        'fib_slow_autotest',
-        'fib_perf_autotest',
-        'red_all',
-        'pie_all',
-        'barrier_autotest',
-        'hash_multiwriter_autotest',
-        'timer_racecond_autotest',
-        'efd_autotest',
-        'hash_functions_autotest',
-        'member_perf_autotest',
-        'efd_perf_autotest',
-        'lpm6_perf_autotest',
-        'rib6_slow_autotest',
-        'fib6_slow_autotest',
-        'fib6_perf_autotest',
-        'rcu_qsbr_perf_autotest',
-        'red_perf',
-        'pie_perf',
-        'distributor_perf_autotest',
-        'pmd_perf_autotest',
-        'service_perf_autotest',
-        'stack_perf_autotest',
-        'stack_lf_perf_autotest',
-        'rand_perf_autotest',
-        'hash_readwrite_perf_autotest',
-        'hash_readwrite_lf_perf_autotest',
-        'trace_perf_autotest',
-        'ipsec_perf_autotest',
-        'thash_perf_autotest',
-        'reassembly_perf_autotest',
-]
-
-driver_test_names = [
-        'cryptodev_aesni_gcm_autotest',
-        'cryptodev_aesni_mb_autotest',
-        'cryptodev_chacha_poly_mb_autotest',
-        'cryptodev_cn10k_autotest',
-        'cryptodev_cn9k_autotest',
-        'cryptodev_cpu_aesni_mb_autotest',
-        'cryptodev_cpu_aesni_gcm_autotest',
-        'cryptodev_dpaa2_sec_autotest',
-        'cryptodev_dpaa_sec_autotest',
-        'cryptodev_null_autotest',
-        'cryptodev_openssl_autotest',
-        'cryptodev_qat_autotest',
-        'cryptodev_qat_asym_autotest',
-        'cryptodev_qat_raw_api_autotest',
-        'cryptodev_sw_armv8_autotest',
-        'cryptodev_sw_kasumi_autotest',
-        'cryptodev_sw_mvsam_autotest',
-        'cryptodev_sw_snow3g_autotest',
-        'cryptodev_sw_zuc_autotest',
-        'cryptodev_uadk_autotest',
-        'dmadev_autotest',
-]
-
-dump_test_names = []
-
-if not is_windows
-    driver_test_names += [
-            'cryptodev_openssl_asym_autotest',
-            'eventdev_selftest_octeontx',
-            'eventdev_selftest_sw',
-    ]
-
-    dump_test_names += [
-            'dump_struct_sizes',
-            'dump_mempool',
-            'dump_malloc_stats',
-            'dump_devargs',
-            'dump_log_types',
-            'dump_ring',
-            'dump_physmem',
-            'dump_memzone',
-    ]
-endif
-
-# The following linkages are an exception to allow running the
-# unit tests without requiring that the developer install the
-# DPDK libraries.  Explicit linkage of drivers (plugin libraries)
-# in applications should not be used.
-if dpdk_conf.has('RTE_MEMPOOL_RING')
-    test_deps += 'mempool_ring'
-endif
-if dpdk_conf.has('RTE_MEMPOOL_STACK')
-    test_deps += 'mempool_stack'
-endif
-if dpdk_conf.has('RTE_EVENT_SKELETON')
-    test_deps += 'event_skeleton'
-endif
-
-if dpdk_conf.has('RTE_LIB_GRAPH')
-    test_sources += 'test_graph.c'
-    fast_tests += [['graph_autotest', true, true]]
-    fast_tests += [['node_list_dump', true, true]]
-    test_sources += 'test_graph_perf.c'
-    perf_test_names += 'graph_perf_autotest'
-endif
-if dpdk_conf.has('RTE_LIB_METRICS')
-    test_sources += ['test_metrics.c']
-    fast_tests += [['metrics_autotest', true, true]]
-endif
-if not is_windows and dpdk_conf.has('RTE_LIB_TELEMETRY')
-    test_sources += ['test_telemetry_json.c', 'test_telemetry_data.c']
-    fast_tests += [['telemetry_json_autotest', true, true]]
-    fast_tests += [['telemetry_data_autotest', true, true]]
-endif
-if dpdk_conf.has('RTE_LIB_PIPELINE')
-# pipeline lib depends on port and table libs, so those must be present
-# if pipeline library is.
-    test_sources += [
-            'test_table.c',
-            'test_table_acl.c',
-            'test_table_combined.c',
-            'test_table_pipeline.c',
-            'test_table_ports.c',
-            'test_table_tables.c',
-    ]
-    fast_tests += [['table_autotest', true, true]]
-endif
-
-# The following linkages of drivers are required because
-# they are used via a driver-specific API.
-if dpdk_conf.has('RTE_NET_BOND')
-    test_deps += 'net_bond'
-    test_sources += ['test_link_bonding.c', 'test_link_bonding_rssconf.c']
-    driver_test_names += ['link_bonding_autotest', 'link_bonding_rssconf_autotest']
-    if dpdk_conf.has('RTE_NET_RING')
-        test_sources += 'test_link_bonding_mode4.c'
-        driver_test_names += 'link_bonding_mode4_autotest'
-    endif
-endif
-if dpdk_conf.has('RTE_LIB_EVENTDEV') and dpdk_conf.has('RTE_NET_RING')
-    test_deps += 'net_ring'
-    test_sources += 'test_pmd_ring_perf.c'
-    test_sources += 'test_pmd_ring.c'
-    test_sources += 'test_event_eth_tx_adapter.c'
-    test_sources += 'sample_packet_forward.c'
-    fast_tests += [['ring_pmd_autotest', true, true]]
-    perf_test_names += 'ring_pmd_perf_autotest'
-    fast_tests += [['event_eth_tx_adapter_autotest', false, true]]
-    if dpdk_conf.has('RTE_LIB_BITRATESTATS')
-        test_sources += 'test_bitratestats.c'
-        fast_tests += [['bitratestats_autotest', true, true]]
-    endif
-    if dpdk_conf.has('RTE_LIB_LATENCYSTATS')
-        test_sources += 'test_latencystats.c'
-        fast_tests += [['latencystats_autotest', true, true]]
-    endif
-    if dpdk_conf.has('RTE_LIB_PDUMP')
-        test_sources += 'test_pdump.c'
-        fast_tests += [['pdump_autotest', true, false]]
-    endif
-endif
-if dpdk_conf.has('RTE_NET_NULL')
-    test_deps += 'net_null'
-    test_sources += 'test_vdev.c'
-    fast_tests += [['vdev_autotest', true, true]]
-endif
-if dpdk_conf.has('RTE_RAW_SKELETON')
-    test_deps += 'raw_skeleton'
-    fast_tests += [['rawdev_autotest', true, true]]
-endif
-
-if dpdk_conf.has('RTE_HAS_LIBPCAP')
-    ext_deps += pcap_dep
-    if dpdk_conf.has('RTE_LIB_PCAPNG')
-        test_sources += 'test_pcapng.c'
+# Copyright(c) 2017-2023 Intel Corporation
+
+# the main test files [test.c and commands.c] relies on these libraries
+deps += ['cmdline', 'ring', 'mempool', 'mbuf']
+sources += files('commands.c', 'test.c')
+
+# some other utility C files, providing functions used by various tests
+# so we need to include these deps in the dependency list for the files using those fns.
+packet_burst_generator_deps = ['net']
+sample_packet_forward_deps = ['net_ring', 'ethdev', 'bus_vdev']
+virtual_pmd_deps = ['ethdev', 'net', 'bus_pci']
+# test_cryptodev has material that other crypto tests need
+test_cryptodev_deps = ['bus_vdev', 'net', 'cryptodev', 'crypto_scheduler', 'security']
+
+source_file_deps = {
+    # The C files providing functionality to other test cases
+    'packet_burst_generator.c': packet_burst_generator_deps,
+#    'resource.c': [],          # unused currently.
+    'sample_packet_forward.c': sample_packet_forward_deps,
+    'virtual_pmd.c': virtual_pmd_deps,
+
+    # the various test_*.c files
+    'test_acl.c': ['net', 'acl'],
+    'test_alarm.c': [],
+    'test_atomic.c': ['hash'],
+    'test_barrier.c': [],
+    'test_bitmap.c': [],
+    'test_bitops.c': [],
+    'test_bitratestats.c': ['metrics', 'bitratestats', 'ethdev'] + sample_packet_forward_deps,
+    'test_bpf.c': ['bpf', 'net'],
+    'test_byteorder.c': [],
+#    'test_cfgfile.c': ['cfgfile'],
+    'test_cksum.c': ['net'],
+    'test_cksum_perf.c': ['net'],
+    'test_cmdline.c': [],
+    'test_cmdline_cirbuf.c': [],
+    'test_cmdline_etheraddr.c': ['net'],
+    'test_cmdline_ipaddr.c': [],
+    'test_cmdline_lib.c': [],
+    'test_cmdline_num.c': [],
+    'test_cmdline_portlist.c': [],
+    'test_cmdline_string.c': [],
+    'test_common.c': [],
+    'test_compressdev.c': ['compressdev'],
+    'test_cpuflags.c': [],
+    'test_crc.c': ['net'],
+    'test_cryptodev.c': test_cryptodev_deps,
+    'test_cryptodev_asym.c': ['bus_vdev'] + test_cryptodev_deps,
+    'test_cryptodev_blockcipher.c': test_cryptodev_deps,
+    'test_cryptodev_crosscheck.c': test_cryptodev_deps,
+    'test_cryptodev_security_ipsec.c': test_cryptodev_deps,
+    'test_cryptodev_security_pdcp.c': test_cryptodev_deps,
+    'test_cycles.c': [],
+    'test_debug.c': [],
+    'test_devargs.c': ['kvargs'],
+    'test_distributor.c': ['distributor'],
+    'test_distributor_perf.c': ['distributor'],
+    'test_dmadev.c': ['dmadev', 'bus_vdev'],
+    'test_dmadev_api.c': ['dmadev'],
+    'test_eal_flags.c': [],
+    'test_eal_fs.c': [],
+    'test_efd.c': ['efd', 'net'],
+    'test_efd_perf.c': ['efd', 'hash'],
+    'test_errno.c': [],
+    'test_ethdev_link.c': ['ethdev'],
+    'test_event_crypto_adapter.c': ['cryptodev', 'eventdev', 'bus_vdev'],
+    'test_event_eth_rx_adapter.c': ['ethdev', 'eventdev', 'bus_vdev'],
+    'test_event_eth_tx_adapter.c': ['bus_vdev', 'ethdev', 'net_ring', 'eventdev'],
+    'test_event_ring.c': ['eventdev'],
+    'test_event_timer_adapter.c': ['ethdev', 'eventdev', 'bus_vdev'],
+    'test_eventdev.c': ['eventdev', 'bus_vdev'],
+    'test_external_mem.c': [],
+    'test_fbarray.c': [],
+    'test_fib.c': ['net', 'fib'],
+    'test_fib6.c': ['rib', 'fib'],
+    'test_fib6_perf.c': ['fib'],
+    'test_fib_perf.c': ['net', 'fib'],
+    'test_flow_classify.c': ['net', 'acl', 'table', 'ethdev', 'flow_classify'],
+    'test_func_reentrancy.c': ['hash', 'lpm'],
+    'test_graph.c': ['graph'],
+    'test_graph_perf.c': ['graph'],
+    'test_hash.c': ['net', 'hash'],
+    'test_hash_functions.c': ['hash'],
+    'test_hash_multiwriter.c': ['hash'],
+    'test_hash_perf.c': ['hash'],
+    'test_hash_readwrite.c': ['hash'],
+    'test_hash_readwrite_lf_perf.c': ['hash'],
+    'test_interrupts.c': [],
+    'test_ipfrag.c': ['net', 'ip_frag'],
+    'test_ipsec.c': ['bus_vdev', 'net', 'cryptodev', 'ipsec', 'security'],
+    'test_ipsec_perf.c': ['net', 'ipsec'],
+    'test_ipsec_sad.c': ['ipsec'],
+    'test_kvargs.c': ['kvargs'],
+    'test_latencystats.c': ['ethdev', 'latencystats', 'metrics'] + sample_packet_forward_deps,
+    'test_lcores.c': [],
+    'test_link_bonding.c': ['ethdev', 'net_bond',
+        'net'] + packet_burst_generator_deps + virtual_pmd_deps,
+    'test_link_bonding_mode4.c': ['ethdev', 'net_ring', 'net_bond',
+        'net'] + packet_burst_generator_deps,
+    'test_link_bonding_rssconf.c': ['ethdev', 'bus_vdev', 'net_bond'],
+    'test_logs.c': [],
+    'test_lpm.c': ['net', 'lpm'],
+    'test_lpm6.c': ['lpm'],
+    'test_lpm6_perf.c': ['lpm'],
+    'test_lpm_perf.c': ['net', 'lpm'],
+    'test_malloc.c': [],
+    'test_malloc_perf.c': [],
+    'test_mbuf.c': ['net'],
+    'test_mcslock.c': [],
+    'test_member.c': ['member', 'net'],
+    'test_member_perf.c': ['hash', 'member'],
+    'test_memcpy.c': [],
+    'test_memcpy_perf.c': [],
+    'test_memory.c': [],
+    'test_mempool.c': [],
+    'test_mempool_perf.c': [],
+    'test_memzone.c': [],
+    'test_meter.c': ['meter'],
+    'test_metrics.c': ['metrics'],
+    'test_mp_secondary.c': ['hash', 'lpm'],
+    'test_pcapng.c': ['ethdev', 'net', 'pcapng'],
+    'test_pdcp.c': ['eventdev', 'pdcp', 'net', 'timer', 'security'],
+    'test_pdump.c': ['pdump'] + sample_packet_forward_deps,
+    'test_per_lcore.c': [],
+    'test_pflock.c': [],
+    'test_pie.c': ['sched'],
+    'test_pmd_perf.c': ['ethdev', 'net'] + packet_burst_generator_deps,
+    'test_pmd_ring.c': ['net_ring', 'ethdev', 'bus_vdev'],
+    'test_pmd_ring_perf.c': ['ethdev', 'net_ring', 'bus_vdev'],
+    'test_power.c': ['power'],
+    'test_power_cpufreq.c': ['power'],
+    'test_power_intel_uncore.c': ['power'],
+    'test_power_kvm_vm.c': ['power'],
+    'test_prefetch.c': [],
+    'test_rand_perf.c': [],
+    'test_rawdev.c': ['rawdev', 'bus_vdev'],
+    'test_rcu_qsbr.c': ['rcu', 'hash'],
+    'test_rcu_qsbr_perf.c': ['rcu', 'hash'],
+    'test_reassembly_perf.c': ['net', 'ip_frag'],
+    'test_reciprocal_division.c': [],
+    'test_reciprocal_division_perf.c': [],
+    'test_red.c': ['sched'],
+    'test_reorder.c': ['reorder'],
+#    'test_resource.c': [],
+    'test_rib.c': ['net', 'rib'],
+    'test_rib6.c': ['net', 'rib'],
+    'test_ring.c': [],
+    'test_ring_hts_stress.c': [],
+    'test_ring_mpmc_stress.c': [],
+    'test_ring_mt_peek_stress.c': [],
+    'test_ring_mt_peek_stress_zc.c': [],
+    'test_ring_perf.c': [],
+    'test_ring_rts_stress.c': [],
+    'test_ring_st_peek_stress.c': [],
+    'test_ring_st_peek_stress_zc.c': [],
+    'test_ring_stress.c': [],
+    'test_rwlock.c': [],
+    'test_sched.c': ['net', 'sched'],
+    'test_security.c': ['net', 'security'],
+    'test_security_inline_macsec.c': ['ethdev', 'security'],
+    'test_security_inline_proto.c': ['ethdev', 'security', 'eventdev'] + test_cryptodev_deps,
+    'test_seqlock.c': [],
+    'test_service_cores.c': [],
+    'test_spinlock.c': [],
+    'test_stack.c': ['stack'],
+    'test_stack_perf.c': ['stack'],
+    'test_string_fns.c': [],
+    'test_table.c': ['table', 'pipeline', 'port'],
+    'test_table_acl.c': ['net', 'table', 'pipeline', 'port'],
+    'test_table_combined.c': ['table', 'pipeline', 'port'],
+    'test_table_pipeline.c': ['pipeline', 'table', 'port'],
+    'test_table_ports.c': ['table', 'pipeline', 'port'],
+    'test_table_tables.c': ['table', 'pipeline', 'port'],
+    'test_tailq.c': [],
+    'test_telemetry_data.c': ['telemetry'],
+    'test_telemetry_json.c': ['telemetry'],
+    'test_thash.c': ['net', 'hash'],
+    'test_thash_perf.c': ['hash'],
+    'test_threads.c': [],
+    'test_ticketlock.c': [],
+    'test_timer.c': ['timer'],
+    'test_timer_perf.c': ['timer'],
+    'test_timer_racecond.c': ['timer'],
+    'test_timer_secondary.c': ['timer'],
+    'test_trace.c': [],
+    'test_trace_perf.c': [],
+    'test_trace_register.c': [],
+    'test_vdev.c': ['kvargs', 'bus_vdev'],
+    'test_version.c': [],
+}
+
+source_file_ext_deps = {
+    'test_compressdev.c': ['zlib'],
+    'test_pcapng.c': ['pcap'],
+}
+
+def_lib = get_option('default_library')
+foreach f, f_deps : source_file_deps
+    has_deps = true
+    foreach d : f_deps
+        if not is_variable(def_lib + '_rte_' + d)
+            has_deps = false
+            break
+        else
+            # technically we might not need this dep, but adding it is harmless
+            if d not in deps
+                deps += d
+            endif
+        endif
+    endforeach
+    # check for any external dependencies for this file
+    if source_file_ext_deps.has_key(f)
+        foreach d: source_file_ext_deps.get(f)
+            dep = dependency(d, required: false, method: 'pkg-config')
+            if not dep.found()
+                message('Skipping test file @0@ due to missing external dependency @1@'.format(f, d))
+                has_deps = false
+            else
+                ext_deps += dep
+            endif
+        endforeach
+    endif
+    if has_deps
+        sources += files(f)
     endif
-endif
-
-if dpdk_conf.has('RTE_LIB_PDCP')
-    test_sources += 'test_pdcp.c'
-    fast_tests += [['pdcp_autotest', false, true]]
-endif
+endforeach
 
 if cc.has_argument('-Wno-format-truncation')
     cflags += '-Wno-format-truncation'
@@ -450,154 +234,13 @@ endif
 cflags += '-fno-strict-aliasing'
 
 # Enable using internal APIs in unit tests
-cflags += ['-DALLOW_INTERNAL_API']
-
-test_dep_objs = []
-if dpdk_conf.has('RTE_LIB_COMPRESSDEV')
-    compress_test_dep = dependency('zlib', required: false, method: 'pkg-config')
-    if compress_test_dep.found()
-        test_dep_objs += compress_test_dep
-        test_sources += 'test_compressdev.c'
-        fast_tests += [['compressdev_autotest', false, true]]
-    endif
-endif
-
-if dpdk_conf.has('RTE_CRYPTO_SCHEDULER')
-    driver_test_names += 'cryptodev_scheduler_autotest'
-    test_deps += 'crypto_scheduler'
-endif
-
-foreach d:test_deps
-    def_lib = get_option('default_library')
-    test_dep_objs += get_variable(def_lib + '_rte_' + d)
-endforeach
-
-link_libs = []
-if get_option('default_library') == 'static'
-    link_libs = dpdk_static_libraries + dpdk_drivers
-endif
-
-dpdk_test = executable('dpdk-test',
-        test_sources,
-        link_whole: link_libs,
-        dependencies: test_dep_objs + ext_deps,
-        c_args: cflags,
-        install_rpath: join_paths(get_option('prefix'),
-             driver_install_path),
-        install: true)
-
-has_hugepage = run_command(py3, files('has_hugepage.py'), check: true).stdout().strip() != '0'
-message('hugepage availability: @0@'.format(has_hugepage))
-
-# some perf tests (eg: memcpy perf autotest)take very long
-# to complete, so timeout to 10 minutes
-timeout_seconds = 600
-timeout_seconds_fast = 10
-
-test_no_huge_args = ['--no-huge', '-m', '2048']
-
-foreach arg : fast_tests
-    test_args = []
-    run_test = true
-    if not has_hugepage
-        if arg[1]
-            test_args += test_no_huge_args
-        else
-            run_test = false
-        endif
-    endif
-
-    if get_option('b_sanitize') == 'address' or get_option('b_sanitize') == 'address,undefined'
-        if not arg[2]
-            run_test = false
-        endif
-    endif
-
-    if (get_option('default_library') == 'shared' and
-        arg[0] == 'event_eth_tx_adapter_autotest')
-        test_args += ['-d', dpdk_drivers_build_dir]
-    endif
-    if is_linux
-        test_args += ['--file-prefix=@0@'.format(arg[0])]
-    endif
-
-    if run_test
-        test(arg[0], dpdk_test,
-                env : ['DPDK_TEST=' + arg[0]],
-                args : test_args,
-                timeout : timeout_seconds_fast,
-                is_parallel : false,
-                suite : 'fast-tests')
-        if not is_windows and arg[0] == 'trace_autotest'
-            test_args += ['--trace=.*']
-            test_args += ['--trace-dir=@0@'.format(meson.current_build_dir())]
-            test(arg[0] + '_with_traces', dpdk_test,
-                    env : ['DPDK_TEST=' + arg[0]],
-                    args : test_args,
-                    timeout : timeout_seconds_fast,
-                    is_parallel : false,
-                    suite : 'fast-tests')
-        endif
-    endif
-endforeach
+cflags += '-DALLOW_INTERNAL_API'
 
-if not is_windows and dpdk_conf.has('RTE_LIB_TELEMETRY')
-    test_args = [dpdk_test]
-    test_args += test_no_huge_args
-    if get_option('default_library') == 'shared'
-        test_args += ['-d', dpdk_drivers_build_dir]
-    endif
-    if dpdk_conf.has('RTE_CRYPTO_NULL')
-        test_args += ['--vdev=crypto_null0']
-    endif
-    if dpdk_conf.has('RTE_DMA_SKELETON')
-        test_args += ['--vdev=dma_skeleton0']
-    endif
-    if dpdk_conf.has('RTE_EVENT_SKELETON')
-        test_args += ['--vdev=event_skeleton0']
-    endif
-    if dpdk_conf.has('RTE_NET_NULL')
-        test_args += ['--vdev=net_null0']
-    endif
-    if dpdk_conf.has('RTE_RAW_SKELETON')
-        test_args += ['--vdev=rawdev_skeleton0']
-    endif
-    test_args += ['-a', '0000:00:00.0']
-    test('telemetry_all', find_program('test_telemetry.sh'),
-            args: test_args,
-            timeout : timeout_seconds_fast,
-            is_parallel : false,
-            suite : 'fast-tests')
+# create a symlink in the app/test directory for the binary, for backward compatibility
+if not is_windows
+    custom_target('test_symlink',
+            output: 'dpdk-test',
+            command: ['ln', '-sf', '../dpdk-test', '@OUTPUT@'],
+            build_by_default: true,
+            install: false)
 endif
-
-foreach arg : perf_test_names
-    test(arg, dpdk_test,
-            env : ['DPDK_TEST=' + arg],
-            timeout : timeout_seconds,
-            is_parallel : false,
-            suite : 'perf-tests')
-endforeach
-
-foreach arg : driver_test_names
-    test(arg, dpdk_test,
-            env : ['DPDK_TEST=' + arg],
-            timeout : timeout_seconds,
-            is_parallel : false,
-            suite : 'driver-tests')
-endforeach
-
-foreach arg : dump_test_names
-    test(arg, dpdk_test,
-            env : ['DPDK_TEST=' + arg],
-            timeout : timeout_seconds,
-            is_parallel : false,
-            suite : 'debug-tests')
-endforeach
-
-foreach arg : extra_test_names
-    test(arg, dpdk_test,
-            env : ['DPDK_TEST=' + arg],
-            timeout : timeout_seconds,
-            is_parallel : false,
-            suite : 'extra-tests')
-endforeach
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v3 5/8] app/test: define unit tests suites based on test macros
  2023-08-14 18:20 ` [PATCH v3 " Bruce Richardson
                     ` (3 preceding siblings ...)
  2023-08-14 18:21   ` [PATCH v3 4/8] app/test: build using per-file dependency matrix Bruce Richardson
@ 2023-08-14 18:21   ` Bruce Richardson
  2023-08-14 18:21   ` [PATCH v3 6/8] examples/l3fwd: make eventdev an optional dependency Bruce Richardson
                     ` (3 subsequent siblings)
  8 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-14 18:21 UTC (permalink / raw)
  To: dev; +Cc: ci, Bruce Richardson

Rather than having the test suites listed out in the meson.build files
and having to have them enabled/disabled selectively based on what libs
are being built, pull the tests to run from the source files which were
added to the build.

Most test suites require no additional info other than the list of test
names in the suite. However the fast-test are special that they have
additional parameters associated with them. This requires some
additional work in the test extraction script and in the meson.build
file for processing the output.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 app/meson.build                               |  7 +-
 app/test/suites/meson.build                   | 76 +++++++++++++++++++
 buildtools/get-test-suites.py                 | 33 ++++++++
 .../has-hugepages.py                          |  0
 buildtools/meson.build                        |  2 +
 5 files changed, 117 insertions(+), 1 deletion(-)
 create mode 100644 app/test/suites/meson.build
 create mode 100644 buildtools/get-test-suites.py
 rename app/test/has_hugepage.py => buildtools/has-hugepages.py (100%)

diff --git a/app/meson.build b/app/meson.build
index 0d8b618e7f..c14dc80892 100644
--- a/app/meson.build
+++ b/app/meson.build
@@ -101,7 +101,7 @@ foreach app:apps
         link_libs = dpdk_static_libraries + dpdk_drivers
     endif
 
-    executable('dpdk-' + name,
+    exec = executable('dpdk-' + name,
             sources,
             c_args: cflags,
             link_args: ldflags,
@@ -110,4 +110,9 @@ foreach app:apps
             include_directories: includes,
             install_rpath: join_paths(get_option('prefix'), driver_install_path),
             install: true)
+    if name == 'test'
+        dpdk_test = exec
+        autotest_sources = sources
+        subdir('test/suites')  # define the pre-canned test suites
+    endif
 endforeach
diff --git a/app/test/suites/meson.build b/app/test/suites/meson.build
new file mode 100644
index 0000000000..df0c66ff0d
--- /dev/null
+++ b/app/test/suites/meson.build
@@ -0,0 +1,76 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2023 Intel Corporation
+
+# some perf tests (eg: memcpy perf autotest)take very long
+# to complete, so timeout to 10 minutes
+timeout_seconds = 600
+timeout_seconds_fast = 10
+
+test_no_huge_args = ['--no-huge', '-m', '2048']
+has_hugepage = run_command(has_hugepages_cmd, check: true).stdout().strip() != '0'
+message('hugepage availability: @0@'.format(has_hugepage))
+
+# process source files to determine the different unit test suites
+# - fast_tests
+# - perf_tests
+# - driver_tests
+test_suites = run_command(get_test_suites_cmd, autotest_sources,
+         check: true).stdout().strip().split()
+foreach suite:test_suites
+    # simple cases - tests without parameters or special handling
+    suite = suite.split('=')
+    suite_name = suite[0]
+    suite_tests = suite[1].split(',')
+    if suite_name != 'fast-tests'
+        # simple cases - tests without parameters or special handling
+        foreach t: suite_tests
+            test(t, dpdk_test,
+                    args: ['--', t],
+                    timeout: timeout_seconds,
+                    is_parallel: false,
+                    suite: suite_name)
+        endforeach
+    else
+    # special fast-test handling here
+        foreach t: suite_tests
+            params = t.split(':')
+            test_name = params[0]
+            nohuge = params[1] == 'true'
+            asan = params[2] == 'true'
+
+            test_args = []
+            if nohuge
+                test_args += test_no_huge_args
+            elif not has_hugepage
+                continue  #skip this tests
+            endif
+            if not asan and (get_option('b_sanitize') == 'address'
+                    or get_option('b_sanitize') == 'address,undefined')
+                continue  # skip this test
+            endif
+
+            if (get_option('default_library') == 'shared'
+                    and test_name == 'event_eth_tx_adapter_autotest')
+                test_args += ['-d', dpdk_drivers_build_dir]
+            endif
+            if is_linux
+                test_args += ['--file-prefix=@0@'.format(arg[0])]
+            endif
+
+            test(test_name, dpdk_test,
+                args : test_args + ['--', test_name],
+                timeout : timeout_seconds_fast,
+                is_parallel : false,
+                suite : 'fast-tests')
+            if not is_windows and test_name == 'trace_autotest'
+                test_args += ['--trace=.*']
+                test_args += ['--trace-dir=@0@'.format(meson.current_build_dir())]
+                test(test_name + '_with_traces', dpdk_test,
+                    args : test_args + ['--', test_name],
+                    timeout : timeout_seconds_fast,
+                    is_parallel : false,
+                    suite : 'fast-tests')
+            endif
+        endforeach
+    endif
+endforeach
diff --git a/buildtools/get-test-suites.py b/buildtools/get-test-suites.py
new file mode 100644
index 0000000000..95a9cad4c8
--- /dev/null
+++ b/buildtools/get-test-suites.py
@@ -0,0 +1,33 @@
+#! /usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2023 Intel Corporation
+
+import sys
+import re
+
+input_list = sys.argv[1:]
+test_def_regex = re.compile("REGISTER_([A-Z]+)_TEST\s*\(\s*([a-z0-9_]+)")
+test_suites = {}
+
+def get_fast_test_params(test_name, ln):
+    "Extract the extra fast-test parameters from the line"
+    #print(f"ln: {ln.rstrip()}, test_name: {test_name}, split: {ln.split(test_name, 1)}")
+    (_, rest_of_line) = ln.split(test_name, 1)
+    (_, nohuge, asan, _func) = rest_of_line.split(',', 3)
+    return f":{nohuge.strip().lower()}:{asan.strip().lower()}"
+
+for fname in input_list:
+    with open(fname) as f:
+        contents = [ln for ln in f.readlines() if test_def_regex.match(ln.strip())]
+    for ln in contents:
+        (test_suite, test_name) = test_def_regex.match(ln).group(1, 2)
+        suite_name = f"{test_suite.lower()}-tests"
+        if suite_name in test_suites:
+            test_suites[suite_name].append(test_name)
+        else:
+            test_suites[suite_name] = [test_name]
+        if suite_name == "fast-tests":
+            test_suites["fast-tests"][-1] += get_fast_test_params(test_name, ln)
+
+for suite in test_suites.keys():
+    print(f"{suite}={','.join(test_suites[suite])}")
diff --git a/app/test/has_hugepage.py b/buildtools/has-hugepages.py
similarity index 100%
rename from app/test/has_hugepage.py
rename to buildtools/has-hugepages.py
diff --git a/buildtools/meson.build b/buildtools/meson.build
index e1c600e40f..ac5e4dcf08 100644
--- a/buildtools/meson.build
+++ b/buildtools/meson.build
@@ -18,6 +18,8 @@ map_to_win_cmd = py3 + files('map_to_win.py')
 sphinx_wrapper = py3 + files('call-sphinx-build.py')
 get_cpu_count_cmd = py3 + files('get-cpu-count.py')
 get_numa_count_cmd = py3 + files('get-numa-count.py')
+get_test_suites_cmd = py3 + files('get-test-suites.py')
+has_hugepages_cmd = py3 + files('has-hugepages.py')
 binutils_avx512_check = (py3 + files('binutils-avx512-check.py') +
                         [objdump] + cc.cmd_array())
 
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v3 6/8] examples/l3fwd: make eventdev an optional dependency
  2023-08-14 18:20 ` [PATCH v3 " Bruce Richardson
                     ` (4 preceding siblings ...)
  2023-08-14 18:21   ` [PATCH v3 5/8] app/test: define unit tests suites based on test macros Bruce Richardson
@ 2023-08-14 18:21   ` Bruce Richardson
  2023-08-14 18:21   ` [PATCH v3 7/8] build: make most device classes optional Bruce Richardson
                     ` (2 subsequent siblings)
  8 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-14 18:21 UTC (permalink / raw)
  To: dev; +Cc: ci, Bruce Richardson

With l3fwd being a very commonly used example app, and built as part of
our CI tests, we need to ensure it's buildable with just about all
supported DPDK configurations.  To enable l3fwd application to be built
when the eventdev library is disabled, we need to compile in the
eventdev support conditionally.  Thankfully, the eventdev support is
pretty self-contained, with only the main.c file having more than a
couple of ifdefs.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 examples/l3fwd/l3fwd_em.c                  |  2 +
 examples/l3fwd/l3fwd_em_hlm.h              |  2 +
 examples/l3fwd/l3fwd_event.c               |  2 +
 examples/l3fwd/l3fwd_event.h               |  7 ++-
 examples/l3fwd/l3fwd_event_generic.c       |  2 +
 examples/l3fwd/l3fwd_event_internal_port.c |  2 +
 examples/l3fwd/l3fwd_fib.c                 |  2 +
 examples/l3fwd/l3fwd_lpm.c                 |  2 +
 examples/l3fwd/main.c                      | 65 +++++++++++++++++-----
 examples/l3fwd/meson.build                 |  5 +-
 10 files changed, 74 insertions(+), 17 deletions(-)

diff --git a/examples/l3fwd/l3fwd_em.c b/examples/l3fwd/l3fwd_em.c
index 476ac0c54f..40e102b38a 100644
--- a/examples/l3fwd/l3fwd_em.c
+++ b/examples/l3fwd/l3fwd_em.c
@@ -663,6 +663,7 @@ em_main_loop(__rte_unused void *dummy)
 	return 0;
 }
 
+#ifdef RTE_LIB_EVENTDEV
 static __rte_always_inline void
 em_event_loop_single(struct l3fwd_event_resources *evt_rsrc,
 		const uint8_t flags)
@@ -959,6 +960,7 @@ em_event_main_loop_tx_q_burst_vector(__rte_unused void *dummy)
 	em_event_loop_vector(evt_rsrc, L3FWD_EVENT_TX_ENQ);
 	return 0;
 }
+#endif
 
 /* Initialize exact match (hash) parameters. 8< */
 void
diff --git a/examples/l3fwd/l3fwd_em_hlm.h b/examples/l3fwd/l3fwd_em_hlm.h
index 2e11eefad7..31cda9ddc1 100644
--- a/examples/l3fwd/l3fwd_em_hlm.h
+++ b/examples/l3fwd/l3fwd_em_hlm.h
@@ -255,6 +255,7 @@ l3fwd_em_send_packets(int nb_rx, struct rte_mbuf **pkts_burst, uint16_t portid,
 	send_packets_multi(qconf, pkts_burst, dst_port, nb_rx);
 }
 
+#ifdef RTE_LIB_EVENTDEV
 /*
  * Buffer optimized handling of events, invoked
  * from main_loop.
@@ -347,5 +348,6 @@ l3fwd_em_process_event_vector(struct rte_event_vector *vec,
 
 	process_event_vector(vec, dst_port);
 }
+#endif /* RTE_LIB_EVENTDEV */
 
 #endif /* __L3FWD_EM_HLM_H__ */
diff --git a/examples/l3fwd/l3fwd_event.c b/examples/l3fwd/l3fwd_event.c
index 32906ab08d..d72a4138cc 100644
--- a/examples/l3fwd/l3fwd_event.c
+++ b/examples/l3fwd/l3fwd_event.c
@@ -2,6 +2,7 @@
  * Copyright(C) 2019 Marvell International Ltd.
  */
 
+#ifdef RTE_LIB_EVENTDEV
 #include <stdbool.h>
 #include <getopt.h>
 
@@ -341,3 +342,4 @@ l3fwd_event_worker_cleanup(uint8_t event_d_id, uint8_t event_p_id,
 	rte_event_port_quiesce(event_d_id, event_p_id, l3fwd_event_port_flush,
 			       NULL);
 }
+#endif /* #ifdef RTE_LIB_EVENTDEV */
diff --git a/examples/l3fwd/l3fwd_event.h b/examples/l3fwd/l3fwd_event.h
index e21817c36b..1fd6fe4a78 100644
--- a/examples/l3fwd/l3fwd_event.h
+++ b/examples/l3fwd/l3fwd_event.h
@@ -6,11 +6,13 @@
 #define __L3FWD_EVENTDEV_H__
 
 #include <rte_common.h>
+#include <rte_service.h>
+#include <rte_spinlock.h>
+
+#ifdef RTE_LIB_EVENTDEV
 #include <rte_eventdev.h>
 #include <rte_event_eth_rx_adapter.h>
 #include <rte_event_eth_tx_adapter.h>
-#include <rte_service.h>
-#include <rte_spinlock.h>
 
 #include "l3fwd.h"
 
@@ -164,4 +166,5 @@ void l3fwd_event_worker_cleanup(uint8_t event_d_id, uint8_t event_p_id,
 				struct rte_event events[], uint16_t nb_enq,
 				uint16_t nb_deq, uint8_t is_vector);
 
+#endif /* ifdef RTE_LIB_EVENTDEV */
 #endif /* __L3FWD_EVENTDEV_H__ */
diff --git a/examples/l3fwd/l3fwd_event_generic.c b/examples/l3fwd/l3fwd_event_generic.c
index c80573fc58..ddb6e5c38d 100644
--- a/examples/l3fwd/l3fwd_event_generic.c
+++ b/examples/l3fwd/l3fwd_event_generic.c
@@ -2,6 +2,7 @@
  * Copyright(C) 2019 Marvell International Ltd.
  */
 
+#ifdef RTE_LIB_EVENTDEV
 #include <stdbool.h>
 
 #include "l3fwd.h"
@@ -309,3 +310,4 @@ l3fwd_event_set_generic_ops(struct l3fwd_event_setup_ops *ops)
 	ops->event_port_setup = l3fwd_event_port_setup_generic;
 	ops->adapter_setup = l3fwd_rx_tx_adapter_setup_generic;
 }
+#endif /* RTE_LIB_EVENTDEV */
diff --git a/examples/l3fwd/l3fwd_event_internal_port.c b/examples/l3fwd/l3fwd_event_internal_port.c
index 32cf657148..cb49a8b9fa 100644
--- a/examples/l3fwd/l3fwd_event_internal_port.c
+++ b/examples/l3fwd/l3fwd_event_internal_port.c
@@ -2,6 +2,7 @@
  * Copyright(C) 2019 Marvell International Ltd.
  */
 
+#ifdef RTE_LIB_EVENTDEV
 #include <stdbool.h>
 
 #include "l3fwd.h"
@@ -311,3 +312,4 @@ l3fwd_event_set_internal_port_ops(struct l3fwd_event_setup_ops *ops)
 	ops->event_port_setup = l3fwd_event_port_setup_internal_port;
 	ops->adapter_setup = l3fwd_rx_tx_adapter_setup_internal_port;
 }
+#endif /* RTE_LIB_EVENTDEV */
diff --git a/examples/l3fwd/l3fwd_fib.c b/examples/l3fwd/l3fwd_fib.c
index 18398492ae..6a21984415 100644
--- a/examples/l3fwd/l3fwd_fib.c
+++ b/examples/l3fwd/l3fwd_fib.c
@@ -253,6 +253,7 @@ fib_main_loop(__rte_unused void *dummy)
 	return 0;
 }
 
+#ifdef RTE_LIB_EVENTDEV
 /* One eventdev loop for single and burst using fib. */
 static __rte_always_inline void
 fib_event_loop(struct l3fwd_event_resources *evt_rsrc,
@@ -635,6 +636,7 @@ fib_event_main_loop_tx_q_burst_vector(__rte_unused void *dummy)
 	fib_event_loop_vector(evt_rsrc, L3FWD_EVENT_TX_ENQ);
 	return 0;
 }
+#endif
 
 /* Function to setup fib. 8< */
 void
diff --git a/examples/l3fwd/l3fwd_lpm.c b/examples/l3fwd/l3fwd_lpm.c
index 4ac1925c84..a484a33089 100644
--- a/examples/l3fwd/l3fwd_lpm.c
+++ b/examples/l3fwd/l3fwd_lpm.c
@@ -226,6 +226,7 @@ lpm_main_loop(__rte_unused void *dummy)
 	return 0;
 }
 
+#ifdef RTE_LIB_EVENTDEV
 static __rte_always_inline uint16_t
 lpm_process_event_pkt(const struct lcore_conf *lconf, struct rte_mbuf *mbuf)
 {
@@ -554,6 +555,7 @@ lpm_event_main_loop_tx_q_burst_vector(__rte_unused void *dummy)
 	lpm_event_loop_vector(evt_rsrc, L3FWD_EVENT_TX_ENQ);
 	return 0;
 }
+#endif
 
 void
 setup_lpm(const int socketid)
diff --git a/examples/l3fwd/main.c b/examples/l3fwd/main.c
index a4f061537e..6063eb1399 100644
--- a/examples/l3fwd/main.c
+++ b/examples/l3fwd/main.c
@@ -135,8 +135,10 @@ static struct rte_eth_conf port_conf = {
 
 uint32_t max_pkt_len;
 
-static struct rte_mempool *pktmbuf_pool[RTE_MAX_ETHPORTS][NB_SOCKETS];
+#ifdef RTE_LIB_EVENTDEV
 static struct rte_mempool *vector_pool[RTE_MAX_ETHPORTS];
+#endif
+static struct rte_mempool *pktmbuf_pool[RTE_MAX_ETHPORTS][NB_SOCKETS];
 static uint8_t lkp_per_socket[NB_SOCKETS];
 
 struct l3fwd_lkp_mode {
@@ -398,8 +400,10 @@ print_usage(const char *prgname)
 		" [--parse-ptype]"
 		" [--per-port-pool]"
 		" [--mode]"
+#ifdef RTE_LIB_EVENTDEV
 		" [--eventq-sched]"
 		" [--event-vector [--event-vector-size SIZE] [--event-vector-tmo NS]]"
+#endif
 		" [-E]"
 		" [-L]\n\n"
 
@@ -422,6 +426,7 @@ print_usage(const char *prgname)
 		"  --per-port-pool: Use separate buffer pool per port\n"
 		"  --mode: Packet transfer mode for I/O, poll or eventdev\n"
 		"          Default mode = poll\n"
+#ifdef RTE_LIB_EVENTDEV
 		"  --eventq-sched: Event queue synchronization method\n"
 		"                  ordered, atomic or parallel.\n"
 		"                  Default: atomic\n"
@@ -432,6 +437,7 @@ print_usage(const char *prgname)
 		"  --event-vector:  Enable event vectorization.\n"
 		"  --event-vector-size: Max vector size if event vectorization is enabled.\n"
 		"  --event-vector-tmo: Max timeout to form vector in nanoseconds if event vectorization is enabled\n"
+#endif
 		"  -E : Enable exact match, legacy flag please use --lookup=em instead\n"
 		"  -L : Enable longest prefix match, legacy flag please use --lookup=lpm instead\n"
 		"  --rule_ipv4=FILE: Specify the ipv4 rules entries file.\n"
@@ -559,14 +565,16 @@ parse_eth_dest(const char *optarg)
 }
 
 static void
-parse_mode(const char *optarg)
+parse_mode(const char *optarg __rte_unused)
 {
+#ifdef RTE_LIB_EVENTDEV
 	struct l3fwd_event_resources *evt_rsrc = l3fwd_get_eventdev_rsrc();
 
 	if (!strcmp(optarg, "poll"))
 		evt_rsrc->enabled = false;
 	else if (!strcmp(optarg, "eventdev"))
 		evt_rsrc->enabled = true;
+#endif
 }
 
 static void
@@ -601,6 +609,7 @@ parse_queue_size(const char *queue_size_arg, uint16_t *queue_size, int rx)
 	*queue_size = value;
 }
 
+#ifdef RTE_LIB_EVENTDEV
 static void
 parse_eventq_sched(const char *optarg)
 {
@@ -631,6 +640,7 @@ parse_event_eth_rx_queues(const char *eth_rx_queues)
 
 	evt_rsrc->eth_rx_queues = num_eth_rx_queues;
 }
+#endif
 
 static int
 parse_lookup(const char *optarg)
@@ -756,9 +766,11 @@ parse_args(int argc, char **argv)
 	int option_index;
 	char *prgname = argv[0];
 	uint8_t lcore_params = 0;
+#ifdef RTE_LIB_EVENTDEV
 	uint8_t eventq_sched = 0;
 	uint8_t eth_rx_q = 0;
 	struct l3fwd_event_resources *evt_rsrc = l3fwd_get_eventdev_rsrc();
+#endif
 
 	argvopt = argv;
 
@@ -850,6 +862,7 @@ parse_args(int argc, char **argv)
 			parse_mode(optarg);
 			break;
 
+#ifdef RTE_LIB_EVENTDEV
 		case CMD_LINE_OPT_EVENTQ_SYNC_NUM:
 			parse_eventq_sched(optarg);
 			eventq_sched = 1;
@@ -860,6 +873,20 @@ parse_args(int argc, char **argv)
 			eth_rx_q = 1;
 			break;
 
+		case CMD_LINE_OPT_ENABLE_VECTOR_NUM:
+			printf("event vectorization is enabled\n");
+			evt_rsrc->vector_enabled = 1;
+			break;
+
+		case CMD_LINE_OPT_VECTOR_SIZE_NUM:
+			evt_rsrc->vector_size = strtol(optarg, NULL, 10);
+			break;
+
+		case CMD_LINE_OPT_VECTOR_TMO_NS_NUM:
+			evt_rsrc->vector_tmo_ns = strtoull(optarg, NULL, 10);
+			break;
+#endif
+
 		case CMD_LINE_OPT_LOOKUP_NUM:
 			if (lookup_mode != L3FWD_LOOKUP_DEFAULT) {
 				fprintf(stderr, "Only one lookup mode is allowed at a time!\n");
@@ -875,16 +902,6 @@ parse_args(int argc, char **argv)
 				return -1;
 			break;
 
-		case CMD_LINE_OPT_ENABLE_VECTOR_NUM:
-			printf("event vectorization is enabled\n");
-			evt_rsrc->vector_enabled = 1;
-			break;
-		case CMD_LINE_OPT_VECTOR_SIZE_NUM:
-			evt_rsrc->vector_size = strtol(optarg, NULL, 10);
-			break;
-		case CMD_LINE_OPT_VECTOR_TMO_NS_NUM:
-			evt_rsrc->vector_tmo_ns = strtoull(optarg, NULL, 10);
-			break;
 		case CMD_LINE_OPT_RULE_IPV4_NUM:
 			l3fwd_set_rule_ipv4_name(optarg);
 			break;
@@ -900,6 +917,8 @@ parse_args(int argc, char **argv)
 		}
 	}
 
+	RTE_SET_USED(lcore_params); /* needed if no eventdev block */
+#ifdef RTE_LIB_EVENTDEV
 	if (evt_rsrc->enabled && lcore_params) {
 		fprintf(stderr, "lcore config is not valid when event mode is selected\n");
 		return -1;
@@ -927,6 +946,7 @@ parse_args(int argc, char **argv)
 			"vector timeout set to default (%" PRIu64 " ns)\n",
 			evt_rsrc->vector_tmo_ns);
 	}
+#endif
 
 	/*
 	 * Nothing is selected, pick longest-prefix match
@@ -962,7 +982,9 @@ print_ethaddr(const char *name, const struct rte_ether_addr *eth_addr)
 int
 init_mem(uint16_t portid, unsigned int nb_mbuf)
 {
+#ifdef RTE_LIB_EVENTDEV
 	struct l3fwd_event_resources *evt_rsrc = l3fwd_get_eventdev_rsrc();
+#endif
 	struct lcore_conf *qconf;
 	int socketid;
 	unsigned lcore_id;
@@ -1007,6 +1029,7 @@ init_mem(uint16_t portid, unsigned int nb_mbuf)
 			}
 		}
 
+#ifdef RTE_LIB_EVENTDEV
 		if (evt_rsrc->vector_enabled && vector_pool[portid] == NULL) {
 			unsigned int nb_vec;
 
@@ -1025,6 +1048,7 @@ init_mem(uint16_t portid, unsigned int nb_mbuf)
 				printf("Allocated vector pool for port %d\n",
 				       portid);
 		}
+#endif
 
 		qconf = &lcore_conf[lcore_id];
 		qconf->ipv4_lookup_struct =
@@ -1406,6 +1430,7 @@ l3fwd_service_enable(uint32_t service_id)
 	return 0;
 }
 
+#ifdef RTE_LIB_EVENTDEV
 static void
 l3fwd_event_service_setup(void)
 {
@@ -1458,16 +1483,20 @@ l3fwd_event_service_setup(void)
 		l3fwd_service_enable(service_id);
 	}
 }
+#endif
 
 int
 main(int argc, char **argv)
 {
+#ifdef RTE_LIB_EVENTDEV
 	struct l3fwd_event_resources *evt_rsrc;
+	int i;
+#endif
 	struct lcore_conf *qconf;
 	uint16_t queueid, portid;
 	unsigned int lcore_id;
 	uint8_t queue;
-	int i, ret;
+	int ret;
 
 	/* init EAL */
 	ret = rte_eal_init(argc, argv);
@@ -1487,7 +1516,9 @@ main(int argc, char **argv)
 		*(uint64_t *)(val_eth + portid) = dest_eth_addr[portid];
 	}
 
+#ifdef RTE_LIB_EVENTDEV
 	evt_rsrc = l3fwd_get_eventdev_rsrc();
+#endif
 	/* parse application arguments (after the EAL ones) */
 	ret = parse_args(argc, argv);
 	if (ret < 0)
@@ -1499,6 +1530,7 @@ main(int argc, char **argv)
 	/* Add the config file rules */
 	l3fwd_lkp.read_config_files();
 
+#ifdef RTE_LIB_EVENTDEV
 	evt_rsrc->per_port_pool = per_port_pool;
 	evt_rsrc->pkt_pool = pktmbuf_pool;
 	evt_rsrc->vec_pool = vector_pool;
@@ -1514,6 +1546,7 @@ main(int argc, char **argv)
 			l3fwd_lkp.main_loop = evt_rsrc->ops.lpm_event_loop;
 		l3fwd_event_service_setup();
 	} else
+#endif
 		l3fwd_poll_resource_setup();
 
 	/* start ports */
@@ -1562,6 +1595,8 @@ main(int argc, char **argv)
 	ret = 0;
 	/* launch per-lcore init on every lcore */
 	rte_eal_mp_remote_launch(l3fwd_lkp.main_loop, NULL, CALL_MAIN);
+
+#ifdef RTE_LIB_EVENTDEV
 	if (evt_rsrc->enabled) {
 		for (i = 0; i < evt_rsrc->rx_adptr.nb_rx_adptr; i++)
 			rte_event_eth_rx_adapter_stop(
@@ -1589,7 +1624,9 @@ main(int argc, char **argv)
 		rte_event_dev_stop(evt_rsrc->event_d_id);
 		rte_event_dev_close(evt_rsrc->event_d_id);
 
-	} else {
+	} else
+#endif
+	{
 		rte_eal_mp_wait_lcore();
 
 		RTE_ETH_FOREACH_DEV(portid) {
diff --git a/examples/l3fwd/meson.build b/examples/l3fwd/meson.build
index b40244a941..c25de77bba 100644
--- a/examples/l3fwd/meson.build
+++ b/examples/l3fwd/meson.build
@@ -7,7 +7,7 @@
 # DPDK instance, use 'make'
 
 allow_experimental_apis = true
-deps += ['acl', 'hash', 'lpm', 'fib', 'eventdev']
+deps += ['acl', 'hash', 'lpm', 'fib']
 sources = files(
         'l3fwd_acl.c',
         'l3fwd_em.c',
@@ -18,3 +18,6 @@ sources = files(
         'l3fwd_lpm.c',
         'main.c',
 )
+if dpdk_conf.has('RTE_LIB_EVENTDEV')
+    deps += 'eventdev'
+endif
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v3 7/8] build: make most device classes optional
  2023-08-14 18:20 ` [PATCH v3 " Bruce Richardson
                     ` (5 preceding siblings ...)
  2023-08-14 18:21   ` [PATCH v3 6/8] examples/l3fwd: make eventdev an optional dependency Bruce Richardson
@ 2023-08-14 18:21   ` Bruce Richardson
  2023-08-14 18:21   ` [PATCH v3 8/8] build: expand list of optional libraries Bruce Richardson
  2023-08-14 18:33   ` [PATCH v3 0/8] " Morten Brørup
  8 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-14 18:21 UTC (permalink / raw)
  To: dev; +Cc: ci, Bruce Richardson, Morten Brørup

Apart from ethdev and cryptodev, which have lots of components and tests
which depend on them, we can make the device class libraries optional
without too much work.

This patch marks:
* bbdev,
* compressdev,
* dmadev,
* eventdev,
* mldev,
* rawdev,
* regexdev
optional, and ensures that DPDK - including tests - can be built with
these components disabled.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
---
 lib/meson.build | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/lib/meson.build b/lib/meson.build
index 92cbd6bb66..1a78c8c984 100644
--- a/lib/meson.build
+++ b/lib/meson.build
@@ -67,8 +67,12 @@ libraries = [
 ]
 
 optional_libs = [
+        'bbdev',
         'bitratestats',
         'cfgfile',
+        'compressdev',
+        'dmadev',
+        'eventdev',
         'gpudev',
         'graph',
         'gro',
@@ -76,11 +80,14 @@ optional_libs = [
         'jobstats',
         'latencystats',
         'metrics',
+        'mldev',
         'node',
         'pdump',
         'pipeline',
         'port',
         'power',
+        'rawdev',
+        'regexdev',
         'table',
         'vhost',
 ]
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v3 8/8] build: expand list of optional libraries
  2023-08-14 18:20 ` [PATCH v3 " Bruce Richardson
                     ` (6 preceding siblings ...)
  2023-08-14 18:21   ` [PATCH v3 7/8] build: make most device classes optional Bruce Richardson
@ 2023-08-14 18:21   ` Bruce Richardson
  2023-08-14 18:33   ` [PATCH v3 0/8] " Morten Brørup
  8 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-14 18:21 UTC (permalink / raw)
  To: dev; +Cc: ci, Bruce Richardson

With the unit test build now with individual per-file dependencies, we
can more easily expand the list of optional libraries. Add 8 new
libraries to the optional list.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 lib/meson.build | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/lib/meson.build b/lib/meson.build
index 1a78c8c984..0c9dc8b3b6 100644
--- a/lib/meson.build
+++ b/lib/meson.build
@@ -69,25 +69,33 @@ libraries = [
 optional_libs = [
         'bbdev',
         'bitratestats',
+        'bpf',
         'cfgfile',
         'compressdev',
+        'distributor',
         'dmadev',
+        'efd',
         'eventdev',
         'gpudev',
         'graph',
         'gro',
         'gso',
+        'ip_frag',
         'jobstats',
         'latencystats',
+        'member',
         'metrics',
         'mldev',
         'node',
+        'pcapng',
         'pdump',
         'pipeline',
         'port',
         'power',
         'rawdev',
         'regexdev',
+        'reorder',
+        'sched',
         'table',
         'vhost',
 ]
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* RE: [PATCH v3 0/8] expand list of optional libraries
  2023-08-14 18:20 ` [PATCH v3 " Bruce Richardson
                     ` (7 preceding siblings ...)
  2023-08-14 18:21   ` [PATCH v3 8/8] build: expand list of optional libraries Bruce Richardson
@ 2023-08-14 18:33   ` Morten Brørup
  8 siblings, 0 replies; 93+ messages in thread
From: Morten Brørup @ 2023-08-14 18:33 UTC (permalink / raw)
  To: Bruce Richardson, dev; +Cc: ci

> From: Bruce Richardson [mailto:bruce.richardson@intel.com]
> Sent: Monday, 14 August 2023 20.21
> 
> This patchset contains changes to DPDK to make the build more
> configurable
> overall. The changes can be grouped into two areas:
> 
> * Firstly, there are changes to make the build of the unit tests more
>   flexible and maintainable. These 5 patches switch the unit tests from
>   having separate hard-coded lists of files and tests, with lots of
>   conditional logic to add/remove items from the list, to instead being
>   a single list of files and dependencies per-file. The actual lists of
>   test-suites are built up dynamically from the list of files whose
>   dependencies are met and therefore will be part of the build.
>   This rework enables...
> * The second set of patches which expands the list of libraries which
>   can be disabled at build-time, while still giving a successful build
>   of DPDK. Overall, after this set is applied, the list of optional
>   libraries in DPDK is approx doubled in size. [15 additions giving
>   31 optional libs]
> 
> NOTES:
> * as stated in the RFC, the test binary moves in the build as a result
>   of this set - from app/test/dpdk-test to app/dpdk-test.
> * running the suites of tests no longer works after patch 4 until patch
>   5 is applied. If this is an issue, the two patches can be merged on
>   apply. However, the individual tests are still runnable through the
>   test binary - it's only through "meson test" that there are issues, so
>   I'm keeping the patches separate since each is relatively complex in
> its
>   own right.
> * A number of files in the test directory were not being built, and seem
>   to have never been built with meson. This requires separate work,
> outside
>   the scope of the patchset to fix.
> * Similarly, some unit tests were never added to test suites. These can
>   now be found by using grep to find the old REGISTER_TEST_COMMAND
> macro.
>   Again, adding orphan tests to suites is outside the scope of this set.
> 
> V3:
> - Create symlink to old path to test binary, to ensure all automated CI,
>   and any other use of the test binary still works.
> 
> V2:
> - Changes since RFC:
>    -  merged two previous sets - reworking test build, and disabling
>       libs - into one, since they depend on each other
>    -  Reworked the list of dependencies for each file in the tests,
>       so that files that depend on other files have those deps called
>       out.
>    -  Added list of external dependencies to the test builds
>    -  For disabling libs, merged patches disabling individual libs
>       together into one patch, since all unit test build changes were
>       unnecessary. This made each patch a single-line addition to
>       the optional list in lib/meson.build.

Series-acked-by: Morten Brørup <mb@smartsharesystems.com>


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v4 0/8] expand list of optional libraries
  2023-07-21 11:51 [RFC PATCH 0/5] replace build code for unit tests Bruce Richardson
                   ` (7 preceding siblings ...)
  2023-08-14 18:20 ` [PATCH v3 " Bruce Richardson
@ 2023-08-15 13:13 ` Bruce Richardson
  2023-08-15 13:13   ` [PATCH v4 1/8] app/test: add new macros for various test types Bruce Richardson
                     ` (8 more replies)
  2023-08-15 15:10 ` [PATCH v5 00/10] " Bruce Richardson
  2023-08-16 15:34 ` [PATCH v6 00/11] " Bruce Richardson
  10 siblings, 9 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-15 13:13 UTC (permalink / raw)
  To: dev; +Cc: ci, Bruce Richardson

This patchset contains changes to DPDK to make the build more configurable
overall. The changes can be grouped into two areas:

* Firstly, there are changes to make the build of the unit tests more
  flexible and maintainable. These 5 patches switch the unit tests from
  having separate hard-coded lists of files and tests, with lots of
  conditional logic to add/remove items from the list, to instead being
  a single list of files and dependencies per-file. The actual lists of
  test-suites are built up dynamically from the list of files whose
  dependencies are met and therefore will be part of the build.
  This rework enables...
* The second set of patches which expands the list of libraries which
  can be disabled at build-time, while still giving a successful build
  of DPDK. Overall, after this set is applied, the list of optional
  libraries in DPDK is approx doubled in size. [15 additions giving
  31 optional libs]

NOTES:
* as stated in the RFC, the test binary moves in the build as a result
  of this set - from app/test/dpdk-test to app/dpdk-test.
* running the suites of tests no longer works after patch 4 until patch
  5 is applied. If this is an issue, the two patches can be merged on
  apply. However, the individual tests are still runnable through the
  test binary - it's only through "meson test" that there are issues, so
  I'm keeping the patches separate since each is relatively complex in its
  own right.
* A number of files in the test directory were not being built, and seem
  to have never been built with meson. This requires separate work, outside
  the scope of the patchset to fix.
* Similarly, some unit tests were never added to test suites. These can
  now be found by using grep to find the old REGISTER_TEST_COMMAND macro.
  Again, adding orphan tests to suites is outside the scope of this set.

V4:
- improve test suite definition, hopefully improving test pass rate:
  - drop unnecessary file-prefix for fast tests, since tests are not parallel
  - always pass driver path flag in "meson test" for shared builds.

V3:
- Create symlink to old path to test binary, to ensure all automated CI,
  and any other use of the test binary still works.

V2:
- Changes since RFC:
   -  merged two previous sets - reworking test build, and disabling
      libs - into one, since they depend on each other
   -  Reworked the list of dependencies for each file in the tests,
      so that files that depend on other files have those deps called
      out.
   -  Added list of external dependencies to the test builds
   -  For disabling libs, merged patches disabling individual libs
      together into one patch, since all unit test build changes were
      unnecessary. This made each patch a single-line addition to
      the optional list in lib/meson.build.

Bruce Richardson (8):
  app/test: add new macros for various test types
  app/test: tag tests with the test type
  app/test: make telemetry data test buildable on windows
  app/test: build using per-file dependency matrix
  app/test: define unit tests suites based on test macros
  examples/l3fwd: make eventdev an optional dependency
  build: make most device classes optional
  build: expand list of optional libraries

 app/meson.build                               |  15 +-
 app/test/meson.build                          | 823 +++++-------------
 app/test/suites/meson.build                   |  72 ++
 app/test/test.h                               |   9 +-
 app/test/test_acl.c                           |   2 +-
 app/test/test_atomic.c                        |   2 +-
 app/test/test_barrier.c                       |   2 +-
 app/test/test_bitmap.c                        |   2 +-
 app/test/test_bitops.c                        |   2 +-
 app/test/test_bpf.c                           |   4 +-
 app/test/test_byteorder.c                     |   2 +-
 app/test/test_cksum.c                         |   2 +-
 app/test/test_cmdline.c                       |   2 +-
 app/test/test_common.c                        |   2 +-
 app/test/test_cpuflags.c                      |   2 +-
 app/test/test_crc.c                           |   2 +-
 app/test/test_cryptodev.c                     |  38 +-
 app/test/test_cryptodev_asym.c                |   2 +-
 app/test/test_cycles.c                        |   2 +-
 app/test/test_debug.c                         |   2 +-
 app/test/test_devargs.c                       |   2 +-
 app/test/test_distributor.c                   |   2 +-
 app/test/test_distributor_perf.c              |   2 +-
 app/test/test_dmadev.c                        |   2 +-
 app/test/test_eal_flags.c                     |  24 +-
 app/test/test_eal_fs.c                        |   2 +-
 app/test/test_efd.c                           |   2 +-
 app/test/test_efd_perf.c                      |   2 +-
 app/test/test_errno.c                         |   2 +-
 app/test/test_ethdev_link.c                   |   2 +-
 app/test/test_event_ring.c                    |   2 +-
 app/test/test_eventdev.c                      |   2 +-
 app/test/test_fbarray.c                       |   2 +-
 app/test/test_fib.c                           |   4 +-
 app/test/test_fib6.c                          |   4 +-
 app/test/test_fib6_perf.c                     |   2 +-
 app/test/test_fib_perf.c                      |   2 +-
 app/test/test_func_reentrancy.c               |   2 +-
 app/test/test_hash.c                          |   2 +-
 app/test/test_hash_functions.c                |   2 +-
 app/test/test_hash_multiwriter.c              |   2 +-
 app/test/test_hash_perf.c                     |   2 +-
 app/test/test_hash_readwrite.c                |   4 +-
 app/test/test_hash_readwrite_lf_perf.c        |   2 +-
 app/test/test_interrupts.c                    |   2 +-
 app/test/test_ipfrag.c                        |   2 +-
 app/test/test_ipsec.c                         |   2 +-
 app/test/test_ipsec_perf.c                    |   2 +-
 app/test/test_kvargs.c                        |   2 +-
 app/test/test_lcores.c                        |   2 +-
 app/test/test_logs.c                          |   2 +-
 app/test/test_lpm.c                           |   2 +-
 app/test/test_lpm6.c                          |   2 +-
 app/test/test_lpm6_perf.c                     |   2 +-
 app/test/test_lpm_perf.c                      |   2 +-
 app/test/test_malloc.c                        |   2 +-
 app/test/test_malloc_perf.c                   |   2 +-
 app/test/test_mbuf.c                          |   2 +-
 app/test/test_mcslock.c                       |   2 +-
 app/test/test_member.c                        |   2 +-
 app/test/test_member_perf.c                   |   2 +-
 app/test/test_memcpy.c                        |   2 +-
 app/test/test_memcpy_perf.c                   |   2 +-
 app/test/test_memory.c                        |   2 +-
 app/test/test_mempool.c                       |   2 +-
 app/test/test_mempool_perf.c                  |   2 +-
 app/test/test_memzone.c                       |   2 +-
 app/test/test_meter.c                         |   2 +-
 app/test/test_mp_secondary.c                  |   2 +-
 app/test/test_per_lcore.c                     |   2 +-
 app/test/test_pflock.c                        |   2 +-
 app/test/test_pie.c                           |   6 +-
 app/test/test_pmd_perf.c                      |   2 +-
 app/test/test_power.c                         |   2 +-
 app/test/test_power_cpufreq.c                 |   2 +-
 app/test/test_power_intel_uncore.c            |   2 +-
 app/test/test_power_kvm_vm.c                  |   2 +-
 app/test/test_prefetch.c                      |   2 +-
 app/test/test_rand_perf.c                     |   2 +-
 app/test/test_rcu_qsbr.c                      |   2 +-
 app/test/test_rcu_qsbr_perf.c                 |   2 +-
 app/test/test_reassembly_perf.c               |   2 +-
 app/test/test_reciprocal_division.c           |   2 +-
 app/test/test_reciprocal_division_perf.c      |   2 +-
 app/test/test_red.c                           |   4 +-
 app/test/test_reorder.c                       |   2 +-
 app/test/test_rib.c                           |   4 +-
 app/test/test_rib6.c                          |   4 +-
 app/test/test_ring.c                          |   2 +-
 app/test/test_ring_perf.c                     |   2 +-
 app/test/test_rwlock.c                        |   8 +-
 app/test/test_sched.c                         |   2 +-
 app/test/test_security.c                      |   2 +-
 app/test/test_seqlock.c                       |   2 +-
 app/test/test_service_cores.c                 |   4 +-
 app/test/test_spinlock.c                      |   2 +-
 app/test/test_stack.c                         |   4 +-
 app/test/test_stack_perf.c                    |   4 +-
 app/test/test_string_fns.c                    |   2 +-
 app/test/test_tailq.c                         |   2 +-
 app/test/test_telemetry_data.c                |  14 +-
 app/test/test_thash.c                         |   2 +-
 app/test/test_thash_perf.c                    |   2 +-
 app/test/test_threads.c                       |   2 +-
 app/test/test_ticketlock.c                    |   2 +-
 app/test/test_timer.c                         |   2 +-
 app/test/test_timer_perf.c                    |   2 +-
 app/test/test_timer_racecond.c                |   2 +-
 app/test/test_trace.c                         |   2 +-
 app/test/test_trace_perf.c                    |   2 +-
 app/test/test_version.c                       |   2 +-
 buildtools/get-test-suites.py                 |  33 +
 .../has-hugepages.py                          |   0
 buildtools/meson.build                        |   2 +
 examples/l3fwd/l3fwd_em.c                     |   2 +
 examples/l3fwd/l3fwd_em_hlm.h                 |   2 +
 examples/l3fwd/l3fwd_event.c                  |   2 +
 examples/l3fwd/l3fwd_event.h                  |   7 +-
 examples/l3fwd/l3fwd_event_generic.c          |   2 +
 examples/l3fwd/l3fwd_event_internal_port.c    |   2 +
 examples/l3fwd/l3fwd_fib.c                    |   2 +
 examples/l3fwd/l3fwd_lpm.c                    |   2 +
 examples/l3fwd/main.c                         |  65 +-
 examples/l3fwd/meson.build                    |   5 +-
 lib/meson.build                               |  15 +
 125 files changed, 610 insertions(+), 764 deletions(-)
 create mode 100644 app/test/suites/meson.build
 create mode 100644 buildtools/get-test-suites.py
 rename app/test/has_hugepage.py => buildtools/has-hugepages.py (100%)

--
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v4 1/8] app/test: add new macros for various test types
  2023-08-15 13:13 ` [PATCH v4 " Bruce Richardson
@ 2023-08-15 13:13   ` Bruce Richardson
  2023-08-15 13:13   ` [PATCH v4 2/8] app/test: tag tests with the test type Bruce Richardson
                     ` (7 subsequent siblings)
  8 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-15 13:13 UTC (permalink / raw)
  To: dev; +Cc: ci, Bruce Richardson

Rather than just registering all tests using a single generic macro,
add macros which identify the test as being of a particular type.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 app/test/test.h | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/app/test/test.h b/app/test/test.h
index 85f57efbc6..a91ded76af 100644
--- a/app/test/test.h
+++ b/app/test/test.h
@@ -190,7 +190,7 @@ struct test_command {
 
 void add_test_command(struct test_command *t);
 
-/* Register a test function with its command string */
+/* Register a test function with its command string. Should not be used directly */
 #define REGISTER_TEST_COMMAND(cmd, func) \
 	static struct test_command test_struct_##cmd = { \
 		.command = RTE_STR(cmd), \
@@ -201,4 +201,11 @@ void add_test_command(struct test_command *t);
 		add_test_command(&test_struct_##cmd); \
 	}
 
+/* Register a test function as a particular type.
+ * These can be used to build up test suites automatically
+ */
+#define REGISTER_FAST_TEST(cmd, no_huge, ASan, func)  REGISTER_TEST_COMMAND(cmd, func)
+#define REGISTER_PERF_TEST REGISTER_TEST_COMMAND
+#define REGISTER_DRIVER_TEST REGISTER_TEST_COMMAND
+
 #endif
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v4 2/8] app/test: tag tests with the test type
  2023-08-15 13:13 ` [PATCH v4 " Bruce Richardson
  2023-08-15 13:13   ` [PATCH v4 1/8] app/test: add new macros for various test types Bruce Richardson
@ 2023-08-15 13:13   ` Bruce Richardson
  2023-08-15 13:13   ` [PATCH v4 3/8] app/test: make telemetry data test buildable on windows Bruce Richardson
                     ` (6 subsequent siblings)
  8 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-15 13:13 UTC (permalink / raw)
  To: dev; +Cc: ci, Bruce Richardson

Rather than having the test types called out in the meson.build file, we
can use macros to identify the test type in the C file itself and then
dynamically build up the tests lists at config time.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 app/test/test_acl.c                      |  2 +-
 app/test/test_atomic.c                   |  2 +-
 app/test/test_barrier.c                  |  2 +-
 app/test/test_bitmap.c                   |  2 +-
 app/test/test_bitops.c                   |  2 +-
 app/test/test_bpf.c                      |  4 +--
 app/test/test_byteorder.c                |  2 +-
 app/test/test_cksum.c                    |  2 +-
 app/test/test_cmdline.c                  |  2 +-
 app/test/test_common.c                   |  2 +-
 app/test/test_cpuflags.c                 |  2 +-
 app/test/test_crc.c                      |  2 +-
 app/test/test_cryptodev.c                | 38 ++++++++++++------------
 app/test/test_cryptodev_asym.c           |  2 +-
 app/test/test_cycles.c                   |  2 +-
 app/test/test_debug.c                    |  2 +-
 app/test/test_devargs.c                  |  2 +-
 app/test/test_distributor.c              |  2 +-
 app/test/test_distributor_perf.c         |  2 +-
 app/test/test_dmadev.c                   |  2 +-
 app/test/test_eal_flags.c                | 24 +++++++--------
 app/test/test_eal_fs.c                   |  2 +-
 app/test/test_efd.c                      |  2 +-
 app/test/test_efd_perf.c                 |  2 +-
 app/test/test_errno.c                    |  2 +-
 app/test/test_ethdev_link.c              |  2 +-
 app/test/test_event_ring.c               |  2 +-
 app/test/test_eventdev.c                 |  2 +-
 app/test/test_fbarray.c                  |  2 +-
 app/test/test_fib.c                      |  4 +--
 app/test/test_fib6.c                     |  4 +--
 app/test/test_fib6_perf.c                |  2 +-
 app/test/test_fib_perf.c                 |  2 +-
 app/test/test_func_reentrancy.c          |  2 +-
 app/test/test_hash.c                     |  2 +-
 app/test/test_hash_functions.c           |  2 +-
 app/test/test_hash_multiwriter.c         |  2 +-
 app/test/test_hash_perf.c                |  2 +-
 app/test/test_hash_readwrite.c           |  4 +--
 app/test/test_hash_readwrite_lf_perf.c   |  2 +-
 app/test/test_interrupts.c               |  2 +-
 app/test/test_ipfrag.c                   |  2 +-
 app/test/test_ipsec.c                    |  2 +-
 app/test/test_ipsec_perf.c               |  2 +-
 app/test/test_kvargs.c                   |  2 +-
 app/test/test_lcores.c                   |  2 +-
 app/test/test_logs.c                     |  2 +-
 app/test/test_lpm.c                      |  2 +-
 app/test/test_lpm6.c                     |  2 +-
 app/test/test_lpm6_perf.c                |  2 +-
 app/test/test_lpm_perf.c                 |  2 +-
 app/test/test_malloc.c                   |  2 +-
 app/test/test_malloc_perf.c              |  2 +-
 app/test/test_mbuf.c                     |  2 +-
 app/test/test_mcslock.c                  |  2 +-
 app/test/test_member.c                   |  2 +-
 app/test/test_member_perf.c              |  2 +-
 app/test/test_memcpy.c                   |  2 +-
 app/test/test_memcpy_perf.c              |  2 +-
 app/test/test_memory.c                   |  2 +-
 app/test/test_mempool.c                  |  2 +-
 app/test/test_mempool_perf.c             |  2 +-
 app/test/test_memzone.c                  |  2 +-
 app/test/test_meter.c                    |  2 +-
 app/test/test_mp_secondary.c             |  2 +-
 app/test/test_per_lcore.c                |  2 +-
 app/test/test_pflock.c                   |  2 +-
 app/test/test_pie.c                      |  6 ++--
 app/test/test_pmd_perf.c                 |  2 +-
 app/test/test_power.c                    |  2 +-
 app/test/test_power_cpufreq.c            |  2 +-
 app/test/test_power_intel_uncore.c       |  2 +-
 app/test/test_power_kvm_vm.c             |  2 +-
 app/test/test_prefetch.c                 |  2 +-
 app/test/test_rand_perf.c                |  2 +-
 app/test/test_rcu_qsbr.c                 |  2 +-
 app/test/test_rcu_qsbr_perf.c            |  2 +-
 app/test/test_reassembly_perf.c          |  2 +-
 app/test/test_reciprocal_division.c      |  2 +-
 app/test/test_reciprocal_division_perf.c |  2 +-
 app/test/test_red.c                      |  4 +--
 app/test/test_reorder.c                  |  2 +-
 app/test/test_rib.c                      |  4 +--
 app/test/test_rib6.c                     |  4 +--
 app/test/test_ring.c                     |  2 +-
 app/test/test_ring_perf.c                |  2 +-
 app/test/test_rwlock.c                   |  8 ++---
 app/test/test_sched.c                    |  2 +-
 app/test/test_security.c                 |  2 +-
 app/test/test_seqlock.c                  |  2 +-
 app/test/test_service_cores.c            |  4 +--
 app/test/test_spinlock.c                 |  2 +-
 app/test/test_stack.c                    |  4 +--
 app/test/test_stack_perf.c               |  4 +--
 app/test/test_string_fns.c               |  2 +-
 app/test/test_tailq.c                    |  2 +-
 app/test/test_thash.c                    |  2 +-
 app/test/test_thash_perf.c               |  2 +-
 app/test/test_threads.c                  |  2 +-
 app/test/test_ticketlock.c               |  2 +-
 app/test/test_timer.c                    |  2 +-
 app/test/test_timer_perf.c               |  2 +-
 app/test/test_timer_racecond.c           |  2 +-
 app/test/test_trace.c                    |  2 +-
 app/test/test_trace_perf.c               |  2 +-
 app/test/test_version.c                  |  2 +-
 106 files changed, 150 insertions(+), 150 deletions(-)

diff --git a/app/test/test_acl.c b/app/test/test_acl.c
index 623f34682e..bf1466fe11 100644
--- a/app/test/test_acl.c
+++ b/app/test/test_acl.c
@@ -1749,4 +1749,4 @@ test_acl(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(acl_autotest, test_acl);
+REGISTER_FAST_TEST(acl_autotest, True, True, test_acl);
diff --git a/app/test/test_atomic.c b/app/test/test_atomic.c
index e4b997827e..b52420c10b 100644
--- a/app/test/test_atomic.c
+++ b/app/test/test_atomic.c
@@ -631,4 +631,4 @@ test_atomic(void)
 
 	return 0;
 }
-REGISTER_TEST_COMMAND(atomic_autotest, test_atomic);
+REGISTER_FAST_TEST(atomic_autotest, False, True, test_atomic);
diff --git a/app/test/test_barrier.c b/app/test/test_barrier.c
index ec69af25bf..925a88b68a 100644
--- a/app/test/test_barrier.c
+++ b/app/test/test_barrier.c
@@ -285,4 +285,4 @@ test_barrier(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(barrier_autotest, test_barrier);
+REGISTER_PERF_TEST(barrier_autotest, test_barrier);
diff --git a/app/test/test_bitmap.c b/app/test/test_bitmap.c
index e9c61590ae..2014d2682c 100644
--- a/app/test/test_bitmap.c
+++ b/app/test/test_bitmap.c
@@ -269,4 +269,4 @@ test_bitmap(void)
 	return test_bitmap_all_set();
 }
 
-REGISTER_TEST_COMMAND(bitmap_autotest, test_bitmap);
+REGISTER_FAST_TEST(bitmap_autotest, True, True, test_bitmap);
diff --git a/app/test/test_bitops.c b/app/test/test_bitops.c
index c21426bf2f..dac466602b 100644
--- a/app/test/test_bitops.c
+++ b/app/test/test_bitops.c
@@ -135,4 +135,4 @@ test_bitops(void)
 	return TEST_SUCCESS;
 }
 
-REGISTER_TEST_COMMAND(bitops_autotest, test_bitops);
+REGISTER_FAST_TEST(bitops_autotest, True, True, test_bitops);
diff --git a/app/test/test_bpf.c b/app/test/test_bpf.c
index f5af5e8a3f..5fef17fbc2 100644
--- a/app/test/test_bpf.c
+++ b/app/test/test_bpf.c
@@ -3262,7 +3262,7 @@ test_bpf(void)
 
 #endif /* !RTE_LIB_BPF */
 
-REGISTER_TEST_COMMAND(bpf_autotest, test_bpf);
+REGISTER_FAST_TEST(bpf_autotest, True, True, test_bpf);
 
 #ifndef RTE_HAS_LIBPCAP
 
@@ -3473,4 +3473,4 @@ test_bpf_convert(void)
 
 #endif /* RTE_HAS_LIBPCAP */
 
-REGISTER_TEST_COMMAND(bpf_convert_autotest, test_bpf_convert);
+REGISTER_FAST_TEST(bpf_convert_autotest, True, True, test_bpf_convert);
diff --git a/app/test/test_byteorder.c b/app/test/test_byteorder.c
index de14ed539e..2189f94744 100644
--- a/app/test/test_byteorder.c
+++ b/app/test/test_byteorder.c
@@ -63,4 +63,4 @@ test_byteorder(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(byteorder_autotest, test_byteorder);
+REGISTER_FAST_TEST(byteorder_autotest, True, True, test_byteorder);
diff --git a/app/test/test_cksum.c b/app/test/test_cksum.c
index 6c15de9a93..c04fae800e 100644
--- a/app/test/test_cksum.c
+++ b/app/test/test_cksum.c
@@ -267,4 +267,4 @@ test_cksum(void)
 }
 #undef GOTO_FAIL
 
-REGISTER_TEST_COMMAND(cksum_autotest, test_cksum);
+REGISTER_FAST_TEST(cksum_autotest, True, True, test_cksum);
diff --git a/app/test/test_cmdline.c b/app/test/test_cmdline.c
index 115bee966d..c2a375ac1c 100644
--- a/app/test/test_cmdline.c
+++ b/app/test/test_cmdline.c
@@ -60,4 +60,4 @@ test_cmdline(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(cmdline_autotest, test_cmdline);
+REGISTER_FAST_TEST(cmdline_autotest, True, True, test_cmdline);
diff --git a/app/test/test_common.c b/app/test/test_common.c
index f89e1eb7ee..c7b41a3f0e 100644
--- a/app/test/test_common.c
+++ b/app/test/test_common.c
@@ -350,4 +350,4 @@ test_common(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(common_autotest, test_common);
+REGISTER_FAST_TEST(common_autotest, True, True, test_common);
diff --git a/app/test/test_cpuflags.c b/app/test/test_cpuflags.c
index a0e342ae48..c3417dda24 100644
--- a/app/test/test_cpuflags.c
+++ b/app/test/test_cpuflags.c
@@ -334,4 +334,4 @@ test_cpuflags(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(cpuflags_autotest, test_cpuflags);
+REGISTER_FAST_TEST(cpuflags_autotest, True, True, test_cpuflags);
diff --git a/app/test/test_crc.c b/app/test/test_crc.c
index 5edc8fb13b..b267bdde9f 100644
--- a/app/test/test_crc.c
+++ b/app/test/test_crc.c
@@ -171,4 +171,4 @@ test_crc(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(crc_autotest, test_crc);
+REGISTER_FAST_TEST(crc_autotest, True, True, test_crc);
diff --git a/app/test/test_cryptodev.c b/app/test/test_cryptodev.c
index fb2af40b99..956268bfcd 100644
--- a/app/test/test_cryptodev.c
+++ b/app/test/test_cryptodev.c
@@ -17535,33 +17535,33 @@ REGISTER_TEST_COMMAND(cryptodev_dpaa2_sec_raw_api_autotest,
 		test_cryptodev_dpaa2_sec_raw_api);
 REGISTER_TEST_COMMAND(cryptodev_dpaa_sec_raw_api_autotest,
 		test_cryptodev_dpaa_sec_raw_api);
-REGISTER_TEST_COMMAND(cryptodev_qat_raw_api_autotest,
+REGISTER_DRIVER_TEST(cryptodev_qat_raw_api_autotest,
 		test_cryptodev_qat_raw_api);
-REGISTER_TEST_COMMAND(cryptodev_qat_autotest, test_cryptodev_qat);
-REGISTER_TEST_COMMAND(cryptodev_aesni_mb_autotest, test_cryptodev_aesni_mb);
-REGISTER_TEST_COMMAND(cryptodev_cpu_aesni_mb_autotest,
+REGISTER_DRIVER_TEST(cryptodev_qat_autotest, test_cryptodev_qat);
+REGISTER_DRIVER_TEST(cryptodev_aesni_mb_autotest, test_cryptodev_aesni_mb);
+REGISTER_DRIVER_TEST(cryptodev_cpu_aesni_mb_autotest,
 	test_cryptodev_cpu_aesni_mb);
-REGISTER_TEST_COMMAND(cryptodev_chacha_poly_mb_autotest,
+REGISTER_DRIVER_TEST(cryptodev_chacha_poly_mb_autotest,
 	test_cryptodev_chacha_poly_mb);
-REGISTER_TEST_COMMAND(cryptodev_openssl_autotest, test_cryptodev_openssl);
-REGISTER_TEST_COMMAND(cryptodev_aesni_gcm_autotest, test_cryptodev_aesni_gcm);
-REGISTER_TEST_COMMAND(cryptodev_cpu_aesni_gcm_autotest,
+REGISTER_DRIVER_TEST(cryptodev_openssl_autotest, test_cryptodev_openssl);
+REGISTER_DRIVER_TEST(cryptodev_aesni_gcm_autotest, test_cryptodev_aesni_gcm);
+REGISTER_DRIVER_TEST(cryptodev_cpu_aesni_gcm_autotest,
 	test_cryptodev_cpu_aesni_gcm);
 REGISTER_TEST_COMMAND(cryptodev_mlx5_autotest, test_cryptodev_mlx5);
-REGISTER_TEST_COMMAND(cryptodev_null_autotest, test_cryptodev_null);
-REGISTER_TEST_COMMAND(cryptodev_sw_snow3g_autotest, test_cryptodev_sw_snow3g);
-REGISTER_TEST_COMMAND(cryptodev_sw_kasumi_autotest, test_cryptodev_sw_kasumi);
-REGISTER_TEST_COMMAND(cryptodev_sw_zuc_autotest, test_cryptodev_sw_zuc);
-REGISTER_TEST_COMMAND(cryptodev_sw_armv8_autotest, test_cryptodev_armv8);
-REGISTER_TEST_COMMAND(cryptodev_sw_mvsam_autotest, test_cryptodev_mrvl);
-REGISTER_TEST_COMMAND(cryptodev_dpaa2_sec_autotest, test_cryptodev_dpaa2_sec);
-REGISTER_TEST_COMMAND(cryptodev_dpaa_sec_autotest, test_cryptodev_dpaa_sec);
+REGISTER_DRIVER_TEST(cryptodev_null_autotest, test_cryptodev_null);
+REGISTER_DRIVER_TEST(cryptodev_sw_snow3g_autotest, test_cryptodev_sw_snow3g);
+REGISTER_DRIVER_TEST(cryptodev_sw_kasumi_autotest, test_cryptodev_sw_kasumi);
+REGISTER_DRIVER_TEST(cryptodev_sw_zuc_autotest, test_cryptodev_sw_zuc);
+REGISTER_DRIVER_TEST(cryptodev_sw_armv8_autotest, test_cryptodev_armv8);
+REGISTER_DRIVER_TEST(cryptodev_sw_mvsam_autotest, test_cryptodev_mrvl);
+REGISTER_DRIVER_TEST(cryptodev_dpaa2_sec_autotest, test_cryptodev_dpaa2_sec);
+REGISTER_DRIVER_TEST(cryptodev_dpaa_sec_autotest, test_cryptodev_dpaa_sec);
 REGISTER_TEST_COMMAND(cryptodev_ccp_autotest, test_cryptodev_ccp);
-REGISTER_TEST_COMMAND(cryptodev_uadk_autotest, test_cryptodev_uadk);
+REGISTER_DRIVER_TEST(cryptodev_uadk_autotest, test_cryptodev_uadk);
 REGISTER_TEST_COMMAND(cryptodev_virtio_autotest, test_cryptodev_virtio);
 REGISTER_TEST_COMMAND(cryptodev_octeontx_autotest, test_cryptodev_octeontx);
 REGISTER_TEST_COMMAND(cryptodev_caam_jr_autotest, test_cryptodev_caam_jr);
 REGISTER_TEST_COMMAND(cryptodev_nitrox_autotest, test_cryptodev_nitrox);
 REGISTER_TEST_COMMAND(cryptodev_bcmfs_autotest, test_cryptodev_bcmfs);
-REGISTER_TEST_COMMAND(cryptodev_cn9k_autotest, test_cryptodev_cn9k);
-REGISTER_TEST_COMMAND(cryptodev_cn10k_autotest, test_cryptodev_cn10k);
+REGISTER_DRIVER_TEST(cryptodev_cn9k_autotest, test_cryptodev_cn9k);
+REGISTER_DRIVER_TEST(cryptodev_cn10k_autotest, test_cryptodev_cn10k);
diff --git a/app/test/test_cryptodev_asym.c b/app/test/test_cryptodev_asym.c
index 0ef2642fdd..3d5a73bf89 100644
--- a/app/test/test_cryptodev_asym.c
+++ b/app/test/test_cryptodev_asym.c
@@ -2828,7 +2828,7 @@ test_cryptodev_cn10k_asym(void)
 REGISTER_TEST_COMMAND(cryptodev_openssl_asym_autotest,
 					  test_cryptodev_openssl_asym);
 
-REGISTER_TEST_COMMAND(cryptodev_qat_asym_autotest, test_cryptodev_qat_asym);
+REGISTER_DRIVER_TEST(cryptodev_qat_asym_autotest, test_cryptodev_qat_asym);
 
 REGISTER_TEST_COMMAND(cryptodev_octeontx_asym_autotest,
 					  test_cryptodev_octeontx_asym);
diff --git a/app/test/test_cycles.c b/app/test/test_cycles.c
index 66d11e6db8..08c91cef78 100644
--- a/app/test/test_cycles.c
+++ b/app/test/test_cycles.c
@@ -53,4 +53,4 @@ test_user_delay_us(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(user_delay_us, test_user_delay_us);
+REGISTER_FAST_TEST(user_delay_us, True, True, test_user_delay_us);
diff --git a/app/test/test_debug.c b/app/test/test_debug.c
index 2704f5b927..acd9542257 100644
--- a/app/test/test_debug.c
+++ b/app/test/test_debug.c
@@ -140,4 +140,4 @@ test_debug(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(debug_autotest, test_debug);
+REGISTER_FAST_TEST(debug_autotest, True, True, test_debug);
diff --git a/app/test/test_devargs.c b/app/test/test_devargs.c
index 0a4c34a1ad..59ec04957e 100644
--- a/app/test/test_devargs.c
+++ b/app/test/test_devargs.c
@@ -213,4 +213,4 @@ test_devargs(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(devargs_autotest, test_devargs);
+REGISTER_FAST_TEST(devargs_autotest, True, True, test_devargs);
diff --git a/app/test/test_distributor.c b/app/test/test_distributor.c
index 3efa4af104..ee263a7fee 100644
--- a/app/test/test_distributor.c
+++ b/app/test/test_distributor.c
@@ -952,4 +952,4 @@ test_distributor(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(distributor_autotest, test_distributor);
+REGISTER_FAST_TEST(distributor_autotest, False, True, test_distributor);
diff --git a/app/test/test_distributor_perf.c b/app/test/test_distributor_perf.c
index ee4321486d..ca868451d7 100644
--- a/app/test/test_distributor_perf.c
+++ b/app/test/test_distributor_perf.c
@@ -277,4 +277,4 @@ test_distributor_perf(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(distributor_perf_autotest, test_distributor_perf);
+REGISTER_PERF_TEST(distributor_perf_autotest, test_distributor_perf);
diff --git a/app/test/test_dmadev.c b/app/test/test_dmadev.c
index 0736ff2a18..6ef875e545 100644
--- a/app/test/test_dmadev.c
+++ b/app/test/test_dmadev.c
@@ -941,4 +941,4 @@ test_dma(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(dmadev_autotest, test_dma);
+REGISTER_DRIVER_TEST(dmadev_autotest, test_dma);
diff --git a/app/test/test_eal_flags.c b/app/test/test_eal_flags.c
index 148e9098e8..3b7af7405d 100644
--- a/app/test/test_eal_flags.c
+++ b/app/test/test_eal_flags.c
@@ -1644,15 +1644,15 @@ test_memory_flags(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(eal_flags_c_opt_autotest, test_missing_c_flag);
-REGISTER_TEST_COMMAND(eal_flags_main_opt_autotest, test_main_lcore_flag);
-REGISTER_TEST_COMMAND(eal_flags_n_opt_autotest, test_invalid_n_flag);
-REGISTER_TEST_COMMAND(eal_flags_hpet_autotest, test_no_hpet_flag);
-REGISTER_TEST_COMMAND(eal_flags_no_huge_autotest, test_no_huge_flag);
-REGISTER_TEST_COMMAND(eal_flags_a_opt_autotest, test_allow_flag);
-REGISTER_TEST_COMMAND(eal_flags_b_opt_autotest, test_invalid_b_flag);
-REGISTER_TEST_COMMAND(eal_flags_vdev_opt_autotest, test_invalid_vdev_flag);
-REGISTER_TEST_COMMAND(eal_flags_r_opt_autotest, test_invalid_r_flag);
-REGISTER_TEST_COMMAND(eal_flags_mem_autotest, test_memory_flags);
-REGISTER_TEST_COMMAND(eal_flags_file_prefix_autotest, test_file_prefix);
-REGISTER_TEST_COMMAND(eal_flags_misc_autotest, test_misc_flags);
+REGISTER_FAST_TEST(eal_flags_c_opt_autotest, False, False, test_missing_c_flag);
+REGISTER_FAST_TEST(eal_flags_main_opt_autotest, False, False, test_main_lcore_flag);
+REGISTER_FAST_TEST(eal_flags_n_opt_autotest, False, False, test_invalid_n_flag);
+REGISTER_FAST_TEST(eal_flags_hpet_autotest, False, False, test_no_hpet_flag);
+REGISTER_FAST_TEST(eal_flags_no_huge_autotest, False, False, test_no_huge_flag);
+REGISTER_FAST_TEST(eal_flags_a_opt_autotest, False, False, test_allow_flag);
+REGISTER_FAST_TEST(eal_flags_b_opt_autotest, False, False, test_invalid_b_flag);
+REGISTER_FAST_TEST(eal_flags_vdev_opt_autotest, False, False, test_invalid_vdev_flag);
+REGISTER_FAST_TEST(eal_flags_r_opt_autotest, False, False, test_invalid_r_flag);
+REGISTER_FAST_TEST(eal_flags_mem_autotest, False, False, test_memory_flags);
+REGISTER_FAST_TEST(eal_flags_file_prefix_autotest, False, False, test_file_prefix);
+REGISTER_FAST_TEST(eal_flags_misc_autotest, False, False, test_misc_flags);
diff --git a/app/test/test_eal_fs.c b/app/test/test_eal_fs.c
index b3686edcb4..a4bdc1236d 100644
--- a/app/test/test_eal_fs.c
+++ b/app/test/test_eal_fs.c
@@ -185,4 +185,4 @@ test_eal_fs(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(eal_fs_autotest, test_eal_fs);
+REGISTER_FAST_TEST(eal_fs_autotest, True, True, test_eal_fs);
diff --git a/app/test/test_efd.c b/app/test/test_efd.c
index fa29e8f97a..1c0986b9bc 100644
--- a/app/test/test_efd.c
+++ b/app/test/test_efd.c
@@ -473,4 +473,4 @@ test_efd(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(efd_autotest, test_efd);
+REGISTER_PERF_TEST(efd_autotest, test_efd);
diff --git a/app/test/test_efd_perf.c b/app/test/test_efd_perf.c
index 4d04ed93e3..b212e96767 100644
--- a/app/test/test_efd_perf.c
+++ b/app/test/test_efd_perf.c
@@ -393,4 +393,4 @@ test_efd_perf(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(efd_perf_autotest, test_efd_perf);
+REGISTER_PERF_TEST(efd_perf_autotest, test_efd_perf);
diff --git a/app/test/test_errno.c b/app/test/test_errno.c
index 0db4fbc8b3..1082299665 100644
--- a/app/test/test_errno.c
+++ b/app/test/test_errno.c
@@ -94,4 +94,4 @@ test_errno(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(errno_autotest, test_errno);
+REGISTER_FAST_TEST(errno_autotest, True, True, test_errno);
diff --git a/app/test/test_ethdev_link.c b/app/test/test_ethdev_link.c
index ab52385a12..d73e18065d 100644
--- a/app/test/test_ethdev_link.c
+++ b/app/test/test_ethdev_link.c
@@ -167,4 +167,4 @@ test_link_status(void)
 	return unit_test_suite_runner(&link_status_testsuite);
 }
 
-REGISTER_TEST_COMMAND(ethdev_link_status, test_link_status);
+REGISTER_FAST_TEST(ethdev_link_status, True, True, test_link_status);
diff --git a/app/test/test_event_ring.c b/app/test/test_event_ring.c
index bc4a6e73f5..d5c8499cfd 100644
--- a/app/test/test_event_ring.c
+++ b/app/test/test_event_ring.c
@@ -256,4 +256,4 @@ test_event_ring(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(event_ring_autotest, test_event_ring);
+REGISTER_FAST_TEST(event_ring_autotest, True, True, test_event_ring);
diff --git a/app/test/test_eventdev.c b/app/test/test_eventdev.c
index 336529038e..e65d893781 100644
--- a/app/test/test_eventdev.c
+++ b/app/test/test_eventdev.c
@@ -1256,7 +1256,7 @@ test_eventdev_selftest_cn10k(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(eventdev_common_autotest, test_eventdev_common);
+REGISTER_FAST_TEST(eventdev_common_autotest, True, True, test_eventdev_common);
 
 #ifndef RTE_EXEC_ENV_WINDOWS
 REGISTER_TEST_COMMAND(eventdev_selftest_sw, test_eventdev_selftest_sw);
diff --git a/app/test/test_fbarray.c b/app/test/test_fbarray.c
index a691bf4458..03428a45fe 100644
--- a/app/test/test_fbarray.c
+++ b/app/test/test_fbarray.c
@@ -733,4 +733,4 @@ test_fbarray(void)
 	return unit_test_suite_runner(&fbarray_test_suite);
 }
 
-REGISTER_TEST_COMMAND(fbarray_autotest, test_fbarray);
+REGISTER_FAST_TEST(fbarray_autotest, True, True, test_fbarray);
diff --git a/app/test/test_fib.c b/app/test/test_fib.c
index eb69d6e2fd..2a8404db61 100644
--- a/app/test/test_fib.c
+++ b/app/test/test_fib.c
@@ -415,5 +415,5 @@ test_slow_fib(void)
 	return unit_test_suite_runner(&fib_slow_tests);
 }
 
-REGISTER_TEST_COMMAND(fib_autotest, test_fib);
-REGISTER_TEST_COMMAND(fib_slow_autotest, test_slow_fib);
+REGISTER_FAST_TEST(fib_autotest, True, True, test_fib);
+REGISTER_PERF_TEST(fib_slow_autotest, test_slow_fib);
diff --git a/app/test/test_fib6.c b/app/test/test_fib6.c
index 15ad09178a..9b1c0ac8c7 100644
--- a/app/test/test_fib6.c
+++ b/app/test/test_fib6.c
@@ -424,5 +424,5 @@ test_slow_fib6(void)
 	return unit_test_suite_runner(&fib6_slow_tests);
 }
 
-REGISTER_TEST_COMMAND(fib6_autotest, test_fib6);
-REGISTER_TEST_COMMAND(fib6_slow_autotest, test_slow_fib6);
+REGISTER_FAST_TEST(fib6_autotest, True, True, test_fib6);
+REGISTER_PERF_TEST(fib6_slow_autotest, test_slow_fib6);
diff --git a/app/test/test_fib6_perf.c b/app/test/test_fib6_perf.c
index add20c2331..a7abc46af9 100644
--- a/app/test/test_fib6_perf.c
+++ b/app/test/test_fib6_perf.c
@@ -156,4 +156,4 @@ test_fib6_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(fib6_perf_autotest, test_fib6_perf);
+REGISTER_PERF_TEST(fib6_perf_autotest, test_fib6_perf);
diff --git a/app/test/test_fib_perf.c b/app/test/test_fib_perf.c
index b56293e64f..a9119c1bb0 100644
--- a/app/test/test_fib_perf.c
+++ b/app/test/test_fib_perf.c
@@ -409,4 +409,4 @@ test_fib_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(fib_perf_autotest, test_fib_perf);
+REGISTER_PERF_TEST(fib_perf_autotest, test_fib_perf);
diff --git a/app/test/test_func_reentrancy.c b/app/test/test_func_reentrancy.c
index ae9de6f93d..63b00009b5 100644
--- a/app/test/test_func_reentrancy.c
+++ b/app/test/test_func_reentrancy.c
@@ -507,4 +507,4 @@ test_func_reentrancy(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(func_reentrancy_autotest, test_func_reentrancy);
+REGISTER_FAST_TEST(func_reentrancy_autotest, False, True, test_func_reentrancy);
diff --git a/app/test/test_hash.c b/app/test/test_hash.c
index 3e45afaa67..e160b27b4c 100644
--- a/app/test/test_hash.c
+++ b/app/test/test_hash.c
@@ -2264,4 +2264,4 @@ test_hash(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(hash_autotest, test_hash);
+REGISTER_FAST_TEST(hash_autotest, True, True, test_hash);
diff --git a/app/test/test_hash_functions.c b/app/test/test_hash_functions.c
index 76d51b6e71..70820d1f19 100644
--- a/app/test/test_hash_functions.c
+++ b/app/test/test_hash_functions.c
@@ -290,4 +290,4 @@ test_hash_functions(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(hash_functions_autotest, test_hash_functions);
+REGISTER_PERF_TEST(hash_functions_autotest, test_hash_functions);
diff --git a/app/test/test_hash_multiwriter.c b/app/test/test_hash_multiwriter.c
index 0c5a8ca186..dd5ca677b9 100644
--- a/app/test/test_hash_multiwriter.c
+++ b/app/test/test_hash_multiwriter.c
@@ -287,4 +287,4 @@ test_hash_multiwriter_main(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(hash_multiwriter_autotest, test_hash_multiwriter_main);
+REGISTER_PERF_TEST(hash_multiwriter_autotest, test_hash_multiwriter_main);
diff --git a/app/test/test_hash_perf.c b/app/test/test_hash_perf.c
index 14a1283aba..d66b96e5ce 100644
--- a/app/test/test_hash_perf.c
+++ b/app/test/test_hash_perf.c
@@ -757,4 +757,4 @@ test_hash_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(hash_perf_autotest, test_hash_perf);
+REGISTER_PERF_TEST(hash_perf_autotest, test_hash_perf);
diff --git a/app/test/test_hash_readwrite.c b/app/test/test_hash_readwrite.c
index 6373e62d33..9c33b49878 100644
--- a/app/test/test_hash_readwrite.c
+++ b/app/test/test_hash_readwrite.c
@@ -760,5 +760,5 @@ test_hash_rw_func_main(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(hash_readwrite_func_autotest, test_hash_rw_func_main);
-REGISTER_TEST_COMMAND(hash_readwrite_perf_autotest, test_hash_rw_perf_main);
+REGISTER_FAST_TEST(hash_readwrite_func_autotest, False, True, test_hash_rw_func_main);
+REGISTER_PERF_TEST(hash_readwrite_perf_autotest, test_hash_rw_perf_main);
diff --git a/app/test/test_hash_readwrite_lf_perf.c b/app/test/test_hash_readwrite_lf_perf.c
index cf86046a2f..5d18850e19 100644
--- a/app/test/test_hash_readwrite_lf_perf.c
+++ b/app/test/test_hash_readwrite_lf_perf.c
@@ -1579,5 +1579,5 @@ test_hash_readwrite_lf_perf_main(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(hash_readwrite_lf_perf_autotest,
+REGISTER_PERF_TEST(hash_readwrite_lf_perf_autotest,
 	test_hash_readwrite_lf_perf_main);
diff --git a/app/test/test_interrupts.c b/app/test/test_interrupts.c
index b59ab42699..cf6b5a6a8e 100644
--- a/app/test/test_interrupts.c
+++ b/app/test/test_interrupts.c
@@ -594,4 +594,4 @@ test_interrupt(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(interrupt_autotest, test_interrupt);
+REGISTER_FAST_TEST(interrupt_autotest, True, True, test_interrupt);
diff --git a/app/test/test_ipfrag.c b/app/test/test_ipfrag.c
index 402ce361c1..264825634d 100644
--- a/app/test/test_ipfrag.c
+++ b/app/test/test_ipfrag.c
@@ -510,4 +510,4 @@ test_ipfrag(void)
 }
 
 
-REGISTER_TEST_COMMAND(ipfrag_autotest, test_ipfrag);
+REGISTER_FAST_TEST(ipfrag_autotest, False, True, test_ipfrag);
diff --git a/app/test/test_ipsec.c b/app/test/test_ipsec.c
index c2a52ec305..629580bc21 100644
--- a/app/test/test_ipsec.c
+++ b/app/test/test_ipsec.c
@@ -2532,4 +2532,4 @@ test_ipsec(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(ipsec_autotest, test_ipsec);
+REGISTER_FAST_TEST(ipsec_autotest, True, True, test_ipsec);
diff --git a/app/test/test_ipsec_perf.c b/app/test/test_ipsec_perf.c
index b221b7fc32..a32a2086e9 100644
--- a/app/test/test_ipsec_perf.c
+++ b/app/test/test_ipsec_perf.c
@@ -631,4 +631,4 @@ test_libipsec_perf(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(ipsec_perf_autotest, test_libipsec_perf);
+REGISTER_PERF_TEST(ipsec_perf_autotest, test_libipsec_perf);
diff --git a/app/test/test_kvargs.c b/app/test/test_kvargs.c
index b7b97a0dd9..d0f05a55c7 100644
--- a/app/test/test_kvargs.c
+++ b/app/test/test_kvargs.c
@@ -292,4 +292,4 @@ test_kvargs(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(kvargs_autotest, test_kvargs);
+REGISTER_FAST_TEST(kvargs_autotest, True, True, test_kvargs);
diff --git a/app/test/test_lcores.c b/app/test/test_lcores.c
index 2c945b0136..b86629c0db 100644
--- a/app/test/test_lcores.c
+++ b/app/test/test_lcores.c
@@ -412,4 +412,4 @@ test_lcores(void)
 	return TEST_SUCCESS;
 }
 
-REGISTER_TEST_COMMAND(lcores_autotest, test_lcores);
+REGISTER_FAST_TEST(lcores_autotest, True, True, test_lcores);
diff --git a/app/test/test_logs.c b/app/test/test_logs.c
index 8da8824bee..38052910e9 100644
--- a/app/test/test_logs.c
+++ b/app/test/test_logs.c
@@ -158,4 +158,4 @@ test_logs(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(logs_autotest, test_logs);
+REGISTER_FAST_TEST(logs_autotest, True, True, test_logs);
diff --git a/app/test/test_lpm.c b/app/test/test_lpm.c
index 37b460af3a..85a75ccec0 100644
--- a/app/test/test_lpm.c
+++ b/app/test/test_lpm.c
@@ -1584,4 +1584,4 @@ test_lpm(void)
 	return global_status;
 }
 
-REGISTER_TEST_COMMAND(lpm_autotest, test_lpm);
+REGISTER_FAST_TEST(lpm_autotest, True, True, test_lpm);
diff --git a/app/test/test_lpm6.c b/app/test/test_lpm6.c
index b6b6f8615e..4a9eddf276 100644
--- a/app/test/test_lpm6.c
+++ b/app/test/test_lpm6.c
@@ -1793,4 +1793,4 @@ test_lpm6(void)
 	return global_status;
 }
 
-REGISTER_TEST_COMMAND(lpm6_autotest, test_lpm6);
+REGISTER_FAST_TEST(lpm6_autotest, True, True, test_lpm6);
diff --git a/app/test/test_lpm6_perf.c b/app/test/test_lpm6_perf.c
index 5b684686a6..8a49f74c84 100644
--- a/app/test/test_lpm6_perf.c
+++ b/app/test/test_lpm6_perf.c
@@ -161,4 +161,4 @@ test_lpm6_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(lpm6_perf_autotest, test_lpm6_perf);
+REGISTER_PERF_TEST(lpm6_perf_autotest, test_lpm6_perf);
diff --git a/app/test/test_lpm_perf.c b/app/test/test_lpm_perf.c
index e72437ba38..15ff396dd0 100644
--- a/app/test/test_lpm_perf.c
+++ b/app/test/test_lpm_perf.c
@@ -760,4 +760,4 @@ test_lpm_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(lpm_perf_autotest, test_lpm_perf);
+REGISTER_PERF_TEST(lpm_perf_autotest, test_lpm_perf);
diff --git a/app/test/test_malloc.c b/app/test/test_malloc.c
index ff081dd931..6057623473 100644
--- a/app/test/test_malloc.c
+++ b/app/test/test_malloc.c
@@ -1091,4 +1091,4 @@ test_malloc(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(malloc_autotest, test_malloc);
+REGISTER_FAST_TEST(malloc_autotest, False, True, test_malloc);
diff --git a/app/test/test_malloc_perf.c b/app/test/test_malloc_perf.c
index 9bd1662981..a99bfd8531 100644
--- a/app/test/test_malloc_perf.c
+++ b/app/test/test_malloc_perf.c
@@ -171,4 +171,4 @@ test_malloc_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(malloc_perf_autotest, test_malloc_perf);
+REGISTER_PERF_TEST(malloc_perf_autotest, test_malloc_perf);
diff --git a/app/test/test_mbuf.c b/app/test/test_mbuf.c
index efac01806b..2eeed268af 100644
--- a/app/test/test_mbuf.c
+++ b/app/test/test_mbuf.c
@@ -2955,4 +2955,4 @@ test_mbuf(void)
 }
 #undef GOTO_FAIL
 
-REGISTER_TEST_COMMAND(mbuf_autotest, test_mbuf);
+REGISTER_FAST_TEST(mbuf_autotest, False, True, test_mbuf);
diff --git a/app/test/test_mcslock.c b/app/test/test_mcslock.c
index 52e45e7e2a..a3753a4397 100644
--- a/app/test/test_mcslock.c
+++ b/app/test/test_mcslock.c
@@ -241,4 +241,4 @@ test_mcslock(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(mcslock_autotest, test_mcslock);
+REGISTER_FAST_TEST(mcslock_autotest, False, True, test_mcslock);
diff --git a/app/test/test_member.c b/app/test/test_member.c
index 4a93f8bff4..e17a8f3190 100644
--- a/app/test/test_member.c
+++ b/app/test/test_member.c
@@ -996,4 +996,4 @@ test_member(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(member_autotest, test_member);
+REGISTER_FAST_TEST(member_autotest, True, True, test_member);
diff --git a/app/test/test_member_perf.c b/app/test/test_member_perf.c
index 2f79888fbd..db6b8a18ef 100644
--- a/app/test/test_member_perf.c
+++ b/app/test/test_member_perf.c
@@ -780,4 +780,4 @@ test_member_perf(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(member_perf_autotest, test_member_perf);
+REGISTER_PERF_TEST(member_perf_autotest, test_member_perf);
diff --git a/app/test/test_memcpy.c b/app/test/test_memcpy.c
index 1ab86f4967..c81efb85c2 100644
--- a/app/test/test_memcpy.c
+++ b/app/test/test_memcpy.c
@@ -129,4 +129,4 @@ test_memcpy(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(memcpy_autotest, test_memcpy);
+REGISTER_FAST_TEST(memcpy_autotest, True, True, test_memcpy);
diff --git a/app/test/test_memcpy_perf.c b/app/test/test_memcpy_perf.c
index 3727c160e6..5c05a84619 100644
--- a/app/test/test_memcpy_perf.c
+++ b/app/test/test_memcpy_perf.c
@@ -348,4 +348,4 @@ test_memcpy_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(memcpy_perf_autotest, test_memcpy_perf);
+REGISTER_PERF_TEST(memcpy_perf_autotest, test_memcpy_perf);
diff --git a/app/test/test_memory.c b/app/test/test_memory.c
index 440e5ef838..13ae9569c2 100644
--- a/app/test/test_memory.c
+++ b/app/test/test_memory.c
@@ -110,4 +110,4 @@ test_memory(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(memory_autotest, test_memory);
+REGISTER_FAST_TEST(memory_autotest, False, True, test_memory);
diff --git a/app/test/test_mempool.c b/app/test/test_mempool.c
index 8e493eda47..03047e5b14 100644
--- a/app/test/test_mempool.c
+++ b/app/test/test_mempool.c
@@ -1043,4 +1043,4 @@ test_mempool(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(mempool_autotest, test_mempool);
+REGISTER_FAST_TEST(mempool_autotest, False, True, test_mempool);
diff --git a/app/test/test_mempool_perf.c b/app/test/test_mempool_perf.c
index ce7c6241ab..96de347f04 100644
--- a/app/test/test_mempool_perf.c
+++ b/app/test/test_mempool_perf.c
@@ -437,4 +437,4 @@ test_mempool_perf(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(mempool_perf_autotest, test_mempool_perf);
+REGISTER_PERF_TEST(mempool_perf_autotest, test_mempool_perf);
diff --git a/app/test/test_memzone.c b/app/test/test_memzone.c
index f10f4fd9cd..85b5839034 100644
--- a/app/test/test_memzone.c
+++ b/app/test/test_memzone.c
@@ -1163,4 +1163,4 @@ test_memzone(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(memzone_autotest, test_memzone);
+REGISTER_FAST_TEST(memzone_autotest, False, True, test_memzone);
diff --git a/app/test/test_meter.c b/app/test/test_meter.c
index 15d5a4839b..3016681bbc 100644
--- a/app/test/test_meter.c
+++ b/app/test/test_meter.c
@@ -713,4 +713,4 @@ test_meter(void)
 
 }
 
-REGISTER_TEST_COMMAND(meter_autotest, test_meter);
+REGISTER_FAST_TEST(meter_autotest, True, True, test_meter);
diff --git a/app/test/test_mp_secondary.c b/app/test/test_mp_secondary.c
index ad47d578f2..502b0235a7 100644
--- a/app/test/test_mp_secondary.c
+++ b/app/test/test_mp_secondary.c
@@ -223,4 +223,4 @@ test_mp_secondary(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(multiprocess_autotest, test_mp_secondary);
+REGISTER_FAST_TEST(multiprocess_autotest, False, False, test_mp_secondary);
diff --git a/app/test/test_per_lcore.c b/app/test/test_per_lcore.c
index 129578d1a3..50011366a7 100644
--- a/app/test/test_per_lcore.c
+++ b/app/test/test_per_lcore.c
@@ -105,4 +105,4 @@ test_per_lcore(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(per_lcore_autotest, test_per_lcore);
+REGISTER_FAST_TEST(per_lcore_autotest, True, True, test_per_lcore);
diff --git a/app/test/test_pflock.c b/app/test/test_pflock.c
index 38da6bce27..07e4a68c2c 100644
--- a/app/test/test_pflock.c
+++ b/app/test/test_pflock.c
@@ -193,4 +193,4 @@ test_pflock(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(pflock_autotest, test_pflock);
+REGISTER_FAST_TEST(pflock_autotest, True, True, test_pflock);
diff --git a/app/test/test_pie.c b/app/test/test_pie.c
index a3c0f97c9d..5674602b69 100644
--- a/app/test/test_pie.c
+++ b/app/test/test_pie.c
@@ -1087,6 +1087,6 @@ test_pie_all(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(pie_autotest, test_pie);
-REGISTER_TEST_COMMAND(pie_perf, test_pie_perf);
-REGISTER_TEST_COMMAND(pie_all, test_pie_all);
+REGISTER_FAST_TEST(pie_autotest, True, True, test_pie);
+REGISTER_PERF_TEST(pie_perf, test_pie_perf);
+REGISTER_PERF_TEST(pie_all, test_pie_all);
diff --git a/app/test/test_pmd_perf.c b/app/test/test_pmd_perf.c
index 3ef590cb51..f6d97f21c9 100644
--- a/app/test/test_pmd_perf.c
+++ b/app/test/test_pmd_perf.c
@@ -899,4 +899,4 @@ test_set_rxtx_sc(cmdline_fixed_string_t type)
 	return -1;
 }
 
-REGISTER_TEST_COMMAND(pmd_perf_autotest, test_pmd_perf);
+REGISTER_PERF_TEST(pmd_perf_autotest, test_pmd_perf);
diff --git a/app/test/test_power.c b/app/test/test_power.c
index b7b5561348..d4f622ff2f 100644
--- a/app/test/test_power.c
+++ b/app/test/test_power.c
@@ -172,4 +172,4 @@ test_power(void)
 }
 #endif
 
-REGISTER_TEST_COMMAND(power_autotest, test_power);
+REGISTER_FAST_TEST(power_autotest, True, True, test_power);
diff --git a/app/test/test_power_cpufreq.c b/app/test/test_power_cpufreq.c
index 4d013cd7bb..f56053ac10 100644
--- a/app/test/test_power_cpufreq.c
+++ b/app/test/test_power_cpufreq.c
@@ -703,5 +703,5 @@ test_power_caps(void)
 
 #endif
 
-REGISTER_TEST_COMMAND(power_cpufreq_autotest, test_power_cpufreq);
+REGISTER_FAST_TEST(power_cpufreq_autotest, False, True, test_power_cpufreq);
 REGISTER_TEST_COMMAND(power_caps_autotest, test_power_caps);
diff --git a/app/test/test_power_intel_uncore.c b/app/test/test_power_intel_uncore.c
index 31163af84e..c8a8621e44 100644
--- a/app/test/test_power_intel_uncore.c
+++ b/app/test/test_power_intel_uncore.c
@@ -298,4 +298,4 @@ test_power_intel_uncore(void)
 }
 #endif
 
-REGISTER_TEST_COMMAND(power_intel_uncore_autotest, test_power_intel_uncore);
+REGISTER_FAST_TEST(power_intel_uncore_autotest, True, True, test_power_intel_uncore);
diff --git a/app/test/test_power_kvm_vm.c b/app/test/test_power_kvm_vm.c
index cc66b7a8a0..4919df8b62 100644
--- a/app/test/test_power_kvm_vm.c
+++ b/app/test/test_power_kvm_vm.c
@@ -299,4 +299,4 @@ test_power_kvm_vm(void)
 }
 #endif
 
-REGISTER_TEST_COMMAND(power_kvm_vm_autotest, test_power_kvm_vm);
+REGISTER_FAST_TEST(power_kvm_vm_autotest, False, True, test_power_kvm_vm);
diff --git a/app/test/test_prefetch.c b/app/test/test_prefetch.c
index 7b4a8e4144..a72ae80557 100644
--- a/app/test/test_prefetch.c
+++ b/app/test/test_prefetch.c
@@ -35,4 +35,4 @@ test_prefetch(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(prefetch_autotest, test_prefetch);
+REGISTER_FAST_TEST(prefetch_autotest, True, True, test_prefetch);
diff --git a/app/test/test_rand_perf.c b/app/test/test_rand_perf.c
index 26fb1d9a58..30204e12c0 100644
--- a/app/test/test_rand_perf.c
+++ b/app/test/test_rand_perf.c
@@ -96,4 +96,4 @@ test_rand_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(rand_perf_autotest, test_rand_perf);
+REGISTER_PERF_TEST(rand_perf_autotest, test_rand_perf);
diff --git a/app/test/test_rcu_qsbr.c b/app/test/test_rcu_qsbr.c
index 70404e89e6..16c24f58e7 100644
--- a/app/test/test_rcu_qsbr.c
+++ b/app/test/test_rcu_qsbr.c
@@ -1418,4 +1418,4 @@ test_rcu_qsbr_main(void)
 	return -1;
 }
 
-REGISTER_TEST_COMMAND(rcu_qsbr_autotest, test_rcu_qsbr_main);
+REGISTER_FAST_TEST(rcu_qsbr_autotest, True, True, test_rcu_qsbr_main);
diff --git a/app/test/test_rcu_qsbr_perf.c b/app/test/test_rcu_qsbr_perf.c
index b15e5cef88..ce88a7333c 100644
--- a/app/test/test_rcu_qsbr_perf.c
+++ b/app/test/test_rcu_qsbr_perf.c
@@ -690,4 +690,4 @@ test_rcu_qsbr_main(void)
 	return -1;
 }
 
-REGISTER_TEST_COMMAND(rcu_qsbr_perf_autotest, test_rcu_qsbr_main);
+REGISTER_PERF_TEST(rcu_qsbr_perf_autotest, test_rcu_qsbr_main);
diff --git a/app/test/test_reassembly_perf.c b/app/test/test_reassembly_perf.c
index c11b65291f..4b4929d777 100644
--- a/app/test/test_reassembly_perf.c
+++ b/app/test/test_reassembly_perf.c
@@ -1000,4 +1000,4 @@ test_reassembly_perf(void)
 	return TEST_SUCCESS;
 }
 
-REGISTER_TEST_COMMAND(reassembly_perf_autotest, test_reassembly_perf);
+REGISTER_PERF_TEST(reassembly_perf_autotest, test_reassembly_perf);
diff --git a/app/test/test_reciprocal_division.c b/app/test/test_reciprocal_division.c
index 8ea9b1d24d..fb52b2d5a1 100644
--- a/app/test/test_reciprocal_division.c
+++ b/app/test/test_reciprocal_division.c
@@ -164,4 +164,4 @@ test_reciprocal(void)
 	return result;
 }
 
-REGISTER_TEST_COMMAND(reciprocal_division, test_reciprocal);
+REGISTER_PERF_TEST(reciprocal_division, test_reciprocal);
diff --git a/app/test/test_reciprocal_division_perf.c b/app/test/test_reciprocal_division_perf.c
index 4f625873e5..cf96d46a22 100644
--- a/app/test/test_reciprocal_division_perf.c
+++ b/app/test/test_reciprocal_division_perf.c
@@ -205,4 +205,4 @@ test_reciprocal_division_perf(void)
 	return result;
 }
 
-REGISTER_TEST_COMMAND(reciprocal_division_perf, test_reciprocal_division_perf);
+REGISTER_PERF_TEST(reciprocal_division_perf, test_reciprocal_division_perf);
diff --git a/app/test/test_red.c b/app/test/test_red.c
index 84c292f8d8..aa7538d51a 100644
--- a/app/test/test_red.c
+++ b/app/test/test_red.c
@@ -1878,5 +1878,5 @@ test_red_all(void)
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
 REGISTER_TEST_COMMAND(red_autotest, test_red);
-REGISTER_TEST_COMMAND(red_perf, test_red_perf);
-REGISTER_TEST_COMMAND(red_all, test_red_all);
+REGISTER_PERF_TEST(red_perf, test_red_perf);
+REGISTER_PERF_TEST(red_all, test_red_all);
diff --git a/app/test/test_reorder.c b/app/test/test_reorder.c
index c188f6ce67..e66b4b05d4 100644
--- a/app/test/test_reorder.c
+++ b/app/test/test_reorder.c
@@ -548,4 +548,4 @@ test_reorder(void)
 }
 
 
-REGISTER_TEST_COMMAND(reorder_autotest, test_reorder);
+REGISTER_FAST_TEST(reorder_autotest, True, True, test_reorder);
diff --git a/app/test/test_rib.c b/app/test/test_rib.c
index 65b6856410..a8d60acc3a 100644
--- a/app/test/test_rib.c
+++ b/app/test/test_rib.c
@@ -363,5 +363,5 @@ test_slow_rib(void)
 	return unit_test_suite_runner(&rib_slow_tests);
 }
 
-REGISTER_TEST_COMMAND(rib_autotest, test_rib);
-REGISTER_TEST_COMMAND(rib_slow_autotest, test_slow_rib);
+REGISTER_FAST_TEST(rib_autotest, True, True, test_rib);
+REGISTER_PERF_TEST(rib_slow_autotest, test_slow_rib);
diff --git a/app/test/test_rib6.c b/app/test/test_rib6.c
index 336b779d2e..9bbd0f406d 100644
--- a/app/test/test_rib6.c
+++ b/app/test/test_rib6.c
@@ -367,5 +367,5 @@ test_slow_rib6(void)
 	return unit_test_suite_runner(&rib6_slow_tests);
 }
 
-REGISTER_TEST_COMMAND(rib6_autotest, test_rib6);
-REGISTER_TEST_COMMAND(rib6_slow_autotest, test_slow_rib6);
+REGISTER_FAST_TEST(rib6_autotest, True, True, test_rib6);
+REGISTER_PERF_TEST(rib6_slow_autotest, test_slow_rib6);
diff --git a/app/test/test_ring.c b/app/test/test_ring.c
index bde33ab4a1..d2accbabfa 100644
--- a/app/test/test_ring.c
+++ b/app/test/test_ring.c
@@ -1241,4 +1241,4 @@ test_ring(void)
 	return -1;
 }
 
-REGISTER_TEST_COMMAND(ring_autotest, test_ring);
+REGISTER_FAST_TEST(ring_autotest, True, True, test_ring);
diff --git a/app/test/test_ring_perf.c b/app/test/test_ring_perf.c
index 3972fd9db3..d7c5a4c30b 100644
--- a/app/test/test_ring_perf.c
+++ b/app/test/test_ring_perf.c
@@ -579,4 +579,4 @@ test_ring_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(ring_perf_autotest, test_ring_perf);
+REGISTER_PERF_TEST(ring_perf_autotest, test_ring_perf);
diff --git a/app/test/test_rwlock.c b/app/test/test_rwlock.c
index 4ae0bf8deb..e8086888fe 100644
--- a/app/test/test_rwlock.c
+++ b/app/test/test_rwlock.c
@@ -506,7 +506,7 @@ try_rwlock_test_rde_wro(void)
 	return process_try_lcore_stats();
 }
 
-REGISTER_TEST_COMMAND(rwlock_test1_autotest, rwlock_test1);
-REGISTER_TEST_COMMAND(rwlock_rda_autotest, try_rwlock_test_rda);
-REGISTER_TEST_COMMAND(rwlock_rds_wrm_autotest, try_rwlock_test_rds_wrm);
-REGISTER_TEST_COMMAND(rwlock_rde_wro_autotest, try_rwlock_test_rde_wro);
+REGISTER_FAST_TEST(rwlock_test1_autotest, True, True, rwlock_test1);
+REGISTER_FAST_TEST(rwlock_rda_autotest, True, True, try_rwlock_test_rda);
+REGISTER_FAST_TEST(rwlock_rds_wrm_autotest, True, True, try_rwlock_test_rds_wrm);
+REGISTER_FAST_TEST(rwlock_rde_wro_autotest, True, True, try_rwlock_test_rde_wro);
diff --git a/app/test/test_sched.c b/app/test/test_sched.c
index ddec572447..385ca45136 100644
--- a/app/test/test_sched.c
+++ b/app/test/test_sched.c
@@ -215,4 +215,4 @@ test_sched(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(sched_autotest, test_sched);
+REGISTER_FAST_TEST(sched_autotest, True, True, test_sched);
diff --git a/app/test/test_security.c b/app/test/test_security.c
index 4783cd0663..a3b962750a 100644
--- a/app/test/test_security.c
+++ b/app/test/test_security.c
@@ -2314,4 +2314,4 @@ test_security(void)
 	return unit_test_suite_runner(&security_testsuite);
 }
 
-REGISTER_TEST_COMMAND(security_autotest, test_security);
+REGISTER_FAST_TEST(security_autotest, False, True, test_security);
diff --git a/app/test/test_seqlock.c b/app/test/test_seqlock.c
index d26d2c010e..325af38070 100644
--- a/app/test/test_seqlock.c
+++ b/app/test/test_seqlock.c
@@ -187,4 +187,4 @@ test_seqlock(void)
 	return rc;
 }
 
-REGISTER_TEST_COMMAND(seqlock_autotest, test_seqlock);
+REGISTER_FAST_TEST(seqlock_autotest, True, True, test_seqlock);
diff --git a/app/test/test_service_cores.c b/app/test/test_service_cores.c
index 422d2a83e8..6c000417d2 100644
--- a/app/test/test_service_cores.c
+++ b/app/test/test_service_cores.c
@@ -1040,7 +1040,7 @@ test_service_common(void)
 	return unit_test_suite_runner(&service_tests);
 }
 
-REGISTER_TEST_COMMAND(service_autotest, test_service_common);
+REGISTER_FAST_TEST(service_autotest, True, True, test_service_common);
 
 static struct unit_test_suite service_perf_tests  = {
 	.suite_name = "service core performance test suite",
@@ -1062,4 +1062,4 @@ test_service_perf(void)
 	return unit_test_suite_runner(&service_perf_tests);
 }
 
-REGISTER_TEST_COMMAND(service_perf_autotest, test_service_perf);
+REGISTER_PERF_TEST(service_perf_autotest, test_service_perf);
diff --git a/app/test/test_spinlock.c b/app/test/test_spinlock.c
index 3f59372300..d655b606f7 100644
--- a/app/test/test_spinlock.c
+++ b/app/test/test_spinlock.c
@@ -302,4 +302,4 @@ test_spinlock(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(spinlock_autotest, test_spinlock);
+REGISTER_FAST_TEST(spinlock_autotest, True, True, test_spinlock);
diff --git a/app/test/test_stack.c b/app/test/test_stack.c
index bc38961433..93bba18469 100644
--- a/app/test/test_stack.c
+++ b/app/test/test_stack.c
@@ -379,5 +379,5 @@ test_lf_stack(void)
 #endif
 }
 
-REGISTER_TEST_COMMAND(stack_autotest, test_stack);
-REGISTER_TEST_COMMAND(stack_lf_autotest, test_lf_stack);
+REGISTER_FAST_TEST(stack_autotest, False, True, test_stack);
+REGISTER_FAST_TEST(stack_lf_autotest, False, True, test_lf_stack);
diff --git a/app/test/test_stack_perf.c b/app/test/test_stack_perf.c
index 1eae00a334..c5e1caa036 100644
--- a/app/test/test_stack_perf.c
+++ b/app/test/test_stack_perf.c
@@ -354,5 +354,5 @@ test_lf_stack_perf(void)
 #endif
 }
 
-REGISTER_TEST_COMMAND(stack_perf_autotest, test_stack_perf);
-REGISTER_TEST_COMMAND(stack_lf_perf_autotest, test_lf_stack_perf);
+REGISTER_PERF_TEST(stack_perf_autotest, test_stack_perf);
+REGISTER_PERF_TEST(stack_lf_perf_autotest, test_lf_stack_perf);
diff --git a/app/test/test_string_fns.c b/app/test/test_string_fns.c
index 5e105d2bb9..07e72bf680 100644
--- a/app/test/test_string_fns.c
+++ b/app/test/test_string_fns.c
@@ -182,4 +182,4 @@ test_string_fns(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(string_autotest, test_string_fns);
+REGISTER_FAST_TEST(string_autotest, True, True, test_string_fns);
diff --git a/app/test/test_tailq.c b/app/test/test_tailq.c
index 9520219b0a..5598bcd12d 100644
--- a/app/test/test_tailq.c
+++ b/app/test/test_tailq.c
@@ -125,4 +125,4 @@ test_tailq(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(tailq_autotest, test_tailq);
+REGISTER_FAST_TEST(tailq_autotest, True, True, test_tailq);
diff --git a/app/test/test_thash.c b/app/test/test_thash.c
index 53d9611e18..dd6468eb56 100644
--- a/app/test/test_thash.c
+++ b/app/test/test_thash.c
@@ -966,4 +966,4 @@ test_thash(void)
 	return unit_test_suite_runner(&thash_tests);
 }
 
-REGISTER_TEST_COMMAND(thash_autotest, test_thash);
+REGISTER_FAST_TEST(thash_autotest, True, True, test_thash);
diff --git a/app/test/test_thash_perf.c b/app/test/test_thash_perf.c
index 687582aa32..9dfd5d3c21 100644
--- a/app/test/test_thash_perf.c
+++ b/app/test/test_thash_perf.c
@@ -135,4 +135,4 @@ test_thash_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(thash_perf_autotest, test_thash_perf);
+REGISTER_PERF_TEST(thash_perf_autotest, test_thash_perf);
diff --git a/app/test/test_threads.c b/app/test/test_threads.c
index a4c4f651a4..6ed6b168d4 100644
--- a/app/test/test_threads.c
+++ b/app/test/test_threads.c
@@ -279,4 +279,4 @@ test_threads(void)
 	return unit_test_suite_runner(&threads_test_suite);
 }
 
-REGISTER_TEST_COMMAND(threads_autotest, test_threads);
+REGISTER_FAST_TEST(threads_autotest, True, True, test_threads);
diff --git a/app/test/test_ticketlock.c b/app/test/test_ticketlock.c
index 242c136478..25af697c0a 100644
--- a/app/test/test_ticketlock.c
+++ b/app/test/test_ticketlock.c
@@ -314,4 +314,4 @@ test_ticketlock(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(ticketlock_autotest, test_ticketlock);
+REGISTER_FAST_TEST(ticketlock_autotest, True, True, test_ticketlock);
diff --git a/app/test/test_timer.c b/app/test/test_timer.c
index 0c36dc9010..09608a1845 100644
--- a/app/test/test_timer.c
+++ b/app/test/test_timer.c
@@ -594,4 +594,4 @@ test_timer(void)
 	return TEST_SUCCESS;
 }
 
-REGISTER_TEST_COMMAND(timer_autotest, test_timer);
+REGISTER_FAST_TEST(timer_autotest, False, True, test_timer);
diff --git a/app/test/test_timer_perf.c b/app/test/test_timer_perf.c
index 0ede4b3e40..d2d74ebbc6 100644
--- a/app/test/test_timer_perf.c
+++ b/app/test/test_timer_perf.c
@@ -131,4 +131,4 @@ test_timer_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(timer_perf_autotest, test_timer_perf);
+REGISTER_PERF_TEST(timer_perf_autotest, test_timer_perf);
diff --git a/app/test/test_timer_racecond.c b/app/test/test_timer_racecond.c
index bb56ae8324..6f8b448ff8 100644
--- a/app/test/test_timer_racecond.c
+++ b/app/test/test_timer_racecond.c
@@ -172,4 +172,4 @@ test_timer_racecond(void)
 	return TEST_SUCCESS;
 }
 
-REGISTER_TEST_COMMAND(timer_racecond_autotest, test_timer_racecond);
+REGISTER_PERF_TEST(timer_racecond_autotest, test_timer_racecond);
diff --git a/app/test/test_trace.c b/app/test/test_trace.c
index ad4a394a29..96829caed5 100644
--- a/app/test/test_trace.c
+++ b/app/test/test_trace.c
@@ -250,4 +250,4 @@ test_trace(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(trace_autotest, test_trace);
+REGISTER_FAST_TEST(trace_autotest, True, True, test_trace);
diff --git a/app/test/test_trace_perf.c b/app/test/test_trace_perf.c
index 46ae7d8074..a6dd075722 100644
--- a/app/test/test_trace_perf.c
+++ b/app/test/test_trace_perf.c
@@ -179,4 +179,4 @@ test_trace_perf(void)
 	return TEST_SUCCESS;
 }
 
-REGISTER_TEST_COMMAND(trace_perf_autotest, test_trace_perf);
+REGISTER_PERF_TEST(trace_perf_autotest, test_trace_perf);
diff --git a/app/test/test_version.c b/app/test/test_version.c
index 1e1ff18656..21016237bc 100644
--- a/app/test/test_version.c
+++ b/app/test/test_version.c
@@ -25,4 +25,4 @@ test_version(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(version_autotest, test_version);
+REGISTER_FAST_TEST(version_autotest, True, True, test_version);
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v4 3/8] app/test: make telemetry data test buildable on windows
  2023-08-15 13:13 ` [PATCH v4 " Bruce Richardson
  2023-08-15 13:13   ` [PATCH v4 1/8] app/test: add new macros for various test types Bruce Richardson
  2023-08-15 13:13   ` [PATCH v4 2/8] app/test: tag tests with the test type Bruce Richardson
@ 2023-08-15 13:13   ` Bruce Richardson
  2023-08-15 13:13   ` [PATCH v4 4/8] app/test: build using per-file dependency matrix Bruce Richardson
                     ` (5 subsequent siblings)
  8 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-15 13:13 UTC (permalink / raw)
  To: dev; +Cc: ci, Bruce Richardson

To enable the building of the telemetry data tests file when building on
windows, we need to provide a stub implementation. That way, the test
file is buildable any time the library itself is built.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 app/test/test_telemetry_data.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/app/test/test_telemetry_data.c b/app/test/test_telemetry_data.c
index a960f44c1b..b2dc1d75db 100644
--- a/app/test/test_telemetry_data.c
+++ b/app/test/test_telemetry_data.c
@@ -2,11 +2,20 @@
  * Copyright 2020 Intel Corporation
  */
 
+#ifdef RTE_EXEC_ENV_WINDOWS
+#include "test.h"
+
+static int
+telemetry_data_autotest(void)
+{
+	return TEST_SKIPPED;
+}
+
+#else
+
 #include <string.h>
 #include <sys/socket.h>
-#ifndef RTE_EXEC_ENV_WINDOWS
 #include <sys/un.h>
-#endif
 #include <unistd.h>
 #include <limits.h>
 
@@ -604,5 +613,6 @@ telemetry_data_autotest(void)
 	close(sock);
 	return 0;
 }
+#endif /* windows/non-windows */
 
 REGISTER_TEST_COMMAND(telemetry_data_autotest, telemetry_data_autotest);
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v4 4/8] app/test: build using per-file dependency matrix
  2023-08-15 13:13 ` [PATCH v4 " Bruce Richardson
                     ` (2 preceding siblings ...)
  2023-08-15 13:13   ` [PATCH v4 3/8] app/test: make telemetry data test buildable on windows Bruce Richardson
@ 2023-08-15 13:13   ` Bruce Richardson
  2023-08-15 13:13   ` [PATCH v4 5/8] app/test: define unit tests suites based on test macros Bruce Richardson
                     ` (4 subsequent siblings)
  8 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-15 13:13 UTC (permalink / raw)
  To: dev; +Cc: ci, Bruce Richardson

Rather than using if-else constructs to selectively add or remove files
from the UT build, switch to a table-based approach where each file
lists out what libs or drivers it depends upon.

Initial version of this table was generated via analysis of the header
files included in each C file. The basic dependencies of the test binary
[cmdline, ring, mempool and mbuf] were then removed from the per-file
lists, as there is no point in checking them as the whole app will be
disabled if they are not present.

With the file list, the dependencies for the "utility" C-files are kept
separate, so that other tests which depend on the functions provided by
those files can have that dependency recorded properly. The basic
cryptodev tests also fall into this category as functions from the main
cryptodev test file are used by other crypto tests.

As well as the main table for internal dependencies, some test
files have separate external components too. A second, much smaller
table lists these dependencies.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 app/meson.build      |   8 +-
 app/test/meson.build | 823 ++++++++++++-------------------------------
 2 files changed, 238 insertions(+), 593 deletions(-)

diff --git a/app/meson.build b/app/meson.build
index 4fc1a83eba..0d8b618e7f 100644
--- a/app/meson.build
+++ b/app/meson.build
@@ -32,6 +32,11 @@ apps = [
         'test-security-perf',
 ]
 
+if get_option('tests')
+# build the auto test app if enabled.
+    apps += 'test'
+endif
+
 default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API']
 default_ldflags = []
 if get_option('default_library') == 'static' and not is_windows
@@ -106,6 +111,3 @@ foreach app:apps
             install_rpath: join_paths(get_option('prefix'), driver_install_path),
             install: true)
 endforeach
-
-# special case the autotests
-subdir('test')
diff --git a/app/test/meson.build b/app/test/meson.build
index 66897c14a3..2b885ae273 100644
--- a/app/test/meson.build
+++ b/app/test/meson.build
@@ -1,446 +1,230 @@
 # SPDX-License-Identifier: BSD-3-Clause
-# Copyright(c) 2017 Intel Corporation
-
-if not get_option('tests')
-    subdir_done()
-endif
-
-test_sources = files(
-        'commands.c',
-        'packet_burst_generator.c',
-        'test.c',
-        'test_acl.c',
-        'test_alarm.c',
-        'test_atomic.c',
-        'test_barrier.c',
-        'test_bitops.c',
-        'test_bitmap.c',
-        'test_bpf.c',
-        'test_byteorder.c',
-        'test_cksum.c',
-        'test_cksum_perf.c',
-        'test_cmdline.c',
-        'test_cmdline_cirbuf.c',
-        'test_cmdline_etheraddr.c',
-        'test_cmdline_ipaddr.c',
-        'test_cmdline_lib.c',
-        'test_cmdline_num.c',
-        'test_cmdline_portlist.c',
-        'test_cmdline_string.c',
-        'test_common.c',
-        'test_cpuflags.c',
-        'test_crc.c',
-        'test_cryptodev.c',
-        'test_cryptodev_asym.c',
-        'test_cryptodev_blockcipher.c',
-        'test_cryptodev_crosscheck.c',
-        'test_cryptodev_security_ipsec.c',
-        'test_cryptodev_security_pdcp.c',
-        'test_cycles.c',
-        'test_debug.c',
-        'test_devargs.c',
-        'test_distributor.c',
-        'test_distributor_perf.c',
-        'test_dmadev.c',
-        'test_dmadev_api.c',
-        'test_eal_flags.c',
-        'test_eal_fs.c',
-        'test_efd.c',
-        'test_efd_perf.c',
-        'test_errno.c',
-        'test_ethdev_link.c',
-        'test_event_crypto_adapter.c',
-        'test_event_eth_rx_adapter.c',
-        'test_event_ring.c',
-        'test_event_timer_adapter.c',
-        'test_eventdev.c',
-        'test_external_mem.c',
-        'test_fbarray.c',
-        'test_fib.c',
-        'test_fib_perf.c',
-        'test_fib6.c',
-        'test_fib6_perf.c',
-        'test_func_reentrancy.c',
-        'test_hash.c',
-        'test_hash_functions.c',
-        'test_hash_multiwriter.c',
-        'test_hash_readwrite.c',
-        'test_hash_perf.c',
-        'test_hash_readwrite_lf_perf.c',
-        'test_interrupts.c',
-        'test_ipfrag.c',
-        'test_ipsec.c',
-        'test_ipsec_sad.c',
-        'test_ipsec_perf.c',
-        'test_kvargs.c',
-        'test_lcores.c',
-        'test_logs.c',
-        'test_lpm.c',
-        'test_lpm6.c',
-        'test_lpm6_perf.c',
-        'test_lpm_perf.c',
-        'test_malloc.c',
-        'test_malloc_perf.c',
-        'test_mbuf.c',
-        'test_member.c',
-        'test_member_perf.c',
-        'test_memcpy.c',
-        'test_memcpy_perf.c',
-        'test_memory.c',
-        'test_mempool.c',
-        'test_mempool_perf.c',
-        'test_memzone.c',
-        'test_meter.c',
-        'test_mcslock.c',
-        'test_mp_secondary.c',
-        'test_per_lcore.c',
-        'test_pflock.c',
-        'test_pmd_perf.c',
-        'test_power.c',
-        'test_power_cpufreq.c',
-        'test_power_kvm_vm.c',
-        'test_power_intel_uncore.c',
-        'test_prefetch.c',
-        'test_rand_perf.c',
-        'test_rawdev.c',
-        'test_rcu_qsbr.c',
-        'test_rcu_qsbr_perf.c',
-        'test_reassembly_perf.c',
-        'test_reciprocal_division.c',
-        'test_reciprocal_division_perf.c',
-        'test_red.c',
-        'test_pie.c',
-        'test_reorder.c',
-        'test_rib.c',
-        'test_rib6.c',
-        'test_ring.c',
-        'test_ring_mpmc_stress.c',
-        'test_ring_hts_stress.c',
-        'test_ring_mt_peek_stress.c',
-        'test_ring_mt_peek_stress_zc.c',
-        'test_ring_perf.c',
-        'test_ring_rts_stress.c',
-        'test_ring_st_peek_stress.c',
-        'test_ring_st_peek_stress_zc.c',
-        'test_ring_stress.c',
-        'test_rwlock.c',
-        'test_sched.c',
-        'test_security.c',
-        'test_security_inline_macsec.c',
-        'test_security_inline_proto.c',
-        'test_seqlock.c',
-        'test_service_cores.c',
-        'test_spinlock.c',
-        'test_stack.c',
-        'test_stack_perf.c',
-        'test_string_fns.c',
-        'test_tailq.c',
-        'test_thash.c',
-        'test_thash_perf.c',
-        'test_threads.c',
-        'test_timer.c',
-        'test_timer_perf.c',
-        'test_timer_racecond.c',
-        'test_timer_secondary.c',
-        'test_ticketlock.c',
-        'test_trace.c',
-        'test_trace_register.c',
-        'test_trace_perf.c',
-        'test_version.c',
-        'virtual_pmd.c',
-)
-
-test_deps = dpdk_libs_enabled
-# as well as libs, the pci and vdev bus drivers are needed for a lot of tests
-test_deps += ['bus_pci', 'bus_vdev']
-
-# Each test is marked with flags:
-# - the first flag indicates whether the test can run in no-huge mode,
-# - the second flag indicates whether the test can run with ASan enabled,
-fast_tests = [
-        ['acl_autotest', true, true],
-        ['atomic_autotest', false, true],
-        ['bitmap_autotest', true, true],
-        ['bpf_autotest', true, true],
-        ['bpf_convert_autotest', true, true],
-        ['bitops_autotest', true, true],
-        ['byteorder_autotest', true, true],
-        ['cksum_autotest', true, true],
-        ['cmdline_autotest', true, true],
-        ['common_autotest', true, true],
-        ['cpuflags_autotest', true, true],
-        ['debug_autotest', true, true],
-        ['devargs_autotest', true, true],
-        ['eal_flags_c_opt_autotest', false, false],
-        ['eal_flags_main_opt_autotest', false, false],
-        ['eal_flags_n_opt_autotest', false, false],
-        ['eal_flags_hpet_autotest', false, false],
-        ['eal_flags_no_huge_autotest', false, false],
-        ['eal_flags_a_opt_autotest', false, false],
-        ['eal_flags_b_opt_autotest', false, false],
-        ['eal_flags_vdev_opt_autotest', false, false],
-        ['eal_flags_r_opt_autotest', false, false],
-        ['eal_flags_mem_autotest', false, false],
-        ['eal_flags_file_prefix_autotest', false, false],
-        ['eal_flags_misc_autotest', false, false],
-        ['eal_fs_autotest', true, true],
-        ['errno_autotest', true, true],
-        ['ethdev_link_status', true, true],
-        ['event_ring_autotest', true, true],
-        ['fib_autotest', true, true],
-        ['fib6_autotest', true, true],
-        ['func_reentrancy_autotest', false, true],
-        ['hash_autotest', true, true],
-        ['interrupt_autotest', true, true],
-        ['ipfrag_autotest', false, true],
-        ['lcores_autotest', true, true],
-        ['logs_autotest', true, true],
-        ['lpm_autotest', true, true],
-        ['lpm6_autotest', true, true],
-        ['malloc_autotest', false, true],
-        ['mbuf_autotest', false, true],
-        ['mcslock_autotest', false, true],
-        ['memcpy_autotest', true, true],
-        ['memory_autotest', false, true],
-        ['mempool_autotest', false, true],
-        ['memzone_autotest', false, true],
-        ['meter_autotest', true, true],
-        ['multiprocess_autotest', false, false],
-        ['per_lcore_autotest', true, true],
-        ['pflock_autotest', true, true],
-        ['prefetch_autotest', true, true],
-        ['rcu_qsbr_autotest', true, true],
-        ['pie_autotest', true, true],
-        ['rib_autotest', true, true],
-        ['rib6_autotest', true, true],
-        ['ring_autotest', true, true],
-        ['rwlock_test1_autotest', true, true],
-        ['rwlock_rda_autotest', true, true],
-        ['rwlock_rds_wrm_autotest', true, true],
-        ['rwlock_rde_wro_autotest', true, true],
-        ['sched_autotest', true, true],
-        ['security_autotest', false, true],
-        ['seqlock_autotest', true, true],
-        ['spinlock_autotest', true, true],
-        ['stack_autotest', false, true],
-        ['stack_lf_autotest', false, true],
-        ['string_autotest', true, true],
-        ['tailq_autotest', true, true],
-        ['ticketlock_autotest', true, true],
-        ['timer_autotest', false, true],
-        ['user_delay_us', true, true],
-        ['version_autotest', true, true],
-        ['crc_autotest', true, true],
-        ['distributor_autotest', false, true],
-        ['eventdev_common_autotest', true, true],
-        ['fbarray_autotest', true, true],
-        ['hash_readwrite_func_autotest', false, true],
-        ['ipsec_autotest', true, true],
-        ['kvargs_autotest', true, true],
-        ['member_autotest', true, true],
-        ['power_cpufreq_autotest', false, true],
-        ['power_autotest', true, true],
-        ['power_kvm_vm_autotest', false, true],
-        ['power_intel_uncore_autotest', true, true],
-        ['reorder_autotest', true, true],
-        ['service_autotest', true, true],
-        ['thash_autotest', true, true],
-        ['threads_autotest', true, true],
-        ['trace_autotest', true, true],
-]
-
-# Tests known to have issues or which don't belong in other tests lists.
-extra_test_names = [
-        'alarm_autotest', # ee00af60170b ("test: remove strict timing requirements some tests")
-        'red_autotest', # https://bugs.dpdk.org/show_bug.cgi?id=826
-]
-
-perf_test_names = [
-        'ring_perf_autotest',
-        'malloc_perf_autotest',
-        'mempool_perf_autotest',
-        'memcpy_perf_autotest',
-        'hash_perf_autotest',
-        'timer_perf_autotest',
-        'reciprocal_division',
-        'reciprocal_division_perf',
-        'lpm_perf_autotest',
-        'rib_slow_autotest',
-        'fib_slow_autotest',
-        'fib_perf_autotest',
-        'red_all',
-        'pie_all',
-        'barrier_autotest',
-        'hash_multiwriter_autotest',
-        'timer_racecond_autotest',
-        'efd_autotest',
-        'hash_functions_autotest',
-        'member_perf_autotest',
-        'efd_perf_autotest',
-        'lpm6_perf_autotest',
-        'rib6_slow_autotest',
-        'fib6_slow_autotest',
-        'fib6_perf_autotest',
-        'rcu_qsbr_perf_autotest',
-        'red_perf',
-        'pie_perf',
-        'distributor_perf_autotest',
-        'pmd_perf_autotest',
-        'service_perf_autotest',
-        'stack_perf_autotest',
-        'stack_lf_perf_autotest',
-        'rand_perf_autotest',
-        'hash_readwrite_perf_autotest',
-        'hash_readwrite_lf_perf_autotest',
-        'trace_perf_autotest',
-        'ipsec_perf_autotest',
-        'thash_perf_autotest',
-        'reassembly_perf_autotest',
-]
-
-driver_test_names = [
-        'cryptodev_aesni_gcm_autotest',
-        'cryptodev_aesni_mb_autotest',
-        'cryptodev_chacha_poly_mb_autotest',
-        'cryptodev_cn10k_autotest',
-        'cryptodev_cn9k_autotest',
-        'cryptodev_cpu_aesni_mb_autotest',
-        'cryptodev_cpu_aesni_gcm_autotest',
-        'cryptodev_dpaa2_sec_autotest',
-        'cryptodev_dpaa_sec_autotest',
-        'cryptodev_null_autotest',
-        'cryptodev_openssl_autotest',
-        'cryptodev_qat_autotest',
-        'cryptodev_qat_asym_autotest',
-        'cryptodev_qat_raw_api_autotest',
-        'cryptodev_sw_armv8_autotest',
-        'cryptodev_sw_kasumi_autotest',
-        'cryptodev_sw_mvsam_autotest',
-        'cryptodev_sw_snow3g_autotest',
-        'cryptodev_sw_zuc_autotest',
-        'cryptodev_uadk_autotest',
-        'dmadev_autotest',
-]
-
-dump_test_names = []
-
-if not is_windows
-    driver_test_names += [
-            'cryptodev_openssl_asym_autotest',
-            'eventdev_selftest_octeontx',
-            'eventdev_selftest_sw',
-    ]
-
-    dump_test_names += [
-            'dump_struct_sizes',
-            'dump_mempool',
-            'dump_malloc_stats',
-            'dump_devargs',
-            'dump_log_types',
-            'dump_ring',
-            'dump_physmem',
-            'dump_memzone',
-    ]
-endif
-
-# The following linkages are an exception to allow running the
-# unit tests without requiring that the developer install the
-# DPDK libraries.  Explicit linkage of drivers (plugin libraries)
-# in applications should not be used.
-if dpdk_conf.has('RTE_MEMPOOL_RING')
-    test_deps += 'mempool_ring'
-endif
-if dpdk_conf.has('RTE_MEMPOOL_STACK')
-    test_deps += 'mempool_stack'
-endif
-if dpdk_conf.has('RTE_EVENT_SKELETON')
-    test_deps += 'event_skeleton'
-endif
-
-if dpdk_conf.has('RTE_LIB_GRAPH')
-    test_sources += 'test_graph.c'
-    fast_tests += [['graph_autotest', true, true]]
-    fast_tests += [['node_list_dump', true, true]]
-    test_sources += 'test_graph_perf.c'
-    perf_test_names += 'graph_perf_autotest'
-endif
-if dpdk_conf.has('RTE_LIB_METRICS')
-    test_sources += ['test_metrics.c']
-    fast_tests += [['metrics_autotest', true, true]]
-endif
-if not is_windows and dpdk_conf.has('RTE_LIB_TELEMETRY')
-    test_sources += ['test_telemetry_json.c', 'test_telemetry_data.c']
-    fast_tests += [['telemetry_json_autotest', true, true]]
-    fast_tests += [['telemetry_data_autotest', true, true]]
-endif
-if dpdk_conf.has('RTE_LIB_PIPELINE')
-# pipeline lib depends on port and table libs, so those must be present
-# if pipeline library is.
-    test_sources += [
-            'test_table.c',
-            'test_table_acl.c',
-            'test_table_combined.c',
-            'test_table_pipeline.c',
-            'test_table_ports.c',
-            'test_table_tables.c',
-    ]
-    fast_tests += [['table_autotest', true, true]]
-endif
-
-# The following linkages of drivers are required because
-# they are used via a driver-specific API.
-if dpdk_conf.has('RTE_NET_BOND')
-    test_deps += 'net_bond'
-    test_sources += ['test_link_bonding.c', 'test_link_bonding_rssconf.c']
-    driver_test_names += ['link_bonding_autotest', 'link_bonding_rssconf_autotest']
-    if dpdk_conf.has('RTE_NET_RING')
-        test_sources += 'test_link_bonding_mode4.c'
-        driver_test_names += 'link_bonding_mode4_autotest'
-    endif
-endif
-if dpdk_conf.has('RTE_LIB_EVENTDEV') and dpdk_conf.has('RTE_NET_RING')
-    test_deps += 'net_ring'
-    test_sources += 'test_pmd_ring_perf.c'
-    test_sources += 'test_pmd_ring.c'
-    test_sources += 'test_event_eth_tx_adapter.c'
-    test_sources += 'sample_packet_forward.c'
-    fast_tests += [['ring_pmd_autotest', true, true]]
-    perf_test_names += 'ring_pmd_perf_autotest'
-    fast_tests += [['event_eth_tx_adapter_autotest', false, true]]
-    if dpdk_conf.has('RTE_LIB_BITRATESTATS')
-        test_sources += 'test_bitratestats.c'
-        fast_tests += [['bitratestats_autotest', true, true]]
-    endif
-    if dpdk_conf.has('RTE_LIB_LATENCYSTATS')
-        test_sources += 'test_latencystats.c'
-        fast_tests += [['latencystats_autotest', true, true]]
-    endif
-    if dpdk_conf.has('RTE_LIB_PDUMP')
-        test_sources += 'test_pdump.c'
-        fast_tests += [['pdump_autotest', true, false]]
-    endif
-endif
-if dpdk_conf.has('RTE_NET_NULL')
-    test_deps += 'net_null'
-    test_sources += 'test_vdev.c'
-    fast_tests += [['vdev_autotest', true, true]]
-endif
-if dpdk_conf.has('RTE_RAW_SKELETON')
-    test_deps += 'raw_skeleton'
-    fast_tests += [['rawdev_autotest', true, true]]
-endif
-
-if dpdk_conf.has('RTE_HAS_LIBPCAP')
-    ext_deps += pcap_dep
-    if dpdk_conf.has('RTE_LIB_PCAPNG')
-        test_sources += 'test_pcapng.c'
+# Copyright(c) 2017-2023 Intel Corporation
+
+# the main test files [test.c and commands.c] relies on these libraries
+deps += ['cmdline', 'ring', 'mempool', 'mbuf']
+sources += files('commands.c', 'test.c')
+
+# some other utility C files, providing functions used by various tests
+# so we need to include these deps in the dependency list for the files using those fns.
+packet_burst_generator_deps = ['net']
+sample_packet_forward_deps = ['net_ring', 'ethdev', 'bus_vdev']
+virtual_pmd_deps = ['ethdev', 'net', 'bus_pci']
+# test_cryptodev has material that other crypto tests need
+test_cryptodev_deps = ['bus_vdev', 'net', 'cryptodev', 'crypto_scheduler', 'security']
+
+source_file_deps = {
+    # The C files providing functionality to other test cases
+    'packet_burst_generator.c': packet_burst_generator_deps,
+#    'resource.c': [],          # unused currently.
+    'sample_packet_forward.c': sample_packet_forward_deps,
+    'virtual_pmd.c': virtual_pmd_deps,
+
+    # the various test_*.c files
+    'test_acl.c': ['net', 'acl'],
+    'test_alarm.c': [],
+    'test_atomic.c': ['hash'],
+    'test_barrier.c': [],
+    'test_bitmap.c': [],
+    'test_bitops.c': [],
+    'test_bitratestats.c': ['metrics', 'bitratestats', 'ethdev'] + sample_packet_forward_deps,
+    'test_bpf.c': ['bpf', 'net'],
+    'test_byteorder.c': [],
+#    'test_cfgfile.c': ['cfgfile'],
+    'test_cksum.c': ['net'],
+    'test_cksum_perf.c': ['net'],
+    'test_cmdline.c': [],
+    'test_cmdline_cirbuf.c': [],
+    'test_cmdline_etheraddr.c': ['net'],
+    'test_cmdline_ipaddr.c': [],
+    'test_cmdline_lib.c': [],
+    'test_cmdline_num.c': [],
+    'test_cmdline_portlist.c': [],
+    'test_cmdline_string.c': [],
+    'test_common.c': [],
+    'test_compressdev.c': ['compressdev'],
+    'test_cpuflags.c': [],
+    'test_crc.c': ['net'],
+    'test_cryptodev.c': test_cryptodev_deps,
+    'test_cryptodev_asym.c': ['bus_vdev'] + test_cryptodev_deps,
+    'test_cryptodev_blockcipher.c': test_cryptodev_deps,
+    'test_cryptodev_crosscheck.c': test_cryptodev_deps,
+    'test_cryptodev_security_ipsec.c': test_cryptodev_deps,
+    'test_cryptodev_security_pdcp.c': test_cryptodev_deps,
+    'test_cycles.c': [],
+    'test_debug.c': [],
+    'test_devargs.c': ['kvargs'],
+    'test_distributor.c': ['distributor'],
+    'test_distributor_perf.c': ['distributor'],
+    'test_dmadev.c': ['dmadev', 'bus_vdev'],
+    'test_dmadev_api.c': ['dmadev'],
+    'test_eal_flags.c': [],
+    'test_eal_fs.c': [],
+    'test_efd.c': ['efd', 'net'],
+    'test_efd_perf.c': ['efd', 'hash'],
+    'test_errno.c': [],
+    'test_ethdev_link.c': ['ethdev'],
+    'test_event_crypto_adapter.c': ['cryptodev', 'eventdev', 'bus_vdev'],
+    'test_event_eth_rx_adapter.c': ['ethdev', 'eventdev', 'bus_vdev'],
+    'test_event_eth_tx_adapter.c': ['bus_vdev', 'ethdev', 'net_ring', 'eventdev'],
+    'test_event_ring.c': ['eventdev'],
+    'test_event_timer_adapter.c': ['ethdev', 'eventdev', 'bus_vdev'],
+    'test_eventdev.c': ['eventdev', 'bus_vdev'],
+    'test_external_mem.c': [],
+    'test_fbarray.c': [],
+    'test_fib.c': ['net', 'fib'],
+    'test_fib6.c': ['rib', 'fib'],
+    'test_fib6_perf.c': ['fib'],
+    'test_fib_perf.c': ['net', 'fib'],
+    'test_flow_classify.c': ['net', 'acl', 'table', 'ethdev', 'flow_classify'],
+    'test_func_reentrancy.c': ['hash', 'lpm'],
+    'test_graph.c': ['graph'],
+    'test_graph_perf.c': ['graph'],
+    'test_hash.c': ['net', 'hash'],
+    'test_hash_functions.c': ['hash'],
+    'test_hash_multiwriter.c': ['hash'],
+    'test_hash_perf.c': ['hash'],
+    'test_hash_readwrite.c': ['hash'],
+    'test_hash_readwrite_lf_perf.c': ['hash'],
+    'test_interrupts.c': [],
+    'test_ipfrag.c': ['net', 'ip_frag'],
+    'test_ipsec.c': ['bus_vdev', 'net', 'cryptodev', 'ipsec', 'security'],
+    'test_ipsec_perf.c': ['net', 'ipsec'],
+    'test_ipsec_sad.c': ['ipsec'],
+    'test_kvargs.c': ['kvargs'],
+    'test_latencystats.c': ['ethdev', 'latencystats', 'metrics'] + sample_packet_forward_deps,
+    'test_lcores.c': [],
+    'test_link_bonding.c': ['ethdev', 'net_bond',
+        'net'] + packet_burst_generator_deps + virtual_pmd_deps,
+    'test_link_bonding_mode4.c': ['ethdev', 'net_ring', 'net_bond',
+        'net'] + packet_burst_generator_deps,
+    'test_link_bonding_rssconf.c': ['ethdev', 'bus_vdev', 'net_bond'],
+    'test_logs.c': [],
+    'test_lpm.c': ['net', 'lpm'],
+    'test_lpm6.c': ['lpm'],
+    'test_lpm6_perf.c': ['lpm'],
+    'test_lpm_perf.c': ['net', 'lpm'],
+    'test_malloc.c': [],
+    'test_malloc_perf.c': [],
+    'test_mbuf.c': ['net'],
+    'test_mcslock.c': [],
+    'test_member.c': ['member', 'net'],
+    'test_member_perf.c': ['hash', 'member'],
+    'test_memcpy.c': [],
+    'test_memcpy_perf.c': [],
+    'test_memory.c': [],
+    'test_mempool.c': [],
+    'test_mempool_perf.c': [],
+    'test_memzone.c': [],
+    'test_meter.c': ['meter'],
+    'test_metrics.c': ['metrics'],
+    'test_mp_secondary.c': ['hash', 'lpm'],
+    'test_pcapng.c': ['ethdev', 'net', 'pcapng'],
+    'test_pdcp.c': ['eventdev', 'pdcp', 'net', 'timer', 'security'],
+    'test_pdump.c': ['pdump'] + sample_packet_forward_deps,
+    'test_per_lcore.c': [],
+    'test_pflock.c': [],
+    'test_pie.c': ['sched'],
+    'test_pmd_perf.c': ['ethdev', 'net'] + packet_burst_generator_deps,
+    'test_pmd_ring.c': ['net_ring', 'ethdev', 'bus_vdev'],
+    'test_pmd_ring_perf.c': ['ethdev', 'net_ring', 'bus_vdev'],
+    'test_power.c': ['power'],
+    'test_power_cpufreq.c': ['power'],
+    'test_power_intel_uncore.c': ['power'],
+    'test_power_kvm_vm.c': ['power'],
+    'test_prefetch.c': [],
+    'test_rand_perf.c': [],
+    'test_rawdev.c': ['rawdev', 'bus_vdev'],
+    'test_rcu_qsbr.c': ['rcu', 'hash'],
+    'test_rcu_qsbr_perf.c': ['rcu', 'hash'],
+    'test_reassembly_perf.c': ['net', 'ip_frag'],
+    'test_reciprocal_division.c': [],
+    'test_reciprocal_division_perf.c': [],
+    'test_red.c': ['sched'],
+    'test_reorder.c': ['reorder'],
+#    'test_resource.c': [],
+    'test_rib.c': ['net', 'rib'],
+    'test_rib6.c': ['net', 'rib'],
+    'test_ring.c': [],
+    'test_ring_hts_stress.c': [],
+    'test_ring_mpmc_stress.c': [],
+    'test_ring_mt_peek_stress.c': [],
+    'test_ring_mt_peek_stress_zc.c': [],
+    'test_ring_perf.c': [],
+    'test_ring_rts_stress.c': [],
+    'test_ring_st_peek_stress.c': [],
+    'test_ring_st_peek_stress_zc.c': [],
+    'test_ring_stress.c': [],
+    'test_rwlock.c': [],
+    'test_sched.c': ['net', 'sched'],
+    'test_security.c': ['net', 'security'],
+    'test_security_inline_macsec.c': ['ethdev', 'security'],
+    'test_security_inline_proto.c': ['ethdev', 'security', 'eventdev'] + test_cryptodev_deps,
+    'test_seqlock.c': [],
+    'test_service_cores.c': [],
+    'test_spinlock.c': [],
+    'test_stack.c': ['stack'],
+    'test_stack_perf.c': ['stack'],
+    'test_string_fns.c': [],
+    'test_table.c': ['table', 'pipeline', 'port'],
+    'test_table_acl.c': ['net', 'table', 'pipeline', 'port'],
+    'test_table_combined.c': ['table', 'pipeline', 'port'],
+    'test_table_pipeline.c': ['pipeline', 'table', 'port'],
+    'test_table_ports.c': ['table', 'pipeline', 'port'],
+    'test_table_tables.c': ['table', 'pipeline', 'port'],
+    'test_tailq.c': [],
+    'test_telemetry_data.c': ['telemetry'],
+    'test_telemetry_json.c': ['telemetry'],
+    'test_thash.c': ['net', 'hash'],
+    'test_thash_perf.c': ['hash'],
+    'test_threads.c': [],
+    'test_ticketlock.c': [],
+    'test_timer.c': ['timer'],
+    'test_timer_perf.c': ['timer'],
+    'test_timer_racecond.c': ['timer'],
+    'test_timer_secondary.c': ['timer'],
+    'test_trace.c': [],
+    'test_trace_perf.c': [],
+    'test_trace_register.c': [],
+    'test_vdev.c': ['kvargs', 'bus_vdev'],
+    'test_version.c': [],
+}
+
+source_file_ext_deps = {
+    'test_compressdev.c': ['zlib'],
+    'test_pcapng.c': ['pcap'],
+}
+
+def_lib = get_option('default_library')
+foreach f, f_deps : source_file_deps
+    has_deps = true
+    foreach d : f_deps
+        if not is_variable(def_lib + '_rte_' + d)
+            has_deps = false
+            break
+        else
+            # technically we might not need this dep, but adding it is harmless
+            if d not in deps
+                deps += d
+            endif
+        endif
+    endforeach
+    # check for any external dependencies for this file
+    if source_file_ext_deps.has_key(f)
+        foreach d: source_file_ext_deps.get(f)
+            dep = dependency(d, required: false, method: 'pkg-config')
+            if not dep.found()
+                message('Skipping test file @0@ due to missing external dependency @1@'.format(f, d))
+                has_deps = false
+            else
+                ext_deps += dep
+            endif
+        endforeach
+    endif
+    if has_deps
+        sources += files(f)
     endif
-endif
-
-if dpdk_conf.has('RTE_LIB_PDCP')
-    test_sources += 'test_pdcp.c'
-    fast_tests += [['pdcp_autotest', false, true]]
-endif
+endforeach
 
 if cc.has_argument('-Wno-format-truncation')
     cflags += '-Wno-format-truncation'
@@ -450,154 +234,13 @@ endif
 cflags += '-fno-strict-aliasing'
 
 # Enable using internal APIs in unit tests
-cflags += ['-DALLOW_INTERNAL_API']
-
-test_dep_objs = []
-if dpdk_conf.has('RTE_LIB_COMPRESSDEV')
-    compress_test_dep = dependency('zlib', required: false, method: 'pkg-config')
-    if compress_test_dep.found()
-        test_dep_objs += compress_test_dep
-        test_sources += 'test_compressdev.c'
-        fast_tests += [['compressdev_autotest', false, true]]
-    endif
-endif
-
-if dpdk_conf.has('RTE_CRYPTO_SCHEDULER')
-    driver_test_names += 'cryptodev_scheduler_autotest'
-    test_deps += 'crypto_scheduler'
-endif
-
-foreach d:test_deps
-    def_lib = get_option('default_library')
-    test_dep_objs += get_variable(def_lib + '_rte_' + d)
-endforeach
-
-link_libs = []
-if get_option('default_library') == 'static'
-    link_libs = dpdk_static_libraries + dpdk_drivers
-endif
-
-dpdk_test = executable('dpdk-test',
-        test_sources,
-        link_whole: link_libs,
-        dependencies: test_dep_objs + ext_deps,
-        c_args: cflags,
-        install_rpath: join_paths(get_option('prefix'),
-             driver_install_path),
-        install: true)
-
-has_hugepage = run_command(py3, files('has_hugepage.py'), check: true).stdout().strip() != '0'
-message('hugepage availability: @0@'.format(has_hugepage))
-
-# some perf tests (eg: memcpy perf autotest)take very long
-# to complete, so timeout to 10 minutes
-timeout_seconds = 600
-timeout_seconds_fast = 10
-
-test_no_huge_args = ['--no-huge', '-m', '2048']
-
-foreach arg : fast_tests
-    test_args = []
-    run_test = true
-    if not has_hugepage
-        if arg[1]
-            test_args += test_no_huge_args
-        else
-            run_test = false
-        endif
-    endif
-
-    if get_option('b_sanitize') == 'address' or get_option('b_sanitize') == 'address,undefined'
-        if not arg[2]
-            run_test = false
-        endif
-    endif
-
-    if (get_option('default_library') == 'shared' and
-        arg[0] == 'event_eth_tx_adapter_autotest')
-        test_args += ['-d', dpdk_drivers_build_dir]
-    endif
-    if is_linux
-        test_args += ['--file-prefix=@0@'.format(arg[0])]
-    endif
-
-    if run_test
-        test(arg[0], dpdk_test,
-                env : ['DPDK_TEST=' + arg[0]],
-                args : test_args,
-                timeout : timeout_seconds_fast,
-                is_parallel : false,
-                suite : 'fast-tests')
-        if not is_windows and arg[0] == 'trace_autotest'
-            test_args += ['--trace=.*']
-            test_args += ['--trace-dir=@0@'.format(meson.current_build_dir())]
-            test(arg[0] + '_with_traces', dpdk_test,
-                    env : ['DPDK_TEST=' + arg[0]],
-                    args : test_args,
-                    timeout : timeout_seconds_fast,
-                    is_parallel : false,
-                    suite : 'fast-tests')
-        endif
-    endif
-endforeach
+cflags += '-DALLOW_INTERNAL_API'
 
-if not is_windows and dpdk_conf.has('RTE_LIB_TELEMETRY')
-    test_args = [dpdk_test]
-    test_args += test_no_huge_args
-    if get_option('default_library') == 'shared'
-        test_args += ['-d', dpdk_drivers_build_dir]
-    endif
-    if dpdk_conf.has('RTE_CRYPTO_NULL')
-        test_args += ['--vdev=crypto_null0']
-    endif
-    if dpdk_conf.has('RTE_DMA_SKELETON')
-        test_args += ['--vdev=dma_skeleton0']
-    endif
-    if dpdk_conf.has('RTE_EVENT_SKELETON')
-        test_args += ['--vdev=event_skeleton0']
-    endif
-    if dpdk_conf.has('RTE_NET_NULL')
-        test_args += ['--vdev=net_null0']
-    endif
-    if dpdk_conf.has('RTE_RAW_SKELETON')
-        test_args += ['--vdev=rawdev_skeleton0']
-    endif
-    test_args += ['-a', '0000:00:00.0']
-    test('telemetry_all', find_program('test_telemetry.sh'),
-            args: test_args,
-            timeout : timeout_seconds_fast,
-            is_parallel : false,
-            suite : 'fast-tests')
+# create a symlink in the app/test directory for the binary, for backward compatibility
+if not is_windows
+    custom_target('test_symlink',
+            output: 'dpdk-test',
+            command: ['ln', '-sf', '../dpdk-test', '@OUTPUT@'],
+            build_by_default: true,
+            install: false)
 endif
-
-foreach arg : perf_test_names
-    test(arg, dpdk_test,
-            env : ['DPDK_TEST=' + arg],
-            timeout : timeout_seconds,
-            is_parallel : false,
-            suite : 'perf-tests')
-endforeach
-
-foreach arg : driver_test_names
-    test(arg, dpdk_test,
-            env : ['DPDK_TEST=' + arg],
-            timeout : timeout_seconds,
-            is_parallel : false,
-            suite : 'driver-tests')
-endforeach
-
-foreach arg : dump_test_names
-    test(arg, dpdk_test,
-            env : ['DPDK_TEST=' + arg],
-            timeout : timeout_seconds,
-            is_parallel : false,
-            suite : 'debug-tests')
-endforeach
-
-foreach arg : extra_test_names
-    test(arg, dpdk_test,
-            env : ['DPDK_TEST=' + arg],
-            timeout : timeout_seconds,
-            is_parallel : false,
-            suite : 'extra-tests')
-endforeach
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v4 5/8] app/test: define unit tests suites based on test macros
  2023-08-15 13:13 ` [PATCH v4 " Bruce Richardson
                     ` (3 preceding siblings ...)
  2023-08-15 13:13   ` [PATCH v4 4/8] app/test: build using per-file dependency matrix Bruce Richardson
@ 2023-08-15 13:13   ` Bruce Richardson
  2023-08-15 13:13   ` [PATCH v4 6/8] examples/l3fwd: make eventdev an optional dependency Bruce Richardson
                     ` (3 subsequent siblings)
  8 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-15 13:13 UTC (permalink / raw)
  To: dev; +Cc: ci, Bruce Richardson

Rather than having the test suites listed out in the meson.build files
and having to have them enabled/disabled selectively based on what libs
are being built, pull the tests to run from the source files which were
added to the build.

Most test suites require no additional info other than the list of test
names in the suite. However the fast-test are special that they have
additional parameters associated with them. This requires some
additional work in the test extraction script and in the meson.build
file for processing the output.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 app/meson.build                               |  7 +-
 app/test/suites/meson.build                   | 72 +++++++++++++++++++
 buildtools/get-test-suites.py                 | 33 +++++++++
 .../has-hugepages.py                          |  0
 buildtools/meson.build                        |  2 +
 5 files changed, 113 insertions(+), 1 deletion(-)
 create mode 100644 app/test/suites/meson.build
 create mode 100644 buildtools/get-test-suites.py
 rename app/test/has_hugepage.py => buildtools/has-hugepages.py (100%)

diff --git a/app/meson.build b/app/meson.build
index 0d8b618e7f..c14dc80892 100644
--- a/app/meson.build
+++ b/app/meson.build
@@ -101,7 +101,7 @@ foreach app:apps
         link_libs = dpdk_static_libraries + dpdk_drivers
     endif
 
-    executable('dpdk-' + name,
+    exec = executable('dpdk-' + name,
             sources,
             c_args: cflags,
             link_args: ldflags,
@@ -110,4 +110,9 @@ foreach app:apps
             include_directories: includes,
             install_rpath: join_paths(get_option('prefix'), driver_install_path),
             install: true)
+    if name == 'test'
+        dpdk_test = exec
+        autotest_sources = sources
+        subdir('test/suites')  # define the pre-canned test suites
+    endif
 endforeach
diff --git a/app/test/suites/meson.build b/app/test/suites/meson.build
new file mode 100644
index 0000000000..86884480a4
--- /dev/null
+++ b/app/test/suites/meson.build
@@ -0,0 +1,72 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2023 Intel Corporation
+
+# some perf tests (eg: memcpy perf autotest)take very long
+# to complete, so timeout to 10 minutes
+timeout_seconds = 600
+timeout_seconds_fast = 10
+
+test_no_huge_args = ['--no-huge', '-m', '2048']
+has_hugepage = run_command(has_hugepages_cmd, check: true).stdout().strip() != '0'
+message('hugepage availability: @0@'.format(has_hugepage))
+
+# process source files to determine the different unit test suites
+# - fast_tests
+# - perf_tests
+# - driver_tests
+test_suites = run_command(get_test_suites_cmd, autotest_sources,
+         check: true).stdout().strip().split()
+foreach suite:test_suites
+    # simple cases - tests without parameters or special handling
+    suite = suite.split('=')
+    suite_name = suite[0]
+    suite_tests = suite[1].split(',')
+    if suite_name != 'fast-tests'
+        # simple cases - tests without parameters or special handling
+        foreach t: suite_tests
+            test(t, dpdk_test,
+                    args: ['--', t],
+                    timeout: timeout_seconds,
+                    is_parallel: false,
+                    suite: suite_name)
+        endforeach
+    else
+    # special fast-test handling here
+        foreach t: suite_tests
+            params = t.split(':')
+            test_name = params[0]
+            nohuge = params[1] == 'true'
+            asan = params[2] == 'true'
+
+            test_args = []
+            if nohuge
+                test_args += test_no_huge_args
+            elif not has_hugepage
+                continue  #skip this tests
+            endif
+            if not asan and (get_option('b_sanitize') == 'address'
+                    or get_option('b_sanitize') == 'address,undefined')
+                continue  # skip this test
+            endif
+
+            if get_option('default_library') == 'shared'
+                test_args += ['-d', dpdk_drivers_build_dir]
+            endif
+
+            test(test_name, dpdk_test,
+                args : test_args + ['--', test_name],
+                timeout : timeout_seconds_fast,
+                is_parallel : false,
+                suite : 'fast-tests')
+            if not is_windows and test_name == 'trace_autotest'
+                test_args += ['--trace=.*']
+                test_args += ['--trace-dir=@0@'.format(meson.current_build_dir())]
+                test(test_name + '_with_traces', dpdk_test,
+                    args : test_args + ['--', test_name],
+                    timeout : timeout_seconds_fast,
+                    is_parallel : false,
+                    suite : 'fast-tests')
+            endif
+        endforeach
+    endif
+endforeach
diff --git a/buildtools/get-test-suites.py b/buildtools/get-test-suites.py
new file mode 100644
index 0000000000..95a9cad4c8
--- /dev/null
+++ b/buildtools/get-test-suites.py
@@ -0,0 +1,33 @@
+#! /usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2023 Intel Corporation
+
+import sys
+import re
+
+input_list = sys.argv[1:]
+test_def_regex = re.compile("REGISTER_([A-Z]+)_TEST\s*\(\s*([a-z0-9_]+)")
+test_suites = {}
+
+def get_fast_test_params(test_name, ln):
+    "Extract the extra fast-test parameters from the line"
+    #print(f"ln: {ln.rstrip()}, test_name: {test_name}, split: {ln.split(test_name, 1)}")
+    (_, rest_of_line) = ln.split(test_name, 1)
+    (_, nohuge, asan, _func) = rest_of_line.split(',', 3)
+    return f":{nohuge.strip().lower()}:{asan.strip().lower()}"
+
+for fname in input_list:
+    with open(fname) as f:
+        contents = [ln for ln in f.readlines() if test_def_regex.match(ln.strip())]
+    for ln in contents:
+        (test_suite, test_name) = test_def_regex.match(ln).group(1, 2)
+        suite_name = f"{test_suite.lower()}-tests"
+        if suite_name in test_suites:
+            test_suites[suite_name].append(test_name)
+        else:
+            test_suites[suite_name] = [test_name]
+        if suite_name == "fast-tests":
+            test_suites["fast-tests"][-1] += get_fast_test_params(test_name, ln)
+
+for suite in test_suites.keys():
+    print(f"{suite}={','.join(test_suites[suite])}")
diff --git a/app/test/has_hugepage.py b/buildtools/has-hugepages.py
similarity index 100%
rename from app/test/has_hugepage.py
rename to buildtools/has-hugepages.py
diff --git a/buildtools/meson.build b/buildtools/meson.build
index e1c600e40f..ac5e4dcf08 100644
--- a/buildtools/meson.build
+++ b/buildtools/meson.build
@@ -18,6 +18,8 @@ map_to_win_cmd = py3 + files('map_to_win.py')
 sphinx_wrapper = py3 + files('call-sphinx-build.py')
 get_cpu_count_cmd = py3 + files('get-cpu-count.py')
 get_numa_count_cmd = py3 + files('get-numa-count.py')
+get_test_suites_cmd = py3 + files('get-test-suites.py')
+has_hugepages_cmd = py3 + files('has-hugepages.py')
 binutils_avx512_check = (py3 + files('binutils-avx512-check.py') +
                         [objdump] + cc.cmd_array())
 
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v4 6/8] examples/l3fwd: make eventdev an optional dependency
  2023-08-15 13:13 ` [PATCH v4 " Bruce Richardson
                     ` (4 preceding siblings ...)
  2023-08-15 13:13   ` [PATCH v4 5/8] app/test: define unit tests suites based on test macros Bruce Richardson
@ 2023-08-15 13:13   ` Bruce Richardson
  2023-08-15 13:13   ` [PATCH v4 7/8] build: make most device classes optional Bruce Richardson
                     ` (2 subsequent siblings)
  8 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-15 13:13 UTC (permalink / raw)
  To: dev; +Cc: ci, Bruce Richardson

With l3fwd being a very commonly used example app, and built as part of
our CI tests, we need to ensure it's buildable with just about all
supported DPDK configurations.  To enable l3fwd application to be built
when the eventdev library is disabled, we need to compile in the
eventdev support conditionally.  Thankfully, the eventdev support is
pretty self-contained, with only the main.c file having more than a
couple of ifdefs.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 examples/l3fwd/l3fwd_em.c                  |  2 +
 examples/l3fwd/l3fwd_em_hlm.h              |  2 +
 examples/l3fwd/l3fwd_event.c               |  2 +
 examples/l3fwd/l3fwd_event.h               |  7 ++-
 examples/l3fwd/l3fwd_event_generic.c       |  2 +
 examples/l3fwd/l3fwd_event_internal_port.c |  2 +
 examples/l3fwd/l3fwd_fib.c                 |  2 +
 examples/l3fwd/l3fwd_lpm.c                 |  2 +
 examples/l3fwd/main.c                      | 65 +++++++++++++++++-----
 examples/l3fwd/meson.build                 |  5 +-
 10 files changed, 74 insertions(+), 17 deletions(-)

diff --git a/examples/l3fwd/l3fwd_em.c b/examples/l3fwd/l3fwd_em.c
index 476ac0c54f..40e102b38a 100644
--- a/examples/l3fwd/l3fwd_em.c
+++ b/examples/l3fwd/l3fwd_em.c
@@ -663,6 +663,7 @@ em_main_loop(__rte_unused void *dummy)
 	return 0;
 }
 
+#ifdef RTE_LIB_EVENTDEV
 static __rte_always_inline void
 em_event_loop_single(struct l3fwd_event_resources *evt_rsrc,
 		const uint8_t flags)
@@ -959,6 +960,7 @@ em_event_main_loop_tx_q_burst_vector(__rte_unused void *dummy)
 	em_event_loop_vector(evt_rsrc, L3FWD_EVENT_TX_ENQ);
 	return 0;
 }
+#endif
 
 /* Initialize exact match (hash) parameters. 8< */
 void
diff --git a/examples/l3fwd/l3fwd_em_hlm.h b/examples/l3fwd/l3fwd_em_hlm.h
index 2e11eefad7..31cda9ddc1 100644
--- a/examples/l3fwd/l3fwd_em_hlm.h
+++ b/examples/l3fwd/l3fwd_em_hlm.h
@@ -255,6 +255,7 @@ l3fwd_em_send_packets(int nb_rx, struct rte_mbuf **pkts_burst, uint16_t portid,
 	send_packets_multi(qconf, pkts_burst, dst_port, nb_rx);
 }
 
+#ifdef RTE_LIB_EVENTDEV
 /*
  * Buffer optimized handling of events, invoked
  * from main_loop.
@@ -347,5 +348,6 @@ l3fwd_em_process_event_vector(struct rte_event_vector *vec,
 
 	process_event_vector(vec, dst_port);
 }
+#endif /* RTE_LIB_EVENTDEV */
 
 #endif /* __L3FWD_EM_HLM_H__ */
diff --git a/examples/l3fwd/l3fwd_event.c b/examples/l3fwd/l3fwd_event.c
index 32906ab08d..d72a4138cc 100644
--- a/examples/l3fwd/l3fwd_event.c
+++ b/examples/l3fwd/l3fwd_event.c
@@ -2,6 +2,7 @@
  * Copyright(C) 2019 Marvell International Ltd.
  */
 
+#ifdef RTE_LIB_EVENTDEV
 #include <stdbool.h>
 #include <getopt.h>
 
@@ -341,3 +342,4 @@ l3fwd_event_worker_cleanup(uint8_t event_d_id, uint8_t event_p_id,
 	rte_event_port_quiesce(event_d_id, event_p_id, l3fwd_event_port_flush,
 			       NULL);
 }
+#endif /* #ifdef RTE_LIB_EVENTDEV */
diff --git a/examples/l3fwd/l3fwd_event.h b/examples/l3fwd/l3fwd_event.h
index e21817c36b..1fd6fe4a78 100644
--- a/examples/l3fwd/l3fwd_event.h
+++ b/examples/l3fwd/l3fwd_event.h
@@ -6,11 +6,13 @@
 #define __L3FWD_EVENTDEV_H__
 
 #include <rte_common.h>
+#include <rte_service.h>
+#include <rte_spinlock.h>
+
+#ifdef RTE_LIB_EVENTDEV
 #include <rte_eventdev.h>
 #include <rte_event_eth_rx_adapter.h>
 #include <rte_event_eth_tx_adapter.h>
-#include <rte_service.h>
-#include <rte_spinlock.h>
 
 #include "l3fwd.h"
 
@@ -164,4 +166,5 @@ void l3fwd_event_worker_cleanup(uint8_t event_d_id, uint8_t event_p_id,
 				struct rte_event events[], uint16_t nb_enq,
 				uint16_t nb_deq, uint8_t is_vector);
 
+#endif /* ifdef RTE_LIB_EVENTDEV */
 #endif /* __L3FWD_EVENTDEV_H__ */
diff --git a/examples/l3fwd/l3fwd_event_generic.c b/examples/l3fwd/l3fwd_event_generic.c
index c80573fc58..ddb6e5c38d 100644
--- a/examples/l3fwd/l3fwd_event_generic.c
+++ b/examples/l3fwd/l3fwd_event_generic.c
@@ -2,6 +2,7 @@
  * Copyright(C) 2019 Marvell International Ltd.
  */
 
+#ifdef RTE_LIB_EVENTDEV
 #include <stdbool.h>
 
 #include "l3fwd.h"
@@ -309,3 +310,4 @@ l3fwd_event_set_generic_ops(struct l3fwd_event_setup_ops *ops)
 	ops->event_port_setup = l3fwd_event_port_setup_generic;
 	ops->adapter_setup = l3fwd_rx_tx_adapter_setup_generic;
 }
+#endif /* RTE_LIB_EVENTDEV */
diff --git a/examples/l3fwd/l3fwd_event_internal_port.c b/examples/l3fwd/l3fwd_event_internal_port.c
index 32cf657148..cb49a8b9fa 100644
--- a/examples/l3fwd/l3fwd_event_internal_port.c
+++ b/examples/l3fwd/l3fwd_event_internal_port.c
@@ -2,6 +2,7 @@
  * Copyright(C) 2019 Marvell International Ltd.
  */
 
+#ifdef RTE_LIB_EVENTDEV
 #include <stdbool.h>
 
 #include "l3fwd.h"
@@ -311,3 +312,4 @@ l3fwd_event_set_internal_port_ops(struct l3fwd_event_setup_ops *ops)
 	ops->event_port_setup = l3fwd_event_port_setup_internal_port;
 	ops->adapter_setup = l3fwd_rx_tx_adapter_setup_internal_port;
 }
+#endif /* RTE_LIB_EVENTDEV */
diff --git a/examples/l3fwd/l3fwd_fib.c b/examples/l3fwd/l3fwd_fib.c
index 18398492ae..6a21984415 100644
--- a/examples/l3fwd/l3fwd_fib.c
+++ b/examples/l3fwd/l3fwd_fib.c
@@ -253,6 +253,7 @@ fib_main_loop(__rte_unused void *dummy)
 	return 0;
 }
 
+#ifdef RTE_LIB_EVENTDEV
 /* One eventdev loop for single and burst using fib. */
 static __rte_always_inline void
 fib_event_loop(struct l3fwd_event_resources *evt_rsrc,
@@ -635,6 +636,7 @@ fib_event_main_loop_tx_q_burst_vector(__rte_unused void *dummy)
 	fib_event_loop_vector(evt_rsrc, L3FWD_EVENT_TX_ENQ);
 	return 0;
 }
+#endif
 
 /* Function to setup fib. 8< */
 void
diff --git a/examples/l3fwd/l3fwd_lpm.c b/examples/l3fwd/l3fwd_lpm.c
index 4ac1925c84..a484a33089 100644
--- a/examples/l3fwd/l3fwd_lpm.c
+++ b/examples/l3fwd/l3fwd_lpm.c
@@ -226,6 +226,7 @@ lpm_main_loop(__rte_unused void *dummy)
 	return 0;
 }
 
+#ifdef RTE_LIB_EVENTDEV
 static __rte_always_inline uint16_t
 lpm_process_event_pkt(const struct lcore_conf *lconf, struct rte_mbuf *mbuf)
 {
@@ -554,6 +555,7 @@ lpm_event_main_loop_tx_q_burst_vector(__rte_unused void *dummy)
 	lpm_event_loop_vector(evt_rsrc, L3FWD_EVENT_TX_ENQ);
 	return 0;
 }
+#endif
 
 void
 setup_lpm(const int socketid)
diff --git a/examples/l3fwd/main.c b/examples/l3fwd/main.c
index a4f061537e..6063eb1399 100644
--- a/examples/l3fwd/main.c
+++ b/examples/l3fwd/main.c
@@ -135,8 +135,10 @@ static struct rte_eth_conf port_conf = {
 
 uint32_t max_pkt_len;
 
-static struct rte_mempool *pktmbuf_pool[RTE_MAX_ETHPORTS][NB_SOCKETS];
+#ifdef RTE_LIB_EVENTDEV
 static struct rte_mempool *vector_pool[RTE_MAX_ETHPORTS];
+#endif
+static struct rte_mempool *pktmbuf_pool[RTE_MAX_ETHPORTS][NB_SOCKETS];
 static uint8_t lkp_per_socket[NB_SOCKETS];
 
 struct l3fwd_lkp_mode {
@@ -398,8 +400,10 @@ print_usage(const char *prgname)
 		" [--parse-ptype]"
 		" [--per-port-pool]"
 		" [--mode]"
+#ifdef RTE_LIB_EVENTDEV
 		" [--eventq-sched]"
 		" [--event-vector [--event-vector-size SIZE] [--event-vector-tmo NS]]"
+#endif
 		" [-E]"
 		" [-L]\n\n"
 
@@ -422,6 +426,7 @@ print_usage(const char *prgname)
 		"  --per-port-pool: Use separate buffer pool per port\n"
 		"  --mode: Packet transfer mode for I/O, poll or eventdev\n"
 		"          Default mode = poll\n"
+#ifdef RTE_LIB_EVENTDEV
 		"  --eventq-sched: Event queue synchronization method\n"
 		"                  ordered, atomic or parallel.\n"
 		"                  Default: atomic\n"
@@ -432,6 +437,7 @@ print_usage(const char *prgname)
 		"  --event-vector:  Enable event vectorization.\n"
 		"  --event-vector-size: Max vector size if event vectorization is enabled.\n"
 		"  --event-vector-tmo: Max timeout to form vector in nanoseconds if event vectorization is enabled\n"
+#endif
 		"  -E : Enable exact match, legacy flag please use --lookup=em instead\n"
 		"  -L : Enable longest prefix match, legacy flag please use --lookup=lpm instead\n"
 		"  --rule_ipv4=FILE: Specify the ipv4 rules entries file.\n"
@@ -559,14 +565,16 @@ parse_eth_dest(const char *optarg)
 }
 
 static void
-parse_mode(const char *optarg)
+parse_mode(const char *optarg __rte_unused)
 {
+#ifdef RTE_LIB_EVENTDEV
 	struct l3fwd_event_resources *evt_rsrc = l3fwd_get_eventdev_rsrc();
 
 	if (!strcmp(optarg, "poll"))
 		evt_rsrc->enabled = false;
 	else if (!strcmp(optarg, "eventdev"))
 		evt_rsrc->enabled = true;
+#endif
 }
 
 static void
@@ -601,6 +609,7 @@ parse_queue_size(const char *queue_size_arg, uint16_t *queue_size, int rx)
 	*queue_size = value;
 }
 
+#ifdef RTE_LIB_EVENTDEV
 static void
 parse_eventq_sched(const char *optarg)
 {
@@ -631,6 +640,7 @@ parse_event_eth_rx_queues(const char *eth_rx_queues)
 
 	evt_rsrc->eth_rx_queues = num_eth_rx_queues;
 }
+#endif
 
 static int
 parse_lookup(const char *optarg)
@@ -756,9 +766,11 @@ parse_args(int argc, char **argv)
 	int option_index;
 	char *prgname = argv[0];
 	uint8_t lcore_params = 0;
+#ifdef RTE_LIB_EVENTDEV
 	uint8_t eventq_sched = 0;
 	uint8_t eth_rx_q = 0;
 	struct l3fwd_event_resources *evt_rsrc = l3fwd_get_eventdev_rsrc();
+#endif
 
 	argvopt = argv;
 
@@ -850,6 +862,7 @@ parse_args(int argc, char **argv)
 			parse_mode(optarg);
 			break;
 
+#ifdef RTE_LIB_EVENTDEV
 		case CMD_LINE_OPT_EVENTQ_SYNC_NUM:
 			parse_eventq_sched(optarg);
 			eventq_sched = 1;
@@ -860,6 +873,20 @@ parse_args(int argc, char **argv)
 			eth_rx_q = 1;
 			break;
 
+		case CMD_LINE_OPT_ENABLE_VECTOR_NUM:
+			printf("event vectorization is enabled\n");
+			evt_rsrc->vector_enabled = 1;
+			break;
+
+		case CMD_LINE_OPT_VECTOR_SIZE_NUM:
+			evt_rsrc->vector_size = strtol(optarg, NULL, 10);
+			break;
+
+		case CMD_LINE_OPT_VECTOR_TMO_NS_NUM:
+			evt_rsrc->vector_tmo_ns = strtoull(optarg, NULL, 10);
+			break;
+#endif
+
 		case CMD_LINE_OPT_LOOKUP_NUM:
 			if (lookup_mode != L3FWD_LOOKUP_DEFAULT) {
 				fprintf(stderr, "Only one lookup mode is allowed at a time!\n");
@@ -875,16 +902,6 @@ parse_args(int argc, char **argv)
 				return -1;
 			break;
 
-		case CMD_LINE_OPT_ENABLE_VECTOR_NUM:
-			printf("event vectorization is enabled\n");
-			evt_rsrc->vector_enabled = 1;
-			break;
-		case CMD_LINE_OPT_VECTOR_SIZE_NUM:
-			evt_rsrc->vector_size = strtol(optarg, NULL, 10);
-			break;
-		case CMD_LINE_OPT_VECTOR_TMO_NS_NUM:
-			evt_rsrc->vector_tmo_ns = strtoull(optarg, NULL, 10);
-			break;
 		case CMD_LINE_OPT_RULE_IPV4_NUM:
 			l3fwd_set_rule_ipv4_name(optarg);
 			break;
@@ -900,6 +917,8 @@ parse_args(int argc, char **argv)
 		}
 	}
 
+	RTE_SET_USED(lcore_params); /* needed if no eventdev block */
+#ifdef RTE_LIB_EVENTDEV
 	if (evt_rsrc->enabled && lcore_params) {
 		fprintf(stderr, "lcore config is not valid when event mode is selected\n");
 		return -1;
@@ -927,6 +946,7 @@ parse_args(int argc, char **argv)
 			"vector timeout set to default (%" PRIu64 " ns)\n",
 			evt_rsrc->vector_tmo_ns);
 	}
+#endif
 
 	/*
 	 * Nothing is selected, pick longest-prefix match
@@ -962,7 +982,9 @@ print_ethaddr(const char *name, const struct rte_ether_addr *eth_addr)
 int
 init_mem(uint16_t portid, unsigned int nb_mbuf)
 {
+#ifdef RTE_LIB_EVENTDEV
 	struct l3fwd_event_resources *evt_rsrc = l3fwd_get_eventdev_rsrc();
+#endif
 	struct lcore_conf *qconf;
 	int socketid;
 	unsigned lcore_id;
@@ -1007,6 +1029,7 @@ init_mem(uint16_t portid, unsigned int nb_mbuf)
 			}
 		}
 
+#ifdef RTE_LIB_EVENTDEV
 		if (evt_rsrc->vector_enabled && vector_pool[portid] == NULL) {
 			unsigned int nb_vec;
 
@@ -1025,6 +1048,7 @@ init_mem(uint16_t portid, unsigned int nb_mbuf)
 				printf("Allocated vector pool for port %d\n",
 				       portid);
 		}
+#endif
 
 		qconf = &lcore_conf[lcore_id];
 		qconf->ipv4_lookup_struct =
@@ -1406,6 +1430,7 @@ l3fwd_service_enable(uint32_t service_id)
 	return 0;
 }
 
+#ifdef RTE_LIB_EVENTDEV
 static void
 l3fwd_event_service_setup(void)
 {
@@ -1458,16 +1483,20 @@ l3fwd_event_service_setup(void)
 		l3fwd_service_enable(service_id);
 	}
 }
+#endif
 
 int
 main(int argc, char **argv)
 {
+#ifdef RTE_LIB_EVENTDEV
 	struct l3fwd_event_resources *evt_rsrc;
+	int i;
+#endif
 	struct lcore_conf *qconf;
 	uint16_t queueid, portid;
 	unsigned int lcore_id;
 	uint8_t queue;
-	int i, ret;
+	int ret;
 
 	/* init EAL */
 	ret = rte_eal_init(argc, argv);
@@ -1487,7 +1516,9 @@ main(int argc, char **argv)
 		*(uint64_t *)(val_eth + portid) = dest_eth_addr[portid];
 	}
 
+#ifdef RTE_LIB_EVENTDEV
 	evt_rsrc = l3fwd_get_eventdev_rsrc();
+#endif
 	/* parse application arguments (after the EAL ones) */
 	ret = parse_args(argc, argv);
 	if (ret < 0)
@@ -1499,6 +1530,7 @@ main(int argc, char **argv)
 	/* Add the config file rules */
 	l3fwd_lkp.read_config_files();
 
+#ifdef RTE_LIB_EVENTDEV
 	evt_rsrc->per_port_pool = per_port_pool;
 	evt_rsrc->pkt_pool = pktmbuf_pool;
 	evt_rsrc->vec_pool = vector_pool;
@@ -1514,6 +1546,7 @@ main(int argc, char **argv)
 			l3fwd_lkp.main_loop = evt_rsrc->ops.lpm_event_loop;
 		l3fwd_event_service_setup();
 	} else
+#endif
 		l3fwd_poll_resource_setup();
 
 	/* start ports */
@@ -1562,6 +1595,8 @@ main(int argc, char **argv)
 	ret = 0;
 	/* launch per-lcore init on every lcore */
 	rte_eal_mp_remote_launch(l3fwd_lkp.main_loop, NULL, CALL_MAIN);
+
+#ifdef RTE_LIB_EVENTDEV
 	if (evt_rsrc->enabled) {
 		for (i = 0; i < evt_rsrc->rx_adptr.nb_rx_adptr; i++)
 			rte_event_eth_rx_adapter_stop(
@@ -1589,7 +1624,9 @@ main(int argc, char **argv)
 		rte_event_dev_stop(evt_rsrc->event_d_id);
 		rte_event_dev_close(evt_rsrc->event_d_id);
 
-	} else {
+	} else
+#endif
+	{
 		rte_eal_mp_wait_lcore();
 
 		RTE_ETH_FOREACH_DEV(portid) {
diff --git a/examples/l3fwd/meson.build b/examples/l3fwd/meson.build
index b40244a941..c25de77bba 100644
--- a/examples/l3fwd/meson.build
+++ b/examples/l3fwd/meson.build
@@ -7,7 +7,7 @@
 # DPDK instance, use 'make'
 
 allow_experimental_apis = true
-deps += ['acl', 'hash', 'lpm', 'fib', 'eventdev']
+deps += ['acl', 'hash', 'lpm', 'fib']
 sources = files(
         'l3fwd_acl.c',
         'l3fwd_em.c',
@@ -18,3 +18,6 @@ sources = files(
         'l3fwd_lpm.c',
         'main.c',
 )
+if dpdk_conf.has('RTE_LIB_EVENTDEV')
+    deps += 'eventdev'
+endif
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v4 7/8] build: make most device classes optional
  2023-08-15 13:13 ` [PATCH v4 " Bruce Richardson
                     ` (5 preceding siblings ...)
  2023-08-15 13:13   ` [PATCH v4 6/8] examples/l3fwd: make eventdev an optional dependency Bruce Richardson
@ 2023-08-15 13:13   ` Bruce Richardson
  2023-08-15 13:13   ` [PATCH v4 8/8] build: expand list of optional libraries Bruce Richardson
  2023-08-15 13:15   ` [PATCH v4 0/8] " Bruce Richardson
  8 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-15 13:13 UTC (permalink / raw)
  To: dev; +Cc: ci, Bruce Richardson, Morten Brørup

Apart from ethdev and cryptodev, which have lots of components and tests
which depend on them, we can make the device class libraries optional
without too much work.

This patch marks:
* bbdev,
* compressdev,
* dmadev,
* eventdev,
* mldev,
* rawdev,
* regexdev
optional, and ensures that DPDK - including tests - can be built with
these components disabled.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
---
 lib/meson.build | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/lib/meson.build b/lib/meson.build
index 92cbd6bb66..1a78c8c984 100644
--- a/lib/meson.build
+++ b/lib/meson.build
@@ -67,8 +67,12 @@ libraries = [
 ]
 
 optional_libs = [
+        'bbdev',
         'bitratestats',
         'cfgfile',
+        'compressdev',
+        'dmadev',
+        'eventdev',
         'gpudev',
         'graph',
         'gro',
@@ -76,11 +80,14 @@ optional_libs = [
         'jobstats',
         'latencystats',
         'metrics',
+        'mldev',
         'node',
         'pdump',
         'pipeline',
         'port',
         'power',
+        'rawdev',
+        'regexdev',
         'table',
         'vhost',
 ]
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v4 8/8] build: expand list of optional libraries
  2023-08-15 13:13 ` [PATCH v4 " Bruce Richardson
                     ` (6 preceding siblings ...)
  2023-08-15 13:13   ` [PATCH v4 7/8] build: make most device classes optional Bruce Richardson
@ 2023-08-15 13:13   ` Bruce Richardson
  2023-08-15 13:15   ` [PATCH v4 0/8] " Bruce Richardson
  8 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-15 13:13 UTC (permalink / raw)
  To: dev; +Cc: ci, Bruce Richardson

With the unit test build now with individual per-file dependencies, we
can more easily expand the list of optional libraries. Add 8 new
libraries to the optional list.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 lib/meson.build | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/lib/meson.build b/lib/meson.build
index 1a78c8c984..0c9dc8b3b6 100644
--- a/lib/meson.build
+++ b/lib/meson.build
@@ -69,25 +69,33 @@ libraries = [
 optional_libs = [
         'bbdev',
         'bitratestats',
+        'bpf',
         'cfgfile',
         'compressdev',
+        'distributor',
         'dmadev',
+        'efd',
         'eventdev',
         'gpudev',
         'graph',
         'gro',
         'gso',
+        'ip_frag',
         'jobstats',
         'latencystats',
+        'member',
         'metrics',
         'mldev',
         'node',
+        'pcapng',
         'pdump',
         'pipeline',
         'port',
         'power',
         'rawdev',
         'regexdev',
+        'reorder',
+        'sched',
         'table',
         'vhost',
 ]
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* Re: [PATCH v4 0/8] expand list of optional libraries
  2023-08-15 13:13 ` [PATCH v4 " Bruce Richardson
                     ` (7 preceding siblings ...)
  2023-08-15 13:13   ` [PATCH v4 8/8] build: expand list of optional libraries Bruce Richardson
@ 2023-08-15 13:15   ` Bruce Richardson
  8 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-15 13:15 UTC (permalink / raw)
  To: dev; +Cc: ci

On Tue, Aug 15, 2023 at 02:13:12PM +0100, Bruce Richardson wrote:
> This patchset contains changes to DPDK to make the build more configurable
> overall. The changes can be grouped into two areas:
> 
> * Firstly, there are changes to make the build of the unit tests more
>   flexible and maintainable. These 5 patches switch the unit tests from
>   having separate hard-coded lists of files and tests, with lots of
>   conditional logic to add/remove items from the list, to instead being
>   a single list of files and dependencies per-file. The actual lists of
>   test-suites are built up dynamically from the list of files whose
>   dependencies are met and therefore will be part of the build.
>   This rework enables...
> * The second set of patches which expands the list of libraries which
>   can be disabled at build-time, while still giving a successful build
>   of DPDK. Overall, after this set is applied, the list of optional
>   libraries in DPDK is approx doubled in size. [15 additions giving
>   31 optional libs]
> 
> NOTES:
> * as stated in the RFC, the test binary moves in the build as a result
>   of this set - from app/test/dpdk-test to app/dpdk-test.
> * running the suites of tests no longer works after patch 4 until patch
>   5 is applied. If this is an issue, the two patches can be merged on
>   apply. However, the individual tests are still runnable through the
>   test binary - it's only through "meson test" that there are issues, so
>   I'm keeping the patches separate since each is relatively complex in its
>   own right.
> * A number of files in the test directory were not being built, and seem
>   to have never been built with meson. This requires separate work, outside
>   the scope of the patchset to fix.
> * Similarly, some unit tests were never added to test suites. These can
>   now be found by using grep to find the old REGISTER_TEST_COMMAND macro.
>   Again, adding orphan tests to suites is outside the scope of this set.
> 
> V4:
> - improve test suite definition, hopefully improving test pass rate:
>   - drop unnecessary file-prefix for fast tests, since tests are not parallel
>   - always pass driver path flag in "meson test" for shared builds.
> 
Missed applying Morten's ack to the series in V4. Adding it here.

Series-acked-by: Morten Brørup <mb@smartsharesystems.com>

^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v5 00/10] expand list of optional libraries
  2023-07-21 11:51 [RFC PATCH 0/5] replace build code for unit tests Bruce Richardson
                   ` (8 preceding siblings ...)
  2023-08-15 13:13 ` [PATCH v4 " Bruce Richardson
@ 2023-08-15 15:10 ` Bruce Richardson
  2023-08-15 15:10   ` [PATCH v5 01/10] app/test: add new macros for various test types Bruce Richardson
                     ` (9 more replies)
  2023-08-16 15:34 ` [PATCH v6 00/11] " Bruce Richardson
  10 siblings, 10 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-15 15:10 UTC (permalink / raw)
  To: dev; +Cc: ci, Bruce Richardson

This patchset contains changes to DPDK to make the build more configurable
overall. The changes can be grouped into two areas:

* Firstly, there are changes to make the build of the unit tests more
  flexible and maintainable. These 7 patches switch the unit tests from
  having separate hard-coded lists of files and tests, with lots of
  conditional logic to add/remove items from the list, to instead being
  a single list of files and dependencies per-file. The actual lists of
  test-suites are built up dynamically from the list of files whose
  dependencies are met and therefore will be part of the build.
  This rework enables...
* The second set of patches which expands the list of libraries which
  can be disabled at build-time, while still giving a successful build
  of DPDK. Overall, after this set is applied, the list of optional
  libraries in DPDK is approx doubled in size. [15 additions giving
  31 optional libs]

NOTES:
* as stated in the RFC, the test binary moves in the build as a result
  of this set - from app/test/dpdk-test to app/dpdk-test.
* running the suites of tests no longer works after patch 4 until patch
  5-7 is applied. If this is an issue, the patches can be merged on
  apply. However, the individual tests are still runnable through the
  test binary - it's only through "meson test" that there are issues, so
  I'm keeping the patches separate since each is relatively complex in its
  own right.
* A number of files in the test directory were not being built, and seem
  to have never been built with meson. This requires separate work, outside
  the scope of the patchset to fix.
* Similarly, some unit tests were never added to test suites. These can
  now be found by using grep to find the old REGISTER_TEST_COMMAND macro.
  Again, adding orphan tests to suites is outside the scope of this set.

V5:
- revert back to using the environment to pass the test names rather than
  the commandline. This allows use of "--test-args" with meson test to
  pass additional EAL parameters, such as coremasks, to the test run.
- add back in support for the script for checking telemetry commands
- add support for the dump-* tests in the "debug-tests" suite. Sadly,
  even though these are hardcoded in commands.c, they aren't easily
  discoverable at build-time, so we have to use a hard-coded list.

V4:
- improve test suite definition, hopefully improving test pass rate:
  - drop unnecessary file-prefix for fast tests, since tests are not parallel
  - always pass driver path flag in "meson test" for shared builds.

V3:
- Create symlink to old path to test binary, to ensure all automated CI,
  and any other use of the test binary still works.

V2:
- Changes since RFC:
   -  merged two previous sets - reworking test build, and disabling
      libs - into one, since they depend on each other
   -  Reworked the list of dependencies for each file in the tests,
      so that files that depend on other files have those deps called
      out.
   -  Added list of external dependencies to the test builds
   -  For disabling libs, merged patches disabling individual libs
      together into one patch, since all unit test build changes were
      unnecessary. This made each patch a single-line addition to
      the optional list in lib/meson.build.

Bruce Richardson (10):
  app/test: add new macros for various test types
  app/test: tag tests with the test type
  app/test: make telemetry data test buildable on windows
  app/test: build using per-file dependency matrix
  app/test: define unit tests suites based on test macros
  app/test: add test case for scripted telemetry commands
  app/test: add debug test suite
  examples/l3fwd: make eventdev an optional dependency
  build: make most device classes optional
  build: expand list of optional libraries

 app/meson.build                               |  15 +-
 app/test/meson.build                          | 823 +++++-------------
 app/test/suites/meson.build                   | 124 +++
 app/test/{ => suites}/test_telemetry.sh       |   0
 app/test/test.h                               |   9 +-
 app/test/test_acl.c                           |   2 +-
 app/test/test_atomic.c                        |   2 +-
 app/test/test_barrier.c                       |   2 +-
 app/test/test_bitmap.c                        |   2 +-
 app/test/test_bitops.c                        |   2 +-
 app/test/test_bpf.c                           |   4 +-
 app/test/test_byteorder.c                     |   2 +-
 app/test/test_cksum.c                         |   2 +-
 app/test/test_cmdline.c                       |   2 +-
 app/test/test_common.c                        |   2 +-
 app/test/test_cpuflags.c                      |   2 +-
 app/test/test_crc.c                           |   2 +-
 app/test/test_cryptodev.c                     |  38 +-
 app/test/test_cryptodev_asym.c                |   2 +-
 app/test/test_cycles.c                        |   2 +-
 app/test/test_debug.c                         |   2 +-
 app/test/test_devargs.c                       |   2 +-
 app/test/test_distributor.c                   |   2 +-
 app/test/test_distributor_perf.c              |   2 +-
 app/test/test_dmadev.c                        |   2 +-
 app/test/test_eal_flags.c                     |  24 +-
 app/test/test_eal_fs.c                        |   2 +-
 app/test/test_efd.c                           |   2 +-
 app/test/test_efd_perf.c                      |   2 +-
 app/test/test_errno.c                         |   2 +-
 app/test/test_ethdev_link.c                   |   2 +-
 app/test/test_event_ring.c                    |   2 +-
 app/test/test_eventdev.c                      |   2 +-
 app/test/test_fbarray.c                       |   2 +-
 app/test/test_fib.c                           |   4 +-
 app/test/test_fib6.c                          |   4 +-
 app/test/test_fib6_perf.c                     |   2 +-
 app/test/test_fib_perf.c                      |   2 +-
 app/test/test_func_reentrancy.c               |   2 +-
 app/test/test_hash.c                          |   2 +-
 app/test/test_hash_functions.c                |   2 +-
 app/test/test_hash_multiwriter.c              |   2 +-
 app/test/test_hash_perf.c                     |   2 +-
 app/test/test_hash_readwrite.c                |   4 +-
 app/test/test_hash_readwrite_lf_perf.c        |   2 +-
 app/test/test_interrupts.c                    |   2 +-
 app/test/test_ipfrag.c                        |   2 +-
 app/test/test_ipsec.c                         |   2 +-
 app/test/test_ipsec_perf.c                    |   2 +-
 app/test/test_kvargs.c                        |   2 +-
 app/test/test_lcores.c                        |   2 +-
 app/test/test_logs.c                          |   2 +-
 app/test/test_lpm.c                           |   2 +-
 app/test/test_lpm6.c                          |   2 +-
 app/test/test_lpm6_perf.c                     |   2 +-
 app/test/test_lpm_perf.c                      |   2 +-
 app/test/test_malloc.c                        |   2 +-
 app/test/test_malloc_perf.c                   |   2 +-
 app/test/test_mbuf.c                          |   2 +-
 app/test/test_mcslock.c                       |   2 +-
 app/test/test_member.c                        |   2 +-
 app/test/test_member_perf.c                   |   2 +-
 app/test/test_memcpy.c                        |   2 +-
 app/test/test_memcpy_perf.c                   |   2 +-
 app/test/test_memory.c                        |   2 +-
 app/test/test_mempool.c                       |   2 +-
 app/test/test_mempool_perf.c                  |   2 +-
 app/test/test_memzone.c                       |   2 +-
 app/test/test_meter.c                         |   2 +-
 app/test/test_mp_secondary.c                  |   2 +-
 app/test/test_per_lcore.c                     |   2 +-
 app/test/test_pflock.c                        |   2 +-
 app/test/test_pie.c                           |   6 +-
 app/test/test_pmd_perf.c                      |   2 +-
 app/test/test_power.c                         |   2 +-
 app/test/test_power_cpufreq.c                 |   2 +-
 app/test/test_power_intel_uncore.c            |   2 +-
 app/test/test_power_kvm_vm.c                  |   2 +-
 app/test/test_prefetch.c                      |   2 +-
 app/test/test_rand_perf.c                     |   2 +-
 app/test/test_rcu_qsbr.c                      |   2 +-
 app/test/test_rcu_qsbr_perf.c                 |   2 +-
 app/test/test_reassembly_perf.c               |   2 +-
 app/test/test_reciprocal_division.c           |   2 +-
 app/test/test_reciprocal_division_perf.c      |   2 +-
 app/test/test_red.c                           |   4 +-
 app/test/test_reorder.c                       |   2 +-
 app/test/test_rib.c                           |   4 +-
 app/test/test_rib6.c                          |   4 +-
 app/test/test_ring.c                          |   2 +-
 app/test/test_ring_perf.c                     |   2 +-
 app/test/test_rwlock.c                        |   8 +-
 app/test/test_sched.c                         |   2 +-
 app/test/test_security.c                      |   2 +-
 app/test/test_seqlock.c                       |   2 +-
 app/test/test_service_cores.c                 |   4 +-
 app/test/test_spinlock.c                      |   2 +-
 app/test/test_stack.c                         |   4 +-
 app/test/test_stack_perf.c                    |   4 +-
 app/test/test_string_fns.c                    |   2 +-
 app/test/test_tailq.c                         |   2 +-
 app/test/test_telemetry_data.c                |  14 +-
 app/test/test_thash.c                         |   2 +-
 app/test/test_thash_perf.c                    |   2 +-
 app/test/test_threads.c                       |   2 +-
 app/test/test_ticketlock.c                    |   2 +-
 app/test/test_timer.c                         |   2 +-
 app/test/test_timer_perf.c                    |   2 +-
 app/test/test_timer_racecond.c                |   2 +-
 app/test/test_trace.c                         |   2 +-
 app/test/test_trace_perf.c                    |   2 +-
 app/test/test_version.c                       |   2 +-
 buildtools/get-test-suites.py                 |  33 +
 .../has-hugepages.py                          |   0
 buildtools/meson.build                        |   2 +
 examples/l3fwd/l3fwd_em.c                     |   2 +
 examples/l3fwd/l3fwd_em_hlm.h                 |   2 +
 examples/l3fwd/l3fwd_event.c                  |   2 +
 examples/l3fwd/l3fwd_event.h                  |   7 +-
 examples/l3fwd/l3fwd_event_generic.c          |   2 +
 examples/l3fwd/l3fwd_event_internal_port.c    |   2 +
 examples/l3fwd/l3fwd_fib.c                    |   2 +
 examples/l3fwd/l3fwd_lpm.c                    |   2 +
 examples/l3fwd/main.c                         |  65 +-
 examples/l3fwd/meson.build                    |   5 +-
 lib/meson.build                               |  15 +
 126 files changed, 662 insertions(+), 764 deletions(-)
 create mode 100644 app/test/suites/meson.build
 rename app/test/{ => suites}/test_telemetry.sh (100%)
 create mode 100644 buildtools/get-test-suites.py
 rename app/test/has_hugepage.py => buildtools/has-hugepages.py (100%)

--
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v5 01/10] app/test: add new macros for various test types
  2023-08-15 15:10 ` [PATCH v5 00/10] " Bruce Richardson
@ 2023-08-15 15:10   ` Bruce Richardson
  2023-08-15 15:10   ` [PATCH v5 02/10] app/test: tag tests with the test type Bruce Richardson
                     ` (8 subsequent siblings)
  9 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-15 15:10 UTC (permalink / raw)
  To: dev; +Cc: ci, Bruce Richardson, Morten Brørup

Rather than just registering all tests using a single generic macro,
add macros which identify the test as being of a particular type.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
---
 app/test/test.h | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/app/test/test.h b/app/test/test.h
index 85f57efbc6..a91ded76af 100644
--- a/app/test/test.h
+++ b/app/test/test.h
@@ -190,7 +190,7 @@ struct test_command {
 
 void add_test_command(struct test_command *t);
 
-/* Register a test function with its command string */
+/* Register a test function with its command string. Should not be used directly */
 #define REGISTER_TEST_COMMAND(cmd, func) \
 	static struct test_command test_struct_##cmd = { \
 		.command = RTE_STR(cmd), \
@@ -201,4 +201,11 @@ void add_test_command(struct test_command *t);
 		add_test_command(&test_struct_##cmd); \
 	}
 
+/* Register a test function as a particular type.
+ * These can be used to build up test suites automatically
+ */
+#define REGISTER_FAST_TEST(cmd, no_huge, ASan, func)  REGISTER_TEST_COMMAND(cmd, func)
+#define REGISTER_PERF_TEST REGISTER_TEST_COMMAND
+#define REGISTER_DRIVER_TEST REGISTER_TEST_COMMAND
+
 #endif
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v5 02/10] app/test: tag tests with the test type
  2023-08-15 15:10 ` [PATCH v5 00/10] " Bruce Richardson
  2023-08-15 15:10   ` [PATCH v5 01/10] app/test: add new macros for various test types Bruce Richardson
@ 2023-08-15 15:10   ` Bruce Richardson
  2023-08-16 14:56     ` David Marchand
  2023-08-15 15:10   ` [PATCH v5 03/10] app/test: make telemetry data test buildable on windows Bruce Richardson
                     ` (7 subsequent siblings)
  9 siblings, 1 reply; 93+ messages in thread
From: Bruce Richardson @ 2023-08-15 15:10 UTC (permalink / raw)
  To: dev; +Cc: ci, Bruce Richardson, Morten Brørup

Rather than having the test types called out in the meson.build file, we
can use macros to identify the test type in the C file itself and then
dynamically build up the tests lists at config time.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
---
 app/test/test_acl.c                      |  2 +-
 app/test/test_atomic.c                   |  2 +-
 app/test/test_barrier.c                  |  2 +-
 app/test/test_bitmap.c                   |  2 +-
 app/test/test_bitops.c                   |  2 +-
 app/test/test_bpf.c                      |  4 +--
 app/test/test_byteorder.c                |  2 +-
 app/test/test_cksum.c                    |  2 +-
 app/test/test_cmdline.c                  |  2 +-
 app/test/test_common.c                   |  2 +-
 app/test/test_cpuflags.c                 |  2 +-
 app/test/test_crc.c                      |  2 +-
 app/test/test_cryptodev.c                | 38 ++++++++++++------------
 app/test/test_cryptodev_asym.c           |  2 +-
 app/test/test_cycles.c                   |  2 +-
 app/test/test_debug.c                    |  2 +-
 app/test/test_devargs.c                  |  2 +-
 app/test/test_distributor.c              |  2 +-
 app/test/test_distributor_perf.c         |  2 +-
 app/test/test_dmadev.c                   |  2 +-
 app/test/test_eal_flags.c                | 24 +++++++--------
 app/test/test_eal_fs.c                   |  2 +-
 app/test/test_efd.c                      |  2 +-
 app/test/test_efd_perf.c                 |  2 +-
 app/test/test_errno.c                    |  2 +-
 app/test/test_ethdev_link.c              |  2 +-
 app/test/test_event_ring.c               |  2 +-
 app/test/test_eventdev.c                 |  2 +-
 app/test/test_fbarray.c                  |  2 +-
 app/test/test_fib.c                      |  4 +--
 app/test/test_fib6.c                     |  4 +--
 app/test/test_fib6_perf.c                |  2 +-
 app/test/test_fib_perf.c                 |  2 +-
 app/test/test_func_reentrancy.c          |  2 +-
 app/test/test_hash.c                     |  2 +-
 app/test/test_hash_functions.c           |  2 +-
 app/test/test_hash_multiwriter.c         |  2 +-
 app/test/test_hash_perf.c                |  2 +-
 app/test/test_hash_readwrite.c           |  4 +--
 app/test/test_hash_readwrite_lf_perf.c   |  2 +-
 app/test/test_interrupts.c               |  2 +-
 app/test/test_ipfrag.c                   |  2 +-
 app/test/test_ipsec.c                    |  2 +-
 app/test/test_ipsec_perf.c               |  2 +-
 app/test/test_kvargs.c                   |  2 +-
 app/test/test_lcores.c                   |  2 +-
 app/test/test_logs.c                     |  2 +-
 app/test/test_lpm.c                      |  2 +-
 app/test/test_lpm6.c                     |  2 +-
 app/test/test_lpm6_perf.c                |  2 +-
 app/test/test_lpm_perf.c                 |  2 +-
 app/test/test_malloc.c                   |  2 +-
 app/test/test_malloc_perf.c              |  2 +-
 app/test/test_mbuf.c                     |  2 +-
 app/test/test_mcslock.c                  |  2 +-
 app/test/test_member.c                   |  2 +-
 app/test/test_member_perf.c              |  2 +-
 app/test/test_memcpy.c                   |  2 +-
 app/test/test_memcpy_perf.c              |  2 +-
 app/test/test_memory.c                   |  2 +-
 app/test/test_mempool.c                  |  2 +-
 app/test/test_mempool_perf.c             |  2 +-
 app/test/test_memzone.c                  |  2 +-
 app/test/test_meter.c                    |  2 +-
 app/test/test_mp_secondary.c             |  2 +-
 app/test/test_per_lcore.c                |  2 +-
 app/test/test_pflock.c                   |  2 +-
 app/test/test_pie.c                      |  6 ++--
 app/test/test_pmd_perf.c                 |  2 +-
 app/test/test_power.c                    |  2 +-
 app/test/test_power_cpufreq.c            |  2 +-
 app/test/test_power_intel_uncore.c       |  2 +-
 app/test/test_power_kvm_vm.c             |  2 +-
 app/test/test_prefetch.c                 |  2 +-
 app/test/test_rand_perf.c                |  2 +-
 app/test/test_rcu_qsbr.c                 |  2 +-
 app/test/test_rcu_qsbr_perf.c            |  2 +-
 app/test/test_reassembly_perf.c          |  2 +-
 app/test/test_reciprocal_division.c      |  2 +-
 app/test/test_reciprocal_division_perf.c |  2 +-
 app/test/test_red.c                      |  4 +--
 app/test/test_reorder.c                  |  2 +-
 app/test/test_rib.c                      |  4 +--
 app/test/test_rib6.c                     |  4 +--
 app/test/test_ring.c                     |  2 +-
 app/test/test_ring_perf.c                |  2 +-
 app/test/test_rwlock.c                   |  8 ++---
 app/test/test_sched.c                    |  2 +-
 app/test/test_security.c                 |  2 +-
 app/test/test_seqlock.c                  |  2 +-
 app/test/test_service_cores.c            |  4 +--
 app/test/test_spinlock.c                 |  2 +-
 app/test/test_stack.c                    |  4 +--
 app/test/test_stack_perf.c               |  4 +--
 app/test/test_string_fns.c               |  2 +-
 app/test/test_tailq.c                    |  2 +-
 app/test/test_thash.c                    |  2 +-
 app/test/test_thash_perf.c               |  2 +-
 app/test/test_threads.c                  |  2 +-
 app/test/test_ticketlock.c               |  2 +-
 app/test/test_timer.c                    |  2 +-
 app/test/test_timer_perf.c               |  2 +-
 app/test/test_timer_racecond.c           |  2 +-
 app/test/test_trace.c                    |  2 +-
 app/test/test_trace_perf.c               |  2 +-
 app/test/test_version.c                  |  2 +-
 106 files changed, 150 insertions(+), 150 deletions(-)

diff --git a/app/test/test_acl.c b/app/test/test_acl.c
index 623f34682e..bf1466fe11 100644
--- a/app/test/test_acl.c
+++ b/app/test/test_acl.c
@@ -1749,4 +1749,4 @@ test_acl(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(acl_autotest, test_acl);
+REGISTER_FAST_TEST(acl_autotest, True, True, test_acl);
diff --git a/app/test/test_atomic.c b/app/test/test_atomic.c
index e4b997827e..b52420c10b 100644
--- a/app/test/test_atomic.c
+++ b/app/test/test_atomic.c
@@ -631,4 +631,4 @@ test_atomic(void)
 
 	return 0;
 }
-REGISTER_TEST_COMMAND(atomic_autotest, test_atomic);
+REGISTER_FAST_TEST(atomic_autotest, False, True, test_atomic);
diff --git a/app/test/test_barrier.c b/app/test/test_barrier.c
index ec69af25bf..925a88b68a 100644
--- a/app/test/test_barrier.c
+++ b/app/test/test_barrier.c
@@ -285,4 +285,4 @@ test_barrier(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(barrier_autotest, test_barrier);
+REGISTER_PERF_TEST(barrier_autotest, test_barrier);
diff --git a/app/test/test_bitmap.c b/app/test/test_bitmap.c
index e9c61590ae..2014d2682c 100644
--- a/app/test/test_bitmap.c
+++ b/app/test/test_bitmap.c
@@ -269,4 +269,4 @@ test_bitmap(void)
 	return test_bitmap_all_set();
 }
 
-REGISTER_TEST_COMMAND(bitmap_autotest, test_bitmap);
+REGISTER_FAST_TEST(bitmap_autotest, True, True, test_bitmap);
diff --git a/app/test/test_bitops.c b/app/test/test_bitops.c
index c21426bf2f..dac466602b 100644
--- a/app/test/test_bitops.c
+++ b/app/test/test_bitops.c
@@ -135,4 +135,4 @@ test_bitops(void)
 	return TEST_SUCCESS;
 }
 
-REGISTER_TEST_COMMAND(bitops_autotest, test_bitops);
+REGISTER_FAST_TEST(bitops_autotest, True, True, test_bitops);
diff --git a/app/test/test_bpf.c b/app/test/test_bpf.c
index f5af5e8a3f..5fef17fbc2 100644
--- a/app/test/test_bpf.c
+++ b/app/test/test_bpf.c
@@ -3262,7 +3262,7 @@ test_bpf(void)
 
 #endif /* !RTE_LIB_BPF */
 
-REGISTER_TEST_COMMAND(bpf_autotest, test_bpf);
+REGISTER_FAST_TEST(bpf_autotest, True, True, test_bpf);
 
 #ifndef RTE_HAS_LIBPCAP
 
@@ -3473,4 +3473,4 @@ test_bpf_convert(void)
 
 #endif /* RTE_HAS_LIBPCAP */
 
-REGISTER_TEST_COMMAND(bpf_convert_autotest, test_bpf_convert);
+REGISTER_FAST_TEST(bpf_convert_autotest, True, True, test_bpf_convert);
diff --git a/app/test/test_byteorder.c b/app/test/test_byteorder.c
index de14ed539e..2189f94744 100644
--- a/app/test/test_byteorder.c
+++ b/app/test/test_byteorder.c
@@ -63,4 +63,4 @@ test_byteorder(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(byteorder_autotest, test_byteorder);
+REGISTER_FAST_TEST(byteorder_autotest, True, True, test_byteorder);
diff --git a/app/test/test_cksum.c b/app/test/test_cksum.c
index 6c15de9a93..c04fae800e 100644
--- a/app/test/test_cksum.c
+++ b/app/test/test_cksum.c
@@ -267,4 +267,4 @@ test_cksum(void)
 }
 #undef GOTO_FAIL
 
-REGISTER_TEST_COMMAND(cksum_autotest, test_cksum);
+REGISTER_FAST_TEST(cksum_autotest, True, True, test_cksum);
diff --git a/app/test/test_cmdline.c b/app/test/test_cmdline.c
index 115bee966d..c2a375ac1c 100644
--- a/app/test/test_cmdline.c
+++ b/app/test/test_cmdline.c
@@ -60,4 +60,4 @@ test_cmdline(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(cmdline_autotest, test_cmdline);
+REGISTER_FAST_TEST(cmdline_autotest, True, True, test_cmdline);
diff --git a/app/test/test_common.c b/app/test/test_common.c
index f89e1eb7ee..c7b41a3f0e 100644
--- a/app/test/test_common.c
+++ b/app/test/test_common.c
@@ -350,4 +350,4 @@ test_common(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(common_autotest, test_common);
+REGISTER_FAST_TEST(common_autotest, True, True, test_common);
diff --git a/app/test/test_cpuflags.c b/app/test/test_cpuflags.c
index a0e342ae48..c3417dda24 100644
--- a/app/test/test_cpuflags.c
+++ b/app/test/test_cpuflags.c
@@ -334,4 +334,4 @@ test_cpuflags(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(cpuflags_autotest, test_cpuflags);
+REGISTER_FAST_TEST(cpuflags_autotest, True, True, test_cpuflags);
diff --git a/app/test/test_crc.c b/app/test/test_crc.c
index 5edc8fb13b..b267bdde9f 100644
--- a/app/test/test_crc.c
+++ b/app/test/test_crc.c
@@ -171,4 +171,4 @@ test_crc(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(crc_autotest, test_crc);
+REGISTER_FAST_TEST(crc_autotest, True, True, test_crc);
diff --git a/app/test/test_cryptodev.c b/app/test/test_cryptodev.c
index fb2af40b99..956268bfcd 100644
--- a/app/test/test_cryptodev.c
+++ b/app/test/test_cryptodev.c
@@ -17535,33 +17535,33 @@ REGISTER_TEST_COMMAND(cryptodev_dpaa2_sec_raw_api_autotest,
 		test_cryptodev_dpaa2_sec_raw_api);
 REGISTER_TEST_COMMAND(cryptodev_dpaa_sec_raw_api_autotest,
 		test_cryptodev_dpaa_sec_raw_api);
-REGISTER_TEST_COMMAND(cryptodev_qat_raw_api_autotest,
+REGISTER_DRIVER_TEST(cryptodev_qat_raw_api_autotest,
 		test_cryptodev_qat_raw_api);
-REGISTER_TEST_COMMAND(cryptodev_qat_autotest, test_cryptodev_qat);
-REGISTER_TEST_COMMAND(cryptodev_aesni_mb_autotest, test_cryptodev_aesni_mb);
-REGISTER_TEST_COMMAND(cryptodev_cpu_aesni_mb_autotest,
+REGISTER_DRIVER_TEST(cryptodev_qat_autotest, test_cryptodev_qat);
+REGISTER_DRIVER_TEST(cryptodev_aesni_mb_autotest, test_cryptodev_aesni_mb);
+REGISTER_DRIVER_TEST(cryptodev_cpu_aesni_mb_autotest,
 	test_cryptodev_cpu_aesni_mb);
-REGISTER_TEST_COMMAND(cryptodev_chacha_poly_mb_autotest,
+REGISTER_DRIVER_TEST(cryptodev_chacha_poly_mb_autotest,
 	test_cryptodev_chacha_poly_mb);
-REGISTER_TEST_COMMAND(cryptodev_openssl_autotest, test_cryptodev_openssl);
-REGISTER_TEST_COMMAND(cryptodev_aesni_gcm_autotest, test_cryptodev_aesni_gcm);
-REGISTER_TEST_COMMAND(cryptodev_cpu_aesni_gcm_autotest,
+REGISTER_DRIVER_TEST(cryptodev_openssl_autotest, test_cryptodev_openssl);
+REGISTER_DRIVER_TEST(cryptodev_aesni_gcm_autotest, test_cryptodev_aesni_gcm);
+REGISTER_DRIVER_TEST(cryptodev_cpu_aesni_gcm_autotest,
 	test_cryptodev_cpu_aesni_gcm);
 REGISTER_TEST_COMMAND(cryptodev_mlx5_autotest, test_cryptodev_mlx5);
-REGISTER_TEST_COMMAND(cryptodev_null_autotest, test_cryptodev_null);
-REGISTER_TEST_COMMAND(cryptodev_sw_snow3g_autotest, test_cryptodev_sw_snow3g);
-REGISTER_TEST_COMMAND(cryptodev_sw_kasumi_autotest, test_cryptodev_sw_kasumi);
-REGISTER_TEST_COMMAND(cryptodev_sw_zuc_autotest, test_cryptodev_sw_zuc);
-REGISTER_TEST_COMMAND(cryptodev_sw_armv8_autotest, test_cryptodev_armv8);
-REGISTER_TEST_COMMAND(cryptodev_sw_mvsam_autotest, test_cryptodev_mrvl);
-REGISTER_TEST_COMMAND(cryptodev_dpaa2_sec_autotest, test_cryptodev_dpaa2_sec);
-REGISTER_TEST_COMMAND(cryptodev_dpaa_sec_autotest, test_cryptodev_dpaa_sec);
+REGISTER_DRIVER_TEST(cryptodev_null_autotest, test_cryptodev_null);
+REGISTER_DRIVER_TEST(cryptodev_sw_snow3g_autotest, test_cryptodev_sw_snow3g);
+REGISTER_DRIVER_TEST(cryptodev_sw_kasumi_autotest, test_cryptodev_sw_kasumi);
+REGISTER_DRIVER_TEST(cryptodev_sw_zuc_autotest, test_cryptodev_sw_zuc);
+REGISTER_DRIVER_TEST(cryptodev_sw_armv8_autotest, test_cryptodev_armv8);
+REGISTER_DRIVER_TEST(cryptodev_sw_mvsam_autotest, test_cryptodev_mrvl);
+REGISTER_DRIVER_TEST(cryptodev_dpaa2_sec_autotest, test_cryptodev_dpaa2_sec);
+REGISTER_DRIVER_TEST(cryptodev_dpaa_sec_autotest, test_cryptodev_dpaa_sec);
 REGISTER_TEST_COMMAND(cryptodev_ccp_autotest, test_cryptodev_ccp);
-REGISTER_TEST_COMMAND(cryptodev_uadk_autotest, test_cryptodev_uadk);
+REGISTER_DRIVER_TEST(cryptodev_uadk_autotest, test_cryptodev_uadk);
 REGISTER_TEST_COMMAND(cryptodev_virtio_autotest, test_cryptodev_virtio);
 REGISTER_TEST_COMMAND(cryptodev_octeontx_autotest, test_cryptodev_octeontx);
 REGISTER_TEST_COMMAND(cryptodev_caam_jr_autotest, test_cryptodev_caam_jr);
 REGISTER_TEST_COMMAND(cryptodev_nitrox_autotest, test_cryptodev_nitrox);
 REGISTER_TEST_COMMAND(cryptodev_bcmfs_autotest, test_cryptodev_bcmfs);
-REGISTER_TEST_COMMAND(cryptodev_cn9k_autotest, test_cryptodev_cn9k);
-REGISTER_TEST_COMMAND(cryptodev_cn10k_autotest, test_cryptodev_cn10k);
+REGISTER_DRIVER_TEST(cryptodev_cn9k_autotest, test_cryptodev_cn9k);
+REGISTER_DRIVER_TEST(cryptodev_cn10k_autotest, test_cryptodev_cn10k);
diff --git a/app/test/test_cryptodev_asym.c b/app/test/test_cryptodev_asym.c
index 0ef2642fdd..3d5a73bf89 100644
--- a/app/test/test_cryptodev_asym.c
+++ b/app/test/test_cryptodev_asym.c
@@ -2828,7 +2828,7 @@ test_cryptodev_cn10k_asym(void)
 REGISTER_TEST_COMMAND(cryptodev_openssl_asym_autotest,
 					  test_cryptodev_openssl_asym);
 
-REGISTER_TEST_COMMAND(cryptodev_qat_asym_autotest, test_cryptodev_qat_asym);
+REGISTER_DRIVER_TEST(cryptodev_qat_asym_autotest, test_cryptodev_qat_asym);
 
 REGISTER_TEST_COMMAND(cryptodev_octeontx_asym_autotest,
 					  test_cryptodev_octeontx_asym);
diff --git a/app/test/test_cycles.c b/app/test/test_cycles.c
index 66d11e6db8..08c91cef78 100644
--- a/app/test/test_cycles.c
+++ b/app/test/test_cycles.c
@@ -53,4 +53,4 @@ test_user_delay_us(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(user_delay_us, test_user_delay_us);
+REGISTER_FAST_TEST(user_delay_us, True, True, test_user_delay_us);
diff --git a/app/test/test_debug.c b/app/test/test_debug.c
index 2704f5b927..acd9542257 100644
--- a/app/test/test_debug.c
+++ b/app/test/test_debug.c
@@ -140,4 +140,4 @@ test_debug(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(debug_autotest, test_debug);
+REGISTER_FAST_TEST(debug_autotest, True, True, test_debug);
diff --git a/app/test/test_devargs.c b/app/test/test_devargs.c
index 0a4c34a1ad..59ec04957e 100644
--- a/app/test/test_devargs.c
+++ b/app/test/test_devargs.c
@@ -213,4 +213,4 @@ test_devargs(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(devargs_autotest, test_devargs);
+REGISTER_FAST_TEST(devargs_autotest, True, True, test_devargs);
diff --git a/app/test/test_distributor.c b/app/test/test_distributor.c
index 3efa4af104..ee263a7fee 100644
--- a/app/test/test_distributor.c
+++ b/app/test/test_distributor.c
@@ -952,4 +952,4 @@ test_distributor(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(distributor_autotest, test_distributor);
+REGISTER_FAST_TEST(distributor_autotest, False, True, test_distributor);
diff --git a/app/test/test_distributor_perf.c b/app/test/test_distributor_perf.c
index ee4321486d..ca868451d7 100644
--- a/app/test/test_distributor_perf.c
+++ b/app/test/test_distributor_perf.c
@@ -277,4 +277,4 @@ test_distributor_perf(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(distributor_perf_autotest, test_distributor_perf);
+REGISTER_PERF_TEST(distributor_perf_autotest, test_distributor_perf);
diff --git a/app/test/test_dmadev.c b/app/test/test_dmadev.c
index 0736ff2a18..6ef875e545 100644
--- a/app/test/test_dmadev.c
+++ b/app/test/test_dmadev.c
@@ -941,4 +941,4 @@ test_dma(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(dmadev_autotest, test_dma);
+REGISTER_DRIVER_TEST(dmadev_autotest, test_dma);
diff --git a/app/test/test_eal_flags.c b/app/test/test_eal_flags.c
index 148e9098e8..3b7af7405d 100644
--- a/app/test/test_eal_flags.c
+++ b/app/test/test_eal_flags.c
@@ -1644,15 +1644,15 @@ test_memory_flags(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(eal_flags_c_opt_autotest, test_missing_c_flag);
-REGISTER_TEST_COMMAND(eal_flags_main_opt_autotest, test_main_lcore_flag);
-REGISTER_TEST_COMMAND(eal_flags_n_opt_autotest, test_invalid_n_flag);
-REGISTER_TEST_COMMAND(eal_flags_hpet_autotest, test_no_hpet_flag);
-REGISTER_TEST_COMMAND(eal_flags_no_huge_autotest, test_no_huge_flag);
-REGISTER_TEST_COMMAND(eal_flags_a_opt_autotest, test_allow_flag);
-REGISTER_TEST_COMMAND(eal_flags_b_opt_autotest, test_invalid_b_flag);
-REGISTER_TEST_COMMAND(eal_flags_vdev_opt_autotest, test_invalid_vdev_flag);
-REGISTER_TEST_COMMAND(eal_flags_r_opt_autotest, test_invalid_r_flag);
-REGISTER_TEST_COMMAND(eal_flags_mem_autotest, test_memory_flags);
-REGISTER_TEST_COMMAND(eal_flags_file_prefix_autotest, test_file_prefix);
-REGISTER_TEST_COMMAND(eal_flags_misc_autotest, test_misc_flags);
+REGISTER_FAST_TEST(eal_flags_c_opt_autotest, False, False, test_missing_c_flag);
+REGISTER_FAST_TEST(eal_flags_main_opt_autotest, False, False, test_main_lcore_flag);
+REGISTER_FAST_TEST(eal_flags_n_opt_autotest, False, False, test_invalid_n_flag);
+REGISTER_FAST_TEST(eal_flags_hpet_autotest, False, False, test_no_hpet_flag);
+REGISTER_FAST_TEST(eal_flags_no_huge_autotest, False, False, test_no_huge_flag);
+REGISTER_FAST_TEST(eal_flags_a_opt_autotest, False, False, test_allow_flag);
+REGISTER_FAST_TEST(eal_flags_b_opt_autotest, False, False, test_invalid_b_flag);
+REGISTER_FAST_TEST(eal_flags_vdev_opt_autotest, False, False, test_invalid_vdev_flag);
+REGISTER_FAST_TEST(eal_flags_r_opt_autotest, False, False, test_invalid_r_flag);
+REGISTER_FAST_TEST(eal_flags_mem_autotest, False, False, test_memory_flags);
+REGISTER_FAST_TEST(eal_flags_file_prefix_autotest, False, False, test_file_prefix);
+REGISTER_FAST_TEST(eal_flags_misc_autotest, False, False, test_misc_flags);
diff --git a/app/test/test_eal_fs.c b/app/test/test_eal_fs.c
index b3686edcb4..a4bdc1236d 100644
--- a/app/test/test_eal_fs.c
+++ b/app/test/test_eal_fs.c
@@ -185,4 +185,4 @@ test_eal_fs(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(eal_fs_autotest, test_eal_fs);
+REGISTER_FAST_TEST(eal_fs_autotest, True, True, test_eal_fs);
diff --git a/app/test/test_efd.c b/app/test/test_efd.c
index fa29e8f97a..1c0986b9bc 100644
--- a/app/test/test_efd.c
+++ b/app/test/test_efd.c
@@ -473,4 +473,4 @@ test_efd(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(efd_autotest, test_efd);
+REGISTER_PERF_TEST(efd_autotest, test_efd);
diff --git a/app/test/test_efd_perf.c b/app/test/test_efd_perf.c
index 4d04ed93e3..b212e96767 100644
--- a/app/test/test_efd_perf.c
+++ b/app/test/test_efd_perf.c
@@ -393,4 +393,4 @@ test_efd_perf(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(efd_perf_autotest, test_efd_perf);
+REGISTER_PERF_TEST(efd_perf_autotest, test_efd_perf);
diff --git a/app/test/test_errno.c b/app/test/test_errno.c
index 0db4fbc8b3..1082299665 100644
--- a/app/test/test_errno.c
+++ b/app/test/test_errno.c
@@ -94,4 +94,4 @@ test_errno(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(errno_autotest, test_errno);
+REGISTER_FAST_TEST(errno_autotest, True, True, test_errno);
diff --git a/app/test/test_ethdev_link.c b/app/test/test_ethdev_link.c
index ab52385a12..d73e18065d 100644
--- a/app/test/test_ethdev_link.c
+++ b/app/test/test_ethdev_link.c
@@ -167,4 +167,4 @@ test_link_status(void)
 	return unit_test_suite_runner(&link_status_testsuite);
 }
 
-REGISTER_TEST_COMMAND(ethdev_link_status, test_link_status);
+REGISTER_FAST_TEST(ethdev_link_status, True, True, test_link_status);
diff --git a/app/test/test_event_ring.c b/app/test/test_event_ring.c
index bc4a6e73f5..d5c8499cfd 100644
--- a/app/test/test_event_ring.c
+++ b/app/test/test_event_ring.c
@@ -256,4 +256,4 @@ test_event_ring(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(event_ring_autotest, test_event_ring);
+REGISTER_FAST_TEST(event_ring_autotest, True, True, test_event_ring);
diff --git a/app/test/test_eventdev.c b/app/test/test_eventdev.c
index 336529038e..e65d893781 100644
--- a/app/test/test_eventdev.c
+++ b/app/test/test_eventdev.c
@@ -1256,7 +1256,7 @@ test_eventdev_selftest_cn10k(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(eventdev_common_autotest, test_eventdev_common);
+REGISTER_FAST_TEST(eventdev_common_autotest, True, True, test_eventdev_common);
 
 #ifndef RTE_EXEC_ENV_WINDOWS
 REGISTER_TEST_COMMAND(eventdev_selftest_sw, test_eventdev_selftest_sw);
diff --git a/app/test/test_fbarray.c b/app/test/test_fbarray.c
index a691bf4458..03428a45fe 100644
--- a/app/test/test_fbarray.c
+++ b/app/test/test_fbarray.c
@@ -733,4 +733,4 @@ test_fbarray(void)
 	return unit_test_suite_runner(&fbarray_test_suite);
 }
 
-REGISTER_TEST_COMMAND(fbarray_autotest, test_fbarray);
+REGISTER_FAST_TEST(fbarray_autotest, True, True, test_fbarray);
diff --git a/app/test/test_fib.c b/app/test/test_fib.c
index eb69d6e2fd..2a8404db61 100644
--- a/app/test/test_fib.c
+++ b/app/test/test_fib.c
@@ -415,5 +415,5 @@ test_slow_fib(void)
 	return unit_test_suite_runner(&fib_slow_tests);
 }
 
-REGISTER_TEST_COMMAND(fib_autotest, test_fib);
-REGISTER_TEST_COMMAND(fib_slow_autotest, test_slow_fib);
+REGISTER_FAST_TEST(fib_autotest, True, True, test_fib);
+REGISTER_PERF_TEST(fib_slow_autotest, test_slow_fib);
diff --git a/app/test/test_fib6.c b/app/test/test_fib6.c
index 15ad09178a..9b1c0ac8c7 100644
--- a/app/test/test_fib6.c
+++ b/app/test/test_fib6.c
@@ -424,5 +424,5 @@ test_slow_fib6(void)
 	return unit_test_suite_runner(&fib6_slow_tests);
 }
 
-REGISTER_TEST_COMMAND(fib6_autotest, test_fib6);
-REGISTER_TEST_COMMAND(fib6_slow_autotest, test_slow_fib6);
+REGISTER_FAST_TEST(fib6_autotest, True, True, test_fib6);
+REGISTER_PERF_TEST(fib6_slow_autotest, test_slow_fib6);
diff --git a/app/test/test_fib6_perf.c b/app/test/test_fib6_perf.c
index add20c2331..a7abc46af9 100644
--- a/app/test/test_fib6_perf.c
+++ b/app/test/test_fib6_perf.c
@@ -156,4 +156,4 @@ test_fib6_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(fib6_perf_autotest, test_fib6_perf);
+REGISTER_PERF_TEST(fib6_perf_autotest, test_fib6_perf);
diff --git a/app/test/test_fib_perf.c b/app/test/test_fib_perf.c
index b56293e64f..a9119c1bb0 100644
--- a/app/test/test_fib_perf.c
+++ b/app/test/test_fib_perf.c
@@ -409,4 +409,4 @@ test_fib_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(fib_perf_autotest, test_fib_perf);
+REGISTER_PERF_TEST(fib_perf_autotest, test_fib_perf);
diff --git a/app/test/test_func_reentrancy.c b/app/test/test_func_reentrancy.c
index ae9de6f93d..63b00009b5 100644
--- a/app/test/test_func_reentrancy.c
+++ b/app/test/test_func_reentrancy.c
@@ -507,4 +507,4 @@ test_func_reentrancy(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(func_reentrancy_autotest, test_func_reentrancy);
+REGISTER_FAST_TEST(func_reentrancy_autotest, False, True, test_func_reentrancy);
diff --git a/app/test/test_hash.c b/app/test/test_hash.c
index 3e45afaa67..e160b27b4c 100644
--- a/app/test/test_hash.c
+++ b/app/test/test_hash.c
@@ -2264,4 +2264,4 @@ test_hash(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(hash_autotest, test_hash);
+REGISTER_FAST_TEST(hash_autotest, True, True, test_hash);
diff --git a/app/test/test_hash_functions.c b/app/test/test_hash_functions.c
index 76d51b6e71..70820d1f19 100644
--- a/app/test/test_hash_functions.c
+++ b/app/test/test_hash_functions.c
@@ -290,4 +290,4 @@ test_hash_functions(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(hash_functions_autotest, test_hash_functions);
+REGISTER_PERF_TEST(hash_functions_autotest, test_hash_functions);
diff --git a/app/test/test_hash_multiwriter.c b/app/test/test_hash_multiwriter.c
index 0c5a8ca186..dd5ca677b9 100644
--- a/app/test/test_hash_multiwriter.c
+++ b/app/test/test_hash_multiwriter.c
@@ -287,4 +287,4 @@ test_hash_multiwriter_main(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(hash_multiwriter_autotest, test_hash_multiwriter_main);
+REGISTER_PERF_TEST(hash_multiwriter_autotest, test_hash_multiwriter_main);
diff --git a/app/test/test_hash_perf.c b/app/test/test_hash_perf.c
index 14a1283aba..d66b96e5ce 100644
--- a/app/test/test_hash_perf.c
+++ b/app/test/test_hash_perf.c
@@ -757,4 +757,4 @@ test_hash_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(hash_perf_autotest, test_hash_perf);
+REGISTER_PERF_TEST(hash_perf_autotest, test_hash_perf);
diff --git a/app/test/test_hash_readwrite.c b/app/test/test_hash_readwrite.c
index 6373e62d33..9c33b49878 100644
--- a/app/test/test_hash_readwrite.c
+++ b/app/test/test_hash_readwrite.c
@@ -760,5 +760,5 @@ test_hash_rw_func_main(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(hash_readwrite_func_autotest, test_hash_rw_func_main);
-REGISTER_TEST_COMMAND(hash_readwrite_perf_autotest, test_hash_rw_perf_main);
+REGISTER_FAST_TEST(hash_readwrite_func_autotest, False, True, test_hash_rw_func_main);
+REGISTER_PERF_TEST(hash_readwrite_perf_autotest, test_hash_rw_perf_main);
diff --git a/app/test/test_hash_readwrite_lf_perf.c b/app/test/test_hash_readwrite_lf_perf.c
index cf86046a2f..5d18850e19 100644
--- a/app/test/test_hash_readwrite_lf_perf.c
+++ b/app/test/test_hash_readwrite_lf_perf.c
@@ -1579,5 +1579,5 @@ test_hash_readwrite_lf_perf_main(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(hash_readwrite_lf_perf_autotest,
+REGISTER_PERF_TEST(hash_readwrite_lf_perf_autotest,
 	test_hash_readwrite_lf_perf_main);
diff --git a/app/test/test_interrupts.c b/app/test/test_interrupts.c
index b59ab42699..cf6b5a6a8e 100644
--- a/app/test/test_interrupts.c
+++ b/app/test/test_interrupts.c
@@ -594,4 +594,4 @@ test_interrupt(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(interrupt_autotest, test_interrupt);
+REGISTER_FAST_TEST(interrupt_autotest, True, True, test_interrupt);
diff --git a/app/test/test_ipfrag.c b/app/test/test_ipfrag.c
index 402ce361c1..264825634d 100644
--- a/app/test/test_ipfrag.c
+++ b/app/test/test_ipfrag.c
@@ -510,4 +510,4 @@ test_ipfrag(void)
 }
 
 
-REGISTER_TEST_COMMAND(ipfrag_autotest, test_ipfrag);
+REGISTER_FAST_TEST(ipfrag_autotest, False, True, test_ipfrag);
diff --git a/app/test/test_ipsec.c b/app/test/test_ipsec.c
index c2a52ec305..629580bc21 100644
--- a/app/test/test_ipsec.c
+++ b/app/test/test_ipsec.c
@@ -2532,4 +2532,4 @@ test_ipsec(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(ipsec_autotest, test_ipsec);
+REGISTER_FAST_TEST(ipsec_autotest, True, True, test_ipsec);
diff --git a/app/test/test_ipsec_perf.c b/app/test/test_ipsec_perf.c
index b221b7fc32..a32a2086e9 100644
--- a/app/test/test_ipsec_perf.c
+++ b/app/test/test_ipsec_perf.c
@@ -631,4 +631,4 @@ test_libipsec_perf(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(ipsec_perf_autotest, test_libipsec_perf);
+REGISTER_PERF_TEST(ipsec_perf_autotest, test_libipsec_perf);
diff --git a/app/test/test_kvargs.c b/app/test/test_kvargs.c
index b7b97a0dd9..d0f05a55c7 100644
--- a/app/test/test_kvargs.c
+++ b/app/test/test_kvargs.c
@@ -292,4 +292,4 @@ test_kvargs(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(kvargs_autotest, test_kvargs);
+REGISTER_FAST_TEST(kvargs_autotest, True, True, test_kvargs);
diff --git a/app/test/test_lcores.c b/app/test/test_lcores.c
index 2c945b0136..b86629c0db 100644
--- a/app/test/test_lcores.c
+++ b/app/test/test_lcores.c
@@ -412,4 +412,4 @@ test_lcores(void)
 	return TEST_SUCCESS;
 }
 
-REGISTER_TEST_COMMAND(lcores_autotest, test_lcores);
+REGISTER_FAST_TEST(lcores_autotest, True, True, test_lcores);
diff --git a/app/test/test_logs.c b/app/test/test_logs.c
index 8da8824bee..38052910e9 100644
--- a/app/test/test_logs.c
+++ b/app/test/test_logs.c
@@ -158,4 +158,4 @@ test_logs(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(logs_autotest, test_logs);
+REGISTER_FAST_TEST(logs_autotest, True, True, test_logs);
diff --git a/app/test/test_lpm.c b/app/test/test_lpm.c
index 37b460af3a..85a75ccec0 100644
--- a/app/test/test_lpm.c
+++ b/app/test/test_lpm.c
@@ -1584,4 +1584,4 @@ test_lpm(void)
 	return global_status;
 }
 
-REGISTER_TEST_COMMAND(lpm_autotest, test_lpm);
+REGISTER_FAST_TEST(lpm_autotest, True, True, test_lpm);
diff --git a/app/test/test_lpm6.c b/app/test/test_lpm6.c
index b6b6f8615e..4a9eddf276 100644
--- a/app/test/test_lpm6.c
+++ b/app/test/test_lpm6.c
@@ -1793,4 +1793,4 @@ test_lpm6(void)
 	return global_status;
 }
 
-REGISTER_TEST_COMMAND(lpm6_autotest, test_lpm6);
+REGISTER_FAST_TEST(lpm6_autotest, True, True, test_lpm6);
diff --git a/app/test/test_lpm6_perf.c b/app/test/test_lpm6_perf.c
index 5b684686a6..8a49f74c84 100644
--- a/app/test/test_lpm6_perf.c
+++ b/app/test/test_lpm6_perf.c
@@ -161,4 +161,4 @@ test_lpm6_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(lpm6_perf_autotest, test_lpm6_perf);
+REGISTER_PERF_TEST(lpm6_perf_autotest, test_lpm6_perf);
diff --git a/app/test/test_lpm_perf.c b/app/test/test_lpm_perf.c
index e72437ba38..15ff396dd0 100644
--- a/app/test/test_lpm_perf.c
+++ b/app/test/test_lpm_perf.c
@@ -760,4 +760,4 @@ test_lpm_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(lpm_perf_autotest, test_lpm_perf);
+REGISTER_PERF_TEST(lpm_perf_autotest, test_lpm_perf);
diff --git a/app/test/test_malloc.c b/app/test/test_malloc.c
index ff081dd931..6057623473 100644
--- a/app/test/test_malloc.c
+++ b/app/test/test_malloc.c
@@ -1091,4 +1091,4 @@ test_malloc(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(malloc_autotest, test_malloc);
+REGISTER_FAST_TEST(malloc_autotest, False, True, test_malloc);
diff --git a/app/test/test_malloc_perf.c b/app/test/test_malloc_perf.c
index 9bd1662981..a99bfd8531 100644
--- a/app/test/test_malloc_perf.c
+++ b/app/test/test_malloc_perf.c
@@ -171,4 +171,4 @@ test_malloc_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(malloc_perf_autotest, test_malloc_perf);
+REGISTER_PERF_TEST(malloc_perf_autotest, test_malloc_perf);
diff --git a/app/test/test_mbuf.c b/app/test/test_mbuf.c
index efac01806b..2eeed268af 100644
--- a/app/test/test_mbuf.c
+++ b/app/test/test_mbuf.c
@@ -2955,4 +2955,4 @@ test_mbuf(void)
 }
 #undef GOTO_FAIL
 
-REGISTER_TEST_COMMAND(mbuf_autotest, test_mbuf);
+REGISTER_FAST_TEST(mbuf_autotest, False, True, test_mbuf);
diff --git a/app/test/test_mcslock.c b/app/test/test_mcslock.c
index 52e45e7e2a..a3753a4397 100644
--- a/app/test/test_mcslock.c
+++ b/app/test/test_mcslock.c
@@ -241,4 +241,4 @@ test_mcslock(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(mcslock_autotest, test_mcslock);
+REGISTER_FAST_TEST(mcslock_autotest, False, True, test_mcslock);
diff --git a/app/test/test_member.c b/app/test/test_member.c
index 4a93f8bff4..e17a8f3190 100644
--- a/app/test/test_member.c
+++ b/app/test/test_member.c
@@ -996,4 +996,4 @@ test_member(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(member_autotest, test_member);
+REGISTER_FAST_TEST(member_autotest, True, True, test_member);
diff --git a/app/test/test_member_perf.c b/app/test/test_member_perf.c
index 2f79888fbd..db6b8a18ef 100644
--- a/app/test/test_member_perf.c
+++ b/app/test/test_member_perf.c
@@ -780,4 +780,4 @@ test_member_perf(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(member_perf_autotest, test_member_perf);
+REGISTER_PERF_TEST(member_perf_autotest, test_member_perf);
diff --git a/app/test/test_memcpy.c b/app/test/test_memcpy.c
index 1ab86f4967..c81efb85c2 100644
--- a/app/test/test_memcpy.c
+++ b/app/test/test_memcpy.c
@@ -129,4 +129,4 @@ test_memcpy(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(memcpy_autotest, test_memcpy);
+REGISTER_FAST_TEST(memcpy_autotest, True, True, test_memcpy);
diff --git a/app/test/test_memcpy_perf.c b/app/test/test_memcpy_perf.c
index 3727c160e6..5c05a84619 100644
--- a/app/test/test_memcpy_perf.c
+++ b/app/test/test_memcpy_perf.c
@@ -348,4 +348,4 @@ test_memcpy_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(memcpy_perf_autotest, test_memcpy_perf);
+REGISTER_PERF_TEST(memcpy_perf_autotest, test_memcpy_perf);
diff --git a/app/test/test_memory.c b/app/test/test_memory.c
index 440e5ef838..13ae9569c2 100644
--- a/app/test/test_memory.c
+++ b/app/test/test_memory.c
@@ -110,4 +110,4 @@ test_memory(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(memory_autotest, test_memory);
+REGISTER_FAST_TEST(memory_autotest, False, True, test_memory);
diff --git a/app/test/test_mempool.c b/app/test/test_mempool.c
index 8e493eda47..03047e5b14 100644
--- a/app/test/test_mempool.c
+++ b/app/test/test_mempool.c
@@ -1043,4 +1043,4 @@ test_mempool(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(mempool_autotest, test_mempool);
+REGISTER_FAST_TEST(mempool_autotest, False, True, test_mempool);
diff --git a/app/test/test_mempool_perf.c b/app/test/test_mempool_perf.c
index ce7c6241ab..96de347f04 100644
--- a/app/test/test_mempool_perf.c
+++ b/app/test/test_mempool_perf.c
@@ -437,4 +437,4 @@ test_mempool_perf(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(mempool_perf_autotest, test_mempool_perf);
+REGISTER_PERF_TEST(mempool_perf_autotest, test_mempool_perf);
diff --git a/app/test/test_memzone.c b/app/test/test_memzone.c
index f10f4fd9cd..85b5839034 100644
--- a/app/test/test_memzone.c
+++ b/app/test/test_memzone.c
@@ -1163,4 +1163,4 @@ test_memzone(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(memzone_autotest, test_memzone);
+REGISTER_FAST_TEST(memzone_autotest, False, True, test_memzone);
diff --git a/app/test/test_meter.c b/app/test/test_meter.c
index 15d5a4839b..3016681bbc 100644
--- a/app/test/test_meter.c
+++ b/app/test/test_meter.c
@@ -713,4 +713,4 @@ test_meter(void)
 
 }
 
-REGISTER_TEST_COMMAND(meter_autotest, test_meter);
+REGISTER_FAST_TEST(meter_autotest, True, True, test_meter);
diff --git a/app/test/test_mp_secondary.c b/app/test/test_mp_secondary.c
index ad47d578f2..502b0235a7 100644
--- a/app/test/test_mp_secondary.c
+++ b/app/test/test_mp_secondary.c
@@ -223,4 +223,4 @@ test_mp_secondary(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(multiprocess_autotest, test_mp_secondary);
+REGISTER_FAST_TEST(multiprocess_autotest, False, False, test_mp_secondary);
diff --git a/app/test/test_per_lcore.c b/app/test/test_per_lcore.c
index 129578d1a3..50011366a7 100644
--- a/app/test/test_per_lcore.c
+++ b/app/test/test_per_lcore.c
@@ -105,4 +105,4 @@ test_per_lcore(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(per_lcore_autotest, test_per_lcore);
+REGISTER_FAST_TEST(per_lcore_autotest, True, True, test_per_lcore);
diff --git a/app/test/test_pflock.c b/app/test/test_pflock.c
index 38da6bce27..07e4a68c2c 100644
--- a/app/test/test_pflock.c
+++ b/app/test/test_pflock.c
@@ -193,4 +193,4 @@ test_pflock(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(pflock_autotest, test_pflock);
+REGISTER_FAST_TEST(pflock_autotest, True, True, test_pflock);
diff --git a/app/test/test_pie.c b/app/test/test_pie.c
index a3c0f97c9d..5674602b69 100644
--- a/app/test/test_pie.c
+++ b/app/test/test_pie.c
@@ -1087,6 +1087,6 @@ test_pie_all(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(pie_autotest, test_pie);
-REGISTER_TEST_COMMAND(pie_perf, test_pie_perf);
-REGISTER_TEST_COMMAND(pie_all, test_pie_all);
+REGISTER_FAST_TEST(pie_autotest, True, True, test_pie);
+REGISTER_PERF_TEST(pie_perf, test_pie_perf);
+REGISTER_PERF_TEST(pie_all, test_pie_all);
diff --git a/app/test/test_pmd_perf.c b/app/test/test_pmd_perf.c
index 3ef590cb51..f6d97f21c9 100644
--- a/app/test/test_pmd_perf.c
+++ b/app/test/test_pmd_perf.c
@@ -899,4 +899,4 @@ test_set_rxtx_sc(cmdline_fixed_string_t type)
 	return -1;
 }
 
-REGISTER_TEST_COMMAND(pmd_perf_autotest, test_pmd_perf);
+REGISTER_PERF_TEST(pmd_perf_autotest, test_pmd_perf);
diff --git a/app/test/test_power.c b/app/test/test_power.c
index b7b5561348..d4f622ff2f 100644
--- a/app/test/test_power.c
+++ b/app/test/test_power.c
@@ -172,4 +172,4 @@ test_power(void)
 }
 #endif
 
-REGISTER_TEST_COMMAND(power_autotest, test_power);
+REGISTER_FAST_TEST(power_autotest, True, True, test_power);
diff --git a/app/test/test_power_cpufreq.c b/app/test/test_power_cpufreq.c
index 4d013cd7bb..f56053ac10 100644
--- a/app/test/test_power_cpufreq.c
+++ b/app/test/test_power_cpufreq.c
@@ -703,5 +703,5 @@ test_power_caps(void)
 
 #endif
 
-REGISTER_TEST_COMMAND(power_cpufreq_autotest, test_power_cpufreq);
+REGISTER_FAST_TEST(power_cpufreq_autotest, False, True, test_power_cpufreq);
 REGISTER_TEST_COMMAND(power_caps_autotest, test_power_caps);
diff --git a/app/test/test_power_intel_uncore.c b/app/test/test_power_intel_uncore.c
index 31163af84e..c8a8621e44 100644
--- a/app/test/test_power_intel_uncore.c
+++ b/app/test/test_power_intel_uncore.c
@@ -298,4 +298,4 @@ test_power_intel_uncore(void)
 }
 #endif
 
-REGISTER_TEST_COMMAND(power_intel_uncore_autotest, test_power_intel_uncore);
+REGISTER_FAST_TEST(power_intel_uncore_autotest, True, True, test_power_intel_uncore);
diff --git a/app/test/test_power_kvm_vm.c b/app/test/test_power_kvm_vm.c
index cc66b7a8a0..4919df8b62 100644
--- a/app/test/test_power_kvm_vm.c
+++ b/app/test/test_power_kvm_vm.c
@@ -299,4 +299,4 @@ test_power_kvm_vm(void)
 }
 #endif
 
-REGISTER_TEST_COMMAND(power_kvm_vm_autotest, test_power_kvm_vm);
+REGISTER_FAST_TEST(power_kvm_vm_autotest, False, True, test_power_kvm_vm);
diff --git a/app/test/test_prefetch.c b/app/test/test_prefetch.c
index 7b4a8e4144..a72ae80557 100644
--- a/app/test/test_prefetch.c
+++ b/app/test/test_prefetch.c
@@ -35,4 +35,4 @@ test_prefetch(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(prefetch_autotest, test_prefetch);
+REGISTER_FAST_TEST(prefetch_autotest, True, True, test_prefetch);
diff --git a/app/test/test_rand_perf.c b/app/test/test_rand_perf.c
index 26fb1d9a58..30204e12c0 100644
--- a/app/test/test_rand_perf.c
+++ b/app/test/test_rand_perf.c
@@ -96,4 +96,4 @@ test_rand_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(rand_perf_autotest, test_rand_perf);
+REGISTER_PERF_TEST(rand_perf_autotest, test_rand_perf);
diff --git a/app/test/test_rcu_qsbr.c b/app/test/test_rcu_qsbr.c
index 70404e89e6..16c24f58e7 100644
--- a/app/test/test_rcu_qsbr.c
+++ b/app/test/test_rcu_qsbr.c
@@ -1418,4 +1418,4 @@ test_rcu_qsbr_main(void)
 	return -1;
 }
 
-REGISTER_TEST_COMMAND(rcu_qsbr_autotest, test_rcu_qsbr_main);
+REGISTER_FAST_TEST(rcu_qsbr_autotest, True, True, test_rcu_qsbr_main);
diff --git a/app/test/test_rcu_qsbr_perf.c b/app/test/test_rcu_qsbr_perf.c
index b15e5cef88..ce88a7333c 100644
--- a/app/test/test_rcu_qsbr_perf.c
+++ b/app/test/test_rcu_qsbr_perf.c
@@ -690,4 +690,4 @@ test_rcu_qsbr_main(void)
 	return -1;
 }
 
-REGISTER_TEST_COMMAND(rcu_qsbr_perf_autotest, test_rcu_qsbr_main);
+REGISTER_PERF_TEST(rcu_qsbr_perf_autotest, test_rcu_qsbr_main);
diff --git a/app/test/test_reassembly_perf.c b/app/test/test_reassembly_perf.c
index c11b65291f..4b4929d777 100644
--- a/app/test/test_reassembly_perf.c
+++ b/app/test/test_reassembly_perf.c
@@ -1000,4 +1000,4 @@ test_reassembly_perf(void)
 	return TEST_SUCCESS;
 }
 
-REGISTER_TEST_COMMAND(reassembly_perf_autotest, test_reassembly_perf);
+REGISTER_PERF_TEST(reassembly_perf_autotest, test_reassembly_perf);
diff --git a/app/test/test_reciprocal_division.c b/app/test/test_reciprocal_division.c
index 8ea9b1d24d..fb52b2d5a1 100644
--- a/app/test/test_reciprocal_division.c
+++ b/app/test/test_reciprocal_division.c
@@ -164,4 +164,4 @@ test_reciprocal(void)
 	return result;
 }
 
-REGISTER_TEST_COMMAND(reciprocal_division, test_reciprocal);
+REGISTER_PERF_TEST(reciprocal_division, test_reciprocal);
diff --git a/app/test/test_reciprocal_division_perf.c b/app/test/test_reciprocal_division_perf.c
index 4f625873e5..cf96d46a22 100644
--- a/app/test/test_reciprocal_division_perf.c
+++ b/app/test/test_reciprocal_division_perf.c
@@ -205,4 +205,4 @@ test_reciprocal_division_perf(void)
 	return result;
 }
 
-REGISTER_TEST_COMMAND(reciprocal_division_perf, test_reciprocal_division_perf);
+REGISTER_PERF_TEST(reciprocal_division_perf, test_reciprocal_division_perf);
diff --git a/app/test/test_red.c b/app/test/test_red.c
index 84c292f8d8..aa7538d51a 100644
--- a/app/test/test_red.c
+++ b/app/test/test_red.c
@@ -1878,5 +1878,5 @@ test_red_all(void)
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
 REGISTER_TEST_COMMAND(red_autotest, test_red);
-REGISTER_TEST_COMMAND(red_perf, test_red_perf);
-REGISTER_TEST_COMMAND(red_all, test_red_all);
+REGISTER_PERF_TEST(red_perf, test_red_perf);
+REGISTER_PERF_TEST(red_all, test_red_all);
diff --git a/app/test/test_reorder.c b/app/test/test_reorder.c
index c188f6ce67..e66b4b05d4 100644
--- a/app/test/test_reorder.c
+++ b/app/test/test_reorder.c
@@ -548,4 +548,4 @@ test_reorder(void)
 }
 
 
-REGISTER_TEST_COMMAND(reorder_autotest, test_reorder);
+REGISTER_FAST_TEST(reorder_autotest, True, True, test_reorder);
diff --git a/app/test/test_rib.c b/app/test/test_rib.c
index 65b6856410..a8d60acc3a 100644
--- a/app/test/test_rib.c
+++ b/app/test/test_rib.c
@@ -363,5 +363,5 @@ test_slow_rib(void)
 	return unit_test_suite_runner(&rib_slow_tests);
 }
 
-REGISTER_TEST_COMMAND(rib_autotest, test_rib);
-REGISTER_TEST_COMMAND(rib_slow_autotest, test_slow_rib);
+REGISTER_FAST_TEST(rib_autotest, True, True, test_rib);
+REGISTER_PERF_TEST(rib_slow_autotest, test_slow_rib);
diff --git a/app/test/test_rib6.c b/app/test/test_rib6.c
index 336b779d2e..9bbd0f406d 100644
--- a/app/test/test_rib6.c
+++ b/app/test/test_rib6.c
@@ -367,5 +367,5 @@ test_slow_rib6(void)
 	return unit_test_suite_runner(&rib6_slow_tests);
 }
 
-REGISTER_TEST_COMMAND(rib6_autotest, test_rib6);
-REGISTER_TEST_COMMAND(rib6_slow_autotest, test_slow_rib6);
+REGISTER_FAST_TEST(rib6_autotest, True, True, test_rib6);
+REGISTER_PERF_TEST(rib6_slow_autotest, test_slow_rib6);
diff --git a/app/test/test_ring.c b/app/test/test_ring.c
index bde33ab4a1..d2accbabfa 100644
--- a/app/test/test_ring.c
+++ b/app/test/test_ring.c
@@ -1241,4 +1241,4 @@ test_ring(void)
 	return -1;
 }
 
-REGISTER_TEST_COMMAND(ring_autotest, test_ring);
+REGISTER_FAST_TEST(ring_autotest, True, True, test_ring);
diff --git a/app/test/test_ring_perf.c b/app/test/test_ring_perf.c
index 3972fd9db3..d7c5a4c30b 100644
--- a/app/test/test_ring_perf.c
+++ b/app/test/test_ring_perf.c
@@ -579,4 +579,4 @@ test_ring_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(ring_perf_autotest, test_ring_perf);
+REGISTER_PERF_TEST(ring_perf_autotest, test_ring_perf);
diff --git a/app/test/test_rwlock.c b/app/test/test_rwlock.c
index 4ae0bf8deb..e8086888fe 100644
--- a/app/test/test_rwlock.c
+++ b/app/test/test_rwlock.c
@@ -506,7 +506,7 @@ try_rwlock_test_rde_wro(void)
 	return process_try_lcore_stats();
 }
 
-REGISTER_TEST_COMMAND(rwlock_test1_autotest, rwlock_test1);
-REGISTER_TEST_COMMAND(rwlock_rda_autotest, try_rwlock_test_rda);
-REGISTER_TEST_COMMAND(rwlock_rds_wrm_autotest, try_rwlock_test_rds_wrm);
-REGISTER_TEST_COMMAND(rwlock_rde_wro_autotest, try_rwlock_test_rde_wro);
+REGISTER_FAST_TEST(rwlock_test1_autotest, True, True, rwlock_test1);
+REGISTER_FAST_TEST(rwlock_rda_autotest, True, True, try_rwlock_test_rda);
+REGISTER_FAST_TEST(rwlock_rds_wrm_autotest, True, True, try_rwlock_test_rds_wrm);
+REGISTER_FAST_TEST(rwlock_rde_wro_autotest, True, True, try_rwlock_test_rde_wro);
diff --git a/app/test/test_sched.c b/app/test/test_sched.c
index ddec572447..385ca45136 100644
--- a/app/test/test_sched.c
+++ b/app/test/test_sched.c
@@ -215,4 +215,4 @@ test_sched(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(sched_autotest, test_sched);
+REGISTER_FAST_TEST(sched_autotest, True, True, test_sched);
diff --git a/app/test/test_security.c b/app/test/test_security.c
index 4783cd0663..a3b962750a 100644
--- a/app/test/test_security.c
+++ b/app/test/test_security.c
@@ -2314,4 +2314,4 @@ test_security(void)
 	return unit_test_suite_runner(&security_testsuite);
 }
 
-REGISTER_TEST_COMMAND(security_autotest, test_security);
+REGISTER_FAST_TEST(security_autotest, False, True, test_security);
diff --git a/app/test/test_seqlock.c b/app/test/test_seqlock.c
index d26d2c010e..325af38070 100644
--- a/app/test/test_seqlock.c
+++ b/app/test/test_seqlock.c
@@ -187,4 +187,4 @@ test_seqlock(void)
 	return rc;
 }
 
-REGISTER_TEST_COMMAND(seqlock_autotest, test_seqlock);
+REGISTER_FAST_TEST(seqlock_autotest, True, True, test_seqlock);
diff --git a/app/test/test_service_cores.c b/app/test/test_service_cores.c
index 422d2a83e8..6c000417d2 100644
--- a/app/test/test_service_cores.c
+++ b/app/test/test_service_cores.c
@@ -1040,7 +1040,7 @@ test_service_common(void)
 	return unit_test_suite_runner(&service_tests);
 }
 
-REGISTER_TEST_COMMAND(service_autotest, test_service_common);
+REGISTER_FAST_TEST(service_autotest, True, True, test_service_common);
 
 static struct unit_test_suite service_perf_tests  = {
 	.suite_name = "service core performance test suite",
@@ -1062,4 +1062,4 @@ test_service_perf(void)
 	return unit_test_suite_runner(&service_perf_tests);
 }
 
-REGISTER_TEST_COMMAND(service_perf_autotest, test_service_perf);
+REGISTER_PERF_TEST(service_perf_autotest, test_service_perf);
diff --git a/app/test/test_spinlock.c b/app/test/test_spinlock.c
index 3f59372300..d655b606f7 100644
--- a/app/test/test_spinlock.c
+++ b/app/test/test_spinlock.c
@@ -302,4 +302,4 @@ test_spinlock(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(spinlock_autotest, test_spinlock);
+REGISTER_FAST_TEST(spinlock_autotest, True, True, test_spinlock);
diff --git a/app/test/test_stack.c b/app/test/test_stack.c
index bc38961433..93bba18469 100644
--- a/app/test/test_stack.c
+++ b/app/test/test_stack.c
@@ -379,5 +379,5 @@ test_lf_stack(void)
 #endif
 }
 
-REGISTER_TEST_COMMAND(stack_autotest, test_stack);
-REGISTER_TEST_COMMAND(stack_lf_autotest, test_lf_stack);
+REGISTER_FAST_TEST(stack_autotest, False, True, test_stack);
+REGISTER_FAST_TEST(stack_lf_autotest, False, True, test_lf_stack);
diff --git a/app/test/test_stack_perf.c b/app/test/test_stack_perf.c
index 1eae00a334..c5e1caa036 100644
--- a/app/test/test_stack_perf.c
+++ b/app/test/test_stack_perf.c
@@ -354,5 +354,5 @@ test_lf_stack_perf(void)
 #endif
 }
 
-REGISTER_TEST_COMMAND(stack_perf_autotest, test_stack_perf);
-REGISTER_TEST_COMMAND(stack_lf_perf_autotest, test_lf_stack_perf);
+REGISTER_PERF_TEST(stack_perf_autotest, test_stack_perf);
+REGISTER_PERF_TEST(stack_lf_perf_autotest, test_lf_stack_perf);
diff --git a/app/test/test_string_fns.c b/app/test/test_string_fns.c
index 5e105d2bb9..07e72bf680 100644
--- a/app/test/test_string_fns.c
+++ b/app/test/test_string_fns.c
@@ -182,4 +182,4 @@ test_string_fns(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(string_autotest, test_string_fns);
+REGISTER_FAST_TEST(string_autotest, True, True, test_string_fns);
diff --git a/app/test/test_tailq.c b/app/test/test_tailq.c
index 9520219b0a..5598bcd12d 100644
--- a/app/test/test_tailq.c
+++ b/app/test/test_tailq.c
@@ -125,4 +125,4 @@ test_tailq(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(tailq_autotest, test_tailq);
+REGISTER_FAST_TEST(tailq_autotest, True, True, test_tailq);
diff --git a/app/test/test_thash.c b/app/test/test_thash.c
index 53d9611e18..dd6468eb56 100644
--- a/app/test/test_thash.c
+++ b/app/test/test_thash.c
@@ -966,4 +966,4 @@ test_thash(void)
 	return unit_test_suite_runner(&thash_tests);
 }
 
-REGISTER_TEST_COMMAND(thash_autotest, test_thash);
+REGISTER_FAST_TEST(thash_autotest, True, True, test_thash);
diff --git a/app/test/test_thash_perf.c b/app/test/test_thash_perf.c
index 687582aa32..9dfd5d3c21 100644
--- a/app/test/test_thash_perf.c
+++ b/app/test/test_thash_perf.c
@@ -135,4 +135,4 @@ test_thash_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(thash_perf_autotest, test_thash_perf);
+REGISTER_PERF_TEST(thash_perf_autotest, test_thash_perf);
diff --git a/app/test/test_threads.c b/app/test/test_threads.c
index a4c4f651a4..6ed6b168d4 100644
--- a/app/test/test_threads.c
+++ b/app/test/test_threads.c
@@ -279,4 +279,4 @@ test_threads(void)
 	return unit_test_suite_runner(&threads_test_suite);
 }
 
-REGISTER_TEST_COMMAND(threads_autotest, test_threads);
+REGISTER_FAST_TEST(threads_autotest, True, True, test_threads);
diff --git a/app/test/test_ticketlock.c b/app/test/test_ticketlock.c
index 242c136478..25af697c0a 100644
--- a/app/test/test_ticketlock.c
+++ b/app/test/test_ticketlock.c
@@ -314,4 +314,4 @@ test_ticketlock(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(ticketlock_autotest, test_ticketlock);
+REGISTER_FAST_TEST(ticketlock_autotest, True, True, test_ticketlock);
diff --git a/app/test/test_timer.c b/app/test/test_timer.c
index 0c36dc9010..09608a1845 100644
--- a/app/test/test_timer.c
+++ b/app/test/test_timer.c
@@ -594,4 +594,4 @@ test_timer(void)
 	return TEST_SUCCESS;
 }
 
-REGISTER_TEST_COMMAND(timer_autotest, test_timer);
+REGISTER_FAST_TEST(timer_autotest, False, True, test_timer);
diff --git a/app/test/test_timer_perf.c b/app/test/test_timer_perf.c
index 0ede4b3e40..d2d74ebbc6 100644
--- a/app/test/test_timer_perf.c
+++ b/app/test/test_timer_perf.c
@@ -131,4 +131,4 @@ test_timer_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(timer_perf_autotest, test_timer_perf);
+REGISTER_PERF_TEST(timer_perf_autotest, test_timer_perf);
diff --git a/app/test/test_timer_racecond.c b/app/test/test_timer_racecond.c
index bb56ae8324..6f8b448ff8 100644
--- a/app/test/test_timer_racecond.c
+++ b/app/test/test_timer_racecond.c
@@ -172,4 +172,4 @@ test_timer_racecond(void)
 	return TEST_SUCCESS;
 }
 
-REGISTER_TEST_COMMAND(timer_racecond_autotest, test_timer_racecond);
+REGISTER_PERF_TEST(timer_racecond_autotest, test_timer_racecond);
diff --git a/app/test/test_trace.c b/app/test/test_trace.c
index ad4a394a29..96829caed5 100644
--- a/app/test/test_trace.c
+++ b/app/test/test_trace.c
@@ -250,4 +250,4 @@ test_trace(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(trace_autotest, test_trace);
+REGISTER_FAST_TEST(trace_autotest, True, True, test_trace);
diff --git a/app/test/test_trace_perf.c b/app/test/test_trace_perf.c
index 46ae7d8074..a6dd075722 100644
--- a/app/test/test_trace_perf.c
+++ b/app/test/test_trace_perf.c
@@ -179,4 +179,4 @@ test_trace_perf(void)
 	return TEST_SUCCESS;
 }
 
-REGISTER_TEST_COMMAND(trace_perf_autotest, test_trace_perf);
+REGISTER_PERF_TEST(trace_perf_autotest, test_trace_perf);
diff --git a/app/test/test_version.c b/app/test/test_version.c
index 1e1ff18656..21016237bc 100644
--- a/app/test/test_version.c
+++ b/app/test/test_version.c
@@ -25,4 +25,4 @@ test_version(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(version_autotest, test_version);
+REGISTER_FAST_TEST(version_autotest, True, True, test_version);
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v5 03/10] app/test: make telemetry data test buildable on windows
  2023-08-15 15:10 ` [PATCH v5 00/10] " Bruce Richardson
  2023-08-15 15:10   ` [PATCH v5 01/10] app/test: add new macros for various test types Bruce Richardson
  2023-08-15 15:10   ` [PATCH v5 02/10] app/test: tag tests with the test type Bruce Richardson
@ 2023-08-15 15:10   ` Bruce Richardson
  2023-08-21 15:50     ` Tyler Retzlaff
  2023-08-15 15:10   ` [PATCH v5 04/10] app/test: build using per-file dependency matrix Bruce Richardson
                     ` (6 subsequent siblings)
  9 siblings, 1 reply; 93+ messages in thread
From: Bruce Richardson @ 2023-08-15 15:10 UTC (permalink / raw)
  To: dev; +Cc: ci, Bruce Richardson, Morten Brørup

To enable the building of the telemetry data tests file when building on
windows, we need to provide a stub implementation. That way, the test
file is buildable any time the library itself is built.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
---
 app/test/test_telemetry_data.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/app/test/test_telemetry_data.c b/app/test/test_telemetry_data.c
index a960f44c1b..b2dc1d75db 100644
--- a/app/test/test_telemetry_data.c
+++ b/app/test/test_telemetry_data.c
@@ -2,11 +2,20 @@
  * Copyright 2020 Intel Corporation
  */
 
+#ifdef RTE_EXEC_ENV_WINDOWS
+#include "test.h"
+
+static int
+telemetry_data_autotest(void)
+{
+	return TEST_SKIPPED;
+}
+
+#else
+
 #include <string.h>
 #include <sys/socket.h>
-#ifndef RTE_EXEC_ENV_WINDOWS
 #include <sys/un.h>
-#endif
 #include <unistd.h>
 #include <limits.h>
 
@@ -604,5 +613,6 @@ telemetry_data_autotest(void)
 	close(sock);
 	return 0;
 }
+#endif /* windows/non-windows */
 
 REGISTER_TEST_COMMAND(telemetry_data_autotest, telemetry_data_autotest);
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v5 04/10] app/test: build using per-file dependency matrix
  2023-08-15 15:10 ` [PATCH v5 00/10] " Bruce Richardson
                     ` (2 preceding siblings ...)
  2023-08-15 15:10   ` [PATCH v5 03/10] app/test: make telemetry data test buildable on windows Bruce Richardson
@ 2023-08-15 15:10   ` Bruce Richardson
  2023-08-15 19:05     ` Patrick Robb
  2023-08-15 15:10   ` [PATCH v5 05/10] app/test: define unit tests suites based on test macros Bruce Richardson
                     ` (5 subsequent siblings)
  9 siblings, 1 reply; 93+ messages in thread
From: Bruce Richardson @ 2023-08-15 15:10 UTC (permalink / raw)
  To: dev; +Cc: ci, Bruce Richardson, Morten Brørup

Rather than using if-else constructs to selectively add or remove files
from the UT build, switch to a table-based approach where each file
lists out what libs or drivers it depends upon.

Initial version of this table was generated via analysis of the header
files included in each C file. The basic dependencies of the test binary
[cmdline, ring, mempool and mbuf] were then removed from the per-file
lists, as there is no point in checking them as the whole app will be
disabled if they are not present.

With the file list, the dependencies for the "utility" C-files are kept
separate, so that other tests which depend on the functions provided by
those files can have that dependency recorded properly. The basic
cryptodev tests also fall into this category as functions from the main
cryptodev test file are used by other crypto tests.

As well as the main table for internal dependencies, some test
files have separate external components too. A second, much smaller
table lists these dependencies.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
---
 app/meson.build      |   8 +-
 app/test/meson.build | 823 ++++++++++++-------------------------------
 2 files changed, 238 insertions(+), 593 deletions(-)

diff --git a/app/meson.build b/app/meson.build
index 4fc1a83eba..0d8b618e7f 100644
--- a/app/meson.build
+++ b/app/meson.build
@@ -32,6 +32,11 @@ apps = [
         'test-security-perf',
 ]
 
+if get_option('tests')
+# build the auto test app if enabled.
+    apps += 'test'
+endif
+
 default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API']
 default_ldflags = []
 if get_option('default_library') == 'static' and not is_windows
@@ -106,6 +111,3 @@ foreach app:apps
             install_rpath: join_paths(get_option('prefix'), driver_install_path),
             install: true)
 endforeach
-
-# special case the autotests
-subdir('test')
diff --git a/app/test/meson.build b/app/test/meson.build
index 66897c14a3..2b885ae273 100644
--- a/app/test/meson.build
+++ b/app/test/meson.build
@@ -1,446 +1,230 @@
 # SPDX-License-Identifier: BSD-3-Clause
-# Copyright(c) 2017 Intel Corporation
-
-if not get_option('tests')
-    subdir_done()
-endif
-
-test_sources = files(
-        'commands.c',
-        'packet_burst_generator.c',
-        'test.c',
-        'test_acl.c',
-        'test_alarm.c',
-        'test_atomic.c',
-        'test_barrier.c',
-        'test_bitops.c',
-        'test_bitmap.c',
-        'test_bpf.c',
-        'test_byteorder.c',
-        'test_cksum.c',
-        'test_cksum_perf.c',
-        'test_cmdline.c',
-        'test_cmdline_cirbuf.c',
-        'test_cmdline_etheraddr.c',
-        'test_cmdline_ipaddr.c',
-        'test_cmdline_lib.c',
-        'test_cmdline_num.c',
-        'test_cmdline_portlist.c',
-        'test_cmdline_string.c',
-        'test_common.c',
-        'test_cpuflags.c',
-        'test_crc.c',
-        'test_cryptodev.c',
-        'test_cryptodev_asym.c',
-        'test_cryptodev_blockcipher.c',
-        'test_cryptodev_crosscheck.c',
-        'test_cryptodev_security_ipsec.c',
-        'test_cryptodev_security_pdcp.c',
-        'test_cycles.c',
-        'test_debug.c',
-        'test_devargs.c',
-        'test_distributor.c',
-        'test_distributor_perf.c',
-        'test_dmadev.c',
-        'test_dmadev_api.c',
-        'test_eal_flags.c',
-        'test_eal_fs.c',
-        'test_efd.c',
-        'test_efd_perf.c',
-        'test_errno.c',
-        'test_ethdev_link.c',
-        'test_event_crypto_adapter.c',
-        'test_event_eth_rx_adapter.c',
-        'test_event_ring.c',
-        'test_event_timer_adapter.c',
-        'test_eventdev.c',
-        'test_external_mem.c',
-        'test_fbarray.c',
-        'test_fib.c',
-        'test_fib_perf.c',
-        'test_fib6.c',
-        'test_fib6_perf.c',
-        'test_func_reentrancy.c',
-        'test_hash.c',
-        'test_hash_functions.c',
-        'test_hash_multiwriter.c',
-        'test_hash_readwrite.c',
-        'test_hash_perf.c',
-        'test_hash_readwrite_lf_perf.c',
-        'test_interrupts.c',
-        'test_ipfrag.c',
-        'test_ipsec.c',
-        'test_ipsec_sad.c',
-        'test_ipsec_perf.c',
-        'test_kvargs.c',
-        'test_lcores.c',
-        'test_logs.c',
-        'test_lpm.c',
-        'test_lpm6.c',
-        'test_lpm6_perf.c',
-        'test_lpm_perf.c',
-        'test_malloc.c',
-        'test_malloc_perf.c',
-        'test_mbuf.c',
-        'test_member.c',
-        'test_member_perf.c',
-        'test_memcpy.c',
-        'test_memcpy_perf.c',
-        'test_memory.c',
-        'test_mempool.c',
-        'test_mempool_perf.c',
-        'test_memzone.c',
-        'test_meter.c',
-        'test_mcslock.c',
-        'test_mp_secondary.c',
-        'test_per_lcore.c',
-        'test_pflock.c',
-        'test_pmd_perf.c',
-        'test_power.c',
-        'test_power_cpufreq.c',
-        'test_power_kvm_vm.c',
-        'test_power_intel_uncore.c',
-        'test_prefetch.c',
-        'test_rand_perf.c',
-        'test_rawdev.c',
-        'test_rcu_qsbr.c',
-        'test_rcu_qsbr_perf.c',
-        'test_reassembly_perf.c',
-        'test_reciprocal_division.c',
-        'test_reciprocal_division_perf.c',
-        'test_red.c',
-        'test_pie.c',
-        'test_reorder.c',
-        'test_rib.c',
-        'test_rib6.c',
-        'test_ring.c',
-        'test_ring_mpmc_stress.c',
-        'test_ring_hts_stress.c',
-        'test_ring_mt_peek_stress.c',
-        'test_ring_mt_peek_stress_zc.c',
-        'test_ring_perf.c',
-        'test_ring_rts_stress.c',
-        'test_ring_st_peek_stress.c',
-        'test_ring_st_peek_stress_zc.c',
-        'test_ring_stress.c',
-        'test_rwlock.c',
-        'test_sched.c',
-        'test_security.c',
-        'test_security_inline_macsec.c',
-        'test_security_inline_proto.c',
-        'test_seqlock.c',
-        'test_service_cores.c',
-        'test_spinlock.c',
-        'test_stack.c',
-        'test_stack_perf.c',
-        'test_string_fns.c',
-        'test_tailq.c',
-        'test_thash.c',
-        'test_thash_perf.c',
-        'test_threads.c',
-        'test_timer.c',
-        'test_timer_perf.c',
-        'test_timer_racecond.c',
-        'test_timer_secondary.c',
-        'test_ticketlock.c',
-        'test_trace.c',
-        'test_trace_register.c',
-        'test_trace_perf.c',
-        'test_version.c',
-        'virtual_pmd.c',
-)
-
-test_deps = dpdk_libs_enabled
-# as well as libs, the pci and vdev bus drivers are needed for a lot of tests
-test_deps += ['bus_pci', 'bus_vdev']
-
-# Each test is marked with flags:
-# - the first flag indicates whether the test can run in no-huge mode,
-# - the second flag indicates whether the test can run with ASan enabled,
-fast_tests = [
-        ['acl_autotest', true, true],
-        ['atomic_autotest', false, true],
-        ['bitmap_autotest', true, true],
-        ['bpf_autotest', true, true],
-        ['bpf_convert_autotest', true, true],
-        ['bitops_autotest', true, true],
-        ['byteorder_autotest', true, true],
-        ['cksum_autotest', true, true],
-        ['cmdline_autotest', true, true],
-        ['common_autotest', true, true],
-        ['cpuflags_autotest', true, true],
-        ['debug_autotest', true, true],
-        ['devargs_autotest', true, true],
-        ['eal_flags_c_opt_autotest', false, false],
-        ['eal_flags_main_opt_autotest', false, false],
-        ['eal_flags_n_opt_autotest', false, false],
-        ['eal_flags_hpet_autotest', false, false],
-        ['eal_flags_no_huge_autotest', false, false],
-        ['eal_flags_a_opt_autotest', false, false],
-        ['eal_flags_b_opt_autotest', false, false],
-        ['eal_flags_vdev_opt_autotest', false, false],
-        ['eal_flags_r_opt_autotest', false, false],
-        ['eal_flags_mem_autotest', false, false],
-        ['eal_flags_file_prefix_autotest', false, false],
-        ['eal_flags_misc_autotest', false, false],
-        ['eal_fs_autotest', true, true],
-        ['errno_autotest', true, true],
-        ['ethdev_link_status', true, true],
-        ['event_ring_autotest', true, true],
-        ['fib_autotest', true, true],
-        ['fib6_autotest', true, true],
-        ['func_reentrancy_autotest', false, true],
-        ['hash_autotest', true, true],
-        ['interrupt_autotest', true, true],
-        ['ipfrag_autotest', false, true],
-        ['lcores_autotest', true, true],
-        ['logs_autotest', true, true],
-        ['lpm_autotest', true, true],
-        ['lpm6_autotest', true, true],
-        ['malloc_autotest', false, true],
-        ['mbuf_autotest', false, true],
-        ['mcslock_autotest', false, true],
-        ['memcpy_autotest', true, true],
-        ['memory_autotest', false, true],
-        ['mempool_autotest', false, true],
-        ['memzone_autotest', false, true],
-        ['meter_autotest', true, true],
-        ['multiprocess_autotest', false, false],
-        ['per_lcore_autotest', true, true],
-        ['pflock_autotest', true, true],
-        ['prefetch_autotest', true, true],
-        ['rcu_qsbr_autotest', true, true],
-        ['pie_autotest', true, true],
-        ['rib_autotest', true, true],
-        ['rib6_autotest', true, true],
-        ['ring_autotest', true, true],
-        ['rwlock_test1_autotest', true, true],
-        ['rwlock_rda_autotest', true, true],
-        ['rwlock_rds_wrm_autotest', true, true],
-        ['rwlock_rde_wro_autotest', true, true],
-        ['sched_autotest', true, true],
-        ['security_autotest', false, true],
-        ['seqlock_autotest', true, true],
-        ['spinlock_autotest', true, true],
-        ['stack_autotest', false, true],
-        ['stack_lf_autotest', false, true],
-        ['string_autotest', true, true],
-        ['tailq_autotest', true, true],
-        ['ticketlock_autotest', true, true],
-        ['timer_autotest', false, true],
-        ['user_delay_us', true, true],
-        ['version_autotest', true, true],
-        ['crc_autotest', true, true],
-        ['distributor_autotest', false, true],
-        ['eventdev_common_autotest', true, true],
-        ['fbarray_autotest', true, true],
-        ['hash_readwrite_func_autotest', false, true],
-        ['ipsec_autotest', true, true],
-        ['kvargs_autotest', true, true],
-        ['member_autotest', true, true],
-        ['power_cpufreq_autotest', false, true],
-        ['power_autotest', true, true],
-        ['power_kvm_vm_autotest', false, true],
-        ['power_intel_uncore_autotest', true, true],
-        ['reorder_autotest', true, true],
-        ['service_autotest', true, true],
-        ['thash_autotest', true, true],
-        ['threads_autotest', true, true],
-        ['trace_autotest', true, true],
-]
-
-# Tests known to have issues or which don't belong in other tests lists.
-extra_test_names = [
-        'alarm_autotest', # ee00af60170b ("test: remove strict timing requirements some tests")
-        'red_autotest', # https://bugs.dpdk.org/show_bug.cgi?id=826
-]
-
-perf_test_names = [
-        'ring_perf_autotest',
-        'malloc_perf_autotest',
-        'mempool_perf_autotest',
-        'memcpy_perf_autotest',
-        'hash_perf_autotest',
-        'timer_perf_autotest',
-        'reciprocal_division',
-        'reciprocal_division_perf',
-        'lpm_perf_autotest',
-        'rib_slow_autotest',
-        'fib_slow_autotest',
-        'fib_perf_autotest',
-        'red_all',
-        'pie_all',
-        'barrier_autotest',
-        'hash_multiwriter_autotest',
-        'timer_racecond_autotest',
-        'efd_autotest',
-        'hash_functions_autotest',
-        'member_perf_autotest',
-        'efd_perf_autotest',
-        'lpm6_perf_autotest',
-        'rib6_slow_autotest',
-        'fib6_slow_autotest',
-        'fib6_perf_autotest',
-        'rcu_qsbr_perf_autotest',
-        'red_perf',
-        'pie_perf',
-        'distributor_perf_autotest',
-        'pmd_perf_autotest',
-        'service_perf_autotest',
-        'stack_perf_autotest',
-        'stack_lf_perf_autotest',
-        'rand_perf_autotest',
-        'hash_readwrite_perf_autotest',
-        'hash_readwrite_lf_perf_autotest',
-        'trace_perf_autotest',
-        'ipsec_perf_autotest',
-        'thash_perf_autotest',
-        'reassembly_perf_autotest',
-]
-
-driver_test_names = [
-        'cryptodev_aesni_gcm_autotest',
-        'cryptodev_aesni_mb_autotest',
-        'cryptodev_chacha_poly_mb_autotest',
-        'cryptodev_cn10k_autotest',
-        'cryptodev_cn9k_autotest',
-        'cryptodev_cpu_aesni_mb_autotest',
-        'cryptodev_cpu_aesni_gcm_autotest',
-        'cryptodev_dpaa2_sec_autotest',
-        'cryptodev_dpaa_sec_autotest',
-        'cryptodev_null_autotest',
-        'cryptodev_openssl_autotest',
-        'cryptodev_qat_autotest',
-        'cryptodev_qat_asym_autotest',
-        'cryptodev_qat_raw_api_autotest',
-        'cryptodev_sw_armv8_autotest',
-        'cryptodev_sw_kasumi_autotest',
-        'cryptodev_sw_mvsam_autotest',
-        'cryptodev_sw_snow3g_autotest',
-        'cryptodev_sw_zuc_autotest',
-        'cryptodev_uadk_autotest',
-        'dmadev_autotest',
-]
-
-dump_test_names = []
-
-if not is_windows
-    driver_test_names += [
-            'cryptodev_openssl_asym_autotest',
-            'eventdev_selftest_octeontx',
-            'eventdev_selftest_sw',
-    ]
-
-    dump_test_names += [
-            'dump_struct_sizes',
-            'dump_mempool',
-            'dump_malloc_stats',
-            'dump_devargs',
-            'dump_log_types',
-            'dump_ring',
-            'dump_physmem',
-            'dump_memzone',
-    ]
-endif
-
-# The following linkages are an exception to allow running the
-# unit tests without requiring that the developer install the
-# DPDK libraries.  Explicit linkage of drivers (plugin libraries)
-# in applications should not be used.
-if dpdk_conf.has('RTE_MEMPOOL_RING')
-    test_deps += 'mempool_ring'
-endif
-if dpdk_conf.has('RTE_MEMPOOL_STACK')
-    test_deps += 'mempool_stack'
-endif
-if dpdk_conf.has('RTE_EVENT_SKELETON')
-    test_deps += 'event_skeleton'
-endif
-
-if dpdk_conf.has('RTE_LIB_GRAPH')
-    test_sources += 'test_graph.c'
-    fast_tests += [['graph_autotest', true, true]]
-    fast_tests += [['node_list_dump', true, true]]
-    test_sources += 'test_graph_perf.c'
-    perf_test_names += 'graph_perf_autotest'
-endif
-if dpdk_conf.has('RTE_LIB_METRICS')
-    test_sources += ['test_metrics.c']
-    fast_tests += [['metrics_autotest', true, true]]
-endif
-if not is_windows and dpdk_conf.has('RTE_LIB_TELEMETRY')
-    test_sources += ['test_telemetry_json.c', 'test_telemetry_data.c']
-    fast_tests += [['telemetry_json_autotest', true, true]]
-    fast_tests += [['telemetry_data_autotest', true, true]]
-endif
-if dpdk_conf.has('RTE_LIB_PIPELINE')
-# pipeline lib depends on port and table libs, so those must be present
-# if pipeline library is.
-    test_sources += [
-            'test_table.c',
-            'test_table_acl.c',
-            'test_table_combined.c',
-            'test_table_pipeline.c',
-            'test_table_ports.c',
-            'test_table_tables.c',
-    ]
-    fast_tests += [['table_autotest', true, true]]
-endif
-
-# The following linkages of drivers are required because
-# they are used via a driver-specific API.
-if dpdk_conf.has('RTE_NET_BOND')
-    test_deps += 'net_bond'
-    test_sources += ['test_link_bonding.c', 'test_link_bonding_rssconf.c']
-    driver_test_names += ['link_bonding_autotest', 'link_bonding_rssconf_autotest']
-    if dpdk_conf.has('RTE_NET_RING')
-        test_sources += 'test_link_bonding_mode4.c'
-        driver_test_names += 'link_bonding_mode4_autotest'
-    endif
-endif
-if dpdk_conf.has('RTE_LIB_EVENTDEV') and dpdk_conf.has('RTE_NET_RING')
-    test_deps += 'net_ring'
-    test_sources += 'test_pmd_ring_perf.c'
-    test_sources += 'test_pmd_ring.c'
-    test_sources += 'test_event_eth_tx_adapter.c'
-    test_sources += 'sample_packet_forward.c'
-    fast_tests += [['ring_pmd_autotest', true, true]]
-    perf_test_names += 'ring_pmd_perf_autotest'
-    fast_tests += [['event_eth_tx_adapter_autotest', false, true]]
-    if dpdk_conf.has('RTE_LIB_BITRATESTATS')
-        test_sources += 'test_bitratestats.c'
-        fast_tests += [['bitratestats_autotest', true, true]]
-    endif
-    if dpdk_conf.has('RTE_LIB_LATENCYSTATS')
-        test_sources += 'test_latencystats.c'
-        fast_tests += [['latencystats_autotest', true, true]]
-    endif
-    if dpdk_conf.has('RTE_LIB_PDUMP')
-        test_sources += 'test_pdump.c'
-        fast_tests += [['pdump_autotest', true, false]]
-    endif
-endif
-if dpdk_conf.has('RTE_NET_NULL')
-    test_deps += 'net_null'
-    test_sources += 'test_vdev.c'
-    fast_tests += [['vdev_autotest', true, true]]
-endif
-if dpdk_conf.has('RTE_RAW_SKELETON')
-    test_deps += 'raw_skeleton'
-    fast_tests += [['rawdev_autotest', true, true]]
-endif
-
-if dpdk_conf.has('RTE_HAS_LIBPCAP')
-    ext_deps += pcap_dep
-    if dpdk_conf.has('RTE_LIB_PCAPNG')
-        test_sources += 'test_pcapng.c'
+# Copyright(c) 2017-2023 Intel Corporation
+
+# the main test files [test.c and commands.c] relies on these libraries
+deps += ['cmdline', 'ring', 'mempool', 'mbuf']
+sources += files('commands.c', 'test.c')
+
+# some other utility C files, providing functions used by various tests
+# so we need to include these deps in the dependency list for the files using those fns.
+packet_burst_generator_deps = ['net']
+sample_packet_forward_deps = ['net_ring', 'ethdev', 'bus_vdev']
+virtual_pmd_deps = ['ethdev', 'net', 'bus_pci']
+# test_cryptodev has material that other crypto tests need
+test_cryptodev_deps = ['bus_vdev', 'net', 'cryptodev', 'crypto_scheduler', 'security']
+
+source_file_deps = {
+    # The C files providing functionality to other test cases
+    'packet_burst_generator.c': packet_burst_generator_deps,
+#    'resource.c': [],          # unused currently.
+    'sample_packet_forward.c': sample_packet_forward_deps,
+    'virtual_pmd.c': virtual_pmd_deps,
+
+    # the various test_*.c files
+    'test_acl.c': ['net', 'acl'],
+    'test_alarm.c': [],
+    'test_atomic.c': ['hash'],
+    'test_barrier.c': [],
+    'test_bitmap.c': [],
+    'test_bitops.c': [],
+    'test_bitratestats.c': ['metrics', 'bitratestats', 'ethdev'] + sample_packet_forward_deps,
+    'test_bpf.c': ['bpf', 'net'],
+    'test_byteorder.c': [],
+#    'test_cfgfile.c': ['cfgfile'],
+    'test_cksum.c': ['net'],
+    'test_cksum_perf.c': ['net'],
+    'test_cmdline.c': [],
+    'test_cmdline_cirbuf.c': [],
+    'test_cmdline_etheraddr.c': ['net'],
+    'test_cmdline_ipaddr.c': [],
+    'test_cmdline_lib.c': [],
+    'test_cmdline_num.c': [],
+    'test_cmdline_portlist.c': [],
+    'test_cmdline_string.c': [],
+    'test_common.c': [],
+    'test_compressdev.c': ['compressdev'],
+    'test_cpuflags.c': [],
+    'test_crc.c': ['net'],
+    'test_cryptodev.c': test_cryptodev_deps,
+    'test_cryptodev_asym.c': ['bus_vdev'] + test_cryptodev_deps,
+    'test_cryptodev_blockcipher.c': test_cryptodev_deps,
+    'test_cryptodev_crosscheck.c': test_cryptodev_deps,
+    'test_cryptodev_security_ipsec.c': test_cryptodev_deps,
+    'test_cryptodev_security_pdcp.c': test_cryptodev_deps,
+    'test_cycles.c': [],
+    'test_debug.c': [],
+    'test_devargs.c': ['kvargs'],
+    'test_distributor.c': ['distributor'],
+    'test_distributor_perf.c': ['distributor'],
+    'test_dmadev.c': ['dmadev', 'bus_vdev'],
+    'test_dmadev_api.c': ['dmadev'],
+    'test_eal_flags.c': [],
+    'test_eal_fs.c': [],
+    'test_efd.c': ['efd', 'net'],
+    'test_efd_perf.c': ['efd', 'hash'],
+    'test_errno.c': [],
+    'test_ethdev_link.c': ['ethdev'],
+    'test_event_crypto_adapter.c': ['cryptodev', 'eventdev', 'bus_vdev'],
+    'test_event_eth_rx_adapter.c': ['ethdev', 'eventdev', 'bus_vdev'],
+    'test_event_eth_tx_adapter.c': ['bus_vdev', 'ethdev', 'net_ring', 'eventdev'],
+    'test_event_ring.c': ['eventdev'],
+    'test_event_timer_adapter.c': ['ethdev', 'eventdev', 'bus_vdev'],
+    'test_eventdev.c': ['eventdev', 'bus_vdev'],
+    'test_external_mem.c': [],
+    'test_fbarray.c': [],
+    'test_fib.c': ['net', 'fib'],
+    'test_fib6.c': ['rib', 'fib'],
+    'test_fib6_perf.c': ['fib'],
+    'test_fib_perf.c': ['net', 'fib'],
+    'test_flow_classify.c': ['net', 'acl', 'table', 'ethdev', 'flow_classify'],
+    'test_func_reentrancy.c': ['hash', 'lpm'],
+    'test_graph.c': ['graph'],
+    'test_graph_perf.c': ['graph'],
+    'test_hash.c': ['net', 'hash'],
+    'test_hash_functions.c': ['hash'],
+    'test_hash_multiwriter.c': ['hash'],
+    'test_hash_perf.c': ['hash'],
+    'test_hash_readwrite.c': ['hash'],
+    'test_hash_readwrite_lf_perf.c': ['hash'],
+    'test_interrupts.c': [],
+    'test_ipfrag.c': ['net', 'ip_frag'],
+    'test_ipsec.c': ['bus_vdev', 'net', 'cryptodev', 'ipsec', 'security'],
+    'test_ipsec_perf.c': ['net', 'ipsec'],
+    'test_ipsec_sad.c': ['ipsec'],
+    'test_kvargs.c': ['kvargs'],
+    'test_latencystats.c': ['ethdev', 'latencystats', 'metrics'] + sample_packet_forward_deps,
+    'test_lcores.c': [],
+    'test_link_bonding.c': ['ethdev', 'net_bond',
+        'net'] + packet_burst_generator_deps + virtual_pmd_deps,
+    'test_link_bonding_mode4.c': ['ethdev', 'net_ring', 'net_bond',
+        'net'] + packet_burst_generator_deps,
+    'test_link_bonding_rssconf.c': ['ethdev', 'bus_vdev', 'net_bond'],
+    'test_logs.c': [],
+    'test_lpm.c': ['net', 'lpm'],
+    'test_lpm6.c': ['lpm'],
+    'test_lpm6_perf.c': ['lpm'],
+    'test_lpm_perf.c': ['net', 'lpm'],
+    'test_malloc.c': [],
+    'test_malloc_perf.c': [],
+    'test_mbuf.c': ['net'],
+    'test_mcslock.c': [],
+    'test_member.c': ['member', 'net'],
+    'test_member_perf.c': ['hash', 'member'],
+    'test_memcpy.c': [],
+    'test_memcpy_perf.c': [],
+    'test_memory.c': [],
+    'test_mempool.c': [],
+    'test_mempool_perf.c': [],
+    'test_memzone.c': [],
+    'test_meter.c': ['meter'],
+    'test_metrics.c': ['metrics'],
+    'test_mp_secondary.c': ['hash', 'lpm'],
+    'test_pcapng.c': ['ethdev', 'net', 'pcapng'],
+    'test_pdcp.c': ['eventdev', 'pdcp', 'net', 'timer', 'security'],
+    'test_pdump.c': ['pdump'] + sample_packet_forward_deps,
+    'test_per_lcore.c': [],
+    'test_pflock.c': [],
+    'test_pie.c': ['sched'],
+    'test_pmd_perf.c': ['ethdev', 'net'] + packet_burst_generator_deps,
+    'test_pmd_ring.c': ['net_ring', 'ethdev', 'bus_vdev'],
+    'test_pmd_ring_perf.c': ['ethdev', 'net_ring', 'bus_vdev'],
+    'test_power.c': ['power'],
+    'test_power_cpufreq.c': ['power'],
+    'test_power_intel_uncore.c': ['power'],
+    'test_power_kvm_vm.c': ['power'],
+    'test_prefetch.c': [],
+    'test_rand_perf.c': [],
+    'test_rawdev.c': ['rawdev', 'bus_vdev'],
+    'test_rcu_qsbr.c': ['rcu', 'hash'],
+    'test_rcu_qsbr_perf.c': ['rcu', 'hash'],
+    'test_reassembly_perf.c': ['net', 'ip_frag'],
+    'test_reciprocal_division.c': [],
+    'test_reciprocal_division_perf.c': [],
+    'test_red.c': ['sched'],
+    'test_reorder.c': ['reorder'],
+#    'test_resource.c': [],
+    'test_rib.c': ['net', 'rib'],
+    'test_rib6.c': ['net', 'rib'],
+    'test_ring.c': [],
+    'test_ring_hts_stress.c': [],
+    'test_ring_mpmc_stress.c': [],
+    'test_ring_mt_peek_stress.c': [],
+    'test_ring_mt_peek_stress_zc.c': [],
+    'test_ring_perf.c': [],
+    'test_ring_rts_stress.c': [],
+    'test_ring_st_peek_stress.c': [],
+    'test_ring_st_peek_stress_zc.c': [],
+    'test_ring_stress.c': [],
+    'test_rwlock.c': [],
+    'test_sched.c': ['net', 'sched'],
+    'test_security.c': ['net', 'security'],
+    'test_security_inline_macsec.c': ['ethdev', 'security'],
+    'test_security_inline_proto.c': ['ethdev', 'security', 'eventdev'] + test_cryptodev_deps,
+    'test_seqlock.c': [],
+    'test_service_cores.c': [],
+    'test_spinlock.c': [],
+    'test_stack.c': ['stack'],
+    'test_stack_perf.c': ['stack'],
+    'test_string_fns.c': [],
+    'test_table.c': ['table', 'pipeline', 'port'],
+    'test_table_acl.c': ['net', 'table', 'pipeline', 'port'],
+    'test_table_combined.c': ['table', 'pipeline', 'port'],
+    'test_table_pipeline.c': ['pipeline', 'table', 'port'],
+    'test_table_ports.c': ['table', 'pipeline', 'port'],
+    'test_table_tables.c': ['table', 'pipeline', 'port'],
+    'test_tailq.c': [],
+    'test_telemetry_data.c': ['telemetry'],
+    'test_telemetry_json.c': ['telemetry'],
+    'test_thash.c': ['net', 'hash'],
+    'test_thash_perf.c': ['hash'],
+    'test_threads.c': [],
+    'test_ticketlock.c': [],
+    'test_timer.c': ['timer'],
+    'test_timer_perf.c': ['timer'],
+    'test_timer_racecond.c': ['timer'],
+    'test_timer_secondary.c': ['timer'],
+    'test_trace.c': [],
+    'test_trace_perf.c': [],
+    'test_trace_register.c': [],
+    'test_vdev.c': ['kvargs', 'bus_vdev'],
+    'test_version.c': [],
+}
+
+source_file_ext_deps = {
+    'test_compressdev.c': ['zlib'],
+    'test_pcapng.c': ['pcap'],
+}
+
+def_lib = get_option('default_library')
+foreach f, f_deps : source_file_deps
+    has_deps = true
+    foreach d : f_deps
+        if not is_variable(def_lib + '_rte_' + d)
+            has_deps = false
+            break
+        else
+            # technically we might not need this dep, but adding it is harmless
+            if d not in deps
+                deps += d
+            endif
+        endif
+    endforeach
+    # check for any external dependencies for this file
+    if source_file_ext_deps.has_key(f)
+        foreach d: source_file_ext_deps.get(f)
+            dep = dependency(d, required: false, method: 'pkg-config')
+            if not dep.found()
+                message('Skipping test file @0@ due to missing external dependency @1@'.format(f, d))
+                has_deps = false
+            else
+                ext_deps += dep
+            endif
+        endforeach
+    endif
+    if has_deps
+        sources += files(f)
     endif
-endif
-
-if dpdk_conf.has('RTE_LIB_PDCP')
-    test_sources += 'test_pdcp.c'
-    fast_tests += [['pdcp_autotest', false, true]]
-endif
+endforeach
 
 if cc.has_argument('-Wno-format-truncation')
     cflags += '-Wno-format-truncation'
@@ -450,154 +234,13 @@ endif
 cflags += '-fno-strict-aliasing'
 
 # Enable using internal APIs in unit tests
-cflags += ['-DALLOW_INTERNAL_API']
-
-test_dep_objs = []
-if dpdk_conf.has('RTE_LIB_COMPRESSDEV')
-    compress_test_dep = dependency('zlib', required: false, method: 'pkg-config')
-    if compress_test_dep.found()
-        test_dep_objs += compress_test_dep
-        test_sources += 'test_compressdev.c'
-        fast_tests += [['compressdev_autotest', false, true]]
-    endif
-endif
-
-if dpdk_conf.has('RTE_CRYPTO_SCHEDULER')
-    driver_test_names += 'cryptodev_scheduler_autotest'
-    test_deps += 'crypto_scheduler'
-endif
-
-foreach d:test_deps
-    def_lib = get_option('default_library')
-    test_dep_objs += get_variable(def_lib + '_rte_' + d)
-endforeach
-
-link_libs = []
-if get_option('default_library') == 'static'
-    link_libs = dpdk_static_libraries + dpdk_drivers
-endif
-
-dpdk_test = executable('dpdk-test',
-        test_sources,
-        link_whole: link_libs,
-        dependencies: test_dep_objs + ext_deps,
-        c_args: cflags,
-        install_rpath: join_paths(get_option('prefix'),
-             driver_install_path),
-        install: true)
-
-has_hugepage = run_command(py3, files('has_hugepage.py'), check: true).stdout().strip() != '0'
-message('hugepage availability: @0@'.format(has_hugepage))
-
-# some perf tests (eg: memcpy perf autotest)take very long
-# to complete, so timeout to 10 minutes
-timeout_seconds = 600
-timeout_seconds_fast = 10
-
-test_no_huge_args = ['--no-huge', '-m', '2048']
-
-foreach arg : fast_tests
-    test_args = []
-    run_test = true
-    if not has_hugepage
-        if arg[1]
-            test_args += test_no_huge_args
-        else
-            run_test = false
-        endif
-    endif
-
-    if get_option('b_sanitize') == 'address' or get_option('b_sanitize') == 'address,undefined'
-        if not arg[2]
-            run_test = false
-        endif
-    endif
-
-    if (get_option('default_library') == 'shared' and
-        arg[0] == 'event_eth_tx_adapter_autotest')
-        test_args += ['-d', dpdk_drivers_build_dir]
-    endif
-    if is_linux
-        test_args += ['--file-prefix=@0@'.format(arg[0])]
-    endif
-
-    if run_test
-        test(arg[0], dpdk_test,
-                env : ['DPDK_TEST=' + arg[0]],
-                args : test_args,
-                timeout : timeout_seconds_fast,
-                is_parallel : false,
-                suite : 'fast-tests')
-        if not is_windows and arg[0] == 'trace_autotest'
-            test_args += ['--trace=.*']
-            test_args += ['--trace-dir=@0@'.format(meson.current_build_dir())]
-            test(arg[0] + '_with_traces', dpdk_test,
-                    env : ['DPDK_TEST=' + arg[0]],
-                    args : test_args,
-                    timeout : timeout_seconds_fast,
-                    is_parallel : false,
-                    suite : 'fast-tests')
-        endif
-    endif
-endforeach
+cflags += '-DALLOW_INTERNAL_API'
 
-if not is_windows and dpdk_conf.has('RTE_LIB_TELEMETRY')
-    test_args = [dpdk_test]
-    test_args += test_no_huge_args
-    if get_option('default_library') == 'shared'
-        test_args += ['-d', dpdk_drivers_build_dir]
-    endif
-    if dpdk_conf.has('RTE_CRYPTO_NULL')
-        test_args += ['--vdev=crypto_null0']
-    endif
-    if dpdk_conf.has('RTE_DMA_SKELETON')
-        test_args += ['--vdev=dma_skeleton0']
-    endif
-    if dpdk_conf.has('RTE_EVENT_SKELETON')
-        test_args += ['--vdev=event_skeleton0']
-    endif
-    if dpdk_conf.has('RTE_NET_NULL')
-        test_args += ['--vdev=net_null0']
-    endif
-    if dpdk_conf.has('RTE_RAW_SKELETON')
-        test_args += ['--vdev=rawdev_skeleton0']
-    endif
-    test_args += ['-a', '0000:00:00.0']
-    test('telemetry_all', find_program('test_telemetry.sh'),
-            args: test_args,
-            timeout : timeout_seconds_fast,
-            is_parallel : false,
-            suite : 'fast-tests')
+# create a symlink in the app/test directory for the binary, for backward compatibility
+if not is_windows
+    custom_target('test_symlink',
+            output: 'dpdk-test',
+            command: ['ln', '-sf', '../dpdk-test', '@OUTPUT@'],
+            build_by_default: true,
+            install: false)
 endif
-
-foreach arg : perf_test_names
-    test(arg, dpdk_test,
-            env : ['DPDK_TEST=' + arg],
-            timeout : timeout_seconds,
-            is_parallel : false,
-            suite : 'perf-tests')
-endforeach
-
-foreach arg : driver_test_names
-    test(arg, dpdk_test,
-            env : ['DPDK_TEST=' + arg],
-            timeout : timeout_seconds,
-            is_parallel : false,
-            suite : 'driver-tests')
-endforeach
-
-foreach arg : dump_test_names
-    test(arg, dpdk_test,
-            env : ['DPDK_TEST=' + arg],
-            timeout : timeout_seconds,
-            is_parallel : false,
-            suite : 'debug-tests')
-endforeach
-
-foreach arg : extra_test_names
-    test(arg, dpdk_test,
-            env : ['DPDK_TEST=' + arg],
-            timeout : timeout_seconds,
-            is_parallel : false,
-            suite : 'extra-tests')
-endforeach
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v5 05/10] app/test: define unit tests suites based on test macros
  2023-08-15 15:10 ` [PATCH v5 00/10] " Bruce Richardson
                     ` (3 preceding siblings ...)
  2023-08-15 15:10   ` [PATCH v5 04/10] app/test: build using per-file dependency matrix Bruce Richardson
@ 2023-08-15 15:10   ` Bruce Richardson
  2023-08-16 11:02     ` Bruce Richardson
  2023-08-16 14:57     ` David Marchand
  2023-08-15 15:10   ` [PATCH v5 06/10] app/test: add test case for scripted telemetry commands Bruce Richardson
                     ` (4 subsequent siblings)
  9 siblings, 2 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-15 15:10 UTC (permalink / raw)
  To: dev; +Cc: ci, Bruce Richardson, Morten Brørup

Rather than having the test suites listed out in the meson.build files
and having to have them enabled/disabled selectively based on what libs
are being built, pull the tests to run from the source files which were
added to the build.

Most test suites require no additional info other than the list of test
names in the suite. However the fast-test are special that they have
additional parameters associated with them. This requires some
additional work in the test extraction script and in the meson.build
file for processing the output.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
---
 app/meson.build                               |  7 +-
 app/test/suites/meson.build                   | 74 +++++++++++++++++++
 buildtools/get-test-suites.py                 | 33 +++++++++
 .../has-hugepages.py                          |  0
 buildtools/meson.build                        |  2 +
 5 files changed, 115 insertions(+), 1 deletion(-)
 create mode 100644 app/test/suites/meson.build
 create mode 100644 buildtools/get-test-suites.py
 rename app/test/has_hugepage.py => buildtools/has-hugepages.py (100%)

diff --git a/app/meson.build b/app/meson.build
index 0d8b618e7f..c14dc80892 100644
--- a/app/meson.build
+++ b/app/meson.build
@@ -101,7 +101,7 @@ foreach app:apps
         link_libs = dpdk_static_libraries + dpdk_drivers
     endif
 
-    executable('dpdk-' + name,
+    exec = executable('dpdk-' + name,
             sources,
             c_args: cflags,
             link_args: ldflags,
@@ -110,4 +110,9 @@ foreach app:apps
             include_directories: includes,
             install_rpath: join_paths(get_option('prefix'), driver_install_path),
             install: true)
+    if name == 'test'
+        dpdk_test = exec
+        autotest_sources = sources
+        subdir('test/suites')  # define the pre-canned test suites
+    endif
 endforeach
diff --git a/app/test/suites/meson.build b/app/test/suites/meson.build
new file mode 100644
index 0000000000..ec74d8adf2
--- /dev/null
+++ b/app/test/suites/meson.build
@@ -0,0 +1,74 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2023 Intel Corporation
+
+# some perf tests (eg: memcpy perf autotest)take very long
+# to complete, so timeout to 10 minutes
+timeout_seconds = 600
+timeout_seconds_fast = 10
+
+test_no_huge_args = ['--no-huge', '-m', '2048']
+has_hugepage = run_command(has_hugepages_cmd, check: true).stdout().strip() != '0'
+message('hugepage availability: @0@'.format(has_hugepage))
+
+# process source files to determine the different unit test suites
+# - fast_tests
+# - perf_tests
+# - driver_tests
+test_suites = run_command(get_test_suites_cmd, autotest_sources,
+         check: true).stdout().strip().split()
+foreach suite:test_suites
+    # simple cases - tests without parameters or special handling
+    suite = suite.split('=')
+    suite_name = suite[0]
+    suite_tests = suite[1].split(',')
+    if suite_name != 'fast-tests'
+        # simple cases - tests without parameters or special handling
+        foreach t: suite_tests
+            test(t, dpdk_test,
+                    env: ['DPDK_TEST=' + t],
+                    timeout: timeout_seconds,
+                    is_parallel: false,
+                    suite: suite_name)
+        endforeach
+    else
+    # special fast-test handling here
+        foreach t: suite_tests
+            params = t.split(':')
+            test_name = params[0]
+            nohuge = params[1] == 'true'
+            asan = params[2] == 'true'
+
+            test_args = []
+            if nohuge
+                test_args += test_no_huge_args
+            elif not has_hugepage
+                continue  #skip this tests
+            endif
+            if not asan and (get_option('b_sanitize') == 'address'
+                    or get_option('b_sanitize') == 'address,undefined')
+                continue  # skip this test
+            endif
+
+            if get_option('default_library') == 'shared'
+                test_args += ['-d', dpdk_drivers_build_dir]
+            endif
+
+            test(test_name, dpdk_test,
+                args : test_args,
+                env: ['DPDK_TEST=' + test_name],
+                timeout : timeout_seconds_fast,
+                is_parallel : false,
+                suite : 'fast-tests')
+            if not is_windows and test_name == 'trace_autotest'
+                test_args += ['--trace=.*']
+                test_args += ['--trace-dir=@0@'.format(meson.current_build_dir())]
+                test(test_name + '_with_traces', dpdk_test,
+                    args : test_args,
+                    env: ['DPDK_TEST=' + test_name],
+                    timeout : timeout_seconds_fast,
+                    is_parallel : false,
+                    suite : 'fast-tests')
+            endif
+        endforeach
+    endif
+endforeach
diff --git a/buildtools/get-test-suites.py b/buildtools/get-test-suites.py
new file mode 100644
index 0000000000..95a9cad4c8
--- /dev/null
+++ b/buildtools/get-test-suites.py
@@ -0,0 +1,33 @@
+#! /usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2023 Intel Corporation
+
+import sys
+import re
+
+input_list = sys.argv[1:]
+test_def_regex = re.compile("REGISTER_([A-Z]+)_TEST\s*\(\s*([a-z0-9_]+)")
+test_suites = {}
+
+def get_fast_test_params(test_name, ln):
+    "Extract the extra fast-test parameters from the line"
+    #print(f"ln: {ln.rstrip()}, test_name: {test_name}, split: {ln.split(test_name, 1)}")
+    (_, rest_of_line) = ln.split(test_name, 1)
+    (_, nohuge, asan, _func) = rest_of_line.split(',', 3)
+    return f":{nohuge.strip().lower()}:{asan.strip().lower()}"
+
+for fname in input_list:
+    with open(fname) as f:
+        contents = [ln for ln in f.readlines() if test_def_regex.match(ln.strip())]
+    for ln in contents:
+        (test_suite, test_name) = test_def_regex.match(ln).group(1, 2)
+        suite_name = f"{test_suite.lower()}-tests"
+        if suite_name in test_suites:
+            test_suites[suite_name].append(test_name)
+        else:
+            test_suites[suite_name] = [test_name]
+        if suite_name == "fast-tests":
+            test_suites["fast-tests"][-1] += get_fast_test_params(test_name, ln)
+
+for suite in test_suites.keys():
+    print(f"{suite}={','.join(test_suites[suite])}")
diff --git a/app/test/has_hugepage.py b/buildtools/has-hugepages.py
similarity index 100%
rename from app/test/has_hugepage.py
rename to buildtools/has-hugepages.py
diff --git a/buildtools/meson.build b/buildtools/meson.build
index e1c600e40f..ac5e4dcf08 100644
--- a/buildtools/meson.build
+++ b/buildtools/meson.build
@@ -18,6 +18,8 @@ map_to_win_cmd = py3 + files('map_to_win.py')
 sphinx_wrapper = py3 + files('call-sphinx-build.py')
 get_cpu_count_cmd = py3 + files('get-cpu-count.py')
 get_numa_count_cmd = py3 + files('get-numa-count.py')
+get_test_suites_cmd = py3 + files('get-test-suites.py')
+has_hugepages_cmd = py3 + files('has-hugepages.py')
 binutils_avx512_check = (py3 + files('binutils-avx512-check.py') +
                         [objdump] + cc.cmd_array())
 
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v5 06/10] app/test: add test case for scripted telemetry commands
  2023-08-15 15:10 ` [PATCH v5 00/10] " Bruce Richardson
                     ` (4 preceding siblings ...)
  2023-08-15 15:10   ` [PATCH v5 05/10] app/test: define unit tests suites based on test macros Bruce Richardson
@ 2023-08-15 15:10   ` Bruce Richardson
  2023-08-15 15:10   ` [PATCH v5 07/10] app/test: add debug test suite Bruce Richardson
                     ` (3 subsequent siblings)
  9 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-15 15:10 UTC (permalink / raw)
  To: dev; +Cc: ci, Bruce Richardson

The test script for calling all telemetry commands is not discoverable
by checking the C files for the build. Therefore we need to add it in as
a special-case test in the fast-tests group.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 app/test/suites/meson.build             | 30 +++++++++++++++++++++++++
 app/test/{ => suites}/test_telemetry.sh |  0
 2 files changed, 30 insertions(+)
 rename app/test/{ => suites}/test_telemetry.sh (100%)

diff --git a/app/test/suites/meson.build b/app/test/suites/meson.build
index ec74d8adf2..2b428847c6 100644
--- a/app/test/suites/meson.build
+++ b/app/test/suites/meson.build
@@ -72,3 +72,33 @@ foreach suite:test_suites
         endforeach
     endif
 endforeach
+
+# standalone test for telemetry
+if not is_windows and dpdk_conf.has('RTE_LIB_TELEMETRY')
+    test_args = [dpdk_test]
+    test_args += test_no_huge_args
+    if get_option('default_library') == 'shared'
+        test_args += ['-d', dpdk_drivers_build_dir]
+    endif
+    if dpdk_conf.has('RTE_CRYPTO_NULL')
+        test_args += ['--vdev=crypto_null0']
+    endif
+    if dpdk_conf.has('RTE_DMA_SKELETON')
+        test_args += ['--vdev=dma_skeleton0']
+    endif
+    if dpdk_conf.has('RTE_EVENT_SKELETON')
+        test_args += ['--vdev=event_skeleton0']
+    endif
+    if dpdk_conf.has('RTE_NET_NULL')
+        test_args += ['--vdev=net_null0']
+    endif
+    if dpdk_conf.has('RTE_RAW_SKELETON')
+        test_args += ['--vdev=rawdev_skeleton0']
+    endif
+    test_args += ['-a', '0000:00:00.0']
+    test('telemetry_all', find_program('test_telemetry.sh'),
+            args: test_args,
+            timeout : timeout_seconds_fast,
+            is_parallel : false,
+            suite : 'fast-tests')
+endif
diff --git a/app/test/test_telemetry.sh b/app/test/suites/test_telemetry.sh
similarity index 100%
rename from app/test/test_telemetry.sh
rename to app/test/suites/test_telemetry.sh
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v5 07/10] app/test: add debug test suite
  2023-08-15 15:10 ` [PATCH v5 00/10] " Bruce Richardson
                     ` (5 preceding siblings ...)
  2023-08-15 15:10   ` [PATCH v5 06/10] app/test: add test case for scripted telemetry commands Bruce Richardson
@ 2023-08-15 15:10   ` Bruce Richardson
  2023-08-15 15:10   ` [PATCH v5 08/10] examples/l3fwd: make eventdev an optional dependency Bruce Richardson
                     ` (2 subsequent siblings)
  9 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-15 15:10 UTC (permalink / raw)
  To: dev; +Cc: ci, Bruce Richardson

Add in for all platforms, the suite of tests to dump out the structure
information available. Since the commands are defined in commands.c on
all OS's, do not limit their presence to just the non-windows OS's.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 app/test/suites/meson.build | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/app/test/suites/meson.build b/app/test/suites/meson.build
index 2b428847c6..8cf498ac52 100644
--- a/app/test/suites/meson.build
+++ b/app/test/suites/meson.build
@@ -102,3 +102,23 @@ if not is_windows and dpdk_conf.has('RTE_LIB_TELEMETRY')
             is_parallel : false,
             suite : 'fast-tests')
 endif
+
+# dump tests are defined in commands.c, and not easily extractable
+dump_test_names = [
+        'dump_devargs',
+        'dump_log_types',
+        'dump_malloc_heaps',
+        'dump_malloc_stats',
+        'dump_mempool',
+        'dump_memzone',
+        'dump_physmem',
+        'dump_ring',
+        'dump_struct_sizes',
+]
+foreach arg : dump_test_names
+    test(arg, dpdk_test,
+                env : ['DPDK_TEST=' + arg],
+                timeout : timeout_seconds_fast,
+                is_parallel : false,
+                suite : 'debug-tests')
+endforeach
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v5 08/10] examples/l3fwd: make eventdev an optional dependency
  2023-08-15 15:10 ` [PATCH v5 00/10] " Bruce Richardson
                     ` (6 preceding siblings ...)
  2023-08-15 15:10   ` [PATCH v5 07/10] app/test: add debug test suite Bruce Richardson
@ 2023-08-15 15:10   ` Bruce Richardson
  2023-08-15 15:10   ` [PATCH v5 09/10] build: make most device classes optional Bruce Richardson
  2023-08-15 15:12   ` [PATCH v5 10/10] build: expand list of optional libraries Bruce Richardson
  9 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-15 15:10 UTC (permalink / raw)
  To: dev; +Cc: ci, Bruce Richardson, Morten Brørup

With l3fwd being a very commonly used example app, and built as part of
our CI tests, we need to ensure it's buildable with just about all
supported DPDK configurations.  To enable l3fwd application to be built
when the eventdev library is disabled, we need to compile in the
eventdev support conditionally.  Thankfully, the eventdev support is
pretty self-contained, with only the main.c file having more than a
couple of ifdefs.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
---
 examples/l3fwd/l3fwd_em.c                  |  2 +
 examples/l3fwd/l3fwd_em_hlm.h              |  2 +
 examples/l3fwd/l3fwd_event.c               |  2 +
 examples/l3fwd/l3fwd_event.h               |  7 ++-
 examples/l3fwd/l3fwd_event_generic.c       |  2 +
 examples/l3fwd/l3fwd_event_internal_port.c |  2 +
 examples/l3fwd/l3fwd_fib.c                 |  2 +
 examples/l3fwd/l3fwd_lpm.c                 |  2 +
 examples/l3fwd/main.c                      | 65 +++++++++++++++++-----
 examples/l3fwd/meson.build                 |  5 +-
 10 files changed, 74 insertions(+), 17 deletions(-)

diff --git a/examples/l3fwd/l3fwd_em.c b/examples/l3fwd/l3fwd_em.c
index 476ac0c54f..40e102b38a 100644
--- a/examples/l3fwd/l3fwd_em.c
+++ b/examples/l3fwd/l3fwd_em.c
@@ -663,6 +663,7 @@ em_main_loop(__rte_unused void *dummy)
 	return 0;
 }
 
+#ifdef RTE_LIB_EVENTDEV
 static __rte_always_inline void
 em_event_loop_single(struct l3fwd_event_resources *evt_rsrc,
 		const uint8_t flags)
@@ -959,6 +960,7 @@ em_event_main_loop_tx_q_burst_vector(__rte_unused void *dummy)
 	em_event_loop_vector(evt_rsrc, L3FWD_EVENT_TX_ENQ);
 	return 0;
 }
+#endif
 
 /* Initialize exact match (hash) parameters. 8< */
 void
diff --git a/examples/l3fwd/l3fwd_em_hlm.h b/examples/l3fwd/l3fwd_em_hlm.h
index 2e11eefad7..31cda9ddc1 100644
--- a/examples/l3fwd/l3fwd_em_hlm.h
+++ b/examples/l3fwd/l3fwd_em_hlm.h
@@ -255,6 +255,7 @@ l3fwd_em_send_packets(int nb_rx, struct rte_mbuf **pkts_burst, uint16_t portid,
 	send_packets_multi(qconf, pkts_burst, dst_port, nb_rx);
 }
 
+#ifdef RTE_LIB_EVENTDEV
 /*
  * Buffer optimized handling of events, invoked
  * from main_loop.
@@ -347,5 +348,6 @@ l3fwd_em_process_event_vector(struct rte_event_vector *vec,
 
 	process_event_vector(vec, dst_port);
 }
+#endif /* RTE_LIB_EVENTDEV */
 
 #endif /* __L3FWD_EM_HLM_H__ */
diff --git a/examples/l3fwd/l3fwd_event.c b/examples/l3fwd/l3fwd_event.c
index 32906ab08d..d72a4138cc 100644
--- a/examples/l3fwd/l3fwd_event.c
+++ b/examples/l3fwd/l3fwd_event.c
@@ -2,6 +2,7 @@
  * Copyright(C) 2019 Marvell International Ltd.
  */
 
+#ifdef RTE_LIB_EVENTDEV
 #include <stdbool.h>
 #include <getopt.h>
 
@@ -341,3 +342,4 @@ l3fwd_event_worker_cleanup(uint8_t event_d_id, uint8_t event_p_id,
 	rte_event_port_quiesce(event_d_id, event_p_id, l3fwd_event_port_flush,
 			       NULL);
 }
+#endif /* #ifdef RTE_LIB_EVENTDEV */
diff --git a/examples/l3fwd/l3fwd_event.h b/examples/l3fwd/l3fwd_event.h
index e21817c36b..1fd6fe4a78 100644
--- a/examples/l3fwd/l3fwd_event.h
+++ b/examples/l3fwd/l3fwd_event.h
@@ -6,11 +6,13 @@
 #define __L3FWD_EVENTDEV_H__
 
 #include <rte_common.h>
+#include <rte_service.h>
+#include <rte_spinlock.h>
+
+#ifdef RTE_LIB_EVENTDEV
 #include <rte_eventdev.h>
 #include <rte_event_eth_rx_adapter.h>
 #include <rte_event_eth_tx_adapter.h>
-#include <rte_service.h>
-#include <rte_spinlock.h>
 
 #include "l3fwd.h"
 
@@ -164,4 +166,5 @@ void l3fwd_event_worker_cleanup(uint8_t event_d_id, uint8_t event_p_id,
 				struct rte_event events[], uint16_t nb_enq,
 				uint16_t nb_deq, uint8_t is_vector);
 
+#endif /* ifdef RTE_LIB_EVENTDEV */
 #endif /* __L3FWD_EVENTDEV_H__ */
diff --git a/examples/l3fwd/l3fwd_event_generic.c b/examples/l3fwd/l3fwd_event_generic.c
index c80573fc58..ddb6e5c38d 100644
--- a/examples/l3fwd/l3fwd_event_generic.c
+++ b/examples/l3fwd/l3fwd_event_generic.c
@@ -2,6 +2,7 @@
  * Copyright(C) 2019 Marvell International Ltd.
  */
 
+#ifdef RTE_LIB_EVENTDEV
 #include <stdbool.h>
 
 #include "l3fwd.h"
@@ -309,3 +310,4 @@ l3fwd_event_set_generic_ops(struct l3fwd_event_setup_ops *ops)
 	ops->event_port_setup = l3fwd_event_port_setup_generic;
 	ops->adapter_setup = l3fwd_rx_tx_adapter_setup_generic;
 }
+#endif /* RTE_LIB_EVENTDEV */
diff --git a/examples/l3fwd/l3fwd_event_internal_port.c b/examples/l3fwd/l3fwd_event_internal_port.c
index 32cf657148..cb49a8b9fa 100644
--- a/examples/l3fwd/l3fwd_event_internal_port.c
+++ b/examples/l3fwd/l3fwd_event_internal_port.c
@@ -2,6 +2,7 @@
  * Copyright(C) 2019 Marvell International Ltd.
  */
 
+#ifdef RTE_LIB_EVENTDEV
 #include <stdbool.h>
 
 #include "l3fwd.h"
@@ -311,3 +312,4 @@ l3fwd_event_set_internal_port_ops(struct l3fwd_event_setup_ops *ops)
 	ops->event_port_setup = l3fwd_event_port_setup_internal_port;
 	ops->adapter_setup = l3fwd_rx_tx_adapter_setup_internal_port;
 }
+#endif /* RTE_LIB_EVENTDEV */
diff --git a/examples/l3fwd/l3fwd_fib.c b/examples/l3fwd/l3fwd_fib.c
index 18398492ae..6a21984415 100644
--- a/examples/l3fwd/l3fwd_fib.c
+++ b/examples/l3fwd/l3fwd_fib.c
@@ -253,6 +253,7 @@ fib_main_loop(__rte_unused void *dummy)
 	return 0;
 }
 
+#ifdef RTE_LIB_EVENTDEV
 /* One eventdev loop for single and burst using fib. */
 static __rte_always_inline void
 fib_event_loop(struct l3fwd_event_resources *evt_rsrc,
@@ -635,6 +636,7 @@ fib_event_main_loop_tx_q_burst_vector(__rte_unused void *dummy)
 	fib_event_loop_vector(evt_rsrc, L3FWD_EVENT_TX_ENQ);
 	return 0;
 }
+#endif
 
 /* Function to setup fib. 8< */
 void
diff --git a/examples/l3fwd/l3fwd_lpm.c b/examples/l3fwd/l3fwd_lpm.c
index 4ac1925c84..a484a33089 100644
--- a/examples/l3fwd/l3fwd_lpm.c
+++ b/examples/l3fwd/l3fwd_lpm.c
@@ -226,6 +226,7 @@ lpm_main_loop(__rte_unused void *dummy)
 	return 0;
 }
 
+#ifdef RTE_LIB_EVENTDEV
 static __rte_always_inline uint16_t
 lpm_process_event_pkt(const struct lcore_conf *lconf, struct rte_mbuf *mbuf)
 {
@@ -554,6 +555,7 @@ lpm_event_main_loop_tx_q_burst_vector(__rte_unused void *dummy)
 	lpm_event_loop_vector(evt_rsrc, L3FWD_EVENT_TX_ENQ);
 	return 0;
 }
+#endif
 
 void
 setup_lpm(const int socketid)
diff --git a/examples/l3fwd/main.c b/examples/l3fwd/main.c
index a4f061537e..6063eb1399 100644
--- a/examples/l3fwd/main.c
+++ b/examples/l3fwd/main.c
@@ -135,8 +135,10 @@ static struct rte_eth_conf port_conf = {
 
 uint32_t max_pkt_len;
 
-static struct rte_mempool *pktmbuf_pool[RTE_MAX_ETHPORTS][NB_SOCKETS];
+#ifdef RTE_LIB_EVENTDEV
 static struct rte_mempool *vector_pool[RTE_MAX_ETHPORTS];
+#endif
+static struct rte_mempool *pktmbuf_pool[RTE_MAX_ETHPORTS][NB_SOCKETS];
 static uint8_t lkp_per_socket[NB_SOCKETS];
 
 struct l3fwd_lkp_mode {
@@ -398,8 +400,10 @@ print_usage(const char *prgname)
 		" [--parse-ptype]"
 		" [--per-port-pool]"
 		" [--mode]"
+#ifdef RTE_LIB_EVENTDEV
 		" [--eventq-sched]"
 		" [--event-vector [--event-vector-size SIZE] [--event-vector-tmo NS]]"
+#endif
 		" [-E]"
 		" [-L]\n\n"
 
@@ -422,6 +426,7 @@ print_usage(const char *prgname)
 		"  --per-port-pool: Use separate buffer pool per port\n"
 		"  --mode: Packet transfer mode for I/O, poll or eventdev\n"
 		"          Default mode = poll\n"
+#ifdef RTE_LIB_EVENTDEV
 		"  --eventq-sched: Event queue synchronization method\n"
 		"                  ordered, atomic or parallel.\n"
 		"                  Default: atomic\n"
@@ -432,6 +437,7 @@ print_usage(const char *prgname)
 		"  --event-vector:  Enable event vectorization.\n"
 		"  --event-vector-size: Max vector size if event vectorization is enabled.\n"
 		"  --event-vector-tmo: Max timeout to form vector in nanoseconds if event vectorization is enabled\n"
+#endif
 		"  -E : Enable exact match, legacy flag please use --lookup=em instead\n"
 		"  -L : Enable longest prefix match, legacy flag please use --lookup=lpm instead\n"
 		"  --rule_ipv4=FILE: Specify the ipv4 rules entries file.\n"
@@ -559,14 +565,16 @@ parse_eth_dest(const char *optarg)
 }
 
 static void
-parse_mode(const char *optarg)
+parse_mode(const char *optarg __rte_unused)
 {
+#ifdef RTE_LIB_EVENTDEV
 	struct l3fwd_event_resources *evt_rsrc = l3fwd_get_eventdev_rsrc();
 
 	if (!strcmp(optarg, "poll"))
 		evt_rsrc->enabled = false;
 	else if (!strcmp(optarg, "eventdev"))
 		evt_rsrc->enabled = true;
+#endif
 }
 
 static void
@@ -601,6 +609,7 @@ parse_queue_size(const char *queue_size_arg, uint16_t *queue_size, int rx)
 	*queue_size = value;
 }
 
+#ifdef RTE_LIB_EVENTDEV
 static void
 parse_eventq_sched(const char *optarg)
 {
@@ -631,6 +640,7 @@ parse_event_eth_rx_queues(const char *eth_rx_queues)
 
 	evt_rsrc->eth_rx_queues = num_eth_rx_queues;
 }
+#endif
 
 static int
 parse_lookup(const char *optarg)
@@ -756,9 +766,11 @@ parse_args(int argc, char **argv)
 	int option_index;
 	char *prgname = argv[0];
 	uint8_t lcore_params = 0;
+#ifdef RTE_LIB_EVENTDEV
 	uint8_t eventq_sched = 0;
 	uint8_t eth_rx_q = 0;
 	struct l3fwd_event_resources *evt_rsrc = l3fwd_get_eventdev_rsrc();
+#endif
 
 	argvopt = argv;
 
@@ -850,6 +862,7 @@ parse_args(int argc, char **argv)
 			parse_mode(optarg);
 			break;
 
+#ifdef RTE_LIB_EVENTDEV
 		case CMD_LINE_OPT_EVENTQ_SYNC_NUM:
 			parse_eventq_sched(optarg);
 			eventq_sched = 1;
@@ -860,6 +873,20 @@ parse_args(int argc, char **argv)
 			eth_rx_q = 1;
 			break;
 
+		case CMD_LINE_OPT_ENABLE_VECTOR_NUM:
+			printf("event vectorization is enabled\n");
+			evt_rsrc->vector_enabled = 1;
+			break;
+
+		case CMD_LINE_OPT_VECTOR_SIZE_NUM:
+			evt_rsrc->vector_size = strtol(optarg, NULL, 10);
+			break;
+
+		case CMD_LINE_OPT_VECTOR_TMO_NS_NUM:
+			evt_rsrc->vector_tmo_ns = strtoull(optarg, NULL, 10);
+			break;
+#endif
+
 		case CMD_LINE_OPT_LOOKUP_NUM:
 			if (lookup_mode != L3FWD_LOOKUP_DEFAULT) {
 				fprintf(stderr, "Only one lookup mode is allowed at a time!\n");
@@ -875,16 +902,6 @@ parse_args(int argc, char **argv)
 				return -1;
 			break;
 
-		case CMD_LINE_OPT_ENABLE_VECTOR_NUM:
-			printf("event vectorization is enabled\n");
-			evt_rsrc->vector_enabled = 1;
-			break;
-		case CMD_LINE_OPT_VECTOR_SIZE_NUM:
-			evt_rsrc->vector_size = strtol(optarg, NULL, 10);
-			break;
-		case CMD_LINE_OPT_VECTOR_TMO_NS_NUM:
-			evt_rsrc->vector_tmo_ns = strtoull(optarg, NULL, 10);
-			break;
 		case CMD_LINE_OPT_RULE_IPV4_NUM:
 			l3fwd_set_rule_ipv4_name(optarg);
 			break;
@@ -900,6 +917,8 @@ parse_args(int argc, char **argv)
 		}
 	}
 
+	RTE_SET_USED(lcore_params); /* needed if no eventdev block */
+#ifdef RTE_LIB_EVENTDEV
 	if (evt_rsrc->enabled && lcore_params) {
 		fprintf(stderr, "lcore config is not valid when event mode is selected\n");
 		return -1;
@@ -927,6 +946,7 @@ parse_args(int argc, char **argv)
 			"vector timeout set to default (%" PRIu64 " ns)\n",
 			evt_rsrc->vector_tmo_ns);
 	}
+#endif
 
 	/*
 	 * Nothing is selected, pick longest-prefix match
@@ -962,7 +982,9 @@ print_ethaddr(const char *name, const struct rte_ether_addr *eth_addr)
 int
 init_mem(uint16_t portid, unsigned int nb_mbuf)
 {
+#ifdef RTE_LIB_EVENTDEV
 	struct l3fwd_event_resources *evt_rsrc = l3fwd_get_eventdev_rsrc();
+#endif
 	struct lcore_conf *qconf;
 	int socketid;
 	unsigned lcore_id;
@@ -1007,6 +1029,7 @@ init_mem(uint16_t portid, unsigned int nb_mbuf)
 			}
 		}
 
+#ifdef RTE_LIB_EVENTDEV
 		if (evt_rsrc->vector_enabled && vector_pool[portid] == NULL) {
 			unsigned int nb_vec;
 
@@ -1025,6 +1048,7 @@ init_mem(uint16_t portid, unsigned int nb_mbuf)
 				printf("Allocated vector pool for port %d\n",
 				       portid);
 		}
+#endif
 
 		qconf = &lcore_conf[lcore_id];
 		qconf->ipv4_lookup_struct =
@@ -1406,6 +1430,7 @@ l3fwd_service_enable(uint32_t service_id)
 	return 0;
 }
 
+#ifdef RTE_LIB_EVENTDEV
 static void
 l3fwd_event_service_setup(void)
 {
@@ -1458,16 +1483,20 @@ l3fwd_event_service_setup(void)
 		l3fwd_service_enable(service_id);
 	}
 }
+#endif
 
 int
 main(int argc, char **argv)
 {
+#ifdef RTE_LIB_EVENTDEV
 	struct l3fwd_event_resources *evt_rsrc;
+	int i;
+#endif
 	struct lcore_conf *qconf;
 	uint16_t queueid, portid;
 	unsigned int lcore_id;
 	uint8_t queue;
-	int i, ret;
+	int ret;
 
 	/* init EAL */
 	ret = rte_eal_init(argc, argv);
@@ -1487,7 +1516,9 @@ main(int argc, char **argv)
 		*(uint64_t *)(val_eth + portid) = dest_eth_addr[portid];
 	}
 
+#ifdef RTE_LIB_EVENTDEV
 	evt_rsrc = l3fwd_get_eventdev_rsrc();
+#endif
 	/* parse application arguments (after the EAL ones) */
 	ret = parse_args(argc, argv);
 	if (ret < 0)
@@ -1499,6 +1530,7 @@ main(int argc, char **argv)
 	/* Add the config file rules */
 	l3fwd_lkp.read_config_files();
 
+#ifdef RTE_LIB_EVENTDEV
 	evt_rsrc->per_port_pool = per_port_pool;
 	evt_rsrc->pkt_pool = pktmbuf_pool;
 	evt_rsrc->vec_pool = vector_pool;
@@ -1514,6 +1546,7 @@ main(int argc, char **argv)
 			l3fwd_lkp.main_loop = evt_rsrc->ops.lpm_event_loop;
 		l3fwd_event_service_setup();
 	} else
+#endif
 		l3fwd_poll_resource_setup();
 
 	/* start ports */
@@ -1562,6 +1595,8 @@ main(int argc, char **argv)
 	ret = 0;
 	/* launch per-lcore init on every lcore */
 	rte_eal_mp_remote_launch(l3fwd_lkp.main_loop, NULL, CALL_MAIN);
+
+#ifdef RTE_LIB_EVENTDEV
 	if (evt_rsrc->enabled) {
 		for (i = 0; i < evt_rsrc->rx_adptr.nb_rx_adptr; i++)
 			rte_event_eth_rx_adapter_stop(
@@ -1589,7 +1624,9 @@ main(int argc, char **argv)
 		rte_event_dev_stop(evt_rsrc->event_d_id);
 		rte_event_dev_close(evt_rsrc->event_d_id);
 
-	} else {
+	} else
+#endif
+	{
 		rte_eal_mp_wait_lcore();
 
 		RTE_ETH_FOREACH_DEV(portid) {
diff --git a/examples/l3fwd/meson.build b/examples/l3fwd/meson.build
index b40244a941..c25de77bba 100644
--- a/examples/l3fwd/meson.build
+++ b/examples/l3fwd/meson.build
@@ -7,7 +7,7 @@
 # DPDK instance, use 'make'
 
 allow_experimental_apis = true
-deps += ['acl', 'hash', 'lpm', 'fib', 'eventdev']
+deps += ['acl', 'hash', 'lpm', 'fib']
 sources = files(
         'l3fwd_acl.c',
         'l3fwd_em.c',
@@ -18,3 +18,6 @@ sources = files(
         'l3fwd_lpm.c',
         'main.c',
 )
+if dpdk_conf.has('RTE_LIB_EVENTDEV')
+    deps += 'eventdev'
+endif
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v5 09/10] build: make most device classes optional
  2023-08-15 15:10 ` [PATCH v5 00/10] " Bruce Richardson
                     ` (7 preceding siblings ...)
  2023-08-15 15:10   ` [PATCH v5 08/10] examples/l3fwd: make eventdev an optional dependency Bruce Richardson
@ 2023-08-15 15:10   ` Bruce Richardson
  2023-08-15 15:12   ` [PATCH v5 10/10] build: expand list of optional libraries Bruce Richardson
  9 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-15 15:10 UTC (permalink / raw)
  To: dev; +Cc: ci, Bruce Richardson, Morten Brørup

Apart from ethdev and cryptodev, which have lots of components and tests
which depend on them, we can make the device class libraries optional
without too much work.

This patch marks:
* bbdev,
* compressdev,
* dmadev,
* eventdev,
* mldev,
* rawdev,
* regexdev
optional, and ensures that DPDK - including tests - can be built with
these components disabled.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
---
 lib/meson.build | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/lib/meson.build b/lib/meson.build
index 92cbd6bb66..1a78c8c984 100644
--- a/lib/meson.build
+++ b/lib/meson.build
@@ -67,8 +67,12 @@ libraries = [
 ]
 
 optional_libs = [
+        'bbdev',
         'bitratestats',
         'cfgfile',
+        'compressdev',
+        'dmadev',
+        'eventdev',
         'gpudev',
         'graph',
         'gro',
@@ -76,11 +80,14 @@ optional_libs = [
         'jobstats',
         'latencystats',
         'metrics',
+        'mldev',
         'node',
         'pdump',
         'pipeline',
         'port',
         'power',
+        'rawdev',
+        'regexdev',
         'table',
         'vhost',
 ]
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v5 10/10] build: expand list of optional libraries
  2023-08-15 15:10 ` [PATCH v5 00/10] " Bruce Richardson
                     ` (8 preceding siblings ...)
  2023-08-15 15:10   ` [PATCH v5 09/10] build: make most device classes optional Bruce Richardson
@ 2023-08-15 15:12   ` Bruce Richardson
  9 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-15 15:12 UTC (permalink / raw)
  To: dev; +Cc: ci, Bruce Richardson, Morten Brørup

With the unit test build now with individual per-file dependencies, we
can more easily expand the list of optional libraries. Add 8 new
libraries to the optional list.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
---
 lib/meson.build | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/lib/meson.build b/lib/meson.build
index 1a78c8c984..0c9dc8b3b6 100644
--- a/lib/meson.build
+++ b/lib/meson.build
@@ -69,25 +69,33 @@ libraries = [
 optional_libs = [
         'bbdev',
         'bitratestats',
+        'bpf',
         'cfgfile',
         'compressdev',
+        'distributor',
         'dmadev',
+        'efd',
         'eventdev',
         'gpudev',
         'graph',
         'gro',
         'gso',
+        'ip_frag',
         'jobstats',
         'latencystats',
+        'member',
         'metrics',
         'mldev',
         'node',
+        'pcapng',
         'pdump',
         'pipeline',
         'port',
         'power',
         'rawdev',
         'regexdev',
+        'reorder',
+        'sched',
         'table',
         'vhost',
 ]
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* Re: [PATCH v5 04/10] app/test: build using per-file dependency matrix
  2023-08-15 15:10   ` [PATCH v5 04/10] app/test: build using per-file dependency matrix Bruce Richardson
@ 2023-08-15 19:05     ` Patrick Robb
  2023-08-16 10:56       ` Bruce Richardson
  2023-08-16 12:55       ` Bruce Richardson
  0 siblings, 2 replies; 93+ messages in thread
From: Patrick Robb @ 2023-08-15 19:05 UTC (permalink / raw)
  To: Bruce Richardson; +Cc: dev, ci, Morten Brørup

[-- Attachment #1: Type: text/plain, Size: 1391 bytes --]

Adam from our team just raised something important about this patch and UNH
CI which I missed during the RFC discussion.
Presently, eal_flags_file_prefix_autotest fails on arm tx2 systems (arm
people are aware of this, they think it might be a memory leak from mlx5
driver), so at their request we are disabling this unit test (on arm only)
for them for the time being. This failure was discovered when we were
initially standing up arm unit testing on tx2 servers earlier this year -
previously there was no coverage for this. We do this filtering by
going through the /app/test/meson.build file and resetting
the eal_flags_file_prefix_autotest line to an empty string. This process is
broken by your patch series. Again, I'm sorry I didn't catch this concern
when discussing it during the RFC. This is why you are not getting unit
test results for arm64, we can't run unit tests given the changes in this
patch.

Your refactor likely means that going forward, we will no longer be able to
tailor the fast test suite (or any suite) per vendor request. That might
actually be a good thing. In any case, if this is merged now, it is going
to put the tree in a state where our CI doesn't run any unit testing on
ARM. I don't know how close this patch series is to possibly hitting
mainline, but, if possible, can that be delayed, pending us figuring out
how we will respond to this situation?

[-- Attachment #2: Type: text/html, Size: 1545 bytes --]

^ permalink raw reply	[flat|nested] 93+ messages in thread

* Re: [PATCH v5 04/10] app/test: build using per-file dependency matrix
  2023-08-15 19:05     ` Patrick Robb
@ 2023-08-16 10:56       ` Bruce Richardson
  2023-08-16 12:55       ` Bruce Richardson
  1 sibling, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-16 10:56 UTC (permalink / raw)
  To: Patrick Robb; +Cc: dev, ci, Morten Brørup

On Tue, Aug 15, 2023 at 03:05:08PM -0400, Patrick Robb wrote:
>    Adam from our team just raised something important about this patch and
>    UNH CI which I missed during the RFC discussion.
>    Presently, eal_flags_file_prefix_autotest fails on arm tx2 systems (arm
>    people are aware of this, they think it might be a memory leak from
>    mlx5 driver), so at their request we are disabling this unit test (on
>    arm only) for them for the time being. This failure was discovered when
>    we were initially standing up arm unit testing on tx2 servers earlier
>    this year - previously there was no coverage for this. We do this
>    filtering by going through the /app/test/meson.build file and resetting
>    the eal_flags_file_prefix_autotest line to an empty string. This
>    process is broken by your patch series. Again, I'm sorry I didn't catch
>    this concern when discussing it during the RFC. This is why you are not
>    getting unit test results for arm64, we can't run unit tests given the
>    changes in this patch.

If this is a (semi)permanent issue, would it not be better to actually
patch in the disabling of this test into the test sources, i.e. if ARM64
have the test return TEST_SKIPPED?

>    Your refactor likely means that going forward, we will no longer be
>    able to tailor the fast test suite (or any suite) per vendor request.
>    That might actually be a good thing. In any case, if this is merged
>    now, it is going to put the tree in a state where our CI doesn't run
>    any unit testing on ARM. I don't know how close this patch series is to
>    possibly hitting mainline, but, if possible, can that be delayed,
>    pending us figuring out how we will respond to this situation?

There is at least one more issue I am aware of with this set - the failing
mempool tests - which needs to be fixed, so there will be at least one more
version.

If this is likely a common occurrance where we need to disable one
particular test for a particular platform, would it be worth designing such
a feature into the rework. For example, it should be fairly easy to have
the testname-extraction script read a blocklist from the environment, and
omit informing meson of those test cases. Would that work?

/Bruce

^ permalink raw reply	[flat|nested] 93+ messages in thread

* Re: [PATCH v5 05/10] app/test: define unit tests suites based on test macros
  2023-08-15 15:10   ` [PATCH v5 05/10] app/test: define unit tests suites based on test macros Bruce Richardson
@ 2023-08-16 11:02     ` Bruce Richardson
  2023-08-16 11:15       ` David Marchand
  2023-08-16 14:57     ` David Marchand
  1 sibling, 1 reply; 93+ messages in thread
From: Bruce Richardson @ 2023-08-16 11:02 UTC (permalink / raw)
  To: dev; +Cc: ci, Morten Brørup

On Tue, Aug 15, 2023 at 04:10:49PM +0100, Bruce Richardson wrote:
> Rather than having the test suites listed out in the meson.build files
> and having to have them enabled/disabled selectively based on what libs
> are being built, pull the tests to run from the source files which were
> added to the build.
> 
> Most test suites require no additional info other than the list of test
> names in the suite. However the fast-test are special that they have
> additional parameters associated with them. This requires some
> additional work in the test extraction script and in the meson.build
> file for processing the output.
> 
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> Acked-by: Morten Brørup <mb@smartsharesystems.com>
> ---
>  app/meson.build                               |  7 +-
>  app/test/suites/meson.build                   | 74 +++++++++++++++++++
>  buildtools/get-test-suites.py                 | 33 +++++++++
>  .../has-hugepages.py                          |  0
>  buildtools/meson.build                        |  2 +
>  5 files changed, 115 insertions(+), 1 deletion(-)
>  create mode 100644 app/test/suites/meson.build
>  create mode 100644 buildtools/get-test-suites.py
>  rename app/test/has_hugepage.py => buildtools/has-hugepages.py (100%)
> 
> diff --git a/app/meson.build b/app/meson.build
> index 0d8b618e7f..c14dc80892 100644
> --- a/app/meson.build
> +++ b/app/meson.build
> @@ -101,7 +101,7 @@ foreach app:apps
>          link_libs = dpdk_static_libraries + dpdk_drivers
>      endif
>  
> -    executable('dpdk-' + name,
> +    exec = executable('dpdk-' + name,
>              sources,
>              c_args: cflags,
>              link_args: ldflags,
> @@ -110,4 +110,9 @@ foreach app:apps
>              include_directories: includes,
>              install_rpath: join_paths(get_option('prefix'), driver_install_path),
>              install: true)
> +    if name == 'test'
> +        dpdk_test = exec
> +        autotest_sources = sources
> +        subdir('test/suites')  # define the pre-canned test suites
> +    endif
>  endforeach
> diff --git a/app/test/suites/meson.build b/app/test/suites/meson.build
> new file mode 100644
> index 0000000000..ec74d8adf2
> --- /dev/null
> +++ b/app/test/suites/meson.build
> @@ -0,0 +1,74 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2023 Intel Corporation
> +
> +# some perf tests (eg: memcpy perf autotest)take very long
> +# to complete, so timeout to 10 minutes
> +timeout_seconds = 600
> +timeout_seconds_fast = 10
> +
> +test_no_huge_args = ['--no-huge', '-m', '2048']
> +has_hugepage = run_command(has_hugepages_cmd, check: true).stdout().strip() != '0'
> +message('hugepage availability: @0@'.format(has_hugepage))
> +
> +# process source files to determine the different unit test suites
> +# - fast_tests
> +# - perf_tests
> +# - driver_tests
> +test_suites = run_command(get_test_suites_cmd, autotest_sources,
> +         check: true).stdout().strip().split()
> +foreach suite:test_suites
> +    # simple cases - tests without parameters or special handling
> +    suite = suite.split('=')
> +    suite_name = suite[0]
> +    suite_tests = suite[1].split(',')
> +    if suite_name != 'fast-tests'
> +        # simple cases - tests without parameters or special handling
> +        foreach t: suite_tests
> +            test(t, dpdk_test,
> +                    env: ['DPDK_TEST=' + t],
> +                    timeout: timeout_seconds,
> +                    is_parallel: false,
> +                    suite: suite_name)
> +        endforeach
> +    else
> +    # special fast-test handling here
> +        foreach t: suite_tests
> +            params = t.split(':')
> +            test_name = params[0]
> +            nohuge = params[1] == 'true'
> +            asan = params[2] == 'true'
> +
> +            test_args = []
> +            if nohuge
> +                test_args += test_no_huge_args
> +            elif not has_hugepage
> +                continue  #skip this tests
> +            endif
> +            if not asan and (get_option('b_sanitize') == 'address'
> +                    or get_option('b_sanitize') == 'address,undefined')
> +                continue  # skip this test
> +            endif
> +
> +            if get_option('default_library') == 'shared'
> +                test_args += ['-d', dpdk_drivers_build_dir]
> +            endif

These lines here seem to be exposing a bug in the mempool unit tests for
shared builds, which is why we have a CI failure.

The mempool unit tests use the mempool "create_empty" API, and then call
the populate APIs to finish setting up the mempool. However, the
create_empty API does not explicitly configure a particular set of mempool
ops for the new mempool, leaving the ops field at 0. This means that unless
the app explicitly sets other ops, the mempool will use the ops from
whatever driver registers itself first. This occurs even when the driver is
unsuitable, e.g. on my Intel system, the dpaa2 is first on the list,
leading to failures in setting up and using the mempool.

In v6 of this set, I intend to fix this, by changing the create empty API
to explicitly set the ring driver as default for new mempools. It's the
most likely to work for common cases.

/Bruce

^ permalink raw reply	[flat|nested] 93+ messages in thread

* Re: [PATCH v5 05/10] app/test: define unit tests suites based on test macros
  2023-08-16 11:02     ` Bruce Richardson
@ 2023-08-16 11:15       ` David Marchand
  2023-08-16 11:40         ` David Marchand
  0 siblings, 1 reply; 93+ messages in thread
From: David Marchand @ 2023-08-16 11:15 UTC (permalink / raw)
  To: Bruce Richardson; +Cc: dev, ci, Morten Brørup

On Wed, Aug 16, 2023 at 1:02 PM Bruce Richardson
<bruce.richardson@intel.com> wrote:
> These lines here seem to be exposing a bug in the mempool unit tests for
> shared builds, which is why we have a CI failure.
>
> The mempool unit tests use the mempool "create_empty" API, and then call
> the populate APIs to finish setting up the mempool. However, the
> create_empty API does not explicitly configure a particular set of mempool
> ops for the new mempool, leaving the ops field at 0. This means that unless
> the app explicitly sets other ops, the mempool will use the ops from
> whatever driver registers itself first. This occurs even when the driver is
> unsuitable, e.g. on my Intel system, the dpaa2 is first on the list,
> leading to failures in setting up and using the mempool.

Hum, it sounds like a bug to me.
The dpaa2 driver should not be registered as the default (or here,
default platform) mempool.
Other mempool drivers ensure that required hw is available, I guess
dpaa2 is missing some check.


>
> In v6 of this set, I intend to fix this, by changing the create empty API
> to explicitly set the ring driver as default for new mempools. It's the
> most likely to work for common cases.


-- 
David Marchand


^ permalink raw reply	[flat|nested] 93+ messages in thread

* Re: [PATCH v5 05/10] app/test: define unit tests suites based on test macros
  2023-08-16 11:15       ` David Marchand
@ 2023-08-16 11:40         ` David Marchand
  2023-08-16 12:33           ` Bruce Richardson
  0 siblings, 1 reply; 93+ messages in thread
From: David Marchand @ 2023-08-16 11:40 UTC (permalink / raw)
  To: Bruce Richardson; +Cc: dev, ci, Morten Brørup, Olivier Matz

On Wed, Aug 16, 2023 at 1:15 PM David Marchand
<david.marchand@redhat.com> wrote:
>
> On Wed, Aug 16, 2023 at 1:02 PM Bruce Richardson
> <bruce.richardson@intel.com> wrote:
> > These lines here seem to be exposing a bug in the mempool unit tests for
> > shared builds, which is why we have a CI failure.
> >
> > The mempool unit tests use the mempool "create_empty" API, and then call
> > the populate APIs to finish setting up the mempool. However, the
> > create_empty API does not explicitly configure a particular set of mempool
> > ops for the new mempool, leaving the ops field at 0. This means that unless
> > the app explicitly sets other ops, the mempool will use the ops from
> > whatever driver registers itself first. This occurs even when the driver is
> > unsuitable, e.g. on my Intel system, the dpaa2 is first on the list,
> > leading to failures in setting up and using the mempool.
>
> Hum, it sounds like a bug to me.
> The dpaa2 driver should not be registered as the default (or here,
> default platform) mempool.
> Other mempool drivers ensure that required hw is available, I guess
> dpaa2 is missing some check.

Ok, re-reading your last comment, I agree it looks like an issue in
the unit test itself.
Copying Olivier.


-- 
David Marchand


^ permalink raw reply	[flat|nested] 93+ messages in thread

* Re: [PATCH v5 05/10] app/test: define unit tests suites based on test macros
  2023-08-16 11:40         ` David Marchand
@ 2023-08-16 12:33           ` Bruce Richardson
  2023-08-16 13:16             ` Olivier Matz
  0 siblings, 1 reply; 93+ messages in thread
From: Bruce Richardson @ 2023-08-16 12:33 UTC (permalink / raw)
  To: David Marchand; +Cc: dev, ci, Morten Brørup, Olivier Matz

On Wed, Aug 16, 2023 at 01:40:41PM +0200, David Marchand wrote:
> On Wed, Aug 16, 2023 at 1:15 PM David Marchand
> <david.marchand@redhat.com> wrote:
> >
> > On Wed, Aug 16, 2023 at 1:02 PM Bruce Richardson
> > <bruce.richardson@intel.com> wrote:
> > > These lines here seem to be exposing a bug in the mempool unit tests for
> > > shared builds, which is why we have a CI failure.
> > >
> > > The mempool unit tests use the mempool "create_empty" API, and then call
> > > the populate APIs to finish setting up the mempool. However, the
> > > create_empty API does not explicitly configure a particular set of mempool
> > > ops for the new mempool, leaving the ops field at 0. This means that unless
> > > the app explicitly sets other ops, the mempool will use the ops from
> > > whatever driver registers itself first. This occurs even when the driver is
> > > unsuitable, e.g. on my Intel system, the dpaa2 is first on the list,
> > > leading to failures in setting up and using the mempool.
> >
> > Hum, it sounds like a bug to me.
> > The dpaa2 driver should not be registered as the default (or here,
> > default platform) mempool.
> > Other mempool drivers ensure that required hw is available, I guess
> > dpaa2 is missing some check.
> 
> Ok, re-reading your last comment, I agree it looks like an issue in
> the unit test itself.
> Copying Olivier.
> 
No, I think it's not a bug in the test, but rather in the mempool.
Unfortunately, I think the concept of the "default" driver for mempools
seems to exist only when using the mbuf library to create mempools. Even
then, the default mempool is different from what the first entry in the
list is. That's the fundamental issue here, we are using the zero-eth entry
in the ops list, rather than a default one.

/Bruce

^ permalink raw reply	[flat|nested] 93+ messages in thread

* Re: [PATCH v5 04/10] app/test: build using per-file dependency matrix
  2023-08-15 19:05     ` Patrick Robb
  2023-08-16 10:56       ` Bruce Richardson
@ 2023-08-16 12:55       ` Bruce Richardson
  2023-08-16 14:40         ` David Marchand
  1 sibling, 1 reply; 93+ messages in thread
From: Bruce Richardson @ 2023-08-16 12:55 UTC (permalink / raw)
  To: Patrick Robb; +Cc: dev, ci, Morten Brørup

On Tue, Aug 15, 2023 at 03:05:08PM -0400, Patrick Robb wrote:
>    Adam from our team just raised something important about this patch and
>    UNH CI which I missed during the RFC discussion.
>    Presently, eal_flags_file_prefix_autotest fails on arm tx2 systems (arm
>    people are aware of this, they think it might be a memory leak from
>    mlx5 driver), so at their request we are disabling this unit test (on
>    arm only) for them for the time being.

Does the test suite pass if the mlx5 driver is disabled in the build? That
could confirm or refute the suspicion of where the issue is, and also
provide a temporary workaround while this set is merged (possibly including
support for disabling specific tests, as I suggested in my other email).

/Bruce

PS: Are there any other workarounds inside the test/DTS/CI systems that
involve patching sources? If so, it would be good to get a list that we can
work through removing by putting place proper fixes or workarounds, as
changing sources for testing like this blocks future patch acceptance.

^ permalink raw reply	[flat|nested] 93+ messages in thread

* Re: [PATCH v5 05/10] app/test: define unit tests suites based on test macros
  2023-08-16 12:33           ` Bruce Richardson
@ 2023-08-16 13:16             ` Olivier Matz
  2023-08-16 13:35               ` Morten Brørup
  0 siblings, 1 reply; 93+ messages in thread
From: Olivier Matz @ 2023-08-16 13:16 UTC (permalink / raw)
  To: Bruce Richardson; +Cc: David Marchand, dev, ci, Morten Brørup

Hi,

On Wed, Aug 16, 2023 at 01:33:46PM +0100, Bruce Richardson wrote:
> On Wed, Aug 16, 2023 at 01:40:41PM +0200, David Marchand wrote:
> > On Wed, Aug 16, 2023 at 1:15 PM David Marchand
> > <david.marchand@redhat.com> wrote:
> > >
> > > On Wed, Aug 16, 2023 at 1:02 PM Bruce Richardson
> > > <bruce.richardson@intel.com> wrote:
> > > > These lines here seem to be exposing a bug in the mempool unit tests for
> > > > shared builds, which is why we have a CI failure.
> > > >
> > > > The mempool unit tests use the mempool "create_empty" API, and then call
> > > > the populate APIs to finish setting up the mempool. However, the
> > > > create_empty API does not explicitly configure a particular set of mempool
> > > > ops for the new mempool, leaving the ops field at 0. This means that unless
> > > > the app explicitly sets other ops, the mempool will use the ops from
> > > > whatever driver registers itself first. This occurs even when the driver is
> > > > unsuitable, e.g. on my Intel system, the dpaa2 is first on the list,
> > > > leading to failures in setting up and using the mempool.
> > >
> > > Hum, it sounds like a bug to me.
> > > The dpaa2 driver should not be registered as the default (or here,
> > > default platform) mempool.
> > > Other mempool drivers ensure that required hw is available, I guess
> > > dpaa2 is missing some check.
> > 
> > Ok, re-reading your last comment, I agree it looks like an issue in
> > the unit test itself.
> > Copying Olivier.
> > 
> No, I think it's not a bug in the test, but rather in the mempool.
> Unfortunately, I think the concept of the "default" driver for mempools
> seems to exist only when using the mbuf library to create mempools. Even
> then, the default mempool is different from what the first entry in the
> list is. That's the fundamental issue here, we are using the zero-eth entry
> in the ops list, rather than a default one.

Yes, Bruce is right.

As discussed off-list with David, moving rte_mempool_set_ops_byname() from
rte_mempool_create() to rte_mempool_create_empty() would ensure that the ring
driver is the default (and taking flags into account).

^ permalink raw reply	[flat|nested] 93+ messages in thread

* RE: [PATCH v5 05/10] app/test: define unit tests suites based on test macros
  2023-08-16 13:16             ` Olivier Matz
@ 2023-08-16 13:35               ` Morten Brørup
  2023-08-16 13:44                 ` Bruce Richardson
  0 siblings, 1 reply; 93+ messages in thread
From: Morten Brørup @ 2023-08-16 13:35 UTC (permalink / raw)
  To: Olivier Matz, Bruce Richardson; +Cc: David Marchand, dev, ci

> From: Olivier Matz [mailto:olivier.matz@6wind.com]
> Sent: Wednesday, 16 August 2023 15.17
> 
> Hi,
> 
> On Wed, Aug 16, 2023 at 01:33:46PM +0100, Bruce Richardson wrote:
> > On Wed, Aug 16, 2023 at 01:40:41PM +0200, David Marchand wrote:
> > > On Wed, Aug 16, 2023 at 1:15 PM David Marchand
> > > <david.marchand@redhat.com> wrote:
> > > >
> > > > On Wed, Aug 16, 2023 at 1:02 PM Bruce Richardson
> > > > <bruce.richardson@intel.com> wrote:
> > > > > These lines here seem to be exposing a bug in the mempool unit tests
> for
> > > > > shared builds, which is why we have a CI failure.
> > > > >
> > > > > The mempool unit tests use the mempool "create_empty" API, and then
> call
> > > > > the populate APIs to finish setting up the mempool. However, the
> > > > > create_empty API does not explicitly configure a particular set of
> mempool
> > > > > ops for the new mempool, leaving the ops field at 0. This means that
> unless
> > > > > the app explicitly sets other ops, the mempool will use the ops from
> > > > > whatever driver registers itself first. This occurs even when the
> driver is
> > > > > unsuitable, e.g. on my Intel system, the dpaa2 is first on the list,
> > > > > leading to failures in setting up and using the mempool.
> > > >
> > > > Hum, it sounds like a bug to me.
> > > > The dpaa2 driver should not be registered as the default (or here,
> > > > default platform) mempool.
> > > > Other mempool drivers ensure that required hw is available, I guess
> > > > dpaa2 is missing some check.
> > >
> > > Ok, re-reading your last comment, I agree it looks like an issue in
> > > the unit test itself.
> > > Copying Olivier.
> > >
> > No, I think it's not a bug in the test, but rather in the mempool.
> > Unfortunately, I think the concept of the "default" driver for mempools
> > seems to exist only when using the mbuf library to create mempools. Even
> > then, the default mempool is different from what the first entry in the
> > list is. That's the fundamental issue here, we are using the zero-eth entry
> > in the ops list, rather than a default one.
> 
> Yes, Bruce is right.
> 
> As discussed off-list with David, moving rte_mempool_set_ops_byname() from
> rte_mempool_create() to rte_mempool_create_empty() would ensure that the ring
> driver is the default (and taking flags into account).

I took a look at the mempool library source code, and reached the same conclusion.

Your suggested fix is also supported by the documentation of the "flags" parameter to rte_mempool_create_empty(), which - by referring to the "flags" parameter to rte_mempool_create() - says that the mempool ops will be set depending on the RTE_MEMPOOL_F_[S|M][P_PUT|C_GET] flags.

It should probably be flagged as a bug and backported to stable releases.


^ permalink raw reply	[flat|nested] 93+ messages in thread

* Re: [PATCH v5 05/10] app/test: define unit tests suites based on test macros
  2023-08-16 13:35               ` Morten Brørup
@ 2023-08-16 13:44                 ` Bruce Richardson
  0 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-16 13:44 UTC (permalink / raw)
  To: Morten Brørup; +Cc: Olivier Matz, David Marchand, dev, ci

On Wed, Aug 16, 2023 at 03:35:47PM +0200, Morten Brørup wrote:
> > From: Olivier Matz [mailto:olivier.matz@6wind.com]
> > Sent: Wednesday, 16 August 2023 15.17
> > 
> > Hi,
> > 
> > On Wed, Aug 16, 2023 at 01:33:46PM +0100, Bruce Richardson wrote:
> > > On Wed, Aug 16, 2023 at 01:40:41PM +0200, David Marchand wrote:
> > > > On Wed, Aug 16, 2023 at 1:15 PM David Marchand
> > > > <david.marchand@redhat.com> wrote:
> > > > >
> > > > > On Wed, Aug 16, 2023 at 1:02 PM Bruce Richardson
> > > > > <bruce.richardson@intel.com> wrote:
> > > > > > These lines here seem to be exposing a bug in the mempool unit tests
> > for
> > > > > > shared builds, which is why we have a CI failure.
> > > > > >
> > > > > > The mempool unit tests use the mempool "create_empty" API, and then
> > call
> > > > > > the populate APIs to finish setting up the mempool. However, the
> > > > > > create_empty API does not explicitly configure a particular set of
> > mempool
> > > > > > ops for the new mempool, leaving the ops field at 0. This means that
> > unless
> > > > > > the app explicitly sets other ops, the mempool will use the ops from
> > > > > > whatever driver registers itself first. This occurs even when the
> > driver is
> > > > > > unsuitable, e.g. on my Intel system, the dpaa2 is first on the list,
> > > > > > leading to failures in setting up and using the mempool.
> > > > >
> > > > > Hum, it sounds like a bug to me.
> > > > > The dpaa2 driver should not be registered as the default (or here,
> > > > > default platform) mempool.
> > > > > Other mempool drivers ensure that required hw is available, I guess
> > > > > dpaa2 is missing some check.
> > > >
> > > > Ok, re-reading your last comment, I agree it looks like an issue in
> > > > the unit test itself.
> > > > Copying Olivier.
> > > >
> > > No, I think it's not a bug in the test, but rather in the mempool.
> > > Unfortunately, I think the concept of the "default" driver for mempools
> > > seems to exist only when using the mbuf library to create mempools. Even
> > > then, the default mempool is different from what the first entry in the
> > > list is. That's the fundamental issue here, we are using the zero-eth entry
> > > in the ops list, rather than a default one.
> > 
> > Yes, Bruce is right.
> > 
> > As discussed off-list with David, moving rte_mempool_set_ops_byname() from
> > rte_mempool_create() to rte_mempool_create_empty() would ensure that the ring
> > driver is the default (and taking flags into account).
> 
> I took a look at the mempool library source code, and reached the same conclusion.
> 
> Your suggested fix is also supported by the documentation of the "flags" parameter to rte_mempool_create_empty(), which - by referring to the "flags" parameter to rte_mempool_create() - says that the mempool ops will be set depending on the RTE_MEMPOOL_F_[S|M][P_PUT|C_GET] flags.
> 
> It should probably be flagged as a bug and backported to stable releases.
>
Yep. David has kindly done up a patch to fix this, and I'm rolling it into
the v6 of this set.

/Bruce 

^ permalink raw reply	[flat|nested] 93+ messages in thread

* Re: [PATCH v5 04/10] app/test: build using per-file dependency matrix
  2023-08-16 12:55       ` Bruce Richardson
@ 2023-08-16 14:40         ` David Marchand
  2023-08-16 18:29           ` Patrick Robb
  0 siblings, 1 reply; 93+ messages in thread
From: David Marchand @ 2023-08-16 14:40 UTC (permalink / raw)
  To: Bruce Richardson, Patrick Robb
  Cc: dev, ci, Morten Brørup, Honnappa Nagarahalli,
	Ruifeng Wang (Arm Technology China),
	Thomas Monjalon, Aaron Conole

Patrick, Bruce,

On Wed, Aug 16, 2023 at 2:57 PM Bruce Richardson
<bruce.richardson@intel.com> wrote:
>
> On Tue, Aug 15, 2023 at 03:05:08PM -0400, Patrick Robb wrote:
> >    Adam from our team just raised something important about this patch and
> >    UNH CI which I missed during the RFC discussion.
> >    Presently, eal_flags_file_prefix_autotest fails on arm tx2 systems (arm
> >    people are aware of this, they think it might be a memory leak from

That's sad to read.

> >    mlx5 driver), so at their request we are disabling this unit test (on
> >    arm only) for them for the time being.

If it was reported, I either missed it or forgot about it, sorry.
Can you (re)share the context?

There was a couple of mem leak fixes for mlx5 in v23.07.
It may be worth running this test again and providing fresh traces.


>
> Does the test suite pass if the mlx5 driver is disabled in the build? That
> could confirm or refute the suspicion of where the issue is, and also
> provide a temporary workaround while this set is merged (possibly including
> support for disabling specific tests, as I suggested in my other email).

Or disabling the driver as Bruce proposes.

>
> /Bruce
>
> PS: Are there any other workarounds inside the test/DTS/CI systems that
> involve patching sources? If so, it would be good to get a list that we can
> work through removing by putting place proper fixes or workarounds, as
> changing sources for testing like this blocks future patch acceptance.

Patching sources from the test tool is a poor solution.
In general, developers won't be aware of source patching and will
waste time trying to understand why they can't reproduce what the CI
reports (it happened to me with DTS on the interrupt stuff with vhost,
at least).

For this specific case of skipping a test, if nobody can fix the
issue, I prefer if the CI can skip some "known broken in my lab" tests
via some meson configuration.
And, such configuration should be easy to catch in the test report.


-- 
David Marchand


^ permalink raw reply	[flat|nested] 93+ messages in thread

* Re: [PATCH v5 02/10] app/test: tag tests with the test type
  2023-08-15 15:10   ` [PATCH v5 02/10] app/test: tag tests with the test type Bruce Richardson
@ 2023-08-16 14:56     ` David Marchand
  2023-08-16 15:05       ` Bruce Richardson
  0 siblings, 1 reply; 93+ messages in thread
From: David Marchand @ 2023-08-16 14:56 UTC (permalink / raw)
  To: Bruce Richardson; +Cc: dev, ci, Morten Brørup

On Tue, Aug 15, 2023 at 5:23 PM Bruce Richardson
<bruce.richardson@intel.com> wrote:
> diff --git a/app/test/test_acl.c b/app/test/test_acl.c
> index 623f34682e..bf1466fe11 100644
> --- a/app/test/test_acl.c
> +++ b/app/test/test_acl.c
> @@ -1749,4 +1749,4 @@ test_acl(void)
>
>  #endif /* !RTE_EXEC_ENV_WINDOWS */
>
> -REGISTER_TEST_COMMAND(acl_autotest, test_acl);
> +REGISTER_FAST_TEST(acl_autotest, True, True, test_acl);

A bit surprising to read "True" and not "true" in some C code.
I see meson supports both forms.
Do you have a reason to go with "True"?
Maybe you prefer to keep it like this so it is not confused with the C
boolean keyword?


-- 
David Marchand


^ permalink raw reply	[flat|nested] 93+ messages in thread

* Re: [PATCH v5 05/10] app/test: define unit tests suites based on test macros
  2023-08-15 15:10   ` [PATCH v5 05/10] app/test: define unit tests suites based on test macros Bruce Richardson
  2023-08-16 11:02     ` Bruce Richardson
@ 2023-08-16 14:57     ` David Marchand
  2023-08-16 15:06       ` Bruce Richardson
  1 sibling, 1 reply; 93+ messages in thread
From: David Marchand @ 2023-08-16 14:57 UTC (permalink / raw)
  To: Bruce Richardson; +Cc: dev, ci, Morten Brørup

On Tue, Aug 15, 2023 at 5:24 PM Bruce Richardson
<bruce.richardson@intel.com> wrote:
>
> Rather than having the test suites listed out in the meson.build files
> and having to have them enabled/disabled selectively based on what libs
> are being built, pull the tests to run from the source files which were
> added to the build.
>
> Most test suites require no additional info other than the list of test
> names in the suite. However the fast-test are special that they have
> additional parameters associated with them. This requires some
> additional work in the test extraction script and in the meson.build
> file for processing the output.
>
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> Acked-by: Morten Brørup <mb@smartsharesystems.com>
> ---
>  app/meson.build                               |  7 +-
>  app/test/suites/meson.build                   | 74 +++++++++++++++++++
>  buildtools/get-test-suites.py                 | 33 +++++++++
>  .../has-hugepages.py                          |  0
>  buildtools/meson.build                        |  2 +
>  5 files changed, 115 insertions(+), 1 deletion(-)
>  create mode 100644 app/test/suites/meson.build
>  create mode 100644 buildtools/get-test-suites.py
>  rename app/test/has_hugepage.py => buildtools/has-hugepages.py (100%)
>
> diff --git a/app/meson.build b/app/meson.build
> index 0d8b618e7f..c14dc80892 100644
> --- a/app/meson.build
> +++ b/app/meson.build
> @@ -101,7 +101,7 @@ foreach app:apps
>          link_libs = dpdk_static_libraries + dpdk_drivers
>      endif
>
> -    executable('dpdk-' + name,
> +    exec = executable('dpdk-' + name,
>              sources,
>              c_args: cflags,
>              link_args: ldflags,
> @@ -110,4 +110,9 @@ foreach app:apps
>              include_directories: includes,
>              install_rpath: join_paths(get_option('prefix'), driver_install_path),
>              install: true)
> +    if name == 'test'
> +        dpdk_test = exec
> +        autotest_sources = sources
> +        subdir('test/suites')  # define the pre-canned test suites
> +    endif
>  endforeach
> diff --git a/app/test/suites/meson.build b/app/test/suites/meson.build
> new file mode 100644
> index 0000000000..ec74d8adf2
> --- /dev/null
> +++ b/app/test/suites/meson.build
> @@ -0,0 +1,74 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2023 Intel Corporation
> +
> +# some perf tests (eg: memcpy perf autotest)take very long

s/)take/) take/

> +# to complete, so timeout to 10 minutes
> +timeout_seconds = 600
> +timeout_seconds_fast = 10
> +
> +test_no_huge_args = ['--no-huge', '-m', '2048']
> +has_hugepage = run_command(has_hugepages_cmd, check: true).stdout().strip() != '0'
> +message('hugepage availability: @0@'.format(has_hugepage))
> +
> +# process source files to determine the different unit test suites
> +# - fast_tests
> +# - perf_tests
> +# - driver_tests
> +test_suites = run_command(get_test_suites_cmd, autotest_sources,
> +         check: true).stdout().strip().split()
> +foreach suite:test_suites
> +    # simple cases - tests without parameters or special handling
> +    suite = suite.split('=')
> +    suite_name = suite[0]
> +    suite_tests = suite[1].split(',')
> +    if suite_name != 'fast-tests'
> +        # simple cases - tests without parameters or special handling
> +        foreach t: suite_tests
> +            test(t, dpdk_test,
> +                    env: ['DPDK_TEST=' + t],
> +                    timeout: timeout_seconds,
> +                    is_parallel: false,
> +                    suite: suite_name)
> +        endforeach
> +    else
> +    # special fast-test handling here

Indentation of this comment is off.

> +        foreach t: suite_tests
> +            params = t.split(':')
> +            test_name = params[0]
> +            nohuge = params[1] == 'true'
> +            asan = params[2] == 'true'
> +
> +            test_args = []
> +            if nohuge
> +                test_args += test_no_huge_args
> +            elif not has_hugepage
> +                continue  #skip this tests
> +            endif
> +            if not asan and (get_option('b_sanitize') == 'address'
> +                    or get_option('b_sanitize') == 'address,undefined')
> +                continue  # skip this test
> +            endif
> +
> +            if get_option('default_library') == 'shared'
> +                test_args += ['-d', dpdk_drivers_build_dir]
> +            endif
> +
> +            test(test_name, dpdk_test,
> +                args : test_args,
> +                env: ['DPDK_TEST=' + test_name],
> +                timeout : timeout_seconds_fast,
> +                is_parallel : false,
> +                suite : 'fast-tests')
> +            if not is_windows and test_name == 'trace_autotest'
> +                test_args += ['--trace=.*']
> +                test_args += ['--trace-dir=@0@'.format(meson.current_build_dir())]
> +                test(test_name + '_with_traces', dpdk_test,
> +                    args : test_args,
> +                    env: ['DPDK_TEST=' + test_name],
> +                    timeout : timeout_seconds_fast,
> +                    is_parallel : false,
> +                    suite : 'fast-tests')
> +            endif
> +        endforeach
> +    endif
> +endforeach
> diff --git a/buildtools/get-test-suites.py b/buildtools/get-test-suites.py
> new file mode 100644
> index 0000000000..95a9cad4c8
> --- /dev/null
> +++ b/buildtools/get-test-suites.py
> @@ -0,0 +1,33 @@
> +#! /usr/bin/env python3
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2023 Intel Corporation
> +
> +import sys
> +import re
> +
> +input_list = sys.argv[1:]
> +test_def_regex = re.compile("REGISTER_([A-Z]+)_TEST\s*\(\s*([a-z0-9_]+)")
> +test_suites = {}
> +
> +def get_fast_test_params(test_name, ln):
> +    "Extract the extra fast-test parameters from the line"
> +    #print(f"ln: {ln.rstrip()}, test_name: {test_name}, split: {ln.split(test_name, 1)}")
> +    (_, rest_of_line) = ln.split(test_name, 1)
> +    (_, nohuge, asan, _func) = rest_of_line.split(',', 3)
> +    return f":{nohuge.strip().lower()}:{asan.strip().lower()}"
> +
> +for fname in input_list:
> +    with open(fname) as f:
> +        contents = [ln for ln in f.readlines() if test_def_regex.match(ln.strip())]
> +    for ln in contents:
> +        (test_suite, test_name) = test_def_regex.match(ln).group(1, 2)
> +        suite_name = f"{test_suite.lower()}-tests"
> +        if suite_name in test_suites:
> +            test_suites[suite_name].append(test_name)
> +        else:
> +            test_suites[suite_name] = [test_name]
> +        if suite_name == "fast-tests":
> +            test_suites["fast-tests"][-1] += get_fast_test_params(test_name, ln)
> +
> +for suite in test_suites.keys():
> +    print(f"{suite}={','.join(test_suites[suite])}")
> diff --git a/app/test/has_hugepage.py b/buildtools/has-hugepages.py
> similarity index 100%
> rename from app/test/has_hugepage.py
> rename to buildtools/has-hugepages.py

app/test/has_hugepage.py is still referenced in MAINTAINERS.
And new files (in this patch, and later patches) are not affiliated to someone.

> diff --git a/buildtools/meson.build b/buildtools/meson.build
> index e1c600e40f..ac5e4dcf08 100644
> --- a/buildtools/meson.build
> +++ b/buildtools/meson.build
> @@ -18,6 +18,8 @@ map_to_win_cmd = py3 + files('map_to_win.py')
>  sphinx_wrapper = py3 + files('call-sphinx-build.py')
>  get_cpu_count_cmd = py3 + files('get-cpu-count.py')
>  get_numa_count_cmd = py3 + files('get-numa-count.py')
> +get_test_suites_cmd = py3 + files('get-test-suites.py')
> +has_hugepages_cmd = py3 + files('has-hugepages.py')
>  binutils_avx512_check = (py3 + files('binutils-avx512-check.py') +
>                          [objdump] + cc.cmd_array())
>
> --
> 2.39.2
>


-- 
David Marchand


^ permalink raw reply	[flat|nested] 93+ messages in thread

* Re: [PATCH v5 02/10] app/test: tag tests with the test type
  2023-08-16 14:56     ` David Marchand
@ 2023-08-16 15:05       ` Bruce Richardson
  0 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-16 15:05 UTC (permalink / raw)
  To: David Marchand; +Cc: dev, ci, Morten Brørup

On Wed, Aug 16, 2023 at 04:56:23PM +0200, David Marchand wrote:
> On Tue, Aug 15, 2023 at 5:23 PM Bruce Richardson
> <bruce.richardson@intel.com> wrote:
> > diff --git a/app/test/test_acl.c b/app/test/test_acl.c
> > index 623f34682e..bf1466fe11 100644
> > --- a/app/test/test_acl.c
> > +++ b/app/test/test_acl.c
> > @@ -1749,4 +1749,4 @@ test_acl(void)
> >
> >  #endif /* !RTE_EXEC_ENV_WINDOWS */
> >
> > -REGISTER_TEST_COMMAND(acl_autotest, test_acl);
> > +REGISTER_FAST_TEST(acl_autotest, True, True, test_acl);
> 
> A bit surprising to read "True" and not "true" in some C code.
> I see meson supports both forms.
> Do you have a reason to go with "True"?
> Maybe you prefer to keep it like this so it is not confused with the C
> boolean keyword?
> 
This info is pulled out by python script, so I believe it is capitalized so
I can process them as boolean values there.
However, looking at the script, it doesn't actually ever convert to boolean
right now, so this may be a holdover from earlier versions. When we pass
the values to meson, they are all forced to lowercase.

/Bruce

^ permalink raw reply	[flat|nested] 93+ messages in thread

* Re: [PATCH v5 05/10] app/test: define unit tests suites based on test macros
  2023-08-16 14:57     ` David Marchand
@ 2023-08-16 15:06       ` Bruce Richardson
  0 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-16 15:06 UTC (permalink / raw)
  To: David Marchand; +Cc: dev, ci, Morten Brørup

On Wed, Aug 16, 2023 at 04:57:07PM +0200, David Marchand wrote:
> On Tue, Aug 15, 2023 at 5:24 PM Bruce Richardson
> <bruce.richardson@intel.com> wrote:
> >
> > Rather than having the test suites listed out in the meson.build files
> > and having to have them enabled/disabled selectively based on what libs
> > are being built, pull the tests to run from the source files which were
> > added to the build.
> >
> > Most test suites require no additional info other than the list of test
> > names in the suite. However the fast-test are special that they have
> > additional parameters associated with them. This requires some
> > additional work in the test extraction script and in the meson.build
> > file for processing the output.
> >
> > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> > Acked-by: Morten Brørup <mb@smartsharesystems.com>
> > ---
> >  app/meson.build                               |  7 +-
> >  app/test/suites/meson.build                   | 74 +++++++++++++++++++
> >  buildtools/get-test-suites.py                 | 33 +++++++++
> >  .../has-hugepages.py                          |  0
> >  buildtools/meson.build                        |  2 +
> >  5 files changed, 115 insertions(+), 1 deletion(-)
> >  create mode 100644 app/test/suites/meson.build
> >  create mode 100644 buildtools/get-test-suites.py
> >  rename app/test/has_hugepage.py => buildtools/has-hugepages.py (100%)
> >
> > diff --git a/app/meson.build b/app/meson.build
> > index 0d8b618e7f..c14dc80892 100644
> > --- a/app/meson.build
> > +++ b/app/meson.build
> > @@ -101,7 +101,7 @@ foreach app:apps
> >          link_libs = dpdk_static_libraries + dpdk_drivers
> >      endif
> >
> > -    executable('dpdk-' + name,
> > +    exec = executable('dpdk-' + name,
> >              sources,
> >              c_args: cflags,
> >              link_args: ldflags,
> > @@ -110,4 +110,9 @@ foreach app:apps
> >              include_directories: includes,
> >              install_rpath: join_paths(get_option('prefix'), driver_install_path),
> >              install: true)
> > +    if name == 'test'
> > +        dpdk_test = exec
> > +        autotest_sources = sources
> > +        subdir('test/suites')  # define the pre-canned test suites
> > +    endif
> >  endforeach
> > diff --git a/app/test/suites/meson.build b/app/test/suites/meson.build
> > new file mode 100644
> > index 0000000000..ec74d8adf2
> > --- /dev/null
> > +++ b/app/test/suites/meson.build
> > @@ -0,0 +1,74 @@
> > +# SPDX-License-Identifier: BSD-3-Clause
> > +# Copyright(c) 2023 Intel Corporation
> > +
> > +# some perf tests (eg: memcpy perf autotest)take very long
> 
> s/)take/) take/
> 
Ack

> > +# to complete, so timeout to 10 minutes
> > +timeout_seconds = 600
> > +timeout_seconds_fast = 10
> > +
> > +test_no_huge_args = ['--no-huge', '-m', '2048']
> > +has_hugepage = run_command(has_hugepages_cmd, check: true).stdout().strip() != '0'
> > +message('hugepage availability: @0@'.format(has_hugepage))
> > +
> > +# process source files to determine the different unit test suites
> > +# - fast_tests
> > +# - perf_tests
> > +# - driver_tests
> > +test_suites = run_command(get_test_suites_cmd, autotest_sources,
> > +         check: true).stdout().strip().split()
> > +foreach suite:test_suites
> > +    # simple cases - tests without parameters or special handling
> > +    suite = suite.split('=')
> > +    suite_name = suite[0]
> > +    suite_tests = suite[1].split(',')
> > +    if suite_name != 'fast-tests'
> > +        # simple cases - tests without parameters or special handling
> > +        foreach t: suite_tests
> > +            test(t, dpdk_test,
> > +                    env: ['DPDK_TEST=' + t],
> > +                    timeout: timeout_seconds,
> > +                    is_parallel: false,
> > +                    suite: suite_name)
> > +        endforeach
> > +    else
> > +    # special fast-test handling here
> 
> Indentation of this comment is off.
>
Ack
 
> > +        foreach t: suite_tests
> > +            params = t.split(':')
> > +            test_name = params[0]
> > +            nohuge = params[1] == 'true'
> > +            asan = params[2] == 'true'
> > +
> > +            test_args = []
> > +            if nohuge
> > +                test_args += test_no_huge_args
> > +            elif not has_hugepage
> > +                continue  #skip this tests
> > +            endif
> > +            if not asan and (get_option('b_sanitize') == 'address'
> > +                    or get_option('b_sanitize') == 'address,undefined')
> > +                continue  # skip this test
> > +            endif
> > +
> > +            if get_option('default_library') == 'shared'
> > +                test_args += ['-d', dpdk_drivers_build_dir]
> > +            endif
> > +
> > +            test(test_name, dpdk_test,
> > +                args : test_args,
> > +                env: ['DPDK_TEST=' + test_name],
> > +                timeout : timeout_seconds_fast,
> > +                is_parallel : false,
> > +                suite : 'fast-tests')
> > +            if not is_windows and test_name == 'trace_autotest'
> > +                test_args += ['--trace=.*']
> > +                test_args += ['--trace-dir=@0@'.format(meson.current_build_dir())]
> > +                test(test_name + '_with_traces', dpdk_test,
> > +                    args : test_args,
> > +                    env: ['DPDK_TEST=' + test_name],
> > +                    timeout : timeout_seconds_fast,
> > +                    is_parallel : false,
> > +                    suite : 'fast-tests')
> > +            endif
> > +        endforeach
> > +    endif
> > +endforeach
> > diff --git a/buildtools/get-test-suites.py b/buildtools/get-test-suites.py
> > new file mode 100644
> > index 0000000000..95a9cad4c8
> > --- /dev/null
> > +++ b/buildtools/get-test-suites.py
> > @@ -0,0 +1,33 @@
> > +#! /usr/bin/env python3
> > +# SPDX-License-Identifier: BSD-3-Clause
> > +# Copyright(c) 2023 Intel Corporation
> > +
> > +import sys
> > +import re
> > +
> > +input_list = sys.argv[1:]
> > +test_def_regex = re.compile("REGISTER_([A-Z]+)_TEST\s*\(\s*([a-z0-9_]+)")
> > +test_suites = {}
> > +
> > +def get_fast_test_params(test_name, ln):
> > +    "Extract the extra fast-test parameters from the line"
> > +    #print(f"ln: {ln.rstrip()}, test_name: {test_name}, split: {ln.split(test_name, 1)}")
> > +    (_, rest_of_line) = ln.split(test_name, 1)
> > +    (_, nohuge, asan, _func) = rest_of_line.split(',', 3)
> > +    return f":{nohuge.strip().lower()}:{asan.strip().lower()}"
> > +
> > +for fname in input_list:
> > +    with open(fname) as f:
> > +        contents = [ln for ln in f.readlines() if test_def_regex.match(ln.strip())]
> > +    for ln in contents:
> > +        (test_suite, test_name) = test_def_regex.match(ln).group(1, 2)
> > +        suite_name = f"{test_suite.lower()}-tests"
> > +        if suite_name in test_suites:
> > +            test_suites[suite_name].append(test_name)
> > +        else:
> > +            test_suites[suite_name] = [test_name]
> > +        if suite_name == "fast-tests":
> > +            test_suites["fast-tests"][-1] += get_fast_test_params(test_name, ln)
> > +
> > +for suite in test_suites.keys():
> > +    print(f"{suite}={','.join(test_suites[suite])}")
> > diff --git a/app/test/has_hugepage.py b/buildtools/has-hugepages.py
> > similarity index 100%
> > rename from app/test/has_hugepage.py
> > rename to buildtools/has-hugepages.py
> 
> app/test/has_hugepage.py is still referenced in MAINTAINERS.
> And new files (in this patch, and later patches) are not affiliated to someone.
>
Will look to fix in V6.

^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v6 00/11] expand list of optional libraries
  2023-07-21 11:51 [RFC PATCH 0/5] replace build code for unit tests Bruce Richardson
                   ` (9 preceding siblings ...)
  2023-08-15 15:10 ` [PATCH v5 00/10] " Bruce Richardson
@ 2023-08-16 15:34 ` Bruce Richardson
  2023-08-16 15:34   ` [PATCH v6 01/11] mempool: fix default ops for an empty mempool Bruce Richardson
                     ` (11 more replies)
  10 siblings, 12 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-16 15:34 UTC (permalink / raw)
  To: dev; +Cc: david.marchand, probb, mb, Bruce Richardson

This patchset contains changes to DPDK to make the build more configurable
overall. The changes can be grouped into two areas:

* Firstly, there are changes to make the build of the unit tests more
  flexible and maintainable. These 7 patches switch the unit tests from
  having separate hard-coded lists of files and tests, with lots of
  conditional logic to add/remove items from the list, to instead being
  a single list of files and dependencies per-file. The actual lists of
  test-suites are built up dynamically from the list of files whose
  dependencies are met and therefore will be part of the build.
  This rework enables...
* The second set of patches which expands the list of libraries which
  can be disabled at build-time, while still giving a successful build
  of DPDK. Overall, after this set is applied, the list of optional
  libraries in DPDK is approx doubled in size. [15 additions giving
  31 optional libs]

NOTES:
* as stated in the RFC, the test binary moves in the build as a result
  of this set - from app/test/dpdk-test to app/dpdk-test.
* running the suites of tests no longer works after patch 4 until patch
  5-7 is applied. If this is an issue, the patches can be merged on
  apply. However, the individual tests are still runnable through the
  test binary - it's only through "meson test" that there are issues, so
  I'm keeping the patches separate since each is relatively complex in its
  own right.
* A number of files in the test directory were not being built, and seem
  to have never been built with meson. This requires separate work, outside
  the scope of the patchset to fix.
* Similarly, some unit tests were never added to test suites. These can
  now be found by using grep to find the old REGISTER_TEST_COMMAND macro.
  Again, adding orphan tests to suites is outside the scope of this set.

V6:
- include fix from David M. for mempool issue causing CI failures
- change use of "True" to "true" in C file macros for the fast-tests.
- add new/renamed python scripts to MAINTAINERS
- minor typo fixes

V5:
- revert back to using the environment to pass the test names rather than
  the commandline. This allows use of "--test-args" with meson test to
  pass additional EAL parameters, such as coremasks, to the test run.
- add back in support for the script for checking telemetry commands
- add support for the dump-* tests in the "debug-tests" suite. Sadly,
  even though these are hardcoded in commands.c, they aren't easily
  discoverable at build-time, so we have to use a hard-coded list.

V4:
- improve test suite definition, hopefully improving test pass rate:
  - drop unnecessary file-prefix for fast tests, since tests are not parallel
  - always pass driver path flag in "meson test" for shared builds.

V3:
- Create symlink to old path to test binary, to ensure all automated CI,
  and any other use of the test binary still works.

V2:
- Changes since RFC:
   -  merged two previous sets - reworking test build, and disabling
      libs - into one, since they depend on each other
   -  Reworked the list of dependencies for each file in the tests,
      so that files that depend on other files have those deps called
      out.
   -  Added list of external dependencies to the test builds
   -  For disabling libs, merged patches disabling individual libs
      together into one patch, since all unit test build changes were
      unnecessary. This made each patch a single-line addition to
      the optional list in lib/meson.build.

Bruce Richardson (10):
  app/test: add new macros for various test types
  app/test: tag tests with the test type
  app/test: make telemetry data test buildable on windows
  app/test: build using per-file dependency matrix
  app/test: define unit tests suites based on test macros
  app/test: add test case for scripted telemetry commands
  app/test: add debug test suite
  examples/l3fwd: make eventdev an optional dependency
  build: make most device classes optional
  build: expand list of optional libraries

David Marchand (1):
  mempool: fix default ops for an empty mempool

 MAINTAINERS                                   |   3 +-
 app/meson.build                               |  15 +-
 app/test/meson.build                          | 823 +++++-------------
 app/test/suites/meson.build                   | 124 +++
 app/test/{ => suites}/test_telemetry.sh       |   0
 app/test/test.h                               |   9 +-
 app/test/test_acl.c                           |   2 +-
 app/test/test_atomic.c                        |   2 +-
 app/test/test_barrier.c                       |   2 +-
 app/test/test_bitmap.c                        |   2 +-
 app/test/test_bitops.c                        |   2 +-
 app/test/test_bpf.c                           |   4 +-
 app/test/test_byteorder.c                     |   2 +-
 app/test/test_cksum.c                         |   2 +-
 app/test/test_cmdline.c                       |   2 +-
 app/test/test_common.c                        |   2 +-
 app/test/test_cpuflags.c                      |   2 +-
 app/test/test_crc.c                           |   2 +-
 app/test/test_cryptodev.c                     |  38 +-
 app/test/test_cryptodev_asym.c                |   2 +-
 app/test/test_cycles.c                        |   2 +-
 app/test/test_debug.c                         |   2 +-
 app/test/test_devargs.c                       |   2 +-
 app/test/test_distributor.c                   |   2 +-
 app/test/test_distributor_perf.c              |   2 +-
 app/test/test_dmadev.c                        |   2 +-
 app/test/test_eal_flags.c                     |  24 +-
 app/test/test_eal_fs.c                        |   2 +-
 app/test/test_efd.c                           |   2 +-
 app/test/test_efd_perf.c                      |   2 +-
 app/test/test_errno.c                         |   2 +-
 app/test/test_ethdev_link.c                   |   2 +-
 app/test/test_event_ring.c                    |   2 +-
 app/test/test_eventdev.c                      |   2 +-
 app/test/test_fbarray.c                       |   2 +-
 app/test/test_fib.c                           |   4 +-
 app/test/test_fib6.c                          |   4 +-
 app/test/test_fib6_perf.c                     |   2 +-
 app/test/test_fib_perf.c                      |   2 +-
 app/test/test_func_reentrancy.c               |   2 +-
 app/test/test_hash.c                          |   2 +-
 app/test/test_hash_functions.c                |   2 +-
 app/test/test_hash_multiwriter.c              |   2 +-
 app/test/test_hash_perf.c                     |   2 +-
 app/test/test_hash_readwrite.c                |   4 +-
 app/test/test_hash_readwrite_lf_perf.c        |   2 +-
 app/test/test_interrupts.c                    |   2 +-
 app/test/test_ipfrag.c                        |   2 +-
 app/test/test_ipsec.c                         |   2 +-
 app/test/test_ipsec_perf.c                    |   2 +-
 app/test/test_kvargs.c                        |   2 +-
 app/test/test_lcores.c                        |   2 +-
 app/test/test_logs.c                          |   2 +-
 app/test/test_lpm.c                           |   2 +-
 app/test/test_lpm6.c                          |   2 +-
 app/test/test_lpm6_perf.c                     |   2 +-
 app/test/test_lpm_perf.c                      |   2 +-
 app/test/test_malloc.c                        |   2 +-
 app/test/test_malloc_perf.c                   |   2 +-
 app/test/test_mbuf.c                          |   2 +-
 app/test/test_mcslock.c                       |   2 +-
 app/test/test_member.c                        |   2 +-
 app/test/test_member_perf.c                   |   2 +-
 app/test/test_memcpy.c                        |   2 +-
 app/test/test_memcpy_perf.c                   |   2 +-
 app/test/test_memory.c                        |   2 +-
 app/test/test_mempool.c                       |   2 +-
 app/test/test_mempool_perf.c                  |   2 +-
 app/test/test_memzone.c                       |   2 +-
 app/test/test_meter.c                         |   2 +-
 app/test/test_mp_secondary.c                  |   2 +-
 app/test/test_per_lcore.c                     |   2 +-
 app/test/test_pflock.c                        |   2 +-
 app/test/test_pie.c                           |   6 +-
 app/test/test_pmd_perf.c                      |   2 +-
 app/test/test_power.c                         |   2 +-
 app/test/test_power_cpufreq.c                 |   2 +-
 app/test/test_power_intel_uncore.c            |   2 +-
 app/test/test_power_kvm_vm.c                  |   2 +-
 app/test/test_prefetch.c                      |   2 +-
 app/test/test_rand_perf.c                     |   2 +-
 app/test/test_rcu_qsbr.c                      |   2 +-
 app/test/test_rcu_qsbr_perf.c                 |   2 +-
 app/test/test_reassembly_perf.c               |   2 +-
 app/test/test_reciprocal_division.c           |   2 +-
 app/test/test_reciprocal_division_perf.c      |   2 +-
 app/test/test_red.c                           |   4 +-
 app/test/test_reorder.c                       |   2 +-
 app/test/test_rib.c                           |   4 +-
 app/test/test_rib6.c                          |   4 +-
 app/test/test_ring.c                          |   2 +-
 app/test/test_ring_perf.c                     |   2 +-
 app/test/test_rwlock.c                        |   8 +-
 app/test/test_sched.c                         |   2 +-
 app/test/test_security.c                      |   2 +-
 app/test/test_seqlock.c                       |   2 +-
 app/test/test_service_cores.c                 |   4 +-
 app/test/test_spinlock.c                      |   2 +-
 app/test/test_stack.c                         |   4 +-
 app/test/test_stack_perf.c                    |   4 +-
 app/test/test_string_fns.c                    |   2 +-
 app/test/test_tailq.c                         |   2 +-
 app/test/test_telemetry_data.c                |  14 +-
 app/test/test_thash.c                         |   2 +-
 app/test/test_thash_perf.c                    |   2 +-
 app/test/test_threads.c                       |   2 +-
 app/test/test_ticketlock.c                    |   2 +-
 app/test/test_timer.c                         |   2 +-
 app/test/test_timer_perf.c                    |   2 +-
 app/test/test_timer_racecond.c                |   2 +-
 app/test/test_trace.c                         |   2 +-
 app/test/test_trace_perf.c                    |   2 +-
 app/test/test_version.c                       |   2 +-
 buildtools/get-test-suites.py                 |  33 +
 .../has-hugepages.py                          |   0
 buildtools/meson.build                        |   2 +
 examples/l3fwd/l3fwd_em.c                     |   2 +
 examples/l3fwd/l3fwd_em_hlm.h                 |   2 +
 examples/l3fwd/l3fwd_event.c                  |   2 +
 examples/l3fwd/l3fwd_event.h                  |   7 +-
 examples/l3fwd/l3fwd_event_generic.c          |   2 +
 examples/l3fwd/l3fwd_event_internal_port.c    |   2 +
 examples/l3fwd/l3fwd_fib.c                    |   2 +
 examples/l3fwd/l3fwd_lpm.c                    |   2 +
 examples/l3fwd/main.c                         |  65 +-
 examples/l3fwd/meson.build                    |   5 +-
 lib/mempool/rte_mempool.c                     |  33 +-
 lib/meson.build                               |  15 +
 128 files changed, 680 insertions(+), 782 deletions(-)
 create mode 100644 app/test/suites/meson.build
 rename app/test/{ => suites}/test_telemetry.sh (100%)
 create mode 100644 buildtools/get-test-suites.py
 rename app/test/has_hugepage.py => buildtools/has-hugepages.py (100%)

--
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v6 01/11] mempool: fix default ops for an empty mempool
  2023-08-16 15:34 ` [PATCH v6 00/11] " Bruce Richardson
@ 2023-08-16 15:34   ` Bruce Richardson
  2023-08-16 16:32     ` Morten Brørup
  2023-08-16 15:34   ` [PATCH v6 02/11] app/test: add new macros for various test types Bruce Richardson
                     ` (10 subsequent siblings)
  11 siblings, 1 reply; 93+ messages in thread
From: Bruce Richardson @ 2023-08-16 15:34 UTC (permalink / raw)
  To: dev; +Cc: david.marchand, probb, mb, stable, Bruce Richardson

From: David Marchand <david.marchand@redhat.com>

An empty mempool's ops were not initialised to a default value wrt to
what the application requested via the flags parameter.  As
rte_mempool_create() relies on rte_mempool_create_empty(), simply move
this ops initialisation to rte_mempool_create_empty().

Fixes: aa10457eb4c2 ("mempool: make mempool populate and free api public")
Cc: stable@dpdk.org

Signed-off-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Bruce Richardson <bruce.richardson@intel.com>
---
 lib/mempool/rte_mempool.c | 33 ++++++++++++++++-----------------
 1 file changed, 16 insertions(+), 17 deletions(-)

diff --git a/lib/mempool/rte_mempool.c b/lib/mempool/rte_mempool.c
index 4d337fca8d..7a7a9bf6db 100644
--- a/lib/mempool/rte_mempool.c
+++ b/lib/mempool/rte_mempool.c
@@ -914,6 +914,22 @@ rte_mempool_create_empty(const char *name, unsigned n, unsigned elt_size,
 	STAILQ_INIT(&mp->elt_list);
 	STAILQ_INIT(&mp->mem_list);
 
+	/*
+	 * Since we have 4 combinations of the SP/SC/MP/MC examine the flags to
+	 * set the correct index into the table of ops structs.
+	 */
+	if ((flags & RTE_MEMPOOL_F_SP_PUT) && (flags & RTE_MEMPOOL_F_SC_GET))
+		ret = rte_mempool_set_ops_byname(mp, "ring_sp_sc", NULL);
+	else if (flags & RTE_MEMPOOL_F_SP_PUT)
+		ret = rte_mempool_set_ops_byname(mp, "ring_sp_mc", NULL);
+	else if (flags & RTE_MEMPOOL_F_SC_GET)
+		ret = rte_mempool_set_ops_byname(mp, "ring_mp_sc", NULL);
+	else
+		ret = rte_mempool_set_ops_byname(mp, "ring_mp_mc", NULL);
+
+	if (ret)
+		goto exit_unlock;
+
 	/*
 	 * local_cache pointer is set even if cache_size is zero.
 	 * The local_cache points to just past the elt_pa[] array.
@@ -954,7 +970,6 @@ rte_mempool_create(const char *name, unsigned n, unsigned elt_size,
 	rte_mempool_obj_cb_t *obj_init, void *obj_init_arg,
 	int socket_id, unsigned flags)
 {
-	int ret;
 	struct rte_mempool *mp;
 
 	mp = rte_mempool_create_empty(name, n, elt_size, cache_size,
@@ -962,22 +977,6 @@ rte_mempool_create(const char *name, unsigned n, unsigned elt_size,
 	if (mp == NULL)
 		return NULL;
 
-	/*
-	 * Since we have 4 combinations of the SP/SC/MP/MC examine the flags to
-	 * set the correct index into the table of ops structs.
-	 */
-	if ((flags & RTE_MEMPOOL_F_SP_PUT) && (flags & RTE_MEMPOOL_F_SC_GET))
-		ret = rte_mempool_set_ops_byname(mp, "ring_sp_sc", NULL);
-	else if (flags & RTE_MEMPOOL_F_SP_PUT)
-		ret = rte_mempool_set_ops_byname(mp, "ring_sp_mc", NULL);
-	else if (flags & RTE_MEMPOOL_F_SC_GET)
-		ret = rte_mempool_set_ops_byname(mp, "ring_mp_sc", NULL);
-	else
-		ret = rte_mempool_set_ops_byname(mp, "ring_mp_mc", NULL);
-
-	if (ret)
-		goto fail;
-
 	/* call the mempool priv initializer */
 	if (mp_init)
 		mp_init(mp, mp_init_arg);
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v6 02/11] app/test: add new macros for various test types
  2023-08-16 15:34 ` [PATCH v6 00/11] " Bruce Richardson
  2023-08-16 15:34   ` [PATCH v6 01/11] mempool: fix default ops for an empty mempool Bruce Richardson
@ 2023-08-16 15:34   ` Bruce Richardson
  2023-08-16 15:34   ` [PATCH v6 03/11] app/test: tag tests with the test type Bruce Richardson
                     ` (9 subsequent siblings)
  11 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-16 15:34 UTC (permalink / raw)
  To: dev; +Cc: david.marchand, probb, mb, Bruce Richardson

Rather than just registering all tests using a single generic macro,
add macros which identify the test as being of a particular type.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
---
 app/test/test.h | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/app/test/test.h b/app/test/test.h
index 85f57efbc6..a91ded76af 100644
--- a/app/test/test.h
+++ b/app/test/test.h
@@ -190,7 +190,7 @@ struct test_command {
 
 void add_test_command(struct test_command *t);
 
-/* Register a test function with its command string */
+/* Register a test function with its command string. Should not be used directly */
 #define REGISTER_TEST_COMMAND(cmd, func) \
 	static struct test_command test_struct_##cmd = { \
 		.command = RTE_STR(cmd), \
@@ -201,4 +201,11 @@ void add_test_command(struct test_command *t);
 		add_test_command(&test_struct_##cmd); \
 	}
 
+/* Register a test function as a particular type.
+ * These can be used to build up test suites automatically
+ */
+#define REGISTER_FAST_TEST(cmd, no_huge, ASan, func)  REGISTER_TEST_COMMAND(cmd, func)
+#define REGISTER_PERF_TEST REGISTER_TEST_COMMAND
+#define REGISTER_DRIVER_TEST REGISTER_TEST_COMMAND
+
 #endif
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v6 03/11] app/test: tag tests with the test type
  2023-08-16 15:34 ` [PATCH v6 00/11] " Bruce Richardson
  2023-08-16 15:34   ` [PATCH v6 01/11] mempool: fix default ops for an empty mempool Bruce Richardson
  2023-08-16 15:34   ` [PATCH v6 02/11] app/test: add new macros for various test types Bruce Richardson
@ 2023-08-16 15:34   ` Bruce Richardson
  2023-08-16 15:34   ` [PATCH v6 04/11] app/test: make telemetry data test buildable on windows Bruce Richardson
                     ` (8 subsequent siblings)
  11 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-16 15:34 UTC (permalink / raw)
  To: dev; +Cc: david.marchand, probb, mb, Bruce Richardson

Rather than having the test types called out in the meson.build file, we
can use macros to identify the test type in the C file itself and then
dynamically build up the tests lists at config time.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
---
 app/test/test_acl.c                      |  2 +-
 app/test/test_atomic.c                   |  2 +-
 app/test/test_barrier.c                  |  2 +-
 app/test/test_bitmap.c                   |  2 +-
 app/test/test_bitops.c                   |  2 +-
 app/test/test_bpf.c                      |  4 +--
 app/test/test_byteorder.c                |  2 +-
 app/test/test_cksum.c                    |  2 +-
 app/test/test_cmdline.c                  |  2 +-
 app/test/test_common.c                   |  2 +-
 app/test/test_cpuflags.c                 |  2 +-
 app/test/test_crc.c                      |  2 +-
 app/test/test_cryptodev.c                | 38 ++++++++++++------------
 app/test/test_cryptodev_asym.c           |  2 +-
 app/test/test_cycles.c                   |  2 +-
 app/test/test_debug.c                    |  2 +-
 app/test/test_devargs.c                  |  2 +-
 app/test/test_distributor.c              |  2 +-
 app/test/test_distributor_perf.c         |  2 +-
 app/test/test_dmadev.c                   |  2 +-
 app/test/test_eal_flags.c                | 24 +++++++--------
 app/test/test_eal_fs.c                   |  2 +-
 app/test/test_efd.c                      |  2 +-
 app/test/test_efd_perf.c                 |  2 +-
 app/test/test_errno.c                    |  2 +-
 app/test/test_ethdev_link.c              |  2 +-
 app/test/test_event_ring.c               |  2 +-
 app/test/test_eventdev.c                 |  2 +-
 app/test/test_fbarray.c                  |  2 +-
 app/test/test_fib.c                      |  4 +--
 app/test/test_fib6.c                     |  4 +--
 app/test/test_fib6_perf.c                |  2 +-
 app/test/test_fib_perf.c                 |  2 +-
 app/test/test_func_reentrancy.c          |  2 +-
 app/test/test_hash.c                     |  2 +-
 app/test/test_hash_functions.c           |  2 +-
 app/test/test_hash_multiwriter.c         |  2 +-
 app/test/test_hash_perf.c                |  2 +-
 app/test/test_hash_readwrite.c           |  4 +--
 app/test/test_hash_readwrite_lf_perf.c   |  2 +-
 app/test/test_interrupts.c               |  2 +-
 app/test/test_ipfrag.c                   |  2 +-
 app/test/test_ipsec.c                    |  2 +-
 app/test/test_ipsec_perf.c               |  2 +-
 app/test/test_kvargs.c                   |  2 +-
 app/test/test_lcores.c                   |  2 +-
 app/test/test_logs.c                     |  2 +-
 app/test/test_lpm.c                      |  2 +-
 app/test/test_lpm6.c                     |  2 +-
 app/test/test_lpm6_perf.c                |  2 +-
 app/test/test_lpm_perf.c                 |  2 +-
 app/test/test_malloc.c                   |  2 +-
 app/test/test_malloc_perf.c              |  2 +-
 app/test/test_mbuf.c                     |  2 +-
 app/test/test_mcslock.c                  |  2 +-
 app/test/test_member.c                   |  2 +-
 app/test/test_member_perf.c              |  2 +-
 app/test/test_memcpy.c                   |  2 +-
 app/test/test_memcpy_perf.c              |  2 +-
 app/test/test_memory.c                   |  2 +-
 app/test/test_mempool.c                  |  2 +-
 app/test/test_mempool_perf.c             |  2 +-
 app/test/test_memzone.c                  |  2 +-
 app/test/test_meter.c                    |  2 +-
 app/test/test_mp_secondary.c             |  2 +-
 app/test/test_per_lcore.c                |  2 +-
 app/test/test_pflock.c                   |  2 +-
 app/test/test_pie.c                      |  6 ++--
 app/test/test_pmd_perf.c                 |  2 +-
 app/test/test_power.c                    |  2 +-
 app/test/test_power_cpufreq.c            |  2 +-
 app/test/test_power_intel_uncore.c       |  2 +-
 app/test/test_power_kvm_vm.c             |  2 +-
 app/test/test_prefetch.c                 |  2 +-
 app/test/test_rand_perf.c                |  2 +-
 app/test/test_rcu_qsbr.c                 |  2 +-
 app/test/test_rcu_qsbr_perf.c            |  2 +-
 app/test/test_reassembly_perf.c          |  2 +-
 app/test/test_reciprocal_division.c      |  2 +-
 app/test/test_reciprocal_division_perf.c |  2 +-
 app/test/test_red.c                      |  4 +--
 app/test/test_reorder.c                  |  2 +-
 app/test/test_rib.c                      |  4 +--
 app/test/test_rib6.c                     |  4 +--
 app/test/test_ring.c                     |  2 +-
 app/test/test_ring_perf.c                |  2 +-
 app/test/test_rwlock.c                   |  8 ++---
 app/test/test_sched.c                    |  2 +-
 app/test/test_security.c                 |  2 +-
 app/test/test_seqlock.c                  |  2 +-
 app/test/test_service_cores.c            |  4 +--
 app/test/test_spinlock.c                 |  2 +-
 app/test/test_stack.c                    |  4 +--
 app/test/test_stack_perf.c               |  4 +--
 app/test/test_string_fns.c               |  2 +-
 app/test/test_tailq.c                    |  2 +-
 app/test/test_thash.c                    |  2 +-
 app/test/test_thash_perf.c               |  2 +-
 app/test/test_threads.c                  |  2 +-
 app/test/test_ticketlock.c               |  2 +-
 app/test/test_timer.c                    |  2 +-
 app/test/test_timer_perf.c               |  2 +-
 app/test/test_timer_racecond.c           |  2 +-
 app/test/test_trace.c                    |  2 +-
 app/test/test_trace_perf.c               |  2 +-
 app/test/test_version.c                  |  2 +-
 106 files changed, 150 insertions(+), 150 deletions(-)

diff --git a/app/test/test_acl.c b/app/test/test_acl.c
index 623f34682e..8011639ddd 100644
--- a/app/test/test_acl.c
+++ b/app/test/test_acl.c
@@ -1749,4 +1749,4 @@ test_acl(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(acl_autotest, test_acl);
+REGISTER_FAST_TEST(acl_autotest, true, true, test_acl);
diff --git a/app/test/test_atomic.c b/app/test/test_atomic.c
index e4b997827e..db07159e81 100644
--- a/app/test/test_atomic.c
+++ b/app/test/test_atomic.c
@@ -631,4 +631,4 @@ test_atomic(void)
 
 	return 0;
 }
-REGISTER_TEST_COMMAND(atomic_autotest, test_atomic);
+REGISTER_FAST_TEST(atomic_autotest, false, true, test_atomic);
diff --git a/app/test/test_barrier.c b/app/test/test_barrier.c
index ec69af25bf..925a88b68a 100644
--- a/app/test/test_barrier.c
+++ b/app/test/test_barrier.c
@@ -285,4 +285,4 @@ test_barrier(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(barrier_autotest, test_barrier);
+REGISTER_PERF_TEST(barrier_autotest, test_barrier);
diff --git a/app/test/test_bitmap.c b/app/test/test_bitmap.c
index e9c61590ae..9a0536c805 100644
--- a/app/test/test_bitmap.c
+++ b/app/test/test_bitmap.c
@@ -269,4 +269,4 @@ test_bitmap(void)
 	return test_bitmap_all_set();
 }
 
-REGISTER_TEST_COMMAND(bitmap_autotest, test_bitmap);
+REGISTER_FAST_TEST(bitmap_autotest, true, true, test_bitmap);
diff --git a/app/test/test_bitops.c b/app/test/test_bitops.c
index c21426bf2f..0d4ccfb468 100644
--- a/app/test/test_bitops.c
+++ b/app/test/test_bitops.c
@@ -135,4 +135,4 @@ test_bitops(void)
 	return TEST_SUCCESS;
 }
 
-REGISTER_TEST_COMMAND(bitops_autotest, test_bitops);
+REGISTER_FAST_TEST(bitops_autotest, true, true, test_bitops);
diff --git a/app/test/test_bpf.c b/app/test/test_bpf.c
index f5af5e8a3f..f83e72a9ad 100644
--- a/app/test/test_bpf.c
+++ b/app/test/test_bpf.c
@@ -3262,7 +3262,7 @@ test_bpf(void)
 
 #endif /* !RTE_LIB_BPF */
 
-REGISTER_TEST_COMMAND(bpf_autotest, test_bpf);
+REGISTER_FAST_TEST(bpf_autotest, true, true, test_bpf);
 
 #ifndef RTE_HAS_LIBPCAP
 
@@ -3473,4 +3473,4 @@ test_bpf_convert(void)
 
 #endif /* RTE_HAS_LIBPCAP */
 
-REGISTER_TEST_COMMAND(bpf_convert_autotest, test_bpf_convert);
+REGISTER_FAST_TEST(bpf_convert_autotest, true, true, test_bpf_convert);
diff --git a/app/test/test_byteorder.c b/app/test/test_byteorder.c
index de14ed539e..67ca7ebbc8 100644
--- a/app/test/test_byteorder.c
+++ b/app/test/test_byteorder.c
@@ -63,4 +63,4 @@ test_byteorder(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(byteorder_autotest, test_byteorder);
+REGISTER_FAST_TEST(byteorder_autotest, true, true, test_byteorder);
diff --git a/app/test/test_cksum.c b/app/test/test_cksum.c
index 6c15de9a93..f2ab5af5a7 100644
--- a/app/test/test_cksum.c
+++ b/app/test/test_cksum.c
@@ -267,4 +267,4 @@ test_cksum(void)
 }
 #undef GOTO_FAIL
 
-REGISTER_TEST_COMMAND(cksum_autotest, test_cksum);
+REGISTER_FAST_TEST(cksum_autotest, true, true, test_cksum);
diff --git a/app/test/test_cmdline.c b/app/test/test_cmdline.c
index 115bee966d..1d8020995c 100644
--- a/app/test/test_cmdline.c
+++ b/app/test/test_cmdline.c
@@ -60,4 +60,4 @@ test_cmdline(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(cmdline_autotest, test_cmdline);
+REGISTER_FAST_TEST(cmdline_autotest, true, true, test_cmdline);
diff --git a/app/test/test_common.c b/app/test/test_common.c
index f89e1eb7ee..29452f0a49 100644
--- a/app/test/test_common.c
+++ b/app/test/test_common.c
@@ -350,4 +350,4 @@ test_common(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(common_autotest, test_common);
+REGISTER_FAST_TEST(common_autotest, true, true, test_common);
diff --git a/app/test/test_cpuflags.c b/app/test/test_cpuflags.c
index a0e342ae48..4de06983d9 100644
--- a/app/test/test_cpuflags.c
+++ b/app/test/test_cpuflags.c
@@ -334,4 +334,4 @@ test_cpuflags(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(cpuflags_autotest, test_cpuflags);
+REGISTER_FAST_TEST(cpuflags_autotest, true, true, test_cpuflags);
diff --git a/app/test/test_crc.c b/app/test/test_crc.c
index 5edc8fb13b..b85fca35fe 100644
--- a/app/test/test_crc.c
+++ b/app/test/test_crc.c
@@ -171,4 +171,4 @@ test_crc(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(crc_autotest, test_crc);
+REGISTER_FAST_TEST(crc_autotest, true, true, test_crc);
diff --git a/app/test/test_cryptodev.c b/app/test/test_cryptodev.c
index fb2af40b99..956268bfcd 100644
--- a/app/test/test_cryptodev.c
+++ b/app/test/test_cryptodev.c
@@ -17535,33 +17535,33 @@ REGISTER_TEST_COMMAND(cryptodev_dpaa2_sec_raw_api_autotest,
 		test_cryptodev_dpaa2_sec_raw_api);
 REGISTER_TEST_COMMAND(cryptodev_dpaa_sec_raw_api_autotest,
 		test_cryptodev_dpaa_sec_raw_api);
-REGISTER_TEST_COMMAND(cryptodev_qat_raw_api_autotest,
+REGISTER_DRIVER_TEST(cryptodev_qat_raw_api_autotest,
 		test_cryptodev_qat_raw_api);
-REGISTER_TEST_COMMAND(cryptodev_qat_autotest, test_cryptodev_qat);
-REGISTER_TEST_COMMAND(cryptodev_aesni_mb_autotest, test_cryptodev_aesni_mb);
-REGISTER_TEST_COMMAND(cryptodev_cpu_aesni_mb_autotest,
+REGISTER_DRIVER_TEST(cryptodev_qat_autotest, test_cryptodev_qat);
+REGISTER_DRIVER_TEST(cryptodev_aesni_mb_autotest, test_cryptodev_aesni_mb);
+REGISTER_DRIVER_TEST(cryptodev_cpu_aesni_mb_autotest,
 	test_cryptodev_cpu_aesni_mb);
-REGISTER_TEST_COMMAND(cryptodev_chacha_poly_mb_autotest,
+REGISTER_DRIVER_TEST(cryptodev_chacha_poly_mb_autotest,
 	test_cryptodev_chacha_poly_mb);
-REGISTER_TEST_COMMAND(cryptodev_openssl_autotest, test_cryptodev_openssl);
-REGISTER_TEST_COMMAND(cryptodev_aesni_gcm_autotest, test_cryptodev_aesni_gcm);
-REGISTER_TEST_COMMAND(cryptodev_cpu_aesni_gcm_autotest,
+REGISTER_DRIVER_TEST(cryptodev_openssl_autotest, test_cryptodev_openssl);
+REGISTER_DRIVER_TEST(cryptodev_aesni_gcm_autotest, test_cryptodev_aesni_gcm);
+REGISTER_DRIVER_TEST(cryptodev_cpu_aesni_gcm_autotest,
 	test_cryptodev_cpu_aesni_gcm);
 REGISTER_TEST_COMMAND(cryptodev_mlx5_autotest, test_cryptodev_mlx5);
-REGISTER_TEST_COMMAND(cryptodev_null_autotest, test_cryptodev_null);
-REGISTER_TEST_COMMAND(cryptodev_sw_snow3g_autotest, test_cryptodev_sw_snow3g);
-REGISTER_TEST_COMMAND(cryptodev_sw_kasumi_autotest, test_cryptodev_sw_kasumi);
-REGISTER_TEST_COMMAND(cryptodev_sw_zuc_autotest, test_cryptodev_sw_zuc);
-REGISTER_TEST_COMMAND(cryptodev_sw_armv8_autotest, test_cryptodev_armv8);
-REGISTER_TEST_COMMAND(cryptodev_sw_mvsam_autotest, test_cryptodev_mrvl);
-REGISTER_TEST_COMMAND(cryptodev_dpaa2_sec_autotest, test_cryptodev_dpaa2_sec);
-REGISTER_TEST_COMMAND(cryptodev_dpaa_sec_autotest, test_cryptodev_dpaa_sec);
+REGISTER_DRIVER_TEST(cryptodev_null_autotest, test_cryptodev_null);
+REGISTER_DRIVER_TEST(cryptodev_sw_snow3g_autotest, test_cryptodev_sw_snow3g);
+REGISTER_DRIVER_TEST(cryptodev_sw_kasumi_autotest, test_cryptodev_sw_kasumi);
+REGISTER_DRIVER_TEST(cryptodev_sw_zuc_autotest, test_cryptodev_sw_zuc);
+REGISTER_DRIVER_TEST(cryptodev_sw_armv8_autotest, test_cryptodev_armv8);
+REGISTER_DRIVER_TEST(cryptodev_sw_mvsam_autotest, test_cryptodev_mrvl);
+REGISTER_DRIVER_TEST(cryptodev_dpaa2_sec_autotest, test_cryptodev_dpaa2_sec);
+REGISTER_DRIVER_TEST(cryptodev_dpaa_sec_autotest, test_cryptodev_dpaa_sec);
 REGISTER_TEST_COMMAND(cryptodev_ccp_autotest, test_cryptodev_ccp);
-REGISTER_TEST_COMMAND(cryptodev_uadk_autotest, test_cryptodev_uadk);
+REGISTER_DRIVER_TEST(cryptodev_uadk_autotest, test_cryptodev_uadk);
 REGISTER_TEST_COMMAND(cryptodev_virtio_autotest, test_cryptodev_virtio);
 REGISTER_TEST_COMMAND(cryptodev_octeontx_autotest, test_cryptodev_octeontx);
 REGISTER_TEST_COMMAND(cryptodev_caam_jr_autotest, test_cryptodev_caam_jr);
 REGISTER_TEST_COMMAND(cryptodev_nitrox_autotest, test_cryptodev_nitrox);
 REGISTER_TEST_COMMAND(cryptodev_bcmfs_autotest, test_cryptodev_bcmfs);
-REGISTER_TEST_COMMAND(cryptodev_cn9k_autotest, test_cryptodev_cn9k);
-REGISTER_TEST_COMMAND(cryptodev_cn10k_autotest, test_cryptodev_cn10k);
+REGISTER_DRIVER_TEST(cryptodev_cn9k_autotest, test_cryptodev_cn9k);
+REGISTER_DRIVER_TEST(cryptodev_cn10k_autotest, test_cryptodev_cn10k);
diff --git a/app/test/test_cryptodev_asym.c b/app/test/test_cryptodev_asym.c
index 0ef2642fdd..3d5a73bf89 100644
--- a/app/test/test_cryptodev_asym.c
+++ b/app/test/test_cryptodev_asym.c
@@ -2828,7 +2828,7 @@ test_cryptodev_cn10k_asym(void)
 REGISTER_TEST_COMMAND(cryptodev_openssl_asym_autotest,
 					  test_cryptodev_openssl_asym);
 
-REGISTER_TEST_COMMAND(cryptodev_qat_asym_autotest, test_cryptodev_qat_asym);
+REGISTER_DRIVER_TEST(cryptodev_qat_asym_autotest, test_cryptodev_qat_asym);
 
 REGISTER_TEST_COMMAND(cryptodev_octeontx_asym_autotest,
 					  test_cryptodev_octeontx_asym);
diff --git a/app/test/test_cycles.c b/app/test/test_cycles.c
index 66d11e6db8..a7654de176 100644
--- a/app/test/test_cycles.c
+++ b/app/test/test_cycles.c
@@ -53,4 +53,4 @@ test_user_delay_us(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(user_delay_us, test_user_delay_us);
+REGISTER_FAST_TEST(user_delay_us, true, true, test_user_delay_us);
diff --git a/app/test/test_debug.c b/app/test/test_debug.c
index 2704f5b927..8ad6d40fcb 100644
--- a/app/test/test_debug.c
+++ b/app/test/test_debug.c
@@ -140,4 +140,4 @@ test_debug(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(debug_autotest, test_debug);
+REGISTER_FAST_TEST(debug_autotest, true, true, test_debug);
diff --git a/app/test/test_devargs.c b/app/test/test_devargs.c
index 0a4c34a1ad..f977d44448 100644
--- a/app/test/test_devargs.c
+++ b/app/test/test_devargs.c
@@ -213,4 +213,4 @@ test_devargs(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(devargs_autotest, test_devargs);
+REGISTER_FAST_TEST(devargs_autotest, true, true, test_devargs);
diff --git a/app/test/test_distributor.c b/app/test/test_distributor.c
index 3efa4af104..6cb27f4de1 100644
--- a/app/test/test_distributor.c
+++ b/app/test/test_distributor.c
@@ -952,4 +952,4 @@ test_distributor(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(distributor_autotest, test_distributor);
+REGISTER_FAST_TEST(distributor_autotest, false, true, test_distributor);
diff --git a/app/test/test_distributor_perf.c b/app/test/test_distributor_perf.c
index ee4321486d..ca868451d7 100644
--- a/app/test/test_distributor_perf.c
+++ b/app/test/test_distributor_perf.c
@@ -277,4 +277,4 @@ test_distributor_perf(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(distributor_perf_autotest, test_distributor_perf);
+REGISTER_PERF_TEST(distributor_perf_autotest, test_distributor_perf);
diff --git a/app/test/test_dmadev.c b/app/test/test_dmadev.c
index 0736ff2a18..6ef875e545 100644
--- a/app/test/test_dmadev.c
+++ b/app/test/test_dmadev.c
@@ -941,4 +941,4 @@ test_dma(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(dmadev_autotest, test_dma);
+REGISTER_DRIVER_TEST(dmadev_autotest, test_dma);
diff --git a/app/test/test_eal_flags.c b/app/test/test_eal_flags.c
index 148e9098e8..6cb4b06757 100644
--- a/app/test/test_eal_flags.c
+++ b/app/test/test_eal_flags.c
@@ -1644,15 +1644,15 @@ test_memory_flags(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(eal_flags_c_opt_autotest, test_missing_c_flag);
-REGISTER_TEST_COMMAND(eal_flags_main_opt_autotest, test_main_lcore_flag);
-REGISTER_TEST_COMMAND(eal_flags_n_opt_autotest, test_invalid_n_flag);
-REGISTER_TEST_COMMAND(eal_flags_hpet_autotest, test_no_hpet_flag);
-REGISTER_TEST_COMMAND(eal_flags_no_huge_autotest, test_no_huge_flag);
-REGISTER_TEST_COMMAND(eal_flags_a_opt_autotest, test_allow_flag);
-REGISTER_TEST_COMMAND(eal_flags_b_opt_autotest, test_invalid_b_flag);
-REGISTER_TEST_COMMAND(eal_flags_vdev_opt_autotest, test_invalid_vdev_flag);
-REGISTER_TEST_COMMAND(eal_flags_r_opt_autotest, test_invalid_r_flag);
-REGISTER_TEST_COMMAND(eal_flags_mem_autotest, test_memory_flags);
-REGISTER_TEST_COMMAND(eal_flags_file_prefix_autotest, test_file_prefix);
-REGISTER_TEST_COMMAND(eal_flags_misc_autotest, test_misc_flags);
+REGISTER_FAST_TEST(eal_flags_c_opt_autotest, false, false, test_missing_c_flag);
+REGISTER_FAST_TEST(eal_flags_main_opt_autotest, false, false, test_main_lcore_flag);
+REGISTER_FAST_TEST(eal_flags_n_opt_autotest, false, false, test_invalid_n_flag);
+REGISTER_FAST_TEST(eal_flags_hpet_autotest, false, false, test_no_hpet_flag);
+REGISTER_FAST_TEST(eal_flags_no_huge_autotest, false, false, test_no_huge_flag);
+REGISTER_FAST_TEST(eal_flags_a_opt_autotest, false, false, test_allow_flag);
+REGISTER_FAST_TEST(eal_flags_b_opt_autotest, false, false, test_invalid_b_flag);
+REGISTER_FAST_TEST(eal_flags_vdev_opt_autotest, false, false, test_invalid_vdev_flag);
+REGISTER_FAST_TEST(eal_flags_r_opt_autotest, false, false, test_invalid_r_flag);
+REGISTER_FAST_TEST(eal_flags_mem_autotest, false, false, test_memory_flags);
+REGISTER_FAST_TEST(eal_flags_file_prefix_autotest, false, false, test_file_prefix);
+REGISTER_FAST_TEST(eal_flags_misc_autotest, false, false, test_misc_flags);
diff --git a/app/test/test_eal_fs.c b/app/test/test_eal_fs.c
index b3686edcb4..8cd287fa9d 100644
--- a/app/test/test_eal_fs.c
+++ b/app/test/test_eal_fs.c
@@ -185,4 +185,4 @@ test_eal_fs(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(eal_fs_autotest, test_eal_fs);
+REGISTER_FAST_TEST(eal_fs_autotest, true, true, test_eal_fs);
diff --git a/app/test/test_efd.c b/app/test/test_efd.c
index fa29e8f97a..1c0986b9bc 100644
--- a/app/test/test_efd.c
+++ b/app/test/test_efd.c
@@ -473,4 +473,4 @@ test_efd(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(efd_autotest, test_efd);
+REGISTER_PERF_TEST(efd_autotest, test_efd);
diff --git a/app/test/test_efd_perf.c b/app/test/test_efd_perf.c
index 4d04ed93e3..b212e96767 100644
--- a/app/test/test_efd_perf.c
+++ b/app/test/test_efd_perf.c
@@ -393,4 +393,4 @@ test_efd_perf(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(efd_perf_autotest, test_efd_perf);
+REGISTER_PERF_TEST(efd_perf_autotest, test_efd_perf);
diff --git a/app/test/test_errno.c b/app/test/test_errno.c
index 0db4fbc8b3..b429962fb9 100644
--- a/app/test/test_errno.c
+++ b/app/test/test_errno.c
@@ -94,4 +94,4 @@ test_errno(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(errno_autotest, test_errno);
+REGISTER_FAST_TEST(errno_autotest, true, true, test_errno);
diff --git a/app/test/test_ethdev_link.c b/app/test/test_ethdev_link.c
index ab52385a12..f063a5fe26 100644
--- a/app/test/test_ethdev_link.c
+++ b/app/test/test_ethdev_link.c
@@ -167,4 +167,4 @@ test_link_status(void)
 	return unit_test_suite_runner(&link_status_testsuite);
 }
 
-REGISTER_TEST_COMMAND(ethdev_link_status, test_link_status);
+REGISTER_FAST_TEST(ethdev_link_status, true, true, test_link_status);
diff --git a/app/test/test_event_ring.c b/app/test/test_event_ring.c
index bc4a6e73f5..3bfb5109b7 100644
--- a/app/test/test_event_ring.c
+++ b/app/test/test_event_ring.c
@@ -256,4 +256,4 @@ test_event_ring(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(event_ring_autotest, test_event_ring);
+REGISTER_FAST_TEST(event_ring_autotest, true, true, test_event_ring);
diff --git a/app/test/test_eventdev.c b/app/test/test_eventdev.c
index 336529038e..29354a24c9 100644
--- a/app/test/test_eventdev.c
+++ b/app/test/test_eventdev.c
@@ -1256,7 +1256,7 @@ test_eventdev_selftest_cn10k(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(eventdev_common_autotest, test_eventdev_common);
+REGISTER_FAST_TEST(eventdev_common_autotest, true, true, test_eventdev_common);
 
 #ifndef RTE_EXEC_ENV_WINDOWS
 REGISTER_TEST_COMMAND(eventdev_selftest_sw, test_eventdev_selftest_sw);
diff --git a/app/test/test_fbarray.c b/app/test/test_fbarray.c
index a691bf4458..26a51e2a3e 100644
--- a/app/test/test_fbarray.c
+++ b/app/test/test_fbarray.c
@@ -733,4 +733,4 @@ test_fbarray(void)
 	return unit_test_suite_runner(&fbarray_test_suite);
 }
 
-REGISTER_TEST_COMMAND(fbarray_autotest, test_fbarray);
+REGISTER_FAST_TEST(fbarray_autotest, true, true, test_fbarray);
diff --git a/app/test/test_fib.c b/app/test/test_fib.c
index eb69d6e2fd..45dccca1f6 100644
--- a/app/test/test_fib.c
+++ b/app/test/test_fib.c
@@ -415,5 +415,5 @@ test_slow_fib(void)
 	return unit_test_suite_runner(&fib_slow_tests);
 }
 
-REGISTER_TEST_COMMAND(fib_autotest, test_fib);
-REGISTER_TEST_COMMAND(fib_slow_autotest, test_slow_fib);
+REGISTER_FAST_TEST(fib_autotest, true, true, test_fib);
+REGISTER_PERF_TEST(fib_slow_autotest, test_slow_fib);
diff --git a/app/test/test_fib6.c b/app/test/test_fib6.c
index 15ad09178a..2f836238fb 100644
--- a/app/test/test_fib6.c
+++ b/app/test/test_fib6.c
@@ -424,5 +424,5 @@ test_slow_fib6(void)
 	return unit_test_suite_runner(&fib6_slow_tests);
 }
 
-REGISTER_TEST_COMMAND(fib6_autotest, test_fib6);
-REGISTER_TEST_COMMAND(fib6_slow_autotest, test_slow_fib6);
+REGISTER_FAST_TEST(fib6_autotest, true, true, test_fib6);
+REGISTER_PERF_TEST(fib6_slow_autotest, test_slow_fib6);
diff --git a/app/test/test_fib6_perf.c b/app/test/test_fib6_perf.c
index add20c2331..a7abc46af9 100644
--- a/app/test/test_fib6_perf.c
+++ b/app/test/test_fib6_perf.c
@@ -156,4 +156,4 @@ test_fib6_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(fib6_perf_autotest, test_fib6_perf);
+REGISTER_PERF_TEST(fib6_perf_autotest, test_fib6_perf);
diff --git a/app/test/test_fib_perf.c b/app/test/test_fib_perf.c
index b56293e64f..a9119c1bb0 100644
--- a/app/test/test_fib_perf.c
+++ b/app/test/test_fib_perf.c
@@ -409,4 +409,4 @@ test_fib_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(fib_perf_autotest, test_fib_perf);
+REGISTER_PERF_TEST(fib_perf_autotest, test_fib_perf);
diff --git a/app/test/test_func_reentrancy.c b/app/test/test_func_reentrancy.c
index ae9de6f93d..9296de23b7 100644
--- a/app/test/test_func_reentrancy.c
+++ b/app/test/test_func_reentrancy.c
@@ -507,4 +507,4 @@ test_func_reentrancy(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(func_reentrancy_autotest, test_func_reentrancy);
+REGISTER_FAST_TEST(func_reentrancy_autotest, false, true, test_func_reentrancy);
diff --git a/app/test/test_hash.c b/app/test/test_hash.c
index 3e45afaa67..d586878a22 100644
--- a/app/test/test_hash.c
+++ b/app/test/test_hash.c
@@ -2264,4 +2264,4 @@ test_hash(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(hash_autotest, test_hash);
+REGISTER_FAST_TEST(hash_autotest, true, true, test_hash);
diff --git a/app/test/test_hash_functions.c b/app/test/test_hash_functions.c
index 76d51b6e71..70820d1f19 100644
--- a/app/test/test_hash_functions.c
+++ b/app/test/test_hash_functions.c
@@ -290,4 +290,4 @@ test_hash_functions(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(hash_functions_autotest, test_hash_functions);
+REGISTER_PERF_TEST(hash_functions_autotest, test_hash_functions);
diff --git a/app/test/test_hash_multiwriter.c b/app/test/test_hash_multiwriter.c
index 0c5a8ca186..dd5ca677b9 100644
--- a/app/test/test_hash_multiwriter.c
+++ b/app/test/test_hash_multiwriter.c
@@ -287,4 +287,4 @@ test_hash_multiwriter_main(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(hash_multiwriter_autotest, test_hash_multiwriter_main);
+REGISTER_PERF_TEST(hash_multiwriter_autotest, test_hash_multiwriter_main);
diff --git a/app/test/test_hash_perf.c b/app/test/test_hash_perf.c
index 14a1283aba..d66b96e5ce 100644
--- a/app/test/test_hash_perf.c
+++ b/app/test/test_hash_perf.c
@@ -757,4 +757,4 @@ test_hash_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(hash_perf_autotest, test_hash_perf);
+REGISTER_PERF_TEST(hash_perf_autotest, test_hash_perf);
diff --git a/app/test/test_hash_readwrite.c b/app/test/test_hash_readwrite.c
index 6373e62d33..74ca13912f 100644
--- a/app/test/test_hash_readwrite.c
+++ b/app/test/test_hash_readwrite.c
@@ -760,5 +760,5 @@ test_hash_rw_func_main(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(hash_readwrite_func_autotest, test_hash_rw_func_main);
-REGISTER_TEST_COMMAND(hash_readwrite_perf_autotest, test_hash_rw_perf_main);
+REGISTER_FAST_TEST(hash_readwrite_func_autotest, false, true, test_hash_rw_func_main);
+REGISTER_PERF_TEST(hash_readwrite_perf_autotest, test_hash_rw_perf_main);
diff --git a/app/test/test_hash_readwrite_lf_perf.c b/app/test/test_hash_readwrite_lf_perf.c
index cf86046a2f..5d18850e19 100644
--- a/app/test/test_hash_readwrite_lf_perf.c
+++ b/app/test/test_hash_readwrite_lf_perf.c
@@ -1579,5 +1579,5 @@ test_hash_readwrite_lf_perf_main(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(hash_readwrite_lf_perf_autotest,
+REGISTER_PERF_TEST(hash_readwrite_lf_perf_autotest,
 	test_hash_readwrite_lf_perf_main);
diff --git a/app/test/test_interrupts.c b/app/test/test_interrupts.c
index b59ab42699..3952f9685f 100644
--- a/app/test/test_interrupts.c
+++ b/app/test/test_interrupts.c
@@ -594,4 +594,4 @@ test_interrupt(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(interrupt_autotest, test_interrupt);
+REGISTER_FAST_TEST(interrupt_autotest, true, true, test_interrupt);
diff --git a/app/test/test_ipfrag.c b/app/test/test_ipfrag.c
index 402ce361c1..8e4df220a2 100644
--- a/app/test/test_ipfrag.c
+++ b/app/test/test_ipfrag.c
@@ -510,4 +510,4 @@ test_ipfrag(void)
 }
 
 
-REGISTER_TEST_COMMAND(ipfrag_autotest, test_ipfrag);
+REGISTER_FAST_TEST(ipfrag_autotest, false, true, test_ipfrag);
diff --git a/app/test/test_ipsec.c b/app/test/test_ipsec.c
index c2a52ec305..6cb1bac1e7 100644
--- a/app/test/test_ipsec.c
+++ b/app/test/test_ipsec.c
@@ -2532,4 +2532,4 @@ test_ipsec(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(ipsec_autotest, test_ipsec);
+REGISTER_FAST_TEST(ipsec_autotest, true, true, test_ipsec);
diff --git a/app/test/test_ipsec_perf.c b/app/test/test_ipsec_perf.c
index b221b7fc32..a32a2086e9 100644
--- a/app/test/test_ipsec_perf.c
+++ b/app/test/test_ipsec_perf.c
@@ -631,4 +631,4 @@ test_libipsec_perf(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(ipsec_perf_autotest, test_libipsec_perf);
+REGISTER_PERF_TEST(ipsec_perf_autotest, test_libipsec_perf);
diff --git a/app/test/test_kvargs.c b/app/test/test_kvargs.c
index b7b97a0dd9..7a60cac4c1 100644
--- a/app/test/test_kvargs.c
+++ b/app/test/test_kvargs.c
@@ -292,4 +292,4 @@ test_kvargs(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(kvargs_autotest, test_kvargs);
+REGISTER_FAST_TEST(kvargs_autotest, true, true, test_kvargs);
diff --git a/app/test/test_lcores.c b/app/test/test_lcores.c
index 2c945b0136..87dcf0590e 100644
--- a/app/test/test_lcores.c
+++ b/app/test/test_lcores.c
@@ -412,4 +412,4 @@ test_lcores(void)
 	return TEST_SUCCESS;
 }
 
-REGISTER_TEST_COMMAND(lcores_autotest, test_lcores);
+REGISTER_FAST_TEST(lcores_autotest, true, true, test_lcores);
diff --git a/app/test/test_logs.c b/app/test/test_logs.c
index 8da8824bee..43b09704a3 100644
--- a/app/test/test_logs.c
+++ b/app/test/test_logs.c
@@ -158,4 +158,4 @@ test_logs(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(logs_autotest, test_logs);
+REGISTER_FAST_TEST(logs_autotest, true, true, test_logs);
diff --git a/app/test/test_lpm.c b/app/test/test_lpm.c
index 37b460af3a..b93e11d700 100644
--- a/app/test/test_lpm.c
+++ b/app/test/test_lpm.c
@@ -1584,4 +1584,4 @@ test_lpm(void)
 	return global_status;
 }
 
-REGISTER_TEST_COMMAND(lpm_autotest, test_lpm);
+REGISTER_FAST_TEST(lpm_autotest, true, true, test_lpm);
diff --git a/app/test/test_lpm6.c b/app/test/test_lpm6.c
index b6b6f8615e..1d8a0afa11 100644
--- a/app/test/test_lpm6.c
+++ b/app/test/test_lpm6.c
@@ -1793,4 +1793,4 @@ test_lpm6(void)
 	return global_status;
 }
 
-REGISTER_TEST_COMMAND(lpm6_autotest, test_lpm6);
+REGISTER_FAST_TEST(lpm6_autotest, true, true, test_lpm6);
diff --git a/app/test/test_lpm6_perf.c b/app/test/test_lpm6_perf.c
index 5b684686a6..8a49f74c84 100644
--- a/app/test/test_lpm6_perf.c
+++ b/app/test/test_lpm6_perf.c
@@ -161,4 +161,4 @@ test_lpm6_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(lpm6_perf_autotest, test_lpm6_perf);
+REGISTER_PERF_TEST(lpm6_perf_autotest, test_lpm6_perf);
diff --git a/app/test/test_lpm_perf.c b/app/test/test_lpm_perf.c
index e72437ba38..15ff396dd0 100644
--- a/app/test/test_lpm_perf.c
+++ b/app/test/test_lpm_perf.c
@@ -760,4 +760,4 @@ test_lpm_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(lpm_perf_autotest, test_lpm_perf);
+REGISTER_PERF_TEST(lpm_perf_autotest, test_lpm_perf);
diff --git a/app/test/test_malloc.c b/app/test/test_malloc.c
index ff081dd931..cd579c503c 100644
--- a/app/test/test_malloc.c
+++ b/app/test/test_malloc.c
@@ -1091,4 +1091,4 @@ test_malloc(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(malloc_autotest, test_malloc);
+REGISTER_FAST_TEST(malloc_autotest, false, true, test_malloc);
diff --git a/app/test/test_malloc_perf.c b/app/test/test_malloc_perf.c
index 9bd1662981..a99bfd8531 100644
--- a/app/test/test_malloc_perf.c
+++ b/app/test/test_malloc_perf.c
@@ -171,4 +171,4 @@ test_malloc_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(malloc_perf_autotest, test_malloc_perf);
+REGISTER_PERF_TEST(malloc_perf_autotest, test_malloc_perf);
diff --git a/app/test/test_mbuf.c b/app/test/test_mbuf.c
index efac01806b..d7393df7eb 100644
--- a/app/test/test_mbuf.c
+++ b/app/test/test_mbuf.c
@@ -2955,4 +2955,4 @@ test_mbuf(void)
 }
 #undef GOTO_FAIL
 
-REGISTER_TEST_COMMAND(mbuf_autotest, test_mbuf);
+REGISTER_FAST_TEST(mbuf_autotest, false, true, test_mbuf);
diff --git a/app/test/test_mcslock.c b/app/test/test_mcslock.c
index 52e45e7e2a..d522fc6ece 100644
--- a/app/test/test_mcslock.c
+++ b/app/test/test_mcslock.c
@@ -241,4 +241,4 @@ test_mcslock(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(mcslock_autotest, test_mcslock);
+REGISTER_FAST_TEST(mcslock_autotest, false, true, test_mcslock);
diff --git a/app/test/test_member.c b/app/test/test_member.c
index 4a93f8bff4..5a4d2750db 100644
--- a/app/test/test_member.c
+++ b/app/test/test_member.c
@@ -996,4 +996,4 @@ test_member(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(member_autotest, test_member);
+REGISTER_FAST_TEST(member_autotest, true, true, test_member);
diff --git a/app/test/test_member_perf.c b/app/test/test_member_perf.c
index 2f79888fbd..db6b8a18ef 100644
--- a/app/test/test_member_perf.c
+++ b/app/test/test_member_perf.c
@@ -780,4 +780,4 @@ test_member_perf(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(member_perf_autotest, test_member_perf);
+REGISTER_PERF_TEST(member_perf_autotest, test_member_perf);
diff --git a/app/test/test_memcpy.c b/app/test/test_memcpy.c
index 1ab86f4967..802dc4631b 100644
--- a/app/test/test_memcpy.c
+++ b/app/test/test_memcpy.c
@@ -129,4 +129,4 @@ test_memcpy(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(memcpy_autotest, test_memcpy);
+REGISTER_FAST_TEST(memcpy_autotest, true, true, test_memcpy);
diff --git a/app/test/test_memcpy_perf.c b/app/test/test_memcpy_perf.c
index 3727c160e6..5c05a84619 100644
--- a/app/test/test_memcpy_perf.c
+++ b/app/test/test_memcpy_perf.c
@@ -348,4 +348,4 @@ test_memcpy_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(memcpy_perf_autotest, test_memcpy_perf);
+REGISTER_PERF_TEST(memcpy_perf_autotest, test_memcpy_perf);
diff --git a/app/test/test_memory.c b/app/test/test_memory.c
index 440e5ef838..ea37f62338 100644
--- a/app/test/test_memory.c
+++ b/app/test/test_memory.c
@@ -110,4 +110,4 @@ test_memory(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(memory_autotest, test_memory);
+REGISTER_FAST_TEST(memory_autotest, false, true, test_memory);
diff --git a/app/test/test_mempool.c b/app/test/test_mempool.c
index 8e493eda47..ad7ebd6363 100644
--- a/app/test/test_mempool.c
+++ b/app/test/test_mempool.c
@@ -1043,4 +1043,4 @@ test_mempool(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(mempool_autotest, test_mempool);
+REGISTER_FAST_TEST(mempool_autotest, false, true, test_mempool);
diff --git a/app/test/test_mempool_perf.c b/app/test/test_mempool_perf.c
index ce7c6241ab..96de347f04 100644
--- a/app/test/test_mempool_perf.c
+++ b/app/test/test_mempool_perf.c
@@ -437,4 +437,4 @@ test_mempool_perf(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(mempool_perf_autotest, test_mempool_perf);
+REGISTER_PERF_TEST(mempool_perf_autotest, test_mempool_perf);
diff --git a/app/test/test_memzone.c b/app/test/test_memzone.c
index f10f4fd9cd..37ae7afc95 100644
--- a/app/test/test_memzone.c
+++ b/app/test/test_memzone.c
@@ -1163,4 +1163,4 @@ test_memzone(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(memzone_autotest, test_memzone);
+REGISTER_FAST_TEST(memzone_autotest, false, true, test_memzone);
diff --git a/app/test/test_meter.c b/app/test/test_meter.c
index 15d5a4839b..6241b75ba0 100644
--- a/app/test/test_meter.c
+++ b/app/test/test_meter.c
@@ -713,4 +713,4 @@ test_meter(void)
 
 }
 
-REGISTER_TEST_COMMAND(meter_autotest, test_meter);
+REGISTER_FAST_TEST(meter_autotest, true, true, test_meter);
diff --git a/app/test/test_mp_secondary.c b/app/test/test_mp_secondary.c
index ad47d578f2..f3694530a8 100644
--- a/app/test/test_mp_secondary.c
+++ b/app/test/test_mp_secondary.c
@@ -223,4 +223,4 @@ test_mp_secondary(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(multiprocess_autotest, test_mp_secondary);
+REGISTER_FAST_TEST(multiprocess_autotest, false, false, test_mp_secondary);
diff --git a/app/test/test_per_lcore.c b/app/test/test_per_lcore.c
index 129578d1a3..63c5c80c24 100644
--- a/app/test/test_per_lcore.c
+++ b/app/test/test_per_lcore.c
@@ -105,4 +105,4 @@ test_per_lcore(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(per_lcore_autotest, test_per_lcore);
+REGISTER_FAST_TEST(per_lcore_autotest, true, true, test_per_lcore);
diff --git a/app/test/test_pflock.c b/app/test/test_pflock.c
index 38da6bce27..5f77b158c8 100644
--- a/app/test/test_pflock.c
+++ b/app/test/test_pflock.c
@@ -193,4 +193,4 @@ test_pflock(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(pflock_autotest, test_pflock);
+REGISTER_FAST_TEST(pflock_autotest, true, true, test_pflock);
diff --git a/app/test/test_pie.c b/app/test/test_pie.c
index a3c0f97c9d..8036bac1e6 100644
--- a/app/test/test_pie.c
+++ b/app/test/test_pie.c
@@ -1087,6 +1087,6 @@ test_pie_all(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(pie_autotest, test_pie);
-REGISTER_TEST_COMMAND(pie_perf, test_pie_perf);
-REGISTER_TEST_COMMAND(pie_all, test_pie_all);
+REGISTER_FAST_TEST(pie_autotest, true, true, test_pie);
+REGISTER_PERF_TEST(pie_perf, test_pie_perf);
+REGISTER_PERF_TEST(pie_all, test_pie_all);
diff --git a/app/test/test_pmd_perf.c b/app/test/test_pmd_perf.c
index 3ef590cb51..f6d97f21c9 100644
--- a/app/test/test_pmd_perf.c
+++ b/app/test/test_pmd_perf.c
@@ -899,4 +899,4 @@ test_set_rxtx_sc(cmdline_fixed_string_t type)
 	return -1;
 }
 
-REGISTER_TEST_COMMAND(pmd_perf_autotest, test_pmd_perf);
+REGISTER_PERF_TEST(pmd_perf_autotest, test_pmd_perf);
diff --git a/app/test/test_power.c b/app/test/test_power.c
index b7b5561348..02ebc54d19 100644
--- a/app/test/test_power.c
+++ b/app/test/test_power.c
@@ -172,4 +172,4 @@ test_power(void)
 }
 #endif
 
-REGISTER_TEST_COMMAND(power_autotest, test_power);
+REGISTER_FAST_TEST(power_autotest, true, true, test_power);
diff --git a/app/test/test_power_cpufreq.c b/app/test/test_power_cpufreq.c
index 4d013cd7bb..10755a0d41 100644
--- a/app/test/test_power_cpufreq.c
+++ b/app/test/test_power_cpufreq.c
@@ -703,5 +703,5 @@ test_power_caps(void)
 
 #endif
 
-REGISTER_TEST_COMMAND(power_cpufreq_autotest, test_power_cpufreq);
+REGISTER_FAST_TEST(power_cpufreq_autotest, false, true, test_power_cpufreq);
 REGISTER_TEST_COMMAND(power_caps_autotest, test_power_caps);
diff --git a/app/test/test_power_intel_uncore.c b/app/test/test_power_intel_uncore.c
index 31163af84e..6fc7f23bbf 100644
--- a/app/test/test_power_intel_uncore.c
+++ b/app/test/test_power_intel_uncore.c
@@ -298,4 +298,4 @@ test_power_intel_uncore(void)
 }
 #endif
 
-REGISTER_TEST_COMMAND(power_intel_uncore_autotest, test_power_intel_uncore);
+REGISTER_FAST_TEST(power_intel_uncore_autotest, true, true, test_power_intel_uncore);
diff --git a/app/test/test_power_kvm_vm.c b/app/test/test_power_kvm_vm.c
index cc66b7a8a0..464e06002e 100644
--- a/app/test/test_power_kvm_vm.c
+++ b/app/test/test_power_kvm_vm.c
@@ -299,4 +299,4 @@ test_power_kvm_vm(void)
 }
 #endif
 
-REGISTER_TEST_COMMAND(power_kvm_vm_autotest, test_power_kvm_vm);
+REGISTER_FAST_TEST(power_kvm_vm_autotest, false, true, test_power_kvm_vm);
diff --git a/app/test/test_prefetch.c b/app/test/test_prefetch.c
index 7b4a8e4144..46e6828e6a 100644
--- a/app/test/test_prefetch.c
+++ b/app/test/test_prefetch.c
@@ -35,4 +35,4 @@ test_prefetch(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(prefetch_autotest, test_prefetch);
+REGISTER_FAST_TEST(prefetch_autotest, true, true, test_prefetch);
diff --git a/app/test/test_rand_perf.c b/app/test/test_rand_perf.c
index 26fb1d9a58..30204e12c0 100644
--- a/app/test/test_rand_perf.c
+++ b/app/test/test_rand_perf.c
@@ -96,4 +96,4 @@ test_rand_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(rand_perf_autotest, test_rand_perf);
+REGISTER_PERF_TEST(rand_perf_autotest, test_rand_perf);
diff --git a/app/test/test_rcu_qsbr.c b/app/test/test_rcu_qsbr.c
index 70404e89e6..72d8e0377e 100644
--- a/app/test/test_rcu_qsbr.c
+++ b/app/test/test_rcu_qsbr.c
@@ -1418,4 +1418,4 @@ test_rcu_qsbr_main(void)
 	return -1;
 }
 
-REGISTER_TEST_COMMAND(rcu_qsbr_autotest, test_rcu_qsbr_main);
+REGISTER_FAST_TEST(rcu_qsbr_autotest, true, true, test_rcu_qsbr_main);
diff --git a/app/test/test_rcu_qsbr_perf.c b/app/test/test_rcu_qsbr_perf.c
index b15e5cef88..ce88a7333c 100644
--- a/app/test/test_rcu_qsbr_perf.c
+++ b/app/test/test_rcu_qsbr_perf.c
@@ -690,4 +690,4 @@ test_rcu_qsbr_main(void)
 	return -1;
 }
 
-REGISTER_TEST_COMMAND(rcu_qsbr_perf_autotest, test_rcu_qsbr_main);
+REGISTER_PERF_TEST(rcu_qsbr_perf_autotest, test_rcu_qsbr_main);
diff --git a/app/test/test_reassembly_perf.c b/app/test/test_reassembly_perf.c
index c11b65291f..4b4929d777 100644
--- a/app/test/test_reassembly_perf.c
+++ b/app/test/test_reassembly_perf.c
@@ -1000,4 +1000,4 @@ test_reassembly_perf(void)
 	return TEST_SUCCESS;
 }
 
-REGISTER_TEST_COMMAND(reassembly_perf_autotest, test_reassembly_perf);
+REGISTER_PERF_TEST(reassembly_perf_autotest, test_reassembly_perf);
diff --git a/app/test/test_reciprocal_division.c b/app/test/test_reciprocal_division.c
index 8ea9b1d24d..fb52b2d5a1 100644
--- a/app/test/test_reciprocal_division.c
+++ b/app/test/test_reciprocal_division.c
@@ -164,4 +164,4 @@ test_reciprocal(void)
 	return result;
 }
 
-REGISTER_TEST_COMMAND(reciprocal_division, test_reciprocal);
+REGISTER_PERF_TEST(reciprocal_division, test_reciprocal);
diff --git a/app/test/test_reciprocal_division_perf.c b/app/test/test_reciprocal_division_perf.c
index 4f625873e5..cf96d46a22 100644
--- a/app/test/test_reciprocal_division_perf.c
+++ b/app/test/test_reciprocal_division_perf.c
@@ -205,4 +205,4 @@ test_reciprocal_division_perf(void)
 	return result;
 }
 
-REGISTER_TEST_COMMAND(reciprocal_division_perf, test_reciprocal_division_perf);
+REGISTER_PERF_TEST(reciprocal_division_perf, test_reciprocal_division_perf);
diff --git a/app/test/test_red.c b/app/test/test_red.c
index 84c292f8d8..aa7538d51a 100644
--- a/app/test/test_red.c
+++ b/app/test/test_red.c
@@ -1878,5 +1878,5 @@ test_red_all(void)
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
 REGISTER_TEST_COMMAND(red_autotest, test_red);
-REGISTER_TEST_COMMAND(red_perf, test_red_perf);
-REGISTER_TEST_COMMAND(red_all, test_red_all);
+REGISTER_PERF_TEST(red_perf, test_red_perf);
+REGISTER_PERF_TEST(red_all, test_red_all);
diff --git a/app/test/test_reorder.c b/app/test/test_reorder.c
index c188f6ce67..501780cb26 100644
--- a/app/test/test_reorder.c
+++ b/app/test/test_reorder.c
@@ -548,4 +548,4 @@ test_reorder(void)
 }
 
 
-REGISTER_TEST_COMMAND(reorder_autotest, test_reorder);
+REGISTER_FAST_TEST(reorder_autotest, true, true, test_reorder);
diff --git a/app/test/test_rib.c b/app/test/test_rib.c
index 65b6856410..c7454f2c47 100644
--- a/app/test/test_rib.c
+++ b/app/test/test_rib.c
@@ -363,5 +363,5 @@ test_slow_rib(void)
 	return unit_test_suite_runner(&rib_slow_tests);
 }
 
-REGISTER_TEST_COMMAND(rib_autotest, test_rib);
-REGISTER_TEST_COMMAND(rib_slow_autotest, test_slow_rib);
+REGISTER_FAST_TEST(rib_autotest, true, true, test_rib);
+REGISTER_PERF_TEST(rib_slow_autotest, test_slow_rib);
diff --git a/app/test/test_rib6.c b/app/test/test_rib6.c
index 336b779d2e..33596fddb4 100644
--- a/app/test/test_rib6.c
+++ b/app/test/test_rib6.c
@@ -367,5 +367,5 @@ test_slow_rib6(void)
 	return unit_test_suite_runner(&rib6_slow_tests);
 }
 
-REGISTER_TEST_COMMAND(rib6_autotest, test_rib6);
-REGISTER_TEST_COMMAND(rib6_slow_autotest, test_slow_rib6);
+REGISTER_FAST_TEST(rib6_autotest, true, true, test_rib6);
+REGISTER_PERF_TEST(rib6_slow_autotest, test_slow_rib6);
diff --git a/app/test/test_ring.c b/app/test/test_ring.c
index bde33ab4a1..ba1fec1de3 100644
--- a/app/test/test_ring.c
+++ b/app/test/test_ring.c
@@ -1241,4 +1241,4 @@ test_ring(void)
 	return -1;
 }
 
-REGISTER_TEST_COMMAND(ring_autotest, test_ring);
+REGISTER_FAST_TEST(ring_autotest, true, true, test_ring);
diff --git a/app/test/test_ring_perf.c b/app/test/test_ring_perf.c
index 3972fd9db3..d7c5a4c30b 100644
--- a/app/test/test_ring_perf.c
+++ b/app/test/test_ring_perf.c
@@ -579,4 +579,4 @@ test_ring_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(ring_perf_autotest, test_ring_perf);
+REGISTER_PERF_TEST(ring_perf_autotest, test_ring_perf);
diff --git a/app/test/test_rwlock.c b/app/test/test_rwlock.c
index 4ae0bf8deb..50798958d7 100644
--- a/app/test/test_rwlock.c
+++ b/app/test/test_rwlock.c
@@ -506,7 +506,7 @@ try_rwlock_test_rde_wro(void)
 	return process_try_lcore_stats();
 }
 
-REGISTER_TEST_COMMAND(rwlock_test1_autotest, rwlock_test1);
-REGISTER_TEST_COMMAND(rwlock_rda_autotest, try_rwlock_test_rda);
-REGISTER_TEST_COMMAND(rwlock_rds_wrm_autotest, try_rwlock_test_rds_wrm);
-REGISTER_TEST_COMMAND(rwlock_rde_wro_autotest, try_rwlock_test_rde_wro);
+REGISTER_FAST_TEST(rwlock_test1_autotest, true, true, rwlock_test1);
+REGISTER_FAST_TEST(rwlock_rda_autotest, true, true, try_rwlock_test_rda);
+REGISTER_FAST_TEST(rwlock_rds_wrm_autotest, true, true, try_rwlock_test_rds_wrm);
+REGISTER_FAST_TEST(rwlock_rde_wro_autotest, true, true, try_rwlock_test_rde_wro);
diff --git a/app/test/test_sched.c b/app/test/test_sched.c
index ddec572447..6daee90bca 100644
--- a/app/test/test_sched.c
+++ b/app/test/test_sched.c
@@ -215,4 +215,4 @@ test_sched(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(sched_autotest, test_sched);
+REGISTER_FAST_TEST(sched_autotest, true, true, test_sched);
diff --git a/app/test/test_security.c b/app/test/test_security.c
index 4783cd0663..23fc7ffb4f 100644
--- a/app/test/test_security.c
+++ b/app/test/test_security.c
@@ -2314,4 +2314,4 @@ test_security(void)
 	return unit_test_suite_runner(&security_testsuite);
 }
 
-REGISTER_TEST_COMMAND(security_autotest, test_security);
+REGISTER_FAST_TEST(security_autotest, false, true, test_security);
diff --git a/app/test/test_seqlock.c b/app/test/test_seqlock.c
index d26d2c010e..873bd60453 100644
--- a/app/test/test_seqlock.c
+++ b/app/test/test_seqlock.c
@@ -187,4 +187,4 @@ test_seqlock(void)
 	return rc;
 }
 
-REGISTER_TEST_COMMAND(seqlock_autotest, test_seqlock);
+REGISTER_FAST_TEST(seqlock_autotest, true, true, test_seqlock);
diff --git a/app/test/test_service_cores.c b/app/test/test_service_cores.c
index 422d2a83e8..c12d52d8f1 100644
--- a/app/test/test_service_cores.c
+++ b/app/test/test_service_cores.c
@@ -1040,7 +1040,7 @@ test_service_common(void)
 	return unit_test_suite_runner(&service_tests);
 }
 
-REGISTER_TEST_COMMAND(service_autotest, test_service_common);
+REGISTER_FAST_TEST(service_autotest, true, true, test_service_common);
 
 static struct unit_test_suite service_perf_tests  = {
 	.suite_name = "service core performance test suite",
@@ -1062,4 +1062,4 @@ test_service_perf(void)
 	return unit_test_suite_runner(&service_perf_tests);
 }
 
-REGISTER_TEST_COMMAND(service_perf_autotest, test_service_perf);
+REGISTER_PERF_TEST(service_perf_autotest, test_service_perf);
diff --git a/app/test/test_spinlock.c b/app/test/test_spinlock.c
index 3f59372300..9a481f2718 100644
--- a/app/test/test_spinlock.c
+++ b/app/test/test_spinlock.c
@@ -302,4 +302,4 @@ test_spinlock(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(spinlock_autotest, test_spinlock);
+REGISTER_FAST_TEST(spinlock_autotest, true, true, test_spinlock);
diff --git a/app/test/test_stack.c b/app/test/test_stack.c
index bc38961433..9150cc9fed 100644
--- a/app/test/test_stack.c
+++ b/app/test/test_stack.c
@@ -379,5 +379,5 @@ test_lf_stack(void)
 #endif
 }
 
-REGISTER_TEST_COMMAND(stack_autotest, test_stack);
-REGISTER_TEST_COMMAND(stack_lf_autotest, test_lf_stack);
+REGISTER_FAST_TEST(stack_autotest, false, true, test_stack);
+REGISTER_FAST_TEST(stack_lf_autotest, false, true, test_lf_stack);
diff --git a/app/test/test_stack_perf.c b/app/test/test_stack_perf.c
index 1eae00a334..c5e1caa036 100644
--- a/app/test/test_stack_perf.c
+++ b/app/test/test_stack_perf.c
@@ -354,5 +354,5 @@ test_lf_stack_perf(void)
 #endif
 }
 
-REGISTER_TEST_COMMAND(stack_perf_autotest, test_stack_perf);
-REGISTER_TEST_COMMAND(stack_lf_perf_autotest, test_lf_stack_perf);
+REGISTER_PERF_TEST(stack_perf_autotest, test_stack_perf);
+REGISTER_PERF_TEST(stack_lf_perf_autotest, test_lf_stack_perf);
diff --git a/app/test/test_string_fns.c b/app/test/test_string_fns.c
index 5e105d2bb9..ad41106df1 100644
--- a/app/test/test_string_fns.c
+++ b/app/test/test_string_fns.c
@@ -182,4 +182,4 @@ test_string_fns(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(string_autotest, test_string_fns);
+REGISTER_FAST_TEST(string_autotest, true, true, test_string_fns);
diff --git a/app/test/test_tailq.c b/app/test/test_tailq.c
index 9520219b0a..2ff2877344 100644
--- a/app/test/test_tailq.c
+++ b/app/test/test_tailq.c
@@ -125,4 +125,4 @@ test_tailq(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(tailq_autotest, test_tailq);
+REGISTER_FAST_TEST(tailq_autotest, true, true, test_tailq);
diff --git a/app/test/test_thash.c b/app/test/test_thash.c
index 53d9611e18..65d42fd900 100644
--- a/app/test/test_thash.c
+++ b/app/test/test_thash.c
@@ -966,4 +966,4 @@ test_thash(void)
 	return unit_test_suite_runner(&thash_tests);
 }
 
-REGISTER_TEST_COMMAND(thash_autotest, test_thash);
+REGISTER_FAST_TEST(thash_autotest, true, true, test_thash);
diff --git a/app/test/test_thash_perf.c b/app/test/test_thash_perf.c
index 687582aa32..9dfd5d3c21 100644
--- a/app/test/test_thash_perf.c
+++ b/app/test/test_thash_perf.c
@@ -135,4 +135,4 @@ test_thash_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(thash_perf_autotest, test_thash_perf);
+REGISTER_PERF_TEST(thash_perf_autotest, test_thash_perf);
diff --git a/app/test/test_threads.c b/app/test/test_threads.c
index a4c4f651a4..526eaa5bf3 100644
--- a/app/test/test_threads.c
+++ b/app/test/test_threads.c
@@ -279,4 +279,4 @@ test_threads(void)
 	return unit_test_suite_runner(&threads_test_suite);
 }
 
-REGISTER_TEST_COMMAND(threads_autotest, test_threads);
+REGISTER_FAST_TEST(threads_autotest, true, true, test_threads);
diff --git a/app/test/test_ticketlock.c b/app/test/test_ticketlock.c
index 242c136478..1fbbedb33b 100644
--- a/app/test/test_ticketlock.c
+++ b/app/test/test_ticketlock.c
@@ -314,4 +314,4 @@ test_ticketlock(void)
 	return ret;
 }
 
-REGISTER_TEST_COMMAND(ticketlock_autotest, test_ticketlock);
+REGISTER_FAST_TEST(ticketlock_autotest, true, true, test_ticketlock);
diff --git a/app/test/test_timer.c b/app/test/test_timer.c
index 0c36dc9010..cac8fc0114 100644
--- a/app/test/test_timer.c
+++ b/app/test/test_timer.c
@@ -594,4 +594,4 @@ test_timer(void)
 	return TEST_SUCCESS;
 }
 
-REGISTER_TEST_COMMAND(timer_autotest, test_timer);
+REGISTER_FAST_TEST(timer_autotest, false, true, test_timer);
diff --git a/app/test/test_timer_perf.c b/app/test/test_timer_perf.c
index 0ede4b3e40..d2d74ebbc6 100644
--- a/app/test/test_timer_perf.c
+++ b/app/test/test_timer_perf.c
@@ -131,4 +131,4 @@ test_timer_perf(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(timer_perf_autotest, test_timer_perf);
+REGISTER_PERF_TEST(timer_perf_autotest, test_timer_perf);
diff --git a/app/test/test_timer_racecond.c b/app/test/test_timer_racecond.c
index bb56ae8324..6f8b448ff8 100644
--- a/app/test/test_timer_racecond.c
+++ b/app/test/test_timer_racecond.c
@@ -172,4 +172,4 @@ test_timer_racecond(void)
 	return TEST_SUCCESS;
 }
 
-REGISTER_TEST_COMMAND(timer_racecond_autotest, test_timer_racecond);
+REGISTER_PERF_TEST(timer_racecond_autotest, test_timer_racecond);
diff --git a/app/test/test_trace.c b/app/test/test_trace.c
index ad4a394a29..00809f433b 100644
--- a/app/test/test_trace.c
+++ b/app/test/test_trace.c
@@ -250,4 +250,4 @@ test_trace(void)
 
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
-REGISTER_TEST_COMMAND(trace_autotest, test_trace);
+REGISTER_FAST_TEST(trace_autotest, true, true, test_trace);
diff --git a/app/test/test_trace_perf.c b/app/test/test_trace_perf.c
index 46ae7d8074..a6dd075722 100644
--- a/app/test/test_trace_perf.c
+++ b/app/test/test_trace_perf.c
@@ -179,4 +179,4 @@ test_trace_perf(void)
 	return TEST_SUCCESS;
 }
 
-REGISTER_TEST_COMMAND(trace_perf_autotest, test_trace_perf);
+REGISTER_PERF_TEST(trace_perf_autotest, test_trace_perf);
diff --git a/app/test/test_version.c b/app/test/test_version.c
index 1e1ff18656..52f269fb9c 100644
--- a/app/test/test_version.c
+++ b/app/test/test_version.c
@@ -25,4 +25,4 @@ test_version(void)
 	return 0;
 }
 
-REGISTER_TEST_COMMAND(version_autotest, test_version);
+REGISTER_FAST_TEST(version_autotest, true, true, test_version);
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v6 04/11] app/test: make telemetry data test buildable on windows
  2023-08-16 15:34 ` [PATCH v6 00/11] " Bruce Richardson
                     ` (2 preceding siblings ...)
  2023-08-16 15:34   ` [PATCH v6 03/11] app/test: tag tests with the test type Bruce Richardson
@ 2023-08-16 15:34   ` Bruce Richardson
  2023-08-21 15:52     ` Tyler Retzlaff
  2023-08-16 15:34   ` [PATCH v6 05/11] app/test: build using per-file dependency matrix Bruce Richardson
                     ` (7 subsequent siblings)
  11 siblings, 1 reply; 93+ messages in thread
From: Bruce Richardson @ 2023-08-16 15:34 UTC (permalink / raw)
  To: dev; +Cc: david.marchand, probb, mb, Bruce Richardson

To enable the building of the telemetry data tests file when building on
windows, we need to provide a stub implementation. That way, the test
file is buildable any time the library itself is built.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
---
 app/test/test_telemetry_data.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/app/test/test_telemetry_data.c b/app/test/test_telemetry_data.c
index a960f44c1b..b2dc1d75db 100644
--- a/app/test/test_telemetry_data.c
+++ b/app/test/test_telemetry_data.c
@@ -2,11 +2,20 @@
  * Copyright 2020 Intel Corporation
  */
 
+#ifdef RTE_EXEC_ENV_WINDOWS
+#include "test.h"
+
+static int
+telemetry_data_autotest(void)
+{
+	return TEST_SKIPPED;
+}
+
+#else
+
 #include <string.h>
 #include <sys/socket.h>
-#ifndef RTE_EXEC_ENV_WINDOWS
 #include <sys/un.h>
-#endif
 #include <unistd.h>
 #include <limits.h>
 
@@ -604,5 +613,6 @@ telemetry_data_autotest(void)
 	close(sock);
 	return 0;
 }
+#endif /* windows/non-windows */
 
 REGISTER_TEST_COMMAND(telemetry_data_autotest, telemetry_data_autotest);
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v6 05/11] app/test: build using per-file dependency matrix
  2023-08-16 15:34 ` [PATCH v6 00/11] " Bruce Richardson
                     ` (3 preceding siblings ...)
  2023-08-16 15:34   ` [PATCH v6 04/11] app/test: make telemetry data test buildable on windows Bruce Richardson
@ 2023-08-16 15:34   ` Bruce Richardson
  2023-08-16 15:34   ` [PATCH v6 06/11] app/test: define unit tests suites based on test macros Bruce Richardson
                     ` (6 subsequent siblings)
  11 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-16 15:34 UTC (permalink / raw)
  To: dev; +Cc: david.marchand, probb, mb, Bruce Richardson

Rather than using if-else constructs to selectively add or remove files
from the UT build, switch to a table-based approach where each file
lists out what libs or drivers it depends upon.

Initial version of this table was generated via analysis of the header
files included in each C file. The basic dependencies of the test binary
[cmdline, ring, mempool and mbuf] were then removed from the per-file
lists, as there is no point in checking them as the whole app will be
disabled if they are not present.

With the file list, the dependencies for the "utility" C-files are kept
separate, so that other tests which depend on the functions provided by
those files can have that dependency recorded properly. The basic
cryptodev tests also fall into this category as functions from the main
cryptodev test file are used by other crypto tests.

As well as the main table for internal dependencies, some test
files have separate external components too. A second, much smaller
table lists these dependencies.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
---
 app/meson.build      |   8 +-
 app/test/meson.build | 823 ++++++++++++-------------------------------
 2 files changed, 238 insertions(+), 593 deletions(-)

diff --git a/app/meson.build b/app/meson.build
index 4fc1a83eba..0d8b618e7f 100644
--- a/app/meson.build
+++ b/app/meson.build
@@ -32,6 +32,11 @@ apps = [
         'test-security-perf',
 ]
 
+if get_option('tests')
+# build the auto test app if enabled.
+    apps += 'test'
+endif
+
 default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API']
 default_ldflags = []
 if get_option('default_library') == 'static' and not is_windows
@@ -106,6 +111,3 @@ foreach app:apps
             install_rpath: join_paths(get_option('prefix'), driver_install_path),
             install: true)
 endforeach
-
-# special case the autotests
-subdir('test')
diff --git a/app/test/meson.build b/app/test/meson.build
index 66897c14a3..2b885ae273 100644
--- a/app/test/meson.build
+++ b/app/test/meson.build
@@ -1,446 +1,230 @@
 # SPDX-License-Identifier: BSD-3-Clause
-# Copyright(c) 2017 Intel Corporation
-
-if not get_option('tests')
-    subdir_done()
-endif
-
-test_sources = files(
-        'commands.c',
-        'packet_burst_generator.c',
-        'test.c',
-        'test_acl.c',
-        'test_alarm.c',
-        'test_atomic.c',
-        'test_barrier.c',
-        'test_bitops.c',
-        'test_bitmap.c',
-        'test_bpf.c',
-        'test_byteorder.c',
-        'test_cksum.c',
-        'test_cksum_perf.c',
-        'test_cmdline.c',
-        'test_cmdline_cirbuf.c',
-        'test_cmdline_etheraddr.c',
-        'test_cmdline_ipaddr.c',
-        'test_cmdline_lib.c',
-        'test_cmdline_num.c',
-        'test_cmdline_portlist.c',
-        'test_cmdline_string.c',
-        'test_common.c',
-        'test_cpuflags.c',
-        'test_crc.c',
-        'test_cryptodev.c',
-        'test_cryptodev_asym.c',
-        'test_cryptodev_blockcipher.c',
-        'test_cryptodev_crosscheck.c',
-        'test_cryptodev_security_ipsec.c',
-        'test_cryptodev_security_pdcp.c',
-        'test_cycles.c',
-        'test_debug.c',
-        'test_devargs.c',
-        'test_distributor.c',
-        'test_distributor_perf.c',
-        'test_dmadev.c',
-        'test_dmadev_api.c',
-        'test_eal_flags.c',
-        'test_eal_fs.c',
-        'test_efd.c',
-        'test_efd_perf.c',
-        'test_errno.c',
-        'test_ethdev_link.c',
-        'test_event_crypto_adapter.c',
-        'test_event_eth_rx_adapter.c',
-        'test_event_ring.c',
-        'test_event_timer_adapter.c',
-        'test_eventdev.c',
-        'test_external_mem.c',
-        'test_fbarray.c',
-        'test_fib.c',
-        'test_fib_perf.c',
-        'test_fib6.c',
-        'test_fib6_perf.c',
-        'test_func_reentrancy.c',
-        'test_hash.c',
-        'test_hash_functions.c',
-        'test_hash_multiwriter.c',
-        'test_hash_readwrite.c',
-        'test_hash_perf.c',
-        'test_hash_readwrite_lf_perf.c',
-        'test_interrupts.c',
-        'test_ipfrag.c',
-        'test_ipsec.c',
-        'test_ipsec_sad.c',
-        'test_ipsec_perf.c',
-        'test_kvargs.c',
-        'test_lcores.c',
-        'test_logs.c',
-        'test_lpm.c',
-        'test_lpm6.c',
-        'test_lpm6_perf.c',
-        'test_lpm_perf.c',
-        'test_malloc.c',
-        'test_malloc_perf.c',
-        'test_mbuf.c',
-        'test_member.c',
-        'test_member_perf.c',
-        'test_memcpy.c',
-        'test_memcpy_perf.c',
-        'test_memory.c',
-        'test_mempool.c',
-        'test_mempool_perf.c',
-        'test_memzone.c',
-        'test_meter.c',
-        'test_mcslock.c',
-        'test_mp_secondary.c',
-        'test_per_lcore.c',
-        'test_pflock.c',
-        'test_pmd_perf.c',
-        'test_power.c',
-        'test_power_cpufreq.c',
-        'test_power_kvm_vm.c',
-        'test_power_intel_uncore.c',
-        'test_prefetch.c',
-        'test_rand_perf.c',
-        'test_rawdev.c',
-        'test_rcu_qsbr.c',
-        'test_rcu_qsbr_perf.c',
-        'test_reassembly_perf.c',
-        'test_reciprocal_division.c',
-        'test_reciprocal_division_perf.c',
-        'test_red.c',
-        'test_pie.c',
-        'test_reorder.c',
-        'test_rib.c',
-        'test_rib6.c',
-        'test_ring.c',
-        'test_ring_mpmc_stress.c',
-        'test_ring_hts_stress.c',
-        'test_ring_mt_peek_stress.c',
-        'test_ring_mt_peek_stress_zc.c',
-        'test_ring_perf.c',
-        'test_ring_rts_stress.c',
-        'test_ring_st_peek_stress.c',
-        'test_ring_st_peek_stress_zc.c',
-        'test_ring_stress.c',
-        'test_rwlock.c',
-        'test_sched.c',
-        'test_security.c',
-        'test_security_inline_macsec.c',
-        'test_security_inline_proto.c',
-        'test_seqlock.c',
-        'test_service_cores.c',
-        'test_spinlock.c',
-        'test_stack.c',
-        'test_stack_perf.c',
-        'test_string_fns.c',
-        'test_tailq.c',
-        'test_thash.c',
-        'test_thash_perf.c',
-        'test_threads.c',
-        'test_timer.c',
-        'test_timer_perf.c',
-        'test_timer_racecond.c',
-        'test_timer_secondary.c',
-        'test_ticketlock.c',
-        'test_trace.c',
-        'test_trace_register.c',
-        'test_trace_perf.c',
-        'test_version.c',
-        'virtual_pmd.c',
-)
-
-test_deps = dpdk_libs_enabled
-# as well as libs, the pci and vdev bus drivers are needed for a lot of tests
-test_deps += ['bus_pci', 'bus_vdev']
-
-# Each test is marked with flags:
-# - the first flag indicates whether the test can run in no-huge mode,
-# - the second flag indicates whether the test can run with ASan enabled,
-fast_tests = [
-        ['acl_autotest', true, true],
-        ['atomic_autotest', false, true],
-        ['bitmap_autotest', true, true],
-        ['bpf_autotest', true, true],
-        ['bpf_convert_autotest', true, true],
-        ['bitops_autotest', true, true],
-        ['byteorder_autotest', true, true],
-        ['cksum_autotest', true, true],
-        ['cmdline_autotest', true, true],
-        ['common_autotest', true, true],
-        ['cpuflags_autotest', true, true],
-        ['debug_autotest', true, true],
-        ['devargs_autotest', true, true],
-        ['eal_flags_c_opt_autotest', false, false],
-        ['eal_flags_main_opt_autotest', false, false],
-        ['eal_flags_n_opt_autotest', false, false],
-        ['eal_flags_hpet_autotest', false, false],
-        ['eal_flags_no_huge_autotest', false, false],
-        ['eal_flags_a_opt_autotest', false, false],
-        ['eal_flags_b_opt_autotest', false, false],
-        ['eal_flags_vdev_opt_autotest', false, false],
-        ['eal_flags_r_opt_autotest', false, false],
-        ['eal_flags_mem_autotest', false, false],
-        ['eal_flags_file_prefix_autotest', false, false],
-        ['eal_flags_misc_autotest', false, false],
-        ['eal_fs_autotest', true, true],
-        ['errno_autotest', true, true],
-        ['ethdev_link_status', true, true],
-        ['event_ring_autotest', true, true],
-        ['fib_autotest', true, true],
-        ['fib6_autotest', true, true],
-        ['func_reentrancy_autotest', false, true],
-        ['hash_autotest', true, true],
-        ['interrupt_autotest', true, true],
-        ['ipfrag_autotest', false, true],
-        ['lcores_autotest', true, true],
-        ['logs_autotest', true, true],
-        ['lpm_autotest', true, true],
-        ['lpm6_autotest', true, true],
-        ['malloc_autotest', false, true],
-        ['mbuf_autotest', false, true],
-        ['mcslock_autotest', false, true],
-        ['memcpy_autotest', true, true],
-        ['memory_autotest', false, true],
-        ['mempool_autotest', false, true],
-        ['memzone_autotest', false, true],
-        ['meter_autotest', true, true],
-        ['multiprocess_autotest', false, false],
-        ['per_lcore_autotest', true, true],
-        ['pflock_autotest', true, true],
-        ['prefetch_autotest', true, true],
-        ['rcu_qsbr_autotest', true, true],
-        ['pie_autotest', true, true],
-        ['rib_autotest', true, true],
-        ['rib6_autotest', true, true],
-        ['ring_autotest', true, true],
-        ['rwlock_test1_autotest', true, true],
-        ['rwlock_rda_autotest', true, true],
-        ['rwlock_rds_wrm_autotest', true, true],
-        ['rwlock_rde_wro_autotest', true, true],
-        ['sched_autotest', true, true],
-        ['security_autotest', false, true],
-        ['seqlock_autotest', true, true],
-        ['spinlock_autotest', true, true],
-        ['stack_autotest', false, true],
-        ['stack_lf_autotest', false, true],
-        ['string_autotest', true, true],
-        ['tailq_autotest', true, true],
-        ['ticketlock_autotest', true, true],
-        ['timer_autotest', false, true],
-        ['user_delay_us', true, true],
-        ['version_autotest', true, true],
-        ['crc_autotest', true, true],
-        ['distributor_autotest', false, true],
-        ['eventdev_common_autotest', true, true],
-        ['fbarray_autotest', true, true],
-        ['hash_readwrite_func_autotest', false, true],
-        ['ipsec_autotest', true, true],
-        ['kvargs_autotest', true, true],
-        ['member_autotest', true, true],
-        ['power_cpufreq_autotest', false, true],
-        ['power_autotest', true, true],
-        ['power_kvm_vm_autotest', false, true],
-        ['power_intel_uncore_autotest', true, true],
-        ['reorder_autotest', true, true],
-        ['service_autotest', true, true],
-        ['thash_autotest', true, true],
-        ['threads_autotest', true, true],
-        ['trace_autotest', true, true],
-]
-
-# Tests known to have issues or which don't belong in other tests lists.
-extra_test_names = [
-        'alarm_autotest', # ee00af60170b ("test: remove strict timing requirements some tests")
-        'red_autotest', # https://bugs.dpdk.org/show_bug.cgi?id=826
-]
-
-perf_test_names = [
-        'ring_perf_autotest',
-        'malloc_perf_autotest',
-        'mempool_perf_autotest',
-        'memcpy_perf_autotest',
-        'hash_perf_autotest',
-        'timer_perf_autotest',
-        'reciprocal_division',
-        'reciprocal_division_perf',
-        'lpm_perf_autotest',
-        'rib_slow_autotest',
-        'fib_slow_autotest',
-        'fib_perf_autotest',
-        'red_all',
-        'pie_all',
-        'barrier_autotest',
-        'hash_multiwriter_autotest',
-        'timer_racecond_autotest',
-        'efd_autotest',
-        'hash_functions_autotest',
-        'member_perf_autotest',
-        'efd_perf_autotest',
-        'lpm6_perf_autotest',
-        'rib6_slow_autotest',
-        'fib6_slow_autotest',
-        'fib6_perf_autotest',
-        'rcu_qsbr_perf_autotest',
-        'red_perf',
-        'pie_perf',
-        'distributor_perf_autotest',
-        'pmd_perf_autotest',
-        'service_perf_autotest',
-        'stack_perf_autotest',
-        'stack_lf_perf_autotest',
-        'rand_perf_autotest',
-        'hash_readwrite_perf_autotest',
-        'hash_readwrite_lf_perf_autotest',
-        'trace_perf_autotest',
-        'ipsec_perf_autotest',
-        'thash_perf_autotest',
-        'reassembly_perf_autotest',
-]
-
-driver_test_names = [
-        'cryptodev_aesni_gcm_autotest',
-        'cryptodev_aesni_mb_autotest',
-        'cryptodev_chacha_poly_mb_autotest',
-        'cryptodev_cn10k_autotest',
-        'cryptodev_cn9k_autotest',
-        'cryptodev_cpu_aesni_mb_autotest',
-        'cryptodev_cpu_aesni_gcm_autotest',
-        'cryptodev_dpaa2_sec_autotest',
-        'cryptodev_dpaa_sec_autotest',
-        'cryptodev_null_autotest',
-        'cryptodev_openssl_autotest',
-        'cryptodev_qat_autotest',
-        'cryptodev_qat_asym_autotest',
-        'cryptodev_qat_raw_api_autotest',
-        'cryptodev_sw_armv8_autotest',
-        'cryptodev_sw_kasumi_autotest',
-        'cryptodev_sw_mvsam_autotest',
-        'cryptodev_sw_snow3g_autotest',
-        'cryptodev_sw_zuc_autotest',
-        'cryptodev_uadk_autotest',
-        'dmadev_autotest',
-]
-
-dump_test_names = []
-
-if not is_windows
-    driver_test_names += [
-            'cryptodev_openssl_asym_autotest',
-            'eventdev_selftest_octeontx',
-            'eventdev_selftest_sw',
-    ]
-
-    dump_test_names += [
-            'dump_struct_sizes',
-            'dump_mempool',
-            'dump_malloc_stats',
-            'dump_devargs',
-            'dump_log_types',
-            'dump_ring',
-            'dump_physmem',
-            'dump_memzone',
-    ]
-endif
-
-# The following linkages are an exception to allow running the
-# unit tests without requiring that the developer install the
-# DPDK libraries.  Explicit linkage of drivers (plugin libraries)
-# in applications should not be used.
-if dpdk_conf.has('RTE_MEMPOOL_RING')
-    test_deps += 'mempool_ring'
-endif
-if dpdk_conf.has('RTE_MEMPOOL_STACK')
-    test_deps += 'mempool_stack'
-endif
-if dpdk_conf.has('RTE_EVENT_SKELETON')
-    test_deps += 'event_skeleton'
-endif
-
-if dpdk_conf.has('RTE_LIB_GRAPH')
-    test_sources += 'test_graph.c'
-    fast_tests += [['graph_autotest', true, true]]
-    fast_tests += [['node_list_dump', true, true]]
-    test_sources += 'test_graph_perf.c'
-    perf_test_names += 'graph_perf_autotest'
-endif
-if dpdk_conf.has('RTE_LIB_METRICS')
-    test_sources += ['test_metrics.c']
-    fast_tests += [['metrics_autotest', true, true]]
-endif
-if not is_windows and dpdk_conf.has('RTE_LIB_TELEMETRY')
-    test_sources += ['test_telemetry_json.c', 'test_telemetry_data.c']
-    fast_tests += [['telemetry_json_autotest', true, true]]
-    fast_tests += [['telemetry_data_autotest', true, true]]
-endif
-if dpdk_conf.has('RTE_LIB_PIPELINE')
-# pipeline lib depends on port and table libs, so those must be present
-# if pipeline library is.
-    test_sources += [
-            'test_table.c',
-            'test_table_acl.c',
-            'test_table_combined.c',
-            'test_table_pipeline.c',
-            'test_table_ports.c',
-            'test_table_tables.c',
-    ]
-    fast_tests += [['table_autotest', true, true]]
-endif
-
-# The following linkages of drivers are required because
-# they are used via a driver-specific API.
-if dpdk_conf.has('RTE_NET_BOND')
-    test_deps += 'net_bond'
-    test_sources += ['test_link_bonding.c', 'test_link_bonding_rssconf.c']
-    driver_test_names += ['link_bonding_autotest', 'link_bonding_rssconf_autotest']
-    if dpdk_conf.has('RTE_NET_RING')
-        test_sources += 'test_link_bonding_mode4.c'
-        driver_test_names += 'link_bonding_mode4_autotest'
-    endif
-endif
-if dpdk_conf.has('RTE_LIB_EVENTDEV') and dpdk_conf.has('RTE_NET_RING')
-    test_deps += 'net_ring'
-    test_sources += 'test_pmd_ring_perf.c'
-    test_sources += 'test_pmd_ring.c'
-    test_sources += 'test_event_eth_tx_adapter.c'
-    test_sources += 'sample_packet_forward.c'
-    fast_tests += [['ring_pmd_autotest', true, true]]
-    perf_test_names += 'ring_pmd_perf_autotest'
-    fast_tests += [['event_eth_tx_adapter_autotest', false, true]]
-    if dpdk_conf.has('RTE_LIB_BITRATESTATS')
-        test_sources += 'test_bitratestats.c'
-        fast_tests += [['bitratestats_autotest', true, true]]
-    endif
-    if dpdk_conf.has('RTE_LIB_LATENCYSTATS')
-        test_sources += 'test_latencystats.c'
-        fast_tests += [['latencystats_autotest', true, true]]
-    endif
-    if dpdk_conf.has('RTE_LIB_PDUMP')
-        test_sources += 'test_pdump.c'
-        fast_tests += [['pdump_autotest', true, false]]
-    endif
-endif
-if dpdk_conf.has('RTE_NET_NULL')
-    test_deps += 'net_null'
-    test_sources += 'test_vdev.c'
-    fast_tests += [['vdev_autotest', true, true]]
-endif
-if dpdk_conf.has('RTE_RAW_SKELETON')
-    test_deps += 'raw_skeleton'
-    fast_tests += [['rawdev_autotest', true, true]]
-endif
-
-if dpdk_conf.has('RTE_HAS_LIBPCAP')
-    ext_deps += pcap_dep
-    if dpdk_conf.has('RTE_LIB_PCAPNG')
-        test_sources += 'test_pcapng.c'
+# Copyright(c) 2017-2023 Intel Corporation
+
+# the main test files [test.c and commands.c] relies on these libraries
+deps += ['cmdline', 'ring', 'mempool', 'mbuf']
+sources += files('commands.c', 'test.c')
+
+# some other utility C files, providing functions used by various tests
+# so we need to include these deps in the dependency list for the files using those fns.
+packet_burst_generator_deps = ['net']
+sample_packet_forward_deps = ['net_ring', 'ethdev', 'bus_vdev']
+virtual_pmd_deps = ['ethdev', 'net', 'bus_pci']
+# test_cryptodev has material that other crypto tests need
+test_cryptodev_deps = ['bus_vdev', 'net', 'cryptodev', 'crypto_scheduler', 'security']
+
+source_file_deps = {
+    # The C files providing functionality to other test cases
+    'packet_burst_generator.c': packet_burst_generator_deps,
+#    'resource.c': [],          # unused currently.
+    'sample_packet_forward.c': sample_packet_forward_deps,
+    'virtual_pmd.c': virtual_pmd_deps,
+
+    # the various test_*.c files
+    'test_acl.c': ['net', 'acl'],
+    'test_alarm.c': [],
+    'test_atomic.c': ['hash'],
+    'test_barrier.c': [],
+    'test_bitmap.c': [],
+    'test_bitops.c': [],
+    'test_bitratestats.c': ['metrics', 'bitratestats', 'ethdev'] + sample_packet_forward_deps,
+    'test_bpf.c': ['bpf', 'net'],
+    'test_byteorder.c': [],
+#    'test_cfgfile.c': ['cfgfile'],
+    'test_cksum.c': ['net'],
+    'test_cksum_perf.c': ['net'],
+    'test_cmdline.c': [],
+    'test_cmdline_cirbuf.c': [],
+    'test_cmdline_etheraddr.c': ['net'],
+    'test_cmdline_ipaddr.c': [],
+    'test_cmdline_lib.c': [],
+    'test_cmdline_num.c': [],
+    'test_cmdline_portlist.c': [],
+    'test_cmdline_string.c': [],
+    'test_common.c': [],
+    'test_compressdev.c': ['compressdev'],
+    'test_cpuflags.c': [],
+    'test_crc.c': ['net'],
+    'test_cryptodev.c': test_cryptodev_deps,
+    'test_cryptodev_asym.c': ['bus_vdev'] + test_cryptodev_deps,
+    'test_cryptodev_blockcipher.c': test_cryptodev_deps,
+    'test_cryptodev_crosscheck.c': test_cryptodev_deps,
+    'test_cryptodev_security_ipsec.c': test_cryptodev_deps,
+    'test_cryptodev_security_pdcp.c': test_cryptodev_deps,
+    'test_cycles.c': [],
+    'test_debug.c': [],
+    'test_devargs.c': ['kvargs'],
+    'test_distributor.c': ['distributor'],
+    'test_distributor_perf.c': ['distributor'],
+    'test_dmadev.c': ['dmadev', 'bus_vdev'],
+    'test_dmadev_api.c': ['dmadev'],
+    'test_eal_flags.c': [],
+    'test_eal_fs.c': [],
+    'test_efd.c': ['efd', 'net'],
+    'test_efd_perf.c': ['efd', 'hash'],
+    'test_errno.c': [],
+    'test_ethdev_link.c': ['ethdev'],
+    'test_event_crypto_adapter.c': ['cryptodev', 'eventdev', 'bus_vdev'],
+    'test_event_eth_rx_adapter.c': ['ethdev', 'eventdev', 'bus_vdev'],
+    'test_event_eth_tx_adapter.c': ['bus_vdev', 'ethdev', 'net_ring', 'eventdev'],
+    'test_event_ring.c': ['eventdev'],
+    'test_event_timer_adapter.c': ['ethdev', 'eventdev', 'bus_vdev'],
+    'test_eventdev.c': ['eventdev', 'bus_vdev'],
+    'test_external_mem.c': [],
+    'test_fbarray.c': [],
+    'test_fib.c': ['net', 'fib'],
+    'test_fib6.c': ['rib', 'fib'],
+    'test_fib6_perf.c': ['fib'],
+    'test_fib_perf.c': ['net', 'fib'],
+    'test_flow_classify.c': ['net', 'acl', 'table', 'ethdev', 'flow_classify'],
+    'test_func_reentrancy.c': ['hash', 'lpm'],
+    'test_graph.c': ['graph'],
+    'test_graph_perf.c': ['graph'],
+    'test_hash.c': ['net', 'hash'],
+    'test_hash_functions.c': ['hash'],
+    'test_hash_multiwriter.c': ['hash'],
+    'test_hash_perf.c': ['hash'],
+    'test_hash_readwrite.c': ['hash'],
+    'test_hash_readwrite_lf_perf.c': ['hash'],
+    'test_interrupts.c': [],
+    'test_ipfrag.c': ['net', 'ip_frag'],
+    'test_ipsec.c': ['bus_vdev', 'net', 'cryptodev', 'ipsec', 'security'],
+    'test_ipsec_perf.c': ['net', 'ipsec'],
+    'test_ipsec_sad.c': ['ipsec'],
+    'test_kvargs.c': ['kvargs'],
+    'test_latencystats.c': ['ethdev', 'latencystats', 'metrics'] + sample_packet_forward_deps,
+    'test_lcores.c': [],
+    'test_link_bonding.c': ['ethdev', 'net_bond',
+        'net'] + packet_burst_generator_deps + virtual_pmd_deps,
+    'test_link_bonding_mode4.c': ['ethdev', 'net_ring', 'net_bond',
+        'net'] + packet_burst_generator_deps,
+    'test_link_bonding_rssconf.c': ['ethdev', 'bus_vdev', 'net_bond'],
+    'test_logs.c': [],
+    'test_lpm.c': ['net', 'lpm'],
+    'test_lpm6.c': ['lpm'],
+    'test_lpm6_perf.c': ['lpm'],
+    'test_lpm_perf.c': ['net', 'lpm'],
+    'test_malloc.c': [],
+    'test_malloc_perf.c': [],
+    'test_mbuf.c': ['net'],
+    'test_mcslock.c': [],
+    'test_member.c': ['member', 'net'],
+    'test_member_perf.c': ['hash', 'member'],
+    'test_memcpy.c': [],
+    'test_memcpy_perf.c': [],
+    'test_memory.c': [],
+    'test_mempool.c': [],
+    'test_mempool_perf.c': [],
+    'test_memzone.c': [],
+    'test_meter.c': ['meter'],
+    'test_metrics.c': ['metrics'],
+    'test_mp_secondary.c': ['hash', 'lpm'],
+    'test_pcapng.c': ['ethdev', 'net', 'pcapng'],
+    'test_pdcp.c': ['eventdev', 'pdcp', 'net', 'timer', 'security'],
+    'test_pdump.c': ['pdump'] + sample_packet_forward_deps,
+    'test_per_lcore.c': [],
+    'test_pflock.c': [],
+    'test_pie.c': ['sched'],
+    'test_pmd_perf.c': ['ethdev', 'net'] + packet_burst_generator_deps,
+    'test_pmd_ring.c': ['net_ring', 'ethdev', 'bus_vdev'],
+    'test_pmd_ring_perf.c': ['ethdev', 'net_ring', 'bus_vdev'],
+    'test_power.c': ['power'],
+    'test_power_cpufreq.c': ['power'],
+    'test_power_intel_uncore.c': ['power'],
+    'test_power_kvm_vm.c': ['power'],
+    'test_prefetch.c': [],
+    'test_rand_perf.c': [],
+    'test_rawdev.c': ['rawdev', 'bus_vdev'],
+    'test_rcu_qsbr.c': ['rcu', 'hash'],
+    'test_rcu_qsbr_perf.c': ['rcu', 'hash'],
+    'test_reassembly_perf.c': ['net', 'ip_frag'],
+    'test_reciprocal_division.c': [],
+    'test_reciprocal_division_perf.c': [],
+    'test_red.c': ['sched'],
+    'test_reorder.c': ['reorder'],
+#    'test_resource.c': [],
+    'test_rib.c': ['net', 'rib'],
+    'test_rib6.c': ['net', 'rib'],
+    'test_ring.c': [],
+    'test_ring_hts_stress.c': [],
+    'test_ring_mpmc_stress.c': [],
+    'test_ring_mt_peek_stress.c': [],
+    'test_ring_mt_peek_stress_zc.c': [],
+    'test_ring_perf.c': [],
+    'test_ring_rts_stress.c': [],
+    'test_ring_st_peek_stress.c': [],
+    'test_ring_st_peek_stress_zc.c': [],
+    'test_ring_stress.c': [],
+    'test_rwlock.c': [],
+    'test_sched.c': ['net', 'sched'],
+    'test_security.c': ['net', 'security'],
+    'test_security_inline_macsec.c': ['ethdev', 'security'],
+    'test_security_inline_proto.c': ['ethdev', 'security', 'eventdev'] + test_cryptodev_deps,
+    'test_seqlock.c': [],
+    'test_service_cores.c': [],
+    'test_spinlock.c': [],
+    'test_stack.c': ['stack'],
+    'test_stack_perf.c': ['stack'],
+    'test_string_fns.c': [],
+    'test_table.c': ['table', 'pipeline', 'port'],
+    'test_table_acl.c': ['net', 'table', 'pipeline', 'port'],
+    'test_table_combined.c': ['table', 'pipeline', 'port'],
+    'test_table_pipeline.c': ['pipeline', 'table', 'port'],
+    'test_table_ports.c': ['table', 'pipeline', 'port'],
+    'test_table_tables.c': ['table', 'pipeline', 'port'],
+    'test_tailq.c': [],
+    'test_telemetry_data.c': ['telemetry'],
+    'test_telemetry_json.c': ['telemetry'],
+    'test_thash.c': ['net', 'hash'],
+    'test_thash_perf.c': ['hash'],
+    'test_threads.c': [],
+    'test_ticketlock.c': [],
+    'test_timer.c': ['timer'],
+    'test_timer_perf.c': ['timer'],
+    'test_timer_racecond.c': ['timer'],
+    'test_timer_secondary.c': ['timer'],
+    'test_trace.c': [],
+    'test_trace_perf.c': [],
+    'test_trace_register.c': [],
+    'test_vdev.c': ['kvargs', 'bus_vdev'],
+    'test_version.c': [],
+}
+
+source_file_ext_deps = {
+    'test_compressdev.c': ['zlib'],
+    'test_pcapng.c': ['pcap'],
+}
+
+def_lib = get_option('default_library')
+foreach f, f_deps : source_file_deps
+    has_deps = true
+    foreach d : f_deps
+        if not is_variable(def_lib + '_rte_' + d)
+            has_deps = false
+            break
+        else
+            # technically we might not need this dep, but adding it is harmless
+            if d not in deps
+                deps += d
+            endif
+        endif
+    endforeach
+    # check for any external dependencies for this file
+    if source_file_ext_deps.has_key(f)
+        foreach d: source_file_ext_deps.get(f)
+            dep = dependency(d, required: false, method: 'pkg-config')
+            if not dep.found()
+                message('Skipping test file @0@ due to missing external dependency @1@'.format(f, d))
+                has_deps = false
+            else
+                ext_deps += dep
+            endif
+        endforeach
+    endif
+    if has_deps
+        sources += files(f)
     endif
-endif
-
-if dpdk_conf.has('RTE_LIB_PDCP')
-    test_sources += 'test_pdcp.c'
-    fast_tests += [['pdcp_autotest', false, true]]
-endif
+endforeach
 
 if cc.has_argument('-Wno-format-truncation')
     cflags += '-Wno-format-truncation'
@@ -450,154 +234,13 @@ endif
 cflags += '-fno-strict-aliasing'
 
 # Enable using internal APIs in unit tests
-cflags += ['-DALLOW_INTERNAL_API']
-
-test_dep_objs = []
-if dpdk_conf.has('RTE_LIB_COMPRESSDEV')
-    compress_test_dep = dependency('zlib', required: false, method: 'pkg-config')
-    if compress_test_dep.found()
-        test_dep_objs += compress_test_dep
-        test_sources += 'test_compressdev.c'
-        fast_tests += [['compressdev_autotest', false, true]]
-    endif
-endif
-
-if dpdk_conf.has('RTE_CRYPTO_SCHEDULER')
-    driver_test_names += 'cryptodev_scheduler_autotest'
-    test_deps += 'crypto_scheduler'
-endif
-
-foreach d:test_deps
-    def_lib = get_option('default_library')
-    test_dep_objs += get_variable(def_lib + '_rte_' + d)
-endforeach
-
-link_libs = []
-if get_option('default_library') == 'static'
-    link_libs = dpdk_static_libraries + dpdk_drivers
-endif
-
-dpdk_test = executable('dpdk-test',
-        test_sources,
-        link_whole: link_libs,
-        dependencies: test_dep_objs + ext_deps,
-        c_args: cflags,
-        install_rpath: join_paths(get_option('prefix'),
-             driver_install_path),
-        install: true)
-
-has_hugepage = run_command(py3, files('has_hugepage.py'), check: true).stdout().strip() != '0'
-message('hugepage availability: @0@'.format(has_hugepage))
-
-# some perf tests (eg: memcpy perf autotest)take very long
-# to complete, so timeout to 10 minutes
-timeout_seconds = 600
-timeout_seconds_fast = 10
-
-test_no_huge_args = ['--no-huge', '-m', '2048']
-
-foreach arg : fast_tests
-    test_args = []
-    run_test = true
-    if not has_hugepage
-        if arg[1]
-            test_args += test_no_huge_args
-        else
-            run_test = false
-        endif
-    endif
-
-    if get_option('b_sanitize') == 'address' or get_option('b_sanitize') == 'address,undefined'
-        if not arg[2]
-            run_test = false
-        endif
-    endif
-
-    if (get_option('default_library') == 'shared' and
-        arg[0] == 'event_eth_tx_adapter_autotest')
-        test_args += ['-d', dpdk_drivers_build_dir]
-    endif
-    if is_linux
-        test_args += ['--file-prefix=@0@'.format(arg[0])]
-    endif
-
-    if run_test
-        test(arg[0], dpdk_test,
-                env : ['DPDK_TEST=' + arg[0]],
-                args : test_args,
-                timeout : timeout_seconds_fast,
-                is_parallel : false,
-                suite : 'fast-tests')
-        if not is_windows and arg[0] == 'trace_autotest'
-            test_args += ['--trace=.*']
-            test_args += ['--trace-dir=@0@'.format(meson.current_build_dir())]
-            test(arg[0] + '_with_traces', dpdk_test,
-                    env : ['DPDK_TEST=' + arg[0]],
-                    args : test_args,
-                    timeout : timeout_seconds_fast,
-                    is_parallel : false,
-                    suite : 'fast-tests')
-        endif
-    endif
-endforeach
+cflags += '-DALLOW_INTERNAL_API'
 
-if not is_windows and dpdk_conf.has('RTE_LIB_TELEMETRY')
-    test_args = [dpdk_test]
-    test_args += test_no_huge_args
-    if get_option('default_library') == 'shared'
-        test_args += ['-d', dpdk_drivers_build_dir]
-    endif
-    if dpdk_conf.has('RTE_CRYPTO_NULL')
-        test_args += ['--vdev=crypto_null0']
-    endif
-    if dpdk_conf.has('RTE_DMA_SKELETON')
-        test_args += ['--vdev=dma_skeleton0']
-    endif
-    if dpdk_conf.has('RTE_EVENT_SKELETON')
-        test_args += ['--vdev=event_skeleton0']
-    endif
-    if dpdk_conf.has('RTE_NET_NULL')
-        test_args += ['--vdev=net_null0']
-    endif
-    if dpdk_conf.has('RTE_RAW_SKELETON')
-        test_args += ['--vdev=rawdev_skeleton0']
-    endif
-    test_args += ['-a', '0000:00:00.0']
-    test('telemetry_all', find_program('test_telemetry.sh'),
-            args: test_args,
-            timeout : timeout_seconds_fast,
-            is_parallel : false,
-            suite : 'fast-tests')
+# create a symlink in the app/test directory for the binary, for backward compatibility
+if not is_windows
+    custom_target('test_symlink',
+            output: 'dpdk-test',
+            command: ['ln', '-sf', '../dpdk-test', '@OUTPUT@'],
+            build_by_default: true,
+            install: false)
 endif
-
-foreach arg : perf_test_names
-    test(arg, dpdk_test,
-            env : ['DPDK_TEST=' + arg],
-            timeout : timeout_seconds,
-            is_parallel : false,
-            suite : 'perf-tests')
-endforeach
-
-foreach arg : driver_test_names
-    test(arg, dpdk_test,
-            env : ['DPDK_TEST=' + arg],
-            timeout : timeout_seconds,
-            is_parallel : false,
-            suite : 'driver-tests')
-endforeach
-
-foreach arg : dump_test_names
-    test(arg, dpdk_test,
-            env : ['DPDK_TEST=' + arg],
-            timeout : timeout_seconds,
-            is_parallel : false,
-            suite : 'debug-tests')
-endforeach
-
-foreach arg : extra_test_names
-    test(arg, dpdk_test,
-            env : ['DPDK_TEST=' + arg],
-            timeout : timeout_seconds,
-            is_parallel : false,
-            suite : 'extra-tests')
-endforeach
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v6 06/11] app/test: define unit tests suites based on test macros
  2023-08-16 15:34 ` [PATCH v6 00/11] " Bruce Richardson
                     ` (4 preceding siblings ...)
  2023-08-16 15:34   ` [PATCH v6 05/11] app/test: build using per-file dependency matrix Bruce Richardson
@ 2023-08-16 15:34   ` Bruce Richardson
  2023-08-16 15:34   ` [PATCH v6 07/11] app/test: add test case for scripted telemetry commands Bruce Richardson
                     ` (5 subsequent siblings)
  11 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-16 15:34 UTC (permalink / raw)
  To: dev; +Cc: david.marchand, probb, mb, Bruce Richardson

Rather than having the test suites listed out in the meson.build files
and having to have them enabled/disabled selectively based on what libs
are being built, pull the tests to run from the source files which were
added to the build.

Most test suites require no additional info other than the list of test
names in the suite. However the fast-test are special that they have
additional parameters associated with them. This requires some
additional work in the test extraction script and in the meson.build
file for processing the output.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
---
 MAINTAINERS                                   |  3 +-
 app/meson.build                               |  7 +-
 app/test/suites/meson.build                   | 74 +++++++++++++++++++
 buildtools/get-test-suites.py                 | 33 +++++++++
 .../has-hugepages.py                          |  0
 buildtools/meson.build                        |  2 +
 6 files changed, 117 insertions(+), 2 deletions(-)
 create mode 100644 app/test/suites/meson.build
 create mode 100644 buildtools/get-test-suites.py
 rename app/test/has_hugepage.py => buildtools/has-hugepages.py (100%)

diff --git a/MAINTAINERS b/MAINTAINERS
index 8c3f2c993f..c4ce661b6e 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1731,7 +1731,6 @@ Test Applications
 
 Unit tests framework
 F: app/test/commands.c
-F: app/test/has_hugepage.py
 F: app/test/packet_burst_generator.c
 F: app/test/packet_burst_generator.h
 F: app/test/process.h
@@ -1742,6 +1741,8 @@ F: app/test/test_pmd_perf.c
 F: app/test/test_resource.c
 F: app/test/virtual_pmd.c
 F: app/test/virtual_pmd.h
+F: buildtools/has-hugepages.py
+F: buildtools/get-test-suites.py
 
 Sample packet helper functions for unit test
 M: Reshma Pattan <reshma.pattan@intel.com>
diff --git a/app/meson.build b/app/meson.build
index 0d8b618e7f..c14dc80892 100644
--- a/app/meson.build
+++ b/app/meson.build
@@ -101,7 +101,7 @@ foreach app:apps
         link_libs = dpdk_static_libraries + dpdk_drivers
     endif
 
-    executable('dpdk-' + name,
+    exec = executable('dpdk-' + name,
             sources,
             c_args: cflags,
             link_args: ldflags,
@@ -110,4 +110,9 @@ foreach app:apps
             include_directories: includes,
             install_rpath: join_paths(get_option('prefix'), driver_install_path),
             install: true)
+    if name == 'test'
+        dpdk_test = exec
+        autotest_sources = sources
+        subdir('test/suites')  # define the pre-canned test suites
+    endif
 endforeach
diff --git a/app/test/suites/meson.build b/app/test/suites/meson.build
new file mode 100644
index 0000000000..5cdf7c200c
--- /dev/null
+++ b/app/test/suites/meson.build
@@ -0,0 +1,74 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2023 Intel Corporation
+
+# some perf tests (eg: memcpy perf autotest) take a very long
+# to complete, so timeout to 10 minutes
+timeout_seconds = 600
+timeout_seconds_fast = 10
+
+test_no_huge_args = ['--no-huge', '-m', '2048']
+has_hugepage = run_command(has_hugepages_cmd, check: true).stdout().strip() != '0'
+message('hugepage availability: @0@'.format(has_hugepage))
+
+# process source files to determine the different unit test suites
+# - fast_tests
+# - perf_tests
+# - driver_tests
+test_suites = run_command(get_test_suites_cmd, autotest_sources,
+         check: true).stdout().strip().split()
+foreach suite:test_suites
+    # simple cases - tests without parameters or special handling
+    suite = suite.split('=')
+    suite_name = suite[0]
+    suite_tests = suite[1].split(',')
+    if suite_name != 'fast-tests'
+        # simple cases - tests without parameters or special handling
+        foreach t: suite_tests
+            test(t, dpdk_test,
+                    env: ['DPDK_TEST=' + t],
+                    timeout: timeout_seconds,
+                    is_parallel: false,
+                    suite: suite_name)
+        endforeach
+    else
+        # special fast-test handling here
+        foreach t: suite_tests
+            params = t.split(':')
+            test_name = params[0]
+            nohuge = params[1] == 'true'
+            asan = params[2] == 'true'
+
+            test_args = []
+            if nohuge
+                test_args += test_no_huge_args
+            elif not has_hugepage
+                continue  #skip this tests
+            endif
+            if not asan and (get_option('b_sanitize') == 'address'
+                    or get_option('b_sanitize') == 'address,undefined')
+                continue  # skip this test
+            endif
+
+            if get_option('default_library') == 'shared'
+                test_args += ['-d', dpdk_drivers_build_dir]
+            endif
+
+            test(test_name, dpdk_test,
+                args : test_args,
+                env: ['DPDK_TEST=' + test_name],
+                timeout : timeout_seconds_fast,
+                is_parallel : false,
+                suite : 'fast-tests')
+            if not is_windows and test_name == 'trace_autotest'
+                test_args += ['--trace=.*']
+                test_args += ['--trace-dir=@0@'.format(meson.current_build_dir())]
+                test(test_name + '_with_traces', dpdk_test,
+                    args : test_args,
+                    env: ['DPDK_TEST=' + test_name],
+                    timeout : timeout_seconds_fast,
+                    is_parallel : false,
+                    suite : 'fast-tests')
+            endif
+        endforeach
+    endif
+endforeach
diff --git a/buildtools/get-test-suites.py b/buildtools/get-test-suites.py
new file mode 100644
index 0000000000..95a9cad4c8
--- /dev/null
+++ b/buildtools/get-test-suites.py
@@ -0,0 +1,33 @@
+#! /usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2023 Intel Corporation
+
+import sys
+import re
+
+input_list = sys.argv[1:]
+test_def_regex = re.compile("REGISTER_([A-Z]+)_TEST\s*\(\s*([a-z0-9_]+)")
+test_suites = {}
+
+def get_fast_test_params(test_name, ln):
+    "Extract the extra fast-test parameters from the line"
+    #print(f"ln: {ln.rstrip()}, test_name: {test_name}, split: {ln.split(test_name, 1)}")
+    (_, rest_of_line) = ln.split(test_name, 1)
+    (_, nohuge, asan, _func) = rest_of_line.split(',', 3)
+    return f":{nohuge.strip().lower()}:{asan.strip().lower()}"
+
+for fname in input_list:
+    with open(fname) as f:
+        contents = [ln for ln in f.readlines() if test_def_regex.match(ln.strip())]
+    for ln in contents:
+        (test_suite, test_name) = test_def_regex.match(ln).group(1, 2)
+        suite_name = f"{test_suite.lower()}-tests"
+        if suite_name in test_suites:
+            test_suites[suite_name].append(test_name)
+        else:
+            test_suites[suite_name] = [test_name]
+        if suite_name == "fast-tests":
+            test_suites["fast-tests"][-1] += get_fast_test_params(test_name, ln)
+
+for suite in test_suites.keys():
+    print(f"{suite}={','.join(test_suites[suite])}")
diff --git a/app/test/has_hugepage.py b/buildtools/has-hugepages.py
similarity index 100%
rename from app/test/has_hugepage.py
rename to buildtools/has-hugepages.py
diff --git a/buildtools/meson.build b/buildtools/meson.build
index e1c600e40f..ac5e4dcf08 100644
--- a/buildtools/meson.build
+++ b/buildtools/meson.build
@@ -18,6 +18,8 @@ map_to_win_cmd = py3 + files('map_to_win.py')
 sphinx_wrapper = py3 + files('call-sphinx-build.py')
 get_cpu_count_cmd = py3 + files('get-cpu-count.py')
 get_numa_count_cmd = py3 + files('get-numa-count.py')
+get_test_suites_cmd = py3 + files('get-test-suites.py')
+has_hugepages_cmd = py3 + files('has-hugepages.py')
 binutils_avx512_check = (py3 + files('binutils-avx512-check.py') +
                         [objdump] + cc.cmd_array())
 
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v6 07/11] app/test: add test case for scripted telemetry commands
  2023-08-16 15:34 ` [PATCH v6 00/11] " Bruce Richardson
                     ` (5 preceding siblings ...)
  2023-08-16 15:34   ` [PATCH v6 06/11] app/test: define unit tests suites based on test macros Bruce Richardson
@ 2023-08-16 15:34   ` Bruce Richardson
  2023-08-16 15:34   ` [PATCH v6 08/11] app/test: add debug test suite Bruce Richardson
                     ` (4 subsequent siblings)
  11 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-16 15:34 UTC (permalink / raw)
  To: dev; +Cc: david.marchand, probb, mb, Bruce Richardson

The test script for calling all telemetry commands is not discoverable
by checking the C files for the build. Therefore we need to add it in as
a special-case test in the fast-tests group.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 app/test/suites/meson.build             | 30 +++++++++++++++++++++++++
 app/test/{ => suites}/test_telemetry.sh |  0
 2 files changed, 30 insertions(+)
 rename app/test/{ => suites}/test_telemetry.sh (100%)

diff --git a/app/test/suites/meson.build b/app/test/suites/meson.build
index 5cdf7c200c..eddf332156 100644
--- a/app/test/suites/meson.build
+++ b/app/test/suites/meson.build
@@ -72,3 +72,33 @@ foreach suite:test_suites
         endforeach
     endif
 endforeach
+
+# standalone test for telemetry
+if not is_windows and dpdk_conf.has('RTE_LIB_TELEMETRY')
+    test_args = [dpdk_test]
+    test_args += test_no_huge_args
+    if get_option('default_library') == 'shared'
+        test_args += ['-d', dpdk_drivers_build_dir]
+    endif
+    if dpdk_conf.has('RTE_CRYPTO_NULL')
+        test_args += ['--vdev=crypto_null0']
+    endif
+    if dpdk_conf.has('RTE_DMA_SKELETON')
+        test_args += ['--vdev=dma_skeleton0']
+    endif
+    if dpdk_conf.has('RTE_EVENT_SKELETON')
+        test_args += ['--vdev=event_skeleton0']
+    endif
+    if dpdk_conf.has('RTE_NET_NULL')
+        test_args += ['--vdev=net_null0']
+    endif
+    if dpdk_conf.has('RTE_RAW_SKELETON')
+        test_args += ['--vdev=rawdev_skeleton0']
+    endif
+    test_args += ['-a', '0000:00:00.0']
+    test('telemetry_all', find_program('test_telemetry.sh'),
+            args: test_args,
+            timeout : timeout_seconds_fast,
+            is_parallel : false,
+            suite : 'fast-tests')
+endif
diff --git a/app/test/test_telemetry.sh b/app/test/suites/test_telemetry.sh
similarity index 100%
rename from app/test/test_telemetry.sh
rename to app/test/suites/test_telemetry.sh
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v6 08/11] app/test: add debug test suite
  2023-08-16 15:34 ` [PATCH v6 00/11] " Bruce Richardson
                     ` (6 preceding siblings ...)
  2023-08-16 15:34   ` [PATCH v6 07/11] app/test: add test case for scripted telemetry commands Bruce Richardson
@ 2023-08-16 15:34   ` Bruce Richardson
  2023-08-16 15:34   ` [PATCH v6 09/11] examples/l3fwd: make eventdev an optional dependency Bruce Richardson
                     ` (3 subsequent siblings)
  11 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-16 15:34 UTC (permalink / raw)
  To: dev; +Cc: david.marchand, probb, mb, Bruce Richardson

Add in for all platforms, the suite of tests to dump out the structure
information available. Since the commands are defined in commands.c on
all OS's, do not limit their presence to just the non-windows OS's.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 app/test/suites/meson.build | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/app/test/suites/meson.build b/app/test/suites/meson.build
index eddf332156..19a6b902fa 100644
--- a/app/test/suites/meson.build
+++ b/app/test/suites/meson.build
@@ -102,3 +102,23 @@ if not is_windows and dpdk_conf.has('RTE_LIB_TELEMETRY')
             is_parallel : false,
             suite : 'fast-tests')
 endif
+
+# dump tests are defined in commands.c, and not easily extractable
+dump_test_names = [
+        'dump_devargs',
+        'dump_log_types',
+        'dump_malloc_heaps',
+        'dump_malloc_stats',
+        'dump_mempool',
+        'dump_memzone',
+        'dump_physmem',
+        'dump_ring',
+        'dump_struct_sizes',
+]
+foreach arg : dump_test_names
+    test(arg, dpdk_test,
+                env : ['DPDK_TEST=' + arg],
+                timeout : timeout_seconds_fast,
+                is_parallel : false,
+                suite : 'debug-tests')
+endforeach
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v6 09/11] examples/l3fwd: make eventdev an optional dependency
  2023-08-16 15:34 ` [PATCH v6 00/11] " Bruce Richardson
                     ` (7 preceding siblings ...)
  2023-08-16 15:34   ` [PATCH v6 08/11] app/test: add debug test suite Bruce Richardson
@ 2023-08-16 15:34   ` Bruce Richardson
  2023-08-16 15:34   ` [PATCH v6 10/11] build: make most device classes optional Bruce Richardson
                     ` (2 subsequent siblings)
  11 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-16 15:34 UTC (permalink / raw)
  To: dev; +Cc: david.marchand, probb, mb, Bruce Richardson

With l3fwd being a very commonly used example app, and built as part of
our CI tests, we need to ensure it's buildable with just about all
supported DPDK configurations.  To enable l3fwd application to be built
when the eventdev library is disabled, we need to compile in the
eventdev support conditionally.  Thankfully, the eventdev support is
pretty self-contained, with only the main.c file having more than a
couple of ifdefs.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
---
 examples/l3fwd/l3fwd_em.c                  |  2 +
 examples/l3fwd/l3fwd_em_hlm.h              |  2 +
 examples/l3fwd/l3fwd_event.c               |  2 +
 examples/l3fwd/l3fwd_event.h               |  7 ++-
 examples/l3fwd/l3fwd_event_generic.c       |  2 +
 examples/l3fwd/l3fwd_event_internal_port.c |  2 +
 examples/l3fwd/l3fwd_fib.c                 |  2 +
 examples/l3fwd/l3fwd_lpm.c                 |  2 +
 examples/l3fwd/main.c                      | 65 +++++++++++++++++-----
 examples/l3fwd/meson.build                 |  5 +-
 10 files changed, 74 insertions(+), 17 deletions(-)

diff --git a/examples/l3fwd/l3fwd_em.c b/examples/l3fwd/l3fwd_em.c
index 476ac0c54f..40e102b38a 100644
--- a/examples/l3fwd/l3fwd_em.c
+++ b/examples/l3fwd/l3fwd_em.c
@@ -663,6 +663,7 @@ em_main_loop(__rte_unused void *dummy)
 	return 0;
 }
 
+#ifdef RTE_LIB_EVENTDEV
 static __rte_always_inline void
 em_event_loop_single(struct l3fwd_event_resources *evt_rsrc,
 		const uint8_t flags)
@@ -959,6 +960,7 @@ em_event_main_loop_tx_q_burst_vector(__rte_unused void *dummy)
 	em_event_loop_vector(evt_rsrc, L3FWD_EVENT_TX_ENQ);
 	return 0;
 }
+#endif
 
 /* Initialize exact match (hash) parameters. 8< */
 void
diff --git a/examples/l3fwd/l3fwd_em_hlm.h b/examples/l3fwd/l3fwd_em_hlm.h
index 2e11eefad7..31cda9ddc1 100644
--- a/examples/l3fwd/l3fwd_em_hlm.h
+++ b/examples/l3fwd/l3fwd_em_hlm.h
@@ -255,6 +255,7 @@ l3fwd_em_send_packets(int nb_rx, struct rte_mbuf **pkts_burst, uint16_t portid,
 	send_packets_multi(qconf, pkts_burst, dst_port, nb_rx);
 }
 
+#ifdef RTE_LIB_EVENTDEV
 /*
  * Buffer optimized handling of events, invoked
  * from main_loop.
@@ -347,5 +348,6 @@ l3fwd_em_process_event_vector(struct rte_event_vector *vec,
 
 	process_event_vector(vec, dst_port);
 }
+#endif /* RTE_LIB_EVENTDEV */
 
 #endif /* __L3FWD_EM_HLM_H__ */
diff --git a/examples/l3fwd/l3fwd_event.c b/examples/l3fwd/l3fwd_event.c
index 32906ab08d..d72a4138cc 100644
--- a/examples/l3fwd/l3fwd_event.c
+++ b/examples/l3fwd/l3fwd_event.c
@@ -2,6 +2,7 @@
  * Copyright(C) 2019 Marvell International Ltd.
  */
 
+#ifdef RTE_LIB_EVENTDEV
 #include <stdbool.h>
 #include <getopt.h>
 
@@ -341,3 +342,4 @@ l3fwd_event_worker_cleanup(uint8_t event_d_id, uint8_t event_p_id,
 	rte_event_port_quiesce(event_d_id, event_p_id, l3fwd_event_port_flush,
 			       NULL);
 }
+#endif /* #ifdef RTE_LIB_EVENTDEV */
diff --git a/examples/l3fwd/l3fwd_event.h b/examples/l3fwd/l3fwd_event.h
index e21817c36b..1fd6fe4a78 100644
--- a/examples/l3fwd/l3fwd_event.h
+++ b/examples/l3fwd/l3fwd_event.h
@@ -6,11 +6,13 @@
 #define __L3FWD_EVENTDEV_H__
 
 #include <rte_common.h>
+#include <rte_service.h>
+#include <rte_spinlock.h>
+
+#ifdef RTE_LIB_EVENTDEV
 #include <rte_eventdev.h>
 #include <rte_event_eth_rx_adapter.h>
 #include <rte_event_eth_tx_adapter.h>
-#include <rte_service.h>
-#include <rte_spinlock.h>
 
 #include "l3fwd.h"
 
@@ -164,4 +166,5 @@ void l3fwd_event_worker_cleanup(uint8_t event_d_id, uint8_t event_p_id,
 				struct rte_event events[], uint16_t nb_enq,
 				uint16_t nb_deq, uint8_t is_vector);
 
+#endif /* ifdef RTE_LIB_EVENTDEV */
 #endif /* __L3FWD_EVENTDEV_H__ */
diff --git a/examples/l3fwd/l3fwd_event_generic.c b/examples/l3fwd/l3fwd_event_generic.c
index c80573fc58..ddb6e5c38d 100644
--- a/examples/l3fwd/l3fwd_event_generic.c
+++ b/examples/l3fwd/l3fwd_event_generic.c
@@ -2,6 +2,7 @@
  * Copyright(C) 2019 Marvell International Ltd.
  */
 
+#ifdef RTE_LIB_EVENTDEV
 #include <stdbool.h>
 
 #include "l3fwd.h"
@@ -309,3 +310,4 @@ l3fwd_event_set_generic_ops(struct l3fwd_event_setup_ops *ops)
 	ops->event_port_setup = l3fwd_event_port_setup_generic;
 	ops->adapter_setup = l3fwd_rx_tx_adapter_setup_generic;
 }
+#endif /* RTE_LIB_EVENTDEV */
diff --git a/examples/l3fwd/l3fwd_event_internal_port.c b/examples/l3fwd/l3fwd_event_internal_port.c
index 32cf657148..cb49a8b9fa 100644
--- a/examples/l3fwd/l3fwd_event_internal_port.c
+++ b/examples/l3fwd/l3fwd_event_internal_port.c
@@ -2,6 +2,7 @@
  * Copyright(C) 2019 Marvell International Ltd.
  */
 
+#ifdef RTE_LIB_EVENTDEV
 #include <stdbool.h>
 
 #include "l3fwd.h"
@@ -311,3 +312,4 @@ l3fwd_event_set_internal_port_ops(struct l3fwd_event_setup_ops *ops)
 	ops->event_port_setup = l3fwd_event_port_setup_internal_port;
 	ops->adapter_setup = l3fwd_rx_tx_adapter_setup_internal_port;
 }
+#endif /* RTE_LIB_EVENTDEV */
diff --git a/examples/l3fwd/l3fwd_fib.c b/examples/l3fwd/l3fwd_fib.c
index 18398492ae..6a21984415 100644
--- a/examples/l3fwd/l3fwd_fib.c
+++ b/examples/l3fwd/l3fwd_fib.c
@@ -253,6 +253,7 @@ fib_main_loop(__rte_unused void *dummy)
 	return 0;
 }
 
+#ifdef RTE_LIB_EVENTDEV
 /* One eventdev loop for single and burst using fib. */
 static __rte_always_inline void
 fib_event_loop(struct l3fwd_event_resources *evt_rsrc,
@@ -635,6 +636,7 @@ fib_event_main_loop_tx_q_burst_vector(__rte_unused void *dummy)
 	fib_event_loop_vector(evt_rsrc, L3FWD_EVENT_TX_ENQ);
 	return 0;
 }
+#endif
 
 /* Function to setup fib. 8< */
 void
diff --git a/examples/l3fwd/l3fwd_lpm.c b/examples/l3fwd/l3fwd_lpm.c
index 4ac1925c84..a484a33089 100644
--- a/examples/l3fwd/l3fwd_lpm.c
+++ b/examples/l3fwd/l3fwd_lpm.c
@@ -226,6 +226,7 @@ lpm_main_loop(__rte_unused void *dummy)
 	return 0;
 }
 
+#ifdef RTE_LIB_EVENTDEV
 static __rte_always_inline uint16_t
 lpm_process_event_pkt(const struct lcore_conf *lconf, struct rte_mbuf *mbuf)
 {
@@ -554,6 +555,7 @@ lpm_event_main_loop_tx_q_burst_vector(__rte_unused void *dummy)
 	lpm_event_loop_vector(evt_rsrc, L3FWD_EVENT_TX_ENQ);
 	return 0;
 }
+#endif
 
 void
 setup_lpm(const int socketid)
diff --git a/examples/l3fwd/main.c b/examples/l3fwd/main.c
index a4f061537e..6063eb1399 100644
--- a/examples/l3fwd/main.c
+++ b/examples/l3fwd/main.c
@@ -135,8 +135,10 @@ static struct rte_eth_conf port_conf = {
 
 uint32_t max_pkt_len;
 
-static struct rte_mempool *pktmbuf_pool[RTE_MAX_ETHPORTS][NB_SOCKETS];
+#ifdef RTE_LIB_EVENTDEV
 static struct rte_mempool *vector_pool[RTE_MAX_ETHPORTS];
+#endif
+static struct rte_mempool *pktmbuf_pool[RTE_MAX_ETHPORTS][NB_SOCKETS];
 static uint8_t lkp_per_socket[NB_SOCKETS];
 
 struct l3fwd_lkp_mode {
@@ -398,8 +400,10 @@ print_usage(const char *prgname)
 		" [--parse-ptype]"
 		" [--per-port-pool]"
 		" [--mode]"
+#ifdef RTE_LIB_EVENTDEV
 		" [--eventq-sched]"
 		" [--event-vector [--event-vector-size SIZE] [--event-vector-tmo NS]]"
+#endif
 		" [-E]"
 		" [-L]\n\n"
 
@@ -422,6 +426,7 @@ print_usage(const char *prgname)
 		"  --per-port-pool: Use separate buffer pool per port\n"
 		"  --mode: Packet transfer mode for I/O, poll or eventdev\n"
 		"          Default mode = poll\n"
+#ifdef RTE_LIB_EVENTDEV
 		"  --eventq-sched: Event queue synchronization method\n"
 		"                  ordered, atomic or parallel.\n"
 		"                  Default: atomic\n"
@@ -432,6 +437,7 @@ print_usage(const char *prgname)
 		"  --event-vector:  Enable event vectorization.\n"
 		"  --event-vector-size: Max vector size if event vectorization is enabled.\n"
 		"  --event-vector-tmo: Max timeout to form vector in nanoseconds if event vectorization is enabled\n"
+#endif
 		"  -E : Enable exact match, legacy flag please use --lookup=em instead\n"
 		"  -L : Enable longest prefix match, legacy flag please use --lookup=lpm instead\n"
 		"  --rule_ipv4=FILE: Specify the ipv4 rules entries file.\n"
@@ -559,14 +565,16 @@ parse_eth_dest(const char *optarg)
 }
 
 static void
-parse_mode(const char *optarg)
+parse_mode(const char *optarg __rte_unused)
 {
+#ifdef RTE_LIB_EVENTDEV
 	struct l3fwd_event_resources *evt_rsrc = l3fwd_get_eventdev_rsrc();
 
 	if (!strcmp(optarg, "poll"))
 		evt_rsrc->enabled = false;
 	else if (!strcmp(optarg, "eventdev"))
 		evt_rsrc->enabled = true;
+#endif
 }
 
 static void
@@ -601,6 +609,7 @@ parse_queue_size(const char *queue_size_arg, uint16_t *queue_size, int rx)
 	*queue_size = value;
 }
 
+#ifdef RTE_LIB_EVENTDEV
 static void
 parse_eventq_sched(const char *optarg)
 {
@@ -631,6 +640,7 @@ parse_event_eth_rx_queues(const char *eth_rx_queues)
 
 	evt_rsrc->eth_rx_queues = num_eth_rx_queues;
 }
+#endif
 
 static int
 parse_lookup(const char *optarg)
@@ -756,9 +766,11 @@ parse_args(int argc, char **argv)
 	int option_index;
 	char *prgname = argv[0];
 	uint8_t lcore_params = 0;
+#ifdef RTE_LIB_EVENTDEV
 	uint8_t eventq_sched = 0;
 	uint8_t eth_rx_q = 0;
 	struct l3fwd_event_resources *evt_rsrc = l3fwd_get_eventdev_rsrc();
+#endif
 
 	argvopt = argv;
 
@@ -850,6 +862,7 @@ parse_args(int argc, char **argv)
 			parse_mode(optarg);
 			break;
 
+#ifdef RTE_LIB_EVENTDEV
 		case CMD_LINE_OPT_EVENTQ_SYNC_NUM:
 			parse_eventq_sched(optarg);
 			eventq_sched = 1;
@@ -860,6 +873,20 @@ parse_args(int argc, char **argv)
 			eth_rx_q = 1;
 			break;
 
+		case CMD_LINE_OPT_ENABLE_VECTOR_NUM:
+			printf("event vectorization is enabled\n");
+			evt_rsrc->vector_enabled = 1;
+			break;
+
+		case CMD_LINE_OPT_VECTOR_SIZE_NUM:
+			evt_rsrc->vector_size = strtol(optarg, NULL, 10);
+			break;
+
+		case CMD_LINE_OPT_VECTOR_TMO_NS_NUM:
+			evt_rsrc->vector_tmo_ns = strtoull(optarg, NULL, 10);
+			break;
+#endif
+
 		case CMD_LINE_OPT_LOOKUP_NUM:
 			if (lookup_mode != L3FWD_LOOKUP_DEFAULT) {
 				fprintf(stderr, "Only one lookup mode is allowed at a time!\n");
@@ -875,16 +902,6 @@ parse_args(int argc, char **argv)
 				return -1;
 			break;
 
-		case CMD_LINE_OPT_ENABLE_VECTOR_NUM:
-			printf("event vectorization is enabled\n");
-			evt_rsrc->vector_enabled = 1;
-			break;
-		case CMD_LINE_OPT_VECTOR_SIZE_NUM:
-			evt_rsrc->vector_size = strtol(optarg, NULL, 10);
-			break;
-		case CMD_LINE_OPT_VECTOR_TMO_NS_NUM:
-			evt_rsrc->vector_tmo_ns = strtoull(optarg, NULL, 10);
-			break;
 		case CMD_LINE_OPT_RULE_IPV4_NUM:
 			l3fwd_set_rule_ipv4_name(optarg);
 			break;
@@ -900,6 +917,8 @@ parse_args(int argc, char **argv)
 		}
 	}
 
+	RTE_SET_USED(lcore_params); /* needed if no eventdev block */
+#ifdef RTE_LIB_EVENTDEV
 	if (evt_rsrc->enabled && lcore_params) {
 		fprintf(stderr, "lcore config is not valid when event mode is selected\n");
 		return -1;
@@ -927,6 +946,7 @@ parse_args(int argc, char **argv)
 			"vector timeout set to default (%" PRIu64 " ns)\n",
 			evt_rsrc->vector_tmo_ns);
 	}
+#endif
 
 	/*
 	 * Nothing is selected, pick longest-prefix match
@@ -962,7 +982,9 @@ print_ethaddr(const char *name, const struct rte_ether_addr *eth_addr)
 int
 init_mem(uint16_t portid, unsigned int nb_mbuf)
 {
+#ifdef RTE_LIB_EVENTDEV
 	struct l3fwd_event_resources *evt_rsrc = l3fwd_get_eventdev_rsrc();
+#endif
 	struct lcore_conf *qconf;
 	int socketid;
 	unsigned lcore_id;
@@ -1007,6 +1029,7 @@ init_mem(uint16_t portid, unsigned int nb_mbuf)
 			}
 		}
 
+#ifdef RTE_LIB_EVENTDEV
 		if (evt_rsrc->vector_enabled && vector_pool[portid] == NULL) {
 			unsigned int nb_vec;
 
@@ -1025,6 +1048,7 @@ init_mem(uint16_t portid, unsigned int nb_mbuf)
 				printf("Allocated vector pool for port %d\n",
 				       portid);
 		}
+#endif
 
 		qconf = &lcore_conf[lcore_id];
 		qconf->ipv4_lookup_struct =
@@ -1406,6 +1430,7 @@ l3fwd_service_enable(uint32_t service_id)
 	return 0;
 }
 
+#ifdef RTE_LIB_EVENTDEV
 static void
 l3fwd_event_service_setup(void)
 {
@@ -1458,16 +1483,20 @@ l3fwd_event_service_setup(void)
 		l3fwd_service_enable(service_id);
 	}
 }
+#endif
 
 int
 main(int argc, char **argv)
 {
+#ifdef RTE_LIB_EVENTDEV
 	struct l3fwd_event_resources *evt_rsrc;
+	int i;
+#endif
 	struct lcore_conf *qconf;
 	uint16_t queueid, portid;
 	unsigned int lcore_id;
 	uint8_t queue;
-	int i, ret;
+	int ret;
 
 	/* init EAL */
 	ret = rte_eal_init(argc, argv);
@@ -1487,7 +1516,9 @@ main(int argc, char **argv)
 		*(uint64_t *)(val_eth + portid) = dest_eth_addr[portid];
 	}
 
+#ifdef RTE_LIB_EVENTDEV
 	evt_rsrc = l3fwd_get_eventdev_rsrc();
+#endif
 	/* parse application arguments (after the EAL ones) */
 	ret = parse_args(argc, argv);
 	if (ret < 0)
@@ -1499,6 +1530,7 @@ main(int argc, char **argv)
 	/* Add the config file rules */
 	l3fwd_lkp.read_config_files();
 
+#ifdef RTE_LIB_EVENTDEV
 	evt_rsrc->per_port_pool = per_port_pool;
 	evt_rsrc->pkt_pool = pktmbuf_pool;
 	evt_rsrc->vec_pool = vector_pool;
@@ -1514,6 +1546,7 @@ main(int argc, char **argv)
 			l3fwd_lkp.main_loop = evt_rsrc->ops.lpm_event_loop;
 		l3fwd_event_service_setup();
 	} else
+#endif
 		l3fwd_poll_resource_setup();
 
 	/* start ports */
@@ -1562,6 +1595,8 @@ main(int argc, char **argv)
 	ret = 0;
 	/* launch per-lcore init on every lcore */
 	rte_eal_mp_remote_launch(l3fwd_lkp.main_loop, NULL, CALL_MAIN);
+
+#ifdef RTE_LIB_EVENTDEV
 	if (evt_rsrc->enabled) {
 		for (i = 0; i < evt_rsrc->rx_adptr.nb_rx_adptr; i++)
 			rte_event_eth_rx_adapter_stop(
@@ -1589,7 +1624,9 @@ main(int argc, char **argv)
 		rte_event_dev_stop(evt_rsrc->event_d_id);
 		rte_event_dev_close(evt_rsrc->event_d_id);
 
-	} else {
+	} else
+#endif
+	{
 		rte_eal_mp_wait_lcore();
 
 		RTE_ETH_FOREACH_DEV(portid) {
diff --git a/examples/l3fwd/meson.build b/examples/l3fwd/meson.build
index b40244a941..c25de77bba 100644
--- a/examples/l3fwd/meson.build
+++ b/examples/l3fwd/meson.build
@@ -7,7 +7,7 @@
 # DPDK instance, use 'make'
 
 allow_experimental_apis = true
-deps += ['acl', 'hash', 'lpm', 'fib', 'eventdev']
+deps += ['acl', 'hash', 'lpm', 'fib']
 sources = files(
         'l3fwd_acl.c',
         'l3fwd_em.c',
@@ -18,3 +18,6 @@ sources = files(
         'l3fwd_lpm.c',
         'main.c',
 )
+if dpdk_conf.has('RTE_LIB_EVENTDEV')
+    deps += 'eventdev'
+endif
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v6 10/11] build: make most device classes optional
  2023-08-16 15:34 ` [PATCH v6 00/11] " Bruce Richardson
                     ` (8 preceding siblings ...)
  2023-08-16 15:34   ` [PATCH v6 09/11] examples/l3fwd: make eventdev an optional dependency Bruce Richardson
@ 2023-08-16 15:34   ` Bruce Richardson
  2023-08-16 15:34   ` [PATCH v6 11/11] build: expand list of optional libraries Bruce Richardson
  2023-08-23  7:33   ` [PATCH v6 00/11] " David Marchand
  11 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-16 15:34 UTC (permalink / raw)
  To: dev; +Cc: david.marchand, probb, mb, Bruce Richardson

Apart from ethdev and cryptodev, which have lots of components and tests
which depend on them, we can make the device class libraries optional
without too much work.

This patch marks:
* bbdev,
* compressdev,
* dmadev,
* eventdev,
* mldev,
* rawdev,
* regexdev
optional, and ensures that DPDK - including tests - can be built with
these components disabled.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
---
 lib/meson.build | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/lib/meson.build b/lib/meson.build
index 92cbd6bb66..1a78c8c984 100644
--- a/lib/meson.build
+++ b/lib/meson.build
@@ -67,8 +67,12 @@ libraries = [
 ]
 
 optional_libs = [
+        'bbdev',
         'bitratestats',
         'cfgfile',
+        'compressdev',
+        'dmadev',
+        'eventdev',
         'gpudev',
         'graph',
         'gro',
@@ -76,11 +80,14 @@ optional_libs = [
         'jobstats',
         'latencystats',
         'metrics',
+        'mldev',
         'node',
         'pdump',
         'pipeline',
         'port',
         'power',
+        'rawdev',
+        'regexdev',
         'table',
         'vhost',
 ]
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* [PATCH v6 11/11] build: expand list of optional libraries
  2023-08-16 15:34 ` [PATCH v6 00/11] " Bruce Richardson
                     ` (9 preceding siblings ...)
  2023-08-16 15:34   ` [PATCH v6 10/11] build: make most device classes optional Bruce Richardson
@ 2023-08-16 15:34   ` Bruce Richardson
  2023-08-23  7:33   ` [PATCH v6 00/11] " David Marchand
  11 siblings, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-16 15:34 UTC (permalink / raw)
  To: dev; +Cc: david.marchand, probb, mb, Bruce Richardson

With the unit test build now with individual per-file dependencies, we
can more easily expand the list of optional libraries. Add 8 new
libraries to the optional list.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
---
 lib/meson.build | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/lib/meson.build b/lib/meson.build
index 1a78c8c984..0c9dc8b3b6 100644
--- a/lib/meson.build
+++ b/lib/meson.build
@@ -69,25 +69,33 @@ libraries = [
 optional_libs = [
         'bbdev',
         'bitratestats',
+        'bpf',
         'cfgfile',
         'compressdev',
+        'distributor',
         'dmadev',
+        'efd',
         'eventdev',
         'gpudev',
         'graph',
         'gro',
         'gso',
+        'ip_frag',
         'jobstats',
         'latencystats',
+        'member',
         'metrics',
         'mldev',
         'node',
+        'pcapng',
         'pdump',
         'pipeline',
         'port',
         'power',
         'rawdev',
         'regexdev',
+        'reorder',
+        'sched',
         'table',
         'vhost',
 ]
-- 
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

* RE: [PATCH v6 01/11] mempool: fix default ops for an empty mempool
  2023-08-16 15:34   ` [PATCH v6 01/11] mempool: fix default ops for an empty mempool Bruce Richardson
@ 2023-08-16 16:32     ` Morten Brørup
  0 siblings, 0 replies; 93+ messages in thread
From: Morten Brørup @ 2023-08-16 16:32 UTC (permalink / raw)
  To: Bruce Richardson, dev; +Cc: david.marchand, probb, stable

> From: Bruce Richardson [mailto:bruce.richardson@intel.com]
> Sent: Wednesday, 16 August 2023 17.34
> 
> From: David Marchand <david.marchand@redhat.com>
> 
> An empty mempool's ops were not initialised to a default value wrt to
> what the application requested via the flags parameter.  As
> rte_mempool_create() relies on rte_mempool_create_empty(), simply move
> this ops initialisation to rte_mempool_create_empty().
> 
> Fixes: aa10457eb4c2 ("mempool: make mempool populate and free api
> public")
> Cc: stable@dpdk.org
> 
> Signed-off-by: David Marchand <david.marchand@redhat.com>
> Reviewed-by: Bruce Richardson <bruce.richardson@intel.com>
> ---

I would have moved it to the exact same location.

Reviewed-by: Morten Brørup <mb@smartsharesystems.com>


^ permalink raw reply	[flat|nested] 93+ messages in thread

* Re: [PATCH v5 04/10] app/test: build using per-file dependency matrix
  2023-08-16 14:40         ` David Marchand
@ 2023-08-16 18:29           ` Patrick Robb
  2023-08-16 19:26             ` David Marchand
  2023-08-17  7:27             ` David Marchand
  0 siblings, 2 replies; 93+ messages in thread
From: Patrick Robb @ 2023-08-16 18:29 UTC (permalink / raw)
  To: David Marchand
  Cc: Bruce Richardson, dev, ci, Morten Brørup,
	Honnappa Nagarahalli, Ruifeng Wang (Arm Technology China),
	Thomas Monjalon, Aaron Conole

[-- Attachment #1: Type: text/plain, Size: 2872 bytes --]

On Wed, Aug 16, 2023 at 10:40 AM David Marchand <david.marchand@redhat.com>
wrote:

> Patrick, Bruce,
>
> If it was reported, I either missed it or forgot about it, sorry.
> Can you (re)share the context?


> >
> > Does the test suite pass if the mlx5 driver is disabled in the build?
> That
> > could confirm or refute the suspicion of where the issue is, and also
> > provide a temporary workaround while this set is merged (possibly
> including
> > support for disabling specific tests, as I suggested in my other email).
>
> Or disabling the driver as Bruce proposes.
>
 Okay, we ran the test with the mlx5 driver disabled, and it still fails.
So, this might be more of an ARM architecture issue. Ruifeng, are you still
seeing this on your test bed?

@David you didn't miss anything, we had a unicast with ARM when setting up
the new arm container runners for unit testing a few months back. Ruifeng
also noticed the same issue and speculated about mlx5 memory leaks. He
raised the possibility of disabling the mlx5 driver too, but that option
isn't great since we want to have a uniform build process (as much as
possible) for our unit testing. Anyways, now we know that that isn't
relevant. I'll forward the thread to you in any case - let me know if you
have any ideas.

>
> >
> > /Bruce
> >
> > PS: Are there any other workarounds inside the test/DTS/CI systems that
> > involve patching sources? If so, it would be good to get a list that we
> can
> > work through removing by putting place proper fixes or workarounds, as
> > changing sources for testing like this blocks future patch acceptance.


> Patching sources from the test tool is a poor solution.
> In general, developers won't be aware of source patching and will
> waste time trying to understand why they can't reproduce what the CI
> reports (it happened to me with DTS on the interrupt stuff with vhost,
> at least).
>
> For this specific case of skipping a test, if nobody can fix the
> issue, I prefer if the CI can skip some "known broken in my lab" tests
> via some meson configuration.
> And, such configuration should be easy to catch in the test report.
>
> I strongly agree on all points, which is why I said it was probably a good
thing anyhow for us to lose this ability. In the case of the disabled
fast-test for arm, that was a new discovery coming from adding new
environments, not a regression introduced by a patch, and I don't think it
made sense then to block the introduction of the entire unit test coverage
for arm while they looked into this issue. If it's possible to introduce
meson configure functionality to disable specific tests, that does give us
more flexibility. And it's obviously a better process than us doing it at
the CI end.

We don't currently patch source in any other way in our CI testing.

[-- Attachment #2: Type: text/html, Size: 3764 bytes --]

^ permalink raw reply	[flat|nested] 93+ messages in thread

* Re: [PATCH v5 04/10] app/test: build using per-file dependency matrix
  2023-08-16 18:29           ` Patrick Robb
@ 2023-08-16 19:26             ` David Marchand
  2023-08-16 20:38               ` Patrick Robb
  2023-08-18  7:07               ` David Marchand
  2023-08-17  7:27             ` David Marchand
  1 sibling, 2 replies; 93+ messages in thread
From: David Marchand @ 2023-08-16 19:26 UTC (permalink / raw)
  To: Patrick Robb
  Cc: Bruce Richardson, dev, ci, Morten Brørup,
	Honnappa Nagarahalli, Ruifeng Wang (Arm Technology China),
	Thomas Monjalon, Aaron Conole

On Wed, Aug 16, 2023 at 8:30 PM Patrick Robb <probb@iol.unh.edu> wrote:
> On Wed, Aug 16, 2023 at 10:40 AM David Marchand <david.marchand@redhat.com> wrote:
>>
>> Patrick, Bruce,
>>
>> If it was reported, I either missed it or forgot about it, sorry.
>> Can you (re)share the context?
>>
>>
>> >
>> > Does the test suite pass if the mlx5 driver is disabled in the build? That
>> > could confirm or refute the suspicion of where the issue is, and also
>> > provide a temporary workaround while this set is merged (possibly including
>> > support for disabling specific tests, as I suggested in my other email).
>>
>> Or disabling the driver as Bruce proposes.
>
>  Okay, we ran the test with the mlx5 driver disabled, and it still fails. So, this might be more of an ARM architecture issue. Ruifeng, are you still seeing this on your test bed?
>
> @David you didn't miss anything, we had a unicast with ARM when setting up the new arm container runners for unit testing a few months back. Ruifeng also noticed the same issue and speculated about mlx5 memory leaks. He raised the possibility of disabling the mlx5 driver too, but that option isn't great since we want to have a uniform build process (as much as possible) for our unit testing. Anyways, now we know that that isn't relevant. I'll forward the thread to you in any case - let me know if you have any ideas.

The mention of "memtest1" in the mails rings a bell.
I will need more detailed logs, or ideally an env where it is reproduced.


One thing bothers me.. why are we not seeing this failure with ARM for
Bruce v6 series?
Just looking at patchwork, I would think that I can merge Bruce series as is.
https://patchwork.dpdk.org/project/dpdk/patch/20230816153439.551501-12-bruce.richardson@intel.com/


-- 
David Marchand


^ permalink raw reply	[flat|nested] 93+ messages in thread

* Re: [PATCH v5 04/10] app/test: build using per-file dependency matrix
  2023-08-16 19:26             ` David Marchand
@ 2023-08-16 20:38               ` Patrick Robb
  2023-08-18  7:07               ` David Marchand
  1 sibling, 0 replies; 93+ messages in thread
From: Patrick Robb @ 2023-08-16 20:38 UTC (permalink / raw)
  To: David Marchand
  Cc: Bruce Richardson, dev, ci, Morten Brørup,
	Honnappa Nagarahalli, Ruifeng Wang (Arm Technology China),
	Thomas Monjalon, Aaron Conole

[-- Attachment #1: Type: text/plain, Size: 3368 bytes --]

On Wed, Aug 16, 2023 at 3:26 PM David Marchand <david.marchand@redhat.com>
wrote:

> On Wed, Aug 16, 2023 at 8:30 PM Patrick Robb <probb@iol.unh.edu> wrote:
> > On Wed, Aug 16, 2023 at 10:40 AM David Marchand <
> david.marchand@redhat.com> wrote:
> >>
> >> Patrick, Bruce,
> >>
> >> If it was reported, I either missed it or forgot about it, sorry.
> >> Can you (re)share the context?
> >>
> >>
> >> >
> >> > Does the test suite pass if the mlx5 driver is disabled in the build?
> That
> >> > could confirm or refute the suspicion of where the issue is, and also
> >> > provide a temporary workaround while this set is merged (possibly
> including
> >> > support for disabling specific tests, as I suggested in my other
> email).
> >>
> >> Or disabling the driver as Bruce proposes.
> >
> >  Okay, we ran the test with the mlx5 driver disabled, and it still
> fails. So, this might be more of an ARM architecture issue. Ruifeng, are
> you still seeing this on your test bed?
> >
> > @David you didn't miss anything, we had a unicast with ARM when setting
> up the new arm container runners for unit testing a few months back.
> Ruifeng also noticed the same issue and speculated about mlx5 memory leaks.
> He raised the possibility of disabling the mlx5 driver too, but that option
> isn't great since we want to have a uniform build process (as much as
> possible) for our unit testing. Anyways, now we know that that isn't
> relevant. I'll forward the thread to you in any case - let me know if you
> have any ideas.
>
> The mention of "memtest1" in the mails rings a bell.
> I will need more detailed logs, or ideally an env where it is reproduced.
>
> meson-logs/ for the unit tests run with eal_flags_file_prefix_autotest
included shared with you via slack. I also shared the meson test summary,
but of course it's the detailed testlog.txt you care about.

>
> One thing bothers me.. why are we not seeing this failure with ARM for
> Bruce v6 series?
> Just looking at patchwork, I would think that I can merge Bruce series as
> is.
>
> https://patchwork.dpdk.org/project/dpdk/patch/20230816153439.551501-12-bruce.richardson@intel.com/
>
> So, this is a niche edge case, but because we fail to apply the fast-test
filtering script in our jenkinsfile script, we exit without doing any unit
testing and don't save or report any results. Almost always if we fail
doing "unh jenkins scipt" stuff, it's an infra failure, not a problem with
a patch, and we don't want to report a false positive failure result there.
It does further exemplify the danger in our current process, of course.
I'll be glad to not have to do this anymore. I did try to make this point
above, but I don't think I explained it too well.

The only other thing I'll add is that we are going to change our reporting
process soon, to begin our pipeline run on a test/environment combo by
reporting a "pending" result on that test/environent. Then we will
overwrite it with a PASS or FAIL at the end. This helps protect us from
situations like this. For instance, the way this would have played out is
your would have had a label (iol-unit-arm64-testing) which would have had
the initial "PENDING" result reported to it, but it never would have been
updated from pending. So, you would know the CI results were incomplete.

[-- Attachment #2: Type: text/html, Size: 4334 bytes --]

^ permalink raw reply	[flat|nested] 93+ messages in thread

* Re: [PATCH v5 04/10] app/test: build using per-file dependency matrix
  2023-08-16 18:29           ` Patrick Robb
  2023-08-16 19:26             ` David Marchand
@ 2023-08-17  7:27             ` David Marchand
  2023-08-17  8:46               ` Bruce Richardson
  2023-08-17 18:30               ` Patrick Robb
  1 sibling, 2 replies; 93+ messages in thread
From: David Marchand @ 2023-08-17  7:27 UTC (permalink / raw)
  To: Patrick Robb
  Cc: Bruce Richardson, dev, ci, Morten Brørup,
	Honnappa Nagarahalli, Ruifeng Wang (Arm Technology China),
	Thomas Monjalon, Aaron Conole

On Wed, Aug 16, 2023 at 8:30 PM Patrick Robb <probb@iol.unh.edu> wrote:
>> Patching sources from the test tool is a poor solution.
>> In general, developers won't be aware of source patching and will
>> waste time trying to understand why they can't reproduce what the CI
>> reports (it happened to me with DTS on the interrupt stuff with vhost,
>> at least).
>>
>> For this specific case of skipping a test, if nobody can fix the
>> issue, I prefer if the CI can skip some "known broken in my lab" tests
>> via some meson configuration.
>> And, such configuration should be easy to catch in the test report.
>>
> I strongly agree on all points, which is why I said it was probably a good thing anyhow for us to lose this ability. In the case of the disabled fast-test for arm, that was a new discovery coming from adding new environments, not a regression introduced by a patch, and I don't think it made sense then to block the introduction of the entire unit test coverage for arm while they looked into this issue. If it's possible to introduce meson configure functionality to disable specific tests, that does give us more flexibility. And it's obviously a better process than us doing it at the CI end.
>
> We don't currently patch source in any other way in our CI testing.

It is possible to list tests from meson.
Why not simply list the existing tests and filter the non working one?

$ meson test -C <build_dir> --suite fast-tests --list
$ meson test -C <build_dir> <explicit list> --test-args ...

That requires no change in DPDK.


-- 
David Marchand


^ permalink raw reply	[flat|nested] 93+ messages in thread

* Re: [PATCH v5 04/10] app/test: build using per-file dependency matrix
  2023-08-17  7:27             ` David Marchand
@ 2023-08-17  8:46               ` Bruce Richardson
  2023-08-17 18:30               ` Patrick Robb
  1 sibling, 0 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-08-17  8:46 UTC (permalink / raw)
  To: David Marchand
  Cc: Patrick Robb, dev, ci, Morten Brørup, Honnappa Nagarahalli,
	Ruifeng Wang (Arm Technology China),
	Thomas Monjalon, Aaron Conole

On Thu, Aug 17, 2023 at 09:27:50AM +0200, David Marchand wrote:
> On Wed, Aug 16, 2023 at 8:30 PM Patrick Robb <probb@iol.unh.edu> wrote:
> >> Patching sources from the test tool is a poor solution.
> >> In general, developers won't be aware of source patching and will
> >> waste time trying to understand why they can't reproduce what the CI
> >> reports (it happened to me with DTS on the interrupt stuff with vhost,
> >> at least).
> >>
> >> For this specific case of skipping a test, if nobody can fix the
> >> issue, I prefer if the CI can skip some "known broken in my lab" tests
> >> via some meson configuration.
> >> And, such configuration should be easy to catch in the test report.
> >>
> > I strongly agree on all points, which is why I said it was probably a good thing anyhow for us to lose this ability. In the case of the disabled fast-test for arm, that was a new discovery coming from adding new environments, not a regression introduced by a patch, and I don't think it made sense then to block the introduction of the entire unit test coverage for arm while they looked into this issue. If it's possible to introduce meson configure functionality to disable specific tests, that does give us more flexibility. And it's obviously a better process than us doing it at the CI end.
> >
> > We don't currently patch source in any other way in our CI testing.
> 
> It is possible to list tests from meson.
> Why not simply list the existing tests and filter the non working one?
> 
> $ meson test -C <build_dir> --suite fast-tests --list
> $ meson test -C <build_dir> <explicit list> --test-args ...
> 
> That requires no change in DPDK.
> 
The small issue with that is that the broken test is hidden from reports.
While it unblocks things here without having to change DPDK, I wonder if
changing DPDK to support this scenario might be worthwhile. For example, one
idea I'm now thinking of is to have the test binary itself look in the
environment for a DPDK_TEST_SKIPPED variable, and - when processing test
names passed on command-line or environment - report any on that list as
SKIPPED. This at least flags in the reports that a test is not running, and
the test log gives a reason. 

Worth doing? Or is adding a new environment with broken tests rare enough
we don't need this?

Of course, this also seems like a feature request for meson. When running
all tests, meson allows you to skip a test suite, but doesn't allow
skipping a test within a suite.

/Bruce

^ permalink raw reply	[flat|nested] 93+ messages in thread

* Re: [PATCH v5 04/10] app/test: build using per-file dependency matrix
  2023-08-17  7:27             ` David Marchand
  2023-08-17  8:46               ` Bruce Richardson
@ 2023-08-17 18:30               ` Patrick Robb
  1 sibling, 0 replies; 93+ messages in thread
From: Patrick Robb @ 2023-08-17 18:30 UTC (permalink / raw)
  To: David Marchand
  Cc: Bruce Richardson, dev, ci, Morten Brørup,
	Honnappa Nagarahalli, Ruifeng Wang (Arm Technology China),
	Thomas Monjalon, Aaron Conole

[-- Attachment #1: Type: text/plain, Size: 1241 bytes --]

On Thu, Aug 17, 2023 at 3:28 AM David Marchand <david.marchand@redhat.com>
wrote:

> It is possible to list tests from meson.
> Why not simply list the existing tests and filter the non working one?
>
> $ meson test -C <build_dir> --suite fast-tests --list
> $ meson test -C <build_dir> <explicit list> --test-args ...
>
> That requires no change in DPDK.
>
>
> Thanks. I am seeing now that this is possible. So, we could easily solve
this problem using this approach. Thank you David.

At the same time, I do like Bruce's new RFC patch for excluding tests with
env variables. It is a bit cleaner on the testing experience side (I don't
have to list 100 tests in the command) and it makes clear in the logs what
is being skipped which typically would be run. Making that info visible
could be valuable in some cases.

Ultimately, we can proceed on the Community Lab side either way. I guess
the maintainers must decide whether adding in the extra complexity to the
test app is justified in this case.

Also, Bruce, I do see that the symlink you used is working for the case
where we have to (for now) directly use the dpdk-test app. I'm sorry I
didn't have that info on hand this morning during the ci meeting.

[-- Attachment #2: Type: text/html, Size: 1685 bytes --]

^ permalink raw reply	[flat|nested] 93+ messages in thread

* Re: [PATCH v5 04/10] app/test: build using per-file dependency matrix
  2023-08-16 19:26             ` David Marchand
  2023-08-16 20:38               ` Patrick Robb
@ 2023-08-18  7:07               ` David Marchand
  2023-08-18 22:33                 ` Patrick Robb
  1 sibling, 1 reply; 93+ messages in thread
From: David Marchand @ 2023-08-18  7:07 UTC (permalink / raw)
  To: Patrick Robb
  Cc: Bruce Richardson, dev, ci, Morten Brørup,
	Honnappa Nagarahalli, Ruifeng Wang (Arm Technology China),
	Thomas Monjalon, Aaron Conole, Ferruh Yigit

On Wed, Aug 16, 2023 at 9:26 PM David Marchand
<david.marchand@redhat.com> wrote:
>
> On Wed, Aug 16, 2023 at 8:30 PM Patrick Robb <probb@iol.unh.edu> wrote:
> > On Wed, Aug 16, 2023 at 10:40 AM David Marchand <david.marchand@redhat.com> wrote:
> >>
> >> Patrick, Bruce,
> >>
> >> If it was reported, I either missed it or forgot about it, sorry.
> >> Can you (re)share the context?
> >>
> >>
> >> >
> >> > Does the test suite pass if the mlx5 driver is disabled in the build? That
> >> > could confirm or refute the suspicion of where the issue is, and also
> >> > provide a temporary workaround while this set is merged (possibly including
> >> > support for disabling specific tests, as I suggested in my other email).
> >>
> >> Or disabling the driver as Bruce proposes.
> >
> >  Okay, we ran the test with the mlx5 driver disabled, and it still fails. So, this might be more of an ARM architecture issue. Ruifeng, are you still seeing this on your test bed?
> >
> > @David you didn't miss anything, we had a unicast with ARM when setting up the new arm container runners for unit testing a few months back. Ruifeng also noticed the same issue and speculated about mlx5 memory leaks. He raised the possibility of disabling the mlx5 driver too, but that option isn't great since we want to have a uniform build process (as much as possible) for our unit testing. Anyways, now we know that that isn't relevant. I'll forward the thread to you in any case - let me know if you have any ideas.
>
> The mention of "memtest1" in the mails rings a bell.
> I will need more detailed logs, or ideally an env where it is reproduced.

It is a "recurring" yet not so well known issue.
This unit test fails if any part of the DPDK did not release all
(hugepage backed) memory and associated hugepages before exiting.

In your case here, there is a virtio-net device that the container
tries to get its hands on because DPDK scans and probes all available
resources by default (and fails to, in this case, but that's not
important).
Triggering this virtio-net probing makes ethdev allocate its shared
memzone for port data, but nothing in ethdev releases the memzone when
exiting.
Fixing this could be tricky... as the current ethdev code is really
vague around which locks protect what (if anything..).

I think we hit this issue in the past, and avoided it by running the
tests with dynamically linked DPDK binaries (and by doing this, avoid
the net drivers get loaded).

I can see that you are running the unit tests with a static binary in
the report you sent.
I think the default is shared mode, so I wonder what could be the
reason why UNH builds with static here.
In any case, could you have a try and switch to
-Ddefault_library=shared (or remove forcing to static mode)?


-- 
David Marchand


^ permalink raw reply	[flat|nested] 93+ messages in thread

* Re: [PATCH v5 04/10] app/test: build using per-file dependency matrix
  2023-08-18  7:07               ` David Marchand
@ 2023-08-18 22:33                 ` Patrick Robb
  2023-08-18 23:26                   ` Patrick Robb
  0 siblings, 1 reply; 93+ messages in thread
From: Patrick Robb @ 2023-08-18 22:33 UTC (permalink / raw)
  To: David Marchand
  Cc: Bruce Richardson, dev, ci, Morten Brørup,
	Honnappa Nagarahalli, Ruifeng Wang (Arm Technology China),
	Thomas Monjalon, Aaron Conole, Ferruh Yigit

[-- Attachment #1: Type: text/plain, Size: 2348 bytes --]

On Fri, Aug 18, 2023 at 3:07 AM David Marchand <david.marchand@redhat.com>
wrote:

> It is a "recurring" yet not so well known issue.
> This unit test fails if any part of the DPDK did not release all
> (hugepage backed) memory and associated hugepages before exiting.
>
> In your case here, there is a virtio-net device that the container
> tries to get its hands on because DPDK scans and probes all available
> resources by default (and fails to, in this case, but that's not
> important).
> Triggering this virtio-net probing makes ethdev allocate its shared
> memzone for port data, but nothing in ethdev releases the memzone when
> exiting.
> Fixing this could be tricky... as the current ethdev code is really
> vague around which locks protect what (if anything..).
>
> I think we hit this issue in the past, and avoided it by running the
> tests with dynamically linked DPDK binaries (and by doing this, avoid
> the net drivers get loaded).
>
> I can see that you are running the unit tests with a static binary in
> the report you sent.
> I think the default is shared mode, so I wonder what could be the
> reason why UNH builds with static here.
> In any case, could you have a try and switch to
> -Ddefault_library=shared (or remove forcing to static mode)?
>
> Thanks for helping debug this David.

I think the default library actually is static, but let me know if I
misunderstand: https://git.dpdk.org/dpdk/tree/meson.build. If you meson
configure on the output of 'meson setup build' it says static is the
default.

```
        default_options: [
            'buildtype=release',
            'c_std=c11',
            'default_library=static',
            'warning_level=2',
        ],
```

I tried a build forcing default_library=shared, and all fast-tests unit
tests failed. I can look further into this if you think shared mode might
be the best way forward though. But I am assuming for now if static is
indeed the default, that is the way we should build for testing.

For your patch, it looks like there are actually more eal flags test
failures: http://mails.dpdk.org/archives/test-report/2023-August/444170.html.
In any case, for debugging purposes, I will do a run with
the eal_flags_file_prefix_autotest also included and send you the meson
test detail log on slack.

[-- Attachment #2: Type: text/html, Size: 3055 bytes --]

^ permalink raw reply	[flat|nested] 93+ messages in thread

* Re: [PATCH v5 04/10] app/test: build using per-file dependency matrix
  2023-08-18 22:33                 ` Patrick Robb
@ 2023-08-18 23:26                   ` Patrick Robb
  2023-08-21  7:12                     ` David Marchand
  0 siblings, 1 reply; 93+ messages in thread
From: Patrick Robb @ 2023-08-18 23:26 UTC (permalink / raw)
  To: David Marchand
  Cc: Bruce Richardson, dev, ci, Morten Brørup,
	Honnappa Nagarahalli, Ruifeng Wang (Arm Technology China),
	Thomas Monjalon, Aaron Conole, Ferruh Yigit

[-- Attachment #1: Type: text/plain, Size: 593 bytes --]

On Fri, Aug 18, 2023 at 6:33 PM Patrick Robb <probb@iol.unh.edu> wrote:

>
> For your patch, it looks like there are actually more eal flags test
> failures:
> http://mails.dpdk.org/archives/test-report/2023-August/444170.html. In
> any case, for debugging purposes, I will do a run with
> the eal_flags_file_prefix_autotest also included and send you the meson
> test detail log on slack.
>
> Quick follow up - I see with your patch it does pass on
eal_flags_file_prefix_autotest, but as stated fails some others now. The
detailed slack logs are DM'ed to you on slack. Thanks!

[-- Attachment #2: Type: text/html, Size: 1037 bytes --]

^ permalink raw reply	[flat|nested] 93+ messages in thread

* Re: [PATCH v5 04/10] app/test: build using per-file dependency matrix
  2023-08-18 23:26                   ` Patrick Robb
@ 2023-08-21  7:12                     ` David Marchand
  2023-08-21 14:32                       ` Patrick Robb
  0 siblings, 1 reply; 93+ messages in thread
From: David Marchand @ 2023-08-21  7:12 UTC (permalink / raw)
  To: Patrick Robb
  Cc: Bruce Richardson, dev, ci, Morten Brørup,
	Honnappa Nagarahalli, Ruifeng Wang (Arm Technology China),
	Thomas Monjalon, Aaron Conole, Ferruh Yigit

On Sat, Aug 19, 2023 at 1:26 AM Patrick Robb <probb@iol.unh.edu> wrote:
> On Fri, Aug 18, 2023 at 6:33 PM Patrick Robb <probb@iol.unh.edu> wrote:
>>
>>
>> For your patch, it looks like there are actually more eal flags test failures: http://mails.dpdk.org/archives/test-report/2023-August/444170.html. In any case, for debugging purposes, I will do a run with the eal_flags_file_prefix_autotest also included and send you the meson test detail log on slack.
>>
> Quick follow up - I see with your patch it does pass on eal_flags_file_prefix_autotest, but as stated fails some others now. The detailed slack logs are DM'ed to you on slack. Thanks!

Thanks for the report.
Interestingly those failures pointed out that I broke secondary
processes when no port gets initialised in the primary process.

I'll send a v3.


-- 
David Marchand


^ permalink raw reply	[flat|nested] 93+ messages in thread

* Re: [PATCH v5 04/10] app/test: build using per-file dependency matrix
  2023-08-21  7:12                     ` David Marchand
@ 2023-08-21 14:32                       ` Patrick Robb
  0 siblings, 0 replies; 93+ messages in thread
From: Patrick Robb @ 2023-08-21 14:32 UTC (permalink / raw)
  To: David Marchand
  Cc: Bruce Richardson, dev, ci, Morten Brørup,
	Honnappa Nagarahalli, Ruifeng Wang (Arm Technology China),
	Thomas Monjalon, Aaron Conole, Ferruh Yigit

[-- Attachment #1: Type: text/plain, Size: 1746 bytes --]

Hi David,

Responding to your Slack request about merging Bruce's patch ahead of yours
here so there is more visibility.

That is fine, but as you point out it will mean a temporary loss of test
coverage, as it will disable arm unit testing until either 1. your patch
fixing the flags test hits mainline and we can re-enable testing without
failing everyone's patchseries or 2. Bruce's RFC ([RFC PATCH] app/test: add
support for skipping tests) is implemented and made a part of mainline. I
agree with you this is the best thing to do so as to not hold up Bruce and
his series.

On Mon, Aug 21, 2023 at 3:13 AM David Marchand <david.marchand@redhat.com>
wrote:

> On Sat, Aug 19, 2023 at 1:26 AM Patrick Robb <probb@iol.unh.edu> wrote:
> > On Fri, Aug 18, 2023 at 6:33 PM Patrick Robb <probb@iol.unh.edu> wrote:
> >>
> >>
> >> For your patch, it looks like there are actually more eal flags test
> failures:
> http://mails.dpdk.org/archives/test-report/2023-August/444170.html. In
> any case, for debugging purposes, I will do a run with the
> eal_flags_file_prefix_autotest also included and send you the meson test
> detail log on slack.
> >>
> > Quick follow up - I see with your patch it does pass on
> eal_flags_file_prefix_autotest, but as stated fails some others now. The
> detailed slack logs are DM'ed to you on slack. Thanks!
>
> Thanks for the report.
> Interestingly those failures pointed out that I broke secondary
> processes when no port gets initialised in the primary process.
>
> I'll send a v3.
>
>
> --
> David Marchand
>
>

-- 

Patrick Robb

Technical Service Manager

UNH InterOperability Laboratory

21 Madbury Rd, Suite 100, Durham, NH 03824

www.iol.unh.edu

[-- Attachment #2: Type: text/html, Size: 4271 bytes --]

^ permalink raw reply	[flat|nested] 93+ messages in thread

* Re: [PATCH v5 03/10] app/test: make telemetry data test buildable on windows
  2023-08-15 15:10   ` [PATCH v5 03/10] app/test: make telemetry data test buildable on windows Bruce Richardson
@ 2023-08-21 15:50     ` Tyler Retzlaff
  0 siblings, 0 replies; 93+ messages in thread
From: Tyler Retzlaff @ 2023-08-21 15:50 UTC (permalink / raw)
  To: Bruce Richardson; +Cc: dev, ci, Morten Brørup

On Tue, Aug 15, 2023 at 04:10:47PM +0100, Bruce Richardson wrote:
> To enable the building of the telemetry data tests file when building on
> windows, we need to provide a stub implementation. That way, the test
> file is buildable any time the library itself is built.
> 
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> Acked-by: Morten Brørup <mb@smartsharesystems.com>
> ---

Acked-by: Tyler Retzlaff <roretzla@linux.microsoft.com>


^ permalink raw reply	[flat|nested] 93+ messages in thread

* Re: [PATCH v6 04/11] app/test: make telemetry data test buildable on windows
  2023-08-16 15:34   ` [PATCH v6 04/11] app/test: make telemetry data test buildable on windows Bruce Richardson
@ 2023-08-21 15:52     ` Tyler Retzlaff
  0 siblings, 0 replies; 93+ messages in thread
From: Tyler Retzlaff @ 2023-08-21 15:52 UTC (permalink / raw)
  To: Bruce Richardson; +Cc: dev, david.marchand, probb, mb

On Wed, Aug 16, 2023 at 04:34:32PM +0100, Bruce Richardson wrote:
> To enable the building of the telemetry data tests file when building on
> windows, we need to provide a stub implementation. That way, the test
> file is buildable any time the library itself is built.
> 
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> Acked-by: Morten Brørup <mb@smartsharesystems.com>
> ---

let me also ack the latest v6

Acked-by: Tyler Retzlaff <roretzla@linux.microsoft.com>


^ permalink raw reply	[flat|nested] 93+ messages in thread

* Re: [PATCH v6 00/11] expand list of optional libraries
  2023-08-16 15:34 ` [PATCH v6 00/11] " Bruce Richardson
                     ` (10 preceding siblings ...)
  2023-08-16 15:34   ` [PATCH v6 11/11] build: expand list of optional libraries Bruce Richardson
@ 2023-08-23  7:33   ` David Marchand
  11 siblings, 0 replies; 93+ messages in thread
From: David Marchand @ 2023-08-23  7:33 UTC (permalink / raw)
  To: Bruce Richardson; +Cc: dev, probb, mb, Aaron Conole, Thomas Monjalon

Hello Bruce,

On Wed, Aug 16, 2023 at 5:35 PM Bruce Richardson
<bruce.richardson@intel.com> wrote:
>
> This patchset contains changes to DPDK to make the build more configurable
> overall. The changes can be grouped into two areas:
>
> * Firstly, there are changes to make the build of the unit tests more
>   flexible and maintainable. These 7 patches switch the unit tests from
>   having separate hard-coded lists of files and tests, with lots of
>   conditional logic to add/remove items from the list, to instead being
>   a single list of files and dependencies per-file. The actual lists of
>   test-suites are built up dynamically from the list of files whose
>   dependencies are met and therefore will be part of the build.
>   This rework enables...
> * The second set of patches which expands the list of libraries which
>   can be disabled at build-time, while still giving a successful build
>   of DPDK. Overall, after this set is applied, the list of optional
>   libraries in DPDK is approx doubled in size. [15 additions giving
>   31 optional libs]
>
> NOTES:
> * as stated in the RFC, the test binary moves in the build as a result
>   of this set - from app/test/dpdk-test to app/dpdk-test.
> * running the suites of tests no longer works after patch 4 until patch
>   5-7 is applied. If this is an issue, the patches can be merged on
>   apply. However, the individual tests are still runnable through the
>   test binary - it's only through "meson test" that there are issues, so
>   I'm keeping the patches separate since each is relatively complex in its
>   own right.

Compilation is fine between those patches (which is the minimum).
Seeing the size of the changes, I agree it is more readable to keep
them separate.
And I think we can live with unit tests being "broken" (well, absent)
between those patches.


> * A number of files in the test directory were not being built, and seem
>   to have never been built with meson. This requires separate work, outside
>   the scope of the patchset to fix.
> * Similarly, some unit tests were never added to test suites. These can
>   now be found by using grep to find the old REGISTER_TEST_COMMAND macro.
>   Again, adding orphan tests to suites is outside the scope of this set.

As discussed in the thread for patch 5
(https://inbox.dpdk.org/dev/CAJvnSUBcsA3=9CZvnU+g5CJ261X=tJMivcvygO-D=jcu+akEGg@mail.gmail.com/),
merging this series renders ARM container unit tests unavailable at
UNH.
This is not perfect, but I prefer we make progress on the build
framework changes the sooner possible.

This is a temporary state until either the possibility to skip test is
added (and UNH makes use of it instead of patching
app/test/meson.build) or the fixes I proposed for ethdev cleanup are
merged (and UNH removes the patching of meson).
Hopefully, we will solve this remaining issue in the next weeks.


Series applied, thanks for this nice cleanup and enhancement Bruce.

As a followup on the optional libraries topic, next, I will merge the
enable_libs option:
https://patchwork.dpdk.org/project/dpdk/list/?series=29044


-- 
David Marchand


^ permalink raw reply	[flat|nested] 93+ messages in thread

end of thread, other threads:[~2023-08-23  7:33 UTC | newest]

Thread overview: 93+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-07-21 11:51 [RFC PATCH 0/5] replace build code for unit tests Bruce Richardson
2023-07-21 11:51 ` [RFC PATCH 1/5] app/test: add new macros for various test types Bruce Richardson
2023-07-21 11:51 ` [RFC PATCH 2/5] app/test: tag tests with the test type Bruce Richardson
2023-07-21 11:51 ` [RFC PATCH 3/5] app/test: build using per-file dependency matrix Bruce Richardson
2023-07-21 11:51 ` [RFC PATCH 4/5] app/test: define unit tests suites based on test macros Bruce Richardson
2023-07-21 11:51 ` [RFC PATCH 5/5] app/test: add fast test suite to new build infrastructure Bruce Richardson
2023-08-08  8:46 ` [RFC PATCH 0/5] replace build code for unit tests David Marchand
2023-08-09 14:53   ` Patrick Robb
2023-08-09 15:05     ` Bruce Richardson
2023-08-09 15:39       ` Patrick Robb
2023-08-14 15:16 ` [PATCH v2 0/8] expand list of optional libraries Bruce Richardson
2023-08-14 15:16   ` [PATCH v2 1/8] app/test: add new macros for various test types Bruce Richardson
2023-08-14 15:16   ` [PATCH v2 2/8] app/test: tag tests with the test type Bruce Richardson
2023-08-14 15:16   ` [PATCH v2 3/8] app/test: make telemetry data test buildable on windows Bruce Richardson
2023-08-14 15:16   ` [PATCH v2 4/8] app/test: build using per-file dependency matrix Bruce Richardson
2023-08-14 15:16   ` [PATCH v2 5/8] app/test: define unit tests suites based on test macros Bruce Richardson
2023-08-14 15:16   ` [PATCH v2 6/8] examples/l3fwd: make eventdev an optional dependency Bruce Richardson
2023-08-14 15:16   ` [PATCH v2 7/8] build: make most device classes optional Bruce Richardson
2023-08-14 15:16   ` [PATCH v2 8/8] build: expand list of optional libraries Bruce Richardson
2023-08-14 15:22   ` [PATCH v2 0/8] " Bruce Richardson
2023-08-14 18:20 ` [PATCH v3 " Bruce Richardson
2023-08-14 18:20   ` [PATCH v3 1/8] app/test: add new macros for various test types Bruce Richardson
2023-08-14 18:20   ` [PATCH v3 2/8] app/test: tag tests with the test type Bruce Richardson
2023-08-14 18:20   ` [PATCH v3 3/8] app/test: make telemetry data test buildable on windows Bruce Richardson
2023-08-14 18:21   ` [PATCH v3 4/8] app/test: build using per-file dependency matrix Bruce Richardson
2023-08-14 18:21   ` [PATCH v3 5/8] app/test: define unit tests suites based on test macros Bruce Richardson
2023-08-14 18:21   ` [PATCH v3 6/8] examples/l3fwd: make eventdev an optional dependency Bruce Richardson
2023-08-14 18:21   ` [PATCH v3 7/8] build: make most device classes optional Bruce Richardson
2023-08-14 18:21   ` [PATCH v3 8/8] build: expand list of optional libraries Bruce Richardson
2023-08-14 18:33   ` [PATCH v3 0/8] " Morten Brørup
2023-08-15 13:13 ` [PATCH v4 " Bruce Richardson
2023-08-15 13:13   ` [PATCH v4 1/8] app/test: add new macros for various test types Bruce Richardson
2023-08-15 13:13   ` [PATCH v4 2/8] app/test: tag tests with the test type Bruce Richardson
2023-08-15 13:13   ` [PATCH v4 3/8] app/test: make telemetry data test buildable on windows Bruce Richardson
2023-08-15 13:13   ` [PATCH v4 4/8] app/test: build using per-file dependency matrix Bruce Richardson
2023-08-15 13:13   ` [PATCH v4 5/8] app/test: define unit tests suites based on test macros Bruce Richardson
2023-08-15 13:13   ` [PATCH v4 6/8] examples/l3fwd: make eventdev an optional dependency Bruce Richardson
2023-08-15 13:13   ` [PATCH v4 7/8] build: make most device classes optional Bruce Richardson
2023-08-15 13:13   ` [PATCH v4 8/8] build: expand list of optional libraries Bruce Richardson
2023-08-15 13:15   ` [PATCH v4 0/8] " Bruce Richardson
2023-08-15 15:10 ` [PATCH v5 00/10] " Bruce Richardson
2023-08-15 15:10   ` [PATCH v5 01/10] app/test: add new macros for various test types Bruce Richardson
2023-08-15 15:10   ` [PATCH v5 02/10] app/test: tag tests with the test type Bruce Richardson
2023-08-16 14:56     ` David Marchand
2023-08-16 15:05       ` Bruce Richardson
2023-08-15 15:10   ` [PATCH v5 03/10] app/test: make telemetry data test buildable on windows Bruce Richardson
2023-08-21 15:50     ` Tyler Retzlaff
2023-08-15 15:10   ` [PATCH v5 04/10] app/test: build using per-file dependency matrix Bruce Richardson
2023-08-15 19:05     ` Patrick Robb
2023-08-16 10:56       ` Bruce Richardson
2023-08-16 12:55       ` Bruce Richardson
2023-08-16 14:40         ` David Marchand
2023-08-16 18:29           ` Patrick Robb
2023-08-16 19:26             ` David Marchand
2023-08-16 20:38               ` Patrick Robb
2023-08-18  7:07               ` David Marchand
2023-08-18 22:33                 ` Patrick Robb
2023-08-18 23:26                   ` Patrick Robb
2023-08-21  7:12                     ` David Marchand
2023-08-21 14:32                       ` Patrick Robb
2023-08-17  7:27             ` David Marchand
2023-08-17  8:46               ` Bruce Richardson
2023-08-17 18:30               ` Patrick Robb
2023-08-15 15:10   ` [PATCH v5 05/10] app/test: define unit tests suites based on test macros Bruce Richardson
2023-08-16 11:02     ` Bruce Richardson
2023-08-16 11:15       ` David Marchand
2023-08-16 11:40         ` David Marchand
2023-08-16 12:33           ` Bruce Richardson
2023-08-16 13:16             ` Olivier Matz
2023-08-16 13:35               ` Morten Brørup
2023-08-16 13:44                 ` Bruce Richardson
2023-08-16 14:57     ` David Marchand
2023-08-16 15:06       ` Bruce Richardson
2023-08-15 15:10   ` [PATCH v5 06/10] app/test: add test case for scripted telemetry commands Bruce Richardson
2023-08-15 15:10   ` [PATCH v5 07/10] app/test: add debug test suite Bruce Richardson
2023-08-15 15:10   ` [PATCH v5 08/10] examples/l3fwd: make eventdev an optional dependency Bruce Richardson
2023-08-15 15:10   ` [PATCH v5 09/10] build: make most device classes optional Bruce Richardson
2023-08-15 15:12   ` [PATCH v5 10/10] build: expand list of optional libraries Bruce Richardson
2023-08-16 15:34 ` [PATCH v6 00/11] " Bruce Richardson
2023-08-16 15:34   ` [PATCH v6 01/11] mempool: fix default ops for an empty mempool Bruce Richardson
2023-08-16 16:32     ` Morten Brørup
2023-08-16 15:34   ` [PATCH v6 02/11] app/test: add new macros for various test types Bruce Richardson
2023-08-16 15:34   ` [PATCH v6 03/11] app/test: tag tests with the test type Bruce Richardson
2023-08-16 15:34   ` [PATCH v6 04/11] app/test: make telemetry data test buildable on windows Bruce Richardson
2023-08-21 15:52     ` Tyler Retzlaff
2023-08-16 15:34   ` [PATCH v6 05/11] app/test: build using per-file dependency matrix Bruce Richardson
2023-08-16 15:34   ` [PATCH v6 06/11] app/test: define unit tests suites based on test macros Bruce Richardson
2023-08-16 15:34   ` [PATCH v6 07/11] app/test: add test case for scripted telemetry commands Bruce Richardson
2023-08-16 15:34   ` [PATCH v6 08/11] app/test: add debug test suite Bruce Richardson
2023-08-16 15:34   ` [PATCH v6 09/11] examples/l3fwd: make eventdev an optional dependency Bruce Richardson
2023-08-16 15:34   ` [PATCH v6 10/11] build: make most device classes optional Bruce Richardson
2023-08-16 15:34   ` [PATCH v6 11/11] build: expand list of optional libraries Bruce Richardson
2023-08-23  7:33   ` [PATCH v6 00/11] " David Marchand

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).