DPDK patches and discussions
 help / color / mirror / Atom feed
* [RFC PATCH 0/5] replace build code for unit tests
@ 2023-07-21 11:51 Bruce Richardson
  2023-07-21 11:51 ` [RFC PATCH 1/5] app/test: add new macros for various test types Bruce Richardson
                   ` (10 more replies)
  0 siblings, 11 replies; 93+ messages in thread
From: Bruce Richardson @ 2023-07-21 11:51 UTC (permalink / raw)
  To: dev; +Cc: Bruce Richardson

While working on series such as [1] to make DPDK builds more
configurable, the majority of the complexity involved centered on the
unit tests, and ensuring that the individual files were added/removed
from the build as the components were enabled/disabled.

This complexity was exacerbated by:
* C files having multiple dependencies, specifically,
* Some libraries having one set of independent tests, while other
  library tests requiring additional components
* Having to maintain lists of tests in various suites which also needed
  to be updated, depending on component availability.
* Everything having to be checked via if-else clauses, since this was
  not data-driven

This RFC attempts to address these issues by doing the following:
* Creating a per-file dependency list for the autotests, allowing easy
  checks of when a file could be built
* Tagging the various fast-tests/perf-tests/drivers-tests suites via
  macros in the C code. Then scan the C-files we are building to
  construct the test suites based on what is available


This RFC is just an early prototype, and probably needs cleanup and
validation of the dependency lists. Windows support is probably also
broken, given the fact that our current build file maintains custom
lists there. However, I would hope it should not be too big a job to
enable the unit tests for the windows components that are already
available. (Tests for any components not being build on windows are
already removed via the dependency lists).

Another (hopefully less-impactful) side-effect of this change is that
the "test" binary has moved in the build folder. It is now put
along-side the other binaries in "build/app" rather than being in the
subdirectory "build/app/test".

Future work could also be done to scan the C files that are not being
build for tests. Those tests could then be stubbed out in an
auto-genereated file to always return "SKIPPED", meaning that all test
commands are always valid, irrespective of what components are present
or not.

[1] http://patches.dpdk.org/project/dpdk/list/?series=28628&state=10

Bruce Richardson (5):
  app/test: add new macros for various test types
  app/test: tag tests with the test type
  app/test: build using per-file dependency matrix
  app/test: define unit tests suites based on test macros
  app/test: add fast test suite to new build infrastructure

 app/meson.build                               |  15 +-
 app/test/meson.build                          | 796 +++++-------------
 app/test/suites/meson.build                   |  75 ++
 app/test/test.h                               |   9 +-
 app/test/test_acl.c                           |   2 +-
 app/test/test_atomic.c                        |   2 +-
 app/test/test_barrier.c                       |   2 +-
 app/test/test_bitmap.c                        |   2 +-
 app/test/test_bitops.c                        |   2 +-
 app/test/test_bpf.c                           |   4 +-
 app/test/test_byteorder.c                     |   2 +-
 app/test/test_cksum.c                         |   2 +-
 app/test/test_cmdline.c                       |   2 +-
 app/test/test_common.c                        |   2 +-
 app/test/test_cpuflags.c                      |   2 +-
 app/test/test_crc.c                           |   2 +-
 app/test/test_cryptodev.c                     |  38 +-
 app/test/test_cryptodev_asym.c                |   2 +-
 app/test/test_cycles.c                        |   2 +-
 app/test/test_debug.c                         |   2 +-
 app/test/test_devargs.c                       |   2 +-
 app/test/test_distributor.c                   |   2 +-
 app/test/test_distributor_perf.c              |   2 +-
 app/test/test_dmadev.c                        |   2 +-
 app/test/test_eal_flags.c                     |  24 +-
 app/test/test_eal_fs.c                        |   2 +-
 app/test/test_efd.c                           |   2 +-
 app/test/test_efd_perf.c                      |   2 +-
 app/test/test_errno.c                         |   2 +-
 app/test/test_ethdev_link.c                   |   2 +-
 app/test/test_event_ring.c                    |   2 +-
 app/test/test_eventdev.c                      |   2 +-
 app/test/test_fbarray.c                       |   2 +-
 app/test/test_fib.c                           |   4 +-
 app/test/test_fib6.c                          |   4 +-
 app/test/test_fib6_perf.c                     |   2 +-
 app/test/test_fib_perf.c                      |   2 +-
 app/test/test_func_reentrancy.c               |   2 +-
 app/test/test_hash.c                          |   2 +-
 app/test/test_hash_functions.c                |   2 +-
 app/test/test_hash_multiwriter.c              |   2 +-
 app/test/test_hash_perf.c                     |   2 +-
 app/test/test_hash_readwrite.c                |   4 +-
 app/test/test_hash_readwrite_lf_perf.c        |   2 +-
 app/test/test_interrupts.c                    |   2 +-
 app/test/test_ipfrag.c                        |   2 +-
 app/test/test_ipsec.c                         |   2 +-
 app/test/test_ipsec_perf.c                    |   2 +-
 app/test/test_kni.c                           |   2 +-
 app/test/test_kvargs.c                        |   2 +-
 app/test/test_lcores.c                        |   2 +-
 app/test/test_logs.c                          |   2 +-
 app/test/test_lpm.c                           |   2 +-
 app/test/test_lpm6.c                          |   2 +-
 app/test/test_lpm6_perf.c                     |   2 +-
 app/test/test_lpm_perf.c                      |   2 +-
 app/test/test_malloc.c                        |   2 +-
 app/test/test_malloc_perf.c                   |   2 +-
 app/test/test_mbuf.c                          |   2 +-
 app/test/test_mcslock.c                       |   2 +-
 app/test/test_member.c                        |   2 +-
 app/test/test_member_perf.c                   |   2 +-
 app/test/test_memcpy.c                        |   2 +-
 app/test/test_memcpy_perf.c                   |   2 +-
 app/test/test_memory.c                        |   2 +-
 app/test/test_mempool.c                       |   2 +-
 app/test/test_mempool_perf.c                  |   2 +-
 app/test/test_memzone.c                       |   2 +-
 app/test/test_meter.c                         |   2 +-
 app/test/test_mp_secondary.c                  |   2 +-
 app/test/test_per_lcore.c                     |   2 +-
 app/test/test_pflock.c                        |   2 +-
 app/test/test_pie.c                           |   6 +-
 app/test/test_pmd_perf.c                      |   2 +-
 app/test/test_power.c                         |   2 +-
 app/test/test_power_cpufreq.c                 |   2 +-
 app/test/test_power_intel_uncore.c            |   2 +-
 app/test/test_power_kvm_vm.c                  |   2 +-
 app/test/test_prefetch.c                      |   2 +-
 app/test/test_rand_perf.c                     |   2 +-
 app/test/test_rcu_qsbr.c                      |   2 +-
 app/test/test_rcu_qsbr_perf.c                 |   2 +-
 app/test/test_reassembly_perf.c               |   2 +-
 app/test/test_reciprocal_division.c           |   2 +-
 app/test/test_reciprocal_division_perf.c      |   2 +-
 app/test/test_red.c                           |   4 +-
 app/test/test_reorder.c                       |   2 +-
 app/test/test_rib.c                           |   4 +-
 app/test/test_rib6.c                          |   4 +-
 app/test/test_ring.c                          |   2 +-
 app/test/test_ring_perf.c                     |   2 +-
 app/test/test_rwlock.c                        |   8 +-
 app/test/test_sched.c                         |   2 +-
 app/test/test_security.c                      |   2 +-
 app/test/test_seqlock.c                       |   2 +-
 app/test/test_service_cores.c                 |   4 +-
 app/test/test_spinlock.c                      |   2 +-
 app/test/test_stack.c                         |   4 +-
 app/test/test_stack_perf.c                    |   4 +-
 app/test/test_string_fns.c                    |   2 +-
 app/test/test_tailq.c                         |   2 +-
 app/test/test_thash.c                         |   2 +-
 app/test/test_thash_perf.c                    |   2 +-
 app/test/test_threads.c                       |   2 +-
 app/test/test_ticketlock.c                    |   2 +-
 app/test/test_timer.c                         |   2 +-
 app/test/test_timer_perf.c                    |   2 +-
 app/test/test_timer_racecond.c                |   2 +-
 app/test/test_trace.c                         |   2 +-
 app/test/test_trace_perf.c                    |   2 +-
 app/test/test_version.c                       |   2 +-
 buildtools/get-test-suites.py                 |  33 +
 .../has-hugepages.py                          |   0
 buildtools/meson.build                        |   2 +
 114 files changed, 478 insertions(+), 754 deletions(-)
 create mode 100644 app/test/suites/meson.build
 create mode 100644 buildtools/get-test-suites.py
 rename app/test/has_hugepage.py => buildtools/has-hugepages.py (100%)

--
2.39.2


^ permalink raw reply	[flat|nested] 93+ messages in thread

end of thread, other threads:[~2023-08-23  7:33 UTC | newest]

Thread overview: 93+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-07-21 11:51 [RFC PATCH 0/5] replace build code for unit tests Bruce Richardson
2023-07-21 11:51 ` [RFC PATCH 1/5] app/test: add new macros for various test types Bruce Richardson
2023-07-21 11:51 ` [RFC PATCH 2/5] app/test: tag tests with the test type Bruce Richardson
2023-07-21 11:51 ` [RFC PATCH 3/5] app/test: build using per-file dependency matrix Bruce Richardson
2023-07-21 11:51 ` [RFC PATCH 4/5] app/test: define unit tests suites based on test macros Bruce Richardson
2023-07-21 11:51 ` [RFC PATCH 5/5] app/test: add fast test suite to new build infrastructure Bruce Richardson
2023-08-08  8:46 ` [RFC PATCH 0/5] replace build code for unit tests David Marchand
2023-08-09 14:53   ` Patrick Robb
2023-08-09 15:05     ` Bruce Richardson
2023-08-09 15:39       ` Patrick Robb
2023-08-14 15:16 ` [PATCH v2 0/8] expand list of optional libraries Bruce Richardson
2023-08-14 15:16   ` [PATCH v2 1/8] app/test: add new macros for various test types Bruce Richardson
2023-08-14 15:16   ` [PATCH v2 2/8] app/test: tag tests with the test type Bruce Richardson
2023-08-14 15:16   ` [PATCH v2 3/8] app/test: make telemetry data test buildable on windows Bruce Richardson
2023-08-14 15:16   ` [PATCH v2 4/8] app/test: build using per-file dependency matrix Bruce Richardson
2023-08-14 15:16   ` [PATCH v2 5/8] app/test: define unit tests suites based on test macros Bruce Richardson
2023-08-14 15:16   ` [PATCH v2 6/8] examples/l3fwd: make eventdev an optional dependency Bruce Richardson
2023-08-14 15:16   ` [PATCH v2 7/8] build: make most device classes optional Bruce Richardson
2023-08-14 15:16   ` [PATCH v2 8/8] build: expand list of optional libraries Bruce Richardson
2023-08-14 15:22   ` [PATCH v2 0/8] " Bruce Richardson
2023-08-14 18:20 ` [PATCH v3 " Bruce Richardson
2023-08-14 18:20   ` [PATCH v3 1/8] app/test: add new macros for various test types Bruce Richardson
2023-08-14 18:20   ` [PATCH v3 2/8] app/test: tag tests with the test type Bruce Richardson
2023-08-14 18:20   ` [PATCH v3 3/8] app/test: make telemetry data test buildable on windows Bruce Richardson
2023-08-14 18:21   ` [PATCH v3 4/8] app/test: build using per-file dependency matrix Bruce Richardson
2023-08-14 18:21   ` [PATCH v3 5/8] app/test: define unit tests suites based on test macros Bruce Richardson
2023-08-14 18:21   ` [PATCH v3 6/8] examples/l3fwd: make eventdev an optional dependency Bruce Richardson
2023-08-14 18:21   ` [PATCH v3 7/8] build: make most device classes optional Bruce Richardson
2023-08-14 18:21   ` [PATCH v3 8/8] build: expand list of optional libraries Bruce Richardson
2023-08-14 18:33   ` [PATCH v3 0/8] " Morten Brørup
2023-08-15 13:13 ` [PATCH v4 " Bruce Richardson
2023-08-15 13:13   ` [PATCH v4 1/8] app/test: add new macros for various test types Bruce Richardson
2023-08-15 13:13   ` [PATCH v4 2/8] app/test: tag tests with the test type Bruce Richardson
2023-08-15 13:13   ` [PATCH v4 3/8] app/test: make telemetry data test buildable on windows Bruce Richardson
2023-08-15 13:13   ` [PATCH v4 4/8] app/test: build using per-file dependency matrix Bruce Richardson
2023-08-15 13:13   ` [PATCH v4 5/8] app/test: define unit tests suites based on test macros Bruce Richardson
2023-08-15 13:13   ` [PATCH v4 6/8] examples/l3fwd: make eventdev an optional dependency Bruce Richardson
2023-08-15 13:13   ` [PATCH v4 7/8] build: make most device classes optional Bruce Richardson
2023-08-15 13:13   ` [PATCH v4 8/8] build: expand list of optional libraries Bruce Richardson
2023-08-15 13:15   ` [PATCH v4 0/8] " Bruce Richardson
2023-08-15 15:10 ` [PATCH v5 00/10] " Bruce Richardson
2023-08-15 15:10   ` [PATCH v5 01/10] app/test: add new macros for various test types Bruce Richardson
2023-08-15 15:10   ` [PATCH v5 02/10] app/test: tag tests with the test type Bruce Richardson
2023-08-16 14:56     ` David Marchand
2023-08-16 15:05       ` Bruce Richardson
2023-08-15 15:10   ` [PATCH v5 03/10] app/test: make telemetry data test buildable on windows Bruce Richardson
2023-08-21 15:50     ` Tyler Retzlaff
2023-08-15 15:10   ` [PATCH v5 04/10] app/test: build using per-file dependency matrix Bruce Richardson
2023-08-15 19:05     ` Patrick Robb
2023-08-16 10:56       ` Bruce Richardson
2023-08-16 12:55       ` Bruce Richardson
2023-08-16 14:40         ` David Marchand
2023-08-16 18:29           ` Patrick Robb
2023-08-16 19:26             ` David Marchand
2023-08-16 20:38               ` Patrick Robb
2023-08-18  7:07               ` David Marchand
2023-08-18 22:33                 ` Patrick Robb
2023-08-18 23:26                   ` Patrick Robb
2023-08-21  7:12                     ` David Marchand
2023-08-21 14:32                       ` Patrick Robb
2023-08-17  7:27             ` David Marchand
2023-08-17  8:46               ` Bruce Richardson
2023-08-17 18:30               ` Patrick Robb
2023-08-15 15:10   ` [PATCH v5 05/10] app/test: define unit tests suites based on test macros Bruce Richardson
2023-08-16 11:02     ` Bruce Richardson
2023-08-16 11:15       ` David Marchand
2023-08-16 11:40         ` David Marchand
2023-08-16 12:33           ` Bruce Richardson
2023-08-16 13:16             ` Olivier Matz
2023-08-16 13:35               ` Morten Brørup
2023-08-16 13:44                 ` Bruce Richardson
2023-08-16 14:57     ` David Marchand
2023-08-16 15:06       ` Bruce Richardson
2023-08-15 15:10   ` [PATCH v5 06/10] app/test: add test case for scripted telemetry commands Bruce Richardson
2023-08-15 15:10   ` [PATCH v5 07/10] app/test: add debug test suite Bruce Richardson
2023-08-15 15:10   ` [PATCH v5 08/10] examples/l3fwd: make eventdev an optional dependency Bruce Richardson
2023-08-15 15:10   ` [PATCH v5 09/10] build: make most device classes optional Bruce Richardson
2023-08-15 15:12   ` [PATCH v5 10/10] build: expand list of optional libraries Bruce Richardson
2023-08-16 15:34 ` [PATCH v6 00/11] " Bruce Richardson
2023-08-16 15:34   ` [PATCH v6 01/11] mempool: fix default ops for an empty mempool Bruce Richardson
2023-08-16 16:32     ` Morten Brørup
2023-08-16 15:34   ` [PATCH v6 02/11] app/test: add new macros for various test types Bruce Richardson
2023-08-16 15:34   ` [PATCH v6 03/11] app/test: tag tests with the test type Bruce Richardson
2023-08-16 15:34   ` [PATCH v6 04/11] app/test: make telemetry data test buildable on windows Bruce Richardson
2023-08-21 15:52     ` Tyler Retzlaff
2023-08-16 15:34   ` [PATCH v6 05/11] app/test: build using per-file dependency matrix Bruce Richardson
2023-08-16 15:34   ` [PATCH v6 06/11] app/test: define unit tests suites based on test macros Bruce Richardson
2023-08-16 15:34   ` [PATCH v6 07/11] app/test: add test case for scripted telemetry commands Bruce Richardson
2023-08-16 15:34   ` [PATCH v6 08/11] app/test: add debug test suite Bruce Richardson
2023-08-16 15:34   ` [PATCH v6 09/11] examples/l3fwd: make eventdev an optional dependency Bruce Richardson
2023-08-16 15:34   ` [PATCH v6 10/11] build: make most device classes optional Bruce Richardson
2023-08-16 15:34   ` [PATCH v6 11/11] build: expand list of optional libraries Bruce Richardson
2023-08-23  7:33   ` [PATCH v6 00/11] " David Marchand

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).