Re: Unit tests not defined in any test suite

[David Marchand <david.marchand@redhat.com>]

Hello Bruce,

On Thu, Sep 14, 2023 at 6:16 PM Bruce Richardson
<bruce.richardson@intel.com> wrote:
>
> Hi all,
>
> As part of the recent rework of the build of our unit test binary
> (dpdk-test), the individual autotests were updated in the source code to
> specify the test suite they belong to. The replacement of the
> registration macro was done automatically using the existing suite lists
> in the meson.build file, but following the replacement, it appears that
> there are quite a number of test cases which are not in any test suite -
> and as such as not being run automatically for patches in the IOL test
> lab.
>
> Below is a list of these test cases, and the files each is defined in.
> We should endeavour to ensure each test case is in a suite. Currently we
> have 3 suites - fast-test, perf-test, and driver-test, but more suites
> can be added as necessary.
>
> I've attempted to automatically identify the maintainers of the various
> tests using "get-maintainers" script - hence the list of those on CC. If
> possible, can each of you identify what test cases you may be maintainer
> for, and add them to a test suite, to help improve our test coverage.
> [Apologies for any mis-identifications]
>
> Thanks,
> /Bruce
>
> alarm_autotest (test_alarm.c)
> bitratestats_autotest (test_bitratestats.c)

Those two tests above (and the one on the graph library) made me raise
an eyebrow as I remembered them being part of the fast-tests list.
I thought I had checked the tests lists before/after rework..., but
looking again I did not do this properly (the blame is on the
reviewers and commiters).

Before the rework, GHA would run 106 fast-tests ut (look for
fast-tests in follow log):
https://pipelines.actions.githubusercontent.com/serviceHosts/aad9342c-d08c-4ebc-a424-c37e5c0e7e92/_apis/pipelines/1/runs/1767/signedlogcontent/8?urlExpires=2023-09-15T07%3A37%3A45.5644150Z&urlSigningMethod=HMACV1&urlSignature=0g%2BN%2B0Q64RQolmslxH5N%2B91J7kvDAcVAanOmYnvO5cE%3D
After the rework, only 92 remain:
https://pipelines.actions.githubusercontent.com/serviceHosts/aad9342c-d08c-4ebc-a424-c37e5c0e7e92/_apis/pipelines/1/runs/1771/signedlogcontent/8?urlExpires=2023-09-15T07%3A38%3A01.6216684Z&urlSigningMethod=HMACV1&urlSignature=uVfBMml1XXXr1CLVggaPDMLN8US6n9TcudVcyqFgsd0%3D

I have a fix ready that I will send shortly.
And the list of non tagged tests must be reevaluated.



> cfgfile_autotest (test_cfgfile.c)
> cksum_perf_autotest (test_cksum_perf.c)
> compressdev_autotest (test_compressdev.c)
> cryptodev_scheduler_autotest (test_cryptodev.c)
> cryptodev_dpaa2_sec_raw_api_autotest (test_cryptodev.c)
> cryptodev_dpaa_sec_raw_api_autotest (test_cryptodev.c)
> cryptodev_mlx5_autotest (test_cryptodev.c)
> cryptodev_ccp_autotest (test_cryptodev.c)
> cryptodev_virtio_autotest (test_cryptodev.c)
> cryptodev_octeontx_autotest (test_cryptodev.c)
> cryptodev_caam_jr_autotest (test_cryptodev.c)
> cryptodev_nitrox_autotest (test_cryptodev.c)
> cryptodev_bcmfs_autotest (test_cryptodev.c)
> cryptodev_openssl_asym_autotest (test_cryptodev_asym.c)
> cryptodev_octeontx_asym_autotest (test_cryptodev_asym.c)
> cryptodev_cn9k_asym_autotest (test_cryptodev_asym.c)
> cryptodev_cn10k_asym_autotest (test_cryptodev_asym.c)
> cryptodev_crosscheck (test_cryptodev_crosscheck.c)
> event_crypto_adapter_autotest (test_event_crypto_adapter.c)
> event_eth_rx_adapter_autotest (test_event_eth_rx_adapter.c)
> event_eth_rx_intr_adapter_autotest (test_event_eth_rx_adapter.c)
> event_eth_tx_adapter_autotest (test_event_eth_tx_adapter.c)
> event_timer_adapter_test (test_event_timer_adapter.c)
> eventdev_selftest_sw (test_eventdev.c)
> eventdev_selftest_octeontx (test_eventdev.c)
> eventdev_selftest_dpaa2 (test_eventdev.c)
> eventdev_selftest_dlb2 (test_eventdev.c)
> eventdev_selftest_cn9k (test_eventdev.c)
> eventdev_selftest_cn10k (test_eventdev.c)
> external_mem_autotest (test_external_mem.c)
> graph_autotest (test_graph.c)
> node_list_dump (test_graph.c)
> graph_perf_autotest (test_graph_perf.c)
> ipsec_sad_autotest (test_ipsec_sad.c)
> latencystats_autotest (test_latencystats.c)
> link_bonding_autotest (test_link_bonding.c)
> link_bonding_mode4_autotest (test_link_bonding_mode4.c)
> link_bonding_rssconf_autotest (test_link_bonding_rssconf.c)
> metrics_autotest (test_metrics.c)
> pcapng_autotest (test_pcapng.c)
> pdcp_autotest (test_pdcp.c)
> pdump_autotest (test_pdump.c)
> ring_pmd_autotest (test_pmd_ring.c)
> ring_pmd_perf_autotest (test_pmd_ring_perf.c)
> power_caps_autotest (test_power_cpufreq.c)
> rawdev_autotest (test_rawdev.c)
> red_autotest (test_red.c)
> resource_autotest (test_resource.c)
> ring_stress_autotest (test_ring_stress.c)
> inline_macsec_autotest (test_security_inline_macsec.c)
> inline_ipsec_autotest (test_security_inline_proto.c)
> inline_ipsec_sg_autotest (test_security_inline_proto.c)
> event_inline_ipsec_autotest (test_security_inline_proto.c)
> table_autotest (test_table.c)
> timer_secondary_autotest (test_timer_secondary.c)
> vdev_autotest (test_vdev.c)
>


-- 
David Marchand