[dpdk-dev] [PATCH 0/2] next-build: add test app to build

[dpdk-dev] [PATCH 0/2] next-build: add test app to build

[Harry van Haaren]

This patchset adds the test/test/test app to the Meson build.
In doing so, the test app is improved to allow running a unit
test by setting an environment variable. This allows the meson
test infrastructure to integrate with the DPDK tests. Some nice
improvements from the above integration, including debug aids...

Run all autotests:
$ meson test

Run a specific test:
$ meson test ring_perf_autotest

Run a specific test multiple times, eg brute-forcing race conditions:
$ meson test eventdev_sw_autotest --repeat=3

Run a specific test multiple times in gdb, eg to drop to GDB if race found:
$ meson test eventdev_sw_autotest --repeat=3 --gdb


Meson also provides various options to "wrap" the test binary,
which can be used for running in eg: Valgrind or other tools. For
more information about Meson and its testing capabilities, see here:
http://mesonbuild.com/Unit-tests.html

Cheers, -Harry


Harry van Haaren (2):
  test: use env variable to run test if set
  meson: add tests app to build

 meson.build           |   1 +
 meson_options.txt     |   2 +
 test/meson.build      |  32 +++++++
 test/test/meson.build | 248 ++++++++++++++++++++++++++++++++++++++++++++++++++
 test/test/test.c      |  25 ++++-
 5 files changed, 307 insertions(+), 1 deletion(-)
 create mode 100644 test/meson.build
 create mode 100644 test/test/meson.build

-- 
2.7.4

[dpdk-dev] [PATCH 1/2] test: use env variable to run test if set

[Harry van Haaren]

This commit paves the way for the meson tests in the next
patch. With this patch the test binary checks the DPDK_TEST
environment variable and if set, the contents of the var
are inserted on the test app command line, and run.

This allows testing of various different unit tests without
manual interaction with the RTE>> test prompt, instead automating
it using the DPDK_TEST environment variable.

If the DPDK_TEST env variable is not set, or has zero lenght,
the test app behaves as normal.

Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
---
 test/test/test.c | 26 +++++++++++++++++++++++++-
 1 file changed, 25 insertions(+), 1 deletion(-)

diff --git a/test/test/test.c b/test/test/test.c
index 0e6ff7c..fb4d475 100644
--- a/test/test/test.c
+++ b/test/test/test.c
@@ -102,6 +102,8 @@ do_recursive_call(void)
 	return -1;
 }
 
+static int last_test_result;
+
 int
 main(int argc, char **argv)
 {
@@ -140,7 +142,27 @@ main(int argc, char **argv)
 	if (cl == NULL) {
 		return -1;
 	}
-	cmdline_interact(cl);
+
+	char *dpdk_test = getenv("DPDK_TEST");
+	if (dpdk_test && strlen(dpdk_test)) {
+		char buf[1024];
+		snprintf(buf, sizeof(buf), "%s\n", dpdk_test);
+		if (cmdline_in(cl, buf, strlen(buf)) < 0) {
+			printf("error on cmdline input\n");
+			return -1;
+		}
+
+		/* check the last unit test suite return, and error out if
+		 * it failed - this causes Meson to pick up the failure.
+		 */
+		if (last_test_result) {
+			cmdline_stdin_exit(cl);
+			exit(-1);
+		}
+
+	} else {
+		cmdline_interact(cl);
+	}
 	cmdline_stdin_exit(cl);
 #endif
 
@@ -231,6 +253,8 @@ unit_test_suite_runner(struct unit_test_suite *suite)
 	printf(" + Tests Failed :      %2d\n", failed);
 	printf(" + ------------------------------------------------------- +\n");
 
+	last_test_result = failed;
+
 	if (failed)
 		return -1;
 
-- 
2.7.4

[dpdk-dev] [PATCH 2/2] meson: add tests app to build

[Harry van Haaren]

This patch enables the test/test app to be built. It also adds
the test binary to be a meson-test, which allows the meson test
infrastructure to be used to run tests.

Tests are listed using the same test binary, however each test
sets a different DPDK_TEST environment variable. The string contents
of this DPDK_TEST env var is entered in the command line interface.
As such, the familiar test names such as "ring_perf_autotest" etc
are valid tests to run using this meson test infrastructure.

Note that the tests are run serially, given that we cannot run
multiple primary processes at a time. As each test must initialize
EAL this takes some time depending on the number of hugepages.
In future, we could improve this to run multiple tests from one
EAL init, but it is out of scope for this patchset.

Finally, an option to build the tests is added to the meson build
options. When disabled, the unit test code in test/test is not
compiled. The default is set to 'true'. To disable, run:

$ meson configure -Dtests=false

Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
---
 meson.build           |   1 +
 meson_options.txt     |   2 +
 test/meson.build      |  32 +++++++
 test/test/meson.build | 248 ++++++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 283 insertions(+)
 create mode 100644 test/meson.build
 create mode 100644 test/test/meson.build

diff --git a/meson.build b/meson.build
index 04eea72..9132594 100644
--- a/meson.build
+++ b/meson.build
@@ -60,6 +60,7 @@ subdir('drivers')
 # build binaries and installable tools
 subdir('usertools')
 subdir('app')
+subdir('test')
 
 # build any examples explicitly requested - useful for developers
 if get_option('examples') != ''
diff --git a/meson_options.txt b/meson_options.txt
index f2558fe..1a674aa 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -18,3 +18,5 @@ option('per_library_versions', type: 'boolean', value: true,
 	description: 'true: each lib gets its own version number, false: DPDK version used for each lib')
 option('use_hpet', type: 'boolean', value: false,
 	description: 'use HPET timer in EAL')
+option('tests', type: 'boolean', value: true,
+	description: 'build unit tests')
diff --git a/test/meson.build b/test/meson.build
new file mode 100644
index 0000000..4250eb7
--- /dev/null
+++ b/test/meson.build
@@ -0,0 +1,32 @@
+#   BSD LICENSE
+#
+#   Copyright(c) 2017 Intel Corporation. All rights reserved.
+#   All rights reserved.
+#
+#   Redistribution and use in source and binary forms, with or without
+#   modification, are permitted provided that the following conditions
+#   are met:
+#
+#     * Redistributions of source code must retain the above copyright
+#       notice, this list of conditions and the following disclaimer.
+#     * Redistributions in binary form must reproduce the above copyright
+#       notice, this list of conditions and the following disclaimer in
+#       the documentation and/or other materials provided with the
+#       distribution.
+#     * Neither the name of Intel Corporation nor the names of its
+#       contributors may be used to endorse or promote products derived
+#       from this software without specific prior written permission.
+#
+#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+subdir('test')
diff --git a/test/test/meson.build b/test/test/meson.build
new file mode 100644
index 0000000..1a4f894
--- /dev/null
+++ b/test/test/meson.build
@@ -0,0 +1,248 @@
+#   BSD LICENSE
+#
+#   Copyright(c) 2017 Intel Corporation. All rights reserved.
+#   All rights reserved.
+#
+#   Redistribution and use in source and binary forms, with or without
+#   modification, are permitted provided that the following conditions
+#   are met:
+#
+#     * Redistributions of source code must retain the above copyright
+#       notice, this list of conditions and the following disclaimer.
+#     * Redistributions in binary form must reproduce the above copyright
+#       notice, this list of conditions and the following disclaimer in
+#       the documentation and/or other materials provided with the
+#       distribution.
+#     * Neither the name of Intel Corporation nor the names of its
+#       contributors may be used to endorse or promote products derived
+#       from this software without specific prior written permission.
+#
+#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+test_sources = files('commands.c',
+	'packet_burst_generator.c',
+	'test.c',
+	'test_acl.c',
+	'test_alarm.c',
+	'test_atomic.c',
+	'test_byteorder.c',
+	'test_cmdline.c',
+	'test_cmdline_cirbuf.c',
+	'test_cmdline_etheraddr.c',
+	'test_cmdline_ipaddr.c',
+	'test_cmdline_lib.c',
+	'test_cmdline_num.c',
+	'test_cmdline_portlist.c',
+	'test_cmdline_string.c',
+	'test_common.c',
+	'test_cpuflags.c',
+	'test_crc.c',
+	'test_cycles.c',
+	'test_debug.c',
+	'test_devargs.c',
+	'test_distributor.c',
+	'test_distributor_perf.c',
+	'test_eal_flags.c',
+	'test_eal_fs.c',
+	'test_efd.c',
+	'test_efd_perf.c',
+	'test_errno.c',
+	'test_event_ring.c',
+	'test_eventdev.c',
+	'test_eventdev_octeontx.c',
+	'test_eventdev_sw.c',
+	'test_func_reentrancy.c',
+	'test_hash.c',
+	'test_hash_functions.c',
+	'test_hash_multiwriter.c',
+	'test_hash_perf.c',
+	'test_hash_scaling.c',
+	'test_interrupts.c',
+	'test_kni.c',
+	'test_kvargs.c',
+	'test_logs.c',
+	'test_lpm.c',
+	'test_lpm6.c',
+	'test_lpm6_perf.c',
+	'test_lpm_perf.c',
+	'test_malloc.c',
+	'test_mbuf.c',
+	'test_memcpy.c',
+	'test_memcpy_perf.c',
+	'test_memory.c',
+	'test_mempool.c',
+	'test_mempool_perf.c',
+	'test_memzone.c',
+	'test_meter.c',
+	'test_mp_secondary.c',
+	'test_per_lcore.c',
+	'test_pmd_perf.c',
+	'test_power.c',
+	'test_power_acpi_cpufreq.c',
+	'test_power_kvm_vm.c',
+	'test_prefetch.c',
+	'test_red.c',
+	'test_reorder.c',
+	'test_ring.c',
+	'test_ring_perf.c',
+	'test_rwlock.c',
+	'test_sched.c',
+	'test_service_cores.c',
+	'test_spinlock.c',
+	'test_string_fns.c',
+	'test_table.c',
+	'test_table_acl.c',
+	'test_table_combined.c',
+	'test_table_pipeline.c',
+	'test_table_ports.c',
+	'test_table_tables.c',
+	'test_tailq.c',
+	'test_thash.c',
+	'test_timer.c',
+	'test_timer_perf.c',
+	'test_timer_racecond.c',
+	'test_version.c',
+	'virtual_pmd.c'
+)
+
+test_deps = ['acl',
+	'cfgfile',
+	'cmdline',
+	'cryptodev',
+	'distributor',
+	'efd',
+	'ethdev',
+	'eventdev',
+	'hash',
+	'lpm',
+	'pipeline',
+	'port',
+	'power',
+	'reorder',
+	'ring',
+	'timer'
+]
+
+test_names = [
+	'acl_autotest',
+	'alarm_autotest',
+	'atomic_autotest',
+	'byteorder_autotest',
+	'cmdline_autotest',
+	'common_autotest',
+	'cpuflags_autotest',
+	'crc_autotest',
+	'cycles_autotest',
+	'debug_autotest',
+	'devargs_autotest',
+	'distributor_autotest',
+	'distributor_perf_autotest',
+	'eal_flags_autotest',
+	'eal_fs_autotest',
+	'efd_autotest',
+	'efd_perf_autotest',
+	'errno_autotest',
+	'event_ring_autotest',
+	'eventdev_common_autotest',
+	'eventdev_octeontx_autotest',
+	'eventdev_sw_autotest',
+	'func_reentrancy_autotest',
+	'has_scaling_autotest',
+	'hash_autotest',
+	'hash_functions_autotest',
+	'hash_multiwriter_autotest',
+	'hash_perf_autotest',
+	'interrupt_autotest',
+	'kni_autotest',
+	'kvargs_autotest',
+	'logs_autotest',
+	'lpm6_autotest',
+	'lpm6_perf_autotest',
+	'lpm_autotest',
+	'lpm_perf_autotest',
+	'malloc_autotest',
+	'mbuf_autotest',
+	'memcpy_autotest',
+	'memcpy_perf_autotest',
+	'memory_autotest',
+	'mempool_autotest',
+	'mempool_perf_autotest',
+	'memzone_autotest',
+	'meter_autotest',
+	'multiprocess_autotest',
+	'per_lcore_autotest',
+	'pmd_perf_autotest',
+	'power_acpi_cpufreq_autotest',
+	'power_autotest',
+	'power_kvm_vm_autotest',
+	'prefetch_autotest',
+	'red_all',
+	'red_autotest',
+	'red_perf',
+	'reorder_autotest',
+	'ring_autotest',
+	'ring_perf_autotest',
+	'rwlock_autotest',
+	'sched_autotest',
+	'service_autotest',
+	'spinlock_autotest',
+	'table_autotest',
+	'tailq_autotest',
+	'thash_autotest',
+	'timer_autotest',
+	'timer_perf__autotest',
+	'timer_racecond_autotest',
+	'user_delay_us',
+	'version_autotest',
+]
+
+if dpdk_conf.has('RTE_LIBRTE_PDUMP')
+	test_deps += 'pdump'
+endif
+if dpdk_conf.has('RTE_LIBRTE_I40E_PMD')
+	test_deps += 'pmd_i40e'
+endif
+if dpdk_conf.has('RTE_LIBRTE_IXGBE_PMD')
+	test_deps += 'pmd_ixgbe'
+endif
+
+test_dep_objs = []
+foreach d:test_deps
+	def_lib = get_option('default_library')
+	test_dep_objs += get_variable(def_lib + '_rte_' + d)
+endforeach
+
+link_libs = []
+if get_option('default_library') == 'static'
+	link_libs = dpdk_drivers
+endif
+
+if get_option('tests')
+	dpdk_test = executable('dpdk-test',
+		test_sources,
+		link_whole: link_libs,
+		dependencies: test_dep_objs,
+		install_rpath: driver_install_path,
+		install: false)
+
+	# some perf tests (eg: memcpy perf autotest)take very long
+	# to complete, so timeout to 10 minutes
+	timeout_seconds = 600
+
+	foreach t:test_names
+		test(t, dpdk_test,
+			env : ['DPDK_TEST='+t],
+			timeout : timeout_seconds,
+			is_parallel : false)
+	endforeach
+endif
-- 
2.7.4

Re: [dpdk-dev] [PATCH 1/2] test: use env variable to run test if set

[Bruce Richardson]

On Mon, Dec 18, 2017 at 11:53:57AM +0000, Harry van Haaren wrote:
> This commit paves the way for the meson tests in the next
> patch. With this patch the test binary checks the DPDK_TEST
> environment variable and if set, the contents of the var
> are inserted on the test app command line, and run.
> 
> This allows testing of various different unit tests without
> manual interaction with the RTE>> test prompt, instead automating
> it using the DPDK_TEST environment variable.
> 
> If the DPDK_TEST env variable is not set, or has zero lenght,
> the test app behaves as normal.
> 
> Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
> ---

This is very useful, even without the meson test part added in the next
patch.

Probably a matter of personal taste, but I think I would prefer to see
the tests to be run specified as args on the cmdline rather than passed
through the environment. I'd tend to view environment variables as
things to use for long-lasting values, rather than parameters you might
want to change between each run. Using cmdline args would also save you
having to split commands, since you'd get the list pre-split in
argv.

/Bruce

Re: [dpdk-dev] [PATCH 2/2] meson: add tests app to build

[Bruce Richardson]

On Mon, Dec 18, 2017 at 11:53:58AM +0000, Harry van Haaren wrote:
> This patch enables the test/test app to be built. It also adds
> the test binary to be a meson-test, which allows the meson test
> infrastructure to be used to run tests.
> 
> Tests are listed using the same test binary, however each test
> sets a different DPDK_TEST environment variable. The string contents
> of this DPDK_TEST env var is entered in the command line interface.
> As such, the familiar test names such as "ring_perf_autotest" etc
> are valid tests to run using this meson test infrastructure.
> 
> Note that the tests are run serially, given that we cannot run
> multiple primary processes at a time. As each test must initialize
> EAL this takes some time depending on the number of hugepages.
> In future, we could improve this to run multiple tests from one
> EAL init, but it is out of scope for this patchset.
> 
> Finally, an option to build the tests is added to the meson build
> options. When disabled, the unit test code in test/test is not
> compiled. The default is set to 'true'. To disable, run:
> 
> $ meson configure -Dtests=false
> 
> Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
> ---

<snip>
> +
> +test_names = [
> +	'acl_autotest',
> +	'alarm_autotest',
> +	'atomic_autotest',
> +	'byteorder_autotest',
> +	'cmdline_autotest',
> +	'common_autotest',
> +	'cpuflags_autotest',
> +	'crc_autotest',
> +	'cycles_autotest',
> +	'debug_autotest',
> +	'devargs_autotest',
> +	'distributor_autotest',
> +	'distributor_perf_autotest',
> +	'eal_flags_autotest',
> +	'eal_fs_autotest',
> +	'efd_autotest',
> +	'efd_perf_autotest',
> +	'errno_autotest',
> +	'event_ring_autotest',
> +	'eventdev_common_autotest',
> +	'eventdev_octeontx_autotest',
> +	'eventdev_sw_autotest',
> +	'func_reentrancy_autotest',
> +	'has_scaling_autotest',
> +	'hash_autotest',
> +	'hash_functions_autotest',
> +	'hash_multiwriter_autotest',
> +	'hash_perf_autotest',
> +	'interrupt_autotest',
> +	'kni_autotest',
> +	'kvargs_autotest',
> +	'logs_autotest',
> +	'lpm6_autotest',
> +	'lpm6_perf_autotest',
> +	'lpm_autotest',
> +	'lpm_perf_autotest',
> +	'malloc_autotest',
> +	'mbuf_autotest',
> +	'memcpy_autotest',
> +	'memcpy_perf_autotest',
> +	'memory_autotest',
> +	'mempool_autotest',
> +	'mempool_perf_autotest',
> +	'memzone_autotest',
> +	'meter_autotest',
> +	'multiprocess_autotest',
> +	'per_lcore_autotest',
> +	'pmd_perf_autotest',
> +	'power_acpi_cpufreq_autotest',
> +	'power_autotest',
> +	'power_kvm_vm_autotest',
> +	'prefetch_autotest',
> +	'red_all',
> +	'red_autotest',
> +	'red_perf',
> +	'reorder_autotest',
> +	'ring_autotest',
> +	'ring_perf_autotest',
> +	'rwlock_autotest',
> +	'sched_autotest',
> +	'service_autotest',
> +	'spinlock_autotest',
> +	'table_autotest',
> +	'tailq_autotest',
> +	'thash_autotest',
> +	'timer_autotest',
> +	'timer_perf__autotest',
> +	'timer_racecond_autotest',
> +	'user_delay_us',
> +	'version_autotest',
> +]

I think we should find a way to remove this long list of test names in
the build file. The list of available tests is built up dynamically
inside the test binary, so I'm wondering if we can add to the binary an
"all" option to run each autotest straight after each other?

> +
> +if dpdk_conf.has('RTE_LIBRTE_PDUMP')
> +	test_deps += 'pdump'
> +endif
> +if dpdk_conf.has('RTE_LIBRTE_I40E_PMD')
> +	test_deps += 'pmd_i40e'
> +endif
> +if dpdk_conf.has('RTE_LIBRTE_IXGBE_PMD')
> +	test_deps += 'pmd_ixgbe'
> +endif
> +

This should be moved up beside the other dependencies

> +test_dep_objs = []
> +foreach d:test_deps
> +	def_lib = get_option('default_library')
> +	test_dep_objs += get_variable(def_lib + '_rte_' + d)
> +endforeach
> +
> +link_libs = []
> +if get_option('default_library') == 'static'
> +	link_libs = dpdk_drivers
> +endif
> +

A note for the future: since much of this looks similar to that in
testpmd we should look to see if we can find a way to easily consolidate
these build commands inside a foreach loop like is done for the libs,
drivers and examples.

> +if get_option('tests')
> +	dpdk_test = executable('dpdk-test',
> +		test_sources,
> +		link_whole: link_libs,
> +		dependencies: test_dep_objs,
> +		install_rpath: driver_install_path,
> +		install: false)
> +
> +	# some perf tests (eg: memcpy perf autotest)take very long
> +	# to complete, so timeout to 10 minutes
> +	timeout_seconds = 600
> +
> +	foreach t:test_names
> +		test(t, dpdk_test,
> +			env : ['DPDK_TEST='+t],
> +			timeout : timeout_seconds,
> +			is_parallel : false)
> +	endforeach
> +endif
> -- 
> 2.7.4
> 

Re: [dpdk-dev] [PATCH 0/2] next-build: add test app to build

[Bruce Richardson]

On Mon, Dec 18, 2017 at 11:53:56AM +0000, Harry van Haaren wrote:
> This patchset adds the test/test/test app to the Meson build.
> In doing so, the test app is improved to allow running a unit
> test by setting an environment variable. This allows the meson
> test infrastructure to integrate with the DPDK tests. Some nice
> improvements from the above integration, including debug aids...
> 
> Run all autotests:
> $ meson test
> 

meson or ninja?

> Run a specific test:
> $ meson test ring_perf_autotest
> 
> Run a specific test multiple times, eg brute-forcing race conditions:
> $ meson test eventdev_sw_autotest --repeat=3
> 
> Run a specific test multiple times in gdb, eg to drop to GDB if race found:
> $ meson test eventdev_sw_autotest --repeat=3 --gdb
> 

The "repeat" and "gdb" arguments come built-in in meson, right, not from
this patchset?

> 
> Meson also provides various options to "wrap" the test binary,
> which can be used for running in eg: Valgrind or other tools. For
> more information about Meson and its testing capabilities, see here:
> http://mesonbuild.com/Unit-tests.html
> 
> Cheers, -Harry
> 
> 
> Harry van Haaren (2):
>   test: use env variable to run test if set
>   meson: add tests app to build
> 
>  meson.build           |   1 +
>  meson_options.txt     |   2 +
>  test/meson.build      |  32 +++++++
>  test/test/meson.build | 248 ++++++++++++++++++++++++++++++++++++++++++++++++++
>  test/test/test.c      |  25 ++++-
>  5 files changed, 307 insertions(+), 1 deletion(-)
>  create mode 100644 test/meson.build
>  create mode 100644 test/test/meson.build
> 
> -- 
> 2.7.4
> 

Re: [dpdk-dev] [PATCH 1/2] test: use env variable to run test if set

[Jerin Jacob]

-----Original Message-----
> Date: Mon, 18 Dec 2017 11:53:57 +0000
> From: Harry van Haaren <harry.van.haaren@intel.com>
> To: dev@dpdk.org
> CC: bruce.richardson@intel.com, Harry van Haaren
>  <harry.van.haaren@intel.com>
> Subject: [dpdk-dev] [PATCH 1/2] test: use env variable to run test if set
> X-Mailer: git-send-email 2.7.4
> 
> This commit paves the way for the meson tests in the next
> patch. With this patch the test binary checks the DPDK_TEST
> environment variable and if set, the contents of the var
> are inserted on the test app command line, and run.
> 
> This allows testing of various different unit tests without
> manual interaction with the RTE>> test prompt, instead automating
> it using the DPDK_TEST environment variable.

Another alternative is to pipe the command.
example:
echo "eventdev_common_autotest" | sudo ./build/app/test 

Re: [dpdk-dev] [PATCH 0/2] next-build: add test app to build

[Van Haaren, Harry]

> From: Richardson, Bruce
> Sent: Monday, December 18, 2017 1:57 PM
> To: Van Haaren, Harry <harry.van.haaren@intel.com>
> Cc: dev@dpdk.org
> Subject: Re: [PATCH 0/2] next-build: add test app to build
> 
> On Mon, Dec 18, 2017 at 11:53:56AM +0000, Harry van Haaren wrote:
> > This patchset adds the test/test/test app to the Meson build.
> > In doing so, the test app is improved to allow running a unit
> > test by setting an environment variable. This allows the meson
> > test infrastructure to integrate with the DPDK tests. Some nice
> > improvements from the above integration, including debug aids...
> >
> > Run all autotests:
> > $ meson test
> >
> 
> meson or ninja?

meson. If I'm understanding correctly the "meson test" command invokes
the ninja test runner, however some of the argument mushing (eg: --gdb)
is performed by meson before launching ninja.


> > Run a specific test:
> > $ meson test ring_perf_autotest
> >
> > Run a specific test multiple times, eg brute-forcing race conditions:
> > $ meson test eventdev_sw_autotest --repeat=3
> >
> > Run a specific test multiple times in gdb, eg to drop to GDB if race
> found:
> > $ meson test eventdev_sw_autotest --repeat=3 --gdb
> >
> 
> The "repeat" and "gdb" arguments come built-in in meson, right, not from
> this patchset?

Yes correct, Meson provides this functionality.

<snip>

Re: [dpdk-dev] [PATCH 1/2] test: use env variable to run test if set

[Van Haaren, Harry]

> From: Jerin Jacob [mailto:jerin.jacob@caviumnetworks.com]
> Sent: Monday, December 18, 2017 2:59 PM
> To: Van Haaren, Harry <harry.van.haaren@intel.com>
> Cc: dev@dpdk.org; Richardson, Bruce <bruce.richardson@intel.com>
> Subject: Re: [dpdk-dev] [PATCH 1/2] test: use env variable to run test if
> set
> 
> -----Original Message-----
> > Date: Mon, 18 Dec 2017 11:53:57 +0000
> > From: Harry van Haaren <harry.van.haaren@intel.com>
> > To: dev@dpdk.org
> > CC: bruce.richardson@intel.com, Harry van Haaren
> >  <harry.van.haaren@intel.com>
> > Subject: [dpdk-dev] [PATCH 1/2] test: use env variable to run test if set
> > X-Mailer: git-send-email 2.7.4
> >
> > This commit paves the way for the meson tests in the next
> > patch. With this patch the test binary checks the DPDK_TEST
> > environment variable and if set, the contents of the var
> > are inserted on the test app command line, and run.
> >
> > This allows testing of various different unit tests without
> > manual interaction with the RTE>> test prompt, instead automating
> > it using the DPDK_TEST environment variable.
> 
> Another alternative is to pipe the command.
> example:
> echo "eventdev_common_autotest" | sudo ./build/app/test


With the current implementation, meson handles which tests to run, and the command line. This gives us a clean interface from which to run tests. Note that the following command will run the tests requested:

$ meson test ring_autotest ring_perf_autotest acl_autotest

Meson itself supports two methods of launching tests from the same binary: argv and env variables. In this implementation, the DPDK_TEST env is set by the test runner - and the user doesn't have to use it manually at all, and it is not exported in the shell after the tests have run.

In short - I don't see added value in reworking this to argc argv, or in using terminal tricks like echo "test" | sudo ./test.

Actually, the current method has an easter egg included:
If a developer is focused on a single test-case (TDD anyone? :), then they could use the DPDK_TEST env var as a feature, $ export DPDK_TEST=ring_autotest  and run that test automatically when the binary is launched.

Re: [dpdk-dev] [PATCH 2/2] meson: add tests app to build

[Van Haaren, Harry]

> From: Richardson, Bruce
> Sent: Monday, December 18, 2017 1:55 PM
> To: Van Haaren, Harry <harry.van.haaren@intel.com>
> Cc: dev@dpdk.org
> Subject: Re: [PATCH 2/2] meson: add tests app to build
> 
> On Mon, Dec 18, 2017 at 11:53:58AM +0000, Harry van Haaren wrote:
> > This patch enables the test/test app to be built. It also adds
> > the test binary to be a meson-test, which allows the meson test
> > infrastructure to be used to run tests.
> >
> > Tests are listed using the same test binary, however each test
> > sets a different DPDK_TEST environment variable. The string contents
> > of this DPDK_TEST env var is entered in the command line interface.
> > As such, the familiar test names such as "ring_perf_autotest" etc
> > are valid tests to run using this meson test infrastructure.
> >
> > Note that the tests are run serially, given that we cannot run
> > multiple primary processes at a time. As each test must initialize
> > EAL this takes some time depending on the number of hugepages.
> > In future, we could improve this to run multiple tests from one
> > EAL init, but it is out of scope for this patchset.
> >
> > Finally, an option to build the tests is added to the meson build
> > options. When disabled, the unit test code in test/test is not
> > compiled. The default is set to 'true'. To disable, run:
> >
> > $ meson configure -Dtests=false
> >
> > Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
> > ---
> 
> <snip>
> > +
> > +test_names = [
> > +	'acl_autotest',
> > +	'alarm_autotest',
<snip>
> > +	'user_delay_us',
> > +	'version_autotest',
> > +]
> 
> I think we should find a way to remove this long list of test names in
> the build file. The list of available tests is built up dynamically
> inside the test binary, so I'm wondering if we can add to the binary an
> "all" option to run each autotest straight after each other?


Agree - it would be nice if we don't have to specify tests here. We could add a flag the test binary, to output the tests that it contains? Then its about having Meson read this file before running tests, to "gather" the tests available..

There's options, but they will probably all be more complex than just having a list of tests.

Re Run all tests: Currently   $ meson test     will run all unit tests.


> > +
> > +if dpdk_conf.has('RTE_LIBRTE_PDUMP')
> > +	test_deps += 'pdump'
> > +endif
> > +if dpdk_conf.has('RTE_LIBRTE_I40E_PMD')
> > +	test_deps += 'pmd_i40e'
> > +endif
> > +if dpdk_conf.has('RTE_LIBRTE_IXGBE_PMD')
> > +	test_deps += 'pmd_ixgbe'
> > +endif
> > +
> 
> This should be moved up beside the other dependencies
> 
> > +test_dep_objs = []
> > +foreach d:test_deps
> > +	def_lib = get_option('default_library')
> > +	test_dep_objs += get_variable(def_lib + '_rte_' + d)
> > +endforeach
> > +
> > +link_libs = []
> > +if get_option('default_library') == 'static'
> > +	link_libs = dpdk_drivers
> > +endif
> > +
> 
> A note for the future: since much of this looks similar to that in
> testpmd we should look to see if we can find a way to easily consolidate
> these build commands inside a foreach loop like is done for the libs,
> drivers and examples.

+1, good idea for a cleanup later.

Meson >= 0.44 has a "disabler" concept[1], which is a clean way to disable components in the build if dependencies aren't found. Eg: specific tests or sample apps, can be disabled from the build without the if() logic required as below.

[1] http://mesonbuild.com/Disabler.html


> > +if get_option('tests')
> > +	dpdk_test = executable('dpdk-test',
> > +		test_sources,
> > +		link_whole: link_libs,
> > +		dependencies: test_dep_objs,
> > +		install_rpath: driver_install_path,
> > +		install: false)
> > +
> > +	# some perf tests (eg: memcpy perf autotest)take very long
> > +	# to complete, so timeout to 10 minutes
> > +	timeout_seconds = 600
> > +
> > +	foreach t:test_names
> > +		test(t, dpdk_test,
> > +			env : ['DPDK_TEST='+t],
> > +			timeout : timeout_seconds,
> > +			is_parallel : false)
> > +	endforeach
> > +endif
> > --
> > 2.7.4
> >

Re: [dpdk-dev] [PATCH 1/2] test: use env variable to run test if set

[Jerin Jacob]

-----Original Message-----
> Date: Mon, 18 Dec 2017 15:24:22 +0000
> From: "Van Haaren, Harry" <harry.van.haaren@intel.com>
> To: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> CC: "dev@dpdk.org" <dev@dpdk.org>, "Richardson, Bruce"
>  <bruce.richardson@intel.com>
> Subject: RE: [dpdk-dev] [PATCH 1/2] test: use env variable to run test if
>  set
> 
> > From: Jerin Jacob [mailto:jerin.jacob@caviumnetworks.com]
> > Sent: Monday, December 18, 2017 2:59 PM
> > To: Van Haaren, Harry <harry.van.haaren@intel.com>
> > Cc: dev@dpdk.org; Richardson, Bruce <bruce.richardson@intel.com>
> > Subject: Re: [dpdk-dev] [PATCH 1/2] test: use env variable to run test if
> > set
> > 
> > -----Original Message-----
> > > Date: Mon, 18 Dec 2017 11:53:57 +0000
> > > From: Harry van Haaren <harry.van.haaren@intel.com>
> > > To: dev@dpdk.org
> > > CC: bruce.richardson@intel.com, Harry van Haaren
> > >  <harry.van.haaren@intel.com>
> > > Subject: [dpdk-dev] [PATCH 1/2] test: use env variable to run test if set
> > > X-Mailer: git-send-email 2.7.4
> > >
> > > This commit paves the way for the meson tests in the next
> > > patch. With this patch the test binary checks the DPDK_TEST
> > > environment variable and if set, the contents of the var
> > > are inserted on the test app command line, and run.
> > >
> > > This allows testing of various different unit tests without
> > > manual interaction with the RTE>> test prompt, instead automating
> > > it using the DPDK_TEST environment variable.
> > 
> > Another alternative is to pipe the command.
> > example:
> > echo "eventdev_common_autotest" | sudo ./build/app/test
> 
> 
> With the current implementation, meson handles which tests to run, and the command line. This gives us a clean interface from which to run tests. Note that the following command will run the tests requested:
> 
> $ meson test ring_autotest ring_perf_autotest acl_autotest
> 
> Meson itself supports two methods of launching tests from the same binary: argv and env variables. In this implementation, the DPDK_TEST env is set by the test runner - and the user doesn't have to use it manually at all, and it is not exported in the shell after the tests have run.
> 
> In short - I don't see added value in reworking this to argc argv, or in using terminal tricks like echo "test" | sudo ./test.
> 
> Actually, the current method has an easter egg included:
> If a developer is focused on a single test-case (TDD anyone? :), then they could use the DPDK_TEST env var as a feature, $ export DPDK_TEST=ring_autotest  and run that test automatically when the binary is launched.

Yup. I don't see any harm in exposing DPDK_TEST means of test selection.

Re: [dpdk-dev] [PATCH 2/2] meson: add tests app to build

[Bruce Richardson]

On Mon, Dec 18, 2017 at 11:53:58AM +0000, Harry van Haaren wrote:
> This patch enables the test/test app to be built. It also adds
> the test binary to be a meson-test, which allows the meson test
> infrastructure to be used to run tests.
> 
> Tests are listed using the same test binary, however each test
> sets a different DPDK_TEST environment variable. The string contents
> of this DPDK_TEST env var is entered in the command line interface.
> As such, the familiar test names such as "ring_perf_autotest" etc
> are valid tests to run using this meson test infrastructure.
> 
> Note that the tests are run serially, given that we cannot run
> multiple primary processes at a time. As each test must initialize
> EAL this takes some time depending on the number of hugepages.
> In future, we could improve this to run multiple tests from one
> EAL init, but it is out of scope for this patchset.
> 
> Finally, an option to build the tests is added to the meson build
> options. When disabled, the unit test code in test/test is not
> compiled. The default is set to 'true'. To disable, run:
> 
> $ meson configure -Dtests=false
> 
> Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
> ---
>  meson.build           |   1 +
>  meson_options.txt     |   2 +
>  test/meson.build      |  32 +++++++
>  test/test/meson.build | 248 ++++++++++++++++++++++++++++++++++++++++++++++++++
>  4 files changed, 283 insertions(+)
>  create mode 100644 test/meson.build
>  create mode 100644 test/test/meson.build
> 
> diff --git a/meson.build b/meson.build
> index 04eea72..9132594 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -60,6 +60,7 @@ subdir('drivers')
>  # build binaries and installable tools
>  subdir('usertools')
>  subdir('app')
> +subdir('test')
>  
>  # build any examples explicitly requested - useful for developers
>  if get_option('examples') != ''
> diff --git a/meson_options.txt b/meson_options.txt
> index f2558fe..1a674aa 100644
> --- a/meson_options.txt
> +++ b/meson_options.txt
> @@ -18,3 +18,5 @@ option('per_library_versions', type: 'boolean', value: true,
>  	description: 'true: each lib gets its own version number, false: DPDK version used for each lib')
>  option('use_hpet', type: 'boolean', value: false,
>  	description: 'use HPET timer in EAL')
> +option('tests', type: 'boolean', value: true,
> +	description: 'build unit tests')
> diff --git a/test/meson.build b/test/meson.build
> new file mode 100644
> index 0000000..4250eb7
> --- /dev/null
> +++ b/test/meson.build
> @@ -0,0 +1,32 @@
> +#   BSD LICENSE
> +#
> +#   Copyright(c) 2017 Intel Corporation. All rights reserved.
> +#   All rights reserved.
> +#
> +#   Redistribution and use in source and binary forms, with or without
> +#   modification, are permitted provided that the following conditions
> +#   are met:
> +#
> +#     * Redistributions of source code must retain the above copyright
> +#       notice, this list of conditions and the following disclaimer.
> +#     * Redistributions in binary form must reproduce the above copyright
> +#       notice, this list of conditions and the following disclaimer in
> +#       the documentation and/or other materials provided with the
> +#       distribution.
> +#     * Neither the name of Intel Corporation nor the names of its
> +#       contributors may be used to endorse or promote products derived
> +#       from this software without specific prior written permission.
> +#
> +#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> +#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> +#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
> +#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
> +#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
> +#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
> +#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
> +#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
> +#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
> +#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
> +#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
> +
> +subdir('test')

Can you do a V2 of this set with the SPDX tag in place of the license
text. Otherwise, the set looks good enough to me.

/Bruce

[dpdk-dev] [PATCH v2 0/2] next-build: add test app to build

[Harry van Haaren]

This patchset adds the test/test/test app to the Meson build.
In doing so, the test app is improved to allow running a unit
test by setting an environment variable. This allows the meson
test infrastructure to integrate with the DPDK tests. Some nice
improvements from the above integration, including debug aids...

Run all autotests:
$ meson test

Run a specific test:
$ meson test ring_perf_autotest

Run a specific test multiple times, eg brute-forcing race conditions:
$ meson test eventdev_sw_autotest --repeat=3

Run a specific test multiple times in gdb, eg to drop to GDB if race found:
$ meson test eventdev_sw_autotest --repeat=3 --gdb


Meson also provides various options to "wrap" the test binary,
which can be used for running in eg: Valgrind or other tools. For
more information about Meson and its testing capabilities, see here:
http://mesonbuild.com/Unit-tests.html

Cheers, -Harry

---

v2: updated patch 2/2 to SPDX license

Harry van Haaren (2):
  test: use env variable to run test if set
  meson: add tests app to build

 meson.build           |   1 +
 meson_options.txt     |   2 +
 test/meson.build      |   4 +
 test/test/meson.build | 220 ++++++++++++++++++++++++++++++++++++++++++++++++++
 test/test/test.c      |  26 +++++-
 5 files changed, 252 insertions(+), 1 deletion(-)
 create mode 100644 test/meson.build
 create mode 100644 test/test/meson.build

-- 
2.7.4

[dpdk-dev] [PATCH v2 1/2] test: use env variable to run test if set

[Harry van Haaren]

This commit paves the way for the meson tests in the next
patch. With this patch the test binary checks the DPDK_TEST
environment variable and if set, the contents of the var
are inserted on the test app command line, and run.

This allows testing of various different unit tests without
manual interaction with the RTE>> test prompt, instead automating
it using the DPDK_TEST environment variable.

If the DPDK_TEST env variable is not set, or has zero lenght,
the test app behaves as normal.

Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
---
 test/test/test.c | 26 +++++++++++++++++++++++++-
 1 file changed, 25 insertions(+), 1 deletion(-)

diff --git a/test/test/test.c b/test/test/test.c
index 0e6ff7c..fb4d475 100644
--- a/test/test/test.c
+++ b/test/test/test.c
@@ -102,6 +102,8 @@ do_recursive_call(void)
 	return -1;
 }
 
+static int last_test_result;
+
 int
 main(int argc, char **argv)
 {
@@ -140,7 +142,27 @@ main(int argc, char **argv)
 	if (cl == NULL) {
 		return -1;
 	}
-	cmdline_interact(cl);
+
+	char *dpdk_test = getenv("DPDK_TEST");
+	if (dpdk_test && strlen(dpdk_test)) {
+		char buf[1024];
+		snprintf(buf, sizeof(buf), "%s\n", dpdk_test);
+		if (cmdline_in(cl, buf, strlen(buf)) < 0) {
+			printf("error on cmdline input\n");
+			return -1;
+		}
+
+		/* check the last unit test suite return, and error out if
+		 * it failed - this causes Meson to pick up the failure.
+		 */
+		if (last_test_result) {
+			cmdline_stdin_exit(cl);
+			exit(-1);
+		}
+
+	} else {
+		cmdline_interact(cl);
+	}
 	cmdline_stdin_exit(cl);
 #endif
 
@@ -231,6 +253,8 @@ unit_test_suite_runner(struct unit_test_suite *suite)
 	printf(" + Tests Failed :      %2d\n", failed);
 	printf(" + ------------------------------------------------------- +\n");
 
+	last_test_result = failed;
+
 	if (failed)
 		return -1;
 
-- 
2.7.4

[dpdk-dev] [PATCH v2 2/2] meson: add tests app to build

[Harry van Haaren]

This patch enables the test/test app to be built. It also adds
the test binary to be a meson-test, which allows the meson test
infrastructure to be used to run tests.

Tests are listed using the same test binary, however each test
sets a different DPDK_TEST environment variable. The string contents
of this DPDK_TEST env var is entered in the command line interface.
As such, the familiar test names such as "ring_perf_autotest" etc
are valid tests to run using this meson test infrastructure.

Note that the tests are run serially, given that we cannot run
multiple primary processes at a time. As each test must initialize
EAL this takes some time depending on the number of hugepages.
In future, we could improve this to run multiple tests from one
EAL init, but it is out of scope for this patchset.

Finally, an option to build the tests is added to the meson build
options. When disabled, the unit test code in test/test is not
compiled. The default is set to 'true'. To disable, run:

$ meson configure -Dtests=false

Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>

---

v2:
- Updated for SPDX license headers (Bruce)
---
 meson.build           |   1 +
 meson_options.txt     |   2 +
 test/meson.build      |   4 +
 test/test/meson.build | 220 ++++++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 227 insertions(+)
 create mode 100644 test/meson.build
 create mode 100644 test/test/meson.build

diff --git a/meson.build b/meson.build
index 04eea72..9132594 100644
--- a/meson.build
+++ b/meson.build
@@ -60,6 +60,7 @@ subdir('drivers')
 # build binaries and installable tools
 subdir('usertools')
 subdir('app')
+subdir('test')
 
 # build any examples explicitly requested - useful for developers
 if get_option('examples') != ''
diff --git a/meson_options.txt b/meson_options.txt
index f2558fe..1a674aa 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -18,3 +18,5 @@ option('per_library_versions', type: 'boolean', value: true,
 	description: 'true: each lib gets its own version number, false: DPDK version used for each lib')
 option('use_hpet', type: 'boolean', value: false,
 	description: 'use HPET timer in EAL')
+option('tests', type: 'boolean', value: true,
+	description: 'build unit tests')
diff --git a/test/meson.build b/test/meson.build
new file mode 100644
index 0000000..3ad11b3
--- /dev/null
+++ b/test/meson.build
@@ -0,0 +1,4 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2017 Intel Corporation
+
+subdir('test')
diff --git a/test/test/meson.build b/test/test/meson.build
new file mode 100644
index 0000000..435bd78
--- /dev/null
+++ b/test/test/meson.build
@@ -0,0 +1,220 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2017 Intel Corporation
+
+test_sources = files('commands.c',
+	'packet_burst_generator.c',
+	'test.c',
+	'test_acl.c',
+	'test_alarm.c',
+	'test_atomic.c',
+	'test_byteorder.c',
+	'test_cmdline.c',
+	'test_cmdline_cirbuf.c',
+	'test_cmdline_etheraddr.c',
+	'test_cmdline_ipaddr.c',
+	'test_cmdline_lib.c',
+	'test_cmdline_num.c',
+	'test_cmdline_portlist.c',
+	'test_cmdline_string.c',
+	'test_common.c',
+	'test_cpuflags.c',
+	'test_crc.c',
+	'test_cycles.c',
+	'test_debug.c',
+	'test_devargs.c',
+	'test_distributor.c',
+	'test_distributor_perf.c',
+	'test_eal_flags.c',
+	'test_eal_fs.c',
+	'test_efd.c',
+	'test_efd_perf.c',
+	'test_errno.c',
+	'test_event_ring.c',
+	'test_eventdev.c',
+	'test_eventdev_octeontx.c',
+	'test_eventdev_sw.c',
+	'test_func_reentrancy.c',
+	'test_hash.c',
+	'test_hash_functions.c',
+	'test_hash_multiwriter.c',
+	'test_hash_perf.c',
+	'test_hash_scaling.c',
+	'test_interrupts.c',
+	'test_kni.c',
+	'test_kvargs.c',
+	'test_logs.c',
+	'test_lpm.c',
+	'test_lpm6.c',
+	'test_lpm6_perf.c',
+	'test_lpm_perf.c',
+	'test_malloc.c',
+	'test_mbuf.c',
+	'test_memcpy.c',
+	'test_memcpy_perf.c',
+	'test_memory.c',
+	'test_mempool.c',
+	'test_mempool_perf.c',
+	'test_memzone.c',
+	'test_meter.c',
+	'test_mp_secondary.c',
+	'test_per_lcore.c',
+	'test_pmd_perf.c',
+	'test_power.c',
+	'test_power_acpi_cpufreq.c',
+	'test_power_kvm_vm.c',
+	'test_prefetch.c',
+	'test_red.c',
+	'test_reorder.c',
+	'test_ring.c',
+	'test_ring_perf.c',
+	'test_rwlock.c',
+	'test_sched.c',
+	'test_service_cores.c',
+	'test_spinlock.c',
+	'test_string_fns.c',
+	'test_table.c',
+	'test_table_acl.c',
+	'test_table_combined.c',
+	'test_table_pipeline.c',
+	'test_table_ports.c',
+	'test_table_tables.c',
+	'test_tailq.c',
+	'test_thash.c',
+	'test_timer.c',
+	'test_timer_perf.c',
+	'test_timer_racecond.c',
+	'test_version.c',
+	'virtual_pmd.c'
+)
+
+test_deps = ['acl',
+	'cfgfile',
+	'cmdline',
+	'cryptodev',
+	'distributor',
+	'efd',
+	'ethdev',
+	'eventdev',
+	'hash',
+	'lpm',
+	'pipeline',
+	'port',
+	'power',
+	'reorder',
+	'ring',
+	'timer'
+]
+
+test_names = [
+	'acl_autotest',
+	'alarm_autotest',
+	'atomic_autotest',
+	'byteorder_autotest',
+	'cmdline_autotest',
+	'common_autotest',
+	'cpuflags_autotest',
+	'crc_autotest',
+	'cycles_autotest',
+	'debug_autotest',
+	'devargs_autotest',
+	'distributor_autotest',
+	'distributor_perf_autotest',
+	'eal_flags_autotest',
+	'eal_fs_autotest',
+	'efd_autotest',
+	'efd_perf_autotest',
+	'errno_autotest',
+	'event_ring_autotest',
+	'eventdev_common_autotest',
+	'eventdev_octeontx_autotest',
+	'eventdev_sw_autotest',
+	'func_reentrancy_autotest',
+	'has_scaling_autotest',
+	'hash_autotest',
+	'hash_functions_autotest',
+	'hash_multiwriter_autotest',
+	'hash_perf_autotest',
+	'interrupt_autotest',
+	'kni_autotest',
+	'kvargs_autotest',
+	'logs_autotest',
+	'lpm6_autotest',
+	'lpm6_perf_autotest',
+	'lpm_autotest',
+	'lpm_perf_autotest',
+	'malloc_autotest',
+	'mbuf_autotest',
+	'memcpy_autotest',
+	'memcpy_perf_autotest',
+	'memory_autotest',
+	'mempool_autotest',
+	'mempool_perf_autotest',
+	'memzone_autotest',
+	'meter_autotest',
+	'multiprocess_autotest',
+	'per_lcore_autotest',
+	'pmd_perf_autotest',
+	'power_acpi_cpufreq_autotest',
+	'power_autotest',
+	'power_kvm_vm_autotest',
+	'prefetch_autotest',
+	'red_all',
+	'red_autotest',
+	'red_perf',
+	'reorder_autotest',
+	'ring_autotest',
+	'ring_perf_autotest',
+	'rwlock_autotest',
+	'sched_autotest',
+	'service_autotest',
+	'spinlock_autotest',
+	'table_autotest',
+	'tailq_autotest',
+	'thash_autotest',
+	'timer_autotest',
+	'timer_perf__autotest',
+	'timer_racecond_autotest',
+	'user_delay_us',
+	'version_autotest',
+]
+
+if dpdk_conf.has('RTE_LIBRTE_PDUMP')
+	test_deps += 'pdump'
+endif
+if dpdk_conf.has('RTE_LIBRTE_I40E_PMD')
+	test_deps += 'pmd_i40e'
+endif
+if dpdk_conf.has('RTE_LIBRTE_IXGBE_PMD')
+	test_deps += 'pmd_ixgbe'
+endif
+
+test_dep_objs = []
+foreach d:test_deps
+	def_lib = get_option('default_library')
+	test_dep_objs += get_variable(def_lib + '_rte_' + d)
+endforeach
+
+link_libs = []
+if get_option('default_library') == 'static'
+	link_libs = dpdk_drivers
+endif
+
+if get_option('tests')
+	dpdk_test = executable('dpdk-test',
+		test_sources,
+		link_whole: link_libs,
+		dependencies: test_dep_objs,
+		install_rpath: driver_install_path,
+		install: false)
+
+	# some perf tests (eg: memcpy perf autotest)take very long
+	# to complete, so timeout to 10 minutes
+	timeout_seconds = 600
+
+	foreach t:test_names
+		test(t, dpdk_test,
+			env : ['DPDK_TEST='+t],
+			timeout : timeout_seconds,
+			is_parallel : false)
+	endforeach
+endif
-- 
2.7.4

Re: [dpdk-dev] [PATCH v2 1/2] test: use env variable to run test if set

[Bruce Richardson]

On Wed, Dec 20, 2017 at 11:16:31AM +0000, Harry van Haaren wrote:
> This commit paves the way for the meson tests in the next
> patch. With this patch the test binary checks the DPDK_TEST
> environment variable and if set, the contents of the var
> are inserted on the test app command line, and run.
> 
> This allows testing of various different unit tests without
> manual interaction with the RTE>> test prompt, instead automating
> it using the DPDK_TEST environment variable.
> 
> If the DPDK_TEST env variable is not set, or has zero lenght,
> the test app behaves as normal.
> 
> Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
> ---
While I still prefer using cmdline args to environment, this works ok.

Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Re: [dpdk-dev] [PATCH v2 2/2] meson: add tests app to build

[Laatz, Kevin]

> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Harry van Haaren
> Sent: Wednesday, December 20, 2017 11:17 AM
> To: dev@dpdk.org
> Cc: Richardson, Bruce <bruce.richardson@intel.com>; Van Haaren, Harry
> <harry.van.haaren@intel.com>
> Subject: [dpdk-dev] [PATCH v2 2/2] meson: add tests app to build

<snip>

> +test_names = [
> +	'acl_autotest',
> +	'alarm_autotest',
> +	'atomic_autotest',
> +	'byteorder_autotest',
> +	'cmdline_autotest',
> +	'common_autotest',
> +	'cpuflags_autotest',
> +	'crc_autotest',
> +	'cycles_autotest',
> +	'debug_autotest',
> +	'devargs_autotest',
> +	'distributor_autotest',
> +	'distributor_perf_autotest',
> +	'eal_flags_autotest',
> +	'eal_fs_autotest',
> +	'efd_autotest',
> +	'efd_perf_autotest',
> +	'errno_autotest',
> +	'event_ring_autotest',
> +	'eventdev_common_autotest',
> +	'eventdev_octeontx_autotest',
> +	'eventdev_sw_autotest',
> +	'func_reentrancy_autotest',
> +	'has_scaling_autotest',

Minor typo here, should be 'hash_scaling_autotest',

> +	'hash_autotest',
> +	'hash_functions_autotest',
> +	'hash_multiwriter_autotest',
> +	'hash_perf_autotest',

<snip>

Re: [dpdk-dev] [PATCH v2 2/2] meson: add tests app to build

[Bruce Richardson]

On Wed, Dec 20, 2017 at 11:57:59AM +0000, Laatz, Kevin wrote:
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Harry van Haaren
> > Sent: Wednesday, December 20, 2017 11:17 AM
> > To: dev@dpdk.org
> > Cc: Richardson, Bruce <bruce.richardson@intel.com>; Van Haaren, Harry
> > <harry.van.haaren@intel.com>
> > Subject: [dpdk-dev] [PATCH v2 2/2] meson: add tests app to build
> 
> <snip>
> 
> > +test_names = [
> > +	'acl_autotest',
> > +	'alarm_autotest',
> > +	'atomic_autotest',
> > +	'byteorder_autotest',
> > +	'cmdline_autotest',
> > +	'common_autotest',
> > +	'cpuflags_autotest',
> > +	'crc_autotest',
> > +	'cycles_autotest',
> > +	'debug_autotest',
> > +	'devargs_autotest',
> > +	'distributor_autotest',
> > +	'distributor_perf_autotest',
> > +	'eal_flags_autotest',
> > +	'eal_fs_autotest',
> > +	'efd_autotest',
> > +	'efd_perf_autotest',
> > +	'errno_autotest',
> > +	'event_ring_autotest',
> > +	'eventdev_common_autotest',
> > +	'eventdev_octeontx_autotest',
> > +	'eventdev_sw_autotest',
> > +	'func_reentrancy_autotest',
> > +	'has_scaling_autotest',
> 
> Minor typo here, should be 'hash_scaling_autotest',
> 
Good catch. I can fix on apply.
I'll also fix build error caused in this patch by the earlier applied
patch for cross-compile support. The build of the binary needs 
"c_args: machine_arg" added to it following that change.

/Bruce

Re: [dpdk-dev] [PATCH v2 2/2] meson: add tests app to build

[Bruce Richardson]

On Wed, Dec 20, 2017 at 11:16:32AM +0000, Harry van Haaren wrote:
> This patch enables the test/test app to be built. It also adds
> the test binary to be a meson-test, which allows the meson test
> infrastructure to be used to run tests.
> 
> Tests are listed using the same test binary, however each test
> sets a different DPDK_TEST environment variable. The string contents
> of this DPDK_TEST env var is entered in the command line interface.
> As such, the familiar test names such as "ring_perf_autotest" etc
> are valid tests to run using this meson test infrastructure.
> 
> Note that the tests are run serially, given that we cannot run
> multiple primary processes at a time. As each test must initialize
> EAL this takes some time depending on the number of hugepages.
> In future, we could improve this to run multiple tests from one
> EAL init, but it is out of scope for this patchset.
> 
> Finally, an option to build the tests is added to the meson build
> options. When disabled, the unit test code in test/test is not
> compiled. The default is set to 'true'. To disable, run:
> 
> $ meson configure -Dtests=false
> 
> Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
> 
> ---
> 
> v2:
> - Updated for SPDX license headers (Bruce)

Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Re: [dpdk-dev] [PATCH v2 0/2] next-build: add test app to build

[Bruce Richardson]

On Wed, Dec 20, 2017 at 11:16:30AM +0000, Harry van Haaren wrote:
> This patchset adds the test/test/test app to the Meson build.
> In doing so, the test app is improved to allow running a unit
> test by setting an environment variable. This allows the meson
> test infrastructure to integrate with the DPDK tests. Some nice
> improvements from the above integration, including debug aids...
> 
> Run all autotests:
> $ meson test
> 
> Run a specific test:
> $ meson test ring_perf_autotest
> 
> Run a specific test multiple times, eg brute-forcing race conditions:
> $ meson test eventdev_sw_autotest --repeat=3
> 
> Run a specific test multiple times in gdb, eg to drop to GDB if race found:
> $ meson test eventdev_sw_autotest --repeat=3 --gdb
> 
> 
> Meson also provides various options to "wrap" the test binary,
> which can be used for running in eg: Valgrind or other tools. For
> more information about Meson and its testing capabilities, see here:
> http://mesonbuild.com/Unit-tests.html
> 
> Cheers, -Harry
> 
> ---
> 
> v2: updated patch 2/2 to SPDX license
> 
> Harry van Haaren (2):
>   test: use env variable to run test if set
>   meson: add tests app to build
> 
Applied to dpdk-next-build

Thanks,
/Bruce