[RFC PATCH] app/test: allow passing a parameter string to autotests

[RFC PATCH] app/test: allow passing a parameter string to autotests

[Bruce Richardson]

Sometimes it can be nice to have autotests which can take a parameter,
or can be tweaked in some ways, e.g. adjust the number of iterations, or
the burst size used in the test. Currently there is no way to do so -
all test parameters are hardcoded, which makes sense for a generic
regression test to be run quickly, but is a bit lacking for those
looking to use unit tests for investigations.

To that end, we can all the commandline to accept an additional string
parameter after each autotest name, and make that available via global
variable. Any test which wishes to make use of this can do so, to allow
overriding test defaults.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 app/test/commands.c                   | 33 ++++++++++++++++++++++++++-
 app/test/test.h                       |  2 ++
 doc/guides/contributing/unit_test.rst | 15 ++++++++++++
 3 files changed, 49 insertions(+), 1 deletion(-)

diff --git a/app/test/commands.c b/app/test/commands.c
index 497d8e9952..7672e9db6b 100644
--- a/app/test/commands.c
+++ b/app/test/commands.c
@@ -40,6 +40,8 @@
 
 #include "test.h"
 
+char *test_parameter = NULL;
+
 /****************/
 
 static struct test_commands_list commands_list =
@@ -55,14 +57,25 @@ struct cmd_autotest_result {
 	cmdline_fixed_string_t autotest;
 };
 
+struct cmd_autotest_param_result {
+	cmdline_fixed_string_t autotest;
+	cmdline_fixed_string_t param;
+};
+
 static void cmd_autotest_parsed(void *parsed_result,
 				__rte_unused struct cmdline *cl,
-				__rte_unused void *data)
+				void *data)
 {
 	struct test_command *t;
 	struct cmd_autotest_result *res = parsed_result;
 	int ret = 0;
 
+	test_parameter = NULL;
+	if (data == (void *)1) {
+		struct cmd_autotest_param_result *pres = parsed_result;
+		test_parameter = pres->param;
+	}
+
 	TAILQ_FOREACH(t, &commands_list, next) {
 		if (!strcmp(res->autotest, t->command))
 			ret = t->callback();
@@ -92,6 +105,22 @@ cmdline_parse_inst_t cmd_autotest = {
 	},
 };
 
+cmdline_parse_token_string_t cmd_autotest_param_autotest =
+	TOKEN_STRING_INITIALIZER(struct cmd_autotest_param_result, autotest, "");
+cmdline_parse_token_string_t cmd_autotest_param_param =
+	TOKEN_STRING_INITIALIZER(struct cmd_autotest_param_result, param, NULL);
+
+cmdline_parse_inst_t cmd_autotest_with_param = {
+	.f = cmd_autotest_parsed,
+	.data = (void *)1,
+	.help_str = "launch autotest, providing a string parameter",
+	.tokens = {
+			(void *)&cmd_autotest_param_autotest,
+			(void *)&cmd_autotest_param_param,
+			NULL,
+	},
+};
+
 /****************/
 
 struct cmd_dump_result {
@@ -348,6 +377,7 @@ cmdline_parse_inst_t cmd_set_rxtx_sc = {
 
 cmdline_parse_ctx_t main_ctx[] = {
 	(cmdline_parse_inst_t *)&cmd_autotest,
+	(cmdline_parse_inst_t *)&cmd_autotest_with_param,
 	(cmdline_parse_inst_t *)&cmd_dump,
 	(cmdline_parse_inst_t *)&cmd_dump_one,
 	(cmdline_parse_inst_t *)&cmd_quit,
@@ -378,5 +408,6 @@ int commands_init(void)
 	}
 
 	cmd_autotest_autotest.string_data.str = commands;
+	cmd_autotest_param_autotest.string_data.str = commands;
 	return 0;
 }
diff --git a/app/test/test.h b/app/test/test.h
index 15e23d297f..3910317027 100644
--- a/app/test/test.h
+++ b/app/test/test.h
@@ -27,6 +27,8 @@
 
 #include <rte_test.h>
 
+extern char *test_parameter;
+
 #define TEST_ASSERT RTE_TEST_ASSERT
 
 #define TEST_ASSERT_EQUAL RTE_TEST_ASSERT_EQUAL
diff --git a/doc/guides/contributing/unit_test.rst b/doc/guides/contributing/unit_test.rst
index 063cefa192..2ce64e2db7 100644
--- a/doc/guides/contributing/unit_test.rst
+++ b/doc/guides/contributing/unit_test.rst
@@ -374,6 +374,21 @@ Example::
    ...
 
 
+Optional test parameters
+-------------------------
+
+Each unit test suite must be runable without any additional test parameters,
+since automated testing via ``meson test``, and many DPDK Continuous Integration(CI) frameworks,
+will just call each test via cmdline-parameter to the ``dpdk-test`` binary
+or by setting the test name in the ``DPDK_TEST`` environment variable.
+
+However, when run interactively via ``dpdk-test`` CLI,
+an additional parameter may be passed to the autotest command.
+This parameter will be available to the test in the ``test_parameter`` global variable.
+This support is intended for developer use only as a convenience.
+It is up to the each test how to use this parameter and what form it may have.
+
+
 Checking code coverage
 ----------------------
 
-- 
2.40.1

RE: [RFC PATCH] app/test: allow passing a parameter string to autotests

[Morten Brørup]

> From: Bruce Richardson [mailto:bruce.richardson@intel.com]
> Sent: Friday, 15 December 2023 14.07
> 
> Sometimes it can be nice to have autotests which can take a parameter,
> or can be tweaked in some ways, e.g. adjust the number of iterations,
> or
> the burst size used in the test. Currently there is no way to do so -
> all test parameters are hardcoded, which makes sense for a generic
> regression test to be run quickly, but is a bit lacking for those
> looking to use unit tests for investigations.
> 
> To that end, we can all the commandline to accept an additional string
> parameter after each autotest name, and make that available via global
> variable. Any test which wishes to make use of this can do so, to allow
> overriding test defaults.
> 
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> ---

This would be useful for the mempool perf test. :-)

It would be nice standardizing on a parameter syntax, though.
Preferably: <param> ["=" <value>] {" " <param> ["=" <value>]}

Acked-by: Morten Brørup <mb@smartsharesystems.com>

Re: [RFC PATCH] app/test: allow passing a parameter string to autotests

[Stephen Hemminger]

On Fri, 15 Dec 2023 13:06:56 +0000
Bruce Richardson <bruce.richardson@intel.com> wrote:

> Sometimes it can be nice to have autotests which can take a parameter,
> or can be tweaked in some ways, e.g. adjust the number of iterations, or
> the burst size used in the test. Currently there is no way to do so -
> all test parameters are hardcoded, which makes sense for a generic
> regression test to be run quickly, but is a bit lacking for those
> looking to use unit tests for investigations.
> 
> To that end, we can all the commandline to accept an additional string
> parameter after each autotest name, and make that available via global
> variable. Any test which wishes to make use of this can do so, to allow
> overriding test defaults.
> 
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>

LGTM
Maybe add an example to an existing test

Acked-by: Stephen Hemminger <stephen@networkplumber.org>