[24.03 RFC 0/3] Add argument manipulation library

[24.03 RFC 0/3] Add argument manipulation library

[Bruce Richardson]

DPDK has traditionally assumed that apps are written where argc/argv
parameters are passed directly to rte_eal_init() and then app arguments
are handled afterwards, as is done in the DPDK apps and examples.

However, based on other projects, like VPP and OVS, we know that this is
often not the case. Instead, the apps build up argument lists internally
themselves and pass those to the init function, and DPDK never directly
accesses the real argc/argv values.

Therefore, let's make this mode of operation a little easier to use, by
adding in an args library to allow an app to dynamically build up an
args array and then pass that to EAL init. This library allows things
like, for example, initializing a list from argv and then checking if a
particular parameter is present, adding it if not. All the basics of
such a library are included in patch 1, and I've found it of use myself
when writing some simple apps internally using DPDK.

Patches 2 and 3 go a little further than this, and allow for a slightly
different use-case, that where an app may want to allow mixing of EAL
arguments in with app args. These patches allow the user to query if a
particular argument is an EAL arg or not, or a valid app argument. The
idea here would be, that an app could go through it's argument list,
building up an argument list for EAL init by just picking out the EAL
arguments from a full argument list. I'm less convinced of the value of
this compared to the basics in patch 1, but I think the idea is
interesting so seeking feedback on it.

Bruce Richardson (3):
  args: new library to allow easier manipulation of cmdline args
  eal: allow export of the cmdline argument parsing options
  args: add functions to check parameter validity

 doc/api/doxy-api-index.md           |   1 +
 doc/api/doxy-api.conf.in            |   1 +
 lib/args/args.c                     | 301 ++++++++++++++++++++++++++++
 lib/args/meson.build                |   5 +
 lib/args/rte_args.h                 | 255 +++++++++++++++++++++++
 lib/args/version.map                |  22 ++
 lib/eal/common/eal_common_options.c |   9 +
 lib/eal/include/rte_eal.h           |  14 ++
 lib/eal/version.map                 |   1 +
 lib/meson.build                     |   2 +
 10 files changed, 611 insertions(+)
 create mode 100644 lib/args/args.c
 create mode 100644 lib/args/meson.build
 create mode 100644 lib/args/rte_args.h
 create mode 100644 lib/args/version.map

--
2.39.2

[24.03 RFC 1/3] args: new library to allow easier manipulation of cmdline args

[Bruce Richardson]

Add a new small library to make it easier for apps to work with cmdline
arguments and build up args to use when initializing EAL.

This library is optional, and can be disabled at build time using
the disable libraries meson option.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 doc/api/doxy-api-index.md |   1 +
 doc/api/doxy-api.conf.in  |   1 +
 lib/args/args.c           | 179 ++++++++++++++++++++++++++++++++++
 lib/args/meson.build      |   5 +
 lib/args/rte_args.h       | 199 ++++++++++++++++++++++++++++++++++++++
 lib/args/version.map      |  19 ++++
 lib/meson.build           |   2 +
 7 files changed, 406 insertions(+)
 create mode 100644 lib/args/args.c
 create mode 100644 lib/args/meson.build
 create mode 100644 lib/args/rte_args.h
 create mode 100644 lib/args/version.map

diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md
index a6a768bd7c..6ba1bb40e9 100644
--- a/doc/api/doxy-api-index.md
+++ b/doc/api/doxy-api-index.md
@@ -222,6 +222,7 @@ The public API headers are grouped by topics:
   [key/value args](@ref rte_kvargs.h),
   [string](@ref rte_string_fns.h),
   [thread](@ref rte_thread.h)
+  [cmdline args management](@ref rte_args.h),
 
 - **debug**:
   [jobstats](@ref rte_jobstats.h),
diff --git a/doc/api/doxy-api.conf.in b/doc/api/doxy-api.conf.in
index e94c9e4e46..15b540553d 100644
--- a/doc/api/doxy-api.conf.in
+++ b/doc/api/doxy-api.conf.in
@@ -28,6 +28,7 @@ INPUT                   = @TOPDIR@/doc/api/doxy-api-index.md \
                           @TOPDIR@/lib/eal/include \
                           @TOPDIR@/lib/eal/include/generic \
                           @TOPDIR@/lib/acl \
+                          @TOPDIR@/lib/args \
                           @TOPDIR@/lib/bbdev \
                           @TOPDIR@/lib/bitratestats \
                           @TOPDIR@/lib/bpf \
diff --git a/lib/args/args.c b/lib/args/args.c
new file mode 100644
index 0000000000..80eb6670da
--- /dev/null
+++ b/lib/args/args.c
@@ -0,0 +1,179 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2023 Intel Corporation
+ */
+
+#include <stdint.h>
+#include <malloc.h>
+#include <getopt.h>
+#include <stdarg.h>
+#include <errno.h>
+
+#include <rte_eal.h>
+#include <rte_errno.h>
+#include <rte_common.h>
+#include <rte_bitops.h>
+#include "rte_args.h"
+
+struct rte_args {
+	int array_sz;
+	int arg_count;
+	char **args;
+};
+
+#define DEFAULT_SIZE_HINT 8
+
+struct rte_args *
+rte_args_alloc(uint32_t size_hint)
+{
+	struct rte_args *a;
+
+	a = malloc(sizeof(*a));
+	if (a == NULL)
+		return NULL;
+
+	if (size_hint == 0)
+		size_hint = DEFAULT_SIZE_HINT;
+	size_hint = rte_align32pow2(size_hint);
+	a->args = calloc(size_hint, sizeof(a->args[0]));
+	if (a->args == NULL) {
+		free(a);
+		return NULL;
+	}
+
+	memset(a->args, 0, sizeof(a->args[0]) * size_hint);
+	a->arg_count = 0;
+	a->array_sz = size_hint;
+	return a;
+}
+
+struct rte_args *
+rte_args_create(int argc, char **argv)
+{
+	struct rte_args *a;
+
+	if (argc <= 0 || argv == NULL)
+		return NULL;
+
+	a = rte_args_alloc(argc + DEFAULT_SIZE_HINT);
+	if (a == NULL)
+		return NULL;
+
+	if (rte_args_add_argv(a, argc, argv) != 0) {
+		rte_args_free(a);
+		return NULL;
+	}
+	return a;
+}
+
+
+int
+rte_args_add(struct rte_args *a, const char *arg)
+{
+	if (a == NULL)
+		return -1;
+
+	if (a->arg_count == a->array_sz - 1) {  /* need null at end, so subtract 1 */
+		void *new_a = reallocarray(a->args, a->array_sz * 2, sizeof(a->args[0]));
+		if (new_a == NULL)
+			return -1;
+
+		a->args = new_a;
+		/* zero out only new portion of array */
+		memset(&a->args[a->array_sz], 0, a->array_sz * sizeof(a->args[0]));
+		a->array_sz *= 2;
+	}
+
+	a->args[a->arg_count] = strdup(arg);
+	if (a->args[a->arg_count] == NULL)
+		return -1;
+	a->arg_count++;
+	return 0;
+}
+
+int
+rte_args_add_list(struct rte_args *a, int n, ...)
+{
+	va_list ap;
+	va_start(ap, n);
+	while (n > 0) {
+		if (rte_args_add(a, va_arg(ap, char *)) != 0)
+			break;
+		n--;
+	}
+	va_end(ap);
+	return n;  /* return the number failing, 0 if all ok */
+}
+
+int
+rte_args_add_argv(struct rte_args *a, int argc, char **argv)
+{
+	int i;
+
+	for (i = 0; i < argc; i++)
+		if (rte_args_add(a, argv[i]) < 0)
+			break;
+	return argc - i; /* return the number failing, 0 if all ok */
+}
+
+bool
+rte_args_has_arg(struct rte_args *a, const char *arg)
+{
+	int i;
+
+	if (a == NULL || arg == NULL)
+		return false;
+
+	for (i = 0; i < a->arg_count; i++) {
+		if (strcmp(a->args[i], arg) == 0)
+			return true;
+	}
+	return false;
+}
+
+char **
+rte_args_get_argv(struct rte_args *a, int *argc)
+{
+	if (a == NULL)
+		return NULL;
+
+	if (argc != NULL)
+		*argc = a->arg_count;
+	return a->args;
+}
+
+int
+rte_args_get_argc(struct rte_args *a)
+{
+	if (a == NULL)
+		return -1;
+
+	return a->arg_count;
+}
+
+void
+rte_args_free(struct rte_args *a)
+{
+	int i;
+
+	if (a == NULL)
+		return;
+
+	for (i = 0; i < a->arg_count; i++)
+		free(a->args[i]);
+	free(a->args);
+	free(a);
+}
+
+int
+rte_args_eal_init(struct rte_args *a)
+{
+	int argc;
+	char **argv;
+
+	/* rte_args_argv does the check for NULL for us */
+	argv = rte_args_get_argv(a, &argc);
+	if (argv == NULL || argc <= 0)
+		return -1;
+
+	return rte_eal_init(argc, argv);
+}
diff --git a/lib/args/meson.build b/lib/args/meson.build
new file mode 100644
index 0000000000..997f76a8bb
--- /dev/null
+++ b/lib/args/meson.build
@@ -0,0 +1,5 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2023 Intel Corporation
+
+sources = files('args.c')
+headers = files('rte_args.h')
diff --git a/lib/args/rte_args.h b/lib/args/rte_args.h
new file mode 100644
index 0000000000..3b80b9a39c
--- /dev/null
+++ b/lib/args/rte_args.h
@@ -0,0 +1,199 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2023 Intel Corporation
+ */
+#ifndef RTE_EAL_ARGS_H
+#define RTE_EAL_ARGS_H
+
+/**
+ * @file
+ *
+ * Command-line argument manipulation functions
+ *
+ * Simple functions for manipulating sets of commandline arguments, and then
+ * initializing DPDK / EAL based on those.
+ */
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include <stdint.h>
+
+#include <rte_compat.h>
+
+struct rte_args;
+
+/**
+ * Allocate an argument structure
+ *
+ * This functions reserves memory for an argument structure. On success the returned structure
+ * is guaranteed to hold at least size_hint arguments without being resized. It is a hint only -
+ * if more elements are added than that hinted, the rte_args structure will be dynamically resized
+ * as necessary (subject to memory being available to do so).
+ *
+ * @param size_hint
+ *   The returned structure is guaranteed to hold at least this many arguments without resizing.
+ *   If zero, a default non-zero size value will be used.
+ * @return
+ *   An empty rte_args structure, NULL on failure
+ */
+__rte_experimental
+struct rte_args *
+rte_args_alloc(uint32_t size_hint);
+
+/**
+ * Create an argument structure containing the existing parameters
+ *
+ * This functions creates an rte_args structure and initializes it with "argc" values
+ * from the "argv" array.
+ *
+ * @param argc
+ *   The number of elements in argv
+ * @param argv
+ *   Array of arguments to add to the created rte_args structure
+ * @return
+ *   An rte_args structure holding argc elements from argv. NULL on failure.
+ */
+__rte_experimental
+struct rte_args *
+rte_args_create(int argc, char **argv);
+
+/**
+ * Add a new argument to the rte_args structure
+ *
+ * Add the argument "arg" to the rte_args structure, resizing the structure as necessary.
+ * The arg parameter will be copied using strdup, so the original string may be freed or
+ * edited as necessary after the function returns.
+ *
+ * @param a
+ *   The rte_arg structure to be modified
+ * @param arg
+ *   The argument to be added
+ * @return
+ *   0 on success, -1 on failure
+ */
+__rte_experimental
+int
+rte_args_add(struct rte_args *a, const char *arg);
+
+/**
+ * Add a list of arguments to the rte_args structure
+ *
+ * Add "n" arguments, given as extra function parameters to the rte_args structure, resizing
+ * the structure as necessary.
+ * The argument parameters will be copied using strdup, so the original strings may be freed or
+ * edited as necessary after the function returns.
+ *
+ * @param a
+ *   The rte_arg structure to be modified
+ * @param n
+ *   The number of arguments to be added
+ * @return
+ *   0 on success.
+ *   On failure, returns the number of elements NOT added.
+ */
+__rte_experimental
+int
+rte_args_add_list(struct rte_args *a, int n, ...);
+
+/**
+ * Add an array of arguments to the rte_args structure
+ *
+ * Add arguments from an existing argv array to the rte_args structure.
+ * The argument parameters will be copied using strdup, so the original strings/array may
+ * be freed or edited as necessary after the function returns.
+ *
+ * @param a
+ *   The rte_arg structure to be modified
+ * @param argc
+ *   The number of arguments to be added
+ * @param argv
+ *   The array containing the pointers to the elements to be added
+ * @return
+ *   0 on success.
+ *   On failure, returns the number of elements NOT added.
+ */
+__rte_experimental
+int
+rte_args_add_argv(struct rte_args *a, int argc, char **argv);
+
+/**
+ * Query if an args list contains a given argument parameter
+ *
+ * @param a
+ *   The rte_arg structure to be modified
+ * @param arg
+ *   The argument to look for in the structure
+ * @return
+ *   True if parameters are valid and argument is found
+ *   False otherwise
+ */
+__rte_experimental
+bool
+rte_args_has_arg(struct rte_args *a, const char *arg);
+
+/**
+ * Get the array of arguments from the rte_args structure
+ *
+ * Returns an array of arguments from the structure, suitable for passing to a function which
+ * takes (argc, argv) parameters. The argc value will be optionally returned in the final output
+ * parameter if a valid pointer is passed in.
+ *
+ * NOTE: The return value points to the internals of the rte_args structure, and so the
+ * structure should not be freed while the arguments are in use.
+ *
+ * @param a
+ *   The rte_arg structure to be used
+ * @param argc
+ *   Pointer to store the count (argc) of elements returned. Ignored if NULL
+ * @return
+ *   Pointer to an array of arguments. NULL on failure.
+ */
+__rte_experimental
+char **
+rte_args_get_argv(struct rte_args *a, int *argc);
+
+/**
+ * Gets the number of arguments stored in an rte_args structure
+ *
+ * @param a
+ *   The rte_arg structure to be used
+ * @return
+ *   The number of arguments stored in the structure
+ */
+__rte_experimental
+int
+rte_args_get_argc(struct rte_args *a);
+
+/**
+ * Frees an rte_args structure
+ *
+ * @param a
+ *   The rte_arg structure to be freed
+ */
+__rte_experimental
+void
+rte_args_free(struct rte_args *a);
+
+/**
+ * Initialize DPDK EAL using arguments from the rte_args structure
+ *
+ * This function calls rte_eal_init(), passing in to it as parameters the argc, and argv
+ * values got from the rte_args structure "a".
+ *
+ * @param a
+ *   The rte_args structure to be used. Must have at least one argument in it.
+ * @return
+ *   -1 on invalid parameter
+ *   Otherwise, return value from rte_eal_init()
+ */
+__rte_experimental
+int
+rte_args_eal_init(struct rte_args *a);
+
+#ifdef __cplusplus
+}
+#endif
+
+
+#endif /* RTE_EAL_ARGS_H */
diff --git a/lib/args/version.map b/lib/args/version.map
new file mode 100644
index 0000000000..776bd22b82
--- /dev/null
+++ b/lib/args/version.map
@@ -0,0 +1,19 @@
+DPDK_24 {
+	local: *;
+};
+
+EXPERIMENTAL {
+	global:
+
+	# added in 24.03
+	rte_args_add;
+	rte_args_add_argv;
+	rte_args_add_list;
+	rte_args_alloc;
+	rte_args_create;
+	rte_args_eal_init;
+	rte_args_free;
+	rte_args_get_argc;
+	rte_args_get_argv;
+	rte_args_has_arg;
+};
diff --git a/lib/meson.build b/lib/meson.build
index 6c143ce5a6..7049a4b00e 100644
--- a/lib/meson.build
+++ b/lib/meson.build
@@ -13,6 +13,7 @@ libraries = [
         'kvargs', # eal depends on kvargs
         'telemetry', # basic info querying
         'eal', # everything depends on eal
+        'args',
         'ring',
         'rcu', # rcu depends on ring
         'mempool',
@@ -76,6 +77,7 @@ if is_ms_compiler
 endif
 
 optional_libs = [
+        'args',
         'bbdev',
         'bitratestats',
         'bpf',
-- 
2.39.2

[24.03 RFC 2/3] eal: allow export of the cmdline argument parsing options

[Bruce Richardson]

Allow other libraries in DPDK to know about the options EAL takes, by
exporting the long and short getopt options via an internal-only
function.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 lib/eal/common/eal_common_options.c |  9 +++++++++
 lib/eal/include/rte_eal.h           | 14 ++++++++++++++
 lib/eal/version.map                 |  1 +
 3 files changed, 24 insertions(+)

diff --git a/lib/eal/common/eal_common_options.c b/lib/eal/common/eal_common_options.c
index a6d21f1cba..e26f66ea07 100644
--- a/lib/eal/common/eal_common_options.c
+++ b/lib/eal/common/eal_common_options.c
@@ -175,6 +175,15 @@ rte_set_application_usage_hook(rte_usage_hook_t usage_func)
 	return old_func;
 }
 
+__rte_internal
+int
+rte_eal_getopt_params(const char **shortopts, const struct option **longopts)
+{
+	*shortopts = eal_short_options;
+	*longopts = eal_long_options;
+	return 0;
+}
+
 #ifndef RTE_EXEC_ENV_WINDOWS
 static char **eal_args;
 static char **eal_app_args;
diff --git a/lib/eal/include/rte_eal.h b/lib/eal/include/rte_eal.h
index cd318ee141..3bc5447abf 100644
--- a/lib/eal/include/rte_eal.h
+++ b/lib/eal/include/rte_eal.h
@@ -514,6 +514,20 @@ __rte_internal
 int
 rte_eal_parse_coremask(const char *coremask, int *cores);
 
+struct option;
+/**
+ * @internal
+ * Return the getopt parameters used by EAL when parsing cmdline arguments
+ *
+ * @shortopts
+ *   Pointer for the short options for getopt
+ * @longopts
+ *   Pointer for the longer options used by getopt_long
+ */
+__rte_internal
+int
+rte_eal_getopt_params(const char **shortopts, const struct option **longopts);
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/lib/eal/version.map b/lib/eal/version.map
index e00a844805..6c0a7145ff 100644
--- a/lib/eal/version.map
+++ b/lib/eal/version.map
@@ -421,6 +421,7 @@ INTERNAL {
 	rte_bus_register;
 	rte_bus_unregister;
 	rte_eal_get_baseaddr;
+	rte_eal_getopt_params;
 	rte_eal_parse_coremask;
 	rte_firmware_read;
 	rte_intr_allow_others;
-- 
2.39.2

[24.03 RFC 3/3] args: add functions to check parameter validity

[Bruce Richardson]

Add in functions which can be used to check for valid arguments for EAL
or for the application. This can be used to separate out mixed arguments.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 lib/args/args.c      | 122 +++++++++++++++++++++++++++++++++++++++++++
 lib/args/rte_args.h  |  56 ++++++++++++++++++++
 lib/args/version.map |   3 ++
 3 files changed, 181 insertions(+)

diff --git a/lib/args/args.c b/lib/args/args.c
index 80eb6670da..c04faa6323 100644
--- a/lib/args/args.c
+++ b/lib/args/args.c
@@ -22,6 +22,15 @@ struct rte_args {
 
 #define DEFAULT_SIZE_HINT 8
 
+#define INVALID_ARG '?'
+#define VALID_ARG 'x'
+#define PARAM_ARG ':'
+
+static char *app_shortopts;
+static struct option *app_longopts;
+static char *eal_shortopts;
+static struct option *eal_longopts;
+
 struct rte_args *
 rte_args_alloc(uint32_t size_hint)
 {
@@ -177,3 +186,116 @@ rte_args_eal_init(struct rte_args *a)
 
 	return rte_eal_init(argc, argv);
 }
+
+static int
+args_register_opts(const char *shortopts, const struct option *longopts,
+		char **sopt_var, struct option **lopt_var)
+{
+	static struct option emptyopt = {0};
+	struct option *lopts = &emptyopt;
+	char *sopts;
+
+	if (shortopts == NULL) {
+		rte_errno = EINVAL;
+		return -1;
+	}
+
+	/* for short options, we need to pre-pend ':' for extra argument reporting */
+	if (shortopts[0] != '\0' && shortopts[0] != ':') {
+		size_t len = strlen(shortopts);
+
+		sopts = malloc(len + 2);
+		if (sopts == NULL) {
+			rte_errno = ENOMEM;
+			return -1;
+		}
+		snprintf(sopts, len + 2, ":%s", shortopts);
+	} else
+		sopts = strdup(shortopts);
+
+	/* for long options, we need to ensure we always return a value, not setting a flag */
+	if (longopts != NULL) {
+		int i, n = 0;
+		while (memcmp(&longopts[n], &emptyopt, sizeof(emptyopt)) != 0)
+			n++;
+		lopts = calloc(n + 1, sizeof(emptyopt));
+		if (lopts == NULL) {
+			free(sopts);
+			rte_errno = ENOMEM;
+			return -1;
+		}
+		for (i = 0; i < n; i++) {
+			lopts[i].name = longopts[i].name;
+			lopts[i].has_arg = longopts[i].has_arg;
+			lopts[i].flag = NULL;
+			lopts[i].val = VALID_ARG;
+		}
+		memset(&lopts[i], 0, sizeof(lopts[i]));
+	}
+
+	if (*sopt_var != NULL)
+		free(*sopt_var);
+	if (*lopt_var != NULL)
+		free(*lopt_var);
+
+	*sopt_var = sopts;
+	*lopt_var = lopts;
+	return 0;
+}
+
+int
+rte_args_register_app_opts(const char *shortopts, const struct option *longopts)
+{
+	return args_register_opts(shortopts, longopts, &app_shortopts, &app_longopts);
+}
+
+static int
+args_is_valid_arg(const char *arg, const char *sopts, const struct option *lopts)
+{
+	const int save_optind = optind,
+			save_opterr = opterr,
+			save_optopt = optopt;
+
+	int ret = getopt_long(1, (void *)&arg, sopts, lopts, NULL);
+	switch (ret) {
+	case INVALID_ARG:
+		ret = 0; break;
+	case VALID_ARG:
+		ret = 1; break;
+	case PARAM_ARG:
+		ret = 2; break;
+	default:
+		ret = -1; break;
+	}
+
+	/* restore saved values */
+	optind = save_optind;
+	opterr = save_opterr;
+	optopt = save_optopt;
+	return ret;
+}
+
+int
+rte_args_is_app_arg(const char *arg)
+{
+	if (app_shortopts == NULL || app_longopts == NULL) {
+		rte_errno = EINVAL;
+		return -1;
+	}
+
+	return args_is_valid_arg(arg, app_shortopts, app_longopts);
+}
+
+int
+rte_args_is_eal_arg(const char *arg)
+{
+	if (eal_shortopts == NULL || eal_longopts == NULL) {
+		const struct option *l;
+		const char *s;
+
+		rte_eal_getopt_params(&s, &l);
+		if (args_register_opts(s, l, &eal_shortopts, &eal_longopts) < 0)
+			return -1;
+	}
+	return args_is_valid_arg(arg, eal_shortopts, eal_longopts);;
+}
diff --git a/lib/args/rte_args.h b/lib/args/rte_args.h
index 3b80b9a39c..7e2ba5348d 100644
--- a/lib/args/rte_args.h
+++ b/lib/args/rte_args.h
@@ -191,6 +191,62 @@ __rte_experimental
 int
 rte_args_eal_init(struct rte_args *a);
 
+/**
+ * Register the application arguments to allow testing args for validity
+ *
+ * This function should only be called once per application, and each call
+ * will replace any previouly registered values. This function is not
+ * multi-thread safe and should only be called from one thread at a time.
+ *
+ * @param shortopts
+ *   The array of short options, as passed to getopt/getopt_long
+ * @param longopts
+ *   The array of longer options, as passed to getopt_long
+ * @return
+ *    -1 on error, with rte_errno set appropriately;
+ *    0 otherwise
+ */
+__rte_experimental
+int
+rte_args_register_app_opts(const char *shortopts, const struct option *longopts);
+
+/**
+ * Determine if a given argument is valid or not for the app
+ *
+ * Determine if the passed argument is valid or not, based on previously registered
+ * application arguments. This function uses getopt, and so is not multi-thread safe.
+ *
+ * @param arg
+ *   The argument to test
+ * @return
+ *   0 - the argument is not a valid one
+ *   1 - the argument is valid
+ *   2 - the argument is valid, but it takes the next argument as parameter
+ *   or -1 on error, with rte_errno set appropriately
+ */
+__rte_experimental
+int
+rte_args_is_app_arg(const char *arg);
+
+
+/**
+ * Determine if a given argument is an EAL argument or not
+ *
+ * Determine if the passed argument is valid or not for EAL initialization.
+ * This function uses getopt, and so is not multi-thread safe.
+
+ * @param arg
+ *   The argument to test
+ * @return
+ *   0 - the argument is not a valid one for EAL
+ *   1 - the argument is valid for EAL
+ *   2 - the argument is valid for EAL, but it takes the next argument as parameter
+ *   or -1 on error, with rte_errno set appropriately
+ */
+__rte_experimental
+int
+rte_args_is_eal_arg(const char *arg);
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/lib/args/version.map b/lib/args/version.map
index 776bd22b82..ef80f0d762 100644
--- a/lib/args/version.map
+++ b/lib/args/version.map
@@ -16,4 +16,7 @@ EXPERIMENTAL {
 	rte_args_get_argc;
 	rte_args_get_argv;
 	rte_args_has_arg;
+	rte_args_is_app_arg;
+	rte_args_is_eal_arg;
+	rte_args_register_app_opts;
 };
-- 
2.39.2

Re: [24.03 RFC 0/3] Add argument manipulation library

[Stephen Hemminger]

On Thu,  2 Nov 2023 17:28:46 +0000
Bruce Richardson <bruce.richardson@intel.com> wrote:

> DPDK has traditionally assumed that apps are written where argc/argv
> parameters are passed directly to rte_eal_init() and then app arguments
> are handled afterwards, as is done in the DPDK apps and examples.
> 
> However, based on other projects, like VPP and OVS, we know that this is
> often not the case. Instead, the apps build up argument lists internally
> themselves and pass those to the init function, and DPDK never directly
> accesses the real argc/argv values.
> 
> Therefore, let's make this mode of operation a little easier to use, by
> adding in an args library to allow an app to dynamically build up an
> args array and then pass that to EAL init. This library allows things
> like, for example, initializing a list from argv and then checking if a
> particular parameter is present, adding it if not. All the basics of
> such a library are included in patch 1, and I've found it of use myself
> when writing some simple apps internally using DPDK.
> 
> Patches 2 and 3 go a little further than this, and allow for a slightly
> different use-case, that where an app may want to allow mixing of EAL
> arguments in with app args. These patches allow the user to query if a
> particular argument is an EAL arg or not, or a valid app argument. The
> idea here would be, that an app could go through it's argument list,
> building up an argument list for EAL init by just picking out the EAL
> arguments from a full argument list. I'm less convinced of the value of
> this compared to the basics in patch 1, but I think the idea is
> interesting so seeking feedback on it.
> 
> Bruce Richardson (3):
>   args: new library to allow easier manipulation of cmdline args
>   eal: allow export of the cmdline argument parsing options
>   args: add functions to check parameter validity
> 
>  doc/api/doxy-api-index.md           |   1 +
>  doc/api/doxy-api.conf.in            |   1 +
>  lib/args/args.c                     | 301 ++++++++++++++++++++++++++++
>  lib/args/meson.build                |   5 +
>  lib/args/rte_args.h                 | 255 +++++++++++++++++++++++
>  lib/args/version.map                |  22 ++
>  lib/eal/common/eal_common_options.c |   9 +
>  lib/eal/include/rte_eal.h           |  14 ++
>  lib/eal/version.map                 |   1 +
>  lib/meson.build                     |   2 +
>  10 files changed, 611 insertions(+)
>  create mode 100644 lib/args/args.c
>  create mode 100644 lib/args/meson.build
>  create mode 100644 lib/args/rte_args.h
>  create mode 100644 lib/args/version.map
> 
> --
> 2.39.2
> 

It would be good to change rte_eal_init() from:
int rte_eal_init(int argc, char **argv);

to
int rte_eal_init(int argc, char * const *argv);

I.e it should not modify the argv's passed.
This would save unnecessary copy in applications that build synthetic args.
Example already exists in dumpcap.

Re: [24.03 RFC 0/3] Add argument manipulation library

[Bruce Richardson]

On Thu, Nov 02, 2023 at 10:50:29AM -0700, Stephen Hemminger wrote:
> On Thu,  2 Nov 2023 17:28:46 +0000
> Bruce Richardson <bruce.richardson@intel.com> wrote:
> 
> > DPDK has traditionally assumed that apps are written where argc/argv
> > parameters are passed directly to rte_eal_init() and then app arguments
> > are handled afterwards, as is done in the DPDK apps and examples.
> > 
> > However, based on other projects, like VPP and OVS, we know that this is
> > often not the case. Instead, the apps build up argument lists internally
> > themselves and pass those to the init function, and DPDK never directly
> > accesses the real argc/argv values.
> > 
> > Therefore, let's make this mode of operation a little easier to use, by
> > adding in an args library to allow an app to dynamically build up an
> > args array and then pass that to EAL init. This library allows things
> > like, for example, initializing a list from argv and then checking if a
> > particular parameter is present, adding it if not. All the basics of
> > such a library are included in patch 1, and I've found it of use myself
> > when writing some simple apps internally using DPDK.
> > 
> > Patches 2 and 3 go a little further than this, and allow for a slightly
> > different use-case, that where an app may want to allow mixing of EAL
> > arguments in with app args. These patches allow the user to query if a
> > particular argument is an EAL arg or not, or a valid app argument. The
> > idea here would be, that an app could go through it's argument list,
> > building up an argument list for EAL init by just picking out the EAL
> > arguments from a full argument list. I'm less convinced of the value of
> > this compared to the basics in patch 1, but I think the idea is
> > interesting so seeking feedback on it.
> > 
> > Bruce Richardson (3):
> >   args: new library to allow easier manipulation of cmdline args
> >   eal: allow export of the cmdline argument parsing options
> >   args: add functions to check parameter validity
> > 
> >  doc/api/doxy-api-index.md           |   1 +
> >  doc/api/doxy-api.conf.in            |   1 +
> >  lib/args/args.c                     | 301 ++++++++++++++++++++++++++++
> >  lib/args/meson.build                |   5 +
> >  lib/args/rte_args.h                 | 255 +++++++++++++++++++++++
> >  lib/args/version.map                |  22 ++
> >  lib/eal/common/eal_common_options.c |   9 +
> >  lib/eal/include/rte_eal.h           |  14 ++
> >  lib/eal/version.map                 |   1 +
> >  lib/meson.build                     |   2 +
> >  10 files changed, 611 insertions(+)
> >  create mode 100644 lib/args/args.c
> >  create mode 100644 lib/args/meson.build
> >  create mode 100644 lib/args/rte_args.h
> >  create mode 100644 lib/args/version.map
> > 
> > --
> > 2.39.2
> > 
> 
> It would be good to change rte_eal_init() from:
> int rte_eal_init(int argc, char **argv);
> 
> to
> int rte_eal_init(int argc, char * const *argv);
> 
> I.e it should not modify the argv's passed.
> This would save unnecessary copy in applications that build synthetic args.
> Example already exists in dumpcap.
> 

Pushed up a quick patch proposal for that:
http://patches.dpdk.org/project/dpdk/patch/20231102181148.56930-1-bruce.richardson@intel.com/

Re: [24.03 RFC 3/3] args: add functions to check parameter validity

[Thomas Monjalon]

02/11/2023 18:28, Bruce Richardson:
> Add in functions which can be used to check for valid arguments for EAL
> or for the application. This can be used to separate out mixed arguments.
> 
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> ---
>  lib/args/args.c      | 122 +++++++++++++++++++++++++++++++++++++++++++
>  lib/args/rte_args.h  |  56 ++++++++++++++++++++

This is helping with args taken by EAL.
So I think it should be hosted in EAL itself.

Re: [24.03 RFC 1/3] args: new library to allow easier manipulation of cmdline args

[Thomas Monjalon]

02/11/2023 18:28, Bruce Richardson:
> Add a new small library to make it easier for apps to work with cmdline
> arguments and build up args to use when initializing EAL.
> 
> This library is optional, and can be disabled at build time using
> the disable libraries meson option.

This is an optional helper, so why not.

Another help for applications would be to allow initializing DPDK
without the need of passing or building argc/argv arguments.
I think we could add new functions rte_eal_init_*().
Example:
	rte_eal_init_prepare()
	rte_eal_init_memory(memory parameters)
	rte_eal_init_devices(devargs)
	rte_eal_init_threads()

It should be possible to rebuild rte_eal_init()
using above smaller functions to keep the big old
rte_eal_init with argc/argv for compatibility.

RE: [24.03 RFC 1/3] args: new library to allow easier manipulation of cmdline args

[Honnappa Nagarahalli]

<snip>

> 
> 02/11/2023 18:28, Bruce Richardson:
> > Add a new small library to make it easier for apps to work with
> > cmdline arguments and build up args to use when initializing EAL.
> >
> > This library is optional, and can be disabled at build time using the
> > disable libraries meson option.
> 
> This is an optional helper, so why not.
> 
> Another help for applications would be to allow initializing DPDK without the
> need of passing or building argc/argv arguments.
> I think we could add new functions rte_eal_init_*().
> Example:
> 	rte_eal_init_prepare()
> 	rte_eal_init_memory(memory parameters)
> 	rte_eal_init_devices(devargs)
> 	rte_eal_init_threads()
> 
> It should be possible to rebuild rte_eal_init() using above smaller functions to
> keep the big old rte_eal_init with argc/argv for compatibility.
We have to ensure these new APIs and argc/argv are in sync in future changes. For ex: if we add a new parameter to the API, we need a new argv. This is not a problem, but we have to do this manually.

>