* [PATCH v1 01/12] app/mldev: implement test framework for mldev
2022-11-29 7:07 [PATCH v1 00/12] implement mldev test application Srikanth Yalavarthi
@ 2022-11-29 7:07 ` Srikanth Yalavarthi
2022-11-29 8:20 ` [PATCH v2 " Srikanth Yalavarthi
2022-11-29 7:07 ` [PATCH v1 02/12] app/mldev: add common test functions Srikanth Yalavarthi
` (14 subsequent siblings)
15 siblings, 1 reply; 122+ messages in thread
From: Srikanth Yalavarthi @ 2022-11-29 7:07 UTC (permalink / raw)
To: Thomas Monjalon, Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj
Implemented framework for mldev test application. New test cases
can be added using the framework. Support is also enabled to add
options specific to the test cases. User can launch the tests by
specifying the name of test as part of launch arguments.
Code to parse command line arguments is imported from
test-eventdev, with support to parse additional data types.
Common arguments supported include:
test : name of the test application to run
dev_id : device id of the ML device
socket_id : socket_id of application resources
debug : enable debugging
help : print help
Sample launch command:
./dpdk-test-mldev -- --test <testname> --dev_id <dev_id> \
--socket_id <socket_id>
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
Depends-on: patch-119834 ("mldev: support to get debug info and test device")
MAINTAINERS | 1 +
app/meson.build | 1 +
app/test-mldev/meson.build | 17 ++
app/test-mldev/ml_common.h | 29 +++
app/test-mldev/ml_main.c | 118 +++++++++++
app/test-mldev/ml_options.c | 160 +++++++++++++++
app/test-mldev/ml_options.h | 31 +++
app/test-mldev/ml_test.c | 45 +++++
app/test-mldev/ml_test.h | 75 +++++++
app/test-mldev/parser.c | 380 ++++++++++++++++++++++++++++++++++++
app/test-mldev/parser.h | 55 ++++++
11 files changed, 912 insertions(+)
create mode 100644 app/test-mldev/meson.build
create mode 100644 app/test-mldev/ml_common.h
create mode 100644 app/test-mldev/ml_main.c
create mode 100644 app/test-mldev/ml_options.c
create mode 100644 app/test-mldev/ml_options.h
create mode 100644 app/test-mldev/ml_test.c
create mode 100644 app/test-mldev/ml_test.h
create mode 100644 app/test-mldev/parser.c
create mode 100644 app/test-mldev/parser.h
diff --git a/MAINTAINERS b/MAINTAINERS
index 0c3e6d28e9..1edea42fad 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -538,6 +538,7 @@ F: doc/guides/prog_guide/rawdev.rst
ML device API - EXPERIMENTAL
M: Srikanth Yalavarthi <syalavarthi@marvell.com>
F: lib/mldev/
+F: app/test-mldev/
F: doc/guides/prog_guide/mldev.rst
diff --git a/app/meson.build b/app/meson.build
index e32ea4bd5c..74d2420f67 100644
--- a/app/meson.build
+++ b/app/meson.build
@@ -23,6 +23,7 @@ apps = [
'test-fib',
'test-flow-perf',
'test-gpudev',
+ 'test-mldev',
'test-pipeline',
'test-pmd',
'test-regex',
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
new file mode 100644
index 0000000000..8ca2e1a1c1
--- /dev/null
+++ b/app/test-mldev/meson.build
@@ -0,0 +1,17 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright (c) 2022 Marvell.
+
+if is_windows
+ build = false
+ reason = 'not supported on Windows'
+ subdir_done()
+endif
+
+sources = files(
+ 'ml_main.c',
+ 'ml_options.c',
+ 'ml_test.c',
+ 'parser.c',
+)
+
+deps += ['mldev']
diff --git a/app/test-mldev/ml_common.h b/app/test-mldev/ml_common.h
new file mode 100644
index 0000000000..065180b619
--- /dev/null
+++ b/app/test-mldev/ml_common.h
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_COMMON_
+#define _ML_COMMON_
+
+#include <stdio.h>
+
+#define CLNRM "\x1b[0m"
+#define CLRED "\x1b[31m"
+#define CLGRN "\x1b[32m"
+#define CLYEL "\x1b[33m"
+
+#define ML_STR_FMT 20
+
+#define ml_err(fmt, args...) fprintf(stderr, CLRED "error: %s() " fmt CLNRM "\n", __func__, ##args)
+
+#define ml_info(fmt, args...) fprintf(stdout, CLYEL "" fmt CLNRM "\n", ##args)
+
+#define ml_dump(str, fmt, val...) printf("\t%-*s : " fmt "\n", ML_STR_FMT, str, ##val)
+
+#define ml_dump_begin(str) printf("\t%-*s :\n\t{\n", ML_STR_FMT, str)
+
+#define ml_dump_list(str, id, val) printf("\t%*s[%2u] : %s\n", ML_STR_FMT - 4, str, id, val)
+
+#define ml_dump_end printf("\b\t}\n\n")
+
+#endif /* _ML_COMMON_*/
diff --git a/app/test-mldev/ml_main.c b/app/test-mldev/ml_main.c
new file mode 100644
index 0000000000..d6652cd7b7
--- /dev/null
+++ b/app/test-mldev/ml_main.c
@@ -0,0 +1,118 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+
+#include <rte_debug.h>
+#include <rte_eal.h>
+#include <rte_mldev.h>
+
+#include "ml_common.h"
+#include "ml_options.h"
+#include "ml_test.h"
+
+struct ml_options opt;
+struct ml_test *test;
+
+int
+main(int argc, char **argv)
+{
+ uint16_t mldevs;
+ int ret;
+
+ ret = rte_eal_init(argc, argv);
+ if (ret < 0)
+ rte_panic("invalid EAL arguments\n");
+ argc -= ret;
+ argv += ret;
+
+ mldevs = rte_ml_dev_count();
+ if (!mldevs)
+ rte_panic("no mldev devices found\n");
+
+ /* set default values for options */
+ ml_options_default(&opt);
+
+ /* parse the command line arguments */
+ ret = ml_options_parse(&opt, argc, argv);
+ if (ret) {
+ ml_err("parsing one or more user options failed");
+ goto error;
+ }
+
+ /* get test struct from name */
+ test = ml_test_get(opt.test_name);
+ if (test == NULL) {
+ ml_err("failed to find requested test: %s", opt.test_name);
+ goto error;
+ }
+
+ if (test->ops.test_result == NULL) {
+ ml_err("%s: ops.test_result not found", opt.test_name);
+ goto error;
+ }
+
+ /* check test options */
+ if (test->ops.opt_check) {
+ if (test->ops.opt_check(&opt)) {
+ ml_err("invalid command line argument");
+ goto error;
+ }
+ }
+
+ /* check the device capability */
+ if (test->ops.cap_check) {
+ if (test->ops.cap_check(&opt) == false) {
+ ml_info("unsupported test: %s", opt.test_name);
+ ret = ML_TEST_UNSUPPORTED;
+ goto no_cap;
+ }
+ }
+
+ /* dump options */
+ if (opt.debug) {
+ if (test->ops.opt_dump)
+ test->ops.opt_dump(&opt);
+ }
+
+ /* test specific setup */
+ if (test->ops.test_setup) {
+ if (test->ops.test_setup(test, &opt)) {
+ ml_err("failed to setup test: %s", opt.test_name);
+ goto error;
+ }
+ }
+
+ /* test driver */
+ if (test->ops.test_driver)
+ test->ops.test_driver(test, &opt);
+
+ /* get result */
+ if (test->ops.test_result)
+ ret = test->ops.test_result(test, &opt);
+
+ if (test->ops.test_destroy)
+ test->ops.test_destroy(test, &opt);
+
+no_cap:
+ if (ret == ML_TEST_SUCCESS) {
+ printf("Result: " CLGRN "%s" CLNRM "\n", "Success");
+ } else if (ret == ML_TEST_FAILED) {
+ printf("Result: " CLRED "%s" CLNRM "\n", "Failed");
+ return EXIT_FAILURE;
+ } else if (ret == ML_TEST_UNSUPPORTED) {
+ printf("Result: " CLYEL "%s" CLNRM "\n", "Unsupported");
+ }
+
+ rte_eal_cleanup();
+
+ return 0;
+
+error:
+ rte_eal_cleanup();
+
+ return EXIT_FAILURE;
+}
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
new file mode 100644
index 0000000000..8fd7760e36
--- /dev/null
+++ b/app/test-mldev/ml_options.c
@@ -0,0 +1,160 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <errno.h>
+#include <getopt.h>
+#include <stdbool.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include <rte_memory.h>
+#include <rte_mldev.h>
+#include <rte_string_fns.h>
+
+#include "ml_common.h"
+#include "ml_options.h"
+#include "ml_test.h"
+#include "parser.h"
+
+typedef int (*option_parser_t)(struct ml_options *opt, const char *arg);
+
+void
+ml_options_default(struct ml_options *opt)
+{
+ memset(opt, 0, sizeof(*opt));
+ strlcpy(opt->test_name, "ml_test", ML_TEST_NAME_MAX_LEN);
+ opt->dev_id = 0;
+ opt->socket_id = SOCKET_ID_ANY;
+ opt->debug = false;
+}
+
+struct long_opt_parser {
+ const char *lgopt_name;
+ option_parser_t parser_fn;
+};
+
+static int
+ml_parse_test_name(struct ml_options *opt, const char *arg)
+{
+ strlcpy(opt->test_name, arg, ML_TEST_NAME_MAX_LEN);
+ return 0;
+}
+
+static int
+ml_parse_dev_id(struct ml_options *opt, const char *arg)
+{
+ int ret;
+
+ ret = parser_read_int16(&opt->dev_id, arg);
+
+ if (ret < 0)
+ return -EINVAL;
+
+ return ret;
+}
+
+static int
+ml_parse_socket_id(struct ml_options *opt, const char *arg)
+{
+ opt->socket_id = atoi(arg);
+
+ return 0;
+}
+
+static void
+ml_dump_test_options(const char *testname)
+{
+ RTE_SET_USED(testname);
+}
+
+static void
+print_usage(char *program)
+{
+ printf("\nusage : %s [EAL options] -- [application options]\n", program);
+ printf("application options:\n");
+ printf("\t--test : name of the test application to run\n"
+ "\t--dev_id : device id of the ML device\n"
+ "\t--socket_id : socket_id of application resources\n"
+ "\t--debug : enable debug mode\n"
+ "\t--help : print help\n");
+ printf("\n");
+ printf("available tests and test specific application options:\n");
+ ml_test_dump_names(ml_dump_test_options);
+}
+
+static struct option lgopts[] = {{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0},
+ {ML_SOCKET_ID, 1, 0, 0}, {ML_DEBUG, 0, 0, 0},
+ {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
+
+static int
+ml_opts_parse_long(int opt_idx, struct ml_options *opt)
+{
+ unsigned int i;
+
+ struct long_opt_parser parsermap[] = {
+ {ML_TEST, ml_parse_test_name},
+ {ML_DEVICE_ID, ml_parse_dev_id},
+ {ML_SOCKET_ID, ml_parse_socket_id},
+ };
+
+ for (i = 0; i < RTE_DIM(parsermap); i++) {
+ if (strncmp(lgopts[opt_idx].name, parsermap[i].lgopt_name,
+ strlen(lgopts[opt_idx].name)) == 0)
+ return parsermap[i].parser_fn(opt, optarg);
+ }
+
+ return -EINVAL;
+}
+
+int
+ml_options_parse(struct ml_options *opt, int argc, char **argv)
+{
+ int opt_idx;
+ int retval;
+ int opts;
+
+ while ((opts = getopt_long(argc, argv, "", lgopts, &opt_idx)) != EOF) {
+ switch (opts) {
+ case 0: /* parse long options */
+ if (!strcmp(lgopts[opt_idx].name, "debug")) {
+ opt->debug = true;
+ break;
+ }
+
+ if (!strcmp(lgopts[opt_idx].name, "help")) {
+ print_usage(argv[0]);
+ exit(EXIT_SUCCESS);
+ }
+
+ retval = ml_opts_parse_long(opt_idx, opt);
+ if (retval != 0)
+ return retval;
+ break;
+ default:
+ return -EINVAL;
+ }
+ }
+
+ return 0;
+}
+
+void
+ml_options_dump(struct ml_options *opt)
+{
+ struct rte_ml_dev_info dev_info;
+
+ rte_ml_dev_info_get(opt->dev_id, &dev_info);
+
+ ml_dump("driver", "%s", dev_info.driver_name);
+ ml_dump("test", "%s", opt->test_name);
+ ml_dump("dev_id", "%d", opt->dev_id);
+
+ if (opt->socket_id == SOCKET_ID_ANY)
+ ml_dump("socket_id", "%d (SOCKET_ID_ANY)", opt->socket_id);
+ else
+ ml_dump("socket_id", "%d", opt->socket_id);
+
+ ml_dump("debug", "%s", (opt->debug ? "true" : "false"));
+}
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
new file mode 100644
index 0000000000..05311a9a47
--- /dev/null
+++ b/app/test-mldev/ml_options.h
@@ -0,0 +1,31 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_OPTIONS_
+#define _ML_OPTIONS_
+
+#include <stdbool.h>
+#include <stdint.h>
+
+#define ML_TEST_NAME_MAX_LEN 32
+
+/* Options names */
+#define ML_TEST ("test")
+#define ML_DEVICE_ID ("dev_id")
+#define ML_SOCKET_ID ("socket_id")
+#define ML_DEBUG ("debug")
+#define ML_HELP ("help")
+
+struct ml_options {
+ char test_name[ML_TEST_NAME_MAX_LEN];
+ int16_t dev_id;
+ int socket_id;
+ bool debug;
+};
+
+void ml_options_default(struct ml_options *opt);
+int ml_options_parse(struct ml_options *opt, int argc, char **argv);
+void ml_options_dump(struct ml_options *opt);
+
+#endif /* _ML_OPTIONS_ */
diff --git a/app/test-mldev/ml_test.c b/app/test-mldev/ml_test.c
new file mode 100644
index 0000000000..2304712764
--- /dev/null
+++ b/app/test-mldev/ml_test.c
@@ -0,0 +1,45 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <stdio.h>
+#include <string.h>
+#include <sys/queue.h>
+
+#include "ml_test.h"
+
+static STAILQ_HEAD(, ml_test_entry) head = STAILQ_HEAD_INITIALIZER(head);
+
+void
+ml_test_register(struct ml_test_entry *entry)
+{
+ STAILQ_INSERT_TAIL(&head, entry, next);
+}
+
+struct ml_test *
+ml_test_get(const char *name)
+{
+ struct ml_test_entry *entry;
+
+ if (!name)
+ return NULL;
+
+ STAILQ_FOREACH(entry, &head, next)
+ if (!strncmp(entry->test.name, name, strlen(name)))
+ return &entry->test;
+
+ return NULL;
+}
+
+void
+ml_test_dump_names(void (*f)(const char *name))
+{
+ struct ml_test_entry *entry;
+
+ STAILQ_FOREACH(entry, &head, next)
+ {
+ if (entry->test.name)
+ printf("\t %s\n", entry->test.name);
+ f(entry->test.name);
+ }
+}
diff --git a/app/test-mldev/ml_test.h b/app/test-mldev/ml_test.h
new file mode 100644
index 0000000000..4a1430ec1b
--- /dev/null
+++ b/app/test-mldev/ml_test.h
@@ -0,0 +1,75 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_TEST_
+#define _ML_TEST_
+
+#include <stdbool.h>
+#include <string.h>
+#include <sys/queue.h>
+
+#include <rte_common.h>
+
+#include "ml_options.h"
+
+#define ML_TEST_MAX_POOL_SIZE 256
+
+enum ml_test_result {
+ ML_TEST_SUCCESS,
+ ML_TEST_FAILED,
+ ML_TEST_UNSUPPORTED,
+};
+
+struct ml_test;
+
+typedef bool (*ml_test_capability_check_t)(struct ml_options *opt);
+typedef int (*ml_test_options_check_t)(struct ml_options *opt);
+typedef void (*ml_test_options_dump_t)(struct ml_options *opt);
+typedef int (*ml_test_setup_t)(struct ml_test *test, struct ml_options *opt);
+typedef void (*ml_test_destroy_t)(struct ml_test *test, struct ml_options *opt);
+typedef int (*ml_test_driver_t)(struct ml_test *test, struct ml_options *opt);
+typedef int (*ml_test_result_t)(struct ml_test *test, struct ml_options *opt);
+
+struct ml_test_ops {
+ ml_test_capability_check_t cap_check;
+ ml_test_options_check_t opt_check;
+ ml_test_options_dump_t opt_dump;
+ ml_test_setup_t test_setup;
+ ml_test_destroy_t test_destroy;
+ ml_test_driver_t test_driver;
+ ml_test_result_t test_result;
+};
+
+struct ml_test {
+ const char *name;
+ void *test_priv;
+ struct ml_test_ops ops;
+};
+
+struct ml_test_entry {
+ struct ml_test test;
+
+ STAILQ_ENTRY(ml_test_entry) next;
+};
+
+static inline void *
+ml_test_priv(struct ml_test *test)
+{
+ return test->test_priv;
+}
+
+struct ml_test *ml_test_get(const char *name);
+void ml_test_register(struct ml_test_entry *test);
+void ml_test_dump_names(void (*f)(const char *));
+
+#define ML_TEST_REGISTER(nm) \
+ static struct ml_test_entry _ml_test_entry_##nm; \
+ RTE_INIT(ml_test_##nm) \
+ { \
+ _ml_test_entry_##nm.test.name = RTE_STR(nm); \
+ memcpy(&_ml_test_entry_##nm.test.ops, &nm, sizeof(struct ml_test_ops)); \
+ ml_test_register(&_ml_test_entry_##nm); \
+ }
+
+#endif /* _ML_TEST_ */
diff --git a/app/test-mldev/parser.c b/app/test-mldev/parser.c
new file mode 100644
index 0000000000..0b7fb63fe5
--- /dev/null
+++ b/app/test-mldev/parser.c
@@ -0,0 +1,380 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2016 Intel Corporation.
+ * Copyright (c) 2017 Cavium, Inc.
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <ctype.h>
+#include <errno.h>
+#include <stdbool.h>
+#include <stdint.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include <rte_common.h>
+
+#include "parser.h"
+
+static uint32_t
+get_hex_val(char c)
+{
+ switch (c) {
+ case '0':
+ case '1':
+ case '2':
+ case '3':
+ case '4':
+ case '5':
+ case '6':
+ case '7':
+ case '8':
+ case '9':
+ return c - '0';
+ case 'A':
+ case 'B':
+ case 'C':
+ case 'D':
+ case 'E':
+ case 'F':
+ return c - 'A' + 10;
+ case 'a':
+ case 'b':
+ case 'c':
+ case 'd':
+ case 'e':
+ case 'f':
+ return c - 'a' + 10;
+ default:
+ return 0;
+ }
+}
+
+int
+parser_read_arg_bool(const char *p)
+{
+ p = skip_white_spaces(p);
+ int result = -EINVAL;
+
+ if (((p[0] == 'y') && (p[1] == 'e') && (p[2] == 's')) ||
+ ((p[0] == 'Y') && (p[1] == 'E') && (p[2] == 'S'))) {
+ p += 3;
+ result = 1;
+ }
+
+ if (((p[0] == 'o') && (p[1] == 'n')) || ((p[0] == 'O') && (p[1] == 'N'))) {
+ p += 2;
+ result = 1;
+ }
+
+ if (((p[0] == 'n') && (p[1] == 'o')) || ((p[0] == 'N') && (p[1] == 'O'))) {
+ p += 2;
+ result = 0;
+ }
+
+ if (((p[0] == 'o') && (p[1] == 'f') && (p[2] == 'f')) ||
+ ((p[0] == 'O') && (p[1] == 'F') && (p[2] == 'F'))) {
+ p += 3;
+ result = 0;
+ }
+
+ p = skip_white_spaces(p);
+
+ if (p[0] != '\0')
+ return -EINVAL;
+
+ return result;
+}
+
+int
+parser_read_uint64(uint64_t *value, const char *p)
+{
+ char *next;
+ uint64_t val;
+
+ p = skip_white_spaces(p);
+ if (!isdigit(*p))
+ return -EINVAL;
+
+ val = strtoul(p, &next, 10);
+ if (p == next)
+ return -EINVAL;
+
+ p = next;
+ switch (*p) {
+ case 'T':
+ val *= 1024ULL;
+ /* fall through */
+ case 'G':
+ val *= 1024ULL;
+ /* fall through */
+ case 'M':
+ val *= 1024ULL;
+ /* fall through */
+ case 'k':
+ case 'K':
+ val *= 1024ULL;
+ p++;
+ break;
+ }
+
+ p = skip_white_spaces(p);
+ if (*p != '\0')
+ return -EINVAL;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_int32(int32_t *value, const char *p)
+{
+ char *next;
+ int32_t val;
+
+ p = skip_white_spaces(p);
+ if (!isdigit(*p))
+ return -EINVAL;
+
+ val = strtol(p, &next, 10);
+ if (p == next)
+ return -EINVAL;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_int16(int16_t *value, const char *p)
+{
+ char *next;
+ int16_t val;
+
+ p = skip_white_spaces(p);
+ if (!isdigit(*p))
+ return -EINVAL;
+
+ val = strtol(p, &next, 10);
+ if (p == next)
+ return -EINVAL;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint64_hex(uint64_t *value, const char *p)
+{
+ char *next;
+ uint64_t val;
+
+ p = skip_white_spaces(p);
+
+ val = strtoul(p, &next, 16);
+ if (p == next)
+ return -EINVAL;
+
+ p = skip_white_spaces(next);
+ if (*p != '\0')
+ return -EINVAL;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint32(uint32_t *value, const char *p)
+{
+ uint64_t val = 0;
+ int ret = parser_read_uint64(&val, p);
+
+ if (ret < 0)
+ return ret;
+
+ if (val > UINT32_MAX)
+ return -ERANGE;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint32_hex(uint32_t *value, const char *p)
+{
+ uint64_t val = 0;
+ int ret = parser_read_uint64_hex(&val, p);
+
+ if (ret < 0)
+ return ret;
+
+ if (val > UINT32_MAX)
+ return -ERANGE;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint16(uint16_t *value, const char *p)
+{
+ uint64_t val = 0;
+ int ret = parser_read_uint64(&val, p);
+
+ if (ret < 0)
+ return ret;
+
+ if (val > UINT16_MAX)
+ return -ERANGE;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint16_hex(uint16_t *value, const char *p)
+{
+ uint64_t val = 0;
+ int ret = parser_read_uint64_hex(&val, p);
+
+ if (ret < 0)
+ return ret;
+
+ if (val > UINT16_MAX)
+ return -ERANGE;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint8(uint8_t *value, const char *p)
+{
+ uint64_t val = 0;
+ int ret = parser_read_uint64(&val, p);
+
+ if (ret < 0)
+ return ret;
+
+ if (val > UINT8_MAX)
+ return -ERANGE;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint8_hex(uint8_t *value, const char *p)
+{
+ uint64_t val = 0;
+ int ret = parser_read_uint64_hex(&val, p);
+
+ if (ret < 0)
+ return ret;
+
+ if (val > UINT8_MAX)
+ return -ERANGE;
+
+ *value = val;
+ return 0;
+}
+
+int
+parse_tokenize_string(char *string, char *tokens[], uint32_t *n_tokens)
+{
+ uint32_t i;
+
+ if ((string == NULL) || (tokens == NULL) || (*n_tokens < 1))
+ return -EINVAL;
+
+ for (i = 0; i < *n_tokens; i++) {
+ tokens[i] = strtok_r(string, PARSE_DELIMITER, &string);
+ if (tokens[i] == NULL)
+ break;
+ }
+
+ if ((i == *n_tokens) && (strtok_r(string, PARSE_DELIMITER, &string) != NULL))
+ return -E2BIG;
+
+ *n_tokens = i;
+ return 0;
+}
+
+int
+parse_hex_string(char *src, uint8_t *dst, uint32_t *size)
+{
+ char *c;
+ uint32_t len, i;
+
+ /* Check input parameters */
+ if ((src == NULL) || (dst == NULL) || (size == NULL) || (*size == 0))
+ return -1;
+
+ len = strlen(src);
+ if (((len & 3) != 0) || (len > (*size) * 2))
+ return -1;
+ *size = len / 2;
+
+ for (c = src; *c != 0; c++) {
+ if ((((*c) >= '0') && ((*c) <= '9')) || (((*c) >= 'A') && ((*c) <= 'F')) ||
+ (((*c) >= 'a') && ((*c) <= 'f')))
+ continue;
+
+ return -1;
+ }
+
+ /* Convert chars to bytes */
+ for (i = 0; i < *size; i++)
+ dst[i] = get_hex_val(src[2 * i]) * 16 + get_hex_val(src[2 * i + 1]);
+
+ return 0;
+}
+
+int
+parse_lcores_list(bool lcores[], int lcores_num, const char *corelist)
+{
+ int i, idx = 0;
+ int min, max;
+ char *end = NULL;
+
+ if (corelist == NULL)
+ return -1;
+ while (isblank(*corelist))
+ corelist++;
+ i = strlen(corelist);
+ while ((i > 0) && isblank(corelist[i - 1]))
+ i--;
+
+ /* Get list of lcores */
+ min = RTE_MAX_LCORE;
+ do {
+ while (isblank(*corelist))
+ corelist++;
+ if (*corelist == '\0')
+ return -1;
+ idx = strtoul(corelist, &end, 10);
+ if (idx < 0 || idx > lcores_num)
+ return -1;
+
+ if (end == NULL)
+ return -1;
+ while (isblank(*end))
+ end++;
+ if (*end == '-') {
+ min = idx;
+ } else if ((*end == ',') || (*end == '\0')) {
+ max = idx;
+ if (min == RTE_MAX_LCORE)
+ min = idx;
+ for (idx = min; idx <= max; idx++) {
+ if (lcores[idx] == 1)
+ return -E2BIG;
+ lcores[idx] = 1;
+ }
+
+ min = RTE_MAX_LCORE;
+ } else
+ return -1;
+ corelist = end + 1;
+ } while (*end != '\0');
+
+ return 0;
+}
diff --git a/app/test-mldev/parser.h b/app/test-mldev/parser.h
new file mode 100644
index 0000000000..f0d5e79e4b
--- /dev/null
+++ b/app/test-mldev/parser.h
@@ -0,0 +1,55 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2010-2016 Intel Corporation.
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef __INCLUDE_PARSER_H__
+#define __INCLUDE_PARSER_H__
+
+#include <ctype.h>
+#include <stdint.h>
+#include <stdlib.h>
+
+#define PARSE_DELIMITER " \f\n\r\t\v"
+
+#define skip_white_spaces(pos) \
+ ({ \
+ __typeof__(pos) _p = (pos); \
+ for (; isspace(*_p); _p++) \
+ ; \
+ _p; \
+ })
+
+static inline size_t
+skip_digits(const char *src)
+{
+ size_t i;
+
+ for (i = 0; isdigit(src[i]); i++)
+ ;
+
+ return i;
+}
+
+int parser_read_arg_bool(const char *p);
+
+int parser_read_uint64(uint64_t *value, const char *p);
+int parser_read_uint32(uint32_t *value, const char *p);
+int parser_read_uint16(uint16_t *value, const char *p);
+int parser_read_uint8(uint8_t *value, const char *p);
+
+int parser_read_uint64_hex(uint64_t *value, const char *p);
+int parser_read_uint32_hex(uint32_t *value, const char *p);
+int parser_read_uint16_hex(uint16_t *value, const char *p);
+int parser_read_uint8_hex(uint8_t *value, const char *p);
+
+int parser_read_int32(int32_t *value, const char *p);
+int parser_read_int16(int16_t *value, const char *p);
+
+int parse_hex_string(char *src, uint8_t *dst, uint32_t *size);
+
+int parse_tokenize_string(char *string, char *tokens[], uint32_t *n_tokens);
+
+int parse_lcores_list(bool lcores[], int lcores_num, const char *corelist);
+
+#endif /* __INCLUDE_PARSER_H__ */
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v2 01/12] app/mldev: implement test framework for mldev
2022-11-29 7:07 ` [PATCH v1 01/12] app/mldev: implement test framework for mldev Srikanth Yalavarthi
@ 2022-11-29 8:20 ` Srikanth Yalavarthi
2022-11-29 8:20 ` [PATCH v2 02/12] app/mldev: add common test functions Srikanth Yalavarthi
` (11 more replies)
0 siblings, 12 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2022-11-29 8:20 UTC (permalink / raw)
To: Thomas Monjalon, Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj
Implemented framework for mldev test application. New test cases
can be added using the framework. Support is also enabled to add
options specific to the test cases. User can launch the tests by
specifying the name of test as part of launch arguments.
Code to parse command line arguments is imported from
test-eventdev, with support to parse additional data types.
Common arguments supported include:
test : name of the test application to run
dev_id : device id of the ML device
socket_id : socket_id of application resources
debug : enable debugging
help : print help
Sample launch command:
./dpdk-test-mldev -- --test <testname> --dev_id <dev_id> \
--socket_id <socket_id>
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
Depends-on: series-25753 ("mldev: introduce machine learning device library")
MAINTAINERS | 1 +
app/meson.build | 1 +
app/test-mldev/meson.build | 17 ++
app/test-mldev/ml_common.h | 29 +++
app/test-mldev/ml_main.c | 118 +++++++++++
app/test-mldev/ml_options.c | 160 +++++++++++++++
app/test-mldev/ml_options.h | 31 +++
app/test-mldev/ml_test.c | 45 +++++
app/test-mldev/ml_test.h | 75 +++++++
app/test-mldev/parser.c | 380 ++++++++++++++++++++++++++++++++++++
app/test-mldev/parser.h | 55 ++++++
11 files changed, 912 insertions(+)
create mode 100644 app/test-mldev/meson.build
create mode 100644 app/test-mldev/ml_common.h
create mode 100644 app/test-mldev/ml_main.c
create mode 100644 app/test-mldev/ml_options.c
create mode 100644 app/test-mldev/ml_options.h
create mode 100644 app/test-mldev/ml_test.c
create mode 100644 app/test-mldev/ml_test.h
create mode 100644 app/test-mldev/parser.c
create mode 100644 app/test-mldev/parser.h
diff --git a/MAINTAINERS b/MAINTAINERS
index 0c3e6d28e9..1edea42fad 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -538,6 +538,7 @@ F: doc/guides/prog_guide/rawdev.rst
ML device API - EXPERIMENTAL
M: Srikanth Yalavarthi <syalavarthi@marvell.com>
F: lib/mldev/
+F: app/test-mldev/
F: doc/guides/prog_guide/mldev.rst
diff --git a/app/meson.build b/app/meson.build
index e32ea4bd5c..74d2420f67 100644
--- a/app/meson.build
+++ b/app/meson.build
@@ -23,6 +23,7 @@ apps = [
'test-fib',
'test-flow-perf',
'test-gpudev',
+ 'test-mldev',
'test-pipeline',
'test-pmd',
'test-regex',
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
new file mode 100644
index 0000000000..8ca2e1a1c1
--- /dev/null
+++ b/app/test-mldev/meson.build
@@ -0,0 +1,17 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright (c) 2022 Marvell.
+
+if is_windows
+ build = false
+ reason = 'not supported on Windows'
+ subdir_done()
+endif
+
+sources = files(
+ 'ml_main.c',
+ 'ml_options.c',
+ 'ml_test.c',
+ 'parser.c',
+)
+
+deps += ['mldev']
diff --git a/app/test-mldev/ml_common.h b/app/test-mldev/ml_common.h
new file mode 100644
index 0000000000..065180b619
--- /dev/null
+++ b/app/test-mldev/ml_common.h
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_COMMON_
+#define _ML_COMMON_
+
+#include <stdio.h>
+
+#define CLNRM "\x1b[0m"
+#define CLRED "\x1b[31m"
+#define CLGRN "\x1b[32m"
+#define CLYEL "\x1b[33m"
+
+#define ML_STR_FMT 20
+
+#define ml_err(fmt, args...) fprintf(stderr, CLRED "error: %s() " fmt CLNRM "\n", __func__, ##args)
+
+#define ml_info(fmt, args...) fprintf(stdout, CLYEL "" fmt CLNRM "\n", ##args)
+
+#define ml_dump(str, fmt, val...) printf("\t%-*s : " fmt "\n", ML_STR_FMT, str, ##val)
+
+#define ml_dump_begin(str) printf("\t%-*s :\n\t{\n", ML_STR_FMT, str)
+
+#define ml_dump_list(str, id, val) printf("\t%*s[%2u] : %s\n", ML_STR_FMT - 4, str, id, val)
+
+#define ml_dump_end printf("\b\t}\n\n")
+
+#endif /* _ML_COMMON_*/
diff --git a/app/test-mldev/ml_main.c b/app/test-mldev/ml_main.c
new file mode 100644
index 0000000000..d6652cd7b7
--- /dev/null
+++ b/app/test-mldev/ml_main.c
@@ -0,0 +1,118 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+
+#include <rte_debug.h>
+#include <rte_eal.h>
+#include <rte_mldev.h>
+
+#include "ml_common.h"
+#include "ml_options.h"
+#include "ml_test.h"
+
+struct ml_options opt;
+struct ml_test *test;
+
+int
+main(int argc, char **argv)
+{
+ uint16_t mldevs;
+ int ret;
+
+ ret = rte_eal_init(argc, argv);
+ if (ret < 0)
+ rte_panic("invalid EAL arguments\n");
+ argc -= ret;
+ argv += ret;
+
+ mldevs = rte_ml_dev_count();
+ if (!mldevs)
+ rte_panic("no mldev devices found\n");
+
+ /* set default values for options */
+ ml_options_default(&opt);
+
+ /* parse the command line arguments */
+ ret = ml_options_parse(&opt, argc, argv);
+ if (ret) {
+ ml_err("parsing one or more user options failed");
+ goto error;
+ }
+
+ /* get test struct from name */
+ test = ml_test_get(opt.test_name);
+ if (test == NULL) {
+ ml_err("failed to find requested test: %s", opt.test_name);
+ goto error;
+ }
+
+ if (test->ops.test_result == NULL) {
+ ml_err("%s: ops.test_result not found", opt.test_name);
+ goto error;
+ }
+
+ /* check test options */
+ if (test->ops.opt_check) {
+ if (test->ops.opt_check(&opt)) {
+ ml_err("invalid command line argument");
+ goto error;
+ }
+ }
+
+ /* check the device capability */
+ if (test->ops.cap_check) {
+ if (test->ops.cap_check(&opt) == false) {
+ ml_info("unsupported test: %s", opt.test_name);
+ ret = ML_TEST_UNSUPPORTED;
+ goto no_cap;
+ }
+ }
+
+ /* dump options */
+ if (opt.debug) {
+ if (test->ops.opt_dump)
+ test->ops.opt_dump(&opt);
+ }
+
+ /* test specific setup */
+ if (test->ops.test_setup) {
+ if (test->ops.test_setup(test, &opt)) {
+ ml_err("failed to setup test: %s", opt.test_name);
+ goto error;
+ }
+ }
+
+ /* test driver */
+ if (test->ops.test_driver)
+ test->ops.test_driver(test, &opt);
+
+ /* get result */
+ if (test->ops.test_result)
+ ret = test->ops.test_result(test, &opt);
+
+ if (test->ops.test_destroy)
+ test->ops.test_destroy(test, &opt);
+
+no_cap:
+ if (ret == ML_TEST_SUCCESS) {
+ printf("Result: " CLGRN "%s" CLNRM "\n", "Success");
+ } else if (ret == ML_TEST_FAILED) {
+ printf("Result: " CLRED "%s" CLNRM "\n", "Failed");
+ return EXIT_FAILURE;
+ } else if (ret == ML_TEST_UNSUPPORTED) {
+ printf("Result: " CLYEL "%s" CLNRM "\n", "Unsupported");
+ }
+
+ rte_eal_cleanup();
+
+ return 0;
+
+error:
+ rte_eal_cleanup();
+
+ return EXIT_FAILURE;
+}
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
new file mode 100644
index 0000000000..8fd7760e36
--- /dev/null
+++ b/app/test-mldev/ml_options.c
@@ -0,0 +1,160 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <errno.h>
+#include <getopt.h>
+#include <stdbool.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include <rte_memory.h>
+#include <rte_mldev.h>
+#include <rte_string_fns.h>
+
+#include "ml_common.h"
+#include "ml_options.h"
+#include "ml_test.h"
+#include "parser.h"
+
+typedef int (*option_parser_t)(struct ml_options *opt, const char *arg);
+
+void
+ml_options_default(struct ml_options *opt)
+{
+ memset(opt, 0, sizeof(*opt));
+ strlcpy(opt->test_name, "ml_test", ML_TEST_NAME_MAX_LEN);
+ opt->dev_id = 0;
+ opt->socket_id = SOCKET_ID_ANY;
+ opt->debug = false;
+}
+
+struct long_opt_parser {
+ const char *lgopt_name;
+ option_parser_t parser_fn;
+};
+
+static int
+ml_parse_test_name(struct ml_options *opt, const char *arg)
+{
+ strlcpy(opt->test_name, arg, ML_TEST_NAME_MAX_LEN);
+ return 0;
+}
+
+static int
+ml_parse_dev_id(struct ml_options *opt, const char *arg)
+{
+ int ret;
+
+ ret = parser_read_int16(&opt->dev_id, arg);
+
+ if (ret < 0)
+ return -EINVAL;
+
+ return ret;
+}
+
+static int
+ml_parse_socket_id(struct ml_options *opt, const char *arg)
+{
+ opt->socket_id = atoi(arg);
+
+ return 0;
+}
+
+static void
+ml_dump_test_options(const char *testname)
+{
+ RTE_SET_USED(testname);
+}
+
+static void
+print_usage(char *program)
+{
+ printf("\nusage : %s [EAL options] -- [application options]\n", program);
+ printf("application options:\n");
+ printf("\t--test : name of the test application to run\n"
+ "\t--dev_id : device id of the ML device\n"
+ "\t--socket_id : socket_id of application resources\n"
+ "\t--debug : enable debug mode\n"
+ "\t--help : print help\n");
+ printf("\n");
+ printf("available tests and test specific application options:\n");
+ ml_test_dump_names(ml_dump_test_options);
+}
+
+static struct option lgopts[] = {{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0},
+ {ML_SOCKET_ID, 1, 0, 0}, {ML_DEBUG, 0, 0, 0},
+ {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
+
+static int
+ml_opts_parse_long(int opt_idx, struct ml_options *opt)
+{
+ unsigned int i;
+
+ struct long_opt_parser parsermap[] = {
+ {ML_TEST, ml_parse_test_name},
+ {ML_DEVICE_ID, ml_parse_dev_id},
+ {ML_SOCKET_ID, ml_parse_socket_id},
+ };
+
+ for (i = 0; i < RTE_DIM(parsermap); i++) {
+ if (strncmp(lgopts[opt_idx].name, parsermap[i].lgopt_name,
+ strlen(lgopts[opt_idx].name)) == 0)
+ return parsermap[i].parser_fn(opt, optarg);
+ }
+
+ return -EINVAL;
+}
+
+int
+ml_options_parse(struct ml_options *opt, int argc, char **argv)
+{
+ int opt_idx;
+ int retval;
+ int opts;
+
+ while ((opts = getopt_long(argc, argv, "", lgopts, &opt_idx)) != EOF) {
+ switch (opts) {
+ case 0: /* parse long options */
+ if (!strcmp(lgopts[opt_idx].name, "debug")) {
+ opt->debug = true;
+ break;
+ }
+
+ if (!strcmp(lgopts[opt_idx].name, "help")) {
+ print_usage(argv[0]);
+ exit(EXIT_SUCCESS);
+ }
+
+ retval = ml_opts_parse_long(opt_idx, opt);
+ if (retval != 0)
+ return retval;
+ break;
+ default:
+ return -EINVAL;
+ }
+ }
+
+ return 0;
+}
+
+void
+ml_options_dump(struct ml_options *opt)
+{
+ struct rte_ml_dev_info dev_info;
+
+ rte_ml_dev_info_get(opt->dev_id, &dev_info);
+
+ ml_dump("driver", "%s", dev_info.driver_name);
+ ml_dump("test", "%s", opt->test_name);
+ ml_dump("dev_id", "%d", opt->dev_id);
+
+ if (opt->socket_id == SOCKET_ID_ANY)
+ ml_dump("socket_id", "%d (SOCKET_ID_ANY)", opt->socket_id);
+ else
+ ml_dump("socket_id", "%d", opt->socket_id);
+
+ ml_dump("debug", "%s", (opt->debug ? "true" : "false"));
+}
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
new file mode 100644
index 0000000000..05311a9a47
--- /dev/null
+++ b/app/test-mldev/ml_options.h
@@ -0,0 +1,31 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_OPTIONS_
+#define _ML_OPTIONS_
+
+#include <stdbool.h>
+#include <stdint.h>
+
+#define ML_TEST_NAME_MAX_LEN 32
+
+/* Options names */
+#define ML_TEST ("test")
+#define ML_DEVICE_ID ("dev_id")
+#define ML_SOCKET_ID ("socket_id")
+#define ML_DEBUG ("debug")
+#define ML_HELP ("help")
+
+struct ml_options {
+ char test_name[ML_TEST_NAME_MAX_LEN];
+ int16_t dev_id;
+ int socket_id;
+ bool debug;
+};
+
+void ml_options_default(struct ml_options *opt);
+int ml_options_parse(struct ml_options *opt, int argc, char **argv);
+void ml_options_dump(struct ml_options *opt);
+
+#endif /* _ML_OPTIONS_ */
diff --git a/app/test-mldev/ml_test.c b/app/test-mldev/ml_test.c
new file mode 100644
index 0000000000..2304712764
--- /dev/null
+++ b/app/test-mldev/ml_test.c
@@ -0,0 +1,45 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <stdio.h>
+#include <string.h>
+#include <sys/queue.h>
+
+#include "ml_test.h"
+
+static STAILQ_HEAD(, ml_test_entry) head = STAILQ_HEAD_INITIALIZER(head);
+
+void
+ml_test_register(struct ml_test_entry *entry)
+{
+ STAILQ_INSERT_TAIL(&head, entry, next);
+}
+
+struct ml_test *
+ml_test_get(const char *name)
+{
+ struct ml_test_entry *entry;
+
+ if (!name)
+ return NULL;
+
+ STAILQ_FOREACH(entry, &head, next)
+ if (!strncmp(entry->test.name, name, strlen(name)))
+ return &entry->test;
+
+ return NULL;
+}
+
+void
+ml_test_dump_names(void (*f)(const char *name))
+{
+ struct ml_test_entry *entry;
+
+ STAILQ_FOREACH(entry, &head, next)
+ {
+ if (entry->test.name)
+ printf("\t %s\n", entry->test.name);
+ f(entry->test.name);
+ }
+}
diff --git a/app/test-mldev/ml_test.h b/app/test-mldev/ml_test.h
new file mode 100644
index 0000000000..4a1430ec1b
--- /dev/null
+++ b/app/test-mldev/ml_test.h
@@ -0,0 +1,75 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_TEST_
+#define _ML_TEST_
+
+#include <stdbool.h>
+#include <string.h>
+#include <sys/queue.h>
+
+#include <rte_common.h>
+
+#include "ml_options.h"
+
+#define ML_TEST_MAX_POOL_SIZE 256
+
+enum ml_test_result {
+ ML_TEST_SUCCESS,
+ ML_TEST_FAILED,
+ ML_TEST_UNSUPPORTED,
+};
+
+struct ml_test;
+
+typedef bool (*ml_test_capability_check_t)(struct ml_options *opt);
+typedef int (*ml_test_options_check_t)(struct ml_options *opt);
+typedef void (*ml_test_options_dump_t)(struct ml_options *opt);
+typedef int (*ml_test_setup_t)(struct ml_test *test, struct ml_options *opt);
+typedef void (*ml_test_destroy_t)(struct ml_test *test, struct ml_options *opt);
+typedef int (*ml_test_driver_t)(struct ml_test *test, struct ml_options *opt);
+typedef int (*ml_test_result_t)(struct ml_test *test, struct ml_options *opt);
+
+struct ml_test_ops {
+ ml_test_capability_check_t cap_check;
+ ml_test_options_check_t opt_check;
+ ml_test_options_dump_t opt_dump;
+ ml_test_setup_t test_setup;
+ ml_test_destroy_t test_destroy;
+ ml_test_driver_t test_driver;
+ ml_test_result_t test_result;
+};
+
+struct ml_test {
+ const char *name;
+ void *test_priv;
+ struct ml_test_ops ops;
+};
+
+struct ml_test_entry {
+ struct ml_test test;
+
+ STAILQ_ENTRY(ml_test_entry) next;
+};
+
+static inline void *
+ml_test_priv(struct ml_test *test)
+{
+ return test->test_priv;
+}
+
+struct ml_test *ml_test_get(const char *name);
+void ml_test_register(struct ml_test_entry *test);
+void ml_test_dump_names(void (*f)(const char *));
+
+#define ML_TEST_REGISTER(nm) \
+ static struct ml_test_entry _ml_test_entry_##nm; \
+ RTE_INIT(ml_test_##nm) \
+ { \
+ _ml_test_entry_##nm.test.name = RTE_STR(nm); \
+ memcpy(&_ml_test_entry_##nm.test.ops, &nm, sizeof(struct ml_test_ops)); \
+ ml_test_register(&_ml_test_entry_##nm); \
+ }
+
+#endif /* _ML_TEST_ */
diff --git a/app/test-mldev/parser.c b/app/test-mldev/parser.c
new file mode 100644
index 0000000000..0b7fb63fe5
--- /dev/null
+++ b/app/test-mldev/parser.c
@@ -0,0 +1,380 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2016 Intel Corporation.
+ * Copyright (c) 2017 Cavium, Inc.
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <ctype.h>
+#include <errno.h>
+#include <stdbool.h>
+#include <stdint.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include <rte_common.h>
+
+#include "parser.h"
+
+static uint32_t
+get_hex_val(char c)
+{
+ switch (c) {
+ case '0':
+ case '1':
+ case '2':
+ case '3':
+ case '4':
+ case '5':
+ case '6':
+ case '7':
+ case '8':
+ case '9':
+ return c - '0';
+ case 'A':
+ case 'B':
+ case 'C':
+ case 'D':
+ case 'E':
+ case 'F':
+ return c - 'A' + 10;
+ case 'a':
+ case 'b':
+ case 'c':
+ case 'd':
+ case 'e':
+ case 'f':
+ return c - 'a' + 10;
+ default:
+ return 0;
+ }
+}
+
+int
+parser_read_arg_bool(const char *p)
+{
+ p = skip_white_spaces(p);
+ int result = -EINVAL;
+
+ if (((p[0] == 'y') && (p[1] == 'e') && (p[2] == 's')) ||
+ ((p[0] == 'Y') && (p[1] == 'E') && (p[2] == 'S'))) {
+ p += 3;
+ result = 1;
+ }
+
+ if (((p[0] == 'o') && (p[1] == 'n')) || ((p[0] == 'O') && (p[1] == 'N'))) {
+ p += 2;
+ result = 1;
+ }
+
+ if (((p[0] == 'n') && (p[1] == 'o')) || ((p[0] == 'N') && (p[1] == 'O'))) {
+ p += 2;
+ result = 0;
+ }
+
+ if (((p[0] == 'o') && (p[1] == 'f') && (p[2] == 'f')) ||
+ ((p[0] == 'O') && (p[1] == 'F') && (p[2] == 'F'))) {
+ p += 3;
+ result = 0;
+ }
+
+ p = skip_white_spaces(p);
+
+ if (p[0] != '\0')
+ return -EINVAL;
+
+ return result;
+}
+
+int
+parser_read_uint64(uint64_t *value, const char *p)
+{
+ char *next;
+ uint64_t val;
+
+ p = skip_white_spaces(p);
+ if (!isdigit(*p))
+ return -EINVAL;
+
+ val = strtoul(p, &next, 10);
+ if (p == next)
+ return -EINVAL;
+
+ p = next;
+ switch (*p) {
+ case 'T':
+ val *= 1024ULL;
+ /* fall through */
+ case 'G':
+ val *= 1024ULL;
+ /* fall through */
+ case 'M':
+ val *= 1024ULL;
+ /* fall through */
+ case 'k':
+ case 'K':
+ val *= 1024ULL;
+ p++;
+ break;
+ }
+
+ p = skip_white_spaces(p);
+ if (*p != '\0')
+ return -EINVAL;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_int32(int32_t *value, const char *p)
+{
+ char *next;
+ int32_t val;
+
+ p = skip_white_spaces(p);
+ if (!isdigit(*p))
+ return -EINVAL;
+
+ val = strtol(p, &next, 10);
+ if (p == next)
+ return -EINVAL;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_int16(int16_t *value, const char *p)
+{
+ char *next;
+ int16_t val;
+
+ p = skip_white_spaces(p);
+ if (!isdigit(*p))
+ return -EINVAL;
+
+ val = strtol(p, &next, 10);
+ if (p == next)
+ return -EINVAL;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint64_hex(uint64_t *value, const char *p)
+{
+ char *next;
+ uint64_t val;
+
+ p = skip_white_spaces(p);
+
+ val = strtoul(p, &next, 16);
+ if (p == next)
+ return -EINVAL;
+
+ p = skip_white_spaces(next);
+ if (*p != '\0')
+ return -EINVAL;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint32(uint32_t *value, const char *p)
+{
+ uint64_t val = 0;
+ int ret = parser_read_uint64(&val, p);
+
+ if (ret < 0)
+ return ret;
+
+ if (val > UINT32_MAX)
+ return -ERANGE;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint32_hex(uint32_t *value, const char *p)
+{
+ uint64_t val = 0;
+ int ret = parser_read_uint64_hex(&val, p);
+
+ if (ret < 0)
+ return ret;
+
+ if (val > UINT32_MAX)
+ return -ERANGE;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint16(uint16_t *value, const char *p)
+{
+ uint64_t val = 0;
+ int ret = parser_read_uint64(&val, p);
+
+ if (ret < 0)
+ return ret;
+
+ if (val > UINT16_MAX)
+ return -ERANGE;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint16_hex(uint16_t *value, const char *p)
+{
+ uint64_t val = 0;
+ int ret = parser_read_uint64_hex(&val, p);
+
+ if (ret < 0)
+ return ret;
+
+ if (val > UINT16_MAX)
+ return -ERANGE;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint8(uint8_t *value, const char *p)
+{
+ uint64_t val = 0;
+ int ret = parser_read_uint64(&val, p);
+
+ if (ret < 0)
+ return ret;
+
+ if (val > UINT8_MAX)
+ return -ERANGE;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint8_hex(uint8_t *value, const char *p)
+{
+ uint64_t val = 0;
+ int ret = parser_read_uint64_hex(&val, p);
+
+ if (ret < 0)
+ return ret;
+
+ if (val > UINT8_MAX)
+ return -ERANGE;
+
+ *value = val;
+ return 0;
+}
+
+int
+parse_tokenize_string(char *string, char *tokens[], uint32_t *n_tokens)
+{
+ uint32_t i;
+
+ if ((string == NULL) || (tokens == NULL) || (*n_tokens < 1))
+ return -EINVAL;
+
+ for (i = 0; i < *n_tokens; i++) {
+ tokens[i] = strtok_r(string, PARSE_DELIMITER, &string);
+ if (tokens[i] == NULL)
+ break;
+ }
+
+ if ((i == *n_tokens) && (strtok_r(string, PARSE_DELIMITER, &string) != NULL))
+ return -E2BIG;
+
+ *n_tokens = i;
+ return 0;
+}
+
+int
+parse_hex_string(char *src, uint8_t *dst, uint32_t *size)
+{
+ char *c;
+ uint32_t len, i;
+
+ /* Check input parameters */
+ if ((src == NULL) || (dst == NULL) || (size == NULL) || (*size == 0))
+ return -1;
+
+ len = strlen(src);
+ if (((len & 3) != 0) || (len > (*size) * 2))
+ return -1;
+ *size = len / 2;
+
+ for (c = src; *c != 0; c++) {
+ if ((((*c) >= '0') && ((*c) <= '9')) || (((*c) >= 'A') && ((*c) <= 'F')) ||
+ (((*c) >= 'a') && ((*c) <= 'f')))
+ continue;
+
+ return -1;
+ }
+
+ /* Convert chars to bytes */
+ for (i = 0; i < *size; i++)
+ dst[i] = get_hex_val(src[2 * i]) * 16 + get_hex_val(src[2 * i + 1]);
+
+ return 0;
+}
+
+int
+parse_lcores_list(bool lcores[], int lcores_num, const char *corelist)
+{
+ int i, idx = 0;
+ int min, max;
+ char *end = NULL;
+
+ if (corelist == NULL)
+ return -1;
+ while (isblank(*corelist))
+ corelist++;
+ i = strlen(corelist);
+ while ((i > 0) && isblank(corelist[i - 1]))
+ i--;
+
+ /* Get list of lcores */
+ min = RTE_MAX_LCORE;
+ do {
+ while (isblank(*corelist))
+ corelist++;
+ if (*corelist == '\0')
+ return -1;
+ idx = strtoul(corelist, &end, 10);
+ if (idx < 0 || idx > lcores_num)
+ return -1;
+
+ if (end == NULL)
+ return -1;
+ while (isblank(*end))
+ end++;
+ if (*end == '-') {
+ min = idx;
+ } else if ((*end == ',') || (*end == '\0')) {
+ max = idx;
+ if (min == RTE_MAX_LCORE)
+ min = idx;
+ for (idx = min; idx <= max; idx++) {
+ if (lcores[idx] == 1)
+ return -E2BIG;
+ lcores[idx] = 1;
+ }
+
+ min = RTE_MAX_LCORE;
+ } else
+ return -1;
+ corelist = end + 1;
+ } while (*end != '\0');
+
+ return 0;
+}
diff --git a/app/test-mldev/parser.h b/app/test-mldev/parser.h
new file mode 100644
index 0000000000..f0d5e79e4b
--- /dev/null
+++ b/app/test-mldev/parser.h
@@ -0,0 +1,55 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2010-2016 Intel Corporation.
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef __INCLUDE_PARSER_H__
+#define __INCLUDE_PARSER_H__
+
+#include <ctype.h>
+#include <stdint.h>
+#include <stdlib.h>
+
+#define PARSE_DELIMITER " \f\n\r\t\v"
+
+#define skip_white_spaces(pos) \
+ ({ \
+ __typeof__(pos) _p = (pos); \
+ for (; isspace(*_p); _p++) \
+ ; \
+ _p; \
+ })
+
+static inline size_t
+skip_digits(const char *src)
+{
+ size_t i;
+
+ for (i = 0; isdigit(src[i]); i++)
+ ;
+
+ return i;
+}
+
+int parser_read_arg_bool(const char *p);
+
+int parser_read_uint64(uint64_t *value, const char *p);
+int parser_read_uint32(uint32_t *value, const char *p);
+int parser_read_uint16(uint16_t *value, const char *p);
+int parser_read_uint8(uint8_t *value, const char *p);
+
+int parser_read_uint64_hex(uint64_t *value, const char *p);
+int parser_read_uint32_hex(uint32_t *value, const char *p);
+int parser_read_uint16_hex(uint16_t *value, const char *p);
+int parser_read_uint8_hex(uint8_t *value, const char *p);
+
+int parser_read_int32(int32_t *value, const char *p);
+int parser_read_int16(int16_t *value, const char *p);
+
+int parse_hex_string(char *src, uint8_t *dst, uint32_t *size);
+
+int parse_tokenize_string(char *string, char *tokens[], uint32_t *n_tokens);
+
+int parse_lcores_list(bool lcores[], int lcores_num, const char *corelist);
+
+#endif /* __INCLUDE_PARSER_H__ */
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v2 02/12] app/mldev: add common test functions
2022-11-29 8:20 ` [PATCH v2 " Srikanth Yalavarthi
@ 2022-11-29 8:20 ` Srikanth Yalavarthi
2022-11-29 8:21 ` [PATCH v2 03/12] app/mldev: add test case to validate device ops Srikanth Yalavarthi
` (10 subsequent siblings)
11 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2022-11-29 8:20 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj
Added common functions used by all tests. Common code
includes functions to check capabilities, options, and
handle ML devices.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/meson.build | 1 +
app/test-mldev/test_common.c | 139 +++++++++++++++++++++++++++++++++++
app/test-mldev/test_common.h | 27 +++++++
3 files changed, 167 insertions(+)
create mode 100644 app/test-mldev/test_common.c
create mode 100644 app/test-mldev/test_common.h
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
index 8ca2e1a1c1..964bb9ddc4 100644
--- a/app/test-mldev/meson.build
+++ b/app/test-mldev/meson.build
@@ -12,6 +12,7 @@ sources = files(
'ml_options.c',
'ml_test.c',
'parser.c',
+ 'test_common.c',
)
deps += ['mldev']
diff --git a/app/test-mldev/test_common.c b/app/test-mldev/test_common.c
new file mode 100644
index 0000000000..b6b32904e4
--- /dev/null
+++ b/app/test-mldev/test_common.c
@@ -0,0 +1,139 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <errno.h>
+#include <stdbool.h>
+#include <stdint.h>
+
+#include <rte_common.h>
+#include <rte_memory.h>
+#include <rte_mldev.h>
+
+#include "ml_common.h"
+#include "ml_options.h"
+#include "test_common.h"
+
+bool
+ml_test_cap_check(struct ml_options *opt)
+{
+ struct rte_ml_dev_info dev_info;
+
+ rte_ml_dev_info_get(opt->dev_id, &dev_info);
+ if (dev_info.max_models == 0) {
+ ml_err("Not enough mldev models supported = %d", dev_info.max_models);
+ return false;
+ }
+
+ return true;
+}
+
+int
+ml_test_opt_check(struct ml_options *opt)
+{
+ uint16_t dev_count;
+ int socket_id;
+
+ RTE_SET_USED(opt);
+
+ dev_count = rte_ml_dev_count();
+ if (dev_count == 0) {
+ ml_err("No ML devices found");
+ return -ENODEV;
+ }
+
+ if (opt->dev_id >= dev_count) {
+ ml_err("Invalid option dev_id = %d", opt->dev_id);
+ return -EINVAL;
+ }
+
+ socket_id = rte_ml_dev_socket_id(opt->dev_id);
+ if (!((opt->socket_id != SOCKET_ID_ANY) || (opt->socket_id != socket_id))) {
+ ml_err("Invalid option, socket_id = %d\n", opt->socket_id);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+void
+ml_test_opt_dump(struct ml_options *opt)
+{
+ ml_options_dump(opt);
+}
+
+int
+ml_test_device_configure(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_common *t = ml_test_priv(test);
+ struct rte_ml_dev_config dev_config;
+ int ret;
+
+ ret = rte_ml_dev_info_get(opt->dev_id, &t->dev_info);
+ if (ret != 0) {
+ ml_err("Failed to get mldev info, dev_id = %d\n", opt->dev_id);
+ return ret;
+ }
+
+ /* configure device */
+ dev_config.socket_id = opt->socket_id;
+ dev_config.nb_models = t->dev_info.max_models;
+ dev_config.nb_queue_pairs = t->dev_info.max_queue_pairs;
+ ret = rte_ml_dev_configure(opt->dev_id, &dev_config);
+ if (ret != 0) {
+ ml_err("Failed to configure ml device, dev_id = %d\n", opt->dev_id);
+ return ret;
+ }
+
+ return 0;
+}
+
+int
+ml_test_device_close(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_common *t = ml_test_priv(test);
+ int ret = 0;
+
+ RTE_SET_USED(t);
+
+ /* close device */
+ ret = rte_ml_dev_close(opt->dev_id);
+ if (ret != 0)
+ ml_err("Failed to close ML device, dev_id = %d\n", opt->dev_id);
+
+ return ret;
+}
+
+int
+ml_test_device_start(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_common *t = ml_test_priv(test);
+ int ret;
+
+ RTE_SET_USED(t);
+
+ /* start device */
+ ret = rte_ml_dev_start(opt->dev_id);
+ if (ret != 0) {
+ ml_err("Failed to start ml device, dev_id = %d\n", opt->dev_id);
+ return ret;
+ }
+
+ return 0;
+}
+
+int
+ml_test_device_stop(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_common *t = ml_test_priv(test);
+ int ret = 0;
+
+ RTE_SET_USED(t);
+
+ /* stop device */
+ ret = rte_ml_dev_stop(opt->dev_id);
+ if (ret != 0)
+ ml_err("Failed to stop ML device, dev_id = %d\n", opt->dev_id);
+
+ return ret;
+}
diff --git a/app/test-mldev/test_common.h b/app/test-mldev/test_common.h
new file mode 100644
index 0000000000..05a2e43e2f
--- /dev/null
+++ b/app/test-mldev/test_common.h
@@ -0,0 +1,27 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_TEST_COMMON_
+#define _ML_TEST_COMMON_
+
+#include <rte_mldev.h>
+
+#include "ml_options.h"
+#include "ml_test.h"
+
+struct test_common {
+ struct ml_options *opt;
+ enum ml_test_result result;
+ struct rte_ml_dev_info dev_info;
+};
+
+bool ml_test_cap_check(struct ml_options *opt);
+int ml_test_opt_check(struct ml_options *opt);
+void ml_test_opt_dump(struct ml_options *opt);
+int ml_test_device_configure(struct ml_test *test, struct ml_options *opt);
+int ml_test_device_close(struct ml_test *test, struct ml_options *opt);
+int ml_test_device_start(struct ml_test *test, struct ml_options *opt);
+int ml_test_device_stop(struct ml_test *test, struct ml_options *opt);
+
+#endif /* _ML_TEST_COMMON_ */
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v2 03/12] app/mldev: add test case to validate device ops
2022-11-29 8:20 ` [PATCH v2 " Srikanth Yalavarthi
2022-11-29 8:20 ` [PATCH v2 02/12] app/mldev: add common test functions Srikanth Yalavarthi
@ 2022-11-29 8:21 ` Srikanth Yalavarthi
2022-11-29 8:21 ` [PATCH v2 04/12] app/mldev: add test case to validate model ops Srikanth Yalavarthi
` (9 subsequent siblings)
11 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2022-11-29 8:21 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj
Added test case to validate device handling operations. Device ops
test is a collection of multiple sub-tests. Enabled sub-test to
validate device reconfiguration. Set device_ops as the default test.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/meson.build | 1 +
app/test-mldev/ml_options.c | 5 +-
app/test-mldev/test_device_ops.c | 234 +++++++++++++++++++++++++++++++
app/test-mldev/test_device_ops.h | 17 +++
4 files changed, 255 insertions(+), 2 deletions(-)
create mode 100644 app/test-mldev/test_device_ops.c
create mode 100644 app/test-mldev/test_device_ops.h
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
index 964bb9ddc4..60ea23d142 100644
--- a/app/test-mldev/meson.build
+++ b/app/test-mldev/meson.build
@@ -13,6 +13,7 @@ sources = files(
'ml_test.c',
'parser.c',
'test_common.c',
+ 'test_device_ops.c',
)
deps += ['mldev']
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 8fd7760e36..2e5f11bca2 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -24,7 +24,7 @@ void
ml_options_default(struct ml_options *opt)
{
memset(opt, 0, sizeof(*opt));
- strlcpy(opt->test_name, "ml_test", ML_TEST_NAME_MAX_LEN);
+ strlcpy(opt->test_name, "device_ops", ML_TEST_NAME_MAX_LEN);
opt->dev_id = 0;
opt->socket_id = SOCKET_ID_ANY;
opt->debug = false;
@@ -66,7 +66,8 @@ ml_parse_socket_id(struct ml_options *opt, const char *arg)
static void
ml_dump_test_options(const char *testname)
{
- RTE_SET_USED(testname);
+ if (strcmp(testname, "device_ops") == 0)
+ printf("\n");
}
static void
diff --git a/app/test-mldev/test_device_ops.c b/app/test-mldev/test_device_ops.c
new file mode 100644
index 0000000000..4cafcf41a6
--- /dev/null
+++ b/app/test-mldev/test_device_ops.c
@@ -0,0 +1,234 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <errno.h>
+#include <stdbool.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+
+#include <rte_common.h>
+#include <rte_malloc.h>
+#include <rte_mldev.h>
+
+#include "ml_common.h"
+#include "ml_options.h"
+#include "test_device_ops.h"
+
+static bool
+test_device_cap_check(struct ml_options *opt)
+{
+ if (!ml_test_cap_check(opt))
+ return false;
+
+ return true;
+}
+
+static int
+test_device_opt_check(struct ml_options *opt)
+{
+ int ret;
+
+ /* check common opts */
+ ret = ml_test_opt_check(opt);
+ if (ret != 0)
+ return ret;
+
+ return 0;
+}
+
+static void
+test_device_opt_dump(struct ml_options *opt)
+{
+ /* dump common opts */
+ ml_test_opt_dump(opt);
+}
+
+static int
+test_device_setup(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_device *t;
+ void *test_device;
+ int ret = 0;
+
+ /* allocate for test structure */
+ test_device = rte_zmalloc_socket(test->name, sizeof(struct test_device),
+ RTE_CACHE_LINE_SIZE, opt->socket_id);
+ if (test_device == NULL) {
+ ml_err("failed to allocate memory for test_model");
+ ret = -ENOMEM;
+ goto error;
+ }
+ test->test_priv = test_device;
+ t = ml_test_priv(test);
+
+ t->cmn.result = ML_TEST_FAILED;
+ t->cmn.opt = opt;
+
+ /* get device info */
+ ret = rte_ml_dev_info_get(opt->dev_id, &t->cmn.dev_info);
+ if (ret < 0) {
+ ml_err("failed to get device info");
+ goto error;
+ }
+
+ return 0;
+
+error:
+ if (test_device != NULL)
+ rte_free(test_device);
+
+ return ret;
+}
+
+static void
+test_device_destroy(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_device *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+ if (t != NULL)
+ rte_free(t);
+}
+
+static int
+test_device_reconfigure(struct ml_test *test, struct ml_options *opt)
+{
+ struct rte_ml_dev_config dev_config;
+ struct rte_ml_dev_qp_conf qp_conf;
+ struct test_device *t;
+ uint16_t qp_id = 0;
+ int ret = 0;
+
+ t = ml_test_priv(test);
+
+ /* configure with default options */
+ ret = ml_test_device_configure(test, opt);
+ if (ret != 0)
+ return ret;
+
+ /* setup one queue pair with nb_desc = 1 */
+ qp_conf.nb_desc = 1;
+ qp_conf.cb = NULL;
+
+ ret = rte_ml_dev_queue_pair_setup(opt->dev_id, qp_id, &qp_conf, opt->socket_id);
+ if (ret != 0) {
+ ml_err("Failed to setup ML device queue-pair, dev_id = %d, qp_id = %u\n",
+ opt->dev_id, qp_id);
+ goto error;
+ }
+
+ /* start device */
+ ret = ml_test_device_start(test, opt);
+ if (ret != 0)
+ goto error;
+
+ /* stop device */
+ ret = ml_test_device_stop(test, opt);
+ if (ret != 0) {
+ ml_err("Failed to stop device");
+ goto error;
+ }
+
+ /* reconfigure device based on dev_info */
+ dev_config.socket_id = opt->socket_id;
+ dev_config.nb_models = t->cmn.dev_info.max_models;
+ dev_config.nb_queue_pairs = t->cmn.dev_info.max_queue_pairs;
+ ret = rte_ml_dev_configure(opt->dev_id, &dev_config);
+ if (ret != 0) {
+ ml_err("Failed to reconfigure ML device, dev_id = %d\n", opt->dev_id);
+ return ret;
+ }
+
+ /* setup queue pairs */
+ for (qp_id = 0; qp_id < t->cmn.dev_info.max_queue_pairs; qp_id++) {
+ qp_conf.nb_desc = t->cmn.dev_info.max_desc;
+ qp_conf.cb = NULL;
+
+ ret = rte_ml_dev_queue_pair_setup(opt->dev_id, qp_id, &qp_conf, opt->socket_id);
+ if (ret != 0) {
+ ml_err("Failed to setup ML device queue-pair, dev_id = %d, qp_id = %u\n",
+ opt->dev_id, qp_id);
+ goto error;
+ }
+ }
+
+ /* start device */
+ ret = ml_test_device_start(test, opt);
+ if (ret != 0)
+ goto error;
+
+ /* stop device */
+ ret = ml_test_device_stop(test, opt);
+ if (ret != 0)
+ goto error;
+
+ /* close device */
+ ret = ml_test_device_close(test, opt);
+ if (ret != 0)
+ return ret;
+
+ return 0;
+
+error:
+ ml_test_device_close(test, opt);
+
+ return ret;
+}
+
+static int
+test_device_driver(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_device *t;
+ int ret = 0;
+
+ t = ml_test_priv(test);
+
+ /* sub-test: device reconfigure */
+ ret = test_device_reconfigure(test, opt);
+ if (ret != 0) {
+ printf("\n");
+ printf("Model Device Reconfigure Test: " CLRED "%s" CLNRM "\n", "Failed");
+ goto error;
+ } else {
+ printf("\n");
+ printf("Model Device Reconfigure Test: " CLYEL "%s" CLNRM "\n", "Passed");
+ }
+
+ printf("\n");
+
+ t->cmn.result = ML_TEST_SUCCESS;
+
+ return 0;
+
+error:
+ t->cmn.result = ML_TEST_FAILED;
+ return -1;
+}
+
+static int
+test_device_result(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_device *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+
+ return t->cmn.result;
+}
+
+static const struct ml_test_ops device_ops = {
+ .cap_check = test_device_cap_check,
+ .opt_check = test_device_opt_check,
+ .opt_dump = test_device_opt_dump,
+ .test_setup = test_device_setup,
+ .test_destroy = test_device_destroy,
+ .test_driver = test_device_driver,
+ .test_result = test_device_result,
+};
+
+ML_TEST_REGISTER(device_ops);
diff --git a/app/test-mldev/test_device_ops.h b/app/test-mldev/test_device_ops.h
new file mode 100644
index 0000000000..115b1072a2
--- /dev/null
+++ b/app/test-mldev/test_device_ops.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_TEST_DEVICE_OPS_
+#define _ML_TEST_DEVICE_OPS_
+
+#include <rte_common.h>
+
+#include "test_common.h"
+
+struct test_device {
+ /* common data */
+ struct test_common cmn;
+} __rte_cache_aligned;
+
+#endif /* _ML_TEST_DEVICE_OPS_ */
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v2 04/12] app/mldev: add test case to validate model ops
2022-11-29 8:20 ` [PATCH v2 " Srikanth Yalavarthi
2022-11-29 8:20 ` [PATCH v2 02/12] app/mldev: add common test functions Srikanth Yalavarthi
2022-11-29 8:21 ` [PATCH v2 03/12] app/mldev: add test case to validate device ops Srikanth Yalavarthi
@ 2022-11-29 8:21 ` Srikanth Yalavarthi
2022-11-29 8:21 ` [PATCH v2 05/12] app/mldev: add ordered inference test case Srikanth Yalavarthi
` (8 subsequent siblings)
11 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2022-11-29 8:21 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj
Added test case to validate model operations. Model ops test
is a collection of sub-tests. Each sub-test invokes the model
operations in a specific order.
Sub-test A: (load -> start -> stop -> unload) x n
Sub-test B: load x n -> start x n -> stop x n -> unload x n
Sub-test C: load x n + (start + stop) x n + unload x n
Sub-test D: (load + start) x n -> (stop + unload) x n
Added internal functions to handle model load, start, stop and
unload. List of models to be used for testing can be specified
through application argument "--models"
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/meson.build | 2 +
app/test-mldev/ml_options.c | 45 ++-
app/test-mldev/ml_options.h | 9 +
app/test-mldev/test_model_common.c | 162 +++++++++++
app/test-mldev/test_model_common.h | 37 +++
app/test-mldev/test_model_ops.c | 433 +++++++++++++++++++++++++++++
app/test-mldev/test_model_ops.h | 21 ++
7 files changed, 706 insertions(+), 3 deletions(-)
create mode 100644 app/test-mldev/test_model_common.c
create mode 100644 app/test-mldev/test_model_common.h
create mode 100644 app/test-mldev/test_model_ops.c
create mode 100644 app/test-mldev/test_model_ops.h
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
index 60ea23d142..b09e1ccc8a 100644
--- a/app/test-mldev/meson.build
+++ b/app/test-mldev/meson.build
@@ -14,6 +14,8 @@ sources = files(
'parser.c',
'test_common.c',
'test_device_ops.c',
+ 'test_model_common.c',
+ 'test_model_ops.c',
)
deps += ['mldev']
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 2e5f11bca2..15043c0992 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -4,6 +4,7 @@
#include <errno.h>
#include <getopt.h>
+#include <linux/limits.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
@@ -27,6 +28,7 @@ ml_options_default(struct ml_options *opt)
strlcpy(opt->test_name, "device_ops", ML_TEST_NAME_MAX_LEN);
opt->dev_id = 0;
opt->socket_id = SOCKET_ID_ANY;
+ opt->nb_filelist = 0;
opt->debug = false;
}
@@ -63,11 +65,47 @@ ml_parse_socket_id(struct ml_options *opt, const char *arg)
return 0;
}
+static int
+ml_parse_models(struct ml_options *opt, const char *arg)
+{
+ const char *delim = ",";
+ char models[PATH_MAX];
+ char *token;
+ int ret = 0;
+
+ strlcpy(models, arg, PATH_MAX);
+
+ token = strtok(models, delim);
+ while (token != NULL) {
+ strlcpy(opt->filelist[opt->nb_filelist].model, token, PATH_MAX);
+ opt->nb_filelist++;
+
+ if (opt->nb_filelist >= ML_TEST_MAX_MODELS) {
+ ml_err("Exceeded model count, max = %d\n", ML_TEST_MAX_MODELS);
+ ret = -EINVAL;
+ break;
+ }
+ token = strtok(NULL, delim);
+ }
+
+ if (opt->nb_filelist == 0) {
+ ml_err("Models list is empty. Need at least one model for the test");
+ ret = -EINVAL;
+ }
+
+ return ret;
+}
+
static void
ml_dump_test_options(const char *testname)
{
if (strcmp(testname, "device_ops") == 0)
printf("\n");
+
+ if (strcmp(testname, "model_ops") == 0) {
+ printf("\t\t--models : comma separated list of models\n");
+ printf("\n");
+ }
}
static void
@@ -85,9 +123,9 @@ print_usage(char *program)
ml_test_dump_names(ml_dump_test_options);
}
-static struct option lgopts[] = {{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0},
- {ML_SOCKET_ID, 1, 0, 0}, {ML_DEBUG, 0, 0, 0},
- {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
+static struct option lgopts[] = {
+ {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0}, {ML_MODELS, 1, 0, 0},
+ {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
static int
ml_opts_parse_long(int opt_idx, struct ml_options *opt)
@@ -98,6 +136,7 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
{ML_TEST, ml_parse_test_name},
{ML_DEVICE_ID, ml_parse_dev_id},
{ML_SOCKET_ID, ml_parse_socket_id},
+ {ML_MODELS, ml_parse_models},
};
for (i = 0; i < RTE_DIM(parsermap); i++) {
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index 05311a9a47..8faf3b5deb 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -5,22 +5,31 @@
#ifndef _ML_OPTIONS_
#define _ML_OPTIONS_
+#include <linux/limits.h>
#include <stdbool.h>
#include <stdint.h>
#define ML_TEST_NAME_MAX_LEN 32
+#define ML_TEST_MAX_MODELS 8
/* Options names */
#define ML_TEST ("test")
#define ML_DEVICE_ID ("dev_id")
#define ML_SOCKET_ID ("socket_id")
+#define ML_MODELS ("models")
#define ML_DEBUG ("debug")
#define ML_HELP ("help")
+struct ml_filelist {
+ char model[PATH_MAX];
+};
+
struct ml_options {
char test_name[ML_TEST_NAME_MAX_LEN];
int16_t dev_id;
int socket_id;
+ struct ml_filelist filelist[ML_TEST_MAX_MODELS];
+ uint8_t nb_filelist;
bool debug;
};
diff --git a/app/test-mldev/test_model_common.c b/app/test-mldev/test_model_common.c
new file mode 100644
index 0000000000..5368be17fe
--- /dev/null
+++ b/app/test-mldev/test_model_common.c
@@ -0,0 +1,162 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <errno.h>
+#include <stdio.h>
+
+#include <rte_common.h>
+#include <rte_malloc.h>
+#include <rte_mldev.h>
+
+#include "ml_common.h"
+#include "ml_options.h"
+#include "ml_test.h"
+#include "test_common.h"
+#include "test_model_common.h"
+
+int
+ml_model_load(struct ml_test *test, struct ml_options *opt, struct ml_model *model, int16_t fid)
+{
+ struct test_common *t = ml_test_priv(test);
+ struct rte_ml_model_params model_params;
+ FILE *fp;
+ int ret;
+
+ if (model->state == MODEL_LOADED)
+ return 0;
+
+ if (model->state != MODEL_INITIAL)
+ return -EINVAL;
+
+ /* read model binary */
+ fp = fopen(opt->filelist[fid].model, "r");
+ if (fp == NULL) {
+ ml_err("Failed to open model file : %s\n", opt->filelist[fid].model);
+ return -1;
+ }
+
+ fseek(fp, 0, SEEK_END);
+ model_params.size = ftell(fp);
+ fseek(fp, 0, SEEK_SET);
+
+ model_params.addr = rte_malloc_socket("ml_model", model_params.size,
+ t->dev_info.min_align_size, opt->socket_id);
+ if (model_params.addr == NULL) {
+ ml_err("Failed to allocate memory for model: %s\n", opt->filelist[fid].model);
+ fclose(fp);
+ return -ENOMEM;
+ }
+
+ if (fread(model_params.addr, 1, model_params.size, fp) != model_params.size) {
+ ml_err("Failed to read model file : %s\n", opt->filelist[fid].model);
+ rte_free(model_params.addr);
+ fclose(fp);
+ return -1;
+ }
+ fclose(fp);
+
+ /* load model to device */
+ ret = rte_ml_model_load(opt->dev_id, &model_params, &model->id);
+ if (ret != 0) {
+ ml_err("Failed to load model : %s\n", opt->filelist[fid].model);
+ model->state = MODEL_ERROR;
+ rte_free(model_params.addr);
+ return ret;
+ }
+
+ /* release mz */
+ rte_free(model_params.addr);
+
+ /* get model info */
+ ret = rte_ml_model_info_get(opt->dev_id, model->id, &model->info);
+ if (ret != 0) {
+ ml_err("Failed to get model info : %s\n", opt->filelist[fid].model);
+ return ret;
+ }
+
+ model->state = MODEL_LOADED;
+
+ return 0;
+}
+
+int
+ml_model_unload(struct ml_test *test, struct ml_options *opt, struct ml_model *model, int16_t fid)
+{
+ struct test_common *t = ml_test_priv(test);
+ int ret;
+
+ RTE_SET_USED(t);
+
+ if (model->state == MODEL_INITIAL)
+ return 0;
+
+ if (model->state != MODEL_LOADED)
+ return -EINVAL;
+
+ /* unload model */
+ ret = rte_ml_model_unload(opt->dev_id, model->id);
+ if (ret != 0) {
+ ml_err("Failed to unload model: %s\n", opt->filelist[fid].model);
+ model->state = MODEL_ERROR;
+ return ret;
+ }
+
+ model->state = MODEL_INITIAL;
+
+ return 0;
+}
+
+int
+ml_model_start(struct ml_test *test, struct ml_options *opt, struct ml_model *model, int16_t fid)
+{
+ struct test_common *t = ml_test_priv(test);
+ int ret;
+
+ RTE_SET_USED(t);
+
+ if (model->state == MODEL_STARTED)
+ return 0;
+
+ if (model->state != MODEL_LOADED)
+ return -EINVAL;
+
+ /* start model */
+ ret = rte_ml_model_start(opt->dev_id, model->id);
+ if (ret != 0) {
+ ml_err("Failed to start model : %s\n", opt->filelist[fid].model);
+ model->state = MODEL_ERROR;
+ return ret;
+ }
+
+ model->state = MODEL_STARTED;
+
+ return 0;
+}
+
+int
+ml_model_stop(struct ml_test *test, struct ml_options *opt, struct ml_model *model, int16_t fid)
+{
+ struct test_common *t = ml_test_priv(test);
+ int ret;
+
+ RTE_SET_USED(t);
+
+ if (model->state == MODEL_LOADED)
+ return 0;
+
+ if (model->state != MODEL_STARTED)
+ return -EINVAL;
+
+ /* stop model */
+ ret = rte_ml_model_stop(opt->dev_id, model->id);
+ if (ret != 0) {
+ ml_err("Failed to stop model: %s\n", opt->filelist[fid].model);
+ model->state = MODEL_ERROR;
+ return ret;
+ }
+
+ model->state = MODEL_LOADED;
+
+ return 0;
+}
diff --git a/app/test-mldev/test_model_common.h b/app/test-mldev/test_model_common.h
new file mode 100644
index 0000000000..302e4eb45f
--- /dev/null
+++ b/app/test-mldev/test_model_common.h
@@ -0,0 +1,37 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_TEST_MODEL_COMMON_
+#define _ML_TEST_MODEL_COMMON_
+
+#include <stdint.h>
+
+#include <rte_mldev.h>
+
+#include "ml_options.h"
+#include "ml_test.h"
+
+enum model_state {
+ MODEL_INITIAL,
+ MODEL_LOADED,
+ MODEL_STARTED,
+ MODEL_ERROR,
+};
+
+struct ml_model {
+ int16_t id;
+ struct rte_ml_model_info info;
+ enum model_state state;
+};
+
+int ml_model_load(struct ml_test *test, struct ml_options *opt, struct ml_model *model,
+ int16_t fid);
+int ml_model_unload(struct ml_test *test, struct ml_options *opt, struct ml_model *model,
+ int16_t fid);
+int ml_model_start(struct ml_test *test, struct ml_options *opt, struct ml_model *model,
+ int16_t fid);
+int ml_model_stop(struct ml_test *test, struct ml_options *opt, struct ml_model *model,
+ int16_t fid);
+
+#endif /* _ML_TEST_MODEL_COMMON_ */
diff --git a/app/test-mldev/test_model_ops.c b/app/test-mldev/test_model_ops.c
new file mode 100644
index 0000000000..69c9df8ed6
--- /dev/null
+++ b/app/test-mldev/test_model_ops.c
@@ -0,0 +1,433 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <stdint.h>
+#include <stdio.h>
+#include <unistd.h>
+
+#include <rte_common.h>
+#include <rte_malloc.h>
+#include <rte_mldev.h>
+
+#include "ml_common.h"
+#include "ml_options.h"
+#include "ml_test.h"
+#include "test_model_ops.h"
+
+static bool
+test_model_ops_cap_check(struct ml_options *opt)
+{
+ if (!ml_test_cap_check(opt))
+ return false;
+
+ return true;
+}
+
+static int
+test_model_ops_opt_check(struct ml_options *opt)
+{
+ uint32_t i;
+ int ret;
+
+ /* check common opts */
+ ret = ml_test_opt_check(opt);
+ if (ret != 0)
+ return ret;
+
+ /* check model file availability */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ if (access(opt->filelist[i].model, F_OK) == -1) {
+ ml_err("Model file not available: id = %u, file = %s", i,
+ opt->filelist[i].model);
+ return -ENOENT;
+ }
+ }
+
+ return 0;
+}
+
+static void
+test_model_ops_opt_dump(struct ml_options *opt)
+{
+ uint32_t i;
+
+ /* dump common opts */
+ ml_test_opt_dump(opt);
+
+ /* dump test specific opts */
+ ml_dump_begin("models");
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_dump_list("model", i, opt->filelist[i].model);
+ ml_dump_end;
+}
+
+static int
+test_model_ops_setup(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+ void *test_model_ops;
+ int ret = 0;
+ uint32_t i;
+
+ /* allocate model ops test structure */
+ test_model_ops = rte_zmalloc_socket(test->name, sizeof(struct test_model_ops),
+ RTE_CACHE_LINE_SIZE, opt->socket_id);
+ if (test_model_ops == NULL) {
+ ml_err("Failed to allocate memory for test_model");
+ ret = -ENOMEM;
+ goto error;
+ }
+ test->test_priv = test_model_ops;
+ t = ml_test_priv(test);
+
+ t->cmn.result = ML_TEST_FAILED;
+ t->cmn.opt = opt;
+
+ /* get device info */
+ ret = rte_ml_dev_info_get(opt->dev_id, &t->cmn.dev_info);
+ if (ret < 0) {
+ ml_err("Failed to get device info");
+ goto error;
+ }
+
+ /* set model initial state */
+ for (i = 0; i < opt->nb_filelist; i++)
+ t->model[i].state = MODEL_INITIAL;
+
+ return 0;
+
+error:
+ if (test_model_ops != NULL)
+ rte_free(test_model_ops);
+
+ return ret;
+}
+
+static void
+test_model_ops_destroy(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+ if (t != NULL)
+ rte_free(t);
+}
+
+static int
+test_model_ops_mldev_setup(struct ml_test *test, struct ml_options *opt)
+{
+ int ret;
+
+ ret = ml_test_device_configure(test, opt);
+ if (ret != 0)
+ return ret;
+
+ ret = ml_test_device_start(test, opt);
+ if (ret != 0)
+ goto error;
+
+ return 0;
+
+error:
+ ml_test_device_close(test, opt);
+
+ return ret;
+}
+
+static int
+test_model_ops_mldev_destroy(struct ml_test *test, struct ml_options *opt)
+{
+ int ret;
+
+ ret = ml_test_device_stop(test, opt);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_test_device_close(test, opt);
+ if (ret != 0)
+ return ret;
+
+ return 0;
+
+error:
+ ml_test_device_close(test, opt);
+
+ return ret;
+}
+
+/* Sub-test A: (load -> start -> stop -> unload) x n */
+static int
+test_model_ops_subtest_a(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+ int ret = 0;
+ uint32_t i;
+
+ t = ml_test_priv(test);
+
+ /* load + start + stop + unload */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_load(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_start(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_stop(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_unload(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+error:
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_stop(test, opt, &t->model[i], i);
+
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_unload(test, opt, &t->model[i], i);
+
+ return ret;
+}
+
+/* Sub-test B: load x n -> start x n -> stop x n -> unload x n */
+static int
+test_model_ops_subtest_b(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+ int ret = 0;
+ uint32_t i;
+
+ t = ml_test_priv(test);
+
+ /* load */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_load(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* start */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_start(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* stop */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_stop(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* unload */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_unload(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ return 0;
+
+error:
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_stop(test, opt, &t->model[i], i);
+
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_unload(test, opt, &t->model[i], i);
+
+ return ret;
+}
+
+/* Sub-test C: load x n + (start + stop) x n + unload x n */
+static int
+test_model_ops_subtest_c(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+ int ret = 0;
+ uint32_t i;
+
+ t = ml_test_priv(test);
+
+ /* load */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_load(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* start + stop */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_start(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_stop(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* unload */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_unload(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ return 0;
+
+error:
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_stop(test, opt, &t->model[i], i);
+
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_unload(test, opt, &t->model[i], i);
+
+ return ret;
+}
+
+/* Sub-test D: (load + start) x n -> (stop + unload) x n */
+static int
+test_model_ops_subtest_d(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+ int ret = 0;
+ uint32_t i;
+
+ t = ml_test_priv(test);
+
+ /* load + start */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_load(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_start(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* stop + unload */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_stop(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_unload(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ return 0;
+
+error:
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_stop(test, opt, &t->model[i], i);
+
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_unload(test, opt, &t->model[i], i);
+
+ return ret;
+}
+
+static int
+test_model_ops_driver(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+ int ret = 0;
+
+ t = ml_test_priv(test);
+
+ /* device setup */
+ ret = test_model_ops_mldev_setup(test, opt);
+ if (ret != 0)
+ return ret;
+
+ printf("\n");
+
+ /* sub-test A */
+ ret = test_model_ops_subtest_a(test, opt);
+ if (ret != 0) {
+ printf("Model Ops Sub-test A: " CLRED "%s" CLNRM "\n", "Failed");
+ goto error;
+ } else {
+ printf("Model Ops Sub-test A: " CLYEL "%s" CLNRM "\n", "Passed");
+ }
+
+ /* sub-test B */
+ ret = test_model_ops_subtest_b(test, opt);
+ if (ret != 0) {
+ printf("Model Ops Sub-test B: " CLRED "%s" CLNRM "\n", "Failed");
+ goto error;
+ } else {
+ printf("Model Ops Sub-test B: " CLYEL "%s" CLNRM "\n", "Passed");
+ }
+
+ /* sub-test C */
+ ret = test_model_ops_subtest_c(test, opt);
+ if (ret != 0) {
+ printf("Model Ops Sub-test C: " CLRED "%s" CLNRM "\n", "Failed");
+ goto error;
+ } else {
+ printf("Model Ops Sub-test C: " CLYEL "%s" CLNRM "\n", "Passed");
+ }
+
+ /* sub-test D */
+ ret = test_model_ops_subtest_d(test, opt);
+ if (ret != 0) {
+ printf("Model Ops Sub-test D: " CLRED "%s" CLNRM "\n", "Failed");
+ goto error;
+ } else {
+ printf("Model Ops Sub-test D: " CLYEL "%s" CLNRM "\n", "Passed");
+ }
+
+ printf("\n");
+
+ /* device destroy */
+ ret = test_model_ops_mldev_destroy(test, opt);
+ if (ret != 0)
+ return ret;
+
+ t->cmn.result = ML_TEST_SUCCESS;
+
+ return 0;
+
+error:
+ test_model_ops_mldev_destroy(test, opt);
+
+ t->cmn.result = ML_TEST_FAILED;
+
+ return ret;
+}
+
+static int
+test_model_ops_result(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+
+ return t->cmn.result;
+}
+
+static const struct ml_test_ops model_ops = {
+ .cap_check = test_model_ops_cap_check,
+ .opt_check = test_model_ops_opt_check,
+ .opt_dump = test_model_ops_opt_dump,
+ .test_setup = test_model_ops_setup,
+ .test_destroy = test_model_ops_destroy,
+ .test_driver = test_model_ops_driver,
+ .test_result = test_model_ops_result,
+};
+
+ML_TEST_REGISTER(model_ops);
diff --git a/app/test-mldev/test_model_ops.h b/app/test-mldev/test_model_ops.h
new file mode 100644
index 0000000000..9dd8402390
--- /dev/null
+++ b/app/test-mldev/test_model_ops.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_TEST_MODEL_OPS_
+#define _ML_TEST_MODEL_OPS_
+
+#include <rte_common.h>
+
+#include "test_common.h"
+#include "test_model_common.h"
+
+struct test_model_ops {
+ /* common data */
+ struct test_common cmn;
+
+ /* test specific data */
+ struct ml_model model[ML_TEST_MAX_MODELS];
+} __rte_cache_aligned;
+
+#endif /* _ML_TEST_MODEL_OPS_ */
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v2 05/12] app/mldev: add ordered inference test case
2022-11-29 8:20 ` [PATCH v2 " Srikanth Yalavarthi
` (2 preceding siblings ...)
2022-11-29 8:21 ` [PATCH v2 04/12] app/mldev: add test case to validate model ops Srikanth Yalavarthi
@ 2022-11-29 8:21 ` Srikanth Yalavarthi
2022-11-29 8:21 ` [PATCH v2 06/12] app/mldev: add test case to interleave inferences Srikanth Yalavarthi
` (7 subsequent siblings)
11 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2022-11-29 8:21 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj
Added an ordered test case to execute inferences with single
or multiple models. In this test case inference requests for
a model are enqueued after completion of all requests for
the previous model. Test supports inference repetitions.
Operations sequence when testing with N models and R reps,
(load -> start -> (enqueue + dequeue) x R -> stop -> unload) x N
Test case can be executed by selecting "inference_ordered" test
and repetitions can be specified through "--repetitions" argument.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/meson.build | 2 +
app/test-mldev/ml_options.c | 73 ++-
app/test-mldev/ml_options.h | 17 +-
app/test-mldev/test_inference_common.c | 565 ++++++++++++++++++++++++
app/test-mldev/test_inference_common.h | 65 +++
app/test-mldev/test_inference_ordered.c | 119 +++++
app/test-mldev/test_model_common.h | 10 +
7 files changed, 839 insertions(+), 12 deletions(-)
create mode 100644 app/test-mldev/test_inference_common.c
create mode 100644 app/test-mldev/test_inference_common.h
create mode 100644 app/test-mldev/test_inference_ordered.c
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
index b09e1ccc8a..475d76d126 100644
--- a/app/test-mldev/meson.build
+++ b/app/test-mldev/meson.build
@@ -16,6 +16,8 @@ sources = files(
'test_device_ops.c',
'test_model_common.c',
'test_model_ops.c',
+ 'test_inference_common.c',
+ 'test_inference_ordered.c',
)
deps += ['mldev']
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 15043c0992..10dad18fff 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -29,6 +29,7 @@ ml_options_default(struct ml_options *opt)
opt->dev_id = 0;
opt->socket_id = SOCKET_ID_ANY;
opt->nb_filelist = 0;
+ opt->repetitions = 1;
opt->debug = false;
}
@@ -96,6 +97,60 @@ ml_parse_models(struct ml_options *opt, const char *arg)
return ret;
}
+static int
+ml_parse_filelist(struct ml_options *opt, const char *arg)
+{
+ const char *delim = ",";
+ char filelist[PATH_MAX];
+ char *token;
+
+ if (opt->nb_filelist >= ML_TEST_MAX_MODELS) {
+ ml_err("Exceeded filelist count, max = %d\n", ML_TEST_MAX_MODELS);
+ return -1;
+ }
+
+ strlcpy(filelist, arg, PATH_MAX);
+
+ /* model */
+ token = strtok(filelist, delim);
+ if (token == NULL) {
+ ml_err("Invalid filelist, model not specified = %s\n", arg);
+ return -EINVAL;
+ }
+ strlcpy(opt->filelist[opt->nb_filelist].model, token, PATH_MAX);
+
+ /* input */
+ token = strtok(NULL, delim);
+ if (token == NULL) {
+ ml_err("Invalid filelist, input not specified = %s\n", arg);
+ return -EINVAL;
+ }
+ strlcpy(opt->filelist[opt->nb_filelist].input, token, PATH_MAX);
+
+ /* output */
+ token = strtok(NULL, delim);
+ if (token == NULL) {
+ ml_err("Invalid filelist, output not specified = %s\n", arg);
+ return -EINVAL;
+ }
+ strlcpy(opt->filelist[opt->nb_filelist].output, token, PATH_MAX);
+
+ opt->nb_filelist++;
+
+ if (opt->nb_filelist == 0) {
+ ml_err("Empty filelist. Need at least one filelist entry for the test.");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int
+ml_parse_repetitions(struct ml_options *opt, const char *arg)
+{
+ return parser_read_uint64(&opt->repetitions, arg);
+}
+
static void
ml_dump_test_options(const char *testname)
{
@@ -106,6 +161,12 @@ ml_dump_test_options(const char *testname)
printf("\t\t--models : comma separated list of models\n");
printf("\n");
}
+
+ if (strcmp(testname, "inference_ordered") == 0) {
+ printf("\t\t--filelist : comma separated list of model, input and output\n"
+ "\t\t--repetitions : number of inference repetitions\n");
+ printf("\n");
+ }
}
static void
@@ -124,8 +185,9 @@ print_usage(char *program)
}
static struct option lgopts[] = {
- {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0}, {ML_MODELS, 1, 0, 0},
- {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
+ {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
+ {ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
+ {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
static int
ml_opts_parse_long(int opt_idx, struct ml_options *opt)
@@ -133,10 +195,9 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
unsigned int i;
struct long_opt_parser parsermap[] = {
- {ML_TEST, ml_parse_test_name},
- {ML_DEVICE_ID, ml_parse_dev_id},
- {ML_SOCKET_ID, ml_parse_socket_id},
- {ML_MODELS, ml_parse_models},
+ {ML_TEST, ml_parse_test_name}, {ML_DEVICE_ID, ml_parse_dev_id},
+ {ML_SOCKET_ID, ml_parse_socket_id}, {ML_MODELS, ml_parse_models},
+ {ML_FILELIST, ml_parse_filelist}, {ML_REPETITIONS, ml_parse_repetitions},
};
for (i = 0; i < RTE_DIM(parsermap); i++) {
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index 8faf3b5deb..ad8aee5964 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -13,15 +13,19 @@
#define ML_TEST_MAX_MODELS 8
/* Options names */
-#define ML_TEST ("test")
-#define ML_DEVICE_ID ("dev_id")
-#define ML_SOCKET_ID ("socket_id")
-#define ML_MODELS ("models")
-#define ML_DEBUG ("debug")
-#define ML_HELP ("help")
+#define ML_TEST ("test")
+#define ML_DEVICE_ID ("dev_id")
+#define ML_SOCKET_ID ("socket_id")
+#define ML_MODELS ("models")
+#define ML_FILELIST ("filelist")
+#define ML_REPETITIONS ("repetitions")
+#define ML_DEBUG ("debug")
+#define ML_HELP ("help")
struct ml_filelist {
char model[PATH_MAX];
+ char input[PATH_MAX];
+ char output[PATH_MAX];
};
struct ml_options {
@@ -30,6 +34,7 @@ struct ml_options {
int socket_id;
struct ml_filelist filelist[ML_TEST_MAX_MODELS];
uint8_t nb_filelist;
+ uint64_t repetitions;
bool debug;
};
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
new file mode 100644
index 0000000000..8b5dc89346
--- /dev/null
+++ b/app/test-mldev/test_inference_common.c
@@ -0,0 +1,565 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <errno.h>
+#include <stdbool.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <unistd.h>
+
+#include <rte_common.h>
+#include <rte_launch.h>
+#include <rte_lcore.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_mldev.h>
+
+#include "ml_common.h"
+#include "ml_options.h"
+#include "ml_test.h"
+#include "test_common.h"
+#include "test_inference_common.h"
+
+/* Enqueue inference requests with burst size equal to 1 */
+static int
+ml_enqueue_single(void *arg)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)arg);
+ struct ml_request *req = NULL;
+ struct rte_ml_op *op = NULL;
+ struct ml_core_args *args;
+ uint64_t model_enq = 0;
+ uint32_t burst_enq;
+ uint32_t lcore_id;
+ int16_t fid;
+ int ret;
+
+ lcore_id = rte_lcore_id();
+ args = &t->args[lcore_id];
+ model_enq = 0;
+
+ if (args->nb_reqs == 0)
+ return 0;
+
+next_rep:
+ fid = args->start_fid;
+
+next_model:
+ ret = rte_mempool_get(t->op_pool, (void **)&op);
+ if (ret != 0)
+ goto next_model;
+
+retry:
+ ret = rte_mempool_get(t->model[fid].io_pool, (void **)&req);
+ if (ret != 0)
+ goto retry;
+
+ op->model_id = t->model[fid].id;
+ op->nb_batches = t->model[fid].info.batch_size;
+ op->mempool = t->op_pool;
+
+ op->input.addr = req->input;
+ op->input.length = t->model[fid].inp_qsize;
+ op->input.next = NULL;
+
+ op->output.addr = req->output;
+ op->output.length = t->model[fid].out_qsize;
+ op->output.next = NULL;
+
+ op->user_ptr = req;
+ req->niters++;
+ req->fid = fid;
+
+enqueue_req:
+ burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, 0, &op, 1);
+ if (burst_enq == 0)
+ goto enqueue_req;
+
+ fid++;
+ if (likely(fid <= args->end_fid))
+ goto next_model;
+
+ model_enq++;
+ if (likely(model_enq < args->nb_reqs))
+ goto next_rep;
+
+ return 0;
+}
+
+/* Dequeue inference requests with burst size equal to 1 */
+static int
+ml_dequeue_single(void *arg)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)arg);
+ struct rte_ml_op_error error;
+ struct rte_ml_op *op = NULL;
+ struct ml_core_args *args;
+ struct ml_request *req;
+ uint64_t total_deq = 0;
+ uint8_t nb_filelist;
+ uint32_t burst_deq;
+ uint32_t lcore_id;
+
+ lcore_id = rte_lcore_id();
+ args = &t->args[lcore_id];
+ nb_filelist = args->end_fid - args->start_fid + 1;
+
+ if (args->nb_reqs == 0)
+ return 0;
+
+dequeue_req:
+ burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, 0, &op, 1);
+
+ if (likely(burst_deq == 1)) {
+ total_deq += burst_deq;
+ if (unlikely(op->status == RTE_ML_OP_STATUS_ERROR)) {
+ rte_ml_op_error_get(t->cmn.opt->dev_id, op, &error);
+ ml_err("error_code = 0x%016lx, error_message = %s\n", error.errcode,
+ error.message);
+ }
+ req = (struct ml_request *)op->user_ptr;
+ rte_mempool_put(t->model[req->fid].io_pool, req);
+ rte_mempool_put(t->op_pool, op);
+ }
+
+ if (likely(total_deq < args->nb_reqs * nb_filelist))
+ goto dequeue_req;
+
+ return 0;
+}
+
+bool
+test_inference_cap_check(struct ml_options *opt)
+{
+ struct rte_ml_dev_info dev_info;
+
+ if (!ml_test_cap_check(opt))
+ return false;
+
+ rte_ml_dev_info_get(opt->dev_id, &dev_info);
+ if (opt->nb_filelist > dev_info.max_models) {
+ ml_err("Insufficient capabilities: Filelist count exceeded device limit, count = %u (max limit = %u)",
+ opt->nb_filelist, dev_info.max_models);
+ return false;
+ }
+
+ return true;
+}
+
+int
+test_inference_opt_check(struct ml_options *opt)
+{
+ uint32_t i;
+ int ret;
+
+ /* check common opts */
+ ret = ml_test_opt_check(opt);
+ if (ret != 0)
+ return ret;
+
+ /* check file availability */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ if (access(opt->filelist[i].model, F_OK) == -1) {
+ ml_err("Model file not accessible: id = %u, file = %s", i,
+ opt->filelist[i].model);
+ return -ENOENT;
+ }
+
+ if (access(opt->filelist[i].input, F_OK) == -1) {
+ ml_err("Input file not accessible: id = %u, file = %s", i,
+ opt->filelist[i].input);
+ return -ENOENT;
+ }
+ }
+
+ if (opt->repetitions == 0) {
+ ml_err("Invalid option, repetitions = %" PRIu64 "\n", opt->repetitions);
+ return -EINVAL;
+ }
+
+ /* check number of available lcores. */
+ if (rte_lcore_count() < 3) {
+ ml_err("Insufficient lcores = %u\n", rte_lcore_count());
+ ml_err("Minimum lcores required to create %u queue-pairs = %u\n", 1, 3);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+void
+test_inference_opt_dump(struct ml_options *opt)
+{
+ uint32_t i;
+
+ /* dump common opts */
+ ml_test_opt_dump(opt);
+
+ /* dump test opts */
+ ml_dump("repetitions", "%" PRIu64, opt->repetitions);
+
+ ml_dump_begin("filelist");
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ml_dump_list("model", i, opt->filelist[i].model);
+ ml_dump_list("input", i, opt->filelist[i].input);
+ ml_dump_list("output", i, opt->filelist[i].output);
+ }
+ ml_dump_end;
+}
+
+int
+test_inference_setup(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t;
+ void *test_inference;
+ int ret = 0;
+ uint32_t i;
+
+ test_inference = rte_zmalloc_socket(test->name, sizeof(struct test_inference),
+ RTE_CACHE_LINE_SIZE, opt->socket_id);
+ if (test_inference == NULL) {
+ ml_err("failed to allocate memory for test_model");
+ ret = -ENOMEM;
+ goto error;
+ }
+ test->test_priv = test_inference;
+ t = ml_test_priv(test);
+
+ t->nb_used = 0;
+ t->cmn.result = ML_TEST_FAILED;
+ t->cmn.opt = opt;
+
+ /* get device info */
+ ret = rte_ml_dev_info_get(opt->dev_id, &t->cmn.dev_info);
+ if (ret < 0) {
+ ml_err("failed to get device info");
+ goto error;
+ }
+
+ t->enqueue = ml_enqueue_single;
+ t->dequeue = ml_dequeue_single;
+
+ /* set model initial state */
+ for (i = 0; i < opt->nb_filelist; i++)
+ t->model[i].state = MODEL_INITIAL;
+
+ return 0;
+
+error:
+ if (test_inference != NULL)
+ rte_free(test_inference);
+
+ return ret;
+}
+
+void
+test_inference_destroy(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+ if (t != NULL)
+ rte_free(t);
+}
+
+int
+ml_inference_mldev_setup(struct ml_test *test, struct ml_options *opt)
+{
+ struct rte_ml_dev_qp_conf qp_conf;
+ struct test_inference *t;
+ int ret;
+
+ t = ml_test_priv(test);
+
+ ret = ml_test_device_configure(test, opt);
+ if (ret != 0)
+ return ret;
+
+ /* setup queue pairs */
+ qp_conf.nb_desc = t->cmn.dev_info.max_desc;
+ qp_conf.cb = NULL;
+
+ ret = rte_ml_dev_queue_pair_setup(opt->dev_id, 0, &qp_conf, opt->socket_id);
+ if (ret != 0) {
+ ml_err("Failed to setup ml device queue-pair, dev_id = %d, qp_id = %u\n",
+ opt->dev_id, 0);
+ goto error;
+ }
+
+ ret = ml_test_device_start(test, opt);
+ if (ret != 0)
+ goto error;
+
+ return 0;
+
+error:
+ ml_test_device_close(test, opt);
+
+ return ret;
+}
+
+int
+ml_inference_mldev_destroy(struct ml_test *test, struct ml_options *opt)
+{
+ int ret;
+
+ ret = ml_test_device_stop(test, opt);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_test_device_close(test, opt);
+ if (ret != 0)
+ return ret;
+
+ return 0;
+
+error:
+ ml_test_device_close(test, opt);
+
+ return ret;
+}
+
+/* Callback for IO pool create. This function would compute the fields of ml_request
+ * structure and prepare the quantized input data.
+ */
+static void
+ml_request_initialize(struct rte_mempool *mp, void *opaque, void *obj, unsigned int obj_idx)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)opaque);
+ struct ml_request *req = (struct ml_request *)obj;
+
+ RTE_SET_USED(mp);
+ RTE_SET_USED(obj_idx);
+
+ req->input = RTE_PTR_ADD(
+ obj, RTE_ALIGN_CEIL(sizeof(struct ml_request), t->cmn.dev_info.min_align_size));
+ req->output = RTE_PTR_ADD(req->input, RTE_ALIGN_CEIL(t->model[t->fid].inp_qsize,
+ t->cmn.dev_info.min_align_size));
+ req->niters = 0;
+
+ /* quantize data */
+ rte_ml_io_quantize(t->cmn.opt->dev_id, t->model[t->fid].id,
+ t->model[t->fid].info.batch_size, t->model[t->fid].input, req->input);
+}
+
+int
+ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, int16_t fid)
+{
+ struct test_inference *t = ml_test_priv(test);
+ char mz_name[RTE_MEMZONE_NAMESIZE];
+ char mp_name[RTE_MEMPOOL_NAMESIZE];
+ const struct rte_memzone *mz;
+ uint64_t nb_buffers;
+ uint32_t buff_size;
+ uint32_t mz_size;
+ uint32_t fsize;
+ FILE *fp;
+ int ret;
+
+ /* get input buffer size */
+ ret = rte_ml_io_input_size_get(opt->dev_id, t->model[fid].id, t->model[fid].info.batch_size,
+ &t->model[fid].inp_qsize, &t->model[fid].inp_dsize);
+ if (ret != 0) {
+ ml_err("Failed to get input size, model : %s\n", opt->filelist[fid].model);
+ return ret;
+ }
+
+ /* get output buffer size */
+ ret = rte_ml_io_output_size_get(opt->dev_id, t->model[fid].id,
+ t->model[fid].info.batch_size, &t->model[fid].out_qsize,
+ &t->model[fid].out_dsize);
+ if (ret != 0) {
+ ml_err("Failed to get input size, model : %s\n", opt->filelist[fid].model);
+ return ret;
+ }
+
+ /* allocate buffer for user data */
+ mz_size = t->model[fid].inp_dsize + t->model[fid].out_dsize;
+ sprintf(mz_name, "ml_user_data_%d", fid);
+ mz = rte_memzone_reserve(mz_name, mz_size, opt->socket_id, 0);
+ if (mz == NULL) {
+ ml_err("Memzone allocation failed for ml_user_data\n");
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ t->model[fid].input = mz->addr;
+ t->model[fid].output = RTE_PTR_ADD(t->model[fid].input, t->model[fid].inp_dsize);
+
+ /* load input file */
+ fp = fopen(opt->filelist[fid].input, "r");
+ if (fp == NULL) {
+ ml_err("Failed to open input file : %s\n", opt->filelist[fid].input);
+ ret = -errno;
+ goto error;
+ }
+
+ fseek(fp, 0, SEEK_END);
+ fsize = ftell(fp);
+ fseek(fp, 0, SEEK_SET);
+ if (fsize != t->model[fid].inp_dsize) {
+ ml_err("Invalid input file, size = %u (expected size = %" PRIu64 ")\n", fsize,
+ t->model[fid].inp_dsize);
+ ret = -EINVAL;
+ fclose(fp);
+ goto error;
+ }
+
+ if (fread(t->model[fid].input, 1, t->model[fid].inp_dsize, fp) != t->model[fid].inp_dsize) {
+ ml_err("Failed to read input file : %s\n", opt->filelist[fid].input);
+ ret = -errno;
+ fclose(fp);
+ goto error;
+ }
+ fclose(fp);
+
+ /* create mempool for quantized input and output buffers. ml_request_initialize is
+ * used as a callback for object creation.
+ */
+ buff_size = RTE_ALIGN_CEIL(sizeof(struct ml_request), t->cmn.dev_info.min_align_size) +
+ RTE_ALIGN_CEIL(t->model[fid].inp_qsize, t->cmn.dev_info.min_align_size) +
+ RTE_ALIGN_CEIL(t->model[fid].out_qsize, t->cmn.dev_info.min_align_size);
+ nb_buffers = RTE_MIN((uint64_t)ML_TEST_MAX_POOL_SIZE, opt->repetitions);
+
+ t->fid = fid;
+ sprintf(mp_name, "ml_io_pool_%d", fid);
+ t->model[fid].io_pool = rte_mempool_create(mp_name, nb_buffers, buff_size, 0, 0, NULL, NULL,
+ ml_request_initialize, test, opt->socket_id, 0);
+ if (t->model[fid].io_pool == NULL) {
+ ml_err("Failed to create io pool : %s\n", "ml_io_pool");
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ return 0;
+
+error:
+ if (mz != NULL)
+ rte_memzone_free(mz);
+
+ if (t->model[fid].io_pool != NULL) {
+ rte_mempool_free(t->model[fid].io_pool);
+ t->model[fid].io_pool = NULL;
+ }
+
+ return ret;
+}
+
+void
+ml_inference_iomem_destroy(struct ml_test *test, struct ml_options *opt, int16_t fid)
+{
+ char mz_name[RTE_MEMZONE_NAMESIZE];
+ char mp_name[RTE_MEMPOOL_NAMESIZE];
+ const struct rte_memzone *mz;
+ struct rte_mempool *mp;
+
+ RTE_SET_USED(test);
+ RTE_SET_USED(opt);
+
+ /* release user data memzone */
+ sprintf(mz_name, "ml_user_data_%d", fid);
+ mz = rte_memzone_lookup(mz_name);
+ if (mz != NULL)
+ rte_memzone_free(mz);
+
+ /* destroy io pool */
+ sprintf(mp_name, "ml_io_pool_%d", fid);
+ mp = rte_mempool_lookup(mp_name);
+ if (mp != NULL)
+ rte_mempool_free(mp);
+}
+
+int
+ml_inference_mem_setup(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t = ml_test_priv(test);
+
+ /* create op pool */
+ t->op_pool = rte_ml_op_pool_create("ml_test_op_pool", ML_TEST_MAX_POOL_SIZE, 0, 0,
+ opt->socket_id);
+ if (t->op_pool == NULL) {
+ ml_err("Failed to create op pool : %s\n", "ml_op_pool");
+ return -ENOMEM;
+ }
+
+ return 0;
+}
+
+void
+ml_inference_mem_destroy(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t = ml_test_priv(test);
+
+ RTE_SET_USED(opt);
+
+ /* release op pool */
+ if (t->op_pool != NULL)
+ rte_mempool_free(t->op_pool);
+}
+
+/* Callback for mempool object iteration. This call would dequantize output data. */
+static void
+ml_request_finish(struct rte_mempool *mp, void *opaque, void *obj, unsigned int obj_idx)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)opaque);
+ struct ml_request *req = (struct ml_request *)obj;
+ struct ml_model *model = &t->model[req->fid];
+
+ RTE_SET_USED(mp);
+ RTE_SET_USED(obj_idx);
+
+ if (req->niters == 0)
+ return;
+
+ t->nb_used++;
+ rte_ml_io_dequantize(t->cmn.opt->dev_id, model->id, t->model[req->fid].info.batch_size,
+ req->output, model->output);
+}
+
+int
+ml_inference_result(struct ml_test *test, struct ml_options *opt, int16_t fid)
+{
+ struct test_inference *t = ml_test_priv(test);
+
+ RTE_SET_USED(opt);
+
+ rte_mempool_obj_iter(t->model[fid].io_pool, ml_request_finish, test);
+
+ if (t->nb_used > 0)
+ t->cmn.result = ML_TEST_SUCCESS;
+ else
+ t->cmn.result = ML_TEST_FAILED;
+
+ return t->cmn.result;
+}
+
+int
+ml_inference_launch_cores(struct ml_test *test, struct ml_options *opt, int16_t start_fid,
+ int16_t end_fid)
+{
+ struct test_inference *t = ml_test_priv(test);
+ uint32_t lcore_id;
+ uint32_t id = 0;
+
+ RTE_LCORE_FOREACH_WORKER(lcore_id)
+ {
+ if (id == 2)
+ break;
+
+ t->args[lcore_id].nb_reqs = opt->repetitions;
+ t->args[lcore_id].start_fid = start_fid;
+ t->args[lcore_id].end_fid = end_fid;
+
+ if (id % 2 == 0)
+ rte_eal_remote_launch(t->enqueue, test, lcore_id);
+ else
+ rte_eal_remote_launch(t->dequeue, test, lcore_id);
+
+ id++;
+ }
+
+ return 0;
+}
diff --git a/app/test-mldev/test_inference_common.h b/app/test-mldev/test_inference_common.h
new file mode 100644
index 0000000000..91007954b4
--- /dev/null
+++ b/app/test-mldev/test_inference_common.h
@@ -0,0 +1,65 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_TEST_INFERENCE_COMMON_
+#define _ML_TEST_INFERENCE_COMMON_
+
+#include <stdbool.h>
+#include <stdint.h>
+
+#include <rte_common.h>
+#include <rte_mempool.h>
+
+#include "ml_options.h"
+#include "ml_test.h"
+#include "test_common.h"
+#include "test_model_common.h"
+
+struct ml_request {
+ void *input;
+ void *output;
+ int16_t fid;
+ uint64_t niters;
+};
+
+struct ml_core_args {
+ uint64_t nb_reqs;
+ int16_t start_fid;
+ int16_t end_fid;
+};
+
+struct test_inference {
+ /* common data */
+ struct test_common cmn;
+
+ /* test specific data */
+ struct ml_model model[ML_TEST_MAX_MODELS];
+ struct rte_mempool *op_pool;
+
+ uint64_t nb_used;
+ int16_t fid;
+
+ int (*enqueue)(void *arg);
+ int (*dequeue)(void *arg);
+
+ struct ml_core_args args[RTE_MAX_LCORE];
+} __rte_cache_aligned;
+
+bool test_inference_cap_check(struct ml_options *opt);
+int test_inference_opt_check(struct ml_options *opt);
+void test_inference_opt_dump(struct ml_options *opt);
+int test_inference_setup(struct ml_test *test, struct ml_options *opt);
+void test_inference_destroy(struct ml_test *test, struct ml_options *opt);
+
+int ml_inference_mldev_setup(struct ml_test *test, struct ml_options *opt);
+int ml_inference_mldev_destroy(struct ml_test *test, struct ml_options *opt);
+int ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, int16_t fid);
+void ml_inference_iomem_destroy(struct ml_test *test, struct ml_options *opt, int16_t fid);
+int ml_inference_mem_setup(struct ml_test *test, struct ml_options *opt);
+void ml_inference_mem_destroy(struct ml_test *test, struct ml_options *opt);
+int ml_inference_result(struct ml_test *test, struct ml_options *opt, int16_t fid);
+int ml_inference_launch_cores(struct ml_test *test, struct ml_options *opt, int16_t start_fid,
+ int16_t end_fid);
+
+#endif /* _ML_TEST_INFERENCE_COMMON_ */
diff --git a/app/test-mldev/test_inference_ordered.c b/app/test-mldev/test_inference_ordered.c
new file mode 100644
index 0000000000..84e6bf9109
--- /dev/null
+++ b/app/test-mldev/test_inference_ordered.c
@@ -0,0 +1,119 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <stdint.h>
+
+#include <rte_common.h>
+#include <rte_launch.h>
+
+#include "ml_common.h"
+#include "ml_test.h"
+#include "test_inference_common.h"
+#include "test_model_common.h"
+
+static int
+test_inference_ordered_driver(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t;
+ int16_t fid = 0;
+ int ret = 0;
+
+ t = ml_test_priv(test);
+
+ ret = ml_inference_mldev_setup(test, opt);
+ if (ret != 0)
+ return ret;
+
+ ret = ml_inference_mem_setup(test, opt);
+ if (ret != 0)
+ return ret;
+
+next_model:
+ /* load model */
+ ret = ml_model_load(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ /* start model */
+ ret = ml_model_start(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_inference_iomem_setup(test, opt, fid);
+ if (ret != 0)
+ goto error;
+
+ /* launch inferences for one model using available queue pairs */
+ ret = ml_inference_launch_cores(test, opt, fid, fid);
+ if (ret != 0) {
+ ml_err("failed to launch cores");
+ goto error;
+ }
+
+ rte_eal_mp_wait_lcore();
+
+ ret = ml_inference_result(test, opt, fid);
+ if (ret != ML_TEST_SUCCESS)
+ goto error;
+
+ ml_inference_iomem_destroy(test, opt, fid);
+
+ /* stop model */
+ ret = ml_model_stop(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ /* unload model */
+ ret = ml_model_unload(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ fid++;
+ if (fid < opt->nb_filelist)
+ goto next_model;
+
+ ml_inference_mem_destroy(test, opt);
+
+ ret = ml_inference_mldev_destroy(test, opt);
+ if (ret != 0)
+ return ret;
+
+ t->cmn.result = ML_TEST_SUCCESS;
+
+ return 0;
+
+error:
+ ml_inference_iomem_destroy(test, opt, fid);
+ ml_inference_mem_destroy(test, opt);
+ ml_model_stop(test, opt, &t->model[fid], fid);
+ ml_model_unload(test, opt, &t->model[fid], fid);
+
+ t->cmn.result = ML_TEST_FAILED;
+
+ return ret;
+}
+
+static int
+test_inference_ordered_result(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+
+ return t->cmn.result;
+}
+
+static const struct ml_test_ops inference_ordered = {
+ .cap_check = test_inference_cap_check,
+ .opt_check = test_inference_opt_check,
+ .opt_dump = test_inference_opt_dump,
+ .test_setup = test_inference_setup,
+ .test_destroy = test_inference_destroy,
+ .test_driver = test_inference_ordered_driver,
+ .test_result = test_inference_ordered_result,
+};
+
+ML_TEST_REGISTER(inference_ordered);
diff --git a/app/test-mldev/test_model_common.h b/app/test-mldev/test_model_common.h
index 302e4eb45f..c45ae80853 100644
--- a/app/test-mldev/test_model_common.h
+++ b/app/test-mldev/test_model_common.h
@@ -23,6 +23,16 @@ struct ml_model {
int16_t id;
struct rte_ml_model_info info;
enum model_state state;
+
+ uint64_t inp_dsize;
+ uint64_t inp_qsize;
+ uint64_t out_dsize;
+ uint64_t out_qsize;
+
+ uint8_t *input;
+ uint8_t *output;
+
+ struct rte_mempool *io_pool;
};
int ml_model_load(struct ml_test *test, struct ml_options *opt, struct ml_model *model,
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v2 06/12] app/mldev: add test case to interleave inferences
2022-11-29 8:20 ` [PATCH v2 " Srikanth Yalavarthi
` (3 preceding siblings ...)
2022-11-29 8:21 ` [PATCH v2 05/12] app/mldev: add ordered inference test case Srikanth Yalavarthi
@ 2022-11-29 8:21 ` Srikanth Yalavarthi
2022-11-29 8:21 ` [PATCH v2 07/12] app/mldev: enable support for burst inferences Srikanth Yalavarthi
` (6 subsequent siblings)
11 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2022-11-29 8:21 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj
Added test case to interleave inference requests from multiple
models. Interleaving would load and start all models and launch
inference requests for the models using available queue-pairs
Operations sequence when testing with N models and R reps,
(load + start) x N -> (enqueue + dequeue) x N x R ...
-> (stop + unload) x N
Test can be executed by selecting "inference_interleave" test.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/meson.build | 1 +
app/test-mldev/ml_options.c | 3 +-
app/test-mldev/test_inference_common.c | 12 +--
app/test-mldev/test_inference_common.h | 4 +-
app/test-mldev/test_inference_interleave.c | 118 +++++++++++++++++++++
5 files changed, 129 insertions(+), 9 deletions(-)
create mode 100644 app/test-mldev/test_inference_interleave.c
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
index 475d76d126..41d22fb22c 100644
--- a/app/test-mldev/meson.build
+++ b/app/test-mldev/meson.build
@@ -18,6 +18,7 @@ sources = files(
'test_model_ops.c',
'test_inference_common.c',
'test_inference_ordered.c',
+ 'test_inference_interleave.c',
)
deps += ['mldev']
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 10dad18fff..01ea050ee7 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -162,7 +162,8 @@ ml_dump_test_options(const char *testname)
printf("\n");
}
- if (strcmp(testname, "inference_ordered") == 0) {
+ if ((strcmp(testname, "inference_ordered") == 0) ||
+ (strcmp(testname, "inference_interleave") == 0)) {
printf("\t\t--filelist : comma separated list of model, input and output\n"
"\t\t--repetitions : number of inference repetitions\n");
printf("\n");
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
index 8b5dc89346..f0b15861a0 100644
--- a/app/test-mldev/test_inference_common.c
+++ b/app/test-mldev/test_inference_common.c
@@ -115,7 +115,7 @@ ml_dequeue_single(void *arg)
total_deq += burst_deq;
if (unlikely(op->status == RTE_ML_OP_STATUS_ERROR)) {
rte_ml_op_error_get(t->cmn.opt->dev_id, op, &error);
- ml_err("error_code = 0x%016lx, error_message = %s\n", error.errcode,
+ ml_err("error_code = 0x%" PRIx64 ", error_message = %s\n", error.errcode,
error.message);
}
req = (struct ml_request *)op->user_ptr;
@@ -334,10 +334,10 @@ ml_request_initialize(struct rte_mempool *mp, void *opaque, void *obj, unsigned
RTE_SET_USED(mp);
RTE_SET_USED(obj_idx);
- req->input = RTE_PTR_ADD(
- obj, RTE_ALIGN_CEIL(sizeof(struct ml_request), t->cmn.dev_info.min_align_size));
- req->output = RTE_PTR_ADD(req->input, RTE_ALIGN_CEIL(t->model[t->fid].inp_qsize,
- t->cmn.dev_info.min_align_size));
+ req->input = (uint8_t *)obj +
+ RTE_ALIGN_CEIL(sizeof(struct ml_request), t->cmn.dev_info.min_align_size);
+ req->output = req->input +
+ RTE_ALIGN_CEIL(t->model[t->fid].inp_qsize, t->cmn.dev_info.min_align_size);
req->niters = 0;
/* quantize data */
@@ -387,7 +387,7 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, int16_t f
}
t->model[fid].input = mz->addr;
- t->model[fid].output = RTE_PTR_ADD(t->model[fid].input, t->model[fid].inp_dsize);
+ t->model[fid].output = t->model[fid].input + t->model[fid].inp_dsize;
/* load input file */
fp = fopen(opt->filelist[fid].input, "r");
diff --git a/app/test-mldev/test_inference_common.h b/app/test-mldev/test_inference_common.h
index 91007954b4..b058abada4 100644
--- a/app/test-mldev/test_inference_common.h
+++ b/app/test-mldev/test_inference_common.h
@@ -17,8 +17,8 @@
#include "test_model_common.h"
struct ml_request {
- void *input;
- void *output;
+ uint8_t *input;
+ uint8_t *output;
int16_t fid;
uint64_t niters;
};
diff --git a/app/test-mldev/test_inference_interleave.c b/app/test-mldev/test_inference_interleave.c
new file mode 100644
index 0000000000..74ad0c597f
--- /dev/null
+++ b/app/test-mldev/test_inference_interleave.c
@@ -0,0 +1,118 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <stdint.h>
+
+#include <rte_common.h>
+#include <rte_launch.h>
+
+#include "ml_common.h"
+#include "ml_test.h"
+#include "test_inference_common.h"
+#include "test_model_common.h"
+
+static int
+test_inference_interleave_driver(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t;
+ int16_t fid = 0;
+ int ret = 0;
+
+ t = ml_test_priv(test);
+
+ ret = ml_inference_mldev_setup(test, opt);
+ if (ret != 0)
+ return ret;
+
+ ret = ml_inference_mem_setup(test, opt);
+ if (ret != 0)
+ return ret;
+
+ /* load and start all models */
+ for (fid = 0; fid < opt->nb_filelist; fid++) {
+ ret = ml_model_load(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_start(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_inference_iomem_setup(test, opt, fid);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* launch inference requests */
+ ret = ml_inference_launch_cores(test, opt, 0, opt->nb_filelist - 1);
+ if (ret != 0) {
+ ml_err("failed to launch cores");
+ goto error;
+ }
+
+ rte_eal_mp_wait_lcore();
+
+ /* stop and unload all models */
+ for (fid = 0; fid < opt->nb_filelist; fid++) {
+ ret = ml_inference_result(test, opt, fid);
+ if (ret != ML_TEST_SUCCESS)
+ goto error;
+
+ ml_inference_iomem_destroy(test, opt, fid);
+
+ ret = ml_model_stop(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_unload(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+ }
+
+ ml_inference_mem_destroy(test, opt);
+
+ ret = ml_inference_mldev_destroy(test, opt);
+ if (ret != 0)
+ return ret;
+
+ t->cmn.result = ML_TEST_SUCCESS;
+
+ return 0;
+
+error:
+ ml_inference_mem_destroy(test, opt);
+ for (fid = 0; fid < opt->nb_filelist; fid++) {
+ ml_inference_iomem_destroy(test, opt, fid);
+ ml_model_stop(test, opt, &t->model[fid], fid);
+ ml_model_unload(test, opt, &t->model[fid], fid);
+ }
+
+ t->cmn.result = ML_TEST_FAILED;
+
+ return ret;
+}
+
+static int
+test_inference_interleave_result(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+
+ return t->cmn.result;
+}
+
+static const struct ml_test_ops inference_interleave = {
+ .cap_check = test_inference_cap_check,
+ .opt_check = test_inference_opt_check,
+ .opt_dump = test_inference_opt_dump,
+ .test_setup = test_inference_setup,
+ .test_destroy = test_inference_destroy,
+ .test_driver = test_inference_interleave_driver,
+ .test_result = test_inference_interleave_result,
+};
+
+ML_TEST_REGISTER(inference_interleave);
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v2 07/12] app/mldev: enable support for burst inferences
2022-11-29 8:20 ` [PATCH v2 " Srikanth Yalavarthi
` (4 preceding siblings ...)
2022-11-29 8:21 ` [PATCH v2 06/12] app/mldev: add test case to interleave inferences Srikanth Yalavarthi
@ 2022-11-29 8:21 ` Srikanth Yalavarthi
2022-11-29 8:21 ` [PATCH v2 08/12] app/mldev: enable support for queue pairs and size Srikanth Yalavarthi
` (5 subsequent siblings)
11 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2022-11-29 8:21 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj
Added 'burst_size' support for inference tests. Burst size
controls the number of inference requests handled during
the burst enqueue and dequeue operations of the test case.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/ml_options.c | 26 ++--
app/test-mldev/ml_options.h | 2 +
app/test-mldev/test_inference_common.c | 159 ++++++++++++++++++++++++-
app/test-mldev/test_inference_common.h | 4 +
4 files changed, 181 insertions(+), 10 deletions(-)
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 01ea050ee7..1990939200 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -30,6 +30,7 @@ ml_options_default(struct ml_options *opt)
opt->socket_id = SOCKET_ID_ANY;
opt->nb_filelist = 0;
opt->repetitions = 1;
+ opt->burst_size = 1;
opt->debug = false;
}
@@ -151,6 +152,12 @@ ml_parse_repetitions(struct ml_options *opt, const char *arg)
return parser_read_uint64(&opt->repetitions, arg);
}
+static int
+ml_parse_burst_size(struct ml_options *opt, const char *arg)
+{
+ return parser_read_uint16(&opt->burst_size, arg);
+}
+
static void
ml_dump_test_options(const char *testname)
{
@@ -165,7 +172,8 @@ ml_dump_test_options(const char *testname)
if ((strcmp(testname, "inference_ordered") == 0) ||
(strcmp(testname, "inference_interleave") == 0)) {
printf("\t\t--filelist : comma separated list of model, input and output\n"
- "\t\t--repetitions : number of inference repetitions\n");
+ "\t\t--repetitions : number of inference repetitions\n"
+ "\t\t--burst_size : inference burst size\n");
printf("\n");
}
}
@@ -185,10 +193,11 @@ print_usage(char *program)
ml_test_dump_names(ml_dump_test_options);
}
-static struct option lgopts[] = {
- {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
- {ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
- {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
+static struct option lgopts[] = {{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0},
+ {ML_SOCKET_ID, 1, 0, 0}, {ML_MODELS, 1, 0, 0},
+ {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
+ {ML_BURST_SIZE, 1, 0, 0}, {ML_DEBUG, 0, 0, 0},
+ {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
static int
ml_opts_parse_long(int opt_idx, struct ml_options *opt)
@@ -196,9 +205,10 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
unsigned int i;
struct long_opt_parser parsermap[] = {
- {ML_TEST, ml_parse_test_name}, {ML_DEVICE_ID, ml_parse_dev_id},
- {ML_SOCKET_ID, ml_parse_socket_id}, {ML_MODELS, ml_parse_models},
- {ML_FILELIST, ml_parse_filelist}, {ML_REPETITIONS, ml_parse_repetitions},
+ {ML_TEST, ml_parse_test_name}, {ML_DEVICE_ID, ml_parse_dev_id},
+ {ML_SOCKET_ID, ml_parse_socket_id}, {ML_MODELS, ml_parse_models},
+ {ML_FILELIST, ml_parse_filelist}, {ML_REPETITIONS, ml_parse_repetitions},
+ {ML_BURST_SIZE, ml_parse_burst_size},
};
for (i = 0; i < RTE_DIM(parsermap); i++) {
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index ad8aee5964..305b39629a 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -19,6 +19,7 @@
#define ML_MODELS ("models")
#define ML_FILELIST ("filelist")
#define ML_REPETITIONS ("repetitions")
+#define ML_BURST_SIZE ("burst_size")
#define ML_DEBUG ("debug")
#define ML_HELP ("help")
@@ -35,6 +36,7 @@ struct ml_options {
struct ml_filelist filelist[ML_TEST_MAX_MODELS];
uint8_t nb_filelist;
uint64_t repetitions;
+ uint16_t burst_size;
bool debug;
};
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
index f0b15861a0..252d77616c 100644
--- a/app/test-mldev/test_inference_common.c
+++ b/app/test-mldev/test_inference_common.c
@@ -129,6 +129,131 @@ ml_dequeue_single(void *arg)
return 0;
}
+/* Enqueue inference requests with burst size greater than 1 */
+static int
+ml_enqueue_burst(void *arg)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)arg);
+ struct ml_core_args *args;
+ uint16_t ops_count;
+ uint64_t model_enq;
+ uint16_t burst_enq;
+ uint32_t lcore_id;
+ uint16_t pending;
+ uint16_t idx;
+ int16_t fid;
+ uint16_t i;
+ int ret;
+
+ lcore_id = rte_lcore_id();
+ args = &t->args[lcore_id];
+ model_enq = 0;
+
+ if (args->nb_reqs == 0)
+ return 0;
+
+next_rep:
+ fid = args->start_fid;
+
+next_model:
+ ops_count = RTE_MIN(t->cmn.opt->burst_size, args->nb_reqs - model_enq);
+ ret = rte_mempool_get_bulk(t->op_pool, (void **)args->enq_ops, ops_count);
+ if (ret != 0)
+ goto next_model;
+
+retry:
+ ret = rte_mempool_get_bulk(t->model[fid].io_pool, (void **)args->reqs, ops_count);
+ if (ret != 0)
+ goto retry;
+
+ for (i = 0; i < ops_count; i++) {
+ args->enq_ops[i]->model_id = t->model[fid].id;
+ args->enq_ops[i]->nb_batches = t->model[fid].info.batch_size;
+ args->enq_ops[i]->mempool = t->op_pool;
+
+ args->enq_ops[i]->input.addr = args->reqs[i]->input;
+ args->enq_ops[i]->input.length = t->model[fid].inp_qsize;
+ args->enq_ops[i]->input.next = NULL;
+
+ args->enq_ops[i]->output.addr = args->reqs[i]->output;
+ args->enq_ops[i]->output.length = t->model[fid].out_qsize;
+ args->enq_ops[i]->output.next = NULL;
+
+ args->enq_ops[i]->user_ptr = args->reqs[i];
+ args->reqs[i]->niters++;
+ args->reqs[i]->fid = fid;
+ }
+
+ idx = 0;
+ pending = ops_count;
+
+enqueue_reqs:
+ burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, 0, &args->enq_ops[idx], pending);
+ pending = pending - burst_enq;
+
+ if (pending > 0) {
+ idx = idx + burst_enq;
+ goto enqueue_reqs;
+ }
+
+ fid++;
+ if (fid <= args->end_fid)
+ goto next_model;
+
+ model_enq = model_enq + ops_count;
+ if (model_enq < args->nb_reqs)
+ goto next_rep;
+
+ return 0;
+}
+
+/* Dequeue inference requests with burst size greater than 1 */
+static int
+ml_dequeue_burst(void *arg)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)arg);
+ struct rte_ml_op_error error;
+ struct ml_core_args *args;
+ struct ml_request *req;
+ uint64_t total_deq = 0;
+ uint16_t burst_deq = 0;
+ uint8_t nb_filelist;
+ uint32_t lcore_id;
+ uint32_t i;
+
+ lcore_id = rte_lcore_id();
+ args = &t->args[lcore_id];
+ nb_filelist = args->end_fid - args->start_fid + 1;
+
+ if (args->nb_reqs == 0)
+ return 0;
+
+dequeue_burst:
+ burst_deq =
+ rte_ml_dequeue_burst(t->cmn.opt->dev_id, 0, args->deq_ops, t->cmn.opt->burst_size);
+
+ if (likely(burst_deq > 0)) {
+ total_deq += burst_deq;
+
+ for (i = 0; i < burst_deq; i++) {
+ if (unlikely(args->deq_ops[i]->status == RTE_ML_OP_STATUS_ERROR)) {
+ rte_ml_op_error_get(t->cmn.opt->dev_id, args->deq_ops[i], &error);
+ ml_err("error_code = 0x%" PRIx64 ", error_message = %s\n",
+ error.errcode, error.message);
+ }
+ req = (struct ml_request *)args->deq_ops[i]->user_ptr;
+ if (req != NULL)
+ rte_mempool_put(t->model[req->fid].io_pool, req);
+ }
+ rte_mempool_put_bulk(t->op_pool, (void *)args->deq_ops, burst_deq);
+ }
+
+ if (total_deq < args->nb_reqs * nb_filelist)
+ goto dequeue_burst;
+
+ return 0;
+}
+
bool
test_inference_cap_check(struct ml_options *opt)
{
@@ -178,6 +303,17 @@ test_inference_opt_check(struct ml_options *opt)
return -EINVAL;
}
+ if (opt->burst_size == 0) {
+ ml_err("Invalid option, burst_size = %u\n", opt->burst_size);
+ return -EINVAL;
+ }
+
+ if (opt->burst_size > ML_TEST_MAX_POOL_SIZE) {
+ ml_err("Invalid option, burst_size = %u (> max supported = %d)\n", opt->burst_size,
+ ML_TEST_MAX_POOL_SIZE);
+ return -EINVAL;
+ }
+
/* check number of available lcores. */
if (rte_lcore_count() < 3) {
ml_err("Insufficient lcores = %u\n", rte_lcore_count());
@@ -198,6 +334,7 @@ test_inference_opt_dump(struct ml_options *opt)
/* dump test opts */
ml_dump("repetitions", "%" PRIu64, opt->repetitions);
+ ml_dump("burst_size", "%u", opt->burst_size);
ml_dump_begin("filelist");
for (i = 0; i < opt->nb_filelist; i++) {
@@ -213,6 +350,7 @@ test_inference_setup(struct ml_test *test, struct ml_options *opt)
{
struct test_inference *t;
void *test_inference;
+ uint32_t lcore_id;
int ret = 0;
uint32_t i;
@@ -237,13 +375,30 @@ test_inference_setup(struct ml_test *test, struct ml_options *opt)
goto error;
}
- t->enqueue = ml_enqueue_single;
- t->dequeue = ml_dequeue_single;
+ if (opt->burst_size == 1) {
+ t->enqueue = ml_enqueue_single;
+ t->dequeue = ml_dequeue_single;
+ } else {
+ t->enqueue = ml_enqueue_burst;
+ t->dequeue = ml_dequeue_burst;
+ }
/* set model initial state */
for (i = 0; i < opt->nb_filelist; i++)
t->model[i].state = MODEL_INITIAL;
+ for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) {
+ t->args[lcore_id].enq_ops = rte_zmalloc_socket(
+ "ml_test_enq_ops", opt->burst_size * sizeof(struct rte_ml_op *),
+ RTE_CACHE_LINE_SIZE, opt->socket_id);
+ t->args[lcore_id].deq_ops = rte_zmalloc_socket(
+ "ml_test_deq_ops", opt->burst_size * sizeof(struct rte_ml_op *),
+ RTE_CACHE_LINE_SIZE, opt->socket_id);
+ t->args[lcore_id].reqs = rte_zmalloc_socket(
+ "ml_test_requests", opt->burst_size * sizeof(struct ml_request *),
+ RTE_CACHE_LINE_SIZE, opt->socket_id);
+ }
+
return 0;
error:
diff --git a/app/test-mldev/test_inference_common.h b/app/test-mldev/test_inference_common.h
index b058abada4..75d588308b 100644
--- a/app/test-mldev/test_inference_common.h
+++ b/app/test-mldev/test_inference_common.h
@@ -27,6 +27,10 @@ struct ml_core_args {
uint64_t nb_reqs;
int16_t start_fid;
int16_t end_fid;
+
+ struct rte_ml_op **enq_ops;
+ struct rte_ml_op **deq_ops;
+ struct ml_request **reqs;
};
struct test_inference {
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v2 08/12] app/mldev: enable support for queue pairs and size
2022-11-29 8:20 ` [PATCH v2 " Srikanth Yalavarthi
` (5 preceding siblings ...)
2022-11-29 8:21 ` [PATCH v2 07/12] app/mldev: enable support for burst inferences Srikanth Yalavarthi
@ 2022-11-29 8:21 ` Srikanth Yalavarthi
2022-11-29 8:21 ` [PATCH v2 09/12] app/mldev: enable support for inference batches Srikanth Yalavarthi
` (4 subsequent siblings)
11 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2022-11-29 8:21 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj
Added support to create multiple queue-pairs per device to
enqueue and dequeue inference requests. Number of queue pairs
to be created can be specified through "--queue_pairs" option.
Support is also enabled to control the number of descriptors
per each queue pair through "--queue_size" option. Inference
requests for a model are distributed across all available
queue-pairs.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/ml_options.c | 40 ++++++++++---
app/test-mldev/ml_options.h | 4 ++
app/test-mldev/test_common.c | 2 +-
app/test-mldev/test_inference_common.c | 79 +++++++++++++++++++++-----
app/test-mldev/test_inference_common.h | 1 +
5 files changed, 102 insertions(+), 24 deletions(-)
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 1990939200..d5182a1701 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -31,6 +31,8 @@ ml_options_default(struct ml_options *opt)
opt->nb_filelist = 0;
opt->repetitions = 1;
opt->burst_size = 1;
+ opt->queue_pairs = 1;
+ opt->queue_size = 1;
opt->debug = false;
}
@@ -158,11 +160,30 @@ ml_parse_burst_size(struct ml_options *opt, const char *arg)
return parser_read_uint16(&opt->burst_size, arg);
}
+static int
+ml_parse_queue_pairs(struct ml_options *opt, const char *arg)
+{
+ int ret;
+
+ ret = parser_read_uint16(&opt->queue_pairs, arg);
+
+ return ret;
+}
+
+static int
+ml_parse_queue_size(struct ml_options *opt, const char *arg)
+{
+ return parser_read_uint16(&opt->queue_size, arg);
+}
+
static void
ml_dump_test_options(const char *testname)
{
- if (strcmp(testname, "device_ops") == 0)
+ if (strcmp(testname, "device_ops") == 0) {
+ printf("\t\t--queue_pairs : number of queue pairs to create\n"
+ "\t\t--queue_size : size fo queue-pair\n");
printf("\n");
+ }
if (strcmp(testname, "model_ops") == 0) {
printf("\t\t--models : comma separated list of models\n");
@@ -173,7 +194,9 @@ ml_dump_test_options(const char *testname)
(strcmp(testname, "inference_interleave") == 0)) {
printf("\t\t--filelist : comma separated list of model, input and output\n"
"\t\t--repetitions : number of inference repetitions\n"
- "\t\t--burst_size : inference burst size\n");
+ "\t\t--burst_size : inference burst size\n"
+ "\t\t--queue_pairs : number of queue pairs to create\n"
+ "\t\t--queue_size : size fo queue-pair\n");
printf("\n");
}
}
@@ -193,11 +216,11 @@ print_usage(char *program)
ml_test_dump_names(ml_dump_test_options);
}
-static struct option lgopts[] = {{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0},
- {ML_SOCKET_ID, 1, 0, 0}, {ML_MODELS, 1, 0, 0},
- {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
- {ML_BURST_SIZE, 1, 0, 0}, {ML_DEBUG, 0, 0, 0},
- {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
+static struct option lgopts[] = {
+ {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
+ {ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
+ {ML_BURST_SIZE, 1, 0, 0}, {ML_QUEUE_PAIRS, 1, 0, 0}, {ML_QUEUE_SIZE, 1, 0, 0},
+ {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
static int
ml_opts_parse_long(int opt_idx, struct ml_options *opt)
@@ -208,7 +231,8 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
{ML_TEST, ml_parse_test_name}, {ML_DEVICE_ID, ml_parse_dev_id},
{ML_SOCKET_ID, ml_parse_socket_id}, {ML_MODELS, ml_parse_models},
{ML_FILELIST, ml_parse_filelist}, {ML_REPETITIONS, ml_parse_repetitions},
- {ML_BURST_SIZE, ml_parse_burst_size},
+ {ML_BURST_SIZE, ml_parse_burst_size}, {ML_QUEUE_PAIRS, ml_parse_queue_pairs},
+ {ML_QUEUE_SIZE, ml_parse_queue_size},
};
for (i = 0; i < RTE_DIM(parsermap); i++) {
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index 305b39629a..6bfef1b979 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -20,6 +20,8 @@
#define ML_FILELIST ("filelist")
#define ML_REPETITIONS ("repetitions")
#define ML_BURST_SIZE ("burst_size")
+#define ML_QUEUE_PAIRS ("queue_pairs")
+#define ML_QUEUE_SIZE ("queue_size")
#define ML_DEBUG ("debug")
#define ML_HELP ("help")
@@ -37,6 +39,8 @@ struct ml_options {
uint8_t nb_filelist;
uint64_t repetitions;
uint16_t burst_size;
+ uint16_t queue_pairs;
+ uint16_t queue_size;
bool debug;
};
diff --git a/app/test-mldev/test_common.c b/app/test-mldev/test_common.c
index b6b32904e4..22e6acb3b6 100644
--- a/app/test-mldev/test_common.c
+++ b/app/test-mldev/test_common.c
@@ -78,7 +78,7 @@ ml_test_device_configure(struct ml_test *test, struct ml_options *opt)
/* configure device */
dev_config.socket_id = opt->socket_id;
dev_config.nb_models = t->dev_info.max_models;
- dev_config.nb_queue_pairs = t->dev_info.max_queue_pairs;
+ dev_config.nb_queue_pairs = opt->queue_pairs;
ret = rte_ml_dev_configure(opt->dev_id, &dev_config);
if (ret != 0) {
ml_err("Failed to configure ml device, dev_id = %d\n", opt->dev_id);
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
index 252d77616c..d680d68f3d 100644
--- a/app/test-mldev/test_inference_common.c
+++ b/app/test-mldev/test_inference_common.c
@@ -72,7 +72,7 @@ ml_enqueue_single(void *arg)
req->fid = fid;
enqueue_req:
- burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, 0, &op, 1);
+ burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, args->qp_id, &op, 1);
if (burst_enq == 0)
goto enqueue_req;
@@ -109,7 +109,7 @@ ml_dequeue_single(void *arg)
return 0;
dequeue_req:
- burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, 0, &op, 1);
+ burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, args->qp_id, &op, 1);
if (likely(burst_deq == 1)) {
total_deq += burst_deq;
@@ -188,7 +188,8 @@ ml_enqueue_burst(void *arg)
pending = ops_count;
enqueue_reqs:
- burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, 0, &args->enq_ops[idx], pending);
+ burst_enq =
+ rte_ml_enqueue_burst(t->cmn.opt->dev_id, args->qp_id, &args->enq_ops[idx], pending);
pending = pending - burst_enq;
if (pending > 0) {
@@ -229,8 +230,8 @@ ml_dequeue_burst(void *arg)
return 0;
dequeue_burst:
- burst_deq =
- rte_ml_dequeue_burst(t->cmn.opt->dev_id, 0, args->deq_ops, t->cmn.opt->burst_size);
+ burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, args->qp_id, args->deq_ops,
+ t->cmn.opt->burst_size);
if (likely(burst_deq > 0)) {
total_deq += burst_deq;
@@ -263,6 +264,19 @@ test_inference_cap_check(struct ml_options *opt)
return false;
rte_ml_dev_info_get(opt->dev_id, &dev_info);
+
+ if (opt->queue_pairs > dev_info.max_queue_pairs) {
+ ml_err("Insufficient capabilities: queue_pairs = %u, max_queue_pairs = %u",
+ opt->queue_pairs, dev_info.max_queue_pairs);
+ return false;
+ }
+
+ if (opt->queue_size > dev_info.max_desc) {
+ ml_err("Insufficient capabilities: queue_size = %u, max_desc = %u", opt->queue_size,
+ dev_info.max_desc);
+ return false;
+ }
+
if (opt->nb_filelist > dev_info.max_models) {
ml_err("Insufficient capabilities: Filelist count exceeded device limit, count = %u (max limit = %u)",
opt->nb_filelist, dev_info.max_models);
@@ -314,10 +328,21 @@ test_inference_opt_check(struct ml_options *opt)
return -EINVAL;
}
+ if (opt->queue_pairs == 0) {
+ ml_err("Invalid option, queue_pairs = %u\n", opt->queue_pairs);
+ return -EINVAL;
+ }
+
+ if (opt->queue_size == 0) {
+ ml_err("Invalid option, queue_size = %u\n", opt->queue_size);
+ return -EINVAL;
+ }
+
/* check number of available lcores. */
- if (rte_lcore_count() < 3) {
+ if (rte_lcore_count() < (uint32_t)(opt->queue_pairs * 2 + 1)) {
ml_err("Insufficient lcores = %u\n", rte_lcore_count());
- ml_err("Minimum lcores required to create %u queue-pairs = %u\n", 1, 3);
+ ml_err("Minimum lcores required to create %u queue-pairs = %u\n", opt->queue_pairs,
+ (opt->queue_pairs * 2 + 1));
return -EINVAL;
}
@@ -335,6 +360,8 @@ test_inference_opt_dump(struct ml_options *opt)
/* dump test opts */
ml_dump("repetitions", "%" PRIu64, opt->repetitions);
ml_dump("burst_size", "%u", opt->burst_size);
+ ml_dump("queue_pairs", "%u", opt->queue_pairs);
+ ml_dump("queue_size", "%u", opt->queue_size);
ml_dump_begin("filelist");
for (i = 0; i < opt->nb_filelist; i++) {
@@ -425,23 +452,31 @@ ml_inference_mldev_setup(struct ml_test *test, struct ml_options *opt)
{
struct rte_ml_dev_qp_conf qp_conf;
struct test_inference *t;
+ uint16_t qp_id;
int ret;
t = ml_test_priv(test);
+ RTE_SET_USED(t);
+
ret = ml_test_device_configure(test, opt);
if (ret != 0)
return ret;
/* setup queue pairs */
- qp_conf.nb_desc = t->cmn.dev_info.max_desc;
+ qp_conf.nb_desc = opt->queue_size;
qp_conf.cb = NULL;
- ret = rte_ml_dev_queue_pair_setup(opt->dev_id, 0, &qp_conf, opt->socket_id);
- if (ret != 0) {
- ml_err("Failed to setup ml device queue-pair, dev_id = %d, qp_id = %u\n",
- opt->dev_id, 0);
- goto error;
+ for (qp_id = 0; qp_id < opt->queue_pairs; qp_id++) {
+ qp_conf.nb_desc = opt->queue_size;
+ qp_conf.cb = NULL;
+
+ ret = rte_ml_dev_queue_pair_setup(opt->dev_id, qp_id, &qp_conf, opt->socket_id);
+ if (ret != 0) {
+ ml_err("Failed to setup ml device queue-pair, dev_id = %d, qp_id = %u\n",
+ opt->dev_id, qp_id);
+ return ret;
+ }
}
ret = ml_test_device_start(test, opt);
@@ -697,14 +732,28 @@ ml_inference_launch_cores(struct ml_test *test, struct ml_options *opt, int16_t
{
struct test_inference *t = ml_test_priv(test);
uint32_t lcore_id;
+ uint32_t nb_reqs;
uint32_t id = 0;
+ uint32_t qp_id;
+
+ nb_reqs = opt->repetitions / opt->queue_pairs;
RTE_LCORE_FOREACH_WORKER(lcore_id)
{
- if (id == 2)
+ if (id >= opt->queue_pairs * 2)
break;
- t->args[lcore_id].nb_reqs = opt->repetitions;
+ qp_id = id / 2;
+ t->args[lcore_id].qp_id = qp_id;
+ t->args[lcore_id].nb_reqs = nb_reqs;
+ if (qp_id == 0)
+ t->args[lcore_id].nb_reqs += opt->repetitions - nb_reqs * opt->queue_pairs;
+
+ if (t->args[lcore_id].nb_reqs == 0) {
+ id++;
+ break;
+ }
+
t->args[lcore_id].start_fid = start_fid;
t->args[lcore_id].end_fid = end_fid;
diff --git a/app/test-mldev/test_inference_common.h b/app/test-mldev/test_inference_common.h
index 75d588308b..1bac2dcfa0 100644
--- a/app/test-mldev/test_inference_common.h
+++ b/app/test-mldev/test_inference_common.h
@@ -27,6 +27,7 @@ struct ml_core_args {
uint64_t nb_reqs;
int16_t start_fid;
int16_t end_fid;
+ uint32_t qp_id;
struct rte_ml_op **enq_ops;
struct rte_ml_op **deq_ops;
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v2 09/12] app/mldev: enable support for inference batches
2022-11-29 8:20 ` [PATCH v2 " Srikanth Yalavarthi
` (6 preceding siblings ...)
2022-11-29 8:21 ` [PATCH v2 08/12] app/mldev: enable support for queue pairs and size Srikanth Yalavarthi
@ 2022-11-29 8:21 ` Srikanth Yalavarthi
2022-11-29 8:21 ` [PATCH v2 10/12] app/mldev: enable support for inference validation Srikanth Yalavarthi
` (3 subsequent siblings)
11 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2022-11-29 8:21 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj
Enabled support to execute multiple batches of inferences
per each enqueue request. Input and reference for the test
should be appropriately provided for multi-batch run. Number
of batches can be specified through "--batches" option.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/ml_options.c | 15 ++++++++++++---
app/test-mldev/ml_options.h | 2 ++
app/test-mldev/test_inference_common.c | 22 +++++++++++++---------
app/test-mldev/test_model_common.c | 6 ++++++
app/test-mldev/test_model_common.h | 1 +
5 files changed, 34 insertions(+), 12 deletions(-)
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index d5182a1701..331ec1704c 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -33,6 +33,7 @@ ml_options_default(struct ml_options *opt)
opt->burst_size = 1;
opt->queue_pairs = 1;
opt->queue_size = 1;
+ opt->batches = 0;
opt->debug = false;
}
@@ -176,6 +177,12 @@ ml_parse_queue_size(struct ml_options *opt, const char *arg)
return parser_read_uint16(&opt->queue_size, arg);
}
+static int
+ml_parse_batches(struct ml_options *opt, const char *arg)
+{
+ return parser_read_uint16(&opt->batches, arg);
+}
+
static void
ml_dump_test_options(const char *testname)
{
@@ -196,7 +203,8 @@ ml_dump_test_options(const char *testname)
"\t\t--repetitions : number of inference repetitions\n"
"\t\t--burst_size : inference burst size\n"
"\t\t--queue_pairs : number of queue pairs to create\n"
- "\t\t--queue_size : size fo queue-pair\n");
+ "\t\t--queue_size : size fo queue-pair\n"
+ "\t\t--batches : number of batches of input\n");
printf("\n");
}
}
@@ -220,7 +228,8 @@ static struct option lgopts[] = {
{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
{ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
{ML_BURST_SIZE, 1, 0, 0}, {ML_QUEUE_PAIRS, 1, 0, 0}, {ML_QUEUE_SIZE, 1, 0, 0},
- {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
+ {ML_BATCHES, 1, 0, 0}, {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0},
+ {NULL, 0, 0, 0}};
static int
ml_opts_parse_long(int opt_idx, struct ml_options *opt)
@@ -232,7 +241,7 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
{ML_SOCKET_ID, ml_parse_socket_id}, {ML_MODELS, ml_parse_models},
{ML_FILELIST, ml_parse_filelist}, {ML_REPETITIONS, ml_parse_repetitions},
{ML_BURST_SIZE, ml_parse_burst_size}, {ML_QUEUE_PAIRS, ml_parse_queue_pairs},
- {ML_QUEUE_SIZE, ml_parse_queue_size},
+ {ML_QUEUE_SIZE, ml_parse_queue_size}, {ML_BATCHES, ml_parse_batches},
};
for (i = 0; i < RTE_DIM(parsermap); i++) {
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index 6bfef1b979..d23e842895 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -22,6 +22,7 @@
#define ML_BURST_SIZE ("burst_size")
#define ML_QUEUE_PAIRS ("queue_pairs")
#define ML_QUEUE_SIZE ("queue_size")
+#define ML_BATCHES ("batches")
#define ML_DEBUG ("debug")
#define ML_HELP ("help")
@@ -41,6 +42,7 @@ struct ml_options {
uint16_t burst_size;
uint16_t queue_pairs;
uint16_t queue_size;
+ uint16_t batches;
bool debug;
};
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
index d680d68f3d..4e29f6c7eb 100644
--- a/app/test-mldev/test_inference_common.c
+++ b/app/test-mldev/test_inference_common.c
@@ -56,7 +56,7 @@ ml_enqueue_single(void *arg)
goto retry;
op->model_id = t->model[fid].id;
- op->nb_batches = t->model[fid].info.batch_size;
+ op->nb_batches = t->model[fid].nb_batches;
op->mempool = t->op_pool;
op->input.addr = req->input;
@@ -168,7 +168,7 @@ ml_enqueue_burst(void *arg)
for (i = 0; i < ops_count; i++) {
args->enq_ops[i]->model_id = t->model[fid].id;
- args->enq_ops[i]->nb_batches = t->model[fid].info.batch_size;
+ args->enq_ops[i]->nb_batches = t->model[fid].nb_batches;
args->enq_ops[i]->mempool = t->op_pool;
args->enq_ops[i]->input.addr = args->reqs[i]->input;
@@ -363,6 +363,11 @@ test_inference_opt_dump(struct ml_options *opt)
ml_dump("queue_pairs", "%u", opt->queue_pairs);
ml_dump("queue_size", "%u", opt->queue_size);
+ if (opt->batches == 0)
+ ml_dump("batches", "%u (default)", opt->batches);
+ else
+ ml_dump("batches", "%u", opt->batches);
+
ml_dump_begin("filelist");
for (i = 0; i < opt->nb_filelist; i++) {
ml_dump_list("model", i, opt->filelist[i].model);
@@ -531,8 +536,8 @@ ml_request_initialize(struct rte_mempool *mp, void *opaque, void *obj, unsigned
req->niters = 0;
/* quantize data */
- rte_ml_io_quantize(t->cmn.opt->dev_id, t->model[t->fid].id,
- t->model[t->fid].info.batch_size, t->model[t->fid].input, req->input);
+ rte_ml_io_quantize(t->cmn.opt->dev_id, t->model[t->fid].id, t->model[t->fid].nb_batches,
+ t->model[t->fid].input, req->input);
}
int
@@ -550,7 +555,7 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, int16_t f
int ret;
/* get input buffer size */
- ret = rte_ml_io_input_size_get(opt->dev_id, t->model[fid].id, t->model[fid].info.batch_size,
+ ret = rte_ml_io_input_size_get(opt->dev_id, t->model[fid].id, t->model[fid].nb_batches,
&t->model[fid].inp_qsize, &t->model[fid].inp_dsize);
if (ret != 0) {
ml_err("Failed to get input size, model : %s\n", opt->filelist[fid].model);
@@ -558,9 +563,8 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, int16_t f
}
/* get output buffer size */
- ret = rte_ml_io_output_size_get(opt->dev_id, t->model[fid].id,
- t->model[fid].info.batch_size, &t->model[fid].out_qsize,
- &t->model[fid].out_dsize);
+ ret = rte_ml_io_output_size_get(opt->dev_id, t->model[fid].id, t->model[fid].nb_batches,
+ &t->model[fid].out_qsize, &t->model[fid].out_dsize);
if (ret != 0) {
ml_err("Failed to get input size, model : %s\n", opt->filelist[fid].model);
return ret;
@@ -705,7 +709,7 @@ ml_request_finish(struct rte_mempool *mp, void *opaque, void *obj, unsigned int
return;
t->nb_used++;
- rte_ml_io_dequantize(t->cmn.opt->dev_id, model->id, t->model[req->fid].info.batch_size,
+ rte_ml_io_dequantize(t->cmn.opt->dev_id, model->id, t->model[req->fid].nb_batches,
req->output, model->output);
}
diff --git a/app/test-mldev/test_model_common.c b/app/test-mldev/test_model_common.c
index 5368be17fe..51260c0789 100644
--- a/app/test-mldev/test_model_common.c
+++ b/app/test-mldev/test_model_common.c
@@ -75,6 +75,12 @@ ml_model_load(struct ml_test *test, struct ml_options *opt, struct ml_model *mod
return ret;
}
+ /* Update number of batches */
+ if (opt->batches == 0)
+ model->nb_batches = model->info.batch_size;
+ else
+ model->nb_batches = opt->batches;
+
model->state = MODEL_LOADED;
return 0;
diff --git a/app/test-mldev/test_model_common.h b/app/test-mldev/test_model_common.h
index c45ae80853..dfbf568f0b 100644
--- a/app/test-mldev/test_model_common.h
+++ b/app/test-mldev/test_model_common.h
@@ -33,6 +33,7 @@ struct ml_model {
uint8_t *output;
struct rte_mempool *io_pool;
+ uint32_t nb_batches;
};
int ml_model_load(struct ml_test *test, struct ml_options *opt, struct ml_model *model,
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v2 10/12] app/mldev: enable support for inference validation
2022-11-29 8:20 ` [PATCH v2 " Srikanth Yalavarthi
` (7 preceding siblings ...)
2022-11-29 8:21 ` [PATCH v2 09/12] app/mldev: enable support for inference batches Srikanth Yalavarthi
@ 2022-11-29 8:21 ` Srikanth Yalavarthi
2022-11-29 8:21 ` [PATCH v2 11/12] app/mldev: enable reporting stats in mldev app Srikanth Yalavarthi
` (2 subsequent siblings)
11 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2022-11-29 8:21 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj
Enabled support to validate inference output with reference
output provided by the user. Validation would be successful
only when the inference outputs are within the 'tolerance'
specified through command line option "--tolerance".
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/meson.build | 2 +-
app/test-mldev/ml_options.c | 36 +++-
app/test-mldev/ml_options.h | 3 +
app/test-mldev/test_inference_common.c | 218 ++++++++++++++++++++++++-
app/test-mldev/test_inference_common.h | 1 +
app/test-mldev/test_model_common.h | 1 +
6 files changed, 250 insertions(+), 11 deletions(-)
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
index 41d22fb22c..15db534dc2 100644
--- a/app/test-mldev/meson.build
+++ b/app/test-mldev/meson.build
@@ -21,4 +21,4 @@ sources = files(
'test_inference_interleave.c',
)
-deps += ['mldev']
+deps += ['mldev', 'hash']
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 331ec1704c..092303903f 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -5,6 +5,7 @@
#include <errno.h>
#include <getopt.h>
#include <linux/limits.h>
+#include <math.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
@@ -34,6 +35,7 @@ ml_options_default(struct ml_options *opt)
opt->queue_pairs = 1;
opt->queue_size = 1;
opt->batches = 0;
+ opt->tolerance = 0.0;
opt->debug = false;
}
@@ -139,6 +141,13 @@ ml_parse_filelist(struct ml_options *opt, const char *arg)
}
strlcpy(opt->filelist[opt->nb_filelist].output, token, PATH_MAX);
+ /* reference - optional */
+ token = strtok(NULL, delim);
+ if (token != NULL)
+ strlcpy(opt->filelist[opt->nb_filelist].reference, token, PATH_MAX);
+ else
+ memset(opt->filelist[opt->nb_filelist].reference, 0, PATH_MAX);
+
opt->nb_filelist++;
if (opt->nb_filelist == 0) {
@@ -183,6 +192,14 @@ ml_parse_batches(struct ml_options *opt, const char *arg)
return parser_read_uint16(&opt->batches, arg);
}
+static int
+ml_parse_tolerance(struct ml_options *opt, const char *arg)
+{
+ opt->tolerance = fabs(atof(arg));
+
+ return 0;
+}
+
static void
ml_dump_test_options(const char *testname)
{
@@ -199,12 +216,13 @@ ml_dump_test_options(const char *testname)
if ((strcmp(testname, "inference_ordered") == 0) ||
(strcmp(testname, "inference_interleave") == 0)) {
- printf("\t\t--filelist : comma separated list of model, input and output\n"
+ printf("\t\t--filelist : comma separated list of model, input, output and reference\n"
"\t\t--repetitions : number of inference repetitions\n"
"\t\t--burst_size : inference burst size\n"
"\t\t--queue_pairs : number of queue pairs to create\n"
"\t\t--queue_size : size fo queue-pair\n"
- "\t\t--batches : number of batches of input\n");
+ "\t\t--batches : number of batches of input\n"
+ "\t\t--tolerance : maximum tolerance (%%) for output validation\n");
printf("\n");
}
}
@@ -224,12 +242,13 @@ print_usage(char *program)
ml_test_dump_names(ml_dump_test_options);
}
-static struct option lgopts[] = {
- {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
- {ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
- {ML_BURST_SIZE, 1, 0, 0}, {ML_QUEUE_PAIRS, 1, 0, 0}, {ML_QUEUE_SIZE, 1, 0, 0},
- {ML_BATCHES, 1, 0, 0}, {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0},
- {NULL, 0, 0, 0}};
+static struct option lgopts[] = {{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0},
+ {ML_SOCKET_ID, 1, 0, 0}, {ML_MODELS, 1, 0, 0},
+ {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
+ {ML_BURST_SIZE, 1, 0, 0}, {ML_QUEUE_PAIRS, 1, 0, 0},
+ {ML_QUEUE_SIZE, 1, 0, 0}, {ML_BATCHES, 1, 0, 0},
+ {ML_TOLERANCE, 1, 0, 0}, {ML_DEBUG, 0, 0, 0},
+ {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
static int
ml_opts_parse_long(int opt_idx, struct ml_options *opt)
@@ -242,6 +261,7 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
{ML_FILELIST, ml_parse_filelist}, {ML_REPETITIONS, ml_parse_repetitions},
{ML_BURST_SIZE, ml_parse_burst_size}, {ML_QUEUE_PAIRS, ml_parse_queue_pairs},
{ML_QUEUE_SIZE, ml_parse_queue_size}, {ML_BATCHES, ml_parse_batches},
+ {ML_TOLERANCE, ml_parse_tolerance},
};
for (i = 0; i < RTE_DIM(parsermap); i++) {
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index d23e842895..79ac54de98 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -23,6 +23,7 @@
#define ML_QUEUE_PAIRS ("queue_pairs")
#define ML_QUEUE_SIZE ("queue_size")
#define ML_BATCHES ("batches")
+#define ML_TOLERANCE ("tolerance")
#define ML_DEBUG ("debug")
#define ML_HELP ("help")
@@ -30,6 +31,7 @@ struct ml_filelist {
char model[PATH_MAX];
char input[PATH_MAX];
char output[PATH_MAX];
+ char reference[PATH_MAX];
};
struct ml_options {
@@ -43,6 +45,7 @@ struct ml_options {
uint16_t queue_pairs;
uint16_t queue_size;
uint16_t batches;
+ float tolerance;
bool debug;
};
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
index 4e29f6c7eb..008cee1023 100644
--- a/app/test-mldev/test_inference_common.c
+++ b/app/test-mldev/test_inference_common.c
@@ -3,12 +3,15 @@
*/
#include <errno.h>
+#include <linux/limits.h>
#include <stdbool.h>
#include <stdint.h>
#include <stdio.h>
+#include <string.h>
#include <unistd.h>
#include <rte_common.h>
+#include <rte_hash_crc.h>
#include <rte_launch.h>
#include <rte_lcore.h>
#include <rte_malloc.h>
@@ -21,6 +24,27 @@
#include "test_common.h"
#include "test_inference_common.h"
+#define ML_TEST_READ_TYPE(buffer, type) (*((type *)buffer))
+
+#define ML_TEST_CHECK_OUTPUT(output, reference, tolerance) \
+ (((float)output - (float)reference) <= (((float)reference * tolerance) / 100.0))
+
+#define ML_OPEN_WRITE_GET_ERR(name, buffer, size, err) \
+ do { \
+ FILE *fp = fopen(name, "w+"); \
+ if (fp == NULL) { \
+ ml_err("Unable to create file: %s, error: %s", name, strerror(errno)); \
+ err = true; \
+ } else { \
+ if (fwrite(buffer, 1, size, fp) != size) { \
+ ml_err("Error writing output, file: %s, error: %s", name, \
+ strerror(errno)); \
+ err = true; \
+ } \
+ fclose(fp); \
+ } \
+ } while (0)
+
/* Enqueue inference requests with burst size equal to 1 */
static int
ml_enqueue_single(void *arg)
@@ -362,6 +386,7 @@ test_inference_opt_dump(struct ml_options *opt)
ml_dump("burst_size", "%u", opt->burst_size);
ml_dump("queue_pairs", "%u", opt->queue_pairs);
ml_dump("queue_size", "%u", opt->queue_size);
+ ml_dump("tolerance", "%-7.3f", opt->tolerance);
if (opt->batches == 0)
ml_dump("batches", "%u (default)", opt->batches);
@@ -373,6 +398,8 @@ test_inference_opt_dump(struct ml_options *opt)
ml_dump_list("model", i, opt->filelist[i].model);
ml_dump_list("input", i, opt->filelist[i].input);
ml_dump_list("output", i, opt->filelist[i].output);
+ if (strcmp(opt->filelist[i].reference, "\0") != 0)
+ ml_dump_list("reference", i, opt->filelist[i].reference);
}
ml_dump_end;
}
@@ -397,6 +424,7 @@ test_inference_setup(struct ml_test *test, struct ml_options *opt)
t = ml_test_priv(test);
t->nb_used = 0;
+ t->nb_valid = 0;
t->cmn.result = ML_TEST_FAILED;
t->cmn.opt = opt;
@@ -572,6 +600,9 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, int16_t f
/* allocate buffer for user data */
mz_size = t->model[fid].inp_dsize + t->model[fid].out_dsize;
+ if (strcmp(opt->filelist[fid].reference, "\0") != 0)
+ mz_size += t->model[fid].out_dsize;
+
sprintf(mz_name, "ml_user_data_%d", fid);
mz = rte_memzone_reserve(mz_name, mz_size, opt->socket_id, 0);
if (mz == NULL) {
@@ -582,6 +613,10 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, int16_t f
t->model[fid].input = mz->addr;
t->model[fid].output = t->model[fid].input + t->model[fid].inp_dsize;
+ if (strcmp(opt->filelist[fid].reference, "\0") != 0)
+ t->model[fid].reference = t->model[fid].output + t->model[fid].out_dsize;
+ else
+ t->model[fid].reference = NULL;
/* load input file */
fp = fopen(opt->filelist[fid].input, "r");
@@ -610,6 +645,27 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, int16_t f
}
fclose(fp);
+ /* load reference file */
+ if (t->model[fid].reference != NULL) {
+ fp = fopen(opt->filelist[fid].reference, "r");
+ if (fp == NULL) {
+ ml_err("Failed to open reference file : %s\n",
+ opt->filelist[fid].reference);
+ ret = -errno;
+ goto error;
+ }
+
+ if (fread(t->model[fid].reference, 1, t->model[fid].out_dsize, fp) !=
+ t->model[fid].out_dsize) {
+ ml_err("Failed to read reference file : %s\n",
+ opt->filelist[fid].reference);
+ ret = -errno;
+ fclose(fp);
+ goto error;
+ }
+ fclose(fp);
+ }
+
/* create mempool for quantized input and output buffers. ml_request_initialize is
* used as a callback for object creation.
*/
@@ -694,6 +750,121 @@ ml_inference_mem_destroy(struct ml_test *test, struct ml_options *opt)
rte_mempool_free(t->op_pool);
}
+static bool
+ml_inference_validation(struct ml_test *test, struct ml_request *req)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)test);
+ struct ml_model *model;
+ uint32_t nb_elements;
+ uint8_t *reference;
+ uint8_t *output;
+ bool match;
+ uint32_t i;
+ uint32_t j;
+
+ model = &t->model[req->fid];
+
+ /* compare crc when tolerance is 0 */
+ if (t->cmn.opt->tolerance == 0.0) {
+ match = (rte_hash_crc(model->output, model->out_dsize, 0) ==
+ rte_hash_crc(model->reference, model->out_dsize, 0));
+ } else {
+ output = model->output;
+ reference = model->reference;
+
+ i = 0;
+next_output:
+ nb_elements =
+ model->info.output_info[i].shape.w * model->info.output_info[i].shape.x *
+ model->info.output_info[i].shape.y * model->info.output_info[i].shape.z;
+ j = 0;
+next_element:
+ match = false;
+ switch (model->info.output_info[i].dtype) {
+ case RTE_ML_IO_TYPE_INT8:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, int8_t),
+ ML_TEST_READ_TYPE(reference, int8_t),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(int8_t);
+ reference += sizeof(int8_t);
+ break;
+ case RTE_ML_IO_TYPE_UINT8:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, uint8_t),
+ ML_TEST_READ_TYPE(reference, uint8_t),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(float);
+ reference += sizeof(float);
+ break;
+ case RTE_ML_IO_TYPE_INT16:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, int16_t),
+ ML_TEST_READ_TYPE(reference, int16_t),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(int16_t);
+ reference += sizeof(int16_t);
+ break;
+ case RTE_ML_IO_TYPE_UINT16:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, uint16_t),
+ ML_TEST_READ_TYPE(reference, uint16_t),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(uint16_t);
+ reference += sizeof(uint16_t);
+ break;
+ case RTE_ML_IO_TYPE_INT32:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, int32_t),
+ ML_TEST_READ_TYPE(reference, int32_t),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(int32_t);
+ reference += sizeof(int32_t);
+ break;
+ case RTE_ML_IO_TYPE_UINT32:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, uint32_t),
+ ML_TEST_READ_TYPE(reference, uint32_t),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(uint32_t);
+ reference += sizeof(uint32_t);
+ break;
+ case RTE_ML_IO_TYPE_FP32:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, float),
+ ML_TEST_READ_TYPE(reference, float),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(float);
+ reference += sizeof(float);
+ break;
+ default: /* other types, fp8, fp16, bfloat16 */
+ match = true;
+ }
+
+ if (!match)
+ goto done;
+ j++;
+ if (j < nb_elements)
+ goto next_element;
+
+ i++;
+ if (i < model->info.nb_outputs)
+ goto next_output;
+ }
+done:
+ if (match)
+ t->nb_valid++;
+
+ return match;
+}
+
/* Callback for mempool object iteration. This call would dequantize output data. */
static void
ml_request_finish(struct rte_mempool *mp, void *opaque, void *obj, unsigned int obj_idx)
@@ -701,9 +872,10 @@ ml_request_finish(struct rte_mempool *mp, void *opaque, void *obj, unsigned int
struct test_inference *t = ml_test_priv((struct ml_test *)opaque);
struct ml_request *req = (struct ml_request *)obj;
struct ml_model *model = &t->model[req->fid];
+ char str[PATH_MAX];
+ bool error = false;
RTE_SET_USED(mp);
- RTE_SET_USED(obj_idx);
if (req->niters == 0)
return;
@@ -711,6 +883,48 @@ ml_request_finish(struct rte_mempool *mp, void *opaque, void *obj, unsigned int
t->nb_used++;
rte_ml_io_dequantize(t->cmn.opt->dev_id, model->id, t->model[req->fid].nb_batches,
req->output, model->output);
+
+ if (model->reference == NULL) {
+ t->nb_valid++;
+ goto dump_output_pass;
+ }
+
+ if (!ml_inference_validation(opaque, req))
+ goto dump_output_fail;
+ else
+ goto dump_output_pass;
+
+dump_output_pass:
+ if (obj_idx == 0) {
+ /* write quantized output */
+ snprintf(str, PATH_MAX, "%s.q", t->cmn.opt->filelist[req->fid].output);
+ ML_OPEN_WRITE_GET_ERR(str, req->output, model->out_qsize, error);
+ if (error)
+ return;
+
+ /* write dequantized output */
+ snprintf(str, PATH_MAX, "%s", t->cmn.opt->filelist[req->fid].output);
+ ML_OPEN_WRITE_GET_ERR(str, model->output, model->out_dsize, error);
+ if (error)
+ return;
+ }
+
+ return;
+
+dump_output_fail:
+ if (t->cmn.opt->debug) {
+ /* dump quantized output buffer */
+ snprintf(str, PATH_MAX, "%s.q.%d", t->cmn.opt->filelist[req->fid].output, obj_idx);
+ ML_OPEN_WRITE_GET_ERR(str, req->output, model->out_qsize, error);
+ if (error)
+ return;
+
+ /* dump dequantized output buffer */
+ snprintf(str, PATH_MAX, "%s.%d", t->cmn.opt->filelist[req->fid].output, obj_idx);
+ ML_OPEN_WRITE_GET_ERR(str, model->output, model->out_dsize, error);
+ if (error)
+ return;
+ }
}
int
@@ -722,7 +936,7 @@ ml_inference_result(struct ml_test *test, struct ml_options *opt, int16_t fid)
rte_mempool_obj_iter(t->model[fid].io_pool, ml_request_finish, test);
- if (t->nb_used > 0)
+ if (t->nb_used == t->nb_valid)
t->cmn.result = ML_TEST_SUCCESS;
else
t->cmn.result = ML_TEST_FAILED;
diff --git a/app/test-mldev/test_inference_common.h b/app/test-mldev/test_inference_common.h
index 1bac2dcfa0..3f2b042360 100644
--- a/app/test-mldev/test_inference_common.h
+++ b/app/test-mldev/test_inference_common.h
@@ -43,6 +43,7 @@ struct test_inference {
struct rte_mempool *op_pool;
uint64_t nb_used;
+ uint64_t nb_valid;
int16_t fid;
int (*enqueue)(void *arg);
diff --git a/app/test-mldev/test_model_common.h b/app/test-mldev/test_model_common.h
index dfbf568f0b..ce12cbfecc 100644
--- a/app/test-mldev/test_model_common.h
+++ b/app/test-mldev/test_model_common.h
@@ -31,6 +31,7 @@ struct ml_model {
uint8_t *input;
uint8_t *output;
+ uint8_t *reference;
struct rte_mempool *io_pool;
uint32_t nb_batches;
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v2 11/12] app/mldev: enable reporting stats in mldev app
2022-11-29 8:20 ` [PATCH v2 " Srikanth Yalavarthi
` (8 preceding siblings ...)
2022-11-29 8:21 ` [PATCH v2 10/12] app/mldev: enable support for inference validation Srikanth Yalavarthi
@ 2022-11-29 8:21 ` Srikanth Yalavarthi
2022-11-29 8:21 ` [PATCH v2 12/12] app/mldev: add documentation for mldev test cases Srikanth Yalavarthi
2022-12-08 19:29 ` [PATCH v3 01/12] app/mldev: implement test framework for mldev Srikanth Yalavarthi
11 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2022-11-29 8:21 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj
Enable reporting driver xstats and inference end-to-end
latency and throughput in mldev inference tests. Reporting
of stats can be enabled using "--stats" option.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/ml_options.c | 22 ++--
app/test-mldev/ml_options.h | 2 +
app/test-mldev/test_inference_common.c | 139 +++++++++++++++++++++
app/test-mldev/test_inference_common.h | 8 ++
app/test-mldev/test_inference_interleave.c | 4 +
app/test-mldev/test_inference_ordered.c | 1 +
6 files changed, 168 insertions(+), 8 deletions(-)
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 092303903f..0e7877eed3 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -36,6 +36,7 @@ ml_options_default(struct ml_options *opt)
opt->queue_size = 1;
opt->batches = 0;
opt->tolerance = 0.0;
+ opt->stats = false;
opt->debug = false;
}
@@ -222,7 +223,8 @@ ml_dump_test_options(const char *testname)
"\t\t--queue_pairs : number of queue pairs to create\n"
"\t\t--queue_size : size fo queue-pair\n"
"\t\t--batches : number of batches of input\n"
- "\t\t--tolerance : maximum tolerance (%%) for output validation\n");
+ "\t\t--tolerance : maximum tolerance (%%) for output validation\n"
+ "\t\t--stats : enable reporting performance statistics\n");
printf("\n");
}
}
@@ -242,13 +244,12 @@ print_usage(char *program)
ml_test_dump_names(ml_dump_test_options);
}
-static struct option lgopts[] = {{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0},
- {ML_SOCKET_ID, 1, 0, 0}, {ML_MODELS, 1, 0, 0},
- {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
- {ML_BURST_SIZE, 1, 0, 0}, {ML_QUEUE_PAIRS, 1, 0, 0},
- {ML_QUEUE_SIZE, 1, 0, 0}, {ML_BATCHES, 1, 0, 0},
- {ML_TOLERANCE, 1, 0, 0}, {ML_DEBUG, 0, 0, 0},
- {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
+static struct option lgopts[] = {
+ {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
+ {ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
+ {ML_BURST_SIZE, 1, 0, 0}, {ML_QUEUE_PAIRS, 1, 0, 0}, {ML_QUEUE_SIZE, 1, 0, 0},
+ {ML_BATCHES, 1, 0, 0}, {ML_TOLERANCE, 1, 0, 0}, {ML_STATS, 0, 0, 0},
+ {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
static int
ml_opts_parse_long(int opt_idx, struct ml_options *opt)
@@ -283,6 +284,11 @@ ml_options_parse(struct ml_options *opt, int argc, char **argv)
while ((opts = getopt_long(argc, argv, "", lgopts, &opt_idx)) != EOF) {
switch (opts) {
case 0: /* parse long options */
+ if (!strcmp(lgopts[opt_idx].name, "stats")) {
+ opt->stats = true;
+ break;
+ }
+
if (!strcmp(lgopts[opt_idx].name, "debug")) {
opt->debug = true;
break;
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index 79ac54de98..a375ae6750 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -24,6 +24,7 @@
#define ML_QUEUE_SIZE ("queue_size")
#define ML_BATCHES ("batches")
#define ML_TOLERANCE ("tolerance")
+#define ML_STATS ("stats")
#define ML_DEBUG ("debug")
#define ML_HELP ("help")
@@ -46,6 +47,7 @@ struct ml_options {
uint16_t queue_size;
uint16_t batches;
float tolerance;
+ bool stats;
bool debug;
};
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
index 008cee1023..8d1fc55c2f 100644
--- a/app/test-mldev/test_inference_common.c
+++ b/app/test-mldev/test_inference_common.c
@@ -11,6 +11,7 @@
#include <unistd.h>
#include <rte_common.h>
+#include <rte_cycles.h>
#include <rte_hash_crc.h>
#include <rte_launch.h>
#include <rte_lcore.h>
@@ -45,6 +46,17 @@
} \
} while (0)
+static void
+print_line(uint16_t len)
+{
+ uint16_t i;
+
+ for (i = 0; i < len; i++)
+ printf("-");
+
+ printf("\n");
+}
+
/* Enqueue inference requests with burst size equal to 1 */
static int
ml_enqueue_single(void *arg)
@@ -54,6 +66,7 @@ ml_enqueue_single(void *arg)
struct rte_ml_op *op = NULL;
struct ml_core_args *args;
uint64_t model_enq = 0;
+ uint64_t start_cycle;
uint32_t burst_enq;
uint32_t lcore_id;
int16_t fid;
@@ -61,6 +74,7 @@ ml_enqueue_single(void *arg)
lcore_id = rte_lcore_id();
args = &t->args[lcore_id];
+ args->start_cycles = 0;
model_enq = 0;
if (args->nb_reqs == 0)
@@ -96,10 +110,12 @@ ml_enqueue_single(void *arg)
req->fid = fid;
enqueue_req:
+ start_cycle = rte_get_tsc_cycles();
burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, args->qp_id, &op, 1);
if (burst_enq == 0)
goto enqueue_req;
+ args->start_cycles += start_cycle;
fid++;
if (likely(fid <= args->end_fid))
goto next_model;
@@ -123,10 +139,12 @@ ml_dequeue_single(void *arg)
uint64_t total_deq = 0;
uint8_t nb_filelist;
uint32_t burst_deq;
+ uint64_t end_cycle;
uint32_t lcore_id;
lcore_id = rte_lcore_id();
args = &t->args[lcore_id];
+ args->end_cycles = 0;
nb_filelist = args->end_fid - args->start_fid + 1;
if (args->nb_reqs == 0)
@@ -134,9 +152,11 @@ ml_dequeue_single(void *arg)
dequeue_req:
burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, args->qp_id, &op, 1);
+ end_cycle = rte_get_tsc_cycles();
if (likely(burst_deq == 1)) {
total_deq += burst_deq;
+ args->end_cycles += end_cycle;
if (unlikely(op->status == RTE_ML_OP_STATUS_ERROR)) {
rte_ml_op_error_get(t->cmn.opt->dev_id, op, &error);
ml_err("error_code = 0x%" PRIx64 ", error_message = %s\n", error.errcode,
@@ -159,6 +179,7 @@ ml_enqueue_burst(void *arg)
{
struct test_inference *t = ml_test_priv((struct ml_test *)arg);
struct ml_core_args *args;
+ uint64_t start_cycle;
uint16_t ops_count;
uint64_t model_enq;
uint16_t burst_enq;
@@ -171,6 +192,7 @@ ml_enqueue_burst(void *arg)
lcore_id = rte_lcore_id();
args = &t->args[lcore_id];
+ args->start_cycles = 0;
model_enq = 0;
if (args->nb_reqs == 0)
@@ -212,8 +234,10 @@ ml_enqueue_burst(void *arg)
pending = ops_count;
enqueue_reqs:
+ start_cycle = rte_get_tsc_cycles();
burst_enq =
rte_ml_enqueue_burst(t->cmn.opt->dev_id, args->qp_id, &args->enq_ops[idx], pending);
+ args->start_cycles += burst_enq * start_cycle;
pending = pending - burst_enq;
if (pending > 0) {
@@ -243,11 +267,13 @@ ml_dequeue_burst(void *arg)
uint64_t total_deq = 0;
uint16_t burst_deq = 0;
uint8_t nb_filelist;
+ uint64_t end_cycle;
uint32_t lcore_id;
uint32_t i;
lcore_id = rte_lcore_id();
args = &t->args[lcore_id];
+ args->end_cycles = 0;
nb_filelist = args->end_fid - args->start_fid + 1;
if (args->nb_reqs == 0)
@@ -256,9 +282,11 @@ ml_dequeue_burst(void *arg)
dequeue_burst:
burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, args->qp_id, args->deq_ops,
t->cmn.opt->burst_size);
+ end_cycle = rte_get_tsc_cycles();
if (likely(burst_deq > 0)) {
total_deq += burst_deq;
+ args->end_cycles += burst_deq * end_cycle;
for (i = 0; i < burst_deq; i++) {
if (unlikely(args->deq_ops[i]->status == RTE_ML_OP_STATUS_ERROR)) {
@@ -387,6 +415,7 @@ test_inference_opt_dump(struct ml_options *opt)
ml_dump("queue_pairs", "%u", opt->queue_pairs);
ml_dump("queue_size", "%u", opt->queue_size);
ml_dump("tolerance", "%-7.3f", opt->tolerance);
+ ml_dump("stats", "%s", (opt->stats ? "true" : "false"));
if (opt->batches == 0)
ml_dump("batches", "%u (default)", opt->batches);
@@ -459,6 +488,11 @@ test_inference_setup(struct ml_test *test, struct ml_options *opt)
RTE_CACHE_LINE_SIZE, opt->socket_id);
}
+ for (i = 0; i < RTE_MAX_LCORE; i++) {
+ t->args[i].start_cycles = 0;
+ t->args[i].end_cycles = 0;
+ }
+
return 0;
error:
@@ -985,3 +1019,108 @@ ml_inference_launch_cores(struct ml_test *test, struct ml_options *opt, int16_t
return 0;
}
+
+int
+ml_inference_stats_get(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t = ml_test_priv(test);
+ uint64_t total_cycles = 0;
+ uint32_t nb_filelist;
+ uint64_t throughput;
+ uint64_t avg_e2e;
+ uint32_t qp_id;
+ uint64_t freq;
+ int ret;
+ int i;
+
+ if (!opt->stats)
+ return 0;
+
+ /* get xstats size */
+ t->xstats_size = rte_ml_dev_xstats_names_get(opt->dev_id, NULL, 0);
+ if (t->xstats_size >= 0) {
+ /* allocate for xstats_map and values */
+ t->xstats_map = rte_malloc(
+ "ml_xstats_map", t->xstats_size * sizeof(struct rte_ml_dev_xstats_map), 0);
+ if (t->xstats_map == NULL) {
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ t->xstats_values =
+ rte_malloc("ml_xstats_values", t->xstats_size * sizeof(uint64_t), 0);
+ if (t->xstats_values == NULL) {
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ ret = rte_ml_dev_xstats_names_get(opt->dev_id, t->xstats_map, t->xstats_size);
+ if (ret != t->xstats_size) {
+ printf("Unable to get xstats names, ret = %d\n", ret);
+ ret = -1;
+ goto error;
+ }
+
+ for (i = 0; i < t->xstats_size; i++)
+ rte_ml_dev_xstats_get(opt->dev_id, &t->xstats_map[i].id,
+ &t->xstats_values[i], 1);
+ }
+
+ /* print xstats*/
+ printf("\n");
+ print_line(80);
+ printf(" ML Device Extended Statistics\n");
+ print_line(80);
+ for (i = 0; i < t->xstats_size; i++)
+ printf(" %-64s = %" PRIu64 "\n", t->xstats_map[i].name, t->xstats_values[i]);
+ print_line(80);
+
+ /* release buffers */
+ if (t->xstats_map)
+ rte_free(t->xstats_map);
+
+ if (t->xstats_values)
+ rte_free(t->xstats_values);
+
+ /* print end-to-end stats */
+ freq = rte_get_tsc_hz();
+ for (qp_id = 0; qp_id < RTE_MAX_LCORE; qp_id++)
+ total_cycles += t->args[qp_id].end_cycles - t->args[qp_id].start_cycles;
+ avg_e2e = total_cycles / opt->repetitions;
+
+ if (freq == 0) {
+ avg_e2e = total_cycles / opt->repetitions;
+ printf(" %-64s = %" PRIu64 "\n", "Average End-to-End Latency (cycles)", avg_e2e);
+ } else {
+ avg_e2e = (total_cycles * NS_PER_S) / (opt->repetitions * freq);
+ printf(" %-64s = %" PRIu64 "\n", "Average End-to-End Latency (ns)", avg_e2e);
+ }
+
+ if (strcmp(opt->test_name, "inference_ordered") == 0)
+ nb_filelist = 1;
+ else
+ nb_filelist = t->cmn.opt->nb_filelist;
+
+ if (freq == 0) {
+ throughput = (nb_filelist * t->cmn.opt->repetitions * 1000000) / total_cycles;
+ printf(" %-64s = %" PRIu64 "\n", "Average Throughput (inferences / million cycles)",
+ throughput);
+ } else {
+ throughput = (nb_filelist * t->cmn.opt->repetitions * freq) / total_cycles;
+ printf(" %-64s = %" PRIu64 "\n", "Average Throughput (inferences / second)",
+ throughput);
+ }
+
+ print_line(80);
+
+ return 0;
+
+error:
+ if (t->xstats_map)
+ rte_free(t->xstats_map);
+
+ if (t->xstats_values)
+ rte_free(t->xstats_values);
+
+ return ret;
+}
diff --git a/app/test-mldev/test_inference_common.h b/app/test-mldev/test_inference_common.h
index 3f2b042360..bb2920cc30 100644
--- a/app/test-mldev/test_inference_common.h
+++ b/app/test-mldev/test_inference_common.h
@@ -32,6 +32,9 @@ struct ml_core_args {
struct rte_ml_op **enq_ops;
struct rte_ml_op **deq_ops;
struct ml_request **reqs;
+
+ uint64_t start_cycles;
+ uint64_t end_cycles;
};
struct test_inference {
@@ -50,6 +53,10 @@ struct test_inference {
int (*dequeue)(void *arg);
struct ml_core_args args[RTE_MAX_LCORE];
+
+ struct rte_ml_dev_xstats_map *xstats_map;
+ uint64_t *xstats_values;
+ int xstats_size;
} __rte_cache_aligned;
bool test_inference_cap_check(struct ml_options *opt);
@@ -67,5 +74,6 @@ void ml_inference_mem_destroy(struct ml_test *test, struct ml_options *opt);
int ml_inference_result(struct ml_test *test, struct ml_options *opt, int16_t fid);
int ml_inference_launch_cores(struct ml_test *test, struct ml_options *opt, int16_t start_fid,
int16_t end_fid);
+int ml_inference_stats_get(struct ml_test *test, struct ml_options *opt);
#endif /* _ML_TEST_INFERENCE_COMMON_ */
diff --git a/app/test-mldev/test_inference_interleave.c b/app/test-mldev/test_inference_interleave.c
index 74ad0c597f..d86838c3fa 100644
--- a/app/test-mldev/test_inference_interleave.c
+++ b/app/test-mldev/test_inference_interleave.c
@@ -60,7 +60,11 @@ test_inference_interleave_driver(struct ml_test *test, struct ml_options *opt)
goto error;
ml_inference_iomem_destroy(test, opt, fid);
+ }
+
+ ml_inference_stats_get(test, opt);
+ for (fid = 0; fid < opt->nb_filelist; fid++) {
ret = ml_model_stop(test, opt, &t->model[fid], fid);
if (ret != 0)
goto error;
diff --git a/app/test-mldev/test_inference_ordered.c b/app/test-mldev/test_inference_ordered.c
index 84e6bf9109..3826121a65 100644
--- a/app/test-mldev/test_inference_ordered.c
+++ b/app/test-mldev/test_inference_ordered.c
@@ -58,6 +58,7 @@ test_inference_ordered_driver(struct ml_test *test, struct ml_options *opt)
goto error;
ml_inference_iomem_destroy(test, opt, fid);
+ ml_inference_stats_get(test, opt);
/* stop model */
ret = ml_model_stop(test, opt, &t->model[fid], fid);
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v2 12/12] app/mldev: add documentation for mldev test cases
2022-11-29 8:20 ` [PATCH v2 " Srikanth Yalavarthi
` (9 preceding siblings ...)
2022-11-29 8:21 ` [PATCH v2 11/12] app/mldev: enable reporting stats in mldev app Srikanth Yalavarthi
@ 2022-11-29 8:21 ` Srikanth Yalavarthi
2022-12-08 19:29 ` [PATCH v3 01/12] app/mldev: implement test framework for mldev Srikanth Yalavarthi
11 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2022-11-29 8:21 UTC (permalink / raw)
To: Thomas Monjalon, Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj
Added documentation specific to mldev test cases. Added details
about all test cases and option supported by individual tests.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
MAINTAINERS | 1 +
.../tools/img/mldev_inference_interleave.svg | 667 ++++++++++++++++++
.../tools/img/mldev_inference_ordered.svg | 526 ++++++++++++++
.../tools/img/mldev_model_ops_subtest_a.svg | 418 +++++++++++
.../tools/img/mldev_model_ops_subtest_b.svg | 421 +++++++++++
.../tools/img/mldev_model_ops_subtest_c.svg | 364 ++++++++++
.../tools/img/mldev_model_ops_subtest_d.svg | 422 +++++++++++
doc/guides/tools/index.rst | 1 +
doc/guides/tools/testmldev.rst | 441 ++++++++++++
9 files changed, 3261 insertions(+)
create mode 100644 doc/guides/tools/img/mldev_inference_interleave.svg
create mode 100644 doc/guides/tools/img/mldev_inference_ordered.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_a.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_b.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_c.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_d.svg
create mode 100644 doc/guides/tools/testmldev.rst
diff --git a/MAINTAINERS b/MAINTAINERS
index 1edea42fad..1cddd6ead2 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -540,6 +540,7 @@ M: Srikanth Yalavarthi <syalavarthi@marvell.com>
F: lib/mldev/
F: app/test-mldev/
F: doc/guides/prog_guide/mldev.rst
+F: doc/guides/tools/testmldev.rst
Memory Pool Drivers
diff --git a/doc/guides/tools/img/mldev_inference_interleave.svg b/doc/guides/tools/img/mldev_inference_interleave.svg
new file mode 100644
index 0000000000..517c53d294
--- /dev/null
+++ b/doc/guides/tools/img/mldev_inference_interleave.svg
@@ -0,0 +1,667 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="224mm"
+ height="150mm"
+ viewBox="0 0 224 150"
+ version="1.1"
+ id="svg5369"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="inference_interleave.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns:xlink="http://www.w3.org/1999/xlink"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg">
+ <sodipodi:namedview
+ id="namedview5371"
+ pagecolor="#ffffff"
+ bordercolor="#000000"
+ borderopacity="0.25"
+ inkscape:showpageshadow="2"
+ inkscape:pageopacity="0.0"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1"
+ inkscape:document-units="mm"
+ showgrid="false"
+ inkscape:zoom="0.74564394"
+ inkscape:cx="415.07747"
+ inkscape:cy="348.6919"
+ inkscape:window-width="1920"
+ inkscape:window-height="1017"
+ inkscape:window-x="1912"
+ inkscape:window-y="-8"
+ inkscape:window-maximized="1"
+ inkscape:current-layer="layer1" />
+ <defs
+ id="defs5366">
+ <linearGradient
+ inkscape:collect="always"
+ id="linearGradient19189">
+ <stop
+ style="stop-color:#ffeeaa;stop-opacity:1;"
+ offset="0"
+ id="stop19185" />
+ <stop
+ style="stop-color:#ffeeaa;stop-opacity:0;"
+ offset="1"
+ id="stop19187" />
+ </linearGradient>
+ <marker
+ style="overflow:visible"
+ id="TriangleStart"
+ refX="4"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="TriangleStart"
+ markerWidth="5.3244081"
+ markerHeight="6.155385"
+ viewBox="0 0 5.3244081 6.1553851"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.5)"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:context-stroke;stroke-width:1pt"
+ d="M 5.77,0 -2.88,5 V -5 Z"
+ id="path135" />
+ </marker>
+ <marker
+ style="overflow:visible"
+ id="RoundedArrow"
+ refX="5"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="RoundedArrow"
+ markerWidth="6.1347523"
+ markerHeight="5.9304948"
+ viewBox="0 0 6.1347524 5.9304951"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.7)"
+ d="m -0.21114562,-4.1055728 6.42229122,3.21114561 a 1,1 90 0 1 0,1.78885438 L -0.21114562,4.1055728 A 1.236068,1.236068 31.717474 0 1 -2,3 v -6 a 1.236068,1.236068 148.28253 0 1 1.78885438,-1.1055728 z"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:none"
+ id="path1367" />
+ </marker>
+ <linearGradient
+ inkscape:collect="always"
+ xlink:href="#linearGradient19189"
+ id="linearGradient19191"
+ x1="12.169352"
+ y1="105"
+ x2="284.83066"
+ y2="105"
+ gradientUnits="userSpaceOnUse"
+ gradientTransform="matrix(0.79055445,0,0,0.74078976,29.505892,28.991272)" />
+ </defs>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1">
+ <g
+ id="g1477"
+ transform="translate(-34.903236,-31.774189)">
+ <rect
+ style="fill:url(#linearGradient19191);fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.396267;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect16635"
+ width="215.18147"
+ height="139.60078"
+ x="39.3125"
+ y="36.973797"
+ ry="2.2354064" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-169.3954"
+ y="214.97237"
+ id="text5181"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan5179"
+ style="font-size:5.64444px;stroke-width:0.75"
+ x="-169.3954"
+ y="214.97237">test: inference_interleave</tspan></text>
+ <path
+ style="display:inline;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 138.05133,87.886263 17.45982,-10e-7"
+ id="path1912"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1724-0"
+ inkscape:connection-end="#rect1679" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 191.51115,87.886262 17.45981,10e-7"
+ id="path1914"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1679"
+ inkscape:connection-end="#rect1724" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 138.05133,109.78102 17.45982,-1e-5"
+ id="path1916"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1724-4-8"
+ inkscape:connection-end="#rect1679-4" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 191.51115,109.78101 17.45981,1e-5"
+ id="path1918"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1679-4"
+ inkscape:connection-end="#rect1724-4" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 138.05133,131.67576 17.45982,-1e-5"
+ id="path1920"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1724-6-7"
+ inkscape:connection-end="#rect1679-8" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 191.51115,131.67575 17.45981,1e-5"
+ id="path1922"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1679-8"
+ inkscape:connection-end="#rect1724-6" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.75;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;marker-start:url(#RoundedArrow);marker-end:url(#RoundedArrow)"
+ d="m 173.51116,60.08164 0,12.907336"
+ id="path1933"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1811"
+ inkscape:connection-end="#rect1924" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.368668;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.737336, 0.737336;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924"
+ width="46.97015"
+ height="73.58287"
+ x="150.02565"
+ y="72.988976"
+ ry="2.4685853" />
+ <text
+ xml:space="preserve"
+ style="font-size:6.35px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-84.44075"
+ y="173.5065"
+ id="text4097"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4095"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-84.44075"
+ y="173.5065">Queue</tspan><tspan
+ sodipodi:role="line"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-92.37825"
+ y="173.5065"
+ id="tspan4099">Pair 0</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:6.35px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-128.2318"
+ y="173.5065"
+ id="text4097-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4095-6"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-128.2318"
+ y="173.5065">Queue</tspan><tspan
+ sodipodi:role="line"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-136.1693"
+ y="173.5065"
+ id="tspan4099-4">Pair 2</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1679"
+ width="36"
+ height="18"
+ x="155.51115"
+ y="78.886261"
+ ry="3" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1679-8"
+ width="36"
+ height="18"
+ x="155.51115"
+ y="122.67575"
+ ry="3" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1679-4"
+ width="36"
+ height="18"
+ x="155.51115"
+ y="100.78101"
+ ry="3" />
+ <text
+ xml:space="preserve"
+ style="font-size:6.35px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-106.33705"
+ y="173.5065"
+ id="text4097-8"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4095-4"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-106.33705"
+ y="173.5065">Queue</tspan><tspan
+ sodipodi:role="line"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-114.27455"
+ y="173.5065"
+ id="tspan4099-5">Pair 1</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.388863;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1811"
+ width="58.909527"
+ height="18.812746"
+ x="144.0564"
+ y="41.268894"
+ ry="2.2255962" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-47.017281"
+ y="173.49187"
+ id="text4156"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4154"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';stroke-width:0.75"
+ x="-47.017281"
+ y="173.49187">Machine Learning</tspan><tspan
+ sodipodi:role="line"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';stroke-width:0.75"
+ x="-54.07283"
+ y="173.49187"
+ id="tspan4158">Hardware Engine</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.368668;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.737336, 0.737336;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924-0"
+ width="46.97015"
+ height="73.58287"
+ x="98.42067"
+ y="72.988976"
+ ry="2.4685853" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-88.32518"
+ y="122.24379"
+ id="text3708"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-88.32518"
+ y="122.24379">lcore 1</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-132.10504"
+ y="121.83865"
+ id="text3708-8"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-7"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-132.10504"
+ y="121.83865">lcore 5</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-0"
+ width="32.290321"
+ height="11.709678"
+ x="105.76101"
+ y="82.031425"
+ ry="3.0161259" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-6-7"
+ width="32.290321"
+ height="11.709678"
+ x="105.76101"
+ y="125.82092"
+ ry="3.0161259"
+ inkscape:connector-avoid="true" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-4-8"
+ width="32.290321"
+ height="11.709678"
+ x="105.76101"
+ y="103.92618"
+ ry="3.0161259"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-110.21718"
+ y="121.85381"
+ id="text3708-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-87"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-110.21718"
+ y="121.85381">lcore 3</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-150.82878"
+ y="121.79179"
+ id="text4542"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4540"
+ style="font-size:5.64444px;stroke-width:0.75"
+ x="-150.82878"
+ y="121.79179">Enqueue Workers</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.368668;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.737336, 0.737336;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924-7"
+ width="46.97015"
+ height="73.58287"
+ x="201.63062"
+ y="72.988976"
+ ry="2.4685853" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-88.32518"
+ y="225.08443"
+ id="text3708-9"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-9"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-88.32518"
+ y="225.08443">lcore 2</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-110.21167"
+ y="225.07202"
+ id="text3708-7"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-8"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-110.21167"
+ y="225.07202">lcore 4</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724"
+ width="32.290321"
+ height="11.709678"
+ x="208.97096"
+ y="82.031425"
+ ry="3.0161259" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-4"
+ width="32.290321"
+ height="11.709678"
+ x="208.97096"
+ y="103.92618"
+ ry="3.0161259" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-6"
+ width="32.290321"
+ height="11.709678"
+ x="208.97096"
+ y="125.82092"
+ ry="3.0161259" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-132.1133"
+ y="225.06514"
+ id="text3708-78"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-0"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-132.1133"
+ y="225.06514">lcore 6</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-150.82878"
+ y="225.00725"
+ id="text4542-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4540-7"
+ style="font-size:5.64444px;stroke-width:0.75"
+ x="-150.82878"
+ y="225.00725">Dequeue Workers</tspan></text>
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 79.368074,78.962117 26.440446,5.546991"
+ id="path6677"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-0"
+ inkscape:connection-start="#rect6252" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="M 78.978262,84.050781 112.13805,103.92618"
+ id="path6679"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-4-8"
+ inkscape:connection-start="#rect6252" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 73.959073,84.25738 42.026977,41.56354"
+ id="path6681"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-6-7"
+ inkscape:connection-start="#rect6252" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="M 79.368074,95.959838 105.76913,90.949016"
+ id="path6683"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-0"
+ inkscape:connection-start="#rect6252-8" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 79.368074,101.28215 26.416716,5.27791"
+ id="path7830"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-4-8"
+ inkscape:connection-start="#rect6252-8" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 79.069199,106.4283 32.903161,19.39262"
+ id="path7862"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-6-7"
+ inkscape:connection-start="#rect6252-8" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="M 79.069201,113.13371 111.97235,93.741103"
+ id="path7900"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-0"
+ inkscape:connection-start="#rect6252-2" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 79.368074,118.27987 26.416716,-5.2779"
+ id="path7932"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-4-8"
+ inkscape:connection-start="#rect6252-2" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 79.368074,123.60218 26.401056,5.01083"
+ id="path7998"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-6-7"
+ inkscape:connection-start="#rect6252-2" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="M 73.959074,135.30464 115.98605,93.741103"
+ id="path8000"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-0"
+ inkscape:connection-start="#rect6252-6" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="M 78.978264,135.51124 112.13805,115.63586"
+ id="path8002"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-4-8"
+ inkscape:connection-start="#rect6252-6" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 79.368074,140.59991 26.440446,-5.54699"
+ id="path8004"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-6-7"
+ inkscape:connection-start="#rect6252-6" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-76.53363"
+ y="65.63237"
+ id="text3708-84"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-4"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-76.53363"
+ y="65.63237">Model 0</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-98.991623"
+ y="66.015465"
+ id="text3708-84-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-4-6"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-98.991623"
+ y="66.015465">Model 1</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-121.44823"
+ y="65.646149"
+ id="text3708-84-9"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-4-1"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-121.44823"
+ y="65.646149">Model 2</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-143.9021"
+ y="65.625481"
+ id="text3708-84-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-4-8"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-143.9021"
+ y="65.625481">Model 3</tspan></text>
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect6252"
+ width="27.32258"
+ height="16.32258"
+ x="52.045494"
+ y="67.934799"
+ ry="1" />
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect6252-8"
+ width="27.32258"
+ height="16.32258"
+ x="52.045494"
+ y="90.391411"
+ ry="1" />
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect6252-2"
+ width="27.32258"
+ height="16.32258"
+ x="52.045494"
+ y="112.84802"
+ ry="1" />
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect6252-6"
+ width="27.32258"
+ height="16.32258"
+ x="52.045494"
+ y="135.30464"
+ ry="1" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.399886;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.799773, 0.799773;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924-1-3"
+ width="43.035683"
+ height="94.487152"
+ x="44.188942"
+ y="62.536991"
+ ry="3.1694498" />
+ <g
+ id="g1010"
+ transform="translate(21.464467,-15.875004)">
+ <text
+ xml:space="preserve"
+ style="font-weight:bold;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:'Arial Bold';text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-185.88483"
+ y="86.542366"
+ id="text15571"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan15569"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.5;stroke-opacity:1"
+ x="-185.88483"
+ y="86.542366">nb_worker_threads = 2 * MIN(nb_queue_pairs, (lcore_count - 1) / 2)</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-weight:bold;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:'Arial Bold';text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-178.43243"
+ y="90.903854"
+ id="text15571-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan15569-9"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.5;stroke-opacity:1"
+ x="-178.43243"
+ y="90.903854">inferences_per_queue_pair = nb_models * (repetitions / nb_queue_pairs)</tspan></text>
+ </g>
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/tools/img/mldev_inference_ordered.svg b/doc/guides/tools/img/mldev_inference_ordered.svg
new file mode 100644
index 0000000000..9d2b2c9246
--- /dev/null
+++ b/doc/guides/tools/img/mldev_inference_ordered.svg
@@ -0,0 +1,526 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="243mm"
+ height="144mm"
+ viewBox="0 0 243 144"
+ version="1.1"
+ id="svg5"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="inference_ordered.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns:xlink="http://www.w3.org/1999/xlink"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg">
+ <sodipodi:namedview
+ id="namedview7"
+ pagecolor="#ffffff"
+ bordercolor="#000000"
+ borderopacity="0.25"
+ inkscape:showpageshadow="2"
+ inkscape:pageopacity="0.0"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1"
+ inkscape:document-units="mm"
+ showgrid="false"
+ inkscape:zoom="0.74564394"
+ inkscape:cx="488.83922"
+ inkscape:cy="234.69647"
+ inkscape:window-width="1920"
+ inkscape:window-height="1017"
+ inkscape:window-x="1912"
+ inkscape:window-y="-8"
+ inkscape:window-maximized="1"
+ inkscape:current-layer="layer1" />
+ <defs
+ id="defs2">
+ <marker
+ style="overflow:visible"
+ id="RoundedArrow"
+ refX="5"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="RoundedArrow"
+ markerWidth="6.1347523"
+ markerHeight="5.9304948"
+ viewBox="0 0 6.1347524 5.9304951"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.7)"
+ d="m -0.21114562,-4.1055728 6.42229122,3.21114561 a 1,1 90 0 1 0,1.78885438 L -0.21114562,4.1055728 A 1.236068,1.236068 31.717474 0 1 -2,3 v -6 a 1.236068,1.236068 148.28253 0 1 1.78885438,-1.1055728 z"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:none"
+ id="path1367" />
+ </marker>
+ <linearGradient
+ inkscape:collect="always"
+ id="linearGradient31002">
+ <stop
+ style="stop-color:#fff6d5;stop-opacity:1;"
+ offset="0"
+ id="stop30998" />
+ <stop
+ style="stop-color:#fff6d5;stop-opacity:0;"
+ offset="1"
+ id="stop31000" />
+ </linearGradient>
+ <marker
+ style="overflow:visible"
+ id="TriangleStart"
+ refX="4"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="TriangleStart"
+ markerWidth="5.3244081"
+ markerHeight="6.155385"
+ viewBox="0 0 5.3244081 6.1553851"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.5)"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:context-stroke;stroke-width:1pt"
+ d="M 5.77,0 -2.88,5 V -5 Z"
+ id="path135" />
+ </marker>
+ <linearGradient
+ inkscape:collect="always"
+ xlink:href="#linearGradient31002"
+ id="linearGradient31004"
+ x1="19.620968"
+ y1="102.90323"
+ x2="279.1532"
+ y2="102.90323"
+ gradientUnits="userSpaceOnUse"
+ gradientTransform="matrix(0.89215122,0,0,0.73190287,13.449912,42.668706)" />
+ </defs>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1">
+ <g
+ id="g1340"
+ transform="translate(-25.225796,-45.983871)">
+ <rect
+ style="fill:url(#linearGradient31004);fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.404032;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect27876"
+ width="231.09595"
+ height="132.45081"
+ x="31.177822"
+ y="51.758469"
+ ry="3.5071263" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1813"
+ width="38.408459"
+ height="45.86002"
+ x="34.901794"
+ y="99.14959"
+ ry="5.2246051"
+ inkscape:connector-avoid="true" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 73.310253,115.94935 36.498807,-11.6509"
+ id="path1906"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1813"
+ inkscape:connection-end="#rect1724-0" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 73.310253,122.0796 36.117817,1e-5"
+ id="path1908"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1813"
+ inkscape:connection-end="#rect1724-4-8" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="M 73.310253,128.20983 109.80905,139.8607"
+ id="path1910"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1813"
+ inkscape:connection-end="#rect1724-6-7" />
+ <path
+ style="display:inline;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 141.71839,99.266314 19.42262,-10e-7"
+ id="path1912"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1724-0"
+ inkscape:connection-end="#rect1679" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 197.14101,99.266313 19.42259,10e-7"
+ id="path1914"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1679"
+ inkscape:connection-end="#rect1724" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 141.71839,122.07961 19.42262,-1e-5"
+ id="path1916"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1724-4-8"
+ inkscape:connection-end="#rect1679-4" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 197.14101,122.0796 19.42259,1e-5"
+ id="path1918"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1679-4"
+ inkscape:connection-end="#rect1724-4" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 141.71839,144.89282 19.42262,0"
+ id="path1920"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1724-6-7"
+ inkscape:connection-end="#rect1679-8" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 197.14101,144.89282 19.42259,0"
+ id="path1922"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1679-8"
+ inkscape:connection-end="#rect1724-6" />
+ <text
+ xml:space="preserve"
+ style="font-size:8.46667px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-121.09793"
+ y="54.031597"
+ id="text4093"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4091"
+ style="font-size:8.46667px;stroke-width:0.75"
+ x="-121.09793"
+ y="54.031597">Model X</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:6.35px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-118.63563"
+ y="179.13635"
+ id="text4097"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4095"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-118.63563"
+ y="179.13635">Queue</tspan><tspan
+ sodipodi:role="line"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-126.57313"
+ y="179.13635"
+ id="tspan4099">Pair 1</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:6.35px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-141.44887"
+ y="179.13635"
+ id="text4097-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4095-6"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-141.44887"
+ y="179.13635">Queue</tspan><tspan
+ sodipodi:role="line"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-149.38637"
+ y="179.13635"
+ id="tspan4099-4">Pair 2</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.354849;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.709699, 0.709699;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924"
+ width="44.145252"
+ height="72.532341"
+ x="157.06865"
+ y="85.813438"
+ ry="4.31247" />
+ <g
+ id="g1224">
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1679"
+ width="36"
+ height="18"
+ x="161.14101"
+ y="90.266312"
+ ry="3" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1679-8"
+ width="36"
+ height="18"
+ x="161.14101"
+ y="135.89282"
+ ry="3" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1679-4"
+ width="36"
+ height="18"
+ x="161.14101"
+ y="113.07959"
+ ry="3" />
+ </g>
+ <text
+ xml:space="preserve"
+ style="font-size:6.35px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-95.820801"
+ y="179.13635"
+ id="text4097-8"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4095-4"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-95.820801"
+ y="179.13635">Queue</tspan><tspan
+ sodipodi:role="line"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-103.7583"
+ y="179.13635"
+ id="tspan4099-5">Pair 0</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.317648;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1811"
+ width="44.196934"
+ height="16.731901"
+ x="157.04254"
+ y="56.49292"
+ ry="2.761292" />
+ <text
+ xml:space="preserve"
+ style="font-size:3.5859px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.317649;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.952945, 0.317649;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-60.009941"
+ y="186.38451"
+ id="text4156"
+ transform="matrix(0,-1.040508,0.96106903,0,0,0)"><tspan
+ sodipodi:role="line"
+ id="tspan4154"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-family:Arial;-inkscape-font-specification:'Arial Bold';stroke-width:0.317648"
+ x="-60.009941"
+ y="186.38451">Machine Learning</tspan><tspan
+ sodipodi:role="line"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-family:Arial;-inkscape-font-specification:'Arial Bold';stroke-width:0.317648"
+ x="-64.492317"
+ y="186.38451"
+ id="tspan4158">Hardware Engine</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-99.705231"
+ y="125.91087"
+ id="text3708"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-99.705231"
+ y="125.91087">lcore 1</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-145.3221"
+ y="125.50572"
+ id="text3708-8"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-7"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-145.3221"
+ y="125.50572">lcore 5</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-122.51577"
+ y="125.52089"
+ id="text3708-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-87"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-122.51577"
+ y="125.52089">lcore 3</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-162.06549"
+ y="125.4589"
+ id="text4542"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4540"
+ style="font-size:5.64444px;stroke-width:0.75"
+ x="-162.06549"
+ y="125.4589">Enqueue Workers</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-99.705231"
+ y="232.67706"
+ id="text3708-9"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-9"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-99.705231"
+ y="232.67706">lcore 2</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-122.51025"
+ y="232.66466"
+ id="text3708-7"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-8"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-122.51025"
+ y="232.66466">lcore 4</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-145.33035"
+ y="232.65778"
+ id="text3708-78"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-0"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-145.33035"
+ y="232.65778">lcore 6</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-162.06549"
+ y="232.59988"
+ id="text4542-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4540-7"
+ style="font-size:5.64444px;stroke-width:0.75"
+ x="-162.06549"
+ y="232.59988">Dequeue Workers</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:6.35px;font-family:Arial;-inkscape-font-specification:'Arial Bold';text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-177.01665"
+ y="220.07283"
+ id="text5181"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan5179"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-177.01665"
+ y="220.07283">test: inference_ordered</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-weight:bold;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:'Arial Bold';text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-178.63324"
+ y="98.67057"
+ id="text15571"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan15569"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.5;stroke-opacity:1"
+ x="-178.63324"
+ y="98.67057">nb_worker_threads = 2 * MIN(nb_queue_pairs, (lcore_count - 1) / 2)</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-weight:bold;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:'Arial Bold';text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-171.18085"
+ y="89.26754"
+ id="text15571-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan15569-9"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.5;stroke-opacity:1"
+ x="-171.18085"
+ y="89.26754">inferences_per_queue_pair = repetitions / nb_queue_pairs</tspan></text>
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.75;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#RoundedArrow);marker-end:url(#RoundedArrow)"
+ d="m 179.14101,85.813438 0,-12.588618"
+ id="path31090"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1924" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724"
+ width="32.290321"
+ height="11.709678"
+ x="216.5636"
+ y="93.411476"
+ ry="2" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-4"
+ width="32.290321"
+ height="11.709678"
+ x="216.5636"
+ y="116.22478"
+ ry="2" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-6"
+ width="32.290321"
+ height="11.709678"
+ x="216.5636"
+ y="139.03798"
+ ry="2" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.354849;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.709699, 0.709699;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924-6"
+ width="44.145252"
+ height="72.532341"
+ x="210.6364"
+ y="85.813477"
+ ry="4.31247" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-0"
+ width="32.290321"
+ height="11.709678"
+ x="109.42807"
+ y="93.411476"
+ ry="2" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-6-7"
+ width="32.290321"
+ height="11.709678"
+ x="109.42807"
+ y="139.03798"
+ ry="2"
+ inkscape:connector-avoid="true" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-4-8"
+ width="32.290321"
+ height="11.709678"
+ x="109.42807"
+ y="116.22478"
+ ry="2"
+ inkscape:connector-avoid="true" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.354849;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.709699, 0.709699;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924-6-3"
+ width="44.145252"
+ height="72.532341"
+ x="103.50092"
+ y="85.813477"
+ ry="4.31247" />
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/tools/img/mldev_model_ops_subtest_a.svg b/doc/guides/tools/img/mldev_model_ops_subtest_a.svg
new file mode 100644
index 0000000000..cce5c3be7c
--- /dev/null
+++ b/doc/guides/tools/img/mldev_model_ops_subtest_a.svg
@@ -0,0 +1,418 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="275mm"
+ height="135mm"
+ viewBox="0 0 275 135"
+ version="1.1"
+ id="svg5"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="mldev_model_ops_subtest_d.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg">
+ <sodipodi:namedview
+ id="namedview7"
+ pagecolor="#ffffff"
+ bordercolor="#000000"
+ borderopacity="0.25"
+ inkscape:showpageshadow="2"
+ inkscape:pageopacity="0.0"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1"
+ inkscape:document-units="mm"
+ showgrid="false"
+ inkscape:zoom="0.72515432"
+ inkscape:cx="372.33454"
+ inkscape:cy="401.29389"
+ inkscape:window-width="1920"
+ inkscape:window-height="986"
+ inkscape:window-x="-11"
+ inkscape:window-y="-11"
+ inkscape:window-maximized="1"
+ inkscape:current-layer="layer1" />
+ <defs
+ id="defs2">
+ <marker
+ style="overflow:visible"
+ id="RoundedArrow"
+ refX="6"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="RoundedArrow"
+ markerWidth="6.1347523"
+ markerHeight="5.9304948"
+ viewBox="0 0 6.1347524 5.9304951"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.7)"
+ d="m -0.21114562,-4.1055728 6.42229122,3.21114561 a 1,1 90 0 1 0,1.78885438 L -0.21114562,4.1055728 A 1.236068,1.236068 31.717474 0 1 -2,3 v -6 a 1.236068,1.236068 148.28253 0 1 1.78885438,-1.1055728 z"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:none"
+ id="path1367" />
+ </marker>
+ </defs>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1">
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 62.203489,25.425124 H 80.823463"
+ id="path1916"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234"
+ inkscape:connection-end="#rect234-07" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 128.19001,25.425124 h 18.6197"
+ id="path1918"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-07"
+ inkscape:connection-end="#rect234-6" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 194.17681,25.425125 h 18.61942"
+ id="path1922"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-6"
+ inkscape:connection-end="#rect234-0-6-6" />
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.633453;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234"
+ width="47.366547"
+ height="17.366549"
+ x="14.836943"
+ y="16.74185"
+ ry="1.4985789"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-25.888845"
+ y="38.514706"
+ id="text290"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-25.888845"
+ y="38.514706">Model 0 / Load</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.633453;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-07"
+ width="47.366547"
+ height="17.366549"
+ x="80.823463"
+ y="16.74185"
+ ry="1.4985789"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-25.888845"
+ y="104.31795"
+ id="text290-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-11"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-25.888845"
+ y="104.31795">Model 0 / Start</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.6329;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-6"
+ width="47.3671"
+ height="17.33604"
+ x="146.80971"
+ y="16.757105"
+ ry="1.4959463" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-25.362436"
+ y="170.39679"
+ id="text290-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-15"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-25.362436"
+ y="170.39679">Model 0 / Stop</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6"
+ width="47.3671"
+ height="17.33604"
+ x="212.79623"
+ y="16.757105"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-25.888849"
+ y="236.47427"
+ id="text290-3-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-25.888849"
+ y="236.47427">Model 0 / Unload</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 236.47978,34.093145 V 44.957249 H 38.520216 v 10.84885"
+ id="path1924"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6"
+ inkscape:connection-end="#rect234-0-6-6-3" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 62.203766,64.474119 H 80.823189"
+ id="path1926"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-3"
+ inkscape:connection-end="#rect234-0-6-6-2" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 128.19029,64.474119 h 18.61942"
+ id="path1928"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-2"
+ inkscape:connection-end="#rect234-0-6-1" />
+ <rect
+ style="fill:#fff6d5;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-3"
+ width="47.3671"
+ height="17.33604"
+ x="14.836666"
+ y="55.806099"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-64.937843"
+ y="38.541786"
+ id="text290-3-2-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-64.937843"
+ y="38.541786">Model 1 / Load</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-2"
+ width="47.3671"
+ height="17.33604"
+ x="80.823189"
+ y="55.806099"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-64.937843"
+ y="104.31795"
+ id="text290-3-2-2-28"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-2"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-64.937843"
+ y="104.31795">Model 1 / Start</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6"
+ width="47.3671"
+ height="17.33604"
+ x="212.79623"
+ y="55.806099"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-64.937843"
+ y="236.47427"
+ id="text290-3-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-64.937843"
+ y="236.47427">Model 1 / Unload</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-1"
+ width="47.3671"
+ height="17.33604"
+ x="146.80971"
+ y="55.806099"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-64.41143"
+ y="170.39679"
+ id="text290-3-2-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-8"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-64.41143"
+ y="170.39679">Model 1 / Stop</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 194.17681,64.474119 h 18.61942"
+ id="path1930"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6"
+ inkscape:connection-start="#rect234-0-6-1" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 236.47978,73.142139 V 83.990988 H 38.520216 V 94.85511"
+ id="path1932"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6-7"
+ inkscape:connection-start="#rect234-0-6" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 62.203766,103.52313 h 9.309711 v 1.3e-4 h 9.309712"
+ id="path1934"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-7" />
+ <rect
+ style="fill:#fff6d5;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-7"
+ width="47.3671"
+ height="17.33604"
+ x="14.836666"
+ y="94.85511"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.98687"
+ y="38.541786"
+ id="text290-3-2-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.98687"
+ y="38.541786">Model N / Load</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0"
+ width="47.3671"
+ height="17.33604"
+ x="80.823189"
+ y="94.855164"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.9869"
+ y="104.31795"
+ id="text290-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.9869"
+ y="104.31795">Model N / Start</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-9"
+ width="47.3671"
+ height="17.33604"
+ x="146.80971"
+ y="94.855164"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.4605"
+ y="170.39679"
+ id="text290-3-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.4605"
+ y="170.39679">Model N / Stop</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.633452;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-5"
+ width="47.366547"
+ height="17.366549"
+ x="212.79651"
+ y="94.839836"
+ ry="1.4985789"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.98682"
+ y="236.47427"
+ id="text290-3-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-3"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.98682"
+ y="236.47427">Model N / Unload</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 128.19029,103.52326 h 18.61942"
+ id="path1936"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 194.17681,103.52326 h 9.30985 v -1.5e-4 h 9.30985"
+ id="path1938"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-5" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#fff6d5;fill-rule:evenodd;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-120.71075"
+ y="220.77164"
+ id="text392"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan390"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';fill:#000000;stroke-width:0.5"
+ x="-120.71075"
+ y="220.77164">mldev: model_ops / subtest D</tspan></text>
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.448724;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect2164"
+ width="259.55127"
+ height="119.55128"
+ x="7.7243652"
+ y="7.7243614"
+ ry="1.17494" />
+ </g>
+</svg>
diff --git a/doc/guides/tools/img/mldev_model_ops_subtest_b.svg b/doc/guides/tools/img/mldev_model_ops_subtest_b.svg
new file mode 100644
index 0000000000..53a49a2823
--- /dev/null
+++ b/doc/guides/tools/img/mldev_model_ops_subtest_b.svg
@@ -0,0 +1,421 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="275mm"
+ height="135mm"
+ viewBox="0 0 275 135"
+ version="1.1"
+ id="svg5"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="mldev_model_ops_subtest_a.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg">
+ <sodipodi:namedview
+ id="namedview7"
+ pagecolor="#ffffff"
+ bordercolor="#000000"
+ borderopacity="0.25"
+ inkscape:showpageshadow="2"
+ inkscape:pageopacity="0.0"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1"
+ inkscape:document-units="mm"
+ showgrid="false"
+ inkscape:zoom="0.72515432"
+ inkscape:cx="277.87189"
+ inkscape:cy="401.29389"
+ inkscape:window-width="1920"
+ inkscape:window-height="986"
+ inkscape:window-x="-11"
+ inkscape:window-y="-11"
+ inkscape:window-maximized="1"
+ inkscape:current-layer="layer1" />
+ <defs
+ id="defs2">
+ <marker
+ style="overflow:visible"
+ id="RoundedArrow"
+ refX="6"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="RoundedArrow"
+ markerWidth="6.1347523"
+ markerHeight="5.9304948"
+ viewBox="0 0 6.1347524 5.9304951"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.7)"
+ d="m -0.21114562,-4.1055728 6.42229122,3.21114561 a 1,1 90 0 1 0,1.78885438 L -0.21114562,4.1055728 A 1.236068,1.236068 31.717474 0 1 -2,3 v -6 a 1.236068,1.236068 148.28253 0 1 1.78885438,-1.1055728 z"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:none"
+ id="path1367" />
+ </marker>
+ </defs>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1">
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.671403;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234"
+ width="53.328598"
+ height="17.328598"
+ x="15.144302"
+ y="12.878438"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-22.00646"
+ y="41.803089"
+ id="text290"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-22.00646"
+ y="41.803089">Model 0 / Load</tspan></text>
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.671403;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-07"
+ width="53.328598"
+ height="17.328598"
+ x="87.552399"
+ y="12.878438"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-22.00646"
+ y="114.21132"
+ id="text290-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-11"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-22.00646"
+ y="114.21132">Model 1 / Load</tspan></text>
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.671403;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-6"
+ width="53.328598"
+ height="17.328598"
+ x="206.5271"
+ y="12.878438"
+ ry="1.4953041" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-22.00646"
+ y="233.18588"
+ id="text290-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-15"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-22.00646"
+ y="233.18588">Model N / Load</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 68.4729,21.542737 H 87.552399"
+ id="path1916"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234"
+ inkscape:connection-end="#rect234-07" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 140.881,21.542737 h 65.6461"
+ id="path1918"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-07"
+ inkscape:connection-end="#rect234-6" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 233.1914,30.207036 0,4.918771 H 41.808601 v 4.918772"
+ id="path1922"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-6"
+ inkscape:connection-end="#rect234-0-6-6" />
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6"
+ width="53.328598"
+ height="17.328598"
+ x="15.144302"
+ y="40.044579"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-49.1726"
+ y="41.619808"
+ id="text290-3-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-49.1726"
+ y="41.619808">Model 0 / Start</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-3"
+ width="53.328598"
+ height="17.328598"
+ x="87.552399"
+ y="40.044579"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-49.1726"
+ y="114.02805"
+ id="text290-3-2-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-49.1726"
+ y="114.02805">Model 1 / Start</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-2"
+ width="53.328598"
+ height="17.328598"
+ x="206.5271"
+ y="40.044579"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-49.1726"
+ y="233.00261"
+ id="text290-3-2-2-28"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-2"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-49.1726"
+ y="233.00261">Model N / Start</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 68.4729,48.708878 H 87.552399"
+ id="path1924"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6"
+ inkscape:connection-end="#rect234-0-6-6-3" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 140.881,48.708878 h 65.6461"
+ id="path1926"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-3"
+ inkscape:connection-end="#rect234-0-6-6-2" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 233.1914,57.373177 V 62.29195 H 41.808599 v 4.918774"
+ id="path1928"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-2"
+ inkscape:connection-end="#rect234-0-6-1" />
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6"
+ width="53.328598"
+ height="17.328598"
+ x="87.552399"
+ y="67.210724"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-75.812294"
+ y="114.12037"
+ id="text290-3-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-75.812294"
+ y="114.12037">Model 1 / Stop</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-1"
+ width="53.328598"
+ height="17.328598"
+ x="15.1443"
+ y="67.210724"
+ ry="1.4953041" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-75.812294"
+ y="41.712139"
+ id="text290-3-2-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-8"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-75.812294"
+ y="41.712139">Model 0 / Stop</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-7"
+ width="53.328598"
+ height="17.328598"
+ x="206.5271"
+ y="67.210724"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-75.812294"
+ y="233.09494"
+ id="text290-3-2-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-75.812294"
+ y="233.09494">Model N / Stop</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 68.472898,75.875023 H 87.552399"
+ id="path1930"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6"
+ inkscape:connection-start="#rect234-0-6-1" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 140.881,75.875023 h 65.6461"
+ id="path1932"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6-7"
+ inkscape:connection-start="#rect234-0-6" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 233.1914,84.539322 0,4.91877 H 41.808602 v 4.91877"
+ id="path1934"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0"
+ inkscape:connection-start="#rect234-0-6-7" />
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0"
+ width="53.328598"
+ height="17.328598"
+ x="15.144303"
+ y="94.376862"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.50489"
+ y="41.803085"
+ id="text290-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.50489"
+ y="41.803085">Model 0 / Unload</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-9"
+ width="53.328598"
+ height="17.328598"
+ x="87.552399"
+ y="94.376862"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.50489"
+ y="114.2113"
+ id="text290-3-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.50489"
+ y="114.2113">Model 1 / Unload</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-5"
+ width="53.328598"
+ height="17.328598"
+ x="206.5271"
+ y="94.376862"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.50489"
+ y="233.18588"
+ id="text290-3-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-3"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.50489"
+ y="233.18588">Model N / Unload</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 68.472901,103.04116 H 87.552399"
+ id="path1936"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-9"
+ inkscape:connection-start="#rect234-0" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 140.881,103.04116 h 65.6461"
+ id="path1938"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-5"
+ inkscape:connection-start="#rect234-0-9" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#fff6d5;fill-rule:evenodd;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-119.78458"
+ y="220.32892"
+ id="text392"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan390"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';fill:#000000;stroke-width:0.5"
+ x="-119.78458"
+ y="220.32892">mldev: model_ops / subtest A</tspan></text>
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.442854;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect3967"
+ width="259.55716"
+ height="119.55714"
+ x="7.7214203"
+ y="7.7214317"
+ ry="1.1806604" />
+ </g>
+</svg>
diff --git a/doc/guides/tools/img/mldev_model_ops_subtest_c.svg b/doc/guides/tools/img/mldev_model_ops_subtest_c.svg
new file mode 100644
index 0000000000..320d4978e3
--- /dev/null
+++ b/doc/guides/tools/img/mldev_model_ops_subtest_c.svg
@@ -0,0 +1,364 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="275mm"
+ height="135mm"
+ viewBox="0 0 275 135"
+ version="1.1"
+ id="svg5"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="mldev_model_ops_subtest_c.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg">
+ <sodipodi:namedview
+ id="namedview7"
+ pagecolor="#ffffff"
+ bordercolor="#000000"
+ borderopacity="0.25"
+ inkscape:showpageshadow="2"
+ inkscape:pageopacity="0.0"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1"
+ inkscape:document-units="mm"
+ showgrid="false"
+ inkscape:zoom="0.72515432"
+ inkscape:cx="479.89785"
+ inkscape:cy="401.29389"
+ inkscape:window-width="1920"
+ inkscape:window-height="986"
+ inkscape:window-x="-11"
+ inkscape:window-y="-11"
+ inkscape:window-maximized="1"
+ inkscape:current-layer="layer1" />
+ <defs
+ id="defs2">
+ <marker
+ style="overflow:visible"
+ id="RoundedArrow"
+ refX="6"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="RoundedArrow"
+ markerWidth="6.1347523"
+ markerHeight="5.9304948"
+ viewBox="0 0 6.1347524 5.9304951"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.7)"
+ d="m -0.21114562,-4.1055728 6.42229122,3.21114561 a 1,1 90 0 1 0,1.78885438 L -0.21114562,4.1055728 A 1.236068,1.236068 31.717474 0 1 -2,3 v -6 a 1.236068,1.236068 148.28253 0 1 1.78885438,-1.1055728 z"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:none"
+ id="path1367" />
+ </marker>
+ </defs>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1">
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.671403;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234"
+ width="53.328598"
+ height="17.328598"
+ x="12.6443"
+ y="13.208546"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-22.336561"
+ y="39.303089"
+ id="text290"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-22.336561"
+ y="39.303089">Model 0 / Load</tspan></text>
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.671403;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-07"
+ width="53.328598"
+ height="17.328598"
+ x="85.052399"
+ y="13.208546"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-22.336561"
+ y="111.71132"
+ id="text290-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-11"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-22.336561"
+ y="111.71132">Model 1 / Load</tspan></text>
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.671403;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-6"
+ width="53.328598"
+ height="17.328598"
+ x="204.0271"
+ y="13.208546"
+ ry="1.4953041" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-22.336561"
+ y="230.68588"
+ id="text290-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-15"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-22.336561"
+ y="230.68588">Model N / Load</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 65.972898,21.872845 H 85.052399"
+ id="path1916"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234"
+ inkscape:connection-end="#rect234-07" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 138.381,21.872845 h 65.6461"
+ id="path1918"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-07"
+ inkscape:connection-end="#rect234-6" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 230.6914,30.537144 V 36.33787 H 39.308599 v 5.800726"
+ id="path1922"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-6"
+ inkscape:connection-end="#rect234-0-6-6" />
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6"
+ width="53.328598"
+ height="17.328598"
+ x="12.6443"
+ y="42.138596"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-51.266617"
+ y="39.119808"
+ id="text290-3-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-51.266617"
+ y="39.119808">Model 0 / Start</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-3"
+ width="53.328598"
+ height="17.328598"
+ x="85.052399"
+ y="42.138596"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-51.266617"
+ y="111.52805"
+ id="text290-3-2-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-51.266617"
+ y="111.52805">Model 0 / Stop</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 65.972898,50.802895 H 85.052399"
+ id="path1924"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6"
+ inkscape:connection-end="#rect234-0-6-6-3" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 184.94759,79.732941 H 204.0271"
+ id="path1930"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6"
+ inkscape:connection-start="#rect234-0-6-1" />
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0"
+ width="53.328598"
+ height="17.328598"
+ x="12.644301"
+ y="99.998688"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-109.12671"
+ y="39.303085"
+ id="text290-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-109.12671"
+ y="39.303085">Model 0 / Unload</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-9"
+ width="53.328598"
+ height="17.328598"
+ x="85.052399"
+ y="99.998688"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-109.12671"
+ y="111.7113"
+ id="text290-3-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-109.12671"
+ y="111.7113">Model 1 / Unload</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-5"
+ width="53.328598"
+ height="17.328598"
+ x="204.0271"
+ y="99.998688"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-109.12671"
+ y="230.68588"
+ id="text290-3-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-3"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-109.12671"
+ y="230.68588">Model N / Unload</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 65.972899,108.66299 h 19.0795"
+ id="path1936"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-9"
+ inkscape:connection-start="#rect234-0" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 138.381,108.66299 h 65.6461"
+ id="path1938"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-5"
+ inkscape:connection-start="#rect234-0-9" />
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6"
+ width="53.328598"
+ height="17.328598"
+ x="204.0271"
+ y="71.068642"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-79.670258"
+ y="230.59494"
+ id="text290-3-2-2-28"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-2"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-79.670258"
+ y="230.59494">Model N / Stop</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-1"
+ width="53.328598"
+ height="17.328598"
+ x="131.61899"
+ y="71.068642"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-80.196663"
+ y="158.0945"
+ id="text290-3-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-80.196663"
+ y="158.0945">Model N / Start</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 138.381,50.802895 h 19.90229 v 20.265747"
+ id="path1158"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-3"
+ inkscape:connection-end="#rect234-0-6-1" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 230.6914,88.39724 v 5.800724 H 39.3086 v 5.800724"
+ id="path1160"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6"
+ inkscape:connection-end="#rect234-0" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#fff6d5;fill-rule:evenodd;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-123.28999"
+ y="217.99152"
+ id="text392"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan390"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';fill:#000000;stroke-width:0.5"
+ x="-123.28999"
+ y="217.99152">mldev: model_ops / subtest C</tspan></text>
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.441855;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect3015"
+ width="259.55814"
+ height="119.55814"
+ x="7.720932"
+ y="7.7209282"
+ ry="1.1638433" />
+ </g>
+</svg>
diff --git a/doc/guides/tools/img/mldev_model_ops_subtest_d.svg b/doc/guides/tools/img/mldev_model_ops_subtest_d.svg
new file mode 100644
index 0000000000..80c1798d99
--- /dev/null
+++ b/doc/guides/tools/img/mldev_model_ops_subtest_d.svg
@@ -0,0 +1,422 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="275mm"
+ height="135mm"
+ viewBox="0 0 275 135"
+ version="1.1"
+ id="svg5"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="mldev_model_ops_subtest_b.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg">
+ <sodipodi:namedview
+ id="namedview7"
+ pagecolor="#ffffff"
+ bordercolor="#000000"
+ borderopacity="0.25"
+ inkscape:showpageshadow="2"
+ inkscape:pageopacity="0.0"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1"
+ inkscape:document-units="mm"
+ showgrid="false"
+ inkscape:zoom="0.72515432"
+ inkscape:cx="324.06895"
+ inkscape:cy="295.1096"
+ inkscape:window-width="1920"
+ inkscape:window-height="986"
+ inkscape:window-x="-11"
+ inkscape:window-y="-11"
+ inkscape:window-maximized="1"
+ inkscape:current-layer="layer1" />
+ <defs
+ id="defs2">
+ <marker
+ style="overflow:visible"
+ id="RoundedArrow"
+ refX="6"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="RoundedArrow"
+ markerWidth="6.1347523"
+ markerHeight="5.9304948"
+ viewBox="0 0 6.1347524 5.9304951"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.7)"
+ d="m -0.21114562,-4.1055728 6.42229122,3.21114561 a 1,1 90 0 1 0,1.78885438 L -0.21114562,4.1055728 A 1.236068,1.236068 31.717474 0 1 -2,3 v -6 a 1.236068,1.236068 148.28253 0 1 1.78885438,-1.1055728 z"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:none"
+ id="path1367" />
+ </marker>
+ </defs>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1">
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.633453;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234"
+ width="47.366547"
+ height="17.366549"
+ x="14.864025"
+ y="14.24185"
+ ry="1.4985789"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-23.388845"
+ y="38.541786"
+ id="text290"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-23.388845"
+ y="38.541786">Model 0 / Load</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.633453;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-07"
+ width="47.366547"
+ height="17.366549"
+ x="81.026939"
+ y="14.24185"
+ ry="1.4985789"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-23.388845"
+ y="104.52142"
+ id="text290-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-11"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-23.388845"
+ y="104.52142">Model 0 / Start</tspan></text>
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.6329;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-6"
+ width="47.3671"
+ height="17.33604"
+ x="147.18958"
+ y="14.257105"
+ ry="1.4959463" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-23.388849"
+ y="170.86761"
+ id="text290-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-15"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-23.388849"
+ y="170.86761">Model 1 / Load</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 62.230572,22.925124 H 81.026939"
+ id="path1916"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234"
+ inkscape:connection-end="#rect234-07" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 128.39349,22.925124 h 18.79609"
+ id="path1918"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-07"
+ inkscape:connection-end="#rect234-6" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 194.55667,22.925125 18.79581,0"
+ id="path1922"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-6"
+ inkscape:connection-end="#rect234-0-6-6" />
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6"
+ width="47.3671"
+ height="17.33604"
+ x="213.35248"
+ y="14.257105"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-23.388849"
+ y="236.84723"
+ id="text290-3-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-23.388849"
+ y="236.84723">Model 1 / Start</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 237.03603,31.593145 0,5.328675 H 38.547297 v 5.313421"
+ id="path1924"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6"
+ inkscape:connection-end="#rect234-0-6-6-3" />
+ <rect
+ style="fill:#fff6d5;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-3"
+ width="47.3671"
+ height="17.33604"
+ x="14.863747"
+ y="42.235241"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-51.366989"
+ y="38.541786"
+ id="text290-3-2-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-51.366989"
+ y="38.541786">Model N / Load</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-2"
+ width="47.3671"
+ height="17.33604"
+ x="81.026657"
+ y="42.235241"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-51.366989"
+ y="104.52142"
+ id="text290-3-2-2-28"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-2"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-51.366989"
+ y="104.52142">Model N / Start</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 62.230846,50.903261 H 81.026657"
+ id="path1926"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-3"
+ inkscape:connection-end="#rect234-0-6-6-2" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 128.39376,50.903261 h 42.47937 v 19.294584"
+ id="path1928"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-2"
+ inkscape:connection-end="#rect234-0-6-1" />
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6"
+ width="47.3671"
+ height="17.33604"
+ x="213.35248"
+ y="70.197845"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-79.32959"
+ y="237.03052"
+ id="text290-3-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-79.32959"
+ y="237.03052">Model N / Unload</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-1"
+ width="47.3671"
+ height="17.33604"
+ x="147.18958"
+ y="70.197845"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-78.803177"
+ y="170.77666"
+ id="text290-3-2-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-8"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-78.803177"
+ y="170.77666">Model N / Stop</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 194.55667,78.865866 h 18.79581"
+ id="path1930"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6"
+ inkscape:connection-start="#rect234-0-6-1" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 237.03603,87.533886 v 5.313417 H 38.547297 v 5.328677"
+ id="path1932"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6-7"
+ inkscape:connection-start="#rect234-0-6" />
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-7"
+ width="47.3671"
+ height="17.33604"
+ x="14.863747"
+ y="98.17598"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-106.78132"
+ y="38.450832"
+ id="text290-3-2-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-106.78132"
+ y="38.450832">Model 1 / Stop</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 62.230846,106.844 H 81.026657"
+ id="path1934"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0"
+ inkscape:connection-start="#rect234-0-6-7" />
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0"
+ width="47.3671"
+ height="17.33604"
+ x="81.026657"
+ y="98.17598"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-107.30773"
+ y="104.7047"
+ id="text290-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-107.30773"
+ y="104.7047">Model 1 / Unload</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-9"
+ width="47.3671"
+ height="17.33604"
+ x="147.18958"
+ y="98.17598"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-106.78133"
+ y="170.77666"
+ id="text290-3-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-106.78133"
+ y="170.77666">Model 0 / Stop</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.633452;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-5"
+ width="47.366547"
+ height="17.366549"
+ x="213.35277"
+ y="98.160721"
+ ry="1.4985789"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-107.30773"
+ y="237.03053"
+ id="text290-3-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-3"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-107.30773"
+ y="237.03053">Model 0 / Unload</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 128.39376,106.844 h 18.79582"
+ id="path1936"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-9"
+ inkscape:connection-start="#rect234-0" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 194.55667,106.844 9.39805,0 v 0 h 9.39805"
+ id="path1938"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-5"
+ inkscape:connection-start="#rect234-0-9" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#fff6d5;fill-rule:evenodd;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-122.00021"
+ y="222.26495"
+ id="text392"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan390"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';fill:#000000;stroke-width:0.5"
+ x="-122.00021"
+ y="222.26495">mldev: model_ops / subest B</tspan></text>
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.462341;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect3475"
+ width="259.53769"
+ height="119.53766"
+ x="7.7311554"
+ y="7.7311707"
+ ry="1.2186421" />
+ </g>
+</svg>
diff --git a/doc/guides/tools/index.rst b/doc/guides/tools/index.rst
index f1f5b94c8c..6f84fc31ff 100644
--- a/doc/guides/tools/index.rst
+++ b/doc/guides/tools/index.rst
@@ -21,4 +21,5 @@ DPDK Tools User Guides
comp_perf
testeventdev
testregex
+ testmldev
dts
diff --git a/doc/guides/tools/testmldev.rst b/doc/guides/tools/testmldev.rst
new file mode 100644
index 0000000000..845c2d9381
--- /dev/null
+++ b/doc/guides/tools/testmldev.rst
@@ -0,0 +1,441 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright (c) 2022 Marvell.
+
+dpdk-test-mldev Application
+===========================
+
+The ``dpdk-test-mldev`` tool is a Data Plane Development Kit (DPDK) application that allows testing
+various mldev use cases. This application has a generic framework to add new mldev based test cases
+to verify functionality and measure the performance of inference execution on DPDK ML devices.
+
+
+Application and Options
+-----------------------
+
+The application has a number of command line options:
+
+.. code-block:: console
+
+ dpdk-test-mldev [EAL Options] -- [application options]
+
+EAL Options
+~~~~~~~~~~~
+
+The following are the EAL command-line options that can be used with the ``dpdk-test-mldev``
+application. See the DPDK Getting Started Guides for more information on these options.
+
+* ``-c <COREMASK>`` or ``-l <CORELIST>``
+
+ Set the hexadecimal bitmask of the cores to run on. The corelist is a list of cores to use.
+
+* ``-a <PCI_ID>``
+
+ Attach a PCI based ML device. Specific to drivers using a PCI based ML devices.
+
+* ``--vdev <driver>``
+
+ Add a virtual mldev device. Specific to drivers using a ML virtual device.
+
+
+Application Options
+~~~~~~~~~~~~~~~~~~~
+
+The following are the command-line options supported by the test application.
+
+* ``--test <name>``
+
+ ML tests are divided into two groups, Model and Device tests and Inference tests. Test
+ name one of the following supported tests.
+
+ **ML Device Tests** ::
+
+ device_ops
+
+ **ML Model Tests** ::
+
+ model_ops
+
+ **ML Inference Tests** ::
+
+ inference_ordered
+ inference_interleave
+
+* ``--dev_id <n>``
+
+ Set the device id of the ML device to be used for the test. Default value is `0`.
+
+* ``--socket_id <n>``
+
+ Set the socket id of the application resources. Default value is `SOCKET_ID_ANY`.
+
+* ``--debug``
+
+ Enable the tests to run in debug mode.
+
+* ``--models <model_list>``
+
+ Set the list of model files to be used for the tests. Application expects the
+ ``model_list`` in comma separated form (i.e. ``--models model_A.bin,model_B.bin``).
+ Maximum number of models supported by the test is ``8``.
+
+* ``--filelist <file_list>``
+
+ Set the list of model, input, output and reference files to be used for the tests.
+ Application expects the ``file_list`` to be in comma separated form
+ (i.e. ``--filelist <model,input,output>[,reference]``).
+
+ Multiple filelist entries can be specified when running the tests with multiple models.
+ Both quantized and dequantized outputs are written to the disk. Dequantized output file
+ would have the name specified by the user through ``--filelist`` option. A suffix ``.q``
+ is appended to quantized output filename. Maximum number of filelist entries supported
+ by the test is ``8``.
+
+* ``--repetitions <n>``
+
+ Set the number of inference repetitions to be executed in the test per each model. Default
+ value is `1`.
+
+* ``--burst_size <n>``
+
+ Set the burst size to be used when enqueuing / dequeuing inferences. Default value is `1`.
+
+* ``--queue_pairs <n>``
+
+ Set the number of queue-pairs to be used for inference enqueue and dequeue operations.
+ Default value is `1`.
+
+* ``--queue_size <n>``
+
+ Set the size of queue-pair to be created for inference enqueue / dequeue operations.
+ Queue size would translate into `rte_ml_dev_qp_conf::nb_desc` field during queue-pair
+ creation. Default value is `1`.
+
+* ``--batches <n>``
+
+ Set the number batches in the input file provided for inference run. When not specified
+ the test would assume the number of batches is equal to the batch size of the model.
+
+* ``--tolerance <n>``
+
+ Set the tolerance value in percentage to be used for output validation. Default value
+ is `0`.
+
+* ``--stats``
+
+ Enable reporting device extended stats.
+
+
+ML Device Tests
+-------------------------
+
+ML device tests are functional tests to validate ML device APIs. Device tests validate the ML device
+handling APIs configure, close, start and stop APIs.
+
+
+Application Options
+~~~~~~~~~~~~~~~~~~~
+
+Supported command line options for the `model_ops` test are following::
+
+ --debug
+ --test
+ --dev_id
+ --socket_id
+ --queue_pairs
+ --queue_size
+
+
+DEVICE_OPS Test
+~~~~~~~~~~~~~~~
+
+Device ops test validates the device configuration and reconfiguration support. The test configures
+ML device based on the option ``--queue_pairs`` and ``--queue_size`` specified by the user, and
+later reconfigures the ML device with the number of queue pairs and queue size based the maximum
+specified through the device info.
+
+
+Example
+^^^^^^^
+
+Command to run device_ops test:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=device_ops
+
+
+Command to run device_ops test with user options:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=device_ops --queue_pairs <M> --queue_size <N>
+
+
+ML Model Tests
+-------------------------
+
+Model tests are functional tests to validate ML model APIs. Model tests validate the functioning
+of APIs to load, start, stop and unload ML models.
+
+
+Application Options
+~~~~~~~~~~~~~~~~~~~
+
+Supported command line options for the `model_ops` test are following::
+
+ --debug
+ --test
+ --dev_id
+ --socket_id
+ --models
+
+
+List of model files to be used for the `model_ops` test can be specified through the option
+``--models <model_list>`` as a comma separated list. Maximum number of models supported in
+the test is `8`.
+
+.. Note::
+
+ * The ``--models <model_list>`` is a mandatory option for running this test.
+ * Options not supported by the test are ignored if specified.
+
+
+MODEL_OPS Test
+~~~~~~~~~~~~~~
+
+The test is a collection of multiple sub-tests, each with a different order of slow-path
+operations when handling with `N` number of models.
+
+
+**Sub-test A:** executes the sequence of load / start / stop / unload for a model in order,
+followed by next model.
+.. _figure_mldev_model_ops_subtest_a:
+
+.. figure:: img/mldev_model_ops_subtest_a.*
+
+ Execution sequence of model_ops subtest A.
+
+
+**Sub-test B:** executes load for all models, followed by a start for all models. Upon successful
+start of all models, stop is invoked for all models followed by unload.
+.. _figure_mldev_model_ops_subtest_b:
+
+.. figure:: img/mldev_model_ops_subtest_b.*
+
+ Execution sequence of model_ops subtest B.
+
+
+**Sub-test C:** loads all models, followed by a start and stop of all models in order. Upon
+completion of stop, unload is invoked for all models.
+.. _figure_mldev_model_ops_subtest_c:
+
+.. figure:: img/mldev_model_ops_subtest_c.*
+
+ Execution sequence of model_ops subtest C.
+
+
+**Sub-test D:** executes load and start for all models available. Upon successful start of all
+models, stop and stop is executed for the models.
+.. _figure_mldev_model_ops_subtest_d:
+
+.. figure:: img/mldev_model_ops_subtest_d.*
+
+ Execution sequence of model_ops subtest D.
+
+
+Example
+^^^^^^^
+
+Command to run model_ops test:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=model_ops --models model_1.bin,model_2.bin,model_3.bin, model_4.bin
+
+
+ML Inference Tests
+------------------
+
+Inference tests are a set of tests to validate end-to-end inference execution on ML device.
+These tests executes the full sequence of operations required to run inferences with one or
+multiple models.
+
+Application Options
+~~~~~~~~~~~~~~~~~~~
+
+Supported command line options for inference tests are following::
+
+ --debug
+ --test
+ --dev_id
+ --socket_id
+ --filelist
+ --repetitions
+ --burst_size
+ --queue_pairs
+ --queue_size
+ --batches
+ --tolerance
+ --stats
+
+
+List of files to be used for the inference tests can be specified through the option
+``--filelist <file_list>`` as a comma separated list. A filelist entry would be of the format
+``--filelist <model_file,input_file,output_file>[,reference_file]`` and is used to specify the
+list of files required to test with a single model. Multiple filelist entries are supported by
+the test, one entry per model. Maximum number of file entries supported by the test is `8`.
+
+When ``--burst_size <num>`` option is specified for the test, enqueue and dequeue burst would
+try to enqueue or dequeue ``num`` number of inferences per each call respectively.
+
+In the inference test, a pair of lcores are mapped to each queue pair. Minimum number of lcores
+required for the tests is equal to ``(queue_pairs * 2 + 1)``.
+
+Output validation of inference would be enabled only when a reference file is specified through
+the ``--filelist`` option. Application would additionally consider the tolerance value provided
+through ``--tolerance`` option during validation. When the tolerance values is 0, CRC32 hash of
+inference output and reference output are compared. When the tolerance is non-zero, element wise
+comparison of output is performed. Validation is considered as successful only when all the
+elements of the output tensor are with in the tolerance range specified.
+
+When ``--debug`` option is specified, tests are run in debug mode.
+
+Enabling ``--stats`` would print the extended stats supported by the driver.
+
+.. Note::
+
+ * The ``--filelist <file_list>`` is a mandatory option for running inference tests.
+ * Options not supported by the tests are ignored if specified.
+ * Element wise comparison is not supported when the output dtype is either fp8, fp16
+ or bfloat16. This is applicable only when the tolerance is greater than zero and for
+ pre-quantized models only.
+
+
+INFERENCE_ORDERED Test
+~~~~~~~~~~~~~~~~~~~~~~
+
+This is a functional test for validating the end-to-end inference execution on ML device. This
+test configures ML device and queue pairs as per the queue-pair related options (queue_pairs and
+queue_size) specified by the user. Upon successful configuration of the device and queue pairs,
+the first model specified through the filelist is loaded to the device and inferences are enqueued
+by a pool of worker threads to the ML device. Total number of inferences enqueued for the model
+are equal to the repetitions specified. A dedicated pool of worker threads would dequeue the
+inferences from the device. The model is unloaded upon completion of all inferences for the model.
+The test would continue loading and executing inference requests for all models specified
+through ``filelist`` option in an ordered manner.
+
+.. _figure_mldev_inference_ordered:
+
+.. figure:: img/mldev_inference_ordered.*
+
+ Execution of inference_ordered on single model.
+
+
+Example
+^^^^^^^
+
+Example command to run inference_ordered test:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_ordered --filelist model.bin,input.bin,output.bin
+
+Example command to run inference_ordered with output validation using tolerance of `1%``:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_ordered --filelist model.bin,input.bin,output.bin,reference.bin \
+ --tolerance 1.0
+
+Example command to run inference_ordered test with multiple queue-pairs and queue size:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_ordered --filelist model.bin,input.bin,output.bin \
+ --queue_pairs 4 --queue_size 16
+
+Example command to run inference_ordered test with a specific burst size:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_ordered --filelist model.bin,input.bin,output.bin \
+ --burst_size 12
+
+
+INFERENCE_INTERLEAVE Test
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This is a stress test for validating the end-to-end inference execution on ML device. The test
+configures the ML device and queue pairs as per the queue-pair related options (queue_pairs
+and queue_size) specified by the user. Upon successful configuration of the device and queue
+pairs, all models specified through the filelist are loaded to the device. Inferences for multiple
+models are enqueued by a pool of worker threads in parallel. Inference execution by the device is
+interleaved between multiple models. Total number of inferences enqueued for a model are equal to
+the repetitions specified. An additional pool of threads would dequeue the inferences from the
+device. Models would be unloaded upon completion of inferences for all models loaded.
+
+
+.. _figure_mldev_inference_interleave:
+
+.. figure:: img/mldev_inference_interleave.*
+
+ Execution of inference_interleave on single model.
+
+
+Example
+^^^^^^^
+
+Example command to run inference_interleave test:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_interleave --filelist model.bin,input.bin,output.bin
+
+
+Example command to run inference_interleave test with multiple models:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_interleave --filelist model_A.bin,input_A.bin,output_A.bin \
+ --filelist model_B.bin,input_B.bin,output_B.bin
+
+
+Example command to run inference_interleave test with multiple models ad output validation
+using tolerance of `2.0%``:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_interleave \
+ --filelist model_A.bin,input_A.bin,output_A.bin,reference_A.bin \
+ --filelist model_B.bin,input_B.bin,output_B.bin,reference_B.bin \
+ --tolerance 2.0
+
+Example command to run inference_interleave test with multiple queue-pairs and queue size
+and burst size:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_interleave --filelist model.bin,input.bin,output.bin \
+ --queue_pairs 8 --queue_size 12 --burst_size 16
+
+
+Debug mode
+----------
+
+ML tests can be executed in debug mode by enabling the option ``--debug``. Execution of tests in
+debug mode would enable additional prints.
+
+When a validation failure is observed, output from that buffer is written to the disk, with the
+filenames having similar convention when the test has passed. Additionally index of the buffer
+would be appended to the filenames.
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v3 01/12] app/mldev: implement test framework for mldev
2022-11-29 8:20 ` [PATCH v2 " Srikanth Yalavarthi
` (10 preceding siblings ...)
2022-11-29 8:21 ` [PATCH v2 12/12] app/mldev: add documentation for mldev test cases Srikanth Yalavarthi
@ 2022-12-08 19:29 ` Srikanth Yalavarthi
2022-12-08 19:29 ` [PATCH v3 02/12] app/mldev: add common test functions Srikanth Yalavarthi
` (10 more replies)
11 siblings, 11 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2022-12-08 19:29 UTC (permalink / raw)
To: Thomas Monjalon, Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu
Implemented framework for mldev test application. New test cases
can be added using the framework. Support is also enabled to add
options specific to the test cases. User can launch the tests by
specifying the name of test as part of launch arguments.
Code to parse command line arguments is imported from
test-eventdev, with support to parse additional data types.
Common arguments supported include:
test : name of the test application to run
dev_id : device id of the ML device
socket_id : socket_id of application resources
debug : enable debugging
help : print help
Sample launch command:
./dpdk-test-mldev -- --test <testname> --dev_id <dev_id> \
--socket_id <socket_id>
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
Depends-on: series-25753 ("mldev: introduce machine learning device library")
MAINTAINERS | 1 +
app/meson.build | 1 +
app/test-mldev/meson.build | 17 ++
app/test-mldev/ml_common.h | 29 +++
app/test-mldev/ml_main.c | 118 +++++++++++
app/test-mldev/ml_options.c | 160 +++++++++++++++
app/test-mldev/ml_options.h | 31 +++
app/test-mldev/ml_test.c | 45 +++++
app/test-mldev/ml_test.h | 75 +++++++
app/test-mldev/parser.c | 380 ++++++++++++++++++++++++++++++++++++
app/test-mldev/parser.h | 55 ++++++
11 files changed, 912 insertions(+)
create mode 100644 app/test-mldev/meson.build
create mode 100644 app/test-mldev/ml_common.h
create mode 100644 app/test-mldev/ml_main.c
create mode 100644 app/test-mldev/ml_options.c
create mode 100644 app/test-mldev/ml_options.h
create mode 100644 app/test-mldev/ml_test.c
create mode 100644 app/test-mldev/ml_test.h
create mode 100644 app/test-mldev/parser.c
create mode 100644 app/test-mldev/parser.h
diff --git a/MAINTAINERS b/MAINTAINERS
index 8a71e2c07a..c76007a340 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -538,6 +538,7 @@ F: doc/guides/prog_guide/rawdev.rst
ML device API - EXPERIMENTAL
M: Srikanth Yalavarthi <syalavarthi@marvell.com>
F: lib/mldev/
+F: app/test-mldev/
F: doc/guides/prog_guide/mldev.rst
diff --git a/app/meson.build b/app/meson.build
index e32ea4bd5c..74d2420f67 100644
--- a/app/meson.build
+++ b/app/meson.build
@@ -23,6 +23,7 @@ apps = [
'test-fib',
'test-flow-perf',
'test-gpudev',
+ 'test-mldev',
'test-pipeline',
'test-pmd',
'test-regex',
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
new file mode 100644
index 0000000000..8ca2e1a1c1
--- /dev/null
+++ b/app/test-mldev/meson.build
@@ -0,0 +1,17 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright (c) 2022 Marvell.
+
+if is_windows
+ build = false
+ reason = 'not supported on Windows'
+ subdir_done()
+endif
+
+sources = files(
+ 'ml_main.c',
+ 'ml_options.c',
+ 'ml_test.c',
+ 'parser.c',
+)
+
+deps += ['mldev']
diff --git a/app/test-mldev/ml_common.h b/app/test-mldev/ml_common.h
new file mode 100644
index 0000000000..065180b619
--- /dev/null
+++ b/app/test-mldev/ml_common.h
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_COMMON_
+#define _ML_COMMON_
+
+#include <stdio.h>
+
+#define CLNRM "\x1b[0m"
+#define CLRED "\x1b[31m"
+#define CLGRN "\x1b[32m"
+#define CLYEL "\x1b[33m"
+
+#define ML_STR_FMT 20
+
+#define ml_err(fmt, args...) fprintf(stderr, CLRED "error: %s() " fmt CLNRM "\n", __func__, ##args)
+
+#define ml_info(fmt, args...) fprintf(stdout, CLYEL "" fmt CLNRM "\n", ##args)
+
+#define ml_dump(str, fmt, val...) printf("\t%-*s : " fmt "\n", ML_STR_FMT, str, ##val)
+
+#define ml_dump_begin(str) printf("\t%-*s :\n\t{\n", ML_STR_FMT, str)
+
+#define ml_dump_list(str, id, val) printf("\t%*s[%2u] : %s\n", ML_STR_FMT - 4, str, id, val)
+
+#define ml_dump_end printf("\b\t}\n\n")
+
+#endif /* _ML_COMMON_*/
diff --git a/app/test-mldev/ml_main.c b/app/test-mldev/ml_main.c
new file mode 100644
index 0000000000..d6652cd7b7
--- /dev/null
+++ b/app/test-mldev/ml_main.c
@@ -0,0 +1,118 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+
+#include <rte_debug.h>
+#include <rte_eal.h>
+#include <rte_mldev.h>
+
+#include "ml_common.h"
+#include "ml_options.h"
+#include "ml_test.h"
+
+struct ml_options opt;
+struct ml_test *test;
+
+int
+main(int argc, char **argv)
+{
+ uint16_t mldevs;
+ int ret;
+
+ ret = rte_eal_init(argc, argv);
+ if (ret < 0)
+ rte_panic("invalid EAL arguments\n");
+ argc -= ret;
+ argv += ret;
+
+ mldevs = rte_ml_dev_count();
+ if (!mldevs)
+ rte_panic("no mldev devices found\n");
+
+ /* set default values for options */
+ ml_options_default(&opt);
+
+ /* parse the command line arguments */
+ ret = ml_options_parse(&opt, argc, argv);
+ if (ret) {
+ ml_err("parsing one or more user options failed");
+ goto error;
+ }
+
+ /* get test struct from name */
+ test = ml_test_get(opt.test_name);
+ if (test == NULL) {
+ ml_err("failed to find requested test: %s", opt.test_name);
+ goto error;
+ }
+
+ if (test->ops.test_result == NULL) {
+ ml_err("%s: ops.test_result not found", opt.test_name);
+ goto error;
+ }
+
+ /* check test options */
+ if (test->ops.opt_check) {
+ if (test->ops.opt_check(&opt)) {
+ ml_err("invalid command line argument");
+ goto error;
+ }
+ }
+
+ /* check the device capability */
+ if (test->ops.cap_check) {
+ if (test->ops.cap_check(&opt) == false) {
+ ml_info("unsupported test: %s", opt.test_name);
+ ret = ML_TEST_UNSUPPORTED;
+ goto no_cap;
+ }
+ }
+
+ /* dump options */
+ if (opt.debug) {
+ if (test->ops.opt_dump)
+ test->ops.opt_dump(&opt);
+ }
+
+ /* test specific setup */
+ if (test->ops.test_setup) {
+ if (test->ops.test_setup(test, &opt)) {
+ ml_err("failed to setup test: %s", opt.test_name);
+ goto error;
+ }
+ }
+
+ /* test driver */
+ if (test->ops.test_driver)
+ test->ops.test_driver(test, &opt);
+
+ /* get result */
+ if (test->ops.test_result)
+ ret = test->ops.test_result(test, &opt);
+
+ if (test->ops.test_destroy)
+ test->ops.test_destroy(test, &opt);
+
+no_cap:
+ if (ret == ML_TEST_SUCCESS) {
+ printf("Result: " CLGRN "%s" CLNRM "\n", "Success");
+ } else if (ret == ML_TEST_FAILED) {
+ printf("Result: " CLRED "%s" CLNRM "\n", "Failed");
+ return EXIT_FAILURE;
+ } else if (ret == ML_TEST_UNSUPPORTED) {
+ printf("Result: " CLYEL "%s" CLNRM "\n", "Unsupported");
+ }
+
+ rte_eal_cleanup();
+
+ return 0;
+
+error:
+ rte_eal_cleanup();
+
+ return EXIT_FAILURE;
+}
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
new file mode 100644
index 0000000000..8fd7760e36
--- /dev/null
+++ b/app/test-mldev/ml_options.c
@@ -0,0 +1,160 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <errno.h>
+#include <getopt.h>
+#include <stdbool.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include <rte_memory.h>
+#include <rte_mldev.h>
+#include <rte_string_fns.h>
+
+#include "ml_common.h"
+#include "ml_options.h"
+#include "ml_test.h"
+#include "parser.h"
+
+typedef int (*option_parser_t)(struct ml_options *opt, const char *arg);
+
+void
+ml_options_default(struct ml_options *opt)
+{
+ memset(opt, 0, sizeof(*opt));
+ strlcpy(opt->test_name, "ml_test", ML_TEST_NAME_MAX_LEN);
+ opt->dev_id = 0;
+ opt->socket_id = SOCKET_ID_ANY;
+ opt->debug = false;
+}
+
+struct long_opt_parser {
+ const char *lgopt_name;
+ option_parser_t parser_fn;
+};
+
+static int
+ml_parse_test_name(struct ml_options *opt, const char *arg)
+{
+ strlcpy(opt->test_name, arg, ML_TEST_NAME_MAX_LEN);
+ return 0;
+}
+
+static int
+ml_parse_dev_id(struct ml_options *opt, const char *arg)
+{
+ int ret;
+
+ ret = parser_read_int16(&opt->dev_id, arg);
+
+ if (ret < 0)
+ return -EINVAL;
+
+ return ret;
+}
+
+static int
+ml_parse_socket_id(struct ml_options *opt, const char *arg)
+{
+ opt->socket_id = atoi(arg);
+
+ return 0;
+}
+
+static void
+ml_dump_test_options(const char *testname)
+{
+ RTE_SET_USED(testname);
+}
+
+static void
+print_usage(char *program)
+{
+ printf("\nusage : %s [EAL options] -- [application options]\n", program);
+ printf("application options:\n");
+ printf("\t--test : name of the test application to run\n"
+ "\t--dev_id : device id of the ML device\n"
+ "\t--socket_id : socket_id of application resources\n"
+ "\t--debug : enable debug mode\n"
+ "\t--help : print help\n");
+ printf("\n");
+ printf("available tests and test specific application options:\n");
+ ml_test_dump_names(ml_dump_test_options);
+}
+
+static struct option lgopts[] = {{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0},
+ {ML_SOCKET_ID, 1, 0, 0}, {ML_DEBUG, 0, 0, 0},
+ {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
+
+static int
+ml_opts_parse_long(int opt_idx, struct ml_options *opt)
+{
+ unsigned int i;
+
+ struct long_opt_parser parsermap[] = {
+ {ML_TEST, ml_parse_test_name},
+ {ML_DEVICE_ID, ml_parse_dev_id},
+ {ML_SOCKET_ID, ml_parse_socket_id},
+ };
+
+ for (i = 0; i < RTE_DIM(parsermap); i++) {
+ if (strncmp(lgopts[opt_idx].name, parsermap[i].lgopt_name,
+ strlen(lgopts[opt_idx].name)) == 0)
+ return parsermap[i].parser_fn(opt, optarg);
+ }
+
+ return -EINVAL;
+}
+
+int
+ml_options_parse(struct ml_options *opt, int argc, char **argv)
+{
+ int opt_idx;
+ int retval;
+ int opts;
+
+ while ((opts = getopt_long(argc, argv, "", lgopts, &opt_idx)) != EOF) {
+ switch (opts) {
+ case 0: /* parse long options */
+ if (!strcmp(lgopts[opt_idx].name, "debug")) {
+ opt->debug = true;
+ break;
+ }
+
+ if (!strcmp(lgopts[opt_idx].name, "help")) {
+ print_usage(argv[0]);
+ exit(EXIT_SUCCESS);
+ }
+
+ retval = ml_opts_parse_long(opt_idx, opt);
+ if (retval != 0)
+ return retval;
+ break;
+ default:
+ return -EINVAL;
+ }
+ }
+
+ return 0;
+}
+
+void
+ml_options_dump(struct ml_options *opt)
+{
+ struct rte_ml_dev_info dev_info;
+
+ rte_ml_dev_info_get(opt->dev_id, &dev_info);
+
+ ml_dump("driver", "%s", dev_info.driver_name);
+ ml_dump("test", "%s", opt->test_name);
+ ml_dump("dev_id", "%d", opt->dev_id);
+
+ if (opt->socket_id == SOCKET_ID_ANY)
+ ml_dump("socket_id", "%d (SOCKET_ID_ANY)", opt->socket_id);
+ else
+ ml_dump("socket_id", "%d", opt->socket_id);
+
+ ml_dump("debug", "%s", (opt->debug ? "true" : "false"));
+}
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
new file mode 100644
index 0000000000..05311a9a47
--- /dev/null
+++ b/app/test-mldev/ml_options.h
@@ -0,0 +1,31 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_OPTIONS_
+#define _ML_OPTIONS_
+
+#include <stdbool.h>
+#include <stdint.h>
+
+#define ML_TEST_NAME_MAX_LEN 32
+
+/* Options names */
+#define ML_TEST ("test")
+#define ML_DEVICE_ID ("dev_id")
+#define ML_SOCKET_ID ("socket_id")
+#define ML_DEBUG ("debug")
+#define ML_HELP ("help")
+
+struct ml_options {
+ char test_name[ML_TEST_NAME_MAX_LEN];
+ int16_t dev_id;
+ int socket_id;
+ bool debug;
+};
+
+void ml_options_default(struct ml_options *opt);
+int ml_options_parse(struct ml_options *opt, int argc, char **argv);
+void ml_options_dump(struct ml_options *opt);
+
+#endif /* _ML_OPTIONS_ */
diff --git a/app/test-mldev/ml_test.c b/app/test-mldev/ml_test.c
new file mode 100644
index 0000000000..2304712764
--- /dev/null
+++ b/app/test-mldev/ml_test.c
@@ -0,0 +1,45 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <stdio.h>
+#include <string.h>
+#include <sys/queue.h>
+
+#include "ml_test.h"
+
+static STAILQ_HEAD(, ml_test_entry) head = STAILQ_HEAD_INITIALIZER(head);
+
+void
+ml_test_register(struct ml_test_entry *entry)
+{
+ STAILQ_INSERT_TAIL(&head, entry, next);
+}
+
+struct ml_test *
+ml_test_get(const char *name)
+{
+ struct ml_test_entry *entry;
+
+ if (!name)
+ return NULL;
+
+ STAILQ_FOREACH(entry, &head, next)
+ if (!strncmp(entry->test.name, name, strlen(name)))
+ return &entry->test;
+
+ return NULL;
+}
+
+void
+ml_test_dump_names(void (*f)(const char *name))
+{
+ struct ml_test_entry *entry;
+
+ STAILQ_FOREACH(entry, &head, next)
+ {
+ if (entry->test.name)
+ printf("\t %s\n", entry->test.name);
+ f(entry->test.name);
+ }
+}
diff --git a/app/test-mldev/ml_test.h b/app/test-mldev/ml_test.h
new file mode 100644
index 0000000000..4a1430ec1b
--- /dev/null
+++ b/app/test-mldev/ml_test.h
@@ -0,0 +1,75 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_TEST_
+#define _ML_TEST_
+
+#include <stdbool.h>
+#include <string.h>
+#include <sys/queue.h>
+
+#include <rte_common.h>
+
+#include "ml_options.h"
+
+#define ML_TEST_MAX_POOL_SIZE 256
+
+enum ml_test_result {
+ ML_TEST_SUCCESS,
+ ML_TEST_FAILED,
+ ML_TEST_UNSUPPORTED,
+};
+
+struct ml_test;
+
+typedef bool (*ml_test_capability_check_t)(struct ml_options *opt);
+typedef int (*ml_test_options_check_t)(struct ml_options *opt);
+typedef void (*ml_test_options_dump_t)(struct ml_options *opt);
+typedef int (*ml_test_setup_t)(struct ml_test *test, struct ml_options *opt);
+typedef void (*ml_test_destroy_t)(struct ml_test *test, struct ml_options *opt);
+typedef int (*ml_test_driver_t)(struct ml_test *test, struct ml_options *opt);
+typedef int (*ml_test_result_t)(struct ml_test *test, struct ml_options *opt);
+
+struct ml_test_ops {
+ ml_test_capability_check_t cap_check;
+ ml_test_options_check_t opt_check;
+ ml_test_options_dump_t opt_dump;
+ ml_test_setup_t test_setup;
+ ml_test_destroy_t test_destroy;
+ ml_test_driver_t test_driver;
+ ml_test_result_t test_result;
+};
+
+struct ml_test {
+ const char *name;
+ void *test_priv;
+ struct ml_test_ops ops;
+};
+
+struct ml_test_entry {
+ struct ml_test test;
+
+ STAILQ_ENTRY(ml_test_entry) next;
+};
+
+static inline void *
+ml_test_priv(struct ml_test *test)
+{
+ return test->test_priv;
+}
+
+struct ml_test *ml_test_get(const char *name);
+void ml_test_register(struct ml_test_entry *test);
+void ml_test_dump_names(void (*f)(const char *));
+
+#define ML_TEST_REGISTER(nm) \
+ static struct ml_test_entry _ml_test_entry_##nm; \
+ RTE_INIT(ml_test_##nm) \
+ { \
+ _ml_test_entry_##nm.test.name = RTE_STR(nm); \
+ memcpy(&_ml_test_entry_##nm.test.ops, &nm, sizeof(struct ml_test_ops)); \
+ ml_test_register(&_ml_test_entry_##nm); \
+ }
+
+#endif /* _ML_TEST_ */
diff --git a/app/test-mldev/parser.c b/app/test-mldev/parser.c
new file mode 100644
index 0000000000..0b7fb63fe5
--- /dev/null
+++ b/app/test-mldev/parser.c
@@ -0,0 +1,380 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2016 Intel Corporation.
+ * Copyright (c) 2017 Cavium, Inc.
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <ctype.h>
+#include <errno.h>
+#include <stdbool.h>
+#include <stdint.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include <rte_common.h>
+
+#include "parser.h"
+
+static uint32_t
+get_hex_val(char c)
+{
+ switch (c) {
+ case '0':
+ case '1':
+ case '2':
+ case '3':
+ case '4':
+ case '5':
+ case '6':
+ case '7':
+ case '8':
+ case '9':
+ return c - '0';
+ case 'A':
+ case 'B':
+ case 'C':
+ case 'D':
+ case 'E':
+ case 'F':
+ return c - 'A' + 10;
+ case 'a':
+ case 'b':
+ case 'c':
+ case 'd':
+ case 'e':
+ case 'f':
+ return c - 'a' + 10;
+ default:
+ return 0;
+ }
+}
+
+int
+parser_read_arg_bool(const char *p)
+{
+ p = skip_white_spaces(p);
+ int result = -EINVAL;
+
+ if (((p[0] == 'y') && (p[1] == 'e') && (p[2] == 's')) ||
+ ((p[0] == 'Y') && (p[1] == 'E') && (p[2] == 'S'))) {
+ p += 3;
+ result = 1;
+ }
+
+ if (((p[0] == 'o') && (p[1] == 'n')) || ((p[0] == 'O') && (p[1] == 'N'))) {
+ p += 2;
+ result = 1;
+ }
+
+ if (((p[0] == 'n') && (p[1] == 'o')) || ((p[0] == 'N') && (p[1] == 'O'))) {
+ p += 2;
+ result = 0;
+ }
+
+ if (((p[0] == 'o') && (p[1] == 'f') && (p[2] == 'f')) ||
+ ((p[0] == 'O') && (p[1] == 'F') && (p[2] == 'F'))) {
+ p += 3;
+ result = 0;
+ }
+
+ p = skip_white_spaces(p);
+
+ if (p[0] != '\0')
+ return -EINVAL;
+
+ return result;
+}
+
+int
+parser_read_uint64(uint64_t *value, const char *p)
+{
+ char *next;
+ uint64_t val;
+
+ p = skip_white_spaces(p);
+ if (!isdigit(*p))
+ return -EINVAL;
+
+ val = strtoul(p, &next, 10);
+ if (p == next)
+ return -EINVAL;
+
+ p = next;
+ switch (*p) {
+ case 'T':
+ val *= 1024ULL;
+ /* fall through */
+ case 'G':
+ val *= 1024ULL;
+ /* fall through */
+ case 'M':
+ val *= 1024ULL;
+ /* fall through */
+ case 'k':
+ case 'K':
+ val *= 1024ULL;
+ p++;
+ break;
+ }
+
+ p = skip_white_spaces(p);
+ if (*p != '\0')
+ return -EINVAL;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_int32(int32_t *value, const char *p)
+{
+ char *next;
+ int32_t val;
+
+ p = skip_white_spaces(p);
+ if (!isdigit(*p))
+ return -EINVAL;
+
+ val = strtol(p, &next, 10);
+ if (p == next)
+ return -EINVAL;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_int16(int16_t *value, const char *p)
+{
+ char *next;
+ int16_t val;
+
+ p = skip_white_spaces(p);
+ if (!isdigit(*p))
+ return -EINVAL;
+
+ val = strtol(p, &next, 10);
+ if (p == next)
+ return -EINVAL;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint64_hex(uint64_t *value, const char *p)
+{
+ char *next;
+ uint64_t val;
+
+ p = skip_white_spaces(p);
+
+ val = strtoul(p, &next, 16);
+ if (p == next)
+ return -EINVAL;
+
+ p = skip_white_spaces(next);
+ if (*p != '\0')
+ return -EINVAL;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint32(uint32_t *value, const char *p)
+{
+ uint64_t val = 0;
+ int ret = parser_read_uint64(&val, p);
+
+ if (ret < 0)
+ return ret;
+
+ if (val > UINT32_MAX)
+ return -ERANGE;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint32_hex(uint32_t *value, const char *p)
+{
+ uint64_t val = 0;
+ int ret = parser_read_uint64_hex(&val, p);
+
+ if (ret < 0)
+ return ret;
+
+ if (val > UINT32_MAX)
+ return -ERANGE;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint16(uint16_t *value, const char *p)
+{
+ uint64_t val = 0;
+ int ret = parser_read_uint64(&val, p);
+
+ if (ret < 0)
+ return ret;
+
+ if (val > UINT16_MAX)
+ return -ERANGE;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint16_hex(uint16_t *value, const char *p)
+{
+ uint64_t val = 0;
+ int ret = parser_read_uint64_hex(&val, p);
+
+ if (ret < 0)
+ return ret;
+
+ if (val > UINT16_MAX)
+ return -ERANGE;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint8(uint8_t *value, const char *p)
+{
+ uint64_t val = 0;
+ int ret = parser_read_uint64(&val, p);
+
+ if (ret < 0)
+ return ret;
+
+ if (val > UINT8_MAX)
+ return -ERANGE;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint8_hex(uint8_t *value, const char *p)
+{
+ uint64_t val = 0;
+ int ret = parser_read_uint64_hex(&val, p);
+
+ if (ret < 0)
+ return ret;
+
+ if (val > UINT8_MAX)
+ return -ERANGE;
+
+ *value = val;
+ return 0;
+}
+
+int
+parse_tokenize_string(char *string, char *tokens[], uint32_t *n_tokens)
+{
+ uint32_t i;
+
+ if ((string == NULL) || (tokens == NULL) || (*n_tokens < 1))
+ return -EINVAL;
+
+ for (i = 0; i < *n_tokens; i++) {
+ tokens[i] = strtok_r(string, PARSE_DELIMITER, &string);
+ if (tokens[i] == NULL)
+ break;
+ }
+
+ if ((i == *n_tokens) && (strtok_r(string, PARSE_DELIMITER, &string) != NULL))
+ return -E2BIG;
+
+ *n_tokens = i;
+ return 0;
+}
+
+int
+parse_hex_string(char *src, uint8_t *dst, uint32_t *size)
+{
+ char *c;
+ uint32_t len, i;
+
+ /* Check input parameters */
+ if ((src == NULL) || (dst == NULL) || (size == NULL) || (*size == 0))
+ return -1;
+
+ len = strlen(src);
+ if (((len & 3) != 0) || (len > (*size) * 2))
+ return -1;
+ *size = len / 2;
+
+ for (c = src; *c != 0; c++) {
+ if ((((*c) >= '0') && ((*c) <= '9')) || (((*c) >= 'A') && ((*c) <= 'F')) ||
+ (((*c) >= 'a') && ((*c) <= 'f')))
+ continue;
+
+ return -1;
+ }
+
+ /* Convert chars to bytes */
+ for (i = 0; i < *size; i++)
+ dst[i] = get_hex_val(src[2 * i]) * 16 + get_hex_val(src[2 * i + 1]);
+
+ return 0;
+}
+
+int
+parse_lcores_list(bool lcores[], int lcores_num, const char *corelist)
+{
+ int i, idx = 0;
+ int min, max;
+ char *end = NULL;
+
+ if (corelist == NULL)
+ return -1;
+ while (isblank(*corelist))
+ corelist++;
+ i = strlen(corelist);
+ while ((i > 0) && isblank(corelist[i - 1]))
+ i--;
+
+ /* Get list of lcores */
+ min = RTE_MAX_LCORE;
+ do {
+ while (isblank(*corelist))
+ corelist++;
+ if (*corelist == '\0')
+ return -1;
+ idx = strtoul(corelist, &end, 10);
+ if (idx < 0 || idx > lcores_num)
+ return -1;
+
+ if (end == NULL)
+ return -1;
+ while (isblank(*end))
+ end++;
+ if (*end == '-') {
+ min = idx;
+ } else if ((*end == ',') || (*end == '\0')) {
+ max = idx;
+ if (min == RTE_MAX_LCORE)
+ min = idx;
+ for (idx = min; idx <= max; idx++) {
+ if (lcores[idx] == 1)
+ return -E2BIG;
+ lcores[idx] = 1;
+ }
+
+ min = RTE_MAX_LCORE;
+ } else
+ return -1;
+ corelist = end + 1;
+ } while (*end != '\0');
+
+ return 0;
+}
diff --git a/app/test-mldev/parser.h b/app/test-mldev/parser.h
new file mode 100644
index 0000000000..f0d5e79e4b
--- /dev/null
+++ b/app/test-mldev/parser.h
@@ -0,0 +1,55 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2010-2016 Intel Corporation.
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef __INCLUDE_PARSER_H__
+#define __INCLUDE_PARSER_H__
+
+#include <ctype.h>
+#include <stdint.h>
+#include <stdlib.h>
+
+#define PARSE_DELIMITER " \f\n\r\t\v"
+
+#define skip_white_spaces(pos) \
+ ({ \
+ __typeof__(pos) _p = (pos); \
+ for (; isspace(*_p); _p++) \
+ ; \
+ _p; \
+ })
+
+static inline size_t
+skip_digits(const char *src)
+{
+ size_t i;
+
+ for (i = 0; isdigit(src[i]); i++)
+ ;
+
+ return i;
+}
+
+int parser_read_arg_bool(const char *p);
+
+int parser_read_uint64(uint64_t *value, const char *p);
+int parser_read_uint32(uint32_t *value, const char *p);
+int parser_read_uint16(uint16_t *value, const char *p);
+int parser_read_uint8(uint8_t *value, const char *p);
+
+int parser_read_uint64_hex(uint64_t *value, const char *p);
+int parser_read_uint32_hex(uint32_t *value, const char *p);
+int parser_read_uint16_hex(uint16_t *value, const char *p);
+int parser_read_uint8_hex(uint8_t *value, const char *p);
+
+int parser_read_int32(int32_t *value, const char *p);
+int parser_read_int16(int16_t *value, const char *p);
+
+int parse_hex_string(char *src, uint8_t *dst, uint32_t *size);
+
+int parse_tokenize_string(char *string, char *tokens[], uint32_t *n_tokens);
+
+int parse_lcores_list(bool lcores[], int lcores_num, const char *corelist);
+
+#endif /* __INCLUDE_PARSER_H__ */
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v3 02/12] app/mldev: add common test functions
2022-12-08 19:29 ` [PATCH v3 01/12] app/mldev: implement test framework for mldev Srikanth Yalavarthi
@ 2022-12-08 19:29 ` Srikanth Yalavarthi
2022-12-08 19:29 ` [PATCH v3 03/12] app/mldev: add test case to validate device ops Srikanth Yalavarthi
` (9 subsequent siblings)
10 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2022-12-08 19:29 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu
Added common functions used by all tests. Common code
includes functions to check capabilities, options, and
handle ML devices.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/meson.build | 1 +
app/test-mldev/test_common.c | 139 +++++++++++++++++++++++++++++++++++
app/test-mldev/test_common.h | 27 +++++++
3 files changed, 167 insertions(+)
create mode 100644 app/test-mldev/test_common.c
create mode 100644 app/test-mldev/test_common.h
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
index 8ca2e1a1c1..964bb9ddc4 100644
--- a/app/test-mldev/meson.build
+++ b/app/test-mldev/meson.build
@@ -12,6 +12,7 @@ sources = files(
'ml_options.c',
'ml_test.c',
'parser.c',
+ 'test_common.c',
)
deps += ['mldev']
diff --git a/app/test-mldev/test_common.c b/app/test-mldev/test_common.c
new file mode 100644
index 0000000000..b6b32904e4
--- /dev/null
+++ b/app/test-mldev/test_common.c
@@ -0,0 +1,139 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <errno.h>
+#include <stdbool.h>
+#include <stdint.h>
+
+#include <rte_common.h>
+#include <rte_memory.h>
+#include <rte_mldev.h>
+
+#include "ml_common.h"
+#include "ml_options.h"
+#include "test_common.h"
+
+bool
+ml_test_cap_check(struct ml_options *opt)
+{
+ struct rte_ml_dev_info dev_info;
+
+ rte_ml_dev_info_get(opt->dev_id, &dev_info);
+ if (dev_info.max_models == 0) {
+ ml_err("Not enough mldev models supported = %d", dev_info.max_models);
+ return false;
+ }
+
+ return true;
+}
+
+int
+ml_test_opt_check(struct ml_options *opt)
+{
+ uint16_t dev_count;
+ int socket_id;
+
+ RTE_SET_USED(opt);
+
+ dev_count = rte_ml_dev_count();
+ if (dev_count == 0) {
+ ml_err("No ML devices found");
+ return -ENODEV;
+ }
+
+ if (opt->dev_id >= dev_count) {
+ ml_err("Invalid option dev_id = %d", opt->dev_id);
+ return -EINVAL;
+ }
+
+ socket_id = rte_ml_dev_socket_id(opt->dev_id);
+ if (!((opt->socket_id != SOCKET_ID_ANY) || (opt->socket_id != socket_id))) {
+ ml_err("Invalid option, socket_id = %d\n", opt->socket_id);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+void
+ml_test_opt_dump(struct ml_options *opt)
+{
+ ml_options_dump(opt);
+}
+
+int
+ml_test_device_configure(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_common *t = ml_test_priv(test);
+ struct rte_ml_dev_config dev_config;
+ int ret;
+
+ ret = rte_ml_dev_info_get(opt->dev_id, &t->dev_info);
+ if (ret != 0) {
+ ml_err("Failed to get mldev info, dev_id = %d\n", opt->dev_id);
+ return ret;
+ }
+
+ /* configure device */
+ dev_config.socket_id = opt->socket_id;
+ dev_config.nb_models = t->dev_info.max_models;
+ dev_config.nb_queue_pairs = t->dev_info.max_queue_pairs;
+ ret = rte_ml_dev_configure(opt->dev_id, &dev_config);
+ if (ret != 0) {
+ ml_err("Failed to configure ml device, dev_id = %d\n", opt->dev_id);
+ return ret;
+ }
+
+ return 0;
+}
+
+int
+ml_test_device_close(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_common *t = ml_test_priv(test);
+ int ret = 0;
+
+ RTE_SET_USED(t);
+
+ /* close device */
+ ret = rte_ml_dev_close(opt->dev_id);
+ if (ret != 0)
+ ml_err("Failed to close ML device, dev_id = %d\n", opt->dev_id);
+
+ return ret;
+}
+
+int
+ml_test_device_start(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_common *t = ml_test_priv(test);
+ int ret;
+
+ RTE_SET_USED(t);
+
+ /* start device */
+ ret = rte_ml_dev_start(opt->dev_id);
+ if (ret != 0) {
+ ml_err("Failed to start ml device, dev_id = %d\n", opt->dev_id);
+ return ret;
+ }
+
+ return 0;
+}
+
+int
+ml_test_device_stop(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_common *t = ml_test_priv(test);
+ int ret = 0;
+
+ RTE_SET_USED(t);
+
+ /* stop device */
+ ret = rte_ml_dev_stop(opt->dev_id);
+ if (ret != 0)
+ ml_err("Failed to stop ML device, dev_id = %d\n", opt->dev_id);
+
+ return ret;
+}
diff --git a/app/test-mldev/test_common.h b/app/test-mldev/test_common.h
new file mode 100644
index 0000000000..05a2e43e2f
--- /dev/null
+++ b/app/test-mldev/test_common.h
@@ -0,0 +1,27 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_TEST_COMMON_
+#define _ML_TEST_COMMON_
+
+#include <rte_mldev.h>
+
+#include "ml_options.h"
+#include "ml_test.h"
+
+struct test_common {
+ struct ml_options *opt;
+ enum ml_test_result result;
+ struct rte_ml_dev_info dev_info;
+};
+
+bool ml_test_cap_check(struct ml_options *opt);
+int ml_test_opt_check(struct ml_options *opt);
+void ml_test_opt_dump(struct ml_options *opt);
+int ml_test_device_configure(struct ml_test *test, struct ml_options *opt);
+int ml_test_device_close(struct ml_test *test, struct ml_options *opt);
+int ml_test_device_start(struct ml_test *test, struct ml_options *opt);
+int ml_test_device_stop(struct ml_test *test, struct ml_options *opt);
+
+#endif /* _ML_TEST_COMMON_ */
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v3 03/12] app/mldev: add test case to validate device ops
2022-12-08 19:29 ` [PATCH v3 01/12] app/mldev: implement test framework for mldev Srikanth Yalavarthi
2022-12-08 19:29 ` [PATCH v3 02/12] app/mldev: add common test functions Srikanth Yalavarthi
@ 2022-12-08 19:29 ` Srikanth Yalavarthi
2022-12-08 19:29 ` [PATCH v3 04/12] app/mldev: add test case to validate model ops Srikanth Yalavarthi
` (8 subsequent siblings)
10 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2022-12-08 19:29 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu
Added test case to validate device handling operations. Device ops
test is a collection of multiple sub-tests. Enabled sub-test to
validate device reconfiguration. Set device_ops as the default test.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/meson.build | 1 +
app/test-mldev/ml_options.c | 5 +-
app/test-mldev/test_device_ops.c | 234 +++++++++++++++++++++++++++++++
app/test-mldev/test_device_ops.h | 17 +++
4 files changed, 255 insertions(+), 2 deletions(-)
create mode 100644 app/test-mldev/test_device_ops.c
create mode 100644 app/test-mldev/test_device_ops.h
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
index 964bb9ddc4..60ea23d142 100644
--- a/app/test-mldev/meson.build
+++ b/app/test-mldev/meson.build
@@ -13,6 +13,7 @@ sources = files(
'ml_test.c',
'parser.c',
'test_common.c',
+ 'test_device_ops.c',
)
deps += ['mldev']
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 8fd7760e36..2e5f11bca2 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -24,7 +24,7 @@ void
ml_options_default(struct ml_options *opt)
{
memset(opt, 0, sizeof(*opt));
- strlcpy(opt->test_name, "ml_test", ML_TEST_NAME_MAX_LEN);
+ strlcpy(opt->test_name, "device_ops", ML_TEST_NAME_MAX_LEN);
opt->dev_id = 0;
opt->socket_id = SOCKET_ID_ANY;
opt->debug = false;
@@ -66,7 +66,8 @@ ml_parse_socket_id(struct ml_options *opt, const char *arg)
static void
ml_dump_test_options(const char *testname)
{
- RTE_SET_USED(testname);
+ if (strcmp(testname, "device_ops") == 0)
+ printf("\n");
}
static void
diff --git a/app/test-mldev/test_device_ops.c b/app/test-mldev/test_device_ops.c
new file mode 100644
index 0000000000..4cafcf41a6
--- /dev/null
+++ b/app/test-mldev/test_device_ops.c
@@ -0,0 +1,234 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <errno.h>
+#include <stdbool.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+
+#include <rte_common.h>
+#include <rte_malloc.h>
+#include <rte_mldev.h>
+
+#include "ml_common.h"
+#include "ml_options.h"
+#include "test_device_ops.h"
+
+static bool
+test_device_cap_check(struct ml_options *opt)
+{
+ if (!ml_test_cap_check(opt))
+ return false;
+
+ return true;
+}
+
+static int
+test_device_opt_check(struct ml_options *opt)
+{
+ int ret;
+
+ /* check common opts */
+ ret = ml_test_opt_check(opt);
+ if (ret != 0)
+ return ret;
+
+ return 0;
+}
+
+static void
+test_device_opt_dump(struct ml_options *opt)
+{
+ /* dump common opts */
+ ml_test_opt_dump(opt);
+}
+
+static int
+test_device_setup(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_device *t;
+ void *test_device;
+ int ret = 0;
+
+ /* allocate for test structure */
+ test_device = rte_zmalloc_socket(test->name, sizeof(struct test_device),
+ RTE_CACHE_LINE_SIZE, opt->socket_id);
+ if (test_device == NULL) {
+ ml_err("failed to allocate memory for test_model");
+ ret = -ENOMEM;
+ goto error;
+ }
+ test->test_priv = test_device;
+ t = ml_test_priv(test);
+
+ t->cmn.result = ML_TEST_FAILED;
+ t->cmn.opt = opt;
+
+ /* get device info */
+ ret = rte_ml_dev_info_get(opt->dev_id, &t->cmn.dev_info);
+ if (ret < 0) {
+ ml_err("failed to get device info");
+ goto error;
+ }
+
+ return 0;
+
+error:
+ if (test_device != NULL)
+ rte_free(test_device);
+
+ return ret;
+}
+
+static void
+test_device_destroy(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_device *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+ if (t != NULL)
+ rte_free(t);
+}
+
+static int
+test_device_reconfigure(struct ml_test *test, struct ml_options *opt)
+{
+ struct rte_ml_dev_config dev_config;
+ struct rte_ml_dev_qp_conf qp_conf;
+ struct test_device *t;
+ uint16_t qp_id = 0;
+ int ret = 0;
+
+ t = ml_test_priv(test);
+
+ /* configure with default options */
+ ret = ml_test_device_configure(test, opt);
+ if (ret != 0)
+ return ret;
+
+ /* setup one queue pair with nb_desc = 1 */
+ qp_conf.nb_desc = 1;
+ qp_conf.cb = NULL;
+
+ ret = rte_ml_dev_queue_pair_setup(opt->dev_id, qp_id, &qp_conf, opt->socket_id);
+ if (ret != 0) {
+ ml_err("Failed to setup ML device queue-pair, dev_id = %d, qp_id = %u\n",
+ opt->dev_id, qp_id);
+ goto error;
+ }
+
+ /* start device */
+ ret = ml_test_device_start(test, opt);
+ if (ret != 0)
+ goto error;
+
+ /* stop device */
+ ret = ml_test_device_stop(test, opt);
+ if (ret != 0) {
+ ml_err("Failed to stop device");
+ goto error;
+ }
+
+ /* reconfigure device based on dev_info */
+ dev_config.socket_id = opt->socket_id;
+ dev_config.nb_models = t->cmn.dev_info.max_models;
+ dev_config.nb_queue_pairs = t->cmn.dev_info.max_queue_pairs;
+ ret = rte_ml_dev_configure(opt->dev_id, &dev_config);
+ if (ret != 0) {
+ ml_err("Failed to reconfigure ML device, dev_id = %d\n", opt->dev_id);
+ return ret;
+ }
+
+ /* setup queue pairs */
+ for (qp_id = 0; qp_id < t->cmn.dev_info.max_queue_pairs; qp_id++) {
+ qp_conf.nb_desc = t->cmn.dev_info.max_desc;
+ qp_conf.cb = NULL;
+
+ ret = rte_ml_dev_queue_pair_setup(opt->dev_id, qp_id, &qp_conf, opt->socket_id);
+ if (ret != 0) {
+ ml_err("Failed to setup ML device queue-pair, dev_id = %d, qp_id = %u\n",
+ opt->dev_id, qp_id);
+ goto error;
+ }
+ }
+
+ /* start device */
+ ret = ml_test_device_start(test, opt);
+ if (ret != 0)
+ goto error;
+
+ /* stop device */
+ ret = ml_test_device_stop(test, opt);
+ if (ret != 0)
+ goto error;
+
+ /* close device */
+ ret = ml_test_device_close(test, opt);
+ if (ret != 0)
+ return ret;
+
+ return 0;
+
+error:
+ ml_test_device_close(test, opt);
+
+ return ret;
+}
+
+static int
+test_device_driver(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_device *t;
+ int ret = 0;
+
+ t = ml_test_priv(test);
+
+ /* sub-test: device reconfigure */
+ ret = test_device_reconfigure(test, opt);
+ if (ret != 0) {
+ printf("\n");
+ printf("Model Device Reconfigure Test: " CLRED "%s" CLNRM "\n", "Failed");
+ goto error;
+ } else {
+ printf("\n");
+ printf("Model Device Reconfigure Test: " CLYEL "%s" CLNRM "\n", "Passed");
+ }
+
+ printf("\n");
+
+ t->cmn.result = ML_TEST_SUCCESS;
+
+ return 0;
+
+error:
+ t->cmn.result = ML_TEST_FAILED;
+ return -1;
+}
+
+static int
+test_device_result(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_device *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+
+ return t->cmn.result;
+}
+
+static const struct ml_test_ops device_ops = {
+ .cap_check = test_device_cap_check,
+ .opt_check = test_device_opt_check,
+ .opt_dump = test_device_opt_dump,
+ .test_setup = test_device_setup,
+ .test_destroy = test_device_destroy,
+ .test_driver = test_device_driver,
+ .test_result = test_device_result,
+};
+
+ML_TEST_REGISTER(device_ops);
diff --git a/app/test-mldev/test_device_ops.h b/app/test-mldev/test_device_ops.h
new file mode 100644
index 0000000000..115b1072a2
--- /dev/null
+++ b/app/test-mldev/test_device_ops.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_TEST_DEVICE_OPS_
+#define _ML_TEST_DEVICE_OPS_
+
+#include <rte_common.h>
+
+#include "test_common.h"
+
+struct test_device {
+ /* common data */
+ struct test_common cmn;
+} __rte_cache_aligned;
+
+#endif /* _ML_TEST_DEVICE_OPS_ */
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v3 04/12] app/mldev: add test case to validate model ops
2022-12-08 19:29 ` [PATCH v3 01/12] app/mldev: implement test framework for mldev Srikanth Yalavarthi
2022-12-08 19:29 ` [PATCH v3 02/12] app/mldev: add common test functions Srikanth Yalavarthi
2022-12-08 19:29 ` [PATCH v3 03/12] app/mldev: add test case to validate device ops Srikanth Yalavarthi
@ 2022-12-08 19:29 ` Srikanth Yalavarthi
2022-12-08 19:29 ` [PATCH v3 05/12] app/mldev: add ordered inference test case Srikanth Yalavarthi
` (7 subsequent siblings)
10 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2022-12-08 19:29 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu
Added test case to validate model operations. Model ops test
is a collection of sub-tests. Each sub-test invokes the model
operations in a specific order.
Sub-test A: (load -> start -> stop -> unload) x n
Sub-test B: load x n -> start x n -> stop x n -> unload x n
Sub-test C: load x n + (start + stop) x n + unload x n
Sub-test D: (load + start) x n -> (stop + unload) x n
Added internal functions to handle model load, start, stop and
unload. List of models to be used for testing can be specified
through application argument "--models"
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/meson.build | 2 +
app/test-mldev/ml_options.c | 45 ++-
app/test-mldev/ml_options.h | 9 +
app/test-mldev/test_model_common.c | 162 +++++++++++
app/test-mldev/test_model_common.h | 37 +++
app/test-mldev/test_model_ops.c | 433 +++++++++++++++++++++++++++++
app/test-mldev/test_model_ops.h | 21 ++
7 files changed, 706 insertions(+), 3 deletions(-)
create mode 100644 app/test-mldev/test_model_common.c
create mode 100644 app/test-mldev/test_model_common.h
create mode 100644 app/test-mldev/test_model_ops.c
create mode 100644 app/test-mldev/test_model_ops.h
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
index 60ea23d142..b09e1ccc8a 100644
--- a/app/test-mldev/meson.build
+++ b/app/test-mldev/meson.build
@@ -14,6 +14,8 @@ sources = files(
'parser.c',
'test_common.c',
'test_device_ops.c',
+ 'test_model_common.c',
+ 'test_model_ops.c',
)
deps += ['mldev']
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 2e5f11bca2..15043c0992 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -4,6 +4,7 @@
#include <errno.h>
#include <getopt.h>
+#include <linux/limits.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
@@ -27,6 +28,7 @@ ml_options_default(struct ml_options *opt)
strlcpy(opt->test_name, "device_ops", ML_TEST_NAME_MAX_LEN);
opt->dev_id = 0;
opt->socket_id = SOCKET_ID_ANY;
+ opt->nb_filelist = 0;
opt->debug = false;
}
@@ -63,11 +65,47 @@ ml_parse_socket_id(struct ml_options *opt, const char *arg)
return 0;
}
+static int
+ml_parse_models(struct ml_options *opt, const char *arg)
+{
+ const char *delim = ",";
+ char models[PATH_MAX];
+ char *token;
+ int ret = 0;
+
+ strlcpy(models, arg, PATH_MAX);
+
+ token = strtok(models, delim);
+ while (token != NULL) {
+ strlcpy(opt->filelist[opt->nb_filelist].model, token, PATH_MAX);
+ opt->nb_filelist++;
+
+ if (opt->nb_filelist >= ML_TEST_MAX_MODELS) {
+ ml_err("Exceeded model count, max = %d\n", ML_TEST_MAX_MODELS);
+ ret = -EINVAL;
+ break;
+ }
+ token = strtok(NULL, delim);
+ }
+
+ if (opt->nb_filelist == 0) {
+ ml_err("Models list is empty. Need at least one model for the test");
+ ret = -EINVAL;
+ }
+
+ return ret;
+}
+
static void
ml_dump_test_options(const char *testname)
{
if (strcmp(testname, "device_ops") == 0)
printf("\n");
+
+ if (strcmp(testname, "model_ops") == 0) {
+ printf("\t\t--models : comma separated list of models\n");
+ printf("\n");
+ }
}
static void
@@ -85,9 +123,9 @@ print_usage(char *program)
ml_test_dump_names(ml_dump_test_options);
}
-static struct option lgopts[] = {{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0},
- {ML_SOCKET_ID, 1, 0, 0}, {ML_DEBUG, 0, 0, 0},
- {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
+static struct option lgopts[] = {
+ {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0}, {ML_MODELS, 1, 0, 0},
+ {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
static int
ml_opts_parse_long(int opt_idx, struct ml_options *opt)
@@ -98,6 +136,7 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
{ML_TEST, ml_parse_test_name},
{ML_DEVICE_ID, ml_parse_dev_id},
{ML_SOCKET_ID, ml_parse_socket_id},
+ {ML_MODELS, ml_parse_models},
};
for (i = 0; i < RTE_DIM(parsermap); i++) {
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index 05311a9a47..8faf3b5deb 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -5,22 +5,31 @@
#ifndef _ML_OPTIONS_
#define _ML_OPTIONS_
+#include <linux/limits.h>
#include <stdbool.h>
#include <stdint.h>
#define ML_TEST_NAME_MAX_LEN 32
+#define ML_TEST_MAX_MODELS 8
/* Options names */
#define ML_TEST ("test")
#define ML_DEVICE_ID ("dev_id")
#define ML_SOCKET_ID ("socket_id")
+#define ML_MODELS ("models")
#define ML_DEBUG ("debug")
#define ML_HELP ("help")
+struct ml_filelist {
+ char model[PATH_MAX];
+};
+
struct ml_options {
char test_name[ML_TEST_NAME_MAX_LEN];
int16_t dev_id;
int socket_id;
+ struct ml_filelist filelist[ML_TEST_MAX_MODELS];
+ uint8_t nb_filelist;
bool debug;
};
diff --git a/app/test-mldev/test_model_common.c b/app/test-mldev/test_model_common.c
new file mode 100644
index 0000000000..5368be17fe
--- /dev/null
+++ b/app/test-mldev/test_model_common.c
@@ -0,0 +1,162 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <errno.h>
+#include <stdio.h>
+
+#include <rte_common.h>
+#include <rte_malloc.h>
+#include <rte_mldev.h>
+
+#include "ml_common.h"
+#include "ml_options.h"
+#include "ml_test.h"
+#include "test_common.h"
+#include "test_model_common.h"
+
+int
+ml_model_load(struct ml_test *test, struct ml_options *opt, struct ml_model *model, int16_t fid)
+{
+ struct test_common *t = ml_test_priv(test);
+ struct rte_ml_model_params model_params;
+ FILE *fp;
+ int ret;
+
+ if (model->state == MODEL_LOADED)
+ return 0;
+
+ if (model->state != MODEL_INITIAL)
+ return -EINVAL;
+
+ /* read model binary */
+ fp = fopen(opt->filelist[fid].model, "r");
+ if (fp == NULL) {
+ ml_err("Failed to open model file : %s\n", opt->filelist[fid].model);
+ return -1;
+ }
+
+ fseek(fp, 0, SEEK_END);
+ model_params.size = ftell(fp);
+ fseek(fp, 0, SEEK_SET);
+
+ model_params.addr = rte_malloc_socket("ml_model", model_params.size,
+ t->dev_info.min_align_size, opt->socket_id);
+ if (model_params.addr == NULL) {
+ ml_err("Failed to allocate memory for model: %s\n", opt->filelist[fid].model);
+ fclose(fp);
+ return -ENOMEM;
+ }
+
+ if (fread(model_params.addr, 1, model_params.size, fp) != model_params.size) {
+ ml_err("Failed to read model file : %s\n", opt->filelist[fid].model);
+ rte_free(model_params.addr);
+ fclose(fp);
+ return -1;
+ }
+ fclose(fp);
+
+ /* load model to device */
+ ret = rte_ml_model_load(opt->dev_id, &model_params, &model->id);
+ if (ret != 0) {
+ ml_err("Failed to load model : %s\n", opt->filelist[fid].model);
+ model->state = MODEL_ERROR;
+ rte_free(model_params.addr);
+ return ret;
+ }
+
+ /* release mz */
+ rte_free(model_params.addr);
+
+ /* get model info */
+ ret = rte_ml_model_info_get(opt->dev_id, model->id, &model->info);
+ if (ret != 0) {
+ ml_err("Failed to get model info : %s\n", opt->filelist[fid].model);
+ return ret;
+ }
+
+ model->state = MODEL_LOADED;
+
+ return 0;
+}
+
+int
+ml_model_unload(struct ml_test *test, struct ml_options *opt, struct ml_model *model, int16_t fid)
+{
+ struct test_common *t = ml_test_priv(test);
+ int ret;
+
+ RTE_SET_USED(t);
+
+ if (model->state == MODEL_INITIAL)
+ return 0;
+
+ if (model->state != MODEL_LOADED)
+ return -EINVAL;
+
+ /* unload model */
+ ret = rte_ml_model_unload(opt->dev_id, model->id);
+ if (ret != 0) {
+ ml_err("Failed to unload model: %s\n", opt->filelist[fid].model);
+ model->state = MODEL_ERROR;
+ return ret;
+ }
+
+ model->state = MODEL_INITIAL;
+
+ return 0;
+}
+
+int
+ml_model_start(struct ml_test *test, struct ml_options *opt, struct ml_model *model, int16_t fid)
+{
+ struct test_common *t = ml_test_priv(test);
+ int ret;
+
+ RTE_SET_USED(t);
+
+ if (model->state == MODEL_STARTED)
+ return 0;
+
+ if (model->state != MODEL_LOADED)
+ return -EINVAL;
+
+ /* start model */
+ ret = rte_ml_model_start(opt->dev_id, model->id);
+ if (ret != 0) {
+ ml_err("Failed to start model : %s\n", opt->filelist[fid].model);
+ model->state = MODEL_ERROR;
+ return ret;
+ }
+
+ model->state = MODEL_STARTED;
+
+ return 0;
+}
+
+int
+ml_model_stop(struct ml_test *test, struct ml_options *opt, struct ml_model *model, int16_t fid)
+{
+ struct test_common *t = ml_test_priv(test);
+ int ret;
+
+ RTE_SET_USED(t);
+
+ if (model->state == MODEL_LOADED)
+ return 0;
+
+ if (model->state != MODEL_STARTED)
+ return -EINVAL;
+
+ /* stop model */
+ ret = rte_ml_model_stop(opt->dev_id, model->id);
+ if (ret != 0) {
+ ml_err("Failed to stop model: %s\n", opt->filelist[fid].model);
+ model->state = MODEL_ERROR;
+ return ret;
+ }
+
+ model->state = MODEL_LOADED;
+
+ return 0;
+}
diff --git a/app/test-mldev/test_model_common.h b/app/test-mldev/test_model_common.h
new file mode 100644
index 0000000000..302e4eb45f
--- /dev/null
+++ b/app/test-mldev/test_model_common.h
@@ -0,0 +1,37 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_TEST_MODEL_COMMON_
+#define _ML_TEST_MODEL_COMMON_
+
+#include <stdint.h>
+
+#include <rte_mldev.h>
+
+#include "ml_options.h"
+#include "ml_test.h"
+
+enum model_state {
+ MODEL_INITIAL,
+ MODEL_LOADED,
+ MODEL_STARTED,
+ MODEL_ERROR,
+};
+
+struct ml_model {
+ int16_t id;
+ struct rte_ml_model_info info;
+ enum model_state state;
+};
+
+int ml_model_load(struct ml_test *test, struct ml_options *opt, struct ml_model *model,
+ int16_t fid);
+int ml_model_unload(struct ml_test *test, struct ml_options *opt, struct ml_model *model,
+ int16_t fid);
+int ml_model_start(struct ml_test *test, struct ml_options *opt, struct ml_model *model,
+ int16_t fid);
+int ml_model_stop(struct ml_test *test, struct ml_options *opt, struct ml_model *model,
+ int16_t fid);
+
+#endif /* _ML_TEST_MODEL_COMMON_ */
diff --git a/app/test-mldev/test_model_ops.c b/app/test-mldev/test_model_ops.c
new file mode 100644
index 0000000000..69c9df8ed6
--- /dev/null
+++ b/app/test-mldev/test_model_ops.c
@@ -0,0 +1,433 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <stdint.h>
+#include <stdio.h>
+#include <unistd.h>
+
+#include <rte_common.h>
+#include <rte_malloc.h>
+#include <rte_mldev.h>
+
+#include "ml_common.h"
+#include "ml_options.h"
+#include "ml_test.h"
+#include "test_model_ops.h"
+
+static bool
+test_model_ops_cap_check(struct ml_options *opt)
+{
+ if (!ml_test_cap_check(opt))
+ return false;
+
+ return true;
+}
+
+static int
+test_model_ops_opt_check(struct ml_options *opt)
+{
+ uint32_t i;
+ int ret;
+
+ /* check common opts */
+ ret = ml_test_opt_check(opt);
+ if (ret != 0)
+ return ret;
+
+ /* check model file availability */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ if (access(opt->filelist[i].model, F_OK) == -1) {
+ ml_err("Model file not available: id = %u, file = %s", i,
+ opt->filelist[i].model);
+ return -ENOENT;
+ }
+ }
+
+ return 0;
+}
+
+static void
+test_model_ops_opt_dump(struct ml_options *opt)
+{
+ uint32_t i;
+
+ /* dump common opts */
+ ml_test_opt_dump(opt);
+
+ /* dump test specific opts */
+ ml_dump_begin("models");
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_dump_list("model", i, opt->filelist[i].model);
+ ml_dump_end;
+}
+
+static int
+test_model_ops_setup(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+ void *test_model_ops;
+ int ret = 0;
+ uint32_t i;
+
+ /* allocate model ops test structure */
+ test_model_ops = rte_zmalloc_socket(test->name, sizeof(struct test_model_ops),
+ RTE_CACHE_LINE_SIZE, opt->socket_id);
+ if (test_model_ops == NULL) {
+ ml_err("Failed to allocate memory for test_model");
+ ret = -ENOMEM;
+ goto error;
+ }
+ test->test_priv = test_model_ops;
+ t = ml_test_priv(test);
+
+ t->cmn.result = ML_TEST_FAILED;
+ t->cmn.opt = opt;
+
+ /* get device info */
+ ret = rte_ml_dev_info_get(opt->dev_id, &t->cmn.dev_info);
+ if (ret < 0) {
+ ml_err("Failed to get device info");
+ goto error;
+ }
+
+ /* set model initial state */
+ for (i = 0; i < opt->nb_filelist; i++)
+ t->model[i].state = MODEL_INITIAL;
+
+ return 0;
+
+error:
+ if (test_model_ops != NULL)
+ rte_free(test_model_ops);
+
+ return ret;
+}
+
+static void
+test_model_ops_destroy(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+ if (t != NULL)
+ rte_free(t);
+}
+
+static int
+test_model_ops_mldev_setup(struct ml_test *test, struct ml_options *opt)
+{
+ int ret;
+
+ ret = ml_test_device_configure(test, opt);
+ if (ret != 0)
+ return ret;
+
+ ret = ml_test_device_start(test, opt);
+ if (ret != 0)
+ goto error;
+
+ return 0;
+
+error:
+ ml_test_device_close(test, opt);
+
+ return ret;
+}
+
+static int
+test_model_ops_mldev_destroy(struct ml_test *test, struct ml_options *opt)
+{
+ int ret;
+
+ ret = ml_test_device_stop(test, opt);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_test_device_close(test, opt);
+ if (ret != 0)
+ return ret;
+
+ return 0;
+
+error:
+ ml_test_device_close(test, opt);
+
+ return ret;
+}
+
+/* Sub-test A: (load -> start -> stop -> unload) x n */
+static int
+test_model_ops_subtest_a(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+ int ret = 0;
+ uint32_t i;
+
+ t = ml_test_priv(test);
+
+ /* load + start + stop + unload */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_load(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_start(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_stop(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_unload(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+error:
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_stop(test, opt, &t->model[i], i);
+
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_unload(test, opt, &t->model[i], i);
+
+ return ret;
+}
+
+/* Sub-test B: load x n -> start x n -> stop x n -> unload x n */
+static int
+test_model_ops_subtest_b(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+ int ret = 0;
+ uint32_t i;
+
+ t = ml_test_priv(test);
+
+ /* load */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_load(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* start */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_start(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* stop */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_stop(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* unload */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_unload(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ return 0;
+
+error:
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_stop(test, opt, &t->model[i], i);
+
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_unload(test, opt, &t->model[i], i);
+
+ return ret;
+}
+
+/* Sub-test C: load x n + (start + stop) x n + unload x n */
+static int
+test_model_ops_subtest_c(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+ int ret = 0;
+ uint32_t i;
+
+ t = ml_test_priv(test);
+
+ /* load */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_load(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* start + stop */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_start(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_stop(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* unload */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_unload(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ return 0;
+
+error:
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_stop(test, opt, &t->model[i], i);
+
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_unload(test, opt, &t->model[i], i);
+
+ return ret;
+}
+
+/* Sub-test D: (load + start) x n -> (stop + unload) x n */
+static int
+test_model_ops_subtest_d(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+ int ret = 0;
+ uint32_t i;
+
+ t = ml_test_priv(test);
+
+ /* load + start */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_load(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_start(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* stop + unload */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_stop(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_unload(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ return 0;
+
+error:
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_stop(test, opt, &t->model[i], i);
+
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_unload(test, opt, &t->model[i], i);
+
+ return ret;
+}
+
+static int
+test_model_ops_driver(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+ int ret = 0;
+
+ t = ml_test_priv(test);
+
+ /* device setup */
+ ret = test_model_ops_mldev_setup(test, opt);
+ if (ret != 0)
+ return ret;
+
+ printf("\n");
+
+ /* sub-test A */
+ ret = test_model_ops_subtest_a(test, opt);
+ if (ret != 0) {
+ printf("Model Ops Sub-test A: " CLRED "%s" CLNRM "\n", "Failed");
+ goto error;
+ } else {
+ printf("Model Ops Sub-test A: " CLYEL "%s" CLNRM "\n", "Passed");
+ }
+
+ /* sub-test B */
+ ret = test_model_ops_subtest_b(test, opt);
+ if (ret != 0) {
+ printf("Model Ops Sub-test B: " CLRED "%s" CLNRM "\n", "Failed");
+ goto error;
+ } else {
+ printf("Model Ops Sub-test B: " CLYEL "%s" CLNRM "\n", "Passed");
+ }
+
+ /* sub-test C */
+ ret = test_model_ops_subtest_c(test, opt);
+ if (ret != 0) {
+ printf("Model Ops Sub-test C: " CLRED "%s" CLNRM "\n", "Failed");
+ goto error;
+ } else {
+ printf("Model Ops Sub-test C: " CLYEL "%s" CLNRM "\n", "Passed");
+ }
+
+ /* sub-test D */
+ ret = test_model_ops_subtest_d(test, opt);
+ if (ret != 0) {
+ printf("Model Ops Sub-test D: " CLRED "%s" CLNRM "\n", "Failed");
+ goto error;
+ } else {
+ printf("Model Ops Sub-test D: " CLYEL "%s" CLNRM "\n", "Passed");
+ }
+
+ printf("\n");
+
+ /* device destroy */
+ ret = test_model_ops_mldev_destroy(test, opt);
+ if (ret != 0)
+ return ret;
+
+ t->cmn.result = ML_TEST_SUCCESS;
+
+ return 0;
+
+error:
+ test_model_ops_mldev_destroy(test, opt);
+
+ t->cmn.result = ML_TEST_FAILED;
+
+ return ret;
+}
+
+static int
+test_model_ops_result(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+
+ return t->cmn.result;
+}
+
+static const struct ml_test_ops model_ops = {
+ .cap_check = test_model_ops_cap_check,
+ .opt_check = test_model_ops_opt_check,
+ .opt_dump = test_model_ops_opt_dump,
+ .test_setup = test_model_ops_setup,
+ .test_destroy = test_model_ops_destroy,
+ .test_driver = test_model_ops_driver,
+ .test_result = test_model_ops_result,
+};
+
+ML_TEST_REGISTER(model_ops);
diff --git a/app/test-mldev/test_model_ops.h b/app/test-mldev/test_model_ops.h
new file mode 100644
index 0000000000..9dd8402390
--- /dev/null
+++ b/app/test-mldev/test_model_ops.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_TEST_MODEL_OPS_
+#define _ML_TEST_MODEL_OPS_
+
+#include <rte_common.h>
+
+#include "test_common.h"
+#include "test_model_common.h"
+
+struct test_model_ops {
+ /* common data */
+ struct test_common cmn;
+
+ /* test specific data */
+ struct ml_model model[ML_TEST_MAX_MODELS];
+} __rte_cache_aligned;
+
+#endif /* _ML_TEST_MODEL_OPS_ */
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v3 05/12] app/mldev: add ordered inference test case
2022-12-08 19:29 ` [PATCH v3 01/12] app/mldev: implement test framework for mldev Srikanth Yalavarthi
` (2 preceding siblings ...)
2022-12-08 19:29 ` [PATCH v3 04/12] app/mldev: add test case to validate model ops Srikanth Yalavarthi
@ 2022-12-08 19:29 ` Srikanth Yalavarthi
2022-12-08 19:29 ` [PATCH v3 06/12] app/mldev: add test case to interleave inferences Srikanth Yalavarthi
` (6 subsequent siblings)
10 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2022-12-08 19:29 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu
Added an ordered test case to execute inferences with single
or multiple models. In this test case inference requests for
a model are enqueued after completion of all requests for
the previous model. Test supports inference repetitions.
Operations sequence when testing with N models and R reps,
(load -> start -> (enqueue + dequeue) x R -> stop -> unload) x N
Test case can be executed by selecting "inference_ordered" test
and repetitions can be specified through "--repetitions" argument.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/meson.build | 2 +
app/test-mldev/ml_options.c | 73 ++-
app/test-mldev/ml_options.h | 17 +-
app/test-mldev/test_inference_common.c | 565 ++++++++++++++++++++++++
app/test-mldev/test_inference_common.h | 65 +++
app/test-mldev/test_inference_ordered.c | 119 +++++
app/test-mldev/test_model_common.h | 10 +
7 files changed, 839 insertions(+), 12 deletions(-)
create mode 100644 app/test-mldev/test_inference_common.c
create mode 100644 app/test-mldev/test_inference_common.h
create mode 100644 app/test-mldev/test_inference_ordered.c
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
index b09e1ccc8a..475d76d126 100644
--- a/app/test-mldev/meson.build
+++ b/app/test-mldev/meson.build
@@ -16,6 +16,8 @@ sources = files(
'test_device_ops.c',
'test_model_common.c',
'test_model_ops.c',
+ 'test_inference_common.c',
+ 'test_inference_ordered.c',
)
deps += ['mldev']
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 15043c0992..10dad18fff 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -29,6 +29,7 @@ ml_options_default(struct ml_options *opt)
opt->dev_id = 0;
opt->socket_id = SOCKET_ID_ANY;
opt->nb_filelist = 0;
+ opt->repetitions = 1;
opt->debug = false;
}
@@ -96,6 +97,60 @@ ml_parse_models(struct ml_options *opt, const char *arg)
return ret;
}
+static int
+ml_parse_filelist(struct ml_options *opt, const char *arg)
+{
+ const char *delim = ",";
+ char filelist[PATH_MAX];
+ char *token;
+
+ if (opt->nb_filelist >= ML_TEST_MAX_MODELS) {
+ ml_err("Exceeded filelist count, max = %d\n", ML_TEST_MAX_MODELS);
+ return -1;
+ }
+
+ strlcpy(filelist, arg, PATH_MAX);
+
+ /* model */
+ token = strtok(filelist, delim);
+ if (token == NULL) {
+ ml_err("Invalid filelist, model not specified = %s\n", arg);
+ return -EINVAL;
+ }
+ strlcpy(opt->filelist[opt->nb_filelist].model, token, PATH_MAX);
+
+ /* input */
+ token = strtok(NULL, delim);
+ if (token == NULL) {
+ ml_err("Invalid filelist, input not specified = %s\n", arg);
+ return -EINVAL;
+ }
+ strlcpy(opt->filelist[opt->nb_filelist].input, token, PATH_MAX);
+
+ /* output */
+ token = strtok(NULL, delim);
+ if (token == NULL) {
+ ml_err("Invalid filelist, output not specified = %s\n", arg);
+ return -EINVAL;
+ }
+ strlcpy(opt->filelist[opt->nb_filelist].output, token, PATH_MAX);
+
+ opt->nb_filelist++;
+
+ if (opt->nb_filelist == 0) {
+ ml_err("Empty filelist. Need at least one filelist entry for the test.");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int
+ml_parse_repetitions(struct ml_options *opt, const char *arg)
+{
+ return parser_read_uint64(&opt->repetitions, arg);
+}
+
static void
ml_dump_test_options(const char *testname)
{
@@ -106,6 +161,12 @@ ml_dump_test_options(const char *testname)
printf("\t\t--models : comma separated list of models\n");
printf("\n");
}
+
+ if (strcmp(testname, "inference_ordered") == 0) {
+ printf("\t\t--filelist : comma separated list of model, input and output\n"
+ "\t\t--repetitions : number of inference repetitions\n");
+ printf("\n");
+ }
}
static void
@@ -124,8 +185,9 @@ print_usage(char *program)
}
static struct option lgopts[] = {
- {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0}, {ML_MODELS, 1, 0, 0},
- {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
+ {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
+ {ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
+ {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
static int
ml_opts_parse_long(int opt_idx, struct ml_options *opt)
@@ -133,10 +195,9 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
unsigned int i;
struct long_opt_parser parsermap[] = {
- {ML_TEST, ml_parse_test_name},
- {ML_DEVICE_ID, ml_parse_dev_id},
- {ML_SOCKET_ID, ml_parse_socket_id},
- {ML_MODELS, ml_parse_models},
+ {ML_TEST, ml_parse_test_name}, {ML_DEVICE_ID, ml_parse_dev_id},
+ {ML_SOCKET_ID, ml_parse_socket_id}, {ML_MODELS, ml_parse_models},
+ {ML_FILELIST, ml_parse_filelist}, {ML_REPETITIONS, ml_parse_repetitions},
};
for (i = 0; i < RTE_DIM(parsermap); i++) {
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index 8faf3b5deb..ad8aee5964 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -13,15 +13,19 @@
#define ML_TEST_MAX_MODELS 8
/* Options names */
-#define ML_TEST ("test")
-#define ML_DEVICE_ID ("dev_id")
-#define ML_SOCKET_ID ("socket_id")
-#define ML_MODELS ("models")
-#define ML_DEBUG ("debug")
-#define ML_HELP ("help")
+#define ML_TEST ("test")
+#define ML_DEVICE_ID ("dev_id")
+#define ML_SOCKET_ID ("socket_id")
+#define ML_MODELS ("models")
+#define ML_FILELIST ("filelist")
+#define ML_REPETITIONS ("repetitions")
+#define ML_DEBUG ("debug")
+#define ML_HELP ("help")
struct ml_filelist {
char model[PATH_MAX];
+ char input[PATH_MAX];
+ char output[PATH_MAX];
};
struct ml_options {
@@ -30,6 +34,7 @@ struct ml_options {
int socket_id;
struct ml_filelist filelist[ML_TEST_MAX_MODELS];
uint8_t nb_filelist;
+ uint64_t repetitions;
bool debug;
};
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
new file mode 100644
index 0000000000..8b5dc89346
--- /dev/null
+++ b/app/test-mldev/test_inference_common.c
@@ -0,0 +1,565 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <errno.h>
+#include <stdbool.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <unistd.h>
+
+#include <rte_common.h>
+#include <rte_launch.h>
+#include <rte_lcore.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_mldev.h>
+
+#include "ml_common.h"
+#include "ml_options.h"
+#include "ml_test.h"
+#include "test_common.h"
+#include "test_inference_common.h"
+
+/* Enqueue inference requests with burst size equal to 1 */
+static int
+ml_enqueue_single(void *arg)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)arg);
+ struct ml_request *req = NULL;
+ struct rte_ml_op *op = NULL;
+ struct ml_core_args *args;
+ uint64_t model_enq = 0;
+ uint32_t burst_enq;
+ uint32_t lcore_id;
+ int16_t fid;
+ int ret;
+
+ lcore_id = rte_lcore_id();
+ args = &t->args[lcore_id];
+ model_enq = 0;
+
+ if (args->nb_reqs == 0)
+ return 0;
+
+next_rep:
+ fid = args->start_fid;
+
+next_model:
+ ret = rte_mempool_get(t->op_pool, (void **)&op);
+ if (ret != 0)
+ goto next_model;
+
+retry:
+ ret = rte_mempool_get(t->model[fid].io_pool, (void **)&req);
+ if (ret != 0)
+ goto retry;
+
+ op->model_id = t->model[fid].id;
+ op->nb_batches = t->model[fid].info.batch_size;
+ op->mempool = t->op_pool;
+
+ op->input.addr = req->input;
+ op->input.length = t->model[fid].inp_qsize;
+ op->input.next = NULL;
+
+ op->output.addr = req->output;
+ op->output.length = t->model[fid].out_qsize;
+ op->output.next = NULL;
+
+ op->user_ptr = req;
+ req->niters++;
+ req->fid = fid;
+
+enqueue_req:
+ burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, 0, &op, 1);
+ if (burst_enq == 0)
+ goto enqueue_req;
+
+ fid++;
+ if (likely(fid <= args->end_fid))
+ goto next_model;
+
+ model_enq++;
+ if (likely(model_enq < args->nb_reqs))
+ goto next_rep;
+
+ return 0;
+}
+
+/* Dequeue inference requests with burst size equal to 1 */
+static int
+ml_dequeue_single(void *arg)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)arg);
+ struct rte_ml_op_error error;
+ struct rte_ml_op *op = NULL;
+ struct ml_core_args *args;
+ struct ml_request *req;
+ uint64_t total_deq = 0;
+ uint8_t nb_filelist;
+ uint32_t burst_deq;
+ uint32_t lcore_id;
+
+ lcore_id = rte_lcore_id();
+ args = &t->args[lcore_id];
+ nb_filelist = args->end_fid - args->start_fid + 1;
+
+ if (args->nb_reqs == 0)
+ return 0;
+
+dequeue_req:
+ burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, 0, &op, 1);
+
+ if (likely(burst_deq == 1)) {
+ total_deq += burst_deq;
+ if (unlikely(op->status == RTE_ML_OP_STATUS_ERROR)) {
+ rte_ml_op_error_get(t->cmn.opt->dev_id, op, &error);
+ ml_err("error_code = 0x%016lx, error_message = %s\n", error.errcode,
+ error.message);
+ }
+ req = (struct ml_request *)op->user_ptr;
+ rte_mempool_put(t->model[req->fid].io_pool, req);
+ rte_mempool_put(t->op_pool, op);
+ }
+
+ if (likely(total_deq < args->nb_reqs * nb_filelist))
+ goto dequeue_req;
+
+ return 0;
+}
+
+bool
+test_inference_cap_check(struct ml_options *opt)
+{
+ struct rte_ml_dev_info dev_info;
+
+ if (!ml_test_cap_check(opt))
+ return false;
+
+ rte_ml_dev_info_get(opt->dev_id, &dev_info);
+ if (opt->nb_filelist > dev_info.max_models) {
+ ml_err("Insufficient capabilities: Filelist count exceeded device limit, count = %u (max limit = %u)",
+ opt->nb_filelist, dev_info.max_models);
+ return false;
+ }
+
+ return true;
+}
+
+int
+test_inference_opt_check(struct ml_options *opt)
+{
+ uint32_t i;
+ int ret;
+
+ /* check common opts */
+ ret = ml_test_opt_check(opt);
+ if (ret != 0)
+ return ret;
+
+ /* check file availability */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ if (access(opt->filelist[i].model, F_OK) == -1) {
+ ml_err("Model file not accessible: id = %u, file = %s", i,
+ opt->filelist[i].model);
+ return -ENOENT;
+ }
+
+ if (access(opt->filelist[i].input, F_OK) == -1) {
+ ml_err("Input file not accessible: id = %u, file = %s", i,
+ opt->filelist[i].input);
+ return -ENOENT;
+ }
+ }
+
+ if (opt->repetitions == 0) {
+ ml_err("Invalid option, repetitions = %" PRIu64 "\n", opt->repetitions);
+ return -EINVAL;
+ }
+
+ /* check number of available lcores. */
+ if (rte_lcore_count() < 3) {
+ ml_err("Insufficient lcores = %u\n", rte_lcore_count());
+ ml_err("Minimum lcores required to create %u queue-pairs = %u\n", 1, 3);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+void
+test_inference_opt_dump(struct ml_options *opt)
+{
+ uint32_t i;
+
+ /* dump common opts */
+ ml_test_opt_dump(opt);
+
+ /* dump test opts */
+ ml_dump("repetitions", "%" PRIu64, opt->repetitions);
+
+ ml_dump_begin("filelist");
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ml_dump_list("model", i, opt->filelist[i].model);
+ ml_dump_list("input", i, opt->filelist[i].input);
+ ml_dump_list("output", i, opt->filelist[i].output);
+ }
+ ml_dump_end;
+}
+
+int
+test_inference_setup(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t;
+ void *test_inference;
+ int ret = 0;
+ uint32_t i;
+
+ test_inference = rte_zmalloc_socket(test->name, sizeof(struct test_inference),
+ RTE_CACHE_LINE_SIZE, opt->socket_id);
+ if (test_inference == NULL) {
+ ml_err("failed to allocate memory for test_model");
+ ret = -ENOMEM;
+ goto error;
+ }
+ test->test_priv = test_inference;
+ t = ml_test_priv(test);
+
+ t->nb_used = 0;
+ t->cmn.result = ML_TEST_FAILED;
+ t->cmn.opt = opt;
+
+ /* get device info */
+ ret = rte_ml_dev_info_get(opt->dev_id, &t->cmn.dev_info);
+ if (ret < 0) {
+ ml_err("failed to get device info");
+ goto error;
+ }
+
+ t->enqueue = ml_enqueue_single;
+ t->dequeue = ml_dequeue_single;
+
+ /* set model initial state */
+ for (i = 0; i < opt->nb_filelist; i++)
+ t->model[i].state = MODEL_INITIAL;
+
+ return 0;
+
+error:
+ if (test_inference != NULL)
+ rte_free(test_inference);
+
+ return ret;
+}
+
+void
+test_inference_destroy(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+ if (t != NULL)
+ rte_free(t);
+}
+
+int
+ml_inference_mldev_setup(struct ml_test *test, struct ml_options *opt)
+{
+ struct rte_ml_dev_qp_conf qp_conf;
+ struct test_inference *t;
+ int ret;
+
+ t = ml_test_priv(test);
+
+ ret = ml_test_device_configure(test, opt);
+ if (ret != 0)
+ return ret;
+
+ /* setup queue pairs */
+ qp_conf.nb_desc = t->cmn.dev_info.max_desc;
+ qp_conf.cb = NULL;
+
+ ret = rte_ml_dev_queue_pair_setup(opt->dev_id, 0, &qp_conf, opt->socket_id);
+ if (ret != 0) {
+ ml_err("Failed to setup ml device queue-pair, dev_id = %d, qp_id = %u\n",
+ opt->dev_id, 0);
+ goto error;
+ }
+
+ ret = ml_test_device_start(test, opt);
+ if (ret != 0)
+ goto error;
+
+ return 0;
+
+error:
+ ml_test_device_close(test, opt);
+
+ return ret;
+}
+
+int
+ml_inference_mldev_destroy(struct ml_test *test, struct ml_options *opt)
+{
+ int ret;
+
+ ret = ml_test_device_stop(test, opt);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_test_device_close(test, opt);
+ if (ret != 0)
+ return ret;
+
+ return 0;
+
+error:
+ ml_test_device_close(test, opt);
+
+ return ret;
+}
+
+/* Callback for IO pool create. This function would compute the fields of ml_request
+ * structure and prepare the quantized input data.
+ */
+static void
+ml_request_initialize(struct rte_mempool *mp, void *opaque, void *obj, unsigned int obj_idx)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)opaque);
+ struct ml_request *req = (struct ml_request *)obj;
+
+ RTE_SET_USED(mp);
+ RTE_SET_USED(obj_idx);
+
+ req->input = RTE_PTR_ADD(
+ obj, RTE_ALIGN_CEIL(sizeof(struct ml_request), t->cmn.dev_info.min_align_size));
+ req->output = RTE_PTR_ADD(req->input, RTE_ALIGN_CEIL(t->model[t->fid].inp_qsize,
+ t->cmn.dev_info.min_align_size));
+ req->niters = 0;
+
+ /* quantize data */
+ rte_ml_io_quantize(t->cmn.opt->dev_id, t->model[t->fid].id,
+ t->model[t->fid].info.batch_size, t->model[t->fid].input, req->input);
+}
+
+int
+ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, int16_t fid)
+{
+ struct test_inference *t = ml_test_priv(test);
+ char mz_name[RTE_MEMZONE_NAMESIZE];
+ char mp_name[RTE_MEMPOOL_NAMESIZE];
+ const struct rte_memzone *mz;
+ uint64_t nb_buffers;
+ uint32_t buff_size;
+ uint32_t mz_size;
+ uint32_t fsize;
+ FILE *fp;
+ int ret;
+
+ /* get input buffer size */
+ ret = rte_ml_io_input_size_get(opt->dev_id, t->model[fid].id, t->model[fid].info.batch_size,
+ &t->model[fid].inp_qsize, &t->model[fid].inp_dsize);
+ if (ret != 0) {
+ ml_err("Failed to get input size, model : %s\n", opt->filelist[fid].model);
+ return ret;
+ }
+
+ /* get output buffer size */
+ ret = rte_ml_io_output_size_get(opt->dev_id, t->model[fid].id,
+ t->model[fid].info.batch_size, &t->model[fid].out_qsize,
+ &t->model[fid].out_dsize);
+ if (ret != 0) {
+ ml_err("Failed to get input size, model : %s\n", opt->filelist[fid].model);
+ return ret;
+ }
+
+ /* allocate buffer for user data */
+ mz_size = t->model[fid].inp_dsize + t->model[fid].out_dsize;
+ sprintf(mz_name, "ml_user_data_%d", fid);
+ mz = rte_memzone_reserve(mz_name, mz_size, opt->socket_id, 0);
+ if (mz == NULL) {
+ ml_err("Memzone allocation failed for ml_user_data\n");
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ t->model[fid].input = mz->addr;
+ t->model[fid].output = RTE_PTR_ADD(t->model[fid].input, t->model[fid].inp_dsize);
+
+ /* load input file */
+ fp = fopen(opt->filelist[fid].input, "r");
+ if (fp == NULL) {
+ ml_err("Failed to open input file : %s\n", opt->filelist[fid].input);
+ ret = -errno;
+ goto error;
+ }
+
+ fseek(fp, 0, SEEK_END);
+ fsize = ftell(fp);
+ fseek(fp, 0, SEEK_SET);
+ if (fsize != t->model[fid].inp_dsize) {
+ ml_err("Invalid input file, size = %u (expected size = %" PRIu64 ")\n", fsize,
+ t->model[fid].inp_dsize);
+ ret = -EINVAL;
+ fclose(fp);
+ goto error;
+ }
+
+ if (fread(t->model[fid].input, 1, t->model[fid].inp_dsize, fp) != t->model[fid].inp_dsize) {
+ ml_err("Failed to read input file : %s\n", opt->filelist[fid].input);
+ ret = -errno;
+ fclose(fp);
+ goto error;
+ }
+ fclose(fp);
+
+ /* create mempool for quantized input and output buffers. ml_request_initialize is
+ * used as a callback for object creation.
+ */
+ buff_size = RTE_ALIGN_CEIL(sizeof(struct ml_request), t->cmn.dev_info.min_align_size) +
+ RTE_ALIGN_CEIL(t->model[fid].inp_qsize, t->cmn.dev_info.min_align_size) +
+ RTE_ALIGN_CEIL(t->model[fid].out_qsize, t->cmn.dev_info.min_align_size);
+ nb_buffers = RTE_MIN((uint64_t)ML_TEST_MAX_POOL_SIZE, opt->repetitions);
+
+ t->fid = fid;
+ sprintf(mp_name, "ml_io_pool_%d", fid);
+ t->model[fid].io_pool = rte_mempool_create(mp_name, nb_buffers, buff_size, 0, 0, NULL, NULL,
+ ml_request_initialize, test, opt->socket_id, 0);
+ if (t->model[fid].io_pool == NULL) {
+ ml_err("Failed to create io pool : %s\n", "ml_io_pool");
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ return 0;
+
+error:
+ if (mz != NULL)
+ rte_memzone_free(mz);
+
+ if (t->model[fid].io_pool != NULL) {
+ rte_mempool_free(t->model[fid].io_pool);
+ t->model[fid].io_pool = NULL;
+ }
+
+ return ret;
+}
+
+void
+ml_inference_iomem_destroy(struct ml_test *test, struct ml_options *opt, int16_t fid)
+{
+ char mz_name[RTE_MEMZONE_NAMESIZE];
+ char mp_name[RTE_MEMPOOL_NAMESIZE];
+ const struct rte_memzone *mz;
+ struct rte_mempool *mp;
+
+ RTE_SET_USED(test);
+ RTE_SET_USED(opt);
+
+ /* release user data memzone */
+ sprintf(mz_name, "ml_user_data_%d", fid);
+ mz = rte_memzone_lookup(mz_name);
+ if (mz != NULL)
+ rte_memzone_free(mz);
+
+ /* destroy io pool */
+ sprintf(mp_name, "ml_io_pool_%d", fid);
+ mp = rte_mempool_lookup(mp_name);
+ if (mp != NULL)
+ rte_mempool_free(mp);
+}
+
+int
+ml_inference_mem_setup(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t = ml_test_priv(test);
+
+ /* create op pool */
+ t->op_pool = rte_ml_op_pool_create("ml_test_op_pool", ML_TEST_MAX_POOL_SIZE, 0, 0,
+ opt->socket_id);
+ if (t->op_pool == NULL) {
+ ml_err("Failed to create op pool : %s\n", "ml_op_pool");
+ return -ENOMEM;
+ }
+
+ return 0;
+}
+
+void
+ml_inference_mem_destroy(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t = ml_test_priv(test);
+
+ RTE_SET_USED(opt);
+
+ /* release op pool */
+ if (t->op_pool != NULL)
+ rte_mempool_free(t->op_pool);
+}
+
+/* Callback for mempool object iteration. This call would dequantize output data. */
+static void
+ml_request_finish(struct rte_mempool *mp, void *opaque, void *obj, unsigned int obj_idx)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)opaque);
+ struct ml_request *req = (struct ml_request *)obj;
+ struct ml_model *model = &t->model[req->fid];
+
+ RTE_SET_USED(mp);
+ RTE_SET_USED(obj_idx);
+
+ if (req->niters == 0)
+ return;
+
+ t->nb_used++;
+ rte_ml_io_dequantize(t->cmn.opt->dev_id, model->id, t->model[req->fid].info.batch_size,
+ req->output, model->output);
+}
+
+int
+ml_inference_result(struct ml_test *test, struct ml_options *opt, int16_t fid)
+{
+ struct test_inference *t = ml_test_priv(test);
+
+ RTE_SET_USED(opt);
+
+ rte_mempool_obj_iter(t->model[fid].io_pool, ml_request_finish, test);
+
+ if (t->nb_used > 0)
+ t->cmn.result = ML_TEST_SUCCESS;
+ else
+ t->cmn.result = ML_TEST_FAILED;
+
+ return t->cmn.result;
+}
+
+int
+ml_inference_launch_cores(struct ml_test *test, struct ml_options *opt, int16_t start_fid,
+ int16_t end_fid)
+{
+ struct test_inference *t = ml_test_priv(test);
+ uint32_t lcore_id;
+ uint32_t id = 0;
+
+ RTE_LCORE_FOREACH_WORKER(lcore_id)
+ {
+ if (id == 2)
+ break;
+
+ t->args[lcore_id].nb_reqs = opt->repetitions;
+ t->args[lcore_id].start_fid = start_fid;
+ t->args[lcore_id].end_fid = end_fid;
+
+ if (id % 2 == 0)
+ rte_eal_remote_launch(t->enqueue, test, lcore_id);
+ else
+ rte_eal_remote_launch(t->dequeue, test, lcore_id);
+
+ id++;
+ }
+
+ return 0;
+}
diff --git a/app/test-mldev/test_inference_common.h b/app/test-mldev/test_inference_common.h
new file mode 100644
index 0000000000..91007954b4
--- /dev/null
+++ b/app/test-mldev/test_inference_common.h
@@ -0,0 +1,65 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_TEST_INFERENCE_COMMON_
+#define _ML_TEST_INFERENCE_COMMON_
+
+#include <stdbool.h>
+#include <stdint.h>
+
+#include <rte_common.h>
+#include <rte_mempool.h>
+
+#include "ml_options.h"
+#include "ml_test.h"
+#include "test_common.h"
+#include "test_model_common.h"
+
+struct ml_request {
+ void *input;
+ void *output;
+ int16_t fid;
+ uint64_t niters;
+};
+
+struct ml_core_args {
+ uint64_t nb_reqs;
+ int16_t start_fid;
+ int16_t end_fid;
+};
+
+struct test_inference {
+ /* common data */
+ struct test_common cmn;
+
+ /* test specific data */
+ struct ml_model model[ML_TEST_MAX_MODELS];
+ struct rte_mempool *op_pool;
+
+ uint64_t nb_used;
+ int16_t fid;
+
+ int (*enqueue)(void *arg);
+ int (*dequeue)(void *arg);
+
+ struct ml_core_args args[RTE_MAX_LCORE];
+} __rte_cache_aligned;
+
+bool test_inference_cap_check(struct ml_options *opt);
+int test_inference_opt_check(struct ml_options *opt);
+void test_inference_opt_dump(struct ml_options *opt);
+int test_inference_setup(struct ml_test *test, struct ml_options *opt);
+void test_inference_destroy(struct ml_test *test, struct ml_options *opt);
+
+int ml_inference_mldev_setup(struct ml_test *test, struct ml_options *opt);
+int ml_inference_mldev_destroy(struct ml_test *test, struct ml_options *opt);
+int ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, int16_t fid);
+void ml_inference_iomem_destroy(struct ml_test *test, struct ml_options *opt, int16_t fid);
+int ml_inference_mem_setup(struct ml_test *test, struct ml_options *opt);
+void ml_inference_mem_destroy(struct ml_test *test, struct ml_options *opt);
+int ml_inference_result(struct ml_test *test, struct ml_options *opt, int16_t fid);
+int ml_inference_launch_cores(struct ml_test *test, struct ml_options *opt, int16_t start_fid,
+ int16_t end_fid);
+
+#endif /* _ML_TEST_INFERENCE_COMMON_ */
diff --git a/app/test-mldev/test_inference_ordered.c b/app/test-mldev/test_inference_ordered.c
new file mode 100644
index 0000000000..84e6bf9109
--- /dev/null
+++ b/app/test-mldev/test_inference_ordered.c
@@ -0,0 +1,119 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <stdint.h>
+
+#include <rte_common.h>
+#include <rte_launch.h>
+
+#include "ml_common.h"
+#include "ml_test.h"
+#include "test_inference_common.h"
+#include "test_model_common.h"
+
+static int
+test_inference_ordered_driver(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t;
+ int16_t fid = 0;
+ int ret = 0;
+
+ t = ml_test_priv(test);
+
+ ret = ml_inference_mldev_setup(test, opt);
+ if (ret != 0)
+ return ret;
+
+ ret = ml_inference_mem_setup(test, opt);
+ if (ret != 0)
+ return ret;
+
+next_model:
+ /* load model */
+ ret = ml_model_load(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ /* start model */
+ ret = ml_model_start(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_inference_iomem_setup(test, opt, fid);
+ if (ret != 0)
+ goto error;
+
+ /* launch inferences for one model using available queue pairs */
+ ret = ml_inference_launch_cores(test, opt, fid, fid);
+ if (ret != 0) {
+ ml_err("failed to launch cores");
+ goto error;
+ }
+
+ rte_eal_mp_wait_lcore();
+
+ ret = ml_inference_result(test, opt, fid);
+ if (ret != ML_TEST_SUCCESS)
+ goto error;
+
+ ml_inference_iomem_destroy(test, opt, fid);
+
+ /* stop model */
+ ret = ml_model_stop(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ /* unload model */
+ ret = ml_model_unload(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ fid++;
+ if (fid < opt->nb_filelist)
+ goto next_model;
+
+ ml_inference_mem_destroy(test, opt);
+
+ ret = ml_inference_mldev_destroy(test, opt);
+ if (ret != 0)
+ return ret;
+
+ t->cmn.result = ML_TEST_SUCCESS;
+
+ return 0;
+
+error:
+ ml_inference_iomem_destroy(test, opt, fid);
+ ml_inference_mem_destroy(test, opt);
+ ml_model_stop(test, opt, &t->model[fid], fid);
+ ml_model_unload(test, opt, &t->model[fid], fid);
+
+ t->cmn.result = ML_TEST_FAILED;
+
+ return ret;
+}
+
+static int
+test_inference_ordered_result(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+
+ return t->cmn.result;
+}
+
+static const struct ml_test_ops inference_ordered = {
+ .cap_check = test_inference_cap_check,
+ .opt_check = test_inference_opt_check,
+ .opt_dump = test_inference_opt_dump,
+ .test_setup = test_inference_setup,
+ .test_destroy = test_inference_destroy,
+ .test_driver = test_inference_ordered_driver,
+ .test_result = test_inference_ordered_result,
+};
+
+ML_TEST_REGISTER(inference_ordered);
diff --git a/app/test-mldev/test_model_common.h b/app/test-mldev/test_model_common.h
index 302e4eb45f..c45ae80853 100644
--- a/app/test-mldev/test_model_common.h
+++ b/app/test-mldev/test_model_common.h
@@ -23,6 +23,16 @@ struct ml_model {
int16_t id;
struct rte_ml_model_info info;
enum model_state state;
+
+ uint64_t inp_dsize;
+ uint64_t inp_qsize;
+ uint64_t out_dsize;
+ uint64_t out_qsize;
+
+ uint8_t *input;
+ uint8_t *output;
+
+ struct rte_mempool *io_pool;
};
int ml_model_load(struct ml_test *test, struct ml_options *opt, struct ml_model *model,
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v3 06/12] app/mldev: add test case to interleave inferences
2022-12-08 19:29 ` [PATCH v3 01/12] app/mldev: implement test framework for mldev Srikanth Yalavarthi
` (3 preceding siblings ...)
2022-12-08 19:29 ` [PATCH v3 05/12] app/mldev: add ordered inference test case Srikanth Yalavarthi
@ 2022-12-08 19:29 ` Srikanth Yalavarthi
2022-12-08 19:29 ` [PATCH v3 07/12] app/mldev: enable support for burst inferences Srikanth Yalavarthi
` (5 subsequent siblings)
10 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2022-12-08 19:29 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu
Added test case to interleave inference requests from multiple
models. Interleaving would load and start all models and launch
inference requests for the models using available queue-pairs
Operations sequence when testing with N models and R reps,
(load + start) x N -> (enqueue + dequeue) x N x R ...
-> (stop + unload) x N
Test can be executed by selecting "inference_interleave" test.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/meson.build | 1 +
app/test-mldev/ml_options.c | 3 +-
app/test-mldev/test_inference_common.c | 12 +--
app/test-mldev/test_inference_common.h | 4 +-
app/test-mldev/test_inference_interleave.c | 118 +++++++++++++++++++++
5 files changed, 129 insertions(+), 9 deletions(-)
create mode 100644 app/test-mldev/test_inference_interleave.c
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
index 475d76d126..41d22fb22c 100644
--- a/app/test-mldev/meson.build
+++ b/app/test-mldev/meson.build
@@ -18,6 +18,7 @@ sources = files(
'test_model_ops.c',
'test_inference_common.c',
'test_inference_ordered.c',
+ 'test_inference_interleave.c',
)
deps += ['mldev']
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 10dad18fff..01ea050ee7 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -162,7 +162,8 @@ ml_dump_test_options(const char *testname)
printf("\n");
}
- if (strcmp(testname, "inference_ordered") == 0) {
+ if ((strcmp(testname, "inference_ordered") == 0) ||
+ (strcmp(testname, "inference_interleave") == 0)) {
printf("\t\t--filelist : comma separated list of model, input and output\n"
"\t\t--repetitions : number of inference repetitions\n");
printf("\n");
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
index 8b5dc89346..f0b15861a0 100644
--- a/app/test-mldev/test_inference_common.c
+++ b/app/test-mldev/test_inference_common.c
@@ -115,7 +115,7 @@ ml_dequeue_single(void *arg)
total_deq += burst_deq;
if (unlikely(op->status == RTE_ML_OP_STATUS_ERROR)) {
rte_ml_op_error_get(t->cmn.opt->dev_id, op, &error);
- ml_err("error_code = 0x%016lx, error_message = %s\n", error.errcode,
+ ml_err("error_code = 0x%" PRIx64 ", error_message = %s\n", error.errcode,
error.message);
}
req = (struct ml_request *)op->user_ptr;
@@ -334,10 +334,10 @@ ml_request_initialize(struct rte_mempool *mp, void *opaque, void *obj, unsigned
RTE_SET_USED(mp);
RTE_SET_USED(obj_idx);
- req->input = RTE_PTR_ADD(
- obj, RTE_ALIGN_CEIL(sizeof(struct ml_request), t->cmn.dev_info.min_align_size));
- req->output = RTE_PTR_ADD(req->input, RTE_ALIGN_CEIL(t->model[t->fid].inp_qsize,
- t->cmn.dev_info.min_align_size));
+ req->input = (uint8_t *)obj +
+ RTE_ALIGN_CEIL(sizeof(struct ml_request), t->cmn.dev_info.min_align_size);
+ req->output = req->input +
+ RTE_ALIGN_CEIL(t->model[t->fid].inp_qsize, t->cmn.dev_info.min_align_size);
req->niters = 0;
/* quantize data */
@@ -387,7 +387,7 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, int16_t f
}
t->model[fid].input = mz->addr;
- t->model[fid].output = RTE_PTR_ADD(t->model[fid].input, t->model[fid].inp_dsize);
+ t->model[fid].output = t->model[fid].input + t->model[fid].inp_dsize;
/* load input file */
fp = fopen(opt->filelist[fid].input, "r");
diff --git a/app/test-mldev/test_inference_common.h b/app/test-mldev/test_inference_common.h
index 91007954b4..b058abada4 100644
--- a/app/test-mldev/test_inference_common.h
+++ b/app/test-mldev/test_inference_common.h
@@ -17,8 +17,8 @@
#include "test_model_common.h"
struct ml_request {
- void *input;
- void *output;
+ uint8_t *input;
+ uint8_t *output;
int16_t fid;
uint64_t niters;
};
diff --git a/app/test-mldev/test_inference_interleave.c b/app/test-mldev/test_inference_interleave.c
new file mode 100644
index 0000000000..74ad0c597f
--- /dev/null
+++ b/app/test-mldev/test_inference_interleave.c
@@ -0,0 +1,118 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <stdint.h>
+
+#include <rte_common.h>
+#include <rte_launch.h>
+
+#include "ml_common.h"
+#include "ml_test.h"
+#include "test_inference_common.h"
+#include "test_model_common.h"
+
+static int
+test_inference_interleave_driver(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t;
+ int16_t fid = 0;
+ int ret = 0;
+
+ t = ml_test_priv(test);
+
+ ret = ml_inference_mldev_setup(test, opt);
+ if (ret != 0)
+ return ret;
+
+ ret = ml_inference_mem_setup(test, opt);
+ if (ret != 0)
+ return ret;
+
+ /* load and start all models */
+ for (fid = 0; fid < opt->nb_filelist; fid++) {
+ ret = ml_model_load(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_start(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_inference_iomem_setup(test, opt, fid);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* launch inference requests */
+ ret = ml_inference_launch_cores(test, opt, 0, opt->nb_filelist - 1);
+ if (ret != 0) {
+ ml_err("failed to launch cores");
+ goto error;
+ }
+
+ rte_eal_mp_wait_lcore();
+
+ /* stop and unload all models */
+ for (fid = 0; fid < opt->nb_filelist; fid++) {
+ ret = ml_inference_result(test, opt, fid);
+ if (ret != ML_TEST_SUCCESS)
+ goto error;
+
+ ml_inference_iomem_destroy(test, opt, fid);
+
+ ret = ml_model_stop(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_unload(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+ }
+
+ ml_inference_mem_destroy(test, opt);
+
+ ret = ml_inference_mldev_destroy(test, opt);
+ if (ret != 0)
+ return ret;
+
+ t->cmn.result = ML_TEST_SUCCESS;
+
+ return 0;
+
+error:
+ ml_inference_mem_destroy(test, opt);
+ for (fid = 0; fid < opt->nb_filelist; fid++) {
+ ml_inference_iomem_destroy(test, opt, fid);
+ ml_model_stop(test, opt, &t->model[fid], fid);
+ ml_model_unload(test, opt, &t->model[fid], fid);
+ }
+
+ t->cmn.result = ML_TEST_FAILED;
+
+ return ret;
+}
+
+static int
+test_inference_interleave_result(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+
+ return t->cmn.result;
+}
+
+static const struct ml_test_ops inference_interleave = {
+ .cap_check = test_inference_cap_check,
+ .opt_check = test_inference_opt_check,
+ .opt_dump = test_inference_opt_dump,
+ .test_setup = test_inference_setup,
+ .test_destroy = test_inference_destroy,
+ .test_driver = test_inference_interleave_driver,
+ .test_result = test_inference_interleave_result,
+};
+
+ML_TEST_REGISTER(inference_interleave);
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v3 07/12] app/mldev: enable support for burst inferences
2022-12-08 19:29 ` [PATCH v3 01/12] app/mldev: implement test framework for mldev Srikanth Yalavarthi
` (4 preceding siblings ...)
2022-12-08 19:29 ` [PATCH v3 06/12] app/mldev: add test case to interleave inferences Srikanth Yalavarthi
@ 2022-12-08 19:29 ` Srikanth Yalavarthi
2022-12-08 19:29 ` [PATCH v3 08/12] app/mldev: enable support for queue pairs and size Srikanth Yalavarthi
` (4 subsequent siblings)
10 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2022-12-08 19:29 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu
Added 'burst_size' support for inference tests. Burst size
controls the number of inference requests handled during
the burst enqueue and dequeue operations of the test case.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/ml_options.c | 26 ++--
app/test-mldev/ml_options.h | 2 +
app/test-mldev/test_inference_common.c | 159 ++++++++++++++++++++++++-
app/test-mldev/test_inference_common.h | 4 +
4 files changed, 181 insertions(+), 10 deletions(-)
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 01ea050ee7..1990939200 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -30,6 +30,7 @@ ml_options_default(struct ml_options *opt)
opt->socket_id = SOCKET_ID_ANY;
opt->nb_filelist = 0;
opt->repetitions = 1;
+ opt->burst_size = 1;
opt->debug = false;
}
@@ -151,6 +152,12 @@ ml_parse_repetitions(struct ml_options *opt, const char *arg)
return parser_read_uint64(&opt->repetitions, arg);
}
+static int
+ml_parse_burst_size(struct ml_options *opt, const char *arg)
+{
+ return parser_read_uint16(&opt->burst_size, arg);
+}
+
static void
ml_dump_test_options(const char *testname)
{
@@ -165,7 +172,8 @@ ml_dump_test_options(const char *testname)
if ((strcmp(testname, "inference_ordered") == 0) ||
(strcmp(testname, "inference_interleave") == 0)) {
printf("\t\t--filelist : comma separated list of model, input and output\n"
- "\t\t--repetitions : number of inference repetitions\n");
+ "\t\t--repetitions : number of inference repetitions\n"
+ "\t\t--burst_size : inference burst size\n");
printf("\n");
}
}
@@ -185,10 +193,11 @@ print_usage(char *program)
ml_test_dump_names(ml_dump_test_options);
}
-static struct option lgopts[] = {
- {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
- {ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
- {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
+static struct option lgopts[] = {{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0},
+ {ML_SOCKET_ID, 1, 0, 0}, {ML_MODELS, 1, 0, 0},
+ {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
+ {ML_BURST_SIZE, 1, 0, 0}, {ML_DEBUG, 0, 0, 0},
+ {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
static int
ml_opts_parse_long(int opt_idx, struct ml_options *opt)
@@ -196,9 +205,10 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
unsigned int i;
struct long_opt_parser parsermap[] = {
- {ML_TEST, ml_parse_test_name}, {ML_DEVICE_ID, ml_parse_dev_id},
- {ML_SOCKET_ID, ml_parse_socket_id}, {ML_MODELS, ml_parse_models},
- {ML_FILELIST, ml_parse_filelist}, {ML_REPETITIONS, ml_parse_repetitions},
+ {ML_TEST, ml_parse_test_name}, {ML_DEVICE_ID, ml_parse_dev_id},
+ {ML_SOCKET_ID, ml_parse_socket_id}, {ML_MODELS, ml_parse_models},
+ {ML_FILELIST, ml_parse_filelist}, {ML_REPETITIONS, ml_parse_repetitions},
+ {ML_BURST_SIZE, ml_parse_burst_size},
};
for (i = 0; i < RTE_DIM(parsermap); i++) {
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index ad8aee5964..305b39629a 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -19,6 +19,7 @@
#define ML_MODELS ("models")
#define ML_FILELIST ("filelist")
#define ML_REPETITIONS ("repetitions")
+#define ML_BURST_SIZE ("burst_size")
#define ML_DEBUG ("debug")
#define ML_HELP ("help")
@@ -35,6 +36,7 @@ struct ml_options {
struct ml_filelist filelist[ML_TEST_MAX_MODELS];
uint8_t nb_filelist;
uint64_t repetitions;
+ uint16_t burst_size;
bool debug;
};
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
index f0b15861a0..252d77616c 100644
--- a/app/test-mldev/test_inference_common.c
+++ b/app/test-mldev/test_inference_common.c
@@ -129,6 +129,131 @@ ml_dequeue_single(void *arg)
return 0;
}
+/* Enqueue inference requests with burst size greater than 1 */
+static int
+ml_enqueue_burst(void *arg)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)arg);
+ struct ml_core_args *args;
+ uint16_t ops_count;
+ uint64_t model_enq;
+ uint16_t burst_enq;
+ uint32_t lcore_id;
+ uint16_t pending;
+ uint16_t idx;
+ int16_t fid;
+ uint16_t i;
+ int ret;
+
+ lcore_id = rte_lcore_id();
+ args = &t->args[lcore_id];
+ model_enq = 0;
+
+ if (args->nb_reqs == 0)
+ return 0;
+
+next_rep:
+ fid = args->start_fid;
+
+next_model:
+ ops_count = RTE_MIN(t->cmn.opt->burst_size, args->nb_reqs - model_enq);
+ ret = rte_mempool_get_bulk(t->op_pool, (void **)args->enq_ops, ops_count);
+ if (ret != 0)
+ goto next_model;
+
+retry:
+ ret = rte_mempool_get_bulk(t->model[fid].io_pool, (void **)args->reqs, ops_count);
+ if (ret != 0)
+ goto retry;
+
+ for (i = 0; i < ops_count; i++) {
+ args->enq_ops[i]->model_id = t->model[fid].id;
+ args->enq_ops[i]->nb_batches = t->model[fid].info.batch_size;
+ args->enq_ops[i]->mempool = t->op_pool;
+
+ args->enq_ops[i]->input.addr = args->reqs[i]->input;
+ args->enq_ops[i]->input.length = t->model[fid].inp_qsize;
+ args->enq_ops[i]->input.next = NULL;
+
+ args->enq_ops[i]->output.addr = args->reqs[i]->output;
+ args->enq_ops[i]->output.length = t->model[fid].out_qsize;
+ args->enq_ops[i]->output.next = NULL;
+
+ args->enq_ops[i]->user_ptr = args->reqs[i];
+ args->reqs[i]->niters++;
+ args->reqs[i]->fid = fid;
+ }
+
+ idx = 0;
+ pending = ops_count;
+
+enqueue_reqs:
+ burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, 0, &args->enq_ops[idx], pending);
+ pending = pending - burst_enq;
+
+ if (pending > 0) {
+ idx = idx + burst_enq;
+ goto enqueue_reqs;
+ }
+
+ fid++;
+ if (fid <= args->end_fid)
+ goto next_model;
+
+ model_enq = model_enq + ops_count;
+ if (model_enq < args->nb_reqs)
+ goto next_rep;
+
+ return 0;
+}
+
+/* Dequeue inference requests with burst size greater than 1 */
+static int
+ml_dequeue_burst(void *arg)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)arg);
+ struct rte_ml_op_error error;
+ struct ml_core_args *args;
+ struct ml_request *req;
+ uint64_t total_deq = 0;
+ uint16_t burst_deq = 0;
+ uint8_t nb_filelist;
+ uint32_t lcore_id;
+ uint32_t i;
+
+ lcore_id = rte_lcore_id();
+ args = &t->args[lcore_id];
+ nb_filelist = args->end_fid - args->start_fid + 1;
+
+ if (args->nb_reqs == 0)
+ return 0;
+
+dequeue_burst:
+ burst_deq =
+ rte_ml_dequeue_burst(t->cmn.opt->dev_id, 0, args->deq_ops, t->cmn.opt->burst_size);
+
+ if (likely(burst_deq > 0)) {
+ total_deq += burst_deq;
+
+ for (i = 0; i < burst_deq; i++) {
+ if (unlikely(args->deq_ops[i]->status == RTE_ML_OP_STATUS_ERROR)) {
+ rte_ml_op_error_get(t->cmn.opt->dev_id, args->deq_ops[i], &error);
+ ml_err("error_code = 0x%" PRIx64 ", error_message = %s\n",
+ error.errcode, error.message);
+ }
+ req = (struct ml_request *)args->deq_ops[i]->user_ptr;
+ if (req != NULL)
+ rte_mempool_put(t->model[req->fid].io_pool, req);
+ }
+ rte_mempool_put_bulk(t->op_pool, (void *)args->deq_ops, burst_deq);
+ }
+
+ if (total_deq < args->nb_reqs * nb_filelist)
+ goto dequeue_burst;
+
+ return 0;
+}
+
bool
test_inference_cap_check(struct ml_options *opt)
{
@@ -178,6 +303,17 @@ test_inference_opt_check(struct ml_options *opt)
return -EINVAL;
}
+ if (opt->burst_size == 0) {
+ ml_err("Invalid option, burst_size = %u\n", opt->burst_size);
+ return -EINVAL;
+ }
+
+ if (opt->burst_size > ML_TEST_MAX_POOL_SIZE) {
+ ml_err("Invalid option, burst_size = %u (> max supported = %d)\n", opt->burst_size,
+ ML_TEST_MAX_POOL_SIZE);
+ return -EINVAL;
+ }
+
/* check number of available lcores. */
if (rte_lcore_count() < 3) {
ml_err("Insufficient lcores = %u\n", rte_lcore_count());
@@ -198,6 +334,7 @@ test_inference_opt_dump(struct ml_options *opt)
/* dump test opts */
ml_dump("repetitions", "%" PRIu64, opt->repetitions);
+ ml_dump("burst_size", "%u", opt->burst_size);
ml_dump_begin("filelist");
for (i = 0; i < opt->nb_filelist; i++) {
@@ -213,6 +350,7 @@ test_inference_setup(struct ml_test *test, struct ml_options *opt)
{
struct test_inference *t;
void *test_inference;
+ uint32_t lcore_id;
int ret = 0;
uint32_t i;
@@ -237,13 +375,30 @@ test_inference_setup(struct ml_test *test, struct ml_options *opt)
goto error;
}
- t->enqueue = ml_enqueue_single;
- t->dequeue = ml_dequeue_single;
+ if (opt->burst_size == 1) {
+ t->enqueue = ml_enqueue_single;
+ t->dequeue = ml_dequeue_single;
+ } else {
+ t->enqueue = ml_enqueue_burst;
+ t->dequeue = ml_dequeue_burst;
+ }
/* set model initial state */
for (i = 0; i < opt->nb_filelist; i++)
t->model[i].state = MODEL_INITIAL;
+ for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) {
+ t->args[lcore_id].enq_ops = rte_zmalloc_socket(
+ "ml_test_enq_ops", opt->burst_size * sizeof(struct rte_ml_op *),
+ RTE_CACHE_LINE_SIZE, opt->socket_id);
+ t->args[lcore_id].deq_ops = rte_zmalloc_socket(
+ "ml_test_deq_ops", opt->burst_size * sizeof(struct rte_ml_op *),
+ RTE_CACHE_LINE_SIZE, opt->socket_id);
+ t->args[lcore_id].reqs = rte_zmalloc_socket(
+ "ml_test_requests", opt->burst_size * sizeof(struct ml_request *),
+ RTE_CACHE_LINE_SIZE, opt->socket_id);
+ }
+
return 0;
error:
diff --git a/app/test-mldev/test_inference_common.h b/app/test-mldev/test_inference_common.h
index b058abada4..75d588308b 100644
--- a/app/test-mldev/test_inference_common.h
+++ b/app/test-mldev/test_inference_common.h
@@ -27,6 +27,10 @@ struct ml_core_args {
uint64_t nb_reqs;
int16_t start_fid;
int16_t end_fid;
+
+ struct rte_ml_op **enq_ops;
+ struct rte_ml_op **deq_ops;
+ struct ml_request **reqs;
};
struct test_inference {
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v3 08/12] app/mldev: enable support for queue pairs and size
2022-12-08 19:29 ` [PATCH v3 01/12] app/mldev: implement test framework for mldev Srikanth Yalavarthi
` (5 preceding siblings ...)
2022-12-08 19:29 ` [PATCH v3 07/12] app/mldev: enable support for burst inferences Srikanth Yalavarthi
@ 2022-12-08 19:29 ` Srikanth Yalavarthi
2022-12-08 19:29 ` [PATCH v3 09/12] app/mldev: enable support for inference batches Srikanth Yalavarthi
` (3 subsequent siblings)
10 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2022-12-08 19:29 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu
Added support to create multiple queue-pairs per device to
enqueue and dequeue inference requests. Number of queue pairs
to be created can be specified through "--queue_pairs" option.
Support is also enabled to control the number of descriptors
per each queue pair through "--queue_size" option. Inference
requests for a model are distributed across all available
queue-pairs.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/ml_options.c | 40 ++++++++++---
app/test-mldev/ml_options.h | 4 ++
app/test-mldev/test_common.c | 2 +-
app/test-mldev/test_inference_common.c | 79 +++++++++++++++++++++-----
app/test-mldev/test_inference_common.h | 1 +
5 files changed, 102 insertions(+), 24 deletions(-)
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 1990939200..d5182a1701 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -31,6 +31,8 @@ ml_options_default(struct ml_options *opt)
opt->nb_filelist = 0;
opt->repetitions = 1;
opt->burst_size = 1;
+ opt->queue_pairs = 1;
+ opt->queue_size = 1;
opt->debug = false;
}
@@ -158,11 +160,30 @@ ml_parse_burst_size(struct ml_options *opt, const char *arg)
return parser_read_uint16(&opt->burst_size, arg);
}
+static int
+ml_parse_queue_pairs(struct ml_options *opt, const char *arg)
+{
+ int ret;
+
+ ret = parser_read_uint16(&opt->queue_pairs, arg);
+
+ return ret;
+}
+
+static int
+ml_parse_queue_size(struct ml_options *opt, const char *arg)
+{
+ return parser_read_uint16(&opt->queue_size, arg);
+}
+
static void
ml_dump_test_options(const char *testname)
{
- if (strcmp(testname, "device_ops") == 0)
+ if (strcmp(testname, "device_ops") == 0) {
+ printf("\t\t--queue_pairs : number of queue pairs to create\n"
+ "\t\t--queue_size : size fo queue-pair\n");
printf("\n");
+ }
if (strcmp(testname, "model_ops") == 0) {
printf("\t\t--models : comma separated list of models\n");
@@ -173,7 +194,9 @@ ml_dump_test_options(const char *testname)
(strcmp(testname, "inference_interleave") == 0)) {
printf("\t\t--filelist : comma separated list of model, input and output\n"
"\t\t--repetitions : number of inference repetitions\n"
- "\t\t--burst_size : inference burst size\n");
+ "\t\t--burst_size : inference burst size\n"
+ "\t\t--queue_pairs : number of queue pairs to create\n"
+ "\t\t--queue_size : size fo queue-pair\n");
printf("\n");
}
}
@@ -193,11 +216,11 @@ print_usage(char *program)
ml_test_dump_names(ml_dump_test_options);
}
-static struct option lgopts[] = {{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0},
- {ML_SOCKET_ID, 1, 0, 0}, {ML_MODELS, 1, 0, 0},
- {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
- {ML_BURST_SIZE, 1, 0, 0}, {ML_DEBUG, 0, 0, 0},
- {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
+static struct option lgopts[] = {
+ {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
+ {ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
+ {ML_BURST_SIZE, 1, 0, 0}, {ML_QUEUE_PAIRS, 1, 0, 0}, {ML_QUEUE_SIZE, 1, 0, 0},
+ {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
static int
ml_opts_parse_long(int opt_idx, struct ml_options *opt)
@@ -208,7 +231,8 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
{ML_TEST, ml_parse_test_name}, {ML_DEVICE_ID, ml_parse_dev_id},
{ML_SOCKET_ID, ml_parse_socket_id}, {ML_MODELS, ml_parse_models},
{ML_FILELIST, ml_parse_filelist}, {ML_REPETITIONS, ml_parse_repetitions},
- {ML_BURST_SIZE, ml_parse_burst_size},
+ {ML_BURST_SIZE, ml_parse_burst_size}, {ML_QUEUE_PAIRS, ml_parse_queue_pairs},
+ {ML_QUEUE_SIZE, ml_parse_queue_size},
};
for (i = 0; i < RTE_DIM(parsermap); i++) {
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index 305b39629a..6bfef1b979 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -20,6 +20,8 @@
#define ML_FILELIST ("filelist")
#define ML_REPETITIONS ("repetitions")
#define ML_BURST_SIZE ("burst_size")
+#define ML_QUEUE_PAIRS ("queue_pairs")
+#define ML_QUEUE_SIZE ("queue_size")
#define ML_DEBUG ("debug")
#define ML_HELP ("help")
@@ -37,6 +39,8 @@ struct ml_options {
uint8_t nb_filelist;
uint64_t repetitions;
uint16_t burst_size;
+ uint16_t queue_pairs;
+ uint16_t queue_size;
bool debug;
};
diff --git a/app/test-mldev/test_common.c b/app/test-mldev/test_common.c
index b6b32904e4..22e6acb3b6 100644
--- a/app/test-mldev/test_common.c
+++ b/app/test-mldev/test_common.c
@@ -78,7 +78,7 @@ ml_test_device_configure(struct ml_test *test, struct ml_options *opt)
/* configure device */
dev_config.socket_id = opt->socket_id;
dev_config.nb_models = t->dev_info.max_models;
- dev_config.nb_queue_pairs = t->dev_info.max_queue_pairs;
+ dev_config.nb_queue_pairs = opt->queue_pairs;
ret = rte_ml_dev_configure(opt->dev_id, &dev_config);
if (ret != 0) {
ml_err("Failed to configure ml device, dev_id = %d\n", opt->dev_id);
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
index 252d77616c..d680d68f3d 100644
--- a/app/test-mldev/test_inference_common.c
+++ b/app/test-mldev/test_inference_common.c
@@ -72,7 +72,7 @@ ml_enqueue_single(void *arg)
req->fid = fid;
enqueue_req:
- burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, 0, &op, 1);
+ burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, args->qp_id, &op, 1);
if (burst_enq == 0)
goto enqueue_req;
@@ -109,7 +109,7 @@ ml_dequeue_single(void *arg)
return 0;
dequeue_req:
- burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, 0, &op, 1);
+ burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, args->qp_id, &op, 1);
if (likely(burst_deq == 1)) {
total_deq += burst_deq;
@@ -188,7 +188,8 @@ ml_enqueue_burst(void *arg)
pending = ops_count;
enqueue_reqs:
- burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, 0, &args->enq_ops[idx], pending);
+ burst_enq =
+ rte_ml_enqueue_burst(t->cmn.opt->dev_id, args->qp_id, &args->enq_ops[idx], pending);
pending = pending - burst_enq;
if (pending > 0) {
@@ -229,8 +230,8 @@ ml_dequeue_burst(void *arg)
return 0;
dequeue_burst:
- burst_deq =
- rte_ml_dequeue_burst(t->cmn.opt->dev_id, 0, args->deq_ops, t->cmn.opt->burst_size);
+ burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, args->qp_id, args->deq_ops,
+ t->cmn.opt->burst_size);
if (likely(burst_deq > 0)) {
total_deq += burst_deq;
@@ -263,6 +264,19 @@ test_inference_cap_check(struct ml_options *opt)
return false;
rte_ml_dev_info_get(opt->dev_id, &dev_info);
+
+ if (opt->queue_pairs > dev_info.max_queue_pairs) {
+ ml_err("Insufficient capabilities: queue_pairs = %u, max_queue_pairs = %u",
+ opt->queue_pairs, dev_info.max_queue_pairs);
+ return false;
+ }
+
+ if (opt->queue_size > dev_info.max_desc) {
+ ml_err("Insufficient capabilities: queue_size = %u, max_desc = %u", opt->queue_size,
+ dev_info.max_desc);
+ return false;
+ }
+
if (opt->nb_filelist > dev_info.max_models) {
ml_err("Insufficient capabilities: Filelist count exceeded device limit, count = %u (max limit = %u)",
opt->nb_filelist, dev_info.max_models);
@@ -314,10 +328,21 @@ test_inference_opt_check(struct ml_options *opt)
return -EINVAL;
}
+ if (opt->queue_pairs == 0) {
+ ml_err("Invalid option, queue_pairs = %u\n", opt->queue_pairs);
+ return -EINVAL;
+ }
+
+ if (opt->queue_size == 0) {
+ ml_err("Invalid option, queue_size = %u\n", opt->queue_size);
+ return -EINVAL;
+ }
+
/* check number of available lcores. */
- if (rte_lcore_count() < 3) {
+ if (rte_lcore_count() < (uint32_t)(opt->queue_pairs * 2 + 1)) {
ml_err("Insufficient lcores = %u\n", rte_lcore_count());
- ml_err("Minimum lcores required to create %u queue-pairs = %u\n", 1, 3);
+ ml_err("Minimum lcores required to create %u queue-pairs = %u\n", opt->queue_pairs,
+ (opt->queue_pairs * 2 + 1));
return -EINVAL;
}
@@ -335,6 +360,8 @@ test_inference_opt_dump(struct ml_options *opt)
/* dump test opts */
ml_dump("repetitions", "%" PRIu64, opt->repetitions);
ml_dump("burst_size", "%u", opt->burst_size);
+ ml_dump("queue_pairs", "%u", opt->queue_pairs);
+ ml_dump("queue_size", "%u", opt->queue_size);
ml_dump_begin("filelist");
for (i = 0; i < opt->nb_filelist; i++) {
@@ -425,23 +452,31 @@ ml_inference_mldev_setup(struct ml_test *test, struct ml_options *opt)
{
struct rte_ml_dev_qp_conf qp_conf;
struct test_inference *t;
+ uint16_t qp_id;
int ret;
t = ml_test_priv(test);
+ RTE_SET_USED(t);
+
ret = ml_test_device_configure(test, opt);
if (ret != 0)
return ret;
/* setup queue pairs */
- qp_conf.nb_desc = t->cmn.dev_info.max_desc;
+ qp_conf.nb_desc = opt->queue_size;
qp_conf.cb = NULL;
- ret = rte_ml_dev_queue_pair_setup(opt->dev_id, 0, &qp_conf, opt->socket_id);
- if (ret != 0) {
- ml_err("Failed to setup ml device queue-pair, dev_id = %d, qp_id = %u\n",
- opt->dev_id, 0);
- goto error;
+ for (qp_id = 0; qp_id < opt->queue_pairs; qp_id++) {
+ qp_conf.nb_desc = opt->queue_size;
+ qp_conf.cb = NULL;
+
+ ret = rte_ml_dev_queue_pair_setup(opt->dev_id, qp_id, &qp_conf, opt->socket_id);
+ if (ret != 0) {
+ ml_err("Failed to setup ml device queue-pair, dev_id = %d, qp_id = %u\n",
+ opt->dev_id, qp_id);
+ return ret;
+ }
}
ret = ml_test_device_start(test, opt);
@@ -697,14 +732,28 @@ ml_inference_launch_cores(struct ml_test *test, struct ml_options *opt, int16_t
{
struct test_inference *t = ml_test_priv(test);
uint32_t lcore_id;
+ uint32_t nb_reqs;
uint32_t id = 0;
+ uint32_t qp_id;
+
+ nb_reqs = opt->repetitions / opt->queue_pairs;
RTE_LCORE_FOREACH_WORKER(lcore_id)
{
- if (id == 2)
+ if (id >= opt->queue_pairs * 2)
break;
- t->args[lcore_id].nb_reqs = opt->repetitions;
+ qp_id = id / 2;
+ t->args[lcore_id].qp_id = qp_id;
+ t->args[lcore_id].nb_reqs = nb_reqs;
+ if (qp_id == 0)
+ t->args[lcore_id].nb_reqs += opt->repetitions - nb_reqs * opt->queue_pairs;
+
+ if (t->args[lcore_id].nb_reqs == 0) {
+ id++;
+ break;
+ }
+
t->args[lcore_id].start_fid = start_fid;
t->args[lcore_id].end_fid = end_fid;
diff --git a/app/test-mldev/test_inference_common.h b/app/test-mldev/test_inference_common.h
index 75d588308b..1bac2dcfa0 100644
--- a/app/test-mldev/test_inference_common.h
+++ b/app/test-mldev/test_inference_common.h
@@ -27,6 +27,7 @@ struct ml_core_args {
uint64_t nb_reqs;
int16_t start_fid;
int16_t end_fid;
+ uint32_t qp_id;
struct rte_ml_op **enq_ops;
struct rte_ml_op **deq_ops;
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v3 09/12] app/mldev: enable support for inference batches
2022-12-08 19:29 ` [PATCH v3 01/12] app/mldev: implement test framework for mldev Srikanth Yalavarthi
` (6 preceding siblings ...)
2022-12-08 19:29 ` [PATCH v3 08/12] app/mldev: enable support for queue pairs and size Srikanth Yalavarthi
@ 2022-12-08 19:29 ` Srikanth Yalavarthi
2022-12-08 19:29 ` [PATCH v3 10/12] app/mldev: enable support for inference validation Srikanth Yalavarthi
` (2 subsequent siblings)
10 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2022-12-08 19:29 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu
Enabled support to execute multiple batches of inferences
per each enqueue request. Input and reference for the test
should be appropriately provided for multi-batch run. Number
of batches can be specified through "--batches" option.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/ml_options.c | 15 ++++++++++++---
app/test-mldev/ml_options.h | 2 ++
app/test-mldev/test_inference_common.c | 22 +++++++++++++---------
app/test-mldev/test_model_common.c | 6 ++++++
app/test-mldev/test_model_common.h | 1 +
5 files changed, 34 insertions(+), 12 deletions(-)
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index d5182a1701..331ec1704c 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -33,6 +33,7 @@ ml_options_default(struct ml_options *opt)
opt->burst_size = 1;
opt->queue_pairs = 1;
opt->queue_size = 1;
+ opt->batches = 0;
opt->debug = false;
}
@@ -176,6 +177,12 @@ ml_parse_queue_size(struct ml_options *opt, const char *arg)
return parser_read_uint16(&opt->queue_size, arg);
}
+static int
+ml_parse_batches(struct ml_options *opt, const char *arg)
+{
+ return parser_read_uint16(&opt->batches, arg);
+}
+
static void
ml_dump_test_options(const char *testname)
{
@@ -196,7 +203,8 @@ ml_dump_test_options(const char *testname)
"\t\t--repetitions : number of inference repetitions\n"
"\t\t--burst_size : inference burst size\n"
"\t\t--queue_pairs : number of queue pairs to create\n"
- "\t\t--queue_size : size fo queue-pair\n");
+ "\t\t--queue_size : size fo queue-pair\n"
+ "\t\t--batches : number of batches of input\n");
printf("\n");
}
}
@@ -220,7 +228,8 @@ static struct option lgopts[] = {
{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
{ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
{ML_BURST_SIZE, 1, 0, 0}, {ML_QUEUE_PAIRS, 1, 0, 0}, {ML_QUEUE_SIZE, 1, 0, 0},
- {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
+ {ML_BATCHES, 1, 0, 0}, {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0},
+ {NULL, 0, 0, 0}};
static int
ml_opts_parse_long(int opt_idx, struct ml_options *opt)
@@ -232,7 +241,7 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
{ML_SOCKET_ID, ml_parse_socket_id}, {ML_MODELS, ml_parse_models},
{ML_FILELIST, ml_parse_filelist}, {ML_REPETITIONS, ml_parse_repetitions},
{ML_BURST_SIZE, ml_parse_burst_size}, {ML_QUEUE_PAIRS, ml_parse_queue_pairs},
- {ML_QUEUE_SIZE, ml_parse_queue_size},
+ {ML_QUEUE_SIZE, ml_parse_queue_size}, {ML_BATCHES, ml_parse_batches},
};
for (i = 0; i < RTE_DIM(parsermap); i++) {
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index 6bfef1b979..d23e842895 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -22,6 +22,7 @@
#define ML_BURST_SIZE ("burst_size")
#define ML_QUEUE_PAIRS ("queue_pairs")
#define ML_QUEUE_SIZE ("queue_size")
+#define ML_BATCHES ("batches")
#define ML_DEBUG ("debug")
#define ML_HELP ("help")
@@ -41,6 +42,7 @@ struct ml_options {
uint16_t burst_size;
uint16_t queue_pairs;
uint16_t queue_size;
+ uint16_t batches;
bool debug;
};
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
index d680d68f3d..4e29f6c7eb 100644
--- a/app/test-mldev/test_inference_common.c
+++ b/app/test-mldev/test_inference_common.c
@@ -56,7 +56,7 @@ ml_enqueue_single(void *arg)
goto retry;
op->model_id = t->model[fid].id;
- op->nb_batches = t->model[fid].info.batch_size;
+ op->nb_batches = t->model[fid].nb_batches;
op->mempool = t->op_pool;
op->input.addr = req->input;
@@ -168,7 +168,7 @@ ml_enqueue_burst(void *arg)
for (i = 0; i < ops_count; i++) {
args->enq_ops[i]->model_id = t->model[fid].id;
- args->enq_ops[i]->nb_batches = t->model[fid].info.batch_size;
+ args->enq_ops[i]->nb_batches = t->model[fid].nb_batches;
args->enq_ops[i]->mempool = t->op_pool;
args->enq_ops[i]->input.addr = args->reqs[i]->input;
@@ -363,6 +363,11 @@ test_inference_opt_dump(struct ml_options *opt)
ml_dump("queue_pairs", "%u", opt->queue_pairs);
ml_dump("queue_size", "%u", opt->queue_size);
+ if (opt->batches == 0)
+ ml_dump("batches", "%u (default)", opt->batches);
+ else
+ ml_dump("batches", "%u", opt->batches);
+
ml_dump_begin("filelist");
for (i = 0; i < opt->nb_filelist; i++) {
ml_dump_list("model", i, opt->filelist[i].model);
@@ -531,8 +536,8 @@ ml_request_initialize(struct rte_mempool *mp, void *opaque, void *obj, unsigned
req->niters = 0;
/* quantize data */
- rte_ml_io_quantize(t->cmn.opt->dev_id, t->model[t->fid].id,
- t->model[t->fid].info.batch_size, t->model[t->fid].input, req->input);
+ rte_ml_io_quantize(t->cmn.opt->dev_id, t->model[t->fid].id, t->model[t->fid].nb_batches,
+ t->model[t->fid].input, req->input);
}
int
@@ -550,7 +555,7 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, int16_t f
int ret;
/* get input buffer size */
- ret = rte_ml_io_input_size_get(opt->dev_id, t->model[fid].id, t->model[fid].info.batch_size,
+ ret = rte_ml_io_input_size_get(opt->dev_id, t->model[fid].id, t->model[fid].nb_batches,
&t->model[fid].inp_qsize, &t->model[fid].inp_dsize);
if (ret != 0) {
ml_err("Failed to get input size, model : %s\n", opt->filelist[fid].model);
@@ -558,9 +563,8 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, int16_t f
}
/* get output buffer size */
- ret = rte_ml_io_output_size_get(opt->dev_id, t->model[fid].id,
- t->model[fid].info.batch_size, &t->model[fid].out_qsize,
- &t->model[fid].out_dsize);
+ ret = rte_ml_io_output_size_get(opt->dev_id, t->model[fid].id, t->model[fid].nb_batches,
+ &t->model[fid].out_qsize, &t->model[fid].out_dsize);
if (ret != 0) {
ml_err("Failed to get input size, model : %s\n", opt->filelist[fid].model);
return ret;
@@ -705,7 +709,7 @@ ml_request_finish(struct rte_mempool *mp, void *opaque, void *obj, unsigned int
return;
t->nb_used++;
- rte_ml_io_dequantize(t->cmn.opt->dev_id, model->id, t->model[req->fid].info.batch_size,
+ rte_ml_io_dequantize(t->cmn.opt->dev_id, model->id, t->model[req->fid].nb_batches,
req->output, model->output);
}
diff --git a/app/test-mldev/test_model_common.c b/app/test-mldev/test_model_common.c
index 5368be17fe..51260c0789 100644
--- a/app/test-mldev/test_model_common.c
+++ b/app/test-mldev/test_model_common.c
@@ -75,6 +75,12 @@ ml_model_load(struct ml_test *test, struct ml_options *opt, struct ml_model *mod
return ret;
}
+ /* Update number of batches */
+ if (opt->batches == 0)
+ model->nb_batches = model->info.batch_size;
+ else
+ model->nb_batches = opt->batches;
+
model->state = MODEL_LOADED;
return 0;
diff --git a/app/test-mldev/test_model_common.h b/app/test-mldev/test_model_common.h
index c45ae80853..dfbf568f0b 100644
--- a/app/test-mldev/test_model_common.h
+++ b/app/test-mldev/test_model_common.h
@@ -33,6 +33,7 @@ struct ml_model {
uint8_t *output;
struct rte_mempool *io_pool;
+ uint32_t nb_batches;
};
int ml_model_load(struct ml_test *test, struct ml_options *opt, struct ml_model *model,
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v3 10/12] app/mldev: enable support for inference validation
2022-12-08 19:29 ` [PATCH v3 01/12] app/mldev: implement test framework for mldev Srikanth Yalavarthi
` (7 preceding siblings ...)
2022-12-08 19:29 ` [PATCH v3 09/12] app/mldev: enable support for inference batches Srikanth Yalavarthi
@ 2022-12-08 19:29 ` Srikanth Yalavarthi
2022-12-08 19:29 ` [PATCH v3 11/12] app/mldev: enable reporting stats in mldev app Srikanth Yalavarthi
2022-12-08 19:29 ` [PATCH v3 12/12] app/mldev: add documentation for mldev test cases Srikanth Yalavarthi
10 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2022-12-08 19:29 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu
Enabled support to validate inference output with reference
output provided by the user. Validation would be successful
only when the inference outputs are within the 'tolerance'
specified through command line option "--tolerance".
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/meson.build | 2 +-
app/test-mldev/ml_options.c | 36 +++-
app/test-mldev/ml_options.h | 3 +
app/test-mldev/test_inference_common.c | 218 ++++++++++++++++++++++++-
app/test-mldev/test_inference_common.h | 1 +
app/test-mldev/test_model_common.h | 1 +
6 files changed, 250 insertions(+), 11 deletions(-)
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
index 41d22fb22c..15db534dc2 100644
--- a/app/test-mldev/meson.build
+++ b/app/test-mldev/meson.build
@@ -21,4 +21,4 @@ sources = files(
'test_inference_interleave.c',
)
-deps += ['mldev']
+deps += ['mldev', 'hash']
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 331ec1704c..092303903f 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -5,6 +5,7 @@
#include <errno.h>
#include <getopt.h>
#include <linux/limits.h>
+#include <math.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
@@ -34,6 +35,7 @@ ml_options_default(struct ml_options *opt)
opt->queue_pairs = 1;
opt->queue_size = 1;
opt->batches = 0;
+ opt->tolerance = 0.0;
opt->debug = false;
}
@@ -139,6 +141,13 @@ ml_parse_filelist(struct ml_options *opt, const char *arg)
}
strlcpy(opt->filelist[opt->nb_filelist].output, token, PATH_MAX);
+ /* reference - optional */
+ token = strtok(NULL, delim);
+ if (token != NULL)
+ strlcpy(opt->filelist[opt->nb_filelist].reference, token, PATH_MAX);
+ else
+ memset(opt->filelist[opt->nb_filelist].reference, 0, PATH_MAX);
+
opt->nb_filelist++;
if (opt->nb_filelist == 0) {
@@ -183,6 +192,14 @@ ml_parse_batches(struct ml_options *opt, const char *arg)
return parser_read_uint16(&opt->batches, arg);
}
+static int
+ml_parse_tolerance(struct ml_options *opt, const char *arg)
+{
+ opt->tolerance = fabs(atof(arg));
+
+ return 0;
+}
+
static void
ml_dump_test_options(const char *testname)
{
@@ -199,12 +216,13 @@ ml_dump_test_options(const char *testname)
if ((strcmp(testname, "inference_ordered") == 0) ||
(strcmp(testname, "inference_interleave") == 0)) {
- printf("\t\t--filelist : comma separated list of model, input and output\n"
+ printf("\t\t--filelist : comma separated list of model, input, output and reference\n"
"\t\t--repetitions : number of inference repetitions\n"
"\t\t--burst_size : inference burst size\n"
"\t\t--queue_pairs : number of queue pairs to create\n"
"\t\t--queue_size : size fo queue-pair\n"
- "\t\t--batches : number of batches of input\n");
+ "\t\t--batches : number of batches of input\n"
+ "\t\t--tolerance : maximum tolerance (%%) for output validation\n");
printf("\n");
}
}
@@ -224,12 +242,13 @@ print_usage(char *program)
ml_test_dump_names(ml_dump_test_options);
}
-static struct option lgopts[] = {
- {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
- {ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
- {ML_BURST_SIZE, 1, 0, 0}, {ML_QUEUE_PAIRS, 1, 0, 0}, {ML_QUEUE_SIZE, 1, 0, 0},
- {ML_BATCHES, 1, 0, 0}, {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0},
- {NULL, 0, 0, 0}};
+static struct option lgopts[] = {{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0},
+ {ML_SOCKET_ID, 1, 0, 0}, {ML_MODELS, 1, 0, 0},
+ {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
+ {ML_BURST_SIZE, 1, 0, 0}, {ML_QUEUE_PAIRS, 1, 0, 0},
+ {ML_QUEUE_SIZE, 1, 0, 0}, {ML_BATCHES, 1, 0, 0},
+ {ML_TOLERANCE, 1, 0, 0}, {ML_DEBUG, 0, 0, 0},
+ {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
static int
ml_opts_parse_long(int opt_idx, struct ml_options *opt)
@@ -242,6 +261,7 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
{ML_FILELIST, ml_parse_filelist}, {ML_REPETITIONS, ml_parse_repetitions},
{ML_BURST_SIZE, ml_parse_burst_size}, {ML_QUEUE_PAIRS, ml_parse_queue_pairs},
{ML_QUEUE_SIZE, ml_parse_queue_size}, {ML_BATCHES, ml_parse_batches},
+ {ML_TOLERANCE, ml_parse_tolerance},
};
for (i = 0; i < RTE_DIM(parsermap); i++) {
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index d23e842895..79ac54de98 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -23,6 +23,7 @@
#define ML_QUEUE_PAIRS ("queue_pairs")
#define ML_QUEUE_SIZE ("queue_size")
#define ML_BATCHES ("batches")
+#define ML_TOLERANCE ("tolerance")
#define ML_DEBUG ("debug")
#define ML_HELP ("help")
@@ -30,6 +31,7 @@ struct ml_filelist {
char model[PATH_MAX];
char input[PATH_MAX];
char output[PATH_MAX];
+ char reference[PATH_MAX];
};
struct ml_options {
@@ -43,6 +45,7 @@ struct ml_options {
uint16_t queue_pairs;
uint16_t queue_size;
uint16_t batches;
+ float tolerance;
bool debug;
};
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
index 4e29f6c7eb..008cee1023 100644
--- a/app/test-mldev/test_inference_common.c
+++ b/app/test-mldev/test_inference_common.c
@@ -3,12 +3,15 @@
*/
#include <errno.h>
+#include <linux/limits.h>
#include <stdbool.h>
#include <stdint.h>
#include <stdio.h>
+#include <string.h>
#include <unistd.h>
#include <rte_common.h>
+#include <rte_hash_crc.h>
#include <rte_launch.h>
#include <rte_lcore.h>
#include <rte_malloc.h>
@@ -21,6 +24,27 @@
#include "test_common.h"
#include "test_inference_common.h"
+#define ML_TEST_READ_TYPE(buffer, type) (*((type *)buffer))
+
+#define ML_TEST_CHECK_OUTPUT(output, reference, tolerance) \
+ (((float)output - (float)reference) <= (((float)reference * tolerance) / 100.0))
+
+#define ML_OPEN_WRITE_GET_ERR(name, buffer, size, err) \
+ do { \
+ FILE *fp = fopen(name, "w+"); \
+ if (fp == NULL) { \
+ ml_err("Unable to create file: %s, error: %s", name, strerror(errno)); \
+ err = true; \
+ } else { \
+ if (fwrite(buffer, 1, size, fp) != size) { \
+ ml_err("Error writing output, file: %s, error: %s", name, \
+ strerror(errno)); \
+ err = true; \
+ } \
+ fclose(fp); \
+ } \
+ } while (0)
+
/* Enqueue inference requests with burst size equal to 1 */
static int
ml_enqueue_single(void *arg)
@@ -362,6 +386,7 @@ test_inference_opt_dump(struct ml_options *opt)
ml_dump("burst_size", "%u", opt->burst_size);
ml_dump("queue_pairs", "%u", opt->queue_pairs);
ml_dump("queue_size", "%u", opt->queue_size);
+ ml_dump("tolerance", "%-7.3f", opt->tolerance);
if (opt->batches == 0)
ml_dump("batches", "%u (default)", opt->batches);
@@ -373,6 +398,8 @@ test_inference_opt_dump(struct ml_options *opt)
ml_dump_list("model", i, opt->filelist[i].model);
ml_dump_list("input", i, opt->filelist[i].input);
ml_dump_list("output", i, opt->filelist[i].output);
+ if (strcmp(opt->filelist[i].reference, "\0") != 0)
+ ml_dump_list("reference", i, opt->filelist[i].reference);
}
ml_dump_end;
}
@@ -397,6 +424,7 @@ test_inference_setup(struct ml_test *test, struct ml_options *opt)
t = ml_test_priv(test);
t->nb_used = 0;
+ t->nb_valid = 0;
t->cmn.result = ML_TEST_FAILED;
t->cmn.opt = opt;
@@ -572,6 +600,9 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, int16_t f
/* allocate buffer for user data */
mz_size = t->model[fid].inp_dsize + t->model[fid].out_dsize;
+ if (strcmp(opt->filelist[fid].reference, "\0") != 0)
+ mz_size += t->model[fid].out_dsize;
+
sprintf(mz_name, "ml_user_data_%d", fid);
mz = rte_memzone_reserve(mz_name, mz_size, opt->socket_id, 0);
if (mz == NULL) {
@@ -582,6 +613,10 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, int16_t f
t->model[fid].input = mz->addr;
t->model[fid].output = t->model[fid].input + t->model[fid].inp_dsize;
+ if (strcmp(opt->filelist[fid].reference, "\0") != 0)
+ t->model[fid].reference = t->model[fid].output + t->model[fid].out_dsize;
+ else
+ t->model[fid].reference = NULL;
/* load input file */
fp = fopen(opt->filelist[fid].input, "r");
@@ -610,6 +645,27 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, int16_t f
}
fclose(fp);
+ /* load reference file */
+ if (t->model[fid].reference != NULL) {
+ fp = fopen(opt->filelist[fid].reference, "r");
+ if (fp == NULL) {
+ ml_err("Failed to open reference file : %s\n",
+ opt->filelist[fid].reference);
+ ret = -errno;
+ goto error;
+ }
+
+ if (fread(t->model[fid].reference, 1, t->model[fid].out_dsize, fp) !=
+ t->model[fid].out_dsize) {
+ ml_err("Failed to read reference file : %s\n",
+ opt->filelist[fid].reference);
+ ret = -errno;
+ fclose(fp);
+ goto error;
+ }
+ fclose(fp);
+ }
+
/* create mempool for quantized input and output buffers. ml_request_initialize is
* used as a callback for object creation.
*/
@@ -694,6 +750,121 @@ ml_inference_mem_destroy(struct ml_test *test, struct ml_options *opt)
rte_mempool_free(t->op_pool);
}
+static bool
+ml_inference_validation(struct ml_test *test, struct ml_request *req)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)test);
+ struct ml_model *model;
+ uint32_t nb_elements;
+ uint8_t *reference;
+ uint8_t *output;
+ bool match;
+ uint32_t i;
+ uint32_t j;
+
+ model = &t->model[req->fid];
+
+ /* compare crc when tolerance is 0 */
+ if (t->cmn.opt->tolerance == 0.0) {
+ match = (rte_hash_crc(model->output, model->out_dsize, 0) ==
+ rte_hash_crc(model->reference, model->out_dsize, 0));
+ } else {
+ output = model->output;
+ reference = model->reference;
+
+ i = 0;
+next_output:
+ nb_elements =
+ model->info.output_info[i].shape.w * model->info.output_info[i].shape.x *
+ model->info.output_info[i].shape.y * model->info.output_info[i].shape.z;
+ j = 0;
+next_element:
+ match = false;
+ switch (model->info.output_info[i].dtype) {
+ case RTE_ML_IO_TYPE_INT8:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, int8_t),
+ ML_TEST_READ_TYPE(reference, int8_t),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(int8_t);
+ reference += sizeof(int8_t);
+ break;
+ case RTE_ML_IO_TYPE_UINT8:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, uint8_t),
+ ML_TEST_READ_TYPE(reference, uint8_t),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(float);
+ reference += sizeof(float);
+ break;
+ case RTE_ML_IO_TYPE_INT16:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, int16_t),
+ ML_TEST_READ_TYPE(reference, int16_t),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(int16_t);
+ reference += sizeof(int16_t);
+ break;
+ case RTE_ML_IO_TYPE_UINT16:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, uint16_t),
+ ML_TEST_READ_TYPE(reference, uint16_t),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(uint16_t);
+ reference += sizeof(uint16_t);
+ break;
+ case RTE_ML_IO_TYPE_INT32:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, int32_t),
+ ML_TEST_READ_TYPE(reference, int32_t),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(int32_t);
+ reference += sizeof(int32_t);
+ break;
+ case RTE_ML_IO_TYPE_UINT32:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, uint32_t),
+ ML_TEST_READ_TYPE(reference, uint32_t),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(uint32_t);
+ reference += sizeof(uint32_t);
+ break;
+ case RTE_ML_IO_TYPE_FP32:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, float),
+ ML_TEST_READ_TYPE(reference, float),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(float);
+ reference += sizeof(float);
+ break;
+ default: /* other types, fp8, fp16, bfloat16 */
+ match = true;
+ }
+
+ if (!match)
+ goto done;
+ j++;
+ if (j < nb_elements)
+ goto next_element;
+
+ i++;
+ if (i < model->info.nb_outputs)
+ goto next_output;
+ }
+done:
+ if (match)
+ t->nb_valid++;
+
+ return match;
+}
+
/* Callback for mempool object iteration. This call would dequantize output data. */
static void
ml_request_finish(struct rte_mempool *mp, void *opaque, void *obj, unsigned int obj_idx)
@@ -701,9 +872,10 @@ ml_request_finish(struct rte_mempool *mp, void *opaque, void *obj, unsigned int
struct test_inference *t = ml_test_priv((struct ml_test *)opaque);
struct ml_request *req = (struct ml_request *)obj;
struct ml_model *model = &t->model[req->fid];
+ char str[PATH_MAX];
+ bool error = false;
RTE_SET_USED(mp);
- RTE_SET_USED(obj_idx);
if (req->niters == 0)
return;
@@ -711,6 +883,48 @@ ml_request_finish(struct rte_mempool *mp, void *opaque, void *obj, unsigned int
t->nb_used++;
rte_ml_io_dequantize(t->cmn.opt->dev_id, model->id, t->model[req->fid].nb_batches,
req->output, model->output);
+
+ if (model->reference == NULL) {
+ t->nb_valid++;
+ goto dump_output_pass;
+ }
+
+ if (!ml_inference_validation(opaque, req))
+ goto dump_output_fail;
+ else
+ goto dump_output_pass;
+
+dump_output_pass:
+ if (obj_idx == 0) {
+ /* write quantized output */
+ snprintf(str, PATH_MAX, "%s.q", t->cmn.opt->filelist[req->fid].output);
+ ML_OPEN_WRITE_GET_ERR(str, req->output, model->out_qsize, error);
+ if (error)
+ return;
+
+ /* write dequantized output */
+ snprintf(str, PATH_MAX, "%s", t->cmn.opt->filelist[req->fid].output);
+ ML_OPEN_WRITE_GET_ERR(str, model->output, model->out_dsize, error);
+ if (error)
+ return;
+ }
+
+ return;
+
+dump_output_fail:
+ if (t->cmn.opt->debug) {
+ /* dump quantized output buffer */
+ snprintf(str, PATH_MAX, "%s.q.%d", t->cmn.opt->filelist[req->fid].output, obj_idx);
+ ML_OPEN_WRITE_GET_ERR(str, req->output, model->out_qsize, error);
+ if (error)
+ return;
+
+ /* dump dequantized output buffer */
+ snprintf(str, PATH_MAX, "%s.%d", t->cmn.opt->filelist[req->fid].output, obj_idx);
+ ML_OPEN_WRITE_GET_ERR(str, model->output, model->out_dsize, error);
+ if (error)
+ return;
+ }
}
int
@@ -722,7 +936,7 @@ ml_inference_result(struct ml_test *test, struct ml_options *opt, int16_t fid)
rte_mempool_obj_iter(t->model[fid].io_pool, ml_request_finish, test);
- if (t->nb_used > 0)
+ if (t->nb_used == t->nb_valid)
t->cmn.result = ML_TEST_SUCCESS;
else
t->cmn.result = ML_TEST_FAILED;
diff --git a/app/test-mldev/test_inference_common.h b/app/test-mldev/test_inference_common.h
index 1bac2dcfa0..3f2b042360 100644
--- a/app/test-mldev/test_inference_common.h
+++ b/app/test-mldev/test_inference_common.h
@@ -43,6 +43,7 @@ struct test_inference {
struct rte_mempool *op_pool;
uint64_t nb_used;
+ uint64_t nb_valid;
int16_t fid;
int (*enqueue)(void *arg);
diff --git a/app/test-mldev/test_model_common.h b/app/test-mldev/test_model_common.h
index dfbf568f0b..ce12cbfecc 100644
--- a/app/test-mldev/test_model_common.h
+++ b/app/test-mldev/test_model_common.h
@@ -31,6 +31,7 @@ struct ml_model {
uint8_t *input;
uint8_t *output;
+ uint8_t *reference;
struct rte_mempool *io_pool;
uint32_t nb_batches;
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v3 11/12] app/mldev: enable reporting stats in mldev app
2022-12-08 19:29 ` [PATCH v3 01/12] app/mldev: implement test framework for mldev Srikanth Yalavarthi
` (8 preceding siblings ...)
2022-12-08 19:29 ` [PATCH v3 10/12] app/mldev: enable support for inference validation Srikanth Yalavarthi
@ 2022-12-08 19:29 ` Srikanth Yalavarthi
2023-02-03 9:49 ` Anup Prabhu
2022-12-08 19:29 ` [PATCH v3 12/12] app/mldev: add documentation for mldev test cases Srikanth Yalavarthi
10 siblings, 1 reply; 122+ messages in thread
From: Srikanth Yalavarthi @ 2022-12-08 19:29 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu
Enable reporting driver xstats and inference end-to-end
latency and throughput in mldev inference tests. Reporting
of stats can be enabled using "--stats" option.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
v3:
* Fixed issue in reporting end-to-end inference stats
v2:
* Fixed typos and formatting issue
app/test-mldev/ml_options.c | 22 ++--
app/test-mldev/ml_options.h | 2 +
app/test-mldev/test_inference_common.c | 139 +++++++++++++++++++++
app/test-mldev/test_inference_common.h | 8 ++
app/test-mldev/test_inference_interleave.c | 4 +
app/test-mldev/test_inference_ordered.c | 1 +
6 files changed, 168 insertions(+), 8 deletions(-)
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 092303903f..0e7877eed3 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -36,6 +36,7 @@ ml_options_default(struct ml_options *opt)
opt->queue_size = 1;
opt->batches = 0;
opt->tolerance = 0.0;
+ opt->stats = false;
opt->debug = false;
}
@@ -222,7 +223,8 @@ ml_dump_test_options(const char *testname)
"\t\t--queue_pairs : number of queue pairs to create\n"
"\t\t--queue_size : size fo queue-pair\n"
"\t\t--batches : number of batches of input\n"
- "\t\t--tolerance : maximum tolerance (%%) for output validation\n");
+ "\t\t--tolerance : maximum tolerance (%%) for output validation\n"
+ "\t\t--stats : enable reporting performance statistics\n");
printf("\n");
}
}
@@ -242,13 +244,12 @@ print_usage(char *program)
ml_test_dump_names(ml_dump_test_options);
}
-static struct option lgopts[] = {{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0},
- {ML_SOCKET_ID, 1, 0, 0}, {ML_MODELS, 1, 0, 0},
- {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
- {ML_BURST_SIZE, 1, 0, 0}, {ML_QUEUE_PAIRS, 1, 0, 0},
- {ML_QUEUE_SIZE, 1, 0, 0}, {ML_BATCHES, 1, 0, 0},
- {ML_TOLERANCE, 1, 0, 0}, {ML_DEBUG, 0, 0, 0},
- {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
+static struct option lgopts[] = {
+ {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
+ {ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
+ {ML_BURST_SIZE, 1, 0, 0}, {ML_QUEUE_PAIRS, 1, 0, 0}, {ML_QUEUE_SIZE, 1, 0, 0},
+ {ML_BATCHES, 1, 0, 0}, {ML_TOLERANCE, 1, 0, 0}, {ML_STATS, 0, 0, 0},
+ {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
static int
ml_opts_parse_long(int opt_idx, struct ml_options *opt)
@@ -283,6 +284,11 @@ ml_options_parse(struct ml_options *opt, int argc, char **argv)
while ((opts = getopt_long(argc, argv, "", lgopts, &opt_idx)) != EOF) {
switch (opts) {
case 0: /* parse long options */
+ if (!strcmp(lgopts[opt_idx].name, "stats")) {
+ opt->stats = true;
+ break;
+ }
+
if (!strcmp(lgopts[opt_idx].name, "debug")) {
opt->debug = true;
break;
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index 79ac54de98..a375ae6750 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -24,6 +24,7 @@
#define ML_QUEUE_SIZE ("queue_size")
#define ML_BATCHES ("batches")
#define ML_TOLERANCE ("tolerance")
+#define ML_STATS ("stats")
#define ML_DEBUG ("debug")
#define ML_HELP ("help")
@@ -46,6 +47,7 @@ struct ml_options {
uint16_t queue_size;
uint16_t batches;
float tolerance;
+ bool stats;
bool debug;
};
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
index 008cee1023..8d1fc55c2f 100644
--- a/app/test-mldev/test_inference_common.c
+++ b/app/test-mldev/test_inference_common.c
@@ -11,6 +11,7 @@
#include <unistd.h>
#include <rte_common.h>
+#include <rte_cycles.h>
#include <rte_hash_crc.h>
#include <rte_launch.h>
#include <rte_lcore.h>
@@ -45,6 +46,17 @@
} \
} while (0)
+static void
+print_line(uint16_t len)
+{
+ uint16_t i;
+
+ for (i = 0; i < len; i++)
+ printf("-");
+
+ printf("\n");
+}
+
/* Enqueue inference requests with burst size equal to 1 */
static int
ml_enqueue_single(void *arg)
@@ -54,6 +66,7 @@ ml_enqueue_single(void *arg)
struct rte_ml_op *op = NULL;
struct ml_core_args *args;
uint64_t model_enq = 0;
+ uint64_t start_cycle;
uint32_t burst_enq;
uint32_t lcore_id;
int16_t fid;
@@ -61,6 +74,7 @@ ml_enqueue_single(void *arg)
lcore_id = rte_lcore_id();
args = &t->args[lcore_id];
+ args->start_cycles = 0;
model_enq = 0;
if (args->nb_reqs == 0)
@@ -96,10 +110,12 @@ ml_enqueue_single(void *arg)
req->fid = fid;
enqueue_req:
+ start_cycle = rte_get_tsc_cycles();
burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, args->qp_id, &op, 1);
if (burst_enq == 0)
goto enqueue_req;
+ args->start_cycles += start_cycle;
fid++;
if (likely(fid <= args->end_fid))
goto next_model;
@@ -123,10 +139,12 @@ ml_dequeue_single(void *arg)
uint64_t total_deq = 0;
uint8_t nb_filelist;
uint32_t burst_deq;
+ uint64_t end_cycle;
uint32_t lcore_id;
lcore_id = rte_lcore_id();
args = &t->args[lcore_id];
+ args->end_cycles = 0;
nb_filelist = args->end_fid - args->start_fid + 1;
if (args->nb_reqs == 0)
@@ -134,9 +152,11 @@ ml_dequeue_single(void *arg)
dequeue_req:
burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, args->qp_id, &op, 1);
+ end_cycle = rte_get_tsc_cycles();
if (likely(burst_deq == 1)) {
total_deq += burst_deq;
+ args->end_cycles += end_cycle;
if (unlikely(op->status == RTE_ML_OP_STATUS_ERROR)) {
rte_ml_op_error_get(t->cmn.opt->dev_id, op, &error);
ml_err("error_code = 0x%" PRIx64 ", error_message = %s\n", error.errcode,
@@ -159,6 +179,7 @@ ml_enqueue_burst(void *arg)
{
struct test_inference *t = ml_test_priv((struct ml_test *)arg);
struct ml_core_args *args;
+ uint64_t start_cycle;
uint16_t ops_count;
uint64_t model_enq;
uint16_t burst_enq;
@@ -171,6 +192,7 @@ ml_enqueue_burst(void *arg)
lcore_id = rte_lcore_id();
args = &t->args[lcore_id];
+ args->start_cycles = 0;
model_enq = 0;
if (args->nb_reqs == 0)
@@ -212,8 +234,10 @@ ml_enqueue_burst(void *arg)
pending = ops_count;
enqueue_reqs:
+ start_cycle = rte_get_tsc_cycles();
burst_enq =
rte_ml_enqueue_burst(t->cmn.opt->dev_id, args->qp_id, &args->enq_ops[idx], pending);
+ args->start_cycles += burst_enq * start_cycle;
pending = pending - burst_enq;
if (pending > 0) {
@@ -243,11 +267,13 @@ ml_dequeue_burst(void *arg)
uint64_t total_deq = 0;
uint16_t burst_deq = 0;
uint8_t nb_filelist;
+ uint64_t end_cycle;
uint32_t lcore_id;
uint32_t i;
lcore_id = rte_lcore_id();
args = &t->args[lcore_id];
+ args->end_cycles = 0;
nb_filelist = args->end_fid - args->start_fid + 1;
if (args->nb_reqs == 0)
@@ -256,9 +282,11 @@ ml_dequeue_burst(void *arg)
dequeue_burst:
burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, args->qp_id, args->deq_ops,
t->cmn.opt->burst_size);
+ end_cycle = rte_get_tsc_cycles();
if (likely(burst_deq > 0)) {
total_deq += burst_deq;
+ args->end_cycles += burst_deq * end_cycle;
for (i = 0; i < burst_deq; i++) {
if (unlikely(args->deq_ops[i]->status == RTE_ML_OP_STATUS_ERROR)) {
@@ -387,6 +415,7 @@ test_inference_opt_dump(struct ml_options *opt)
ml_dump("queue_pairs", "%u", opt->queue_pairs);
ml_dump("queue_size", "%u", opt->queue_size);
ml_dump("tolerance", "%-7.3f", opt->tolerance);
+ ml_dump("stats", "%s", (opt->stats ? "true" : "false"));
if (opt->batches == 0)
ml_dump("batches", "%u (default)", opt->batches);
@@ -459,6 +488,11 @@ test_inference_setup(struct ml_test *test, struct ml_options *opt)
RTE_CACHE_LINE_SIZE, opt->socket_id);
}
+ for (i = 0; i < RTE_MAX_LCORE; i++) {
+ t->args[i].start_cycles = 0;
+ t->args[i].end_cycles = 0;
+ }
+
return 0;
error:
@@ -985,3 +1019,108 @@ ml_inference_launch_cores(struct ml_test *test, struct ml_options *opt, int16_t
return 0;
}
+
+int
+ml_inference_stats_get(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t = ml_test_priv(test);
+ uint64_t total_cycles = 0;
+ uint32_t nb_filelist;
+ uint64_t throughput;
+ uint64_t avg_e2e;
+ uint32_t qp_id;
+ uint64_t freq;
+ int ret;
+ int i;
+
+ if (!opt->stats)
+ return 0;
+
+ /* get xstats size */
+ t->xstats_size = rte_ml_dev_xstats_names_get(opt->dev_id, NULL, 0);
+ if (t->xstats_size >= 0) {
+ /* allocate for xstats_map and values */
+ t->xstats_map = rte_malloc(
+ "ml_xstats_map", t->xstats_size * sizeof(struct rte_ml_dev_xstats_map), 0);
+ if (t->xstats_map == NULL) {
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ t->xstats_values =
+ rte_malloc("ml_xstats_values", t->xstats_size * sizeof(uint64_t), 0);
+ if (t->xstats_values == NULL) {
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ ret = rte_ml_dev_xstats_names_get(opt->dev_id, t->xstats_map, t->xstats_size);
+ if (ret != t->xstats_size) {
+ printf("Unable to get xstats names, ret = %d\n", ret);
+ ret = -1;
+ goto error;
+ }
+
+ for (i = 0; i < t->xstats_size; i++)
+ rte_ml_dev_xstats_get(opt->dev_id, &t->xstats_map[i].id,
+ &t->xstats_values[i], 1);
+ }
+
+ /* print xstats*/
+ printf("\n");
+ print_line(80);
+ printf(" ML Device Extended Statistics\n");
+ print_line(80);
+ for (i = 0; i < t->xstats_size; i++)
+ printf(" %-64s = %" PRIu64 "\n", t->xstats_map[i].name, t->xstats_values[i]);
+ print_line(80);
+
+ /* release buffers */
+ if (t->xstats_map)
+ rte_free(t->xstats_map);
+
+ if (t->xstats_values)
+ rte_free(t->xstats_values);
+
+ /* print end-to-end stats */
+ freq = rte_get_tsc_hz();
+ for (qp_id = 0; qp_id < RTE_MAX_LCORE; qp_id++)
+ total_cycles += t->args[qp_id].end_cycles - t->args[qp_id].start_cycles;
+ avg_e2e = total_cycles / opt->repetitions;
+
+ if (freq == 0) {
+ avg_e2e = total_cycles / opt->repetitions;
+ printf(" %-64s = %" PRIu64 "\n", "Average End-to-End Latency (cycles)", avg_e2e);
+ } else {
+ avg_e2e = (total_cycles * NS_PER_S) / (opt->repetitions * freq);
+ printf(" %-64s = %" PRIu64 "\n", "Average End-to-End Latency (ns)", avg_e2e);
+ }
+
+ if (strcmp(opt->test_name, "inference_ordered") == 0)
+ nb_filelist = 1;
+ else
+ nb_filelist = t->cmn.opt->nb_filelist;
+
+ if (freq == 0) {
+ throughput = (nb_filelist * t->cmn.opt->repetitions * 1000000) / total_cycles;
+ printf(" %-64s = %" PRIu64 "\n", "Average Throughput (inferences / million cycles)",
+ throughput);
+ } else {
+ throughput = (nb_filelist * t->cmn.opt->repetitions * freq) / total_cycles;
+ printf(" %-64s = %" PRIu64 "\n", "Average Throughput (inferences / second)",
+ throughput);
+ }
+
+ print_line(80);
+
+ return 0;
+
+error:
+ if (t->xstats_map)
+ rte_free(t->xstats_map);
+
+ if (t->xstats_values)
+ rte_free(t->xstats_values);
+
+ return ret;
+}
diff --git a/app/test-mldev/test_inference_common.h b/app/test-mldev/test_inference_common.h
index 3f2b042360..bb2920cc30 100644
--- a/app/test-mldev/test_inference_common.h
+++ b/app/test-mldev/test_inference_common.h
@@ -32,6 +32,9 @@ struct ml_core_args {
struct rte_ml_op **enq_ops;
struct rte_ml_op **deq_ops;
struct ml_request **reqs;
+
+ uint64_t start_cycles;
+ uint64_t end_cycles;
};
struct test_inference {
@@ -50,6 +53,10 @@ struct test_inference {
int (*dequeue)(void *arg);
struct ml_core_args args[RTE_MAX_LCORE];
+
+ struct rte_ml_dev_xstats_map *xstats_map;
+ uint64_t *xstats_values;
+ int xstats_size;
} __rte_cache_aligned;
bool test_inference_cap_check(struct ml_options *opt);
@@ -67,5 +74,6 @@ void ml_inference_mem_destroy(struct ml_test *test, struct ml_options *opt);
int ml_inference_result(struct ml_test *test, struct ml_options *opt, int16_t fid);
int ml_inference_launch_cores(struct ml_test *test, struct ml_options *opt, int16_t start_fid,
int16_t end_fid);
+int ml_inference_stats_get(struct ml_test *test, struct ml_options *opt);
#endif /* _ML_TEST_INFERENCE_COMMON_ */
diff --git a/app/test-mldev/test_inference_interleave.c b/app/test-mldev/test_inference_interleave.c
index 74ad0c597f..d86838c3fa 100644
--- a/app/test-mldev/test_inference_interleave.c
+++ b/app/test-mldev/test_inference_interleave.c
@@ -60,7 +60,11 @@ test_inference_interleave_driver(struct ml_test *test, struct ml_options *opt)
goto error;
ml_inference_iomem_destroy(test, opt, fid);
+ }
+
+ ml_inference_stats_get(test, opt);
+ for (fid = 0; fid < opt->nb_filelist; fid++) {
ret = ml_model_stop(test, opt, &t->model[fid], fid);
if (ret != 0)
goto error;
diff --git a/app/test-mldev/test_inference_ordered.c b/app/test-mldev/test_inference_ordered.c
index 84e6bf9109..3826121a65 100644
--- a/app/test-mldev/test_inference_ordered.c
+++ b/app/test-mldev/test_inference_ordered.c
@@ -58,6 +58,7 @@ test_inference_ordered_driver(struct ml_test *test, struct ml_options *opt)
goto error;
ml_inference_iomem_destroy(test, opt, fid);
+ ml_inference_stats_get(test, opt);
/* stop model */
ret = ml_model_stop(test, opt, &t->model[fid], fid);
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v3 12/12] app/mldev: add documentation for mldev test cases
2022-12-08 19:29 ` [PATCH v3 01/12] app/mldev: implement test framework for mldev Srikanth Yalavarthi
` (9 preceding siblings ...)
2022-12-08 19:29 ` [PATCH v3 11/12] app/mldev: enable reporting stats in mldev app Srikanth Yalavarthi
@ 2022-12-08 19:29 ` Srikanth Yalavarthi
2023-02-02 12:39 ` Anup Prabhu
10 siblings, 1 reply; 122+ messages in thread
From: Srikanth Yalavarthi @ 2022-12-08 19:29 UTC (permalink / raw)
To: Thomas Monjalon, Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu
Added documentation specific to mldev test cases. Added details
about all test cases and option supported by individual tests.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
MAINTAINERS | 1 +
app/test-mldev/test_inference_common.c | 27 +-
.../tools/img/mldev_inference_interleave.svg | 667 ++++++++++++++++++
.../tools/img/mldev_inference_ordered.svg | 526 ++++++++++++++
.../tools/img/mldev_model_ops_subtest_a.svg | 418 +++++++++++
.../tools/img/mldev_model_ops_subtest_b.svg | 421 +++++++++++
.../tools/img/mldev_model_ops_subtest_c.svg | 364 ++++++++++
.../tools/img/mldev_model_ops_subtest_d.svg | 422 +++++++++++
doc/guides/tools/index.rst | 1 +
doc/guides/tools/testmldev.rst | 441 ++++++++++++
10 files changed, 3267 insertions(+), 21 deletions(-)
create mode 100644 doc/guides/tools/img/mldev_inference_interleave.svg
create mode 100644 doc/guides/tools/img/mldev_inference_ordered.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_a.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_b.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_c.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_d.svg
create mode 100644 doc/guides/tools/testmldev.rst
diff --git a/MAINTAINERS b/MAINTAINERS
index c76007a340..5fa276fafa 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -540,6 +540,7 @@ M: Srikanth Yalavarthi <syalavarthi@marvell.com>
F: lib/mldev/
F: app/test-mldev/
F: doc/guides/prog_guide/mldev.rst
+F: doc/guides/tools/testmldev.rst
Memory Pool Drivers
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
index 8d1fc55c2f..2c42ce4b1d 100644
--- a/app/test-mldev/test_inference_common.c
+++ b/app/test-mldev/test_inference_common.c
@@ -1029,7 +1029,6 @@ ml_inference_stats_get(struct ml_test *test, struct ml_options *opt)
uint64_t throughput;
uint64_t avg_e2e;
uint32_t qp_id;
- uint64_t freq;
int ret;
int i;
@@ -1083,34 +1082,20 @@ ml_inference_stats_get(struct ml_test *test, struct ml_options *opt)
rte_free(t->xstats_values);
/* print end-to-end stats */
- freq = rte_get_tsc_hz();
for (qp_id = 0; qp_id < RTE_MAX_LCORE; qp_id++)
total_cycles += t->args[qp_id].end_cycles - t->args[qp_id].start_cycles;
- avg_e2e = total_cycles / opt->repetitions;
- if (freq == 0) {
- avg_e2e = total_cycles / opt->repetitions;
- printf(" %-64s = %" PRIu64 "\n", "Average End-to-End Latency (cycles)", avg_e2e);
- } else {
- avg_e2e = (total_cycles * NS_PER_S) / (opt->repetitions * freq);
- printf(" %-64s = %" PRIu64 "\n", "Average End-to-End Latency (ns)", avg_e2e);
- }
+ avg_e2e = ((total_cycles / opt->repetitions) * NS_PER_S) / rte_get_tsc_hz();
+ printf(" %-64s = %" PRIu64 "\n", "Average End-to-End Latency (ns)", avg_e2e);
+ /* print inference throughput */
if (strcmp(opt->test_name, "inference_ordered") == 0)
nb_filelist = 1;
else
- nb_filelist = t->cmn.opt->nb_filelist;
-
- if (freq == 0) {
- throughput = (nb_filelist * t->cmn.opt->repetitions * 1000000) / total_cycles;
- printf(" %-64s = %" PRIu64 "\n", "Average Throughput (inferences / million cycles)",
- throughput);
- } else {
- throughput = (nb_filelist * t->cmn.opt->repetitions * freq) / total_cycles;
- printf(" %-64s = %" PRIu64 "\n", "Average Throughput (inferences / second)",
- throughput);
- }
+ nb_filelist = opt->nb_filelist;
+ throughput = (nb_filelist * NS_PER_S) / avg_e2e;
+ printf(" %-64s = %" PRIu64 "\n", "Average Throughput (inferences / second)", throughput);
print_line(80);
return 0;
diff --git a/doc/guides/tools/img/mldev_inference_interleave.svg b/doc/guides/tools/img/mldev_inference_interleave.svg
new file mode 100644
index 0000000000..517c53d294
--- /dev/null
+++ b/doc/guides/tools/img/mldev_inference_interleave.svg
@@ -0,0 +1,667 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="224mm"
+ height="150mm"
+ viewBox="0 0 224 150"
+ version="1.1"
+ id="svg5369"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="inference_interleave.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns:xlink="http://www.w3.org/1999/xlink"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg">
+ <sodipodi:namedview
+ id="namedview5371"
+ pagecolor="#ffffff"
+ bordercolor="#000000"
+ borderopacity="0.25"
+ inkscape:showpageshadow="2"
+ inkscape:pageopacity="0.0"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1"
+ inkscape:document-units="mm"
+ showgrid="false"
+ inkscape:zoom="0.74564394"
+ inkscape:cx="415.07747"
+ inkscape:cy="348.6919"
+ inkscape:window-width="1920"
+ inkscape:window-height="1017"
+ inkscape:window-x="1912"
+ inkscape:window-y="-8"
+ inkscape:window-maximized="1"
+ inkscape:current-layer="layer1" />
+ <defs
+ id="defs5366">
+ <linearGradient
+ inkscape:collect="always"
+ id="linearGradient19189">
+ <stop
+ style="stop-color:#ffeeaa;stop-opacity:1;"
+ offset="0"
+ id="stop19185" />
+ <stop
+ style="stop-color:#ffeeaa;stop-opacity:0;"
+ offset="1"
+ id="stop19187" />
+ </linearGradient>
+ <marker
+ style="overflow:visible"
+ id="TriangleStart"
+ refX="4"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="TriangleStart"
+ markerWidth="5.3244081"
+ markerHeight="6.155385"
+ viewBox="0 0 5.3244081 6.1553851"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.5)"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:context-stroke;stroke-width:1pt"
+ d="M 5.77,0 -2.88,5 V -5 Z"
+ id="path135" />
+ </marker>
+ <marker
+ style="overflow:visible"
+ id="RoundedArrow"
+ refX="5"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="RoundedArrow"
+ markerWidth="6.1347523"
+ markerHeight="5.9304948"
+ viewBox="0 0 6.1347524 5.9304951"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.7)"
+ d="m -0.21114562,-4.1055728 6.42229122,3.21114561 a 1,1 90 0 1 0,1.78885438 L -0.21114562,4.1055728 A 1.236068,1.236068 31.717474 0 1 -2,3 v -6 a 1.236068,1.236068 148.28253 0 1 1.78885438,-1.1055728 z"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:none"
+ id="path1367" />
+ </marker>
+ <linearGradient
+ inkscape:collect="always"
+ xlink:href="#linearGradient19189"
+ id="linearGradient19191"
+ x1="12.169352"
+ y1="105"
+ x2="284.83066"
+ y2="105"
+ gradientUnits="userSpaceOnUse"
+ gradientTransform="matrix(0.79055445,0,0,0.74078976,29.505892,28.991272)" />
+ </defs>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1">
+ <g
+ id="g1477"
+ transform="translate(-34.903236,-31.774189)">
+ <rect
+ style="fill:url(#linearGradient19191);fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.396267;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect16635"
+ width="215.18147"
+ height="139.60078"
+ x="39.3125"
+ y="36.973797"
+ ry="2.2354064" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-169.3954"
+ y="214.97237"
+ id="text5181"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan5179"
+ style="font-size:5.64444px;stroke-width:0.75"
+ x="-169.3954"
+ y="214.97237">test: inference_interleave</tspan></text>
+ <path
+ style="display:inline;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 138.05133,87.886263 17.45982,-10e-7"
+ id="path1912"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1724-0"
+ inkscape:connection-end="#rect1679" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 191.51115,87.886262 17.45981,10e-7"
+ id="path1914"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1679"
+ inkscape:connection-end="#rect1724" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 138.05133,109.78102 17.45982,-1e-5"
+ id="path1916"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1724-4-8"
+ inkscape:connection-end="#rect1679-4" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 191.51115,109.78101 17.45981,1e-5"
+ id="path1918"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1679-4"
+ inkscape:connection-end="#rect1724-4" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 138.05133,131.67576 17.45982,-1e-5"
+ id="path1920"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1724-6-7"
+ inkscape:connection-end="#rect1679-8" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 191.51115,131.67575 17.45981,1e-5"
+ id="path1922"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1679-8"
+ inkscape:connection-end="#rect1724-6" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.75;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;marker-start:url(#RoundedArrow);marker-end:url(#RoundedArrow)"
+ d="m 173.51116,60.08164 0,12.907336"
+ id="path1933"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1811"
+ inkscape:connection-end="#rect1924" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.368668;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.737336, 0.737336;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924"
+ width="46.97015"
+ height="73.58287"
+ x="150.02565"
+ y="72.988976"
+ ry="2.4685853" />
+ <text
+ xml:space="preserve"
+ style="font-size:6.35px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-84.44075"
+ y="173.5065"
+ id="text4097"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4095"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-84.44075"
+ y="173.5065">Queue</tspan><tspan
+ sodipodi:role="line"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-92.37825"
+ y="173.5065"
+ id="tspan4099">Pair 0</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:6.35px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-128.2318"
+ y="173.5065"
+ id="text4097-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4095-6"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-128.2318"
+ y="173.5065">Queue</tspan><tspan
+ sodipodi:role="line"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-136.1693"
+ y="173.5065"
+ id="tspan4099-4">Pair 2</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1679"
+ width="36"
+ height="18"
+ x="155.51115"
+ y="78.886261"
+ ry="3" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1679-8"
+ width="36"
+ height="18"
+ x="155.51115"
+ y="122.67575"
+ ry="3" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1679-4"
+ width="36"
+ height="18"
+ x="155.51115"
+ y="100.78101"
+ ry="3" />
+ <text
+ xml:space="preserve"
+ style="font-size:6.35px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-106.33705"
+ y="173.5065"
+ id="text4097-8"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4095-4"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-106.33705"
+ y="173.5065">Queue</tspan><tspan
+ sodipodi:role="line"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-114.27455"
+ y="173.5065"
+ id="tspan4099-5">Pair 1</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.388863;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1811"
+ width="58.909527"
+ height="18.812746"
+ x="144.0564"
+ y="41.268894"
+ ry="2.2255962" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-47.017281"
+ y="173.49187"
+ id="text4156"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4154"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';stroke-width:0.75"
+ x="-47.017281"
+ y="173.49187">Machine Learning</tspan><tspan
+ sodipodi:role="line"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';stroke-width:0.75"
+ x="-54.07283"
+ y="173.49187"
+ id="tspan4158">Hardware Engine</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.368668;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.737336, 0.737336;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924-0"
+ width="46.97015"
+ height="73.58287"
+ x="98.42067"
+ y="72.988976"
+ ry="2.4685853" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-88.32518"
+ y="122.24379"
+ id="text3708"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-88.32518"
+ y="122.24379">lcore 1</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-132.10504"
+ y="121.83865"
+ id="text3708-8"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-7"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-132.10504"
+ y="121.83865">lcore 5</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-0"
+ width="32.290321"
+ height="11.709678"
+ x="105.76101"
+ y="82.031425"
+ ry="3.0161259" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-6-7"
+ width="32.290321"
+ height="11.709678"
+ x="105.76101"
+ y="125.82092"
+ ry="3.0161259"
+ inkscape:connector-avoid="true" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-4-8"
+ width="32.290321"
+ height="11.709678"
+ x="105.76101"
+ y="103.92618"
+ ry="3.0161259"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-110.21718"
+ y="121.85381"
+ id="text3708-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-87"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-110.21718"
+ y="121.85381">lcore 3</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-150.82878"
+ y="121.79179"
+ id="text4542"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4540"
+ style="font-size:5.64444px;stroke-width:0.75"
+ x="-150.82878"
+ y="121.79179">Enqueue Workers</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.368668;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.737336, 0.737336;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924-7"
+ width="46.97015"
+ height="73.58287"
+ x="201.63062"
+ y="72.988976"
+ ry="2.4685853" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-88.32518"
+ y="225.08443"
+ id="text3708-9"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-9"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-88.32518"
+ y="225.08443">lcore 2</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-110.21167"
+ y="225.07202"
+ id="text3708-7"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-8"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-110.21167"
+ y="225.07202">lcore 4</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724"
+ width="32.290321"
+ height="11.709678"
+ x="208.97096"
+ y="82.031425"
+ ry="3.0161259" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-4"
+ width="32.290321"
+ height="11.709678"
+ x="208.97096"
+ y="103.92618"
+ ry="3.0161259" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-6"
+ width="32.290321"
+ height="11.709678"
+ x="208.97096"
+ y="125.82092"
+ ry="3.0161259" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-132.1133"
+ y="225.06514"
+ id="text3708-78"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-0"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-132.1133"
+ y="225.06514">lcore 6</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-150.82878"
+ y="225.00725"
+ id="text4542-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4540-7"
+ style="font-size:5.64444px;stroke-width:0.75"
+ x="-150.82878"
+ y="225.00725">Dequeue Workers</tspan></text>
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 79.368074,78.962117 26.440446,5.546991"
+ id="path6677"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-0"
+ inkscape:connection-start="#rect6252" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="M 78.978262,84.050781 112.13805,103.92618"
+ id="path6679"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-4-8"
+ inkscape:connection-start="#rect6252" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 73.959073,84.25738 42.026977,41.56354"
+ id="path6681"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-6-7"
+ inkscape:connection-start="#rect6252" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="M 79.368074,95.959838 105.76913,90.949016"
+ id="path6683"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-0"
+ inkscape:connection-start="#rect6252-8" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 79.368074,101.28215 26.416716,5.27791"
+ id="path7830"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-4-8"
+ inkscape:connection-start="#rect6252-8" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 79.069199,106.4283 32.903161,19.39262"
+ id="path7862"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-6-7"
+ inkscape:connection-start="#rect6252-8" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="M 79.069201,113.13371 111.97235,93.741103"
+ id="path7900"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-0"
+ inkscape:connection-start="#rect6252-2" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 79.368074,118.27987 26.416716,-5.2779"
+ id="path7932"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-4-8"
+ inkscape:connection-start="#rect6252-2" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 79.368074,123.60218 26.401056,5.01083"
+ id="path7998"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-6-7"
+ inkscape:connection-start="#rect6252-2" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="M 73.959074,135.30464 115.98605,93.741103"
+ id="path8000"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-0"
+ inkscape:connection-start="#rect6252-6" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="M 78.978264,135.51124 112.13805,115.63586"
+ id="path8002"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-4-8"
+ inkscape:connection-start="#rect6252-6" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 79.368074,140.59991 26.440446,-5.54699"
+ id="path8004"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-6-7"
+ inkscape:connection-start="#rect6252-6" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-76.53363"
+ y="65.63237"
+ id="text3708-84"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-4"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-76.53363"
+ y="65.63237">Model 0</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-98.991623"
+ y="66.015465"
+ id="text3708-84-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-4-6"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-98.991623"
+ y="66.015465">Model 1</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-121.44823"
+ y="65.646149"
+ id="text3708-84-9"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-4-1"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-121.44823"
+ y="65.646149">Model 2</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-143.9021"
+ y="65.625481"
+ id="text3708-84-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-4-8"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-143.9021"
+ y="65.625481">Model 3</tspan></text>
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect6252"
+ width="27.32258"
+ height="16.32258"
+ x="52.045494"
+ y="67.934799"
+ ry="1" />
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect6252-8"
+ width="27.32258"
+ height="16.32258"
+ x="52.045494"
+ y="90.391411"
+ ry="1" />
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect6252-2"
+ width="27.32258"
+ height="16.32258"
+ x="52.045494"
+ y="112.84802"
+ ry="1" />
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect6252-6"
+ width="27.32258"
+ height="16.32258"
+ x="52.045494"
+ y="135.30464"
+ ry="1" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.399886;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.799773, 0.799773;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924-1-3"
+ width="43.035683"
+ height="94.487152"
+ x="44.188942"
+ y="62.536991"
+ ry="3.1694498" />
+ <g
+ id="g1010"
+ transform="translate(21.464467,-15.875004)">
+ <text
+ xml:space="preserve"
+ style="font-weight:bold;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:'Arial Bold';text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-185.88483"
+ y="86.542366"
+ id="text15571"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan15569"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.5;stroke-opacity:1"
+ x="-185.88483"
+ y="86.542366">nb_worker_threads = 2 * MIN(nb_queue_pairs, (lcore_count - 1) / 2)</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-weight:bold;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:'Arial Bold';text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-178.43243"
+ y="90.903854"
+ id="text15571-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan15569-9"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.5;stroke-opacity:1"
+ x="-178.43243"
+ y="90.903854">inferences_per_queue_pair = nb_models * (repetitions / nb_queue_pairs)</tspan></text>
+ </g>
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/tools/img/mldev_inference_ordered.svg b/doc/guides/tools/img/mldev_inference_ordered.svg
new file mode 100644
index 0000000000..9d2b2c9246
--- /dev/null
+++ b/doc/guides/tools/img/mldev_inference_ordered.svg
@@ -0,0 +1,526 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="243mm"
+ height="144mm"
+ viewBox="0 0 243 144"
+ version="1.1"
+ id="svg5"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="inference_ordered.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns:xlink="http://www.w3.org/1999/xlink"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg">
+ <sodipodi:namedview
+ id="namedview7"
+ pagecolor="#ffffff"
+ bordercolor="#000000"
+ borderopacity="0.25"
+ inkscape:showpageshadow="2"
+ inkscape:pageopacity="0.0"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1"
+ inkscape:document-units="mm"
+ showgrid="false"
+ inkscape:zoom="0.74564394"
+ inkscape:cx="488.83922"
+ inkscape:cy="234.69647"
+ inkscape:window-width="1920"
+ inkscape:window-height="1017"
+ inkscape:window-x="1912"
+ inkscape:window-y="-8"
+ inkscape:window-maximized="1"
+ inkscape:current-layer="layer1" />
+ <defs
+ id="defs2">
+ <marker
+ style="overflow:visible"
+ id="RoundedArrow"
+ refX="5"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="RoundedArrow"
+ markerWidth="6.1347523"
+ markerHeight="5.9304948"
+ viewBox="0 0 6.1347524 5.9304951"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.7)"
+ d="m -0.21114562,-4.1055728 6.42229122,3.21114561 a 1,1 90 0 1 0,1.78885438 L -0.21114562,4.1055728 A 1.236068,1.236068 31.717474 0 1 -2,3 v -6 a 1.236068,1.236068 148.28253 0 1 1.78885438,-1.1055728 z"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:none"
+ id="path1367" />
+ </marker>
+ <linearGradient
+ inkscape:collect="always"
+ id="linearGradient31002">
+ <stop
+ style="stop-color:#fff6d5;stop-opacity:1;"
+ offset="0"
+ id="stop30998" />
+ <stop
+ style="stop-color:#fff6d5;stop-opacity:0;"
+ offset="1"
+ id="stop31000" />
+ </linearGradient>
+ <marker
+ style="overflow:visible"
+ id="TriangleStart"
+ refX="4"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="TriangleStart"
+ markerWidth="5.3244081"
+ markerHeight="6.155385"
+ viewBox="0 0 5.3244081 6.1553851"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.5)"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:context-stroke;stroke-width:1pt"
+ d="M 5.77,0 -2.88,5 V -5 Z"
+ id="path135" />
+ </marker>
+ <linearGradient
+ inkscape:collect="always"
+ xlink:href="#linearGradient31002"
+ id="linearGradient31004"
+ x1="19.620968"
+ y1="102.90323"
+ x2="279.1532"
+ y2="102.90323"
+ gradientUnits="userSpaceOnUse"
+ gradientTransform="matrix(0.89215122,0,0,0.73190287,13.449912,42.668706)" />
+ </defs>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1">
+ <g
+ id="g1340"
+ transform="translate(-25.225796,-45.983871)">
+ <rect
+ style="fill:url(#linearGradient31004);fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.404032;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect27876"
+ width="231.09595"
+ height="132.45081"
+ x="31.177822"
+ y="51.758469"
+ ry="3.5071263" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1813"
+ width="38.408459"
+ height="45.86002"
+ x="34.901794"
+ y="99.14959"
+ ry="5.2246051"
+ inkscape:connector-avoid="true" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 73.310253,115.94935 36.498807,-11.6509"
+ id="path1906"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1813"
+ inkscape:connection-end="#rect1724-0" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 73.310253,122.0796 36.117817,1e-5"
+ id="path1908"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1813"
+ inkscape:connection-end="#rect1724-4-8" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="M 73.310253,128.20983 109.80905,139.8607"
+ id="path1910"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1813"
+ inkscape:connection-end="#rect1724-6-7" />
+ <path
+ style="display:inline;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 141.71839,99.266314 19.42262,-10e-7"
+ id="path1912"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1724-0"
+ inkscape:connection-end="#rect1679" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 197.14101,99.266313 19.42259,10e-7"
+ id="path1914"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1679"
+ inkscape:connection-end="#rect1724" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 141.71839,122.07961 19.42262,-1e-5"
+ id="path1916"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1724-4-8"
+ inkscape:connection-end="#rect1679-4" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 197.14101,122.0796 19.42259,1e-5"
+ id="path1918"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1679-4"
+ inkscape:connection-end="#rect1724-4" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 141.71839,144.89282 19.42262,0"
+ id="path1920"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1724-6-7"
+ inkscape:connection-end="#rect1679-8" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 197.14101,144.89282 19.42259,0"
+ id="path1922"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1679-8"
+ inkscape:connection-end="#rect1724-6" />
+ <text
+ xml:space="preserve"
+ style="font-size:8.46667px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-121.09793"
+ y="54.031597"
+ id="text4093"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4091"
+ style="font-size:8.46667px;stroke-width:0.75"
+ x="-121.09793"
+ y="54.031597">Model X</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:6.35px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-118.63563"
+ y="179.13635"
+ id="text4097"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4095"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-118.63563"
+ y="179.13635">Queue</tspan><tspan
+ sodipodi:role="line"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-126.57313"
+ y="179.13635"
+ id="tspan4099">Pair 1</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:6.35px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-141.44887"
+ y="179.13635"
+ id="text4097-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4095-6"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-141.44887"
+ y="179.13635">Queue</tspan><tspan
+ sodipodi:role="line"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-149.38637"
+ y="179.13635"
+ id="tspan4099-4">Pair 2</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.354849;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.709699, 0.709699;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924"
+ width="44.145252"
+ height="72.532341"
+ x="157.06865"
+ y="85.813438"
+ ry="4.31247" />
+ <g
+ id="g1224">
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1679"
+ width="36"
+ height="18"
+ x="161.14101"
+ y="90.266312"
+ ry="3" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1679-8"
+ width="36"
+ height="18"
+ x="161.14101"
+ y="135.89282"
+ ry="3" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1679-4"
+ width="36"
+ height="18"
+ x="161.14101"
+ y="113.07959"
+ ry="3" />
+ </g>
+ <text
+ xml:space="preserve"
+ style="font-size:6.35px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-95.820801"
+ y="179.13635"
+ id="text4097-8"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4095-4"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-95.820801"
+ y="179.13635">Queue</tspan><tspan
+ sodipodi:role="line"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-103.7583"
+ y="179.13635"
+ id="tspan4099-5">Pair 0</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.317648;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1811"
+ width="44.196934"
+ height="16.731901"
+ x="157.04254"
+ y="56.49292"
+ ry="2.761292" />
+ <text
+ xml:space="preserve"
+ style="font-size:3.5859px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.317649;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.952945, 0.317649;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-60.009941"
+ y="186.38451"
+ id="text4156"
+ transform="matrix(0,-1.040508,0.96106903,0,0,0)"><tspan
+ sodipodi:role="line"
+ id="tspan4154"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-family:Arial;-inkscape-font-specification:'Arial Bold';stroke-width:0.317648"
+ x="-60.009941"
+ y="186.38451">Machine Learning</tspan><tspan
+ sodipodi:role="line"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-family:Arial;-inkscape-font-specification:'Arial Bold';stroke-width:0.317648"
+ x="-64.492317"
+ y="186.38451"
+ id="tspan4158">Hardware Engine</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-99.705231"
+ y="125.91087"
+ id="text3708"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-99.705231"
+ y="125.91087">lcore 1</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-145.3221"
+ y="125.50572"
+ id="text3708-8"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-7"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-145.3221"
+ y="125.50572">lcore 5</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-122.51577"
+ y="125.52089"
+ id="text3708-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-87"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-122.51577"
+ y="125.52089">lcore 3</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-162.06549"
+ y="125.4589"
+ id="text4542"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4540"
+ style="font-size:5.64444px;stroke-width:0.75"
+ x="-162.06549"
+ y="125.4589">Enqueue Workers</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-99.705231"
+ y="232.67706"
+ id="text3708-9"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-9"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-99.705231"
+ y="232.67706">lcore 2</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-122.51025"
+ y="232.66466"
+ id="text3708-7"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-8"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-122.51025"
+ y="232.66466">lcore 4</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-145.33035"
+ y="232.65778"
+ id="text3708-78"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-0"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-145.33035"
+ y="232.65778">lcore 6</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-162.06549"
+ y="232.59988"
+ id="text4542-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4540-7"
+ style="font-size:5.64444px;stroke-width:0.75"
+ x="-162.06549"
+ y="232.59988">Dequeue Workers</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:6.35px;font-family:Arial;-inkscape-font-specification:'Arial Bold';text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-177.01665"
+ y="220.07283"
+ id="text5181"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan5179"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-177.01665"
+ y="220.07283">test: inference_ordered</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-weight:bold;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:'Arial Bold';text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-178.63324"
+ y="98.67057"
+ id="text15571"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan15569"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.5;stroke-opacity:1"
+ x="-178.63324"
+ y="98.67057">nb_worker_threads = 2 * MIN(nb_queue_pairs, (lcore_count - 1) / 2)</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-weight:bold;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:'Arial Bold';text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-171.18085"
+ y="89.26754"
+ id="text15571-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan15569-9"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.5;stroke-opacity:1"
+ x="-171.18085"
+ y="89.26754">inferences_per_queue_pair = repetitions / nb_queue_pairs</tspan></text>
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.75;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#RoundedArrow);marker-end:url(#RoundedArrow)"
+ d="m 179.14101,85.813438 0,-12.588618"
+ id="path31090"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1924" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724"
+ width="32.290321"
+ height="11.709678"
+ x="216.5636"
+ y="93.411476"
+ ry="2" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-4"
+ width="32.290321"
+ height="11.709678"
+ x="216.5636"
+ y="116.22478"
+ ry="2" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-6"
+ width="32.290321"
+ height="11.709678"
+ x="216.5636"
+ y="139.03798"
+ ry="2" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.354849;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.709699, 0.709699;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924-6"
+ width="44.145252"
+ height="72.532341"
+ x="210.6364"
+ y="85.813477"
+ ry="4.31247" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-0"
+ width="32.290321"
+ height="11.709678"
+ x="109.42807"
+ y="93.411476"
+ ry="2" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-6-7"
+ width="32.290321"
+ height="11.709678"
+ x="109.42807"
+ y="139.03798"
+ ry="2"
+ inkscape:connector-avoid="true" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-4-8"
+ width="32.290321"
+ height="11.709678"
+ x="109.42807"
+ y="116.22478"
+ ry="2"
+ inkscape:connector-avoid="true" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.354849;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.709699, 0.709699;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924-6-3"
+ width="44.145252"
+ height="72.532341"
+ x="103.50092"
+ y="85.813477"
+ ry="4.31247" />
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/tools/img/mldev_model_ops_subtest_a.svg b/doc/guides/tools/img/mldev_model_ops_subtest_a.svg
new file mode 100644
index 0000000000..cce5c3be7c
--- /dev/null
+++ b/doc/guides/tools/img/mldev_model_ops_subtest_a.svg
@@ -0,0 +1,418 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="275mm"
+ height="135mm"
+ viewBox="0 0 275 135"
+ version="1.1"
+ id="svg5"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="mldev_model_ops_subtest_d.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg">
+ <sodipodi:namedview
+ id="namedview7"
+ pagecolor="#ffffff"
+ bordercolor="#000000"
+ borderopacity="0.25"
+ inkscape:showpageshadow="2"
+ inkscape:pageopacity="0.0"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1"
+ inkscape:document-units="mm"
+ showgrid="false"
+ inkscape:zoom="0.72515432"
+ inkscape:cx="372.33454"
+ inkscape:cy="401.29389"
+ inkscape:window-width="1920"
+ inkscape:window-height="986"
+ inkscape:window-x="-11"
+ inkscape:window-y="-11"
+ inkscape:window-maximized="1"
+ inkscape:current-layer="layer1" />
+ <defs
+ id="defs2">
+ <marker
+ style="overflow:visible"
+ id="RoundedArrow"
+ refX="6"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="RoundedArrow"
+ markerWidth="6.1347523"
+ markerHeight="5.9304948"
+ viewBox="0 0 6.1347524 5.9304951"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.7)"
+ d="m -0.21114562,-4.1055728 6.42229122,3.21114561 a 1,1 90 0 1 0,1.78885438 L -0.21114562,4.1055728 A 1.236068,1.236068 31.717474 0 1 -2,3 v -6 a 1.236068,1.236068 148.28253 0 1 1.78885438,-1.1055728 z"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:none"
+ id="path1367" />
+ </marker>
+ </defs>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1">
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 62.203489,25.425124 H 80.823463"
+ id="path1916"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234"
+ inkscape:connection-end="#rect234-07" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 128.19001,25.425124 h 18.6197"
+ id="path1918"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-07"
+ inkscape:connection-end="#rect234-6" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 194.17681,25.425125 h 18.61942"
+ id="path1922"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-6"
+ inkscape:connection-end="#rect234-0-6-6" />
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.633453;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234"
+ width="47.366547"
+ height="17.366549"
+ x="14.836943"
+ y="16.74185"
+ ry="1.4985789"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-25.888845"
+ y="38.514706"
+ id="text290"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-25.888845"
+ y="38.514706">Model 0 / Load</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.633453;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-07"
+ width="47.366547"
+ height="17.366549"
+ x="80.823463"
+ y="16.74185"
+ ry="1.4985789"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-25.888845"
+ y="104.31795"
+ id="text290-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-11"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-25.888845"
+ y="104.31795">Model 0 / Start</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.6329;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-6"
+ width="47.3671"
+ height="17.33604"
+ x="146.80971"
+ y="16.757105"
+ ry="1.4959463" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-25.362436"
+ y="170.39679"
+ id="text290-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-15"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-25.362436"
+ y="170.39679">Model 0 / Stop</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6"
+ width="47.3671"
+ height="17.33604"
+ x="212.79623"
+ y="16.757105"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-25.888849"
+ y="236.47427"
+ id="text290-3-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-25.888849"
+ y="236.47427">Model 0 / Unload</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 236.47978,34.093145 V 44.957249 H 38.520216 v 10.84885"
+ id="path1924"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6"
+ inkscape:connection-end="#rect234-0-6-6-3" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 62.203766,64.474119 H 80.823189"
+ id="path1926"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-3"
+ inkscape:connection-end="#rect234-0-6-6-2" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 128.19029,64.474119 h 18.61942"
+ id="path1928"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-2"
+ inkscape:connection-end="#rect234-0-6-1" />
+ <rect
+ style="fill:#fff6d5;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-3"
+ width="47.3671"
+ height="17.33604"
+ x="14.836666"
+ y="55.806099"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-64.937843"
+ y="38.541786"
+ id="text290-3-2-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-64.937843"
+ y="38.541786">Model 1 / Load</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-2"
+ width="47.3671"
+ height="17.33604"
+ x="80.823189"
+ y="55.806099"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-64.937843"
+ y="104.31795"
+ id="text290-3-2-2-28"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-2"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-64.937843"
+ y="104.31795">Model 1 / Start</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6"
+ width="47.3671"
+ height="17.33604"
+ x="212.79623"
+ y="55.806099"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-64.937843"
+ y="236.47427"
+ id="text290-3-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-64.937843"
+ y="236.47427">Model 1 / Unload</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-1"
+ width="47.3671"
+ height="17.33604"
+ x="146.80971"
+ y="55.806099"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-64.41143"
+ y="170.39679"
+ id="text290-3-2-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-8"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-64.41143"
+ y="170.39679">Model 1 / Stop</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 194.17681,64.474119 h 18.61942"
+ id="path1930"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6"
+ inkscape:connection-start="#rect234-0-6-1" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 236.47978,73.142139 V 83.990988 H 38.520216 V 94.85511"
+ id="path1932"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6-7"
+ inkscape:connection-start="#rect234-0-6" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 62.203766,103.52313 h 9.309711 v 1.3e-4 h 9.309712"
+ id="path1934"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-7" />
+ <rect
+ style="fill:#fff6d5;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-7"
+ width="47.3671"
+ height="17.33604"
+ x="14.836666"
+ y="94.85511"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.98687"
+ y="38.541786"
+ id="text290-3-2-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.98687"
+ y="38.541786">Model N / Load</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0"
+ width="47.3671"
+ height="17.33604"
+ x="80.823189"
+ y="94.855164"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.9869"
+ y="104.31795"
+ id="text290-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.9869"
+ y="104.31795">Model N / Start</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-9"
+ width="47.3671"
+ height="17.33604"
+ x="146.80971"
+ y="94.855164"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.4605"
+ y="170.39679"
+ id="text290-3-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.4605"
+ y="170.39679">Model N / Stop</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.633452;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-5"
+ width="47.366547"
+ height="17.366549"
+ x="212.79651"
+ y="94.839836"
+ ry="1.4985789"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.98682"
+ y="236.47427"
+ id="text290-3-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-3"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.98682"
+ y="236.47427">Model N / Unload</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 128.19029,103.52326 h 18.61942"
+ id="path1936"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 194.17681,103.52326 h 9.30985 v -1.5e-4 h 9.30985"
+ id="path1938"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-5" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#fff6d5;fill-rule:evenodd;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-120.71075"
+ y="220.77164"
+ id="text392"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan390"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';fill:#000000;stroke-width:0.5"
+ x="-120.71075"
+ y="220.77164">mldev: model_ops / subtest D</tspan></text>
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.448724;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect2164"
+ width="259.55127"
+ height="119.55128"
+ x="7.7243652"
+ y="7.7243614"
+ ry="1.17494" />
+ </g>
+</svg>
diff --git a/doc/guides/tools/img/mldev_model_ops_subtest_b.svg b/doc/guides/tools/img/mldev_model_ops_subtest_b.svg
new file mode 100644
index 0000000000..53a49a2823
--- /dev/null
+++ b/doc/guides/tools/img/mldev_model_ops_subtest_b.svg
@@ -0,0 +1,421 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="275mm"
+ height="135mm"
+ viewBox="0 0 275 135"
+ version="1.1"
+ id="svg5"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="mldev_model_ops_subtest_a.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg">
+ <sodipodi:namedview
+ id="namedview7"
+ pagecolor="#ffffff"
+ bordercolor="#000000"
+ borderopacity="0.25"
+ inkscape:showpageshadow="2"
+ inkscape:pageopacity="0.0"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1"
+ inkscape:document-units="mm"
+ showgrid="false"
+ inkscape:zoom="0.72515432"
+ inkscape:cx="277.87189"
+ inkscape:cy="401.29389"
+ inkscape:window-width="1920"
+ inkscape:window-height="986"
+ inkscape:window-x="-11"
+ inkscape:window-y="-11"
+ inkscape:window-maximized="1"
+ inkscape:current-layer="layer1" />
+ <defs
+ id="defs2">
+ <marker
+ style="overflow:visible"
+ id="RoundedArrow"
+ refX="6"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="RoundedArrow"
+ markerWidth="6.1347523"
+ markerHeight="5.9304948"
+ viewBox="0 0 6.1347524 5.9304951"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.7)"
+ d="m -0.21114562,-4.1055728 6.42229122,3.21114561 a 1,1 90 0 1 0,1.78885438 L -0.21114562,4.1055728 A 1.236068,1.236068 31.717474 0 1 -2,3 v -6 a 1.236068,1.236068 148.28253 0 1 1.78885438,-1.1055728 z"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:none"
+ id="path1367" />
+ </marker>
+ </defs>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1">
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.671403;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234"
+ width="53.328598"
+ height="17.328598"
+ x="15.144302"
+ y="12.878438"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-22.00646"
+ y="41.803089"
+ id="text290"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-22.00646"
+ y="41.803089">Model 0 / Load</tspan></text>
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.671403;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-07"
+ width="53.328598"
+ height="17.328598"
+ x="87.552399"
+ y="12.878438"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-22.00646"
+ y="114.21132"
+ id="text290-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-11"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-22.00646"
+ y="114.21132">Model 1 / Load</tspan></text>
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.671403;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-6"
+ width="53.328598"
+ height="17.328598"
+ x="206.5271"
+ y="12.878438"
+ ry="1.4953041" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-22.00646"
+ y="233.18588"
+ id="text290-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-15"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-22.00646"
+ y="233.18588">Model N / Load</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 68.4729,21.542737 H 87.552399"
+ id="path1916"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234"
+ inkscape:connection-end="#rect234-07" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 140.881,21.542737 h 65.6461"
+ id="path1918"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-07"
+ inkscape:connection-end="#rect234-6" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 233.1914,30.207036 0,4.918771 H 41.808601 v 4.918772"
+ id="path1922"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-6"
+ inkscape:connection-end="#rect234-0-6-6" />
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6"
+ width="53.328598"
+ height="17.328598"
+ x="15.144302"
+ y="40.044579"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-49.1726"
+ y="41.619808"
+ id="text290-3-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-49.1726"
+ y="41.619808">Model 0 / Start</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-3"
+ width="53.328598"
+ height="17.328598"
+ x="87.552399"
+ y="40.044579"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-49.1726"
+ y="114.02805"
+ id="text290-3-2-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-49.1726"
+ y="114.02805">Model 1 / Start</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-2"
+ width="53.328598"
+ height="17.328598"
+ x="206.5271"
+ y="40.044579"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-49.1726"
+ y="233.00261"
+ id="text290-3-2-2-28"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-2"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-49.1726"
+ y="233.00261">Model N / Start</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 68.4729,48.708878 H 87.552399"
+ id="path1924"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6"
+ inkscape:connection-end="#rect234-0-6-6-3" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 140.881,48.708878 h 65.6461"
+ id="path1926"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-3"
+ inkscape:connection-end="#rect234-0-6-6-2" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 233.1914,57.373177 V 62.29195 H 41.808599 v 4.918774"
+ id="path1928"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-2"
+ inkscape:connection-end="#rect234-0-6-1" />
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6"
+ width="53.328598"
+ height="17.328598"
+ x="87.552399"
+ y="67.210724"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-75.812294"
+ y="114.12037"
+ id="text290-3-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-75.812294"
+ y="114.12037">Model 1 / Stop</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-1"
+ width="53.328598"
+ height="17.328598"
+ x="15.1443"
+ y="67.210724"
+ ry="1.4953041" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-75.812294"
+ y="41.712139"
+ id="text290-3-2-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-8"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-75.812294"
+ y="41.712139">Model 0 / Stop</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-7"
+ width="53.328598"
+ height="17.328598"
+ x="206.5271"
+ y="67.210724"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-75.812294"
+ y="233.09494"
+ id="text290-3-2-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-75.812294"
+ y="233.09494">Model N / Stop</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 68.472898,75.875023 H 87.552399"
+ id="path1930"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6"
+ inkscape:connection-start="#rect234-0-6-1" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 140.881,75.875023 h 65.6461"
+ id="path1932"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6-7"
+ inkscape:connection-start="#rect234-0-6" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 233.1914,84.539322 0,4.91877 H 41.808602 v 4.91877"
+ id="path1934"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0"
+ inkscape:connection-start="#rect234-0-6-7" />
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0"
+ width="53.328598"
+ height="17.328598"
+ x="15.144303"
+ y="94.376862"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.50489"
+ y="41.803085"
+ id="text290-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.50489"
+ y="41.803085">Model 0 / Unload</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-9"
+ width="53.328598"
+ height="17.328598"
+ x="87.552399"
+ y="94.376862"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.50489"
+ y="114.2113"
+ id="text290-3-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.50489"
+ y="114.2113">Model 1 / Unload</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-5"
+ width="53.328598"
+ height="17.328598"
+ x="206.5271"
+ y="94.376862"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.50489"
+ y="233.18588"
+ id="text290-3-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-3"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.50489"
+ y="233.18588">Model N / Unload</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 68.472901,103.04116 H 87.552399"
+ id="path1936"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-9"
+ inkscape:connection-start="#rect234-0" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 140.881,103.04116 h 65.6461"
+ id="path1938"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-5"
+ inkscape:connection-start="#rect234-0-9" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#fff6d5;fill-rule:evenodd;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-119.78458"
+ y="220.32892"
+ id="text392"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan390"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';fill:#000000;stroke-width:0.5"
+ x="-119.78458"
+ y="220.32892">mldev: model_ops / subtest A</tspan></text>
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.442854;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect3967"
+ width="259.55716"
+ height="119.55714"
+ x="7.7214203"
+ y="7.7214317"
+ ry="1.1806604" />
+ </g>
+</svg>
diff --git a/doc/guides/tools/img/mldev_model_ops_subtest_c.svg b/doc/guides/tools/img/mldev_model_ops_subtest_c.svg
new file mode 100644
index 0000000000..320d4978e3
--- /dev/null
+++ b/doc/guides/tools/img/mldev_model_ops_subtest_c.svg
@@ -0,0 +1,364 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="275mm"
+ height="135mm"
+ viewBox="0 0 275 135"
+ version="1.1"
+ id="svg5"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="mldev_model_ops_subtest_c.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg">
+ <sodipodi:namedview
+ id="namedview7"
+ pagecolor="#ffffff"
+ bordercolor="#000000"
+ borderopacity="0.25"
+ inkscape:showpageshadow="2"
+ inkscape:pageopacity="0.0"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1"
+ inkscape:document-units="mm"
+ showgrid="false"
+ inkscape:zoom="0.72515432"
+ inkscape:cx="479.89785"
+ inkscape:cy="401.29389"
+ inkscape:window-width="1920"
+ inkscape:window-height="986"
+ inkscape:window-x="-11"
+ inkscape:window-y="-11"
+ inkscape:window-maximized="1"
+ inkscape:current-layer="layer1" />
+ <defs
+ id="defs2">
+ <marker
+ style="overflow:visible"
+ id="RoundedArrow"
+ refX="6"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="RoundedArrow"
+ markerWidth="6.1347523"
+ markerHeight="5.9304948"
+ viewBox="0 0 6.1347524 5.9304951"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.7)"
+ d="m -0.21114562,-4.1055728 6.42229122,3.21114561 a 1,1 90 0 1 0,1.78885438 L -0.21114562,4.1055728 A 1.236068,1.236068 31.717474 0 1 -2,3 v -6 a 1.236068,1.236068 148.28253 0 1 1.78885438,-1.1055728 z"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:none"
+ id="path1367" />
+ </marker>
+ </defs>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1">
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.671403;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234"
+ width="53.328598"
+ height="17.328598"
+ x="12.6443"
+ y="13.208546"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-22.336561"
+ y="39.303089"
+ id="text290"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-22.336561"
+ y="39.303089">Model 0 / Load</tspan></text>
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.671403;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-07"
+ width="53.328598"
+ height="17.328598"
+ x="85.052399"
+ y="13.208546"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-22.336561"
+ y="111.71132"
+ id="text290-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-11"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-22.336561"
+ y="111.71132">Model 1 / Load</tspan></text>
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.671403;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-6"
+ width="53.328598"
+ height="17.328598"
+ x="204.0271"
+ y="13.208546"
+ ry="1.4953041" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-22.336561"
+ y="230.68588"
+ id="text290-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-15"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-22.336561"
+ y="230.68588">Model N / Load</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 65.972898,21.872845 H 85.052399"
+ id="path1916"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234"
+ inkscape:connection-end="#rect234-07" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 138.381,21.872845 h 65.6461"
+ id="path1918"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-07"
+ inkscape:connection-end="#rect234-6" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 230.6914,30.537144 V 36.33787 H 39.308599 v 5.800726"
+ id="path1922"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-6"
+ inkscape:connection-end="#rect234-0-6-6" />
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6"
+ width="53.328598"
+ height="17.328598"
+ x="12.6443"
+ y="42.138596"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-51.266617"
+ y="39.119808"
+ id="text290-3-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-51.266617"
+ y="39.119808">Model 0 / Start</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-3"
+ width="53.328598"
+ height="17.328598"
+ x="85.052399"
+ y="42.138596"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-51.266617"
+ y="111.52805"
+ id="text290-3-2-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-51.266617"
+ y="111.52805">Model 0 / Stop</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 65.972898,50.802895 H 85.052399"
+ id="path1924"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6"
+ inkscape:connection-end="#rect234-0-6-6-3" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 184.94759,79.732941 H 204.0271"
+ id="path1930"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6"
+ inkscape:connection-start="#rect234-0-6-1" />
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0"
+ width="53.328598"
+ height="17.328598"
+ x="12.644301"
+ y="99.998688"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-109.12671"
+ y="39.303085"
+ id="text290-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-109.12671"
+ y="39.303085">Model 0 / Unload</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-9"
+ width="53.328598"
+ height="17.328598"
+ x="85.052399"
+ y="99.998688"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-109.12671"
+ y="111.7113"
+ id="text290-3-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-109.12671"
+ y="111.7113">Model 1 / Unload</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-5"
+ width="53.328598"
+ height="17.328598"
+ x="204.0271"
+ y="99.998688"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-109.12671"
+ y="230.68588"
+ id="text290-3-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-3"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-109.12671"
+ y="230.68588">Model N / Unload</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 65.972899,108.66299 h 19.0795"
+ id="path1936"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-9"
+ inkscape:connection-start="#rect234-0" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 138.381,108.66299 h 65.6461"
+ id="path1938"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-5"
+ inkscape:connection-start="#rect234-0-9" />
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6"
+ width="53.328598"
+ height="17.328598"
+ x="204.0271"
+ y="71.068642"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-79.670258"
+ y="230.59494"
+ id="text290-3-2-2-28"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-2"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-79.670258"
+ y="230.59494">Model N / Stop</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-1"
+ width="53.328598"
+ height="17.328598"
+ x="131.61899"
+ y="71.068642"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-80.196663"
+ y="158.0945"
+ id="text290-3-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-80.196663"
+ y="158.0945">Model N / Start</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 138.381,50.802895 h 19.90229 v 20.265747"
+ id="path1158"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-3"
+ inkscape:connection-end="#rect234-0-6-1" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 230.6914,88.39724 v 5.800724 H 39.3086 v 5.800724"
+ id="path1160"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6"
+ inkscape:connection-end="#rect234-0" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#fff6d5;fill-rule:evenodd;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-123.28999"
+ y="217.99152"
+ id="text392"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan390"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';fill:#000000;stroke-width:0.5"
+ x="-123.28999"
+ y="217.99152">mldev: model_ops / subtest C</tspan></text>
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.441855;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect3015"
+ width="259.55814"
+ height="119.55814"
+ x="7.720932"
+ y="7.7209282"
+ ry="1.1638433" />
+ </g>
+</svg>
diff --git a/doc/guides/tools/img/mldev_model_ops_subtest_d.svg b/doc/guides/tools/img/mldev_model_ops_subtest_d.svg
new file mode 100644
index 0000000000..80c1798d99
--- /dev/null
+++ b/doc/guides/tools/img/mldev_model_ops_subtest_d.svg
@@ -0,0 +1,422 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="275mm"
+ height="135mm"
+ viewBox="0 0 275 135"
+ version="1.1"
+ id="svg5"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="mldev_model_ops_subtest_b.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg">
+ <sodipodi:namedview
+ id="namedview7"
+ pagecolor="#ffffff"
+ bordercolor="#000000"
+ borderopacity="0.25"
+ inkscape:showpageshadow="2"
+ inkscape:pageopacity="0.0"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1"
+ inkscape:document-units="mm"
+ showgrid="false"
+ inkscape:zoom="0.72515432"
+ inkscape:cx="324.06895"
+ inkscape:cy="295.1096"
+ inkscape:window-width="1920"
+ inkscape:window-height="986"
+ inkscape:window-x="-11"
+ inkscape:window-y="-11"
+ inkscape:window-maximized="1"
+ inkscape:current-layer="layer1" />
+ <defs
+ id="defs2">
+ <marker
+ style="overflow:visible"
+ id="RoundedArrow"
+ refX="6"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="RoundedArrow"
+ markerWidth="6.1347523"
+ markerHeight="5.9304948"
+ viewBox="0 0 6.1347524 5.9304951"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.7)"
+ d="m -0.21114562,-4.1055728 6.42229122,3.21114561 a 1,1 90 0 1 0,1.78885438 L -0.21114562,4.1055728 A 1.236068,1.236068 31.717474 0 1 -2,3 v -6 a 1.236068,1.236068 148.28253 0 1 1.78885438,-1.1055728 z"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:none"
+ id="path1367" />
+ </marker>
+ </defs>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1">
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.633453;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234"
+ width="47.366547"
+ height="17.366549"
+ x="14.864025"
+ y="14.24185"
+ ry="1.4985789"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-23.388845"
+ y="38.541786"
+ id="text290"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-23.388845"
+ y="38.541786">Model 0 / Load</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.633453;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-07"
+ width="47.366547"
+ height="17.366549"
+ x="81.026939"
+ y="14.24185"
+ ry="1.4985789"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-23.388845"
+ y="104.52142"
+ id="text290-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-11"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-23.388845"
+ y="104.52142">Model 0 / Start</tspan></text>
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.6329;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-6"
+ width="47.3671"
+ height="17.33604"
+ x="147.18958"
+ y="14.257105"
+ ry="1.4959463" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-23.388849"
+ y="170.86761"
+ id="text290-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-15"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-23.388849"
+ y="170.86761">Model 1 / Load</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 62.230572,22.925124 H 81.026939"
+ id="path1916"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234"
+ inkscape:connection-end="#rect234-07" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 128.39349,22.925124 h 18.79609"
+ id="path1918"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-07"
+ inkscape:connection-end="#rect234-6" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 194.55667,22.925125 18.79581,0"
+ id="path1922"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-6"
+ inkscape:connection-end="#rect234-0-6-6" />
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6"
+ width="47.3671"
+ height="17.33604"
+ x="213.35248"
+ y="14.257105"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-23.388849"
+ y="236.84723"
+ id="text290-3-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-23.388849"
+ y="236.84723">Model 1 / Start</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 237.03603,31.593145 0,5.328675 H 38.547297 v 5.313421"
+ id="path1924"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6"
+ inkscape:connection-end="#rect234-0-6-6-3" />
+ <rect
+ style="fill:#fff6d5;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-3"
+ width="47.3671"
+ height="17.33604"
+ x="14.863747"
+ y="42.235241"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-51.366989"
+ y="38.541786"
+ id="text290-3-2-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-51.366989"
+ y="38.541786">Model N / Load</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-2"
+ width="47.3671"
+ height="17.33604"
+ x="81.026657"
+ y="42.235241"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-51.366989"
+ y="104.52142"
+ id="text290-3-2-2-28"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-2"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-51.366989"
+ y="104.52142">Model N / Start</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 62.230846,50.903261 H 81.026657"
+ id="path1926"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-3"
+ inkscape:connection-end="#rect234-0-6-6-2" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 128.39376,50.903261 h 42.47937 v 19.294584"
+ id="path1928"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-2"
+ inkscape:connection-end="#rect234-0-6-1" />
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6"
+ width="47.3671"
+ height="17.33604"
+ x="213.35248"
+ y="70.197845"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-79.32959"
+ y="237.03052"
+ id="text290-3-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-79.32959"
+ y="237.03052">Model N / Unload</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-1"
+ width="47.3671"
+ height="17.33604"
+ x="147.18958"
+ y="70.197845"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-78.803177"
+ y="170.77666"
+ id="text290-3-2-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-8"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-78.803177"
+ y="170.77666">Model N / Stop</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 194.55667,78.865866 h 18.79581"
+ id="path1930"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6"
+ inkscape:connection-start="#rect234-0-6-1" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 237.03603,87.533886 v 5.313417 H 38.547297 v 5.328677"
+ id="path1932"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6-7"
+ inkscape:connection-start="#rect234-0-6" />
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-7"
+ width="47.3671"
+ height="17.33604"
+ x="14.863747"
+ y="98.17598"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-106.78132"
+ y="38.450832"
+ id="text290-3-2-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-106.78132"
+ y="38.450832">Model 1 / Stop</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 62.230846,106.844 H 81.026657"
+ id="path1934"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0"
+ inkscape:connection-start="#rect234-0-6-7" />
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0"
+ width="47.3671"
+ height="17.33604"
+ x="81.026657"
+ y="98.17598"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-107.30773"
+ y="104.7047"
+ id="text290-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-107.30773"
+ y="104.7047">Model 1 / Unload</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-9"
+ width="47.3671"
+ height="17.33604"
+ x="147.18958"
+ y="98.17598"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-106.78133"
+ y="170.77666"
+ id="text290-3-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-106.78133"
+ y="170.77666">Model 0 / Stop</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.633452;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-5"
+ width="47.366547"
+ height="17.366549"
+ x="213.35277"
+ y="98.160721"
+ ry="1.4985789"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-107.30773"
+ y="237.03053"
+ id="text290-3-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-3"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-107.30773"
+ y="237.03053">Model 0 / Unload</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 128.39376,106.844 h 18.79582"
+ id="path1936"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-9"
+ inkscape:connection-start="#rect234-0" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 194.55667,106.844 9.39805,0 v 0 h 9.39805"
+ id="path1938"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-5"
+ inkscape:connection-start="#rect234-0-9" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#fff6d5;fill-rule:evenodd;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-122.00021"
+ y="222.26495"
+ id="text392"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan390"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';fill:#000000;stroke-width:0.5"
+ x="-122.00021"
+ y="222.26495">mldev: model_ops / subest B</tspan></text>
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.462341;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect3475"
+ width="259.53769"
+ height="119.53766"
+ x="7.7311554"
+ y="7.7311707"
+ ry="1.2186421" />
+ </g>
+</svg>
diff --git a/doc/guides/tools/index.rst b/doc/guides/tools/index.rst
index f1f5b94c8c..6f84fc31ff 100644
--- a/doc/guides/tools/index.rst
+++ b/doc/guides/tools/index.rst
@@ -21,4 +21,5 @@ DPDK Tools User Guides
comp_perf
testeventdev
testregex
+ testmldev
dts
diff --git a/doc/guides/tools/testmldev.rst b/doc/guides/tools/testmldev.rst
new file mode 100644
index 0000000000..845c2d9381
--- /dev/null
+++ b/doc/guides/tools/testmldev.rst
@@ -0,0 +1,441 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright (c) 2022 Marvell.
+
+dpdk-test-mldev Application
+===========================
+
+The ``dpdk-test-mldev`` tool is a Data Plane Development Kit (DPDK) application that allows testing
+various mldev use cases. This application has a generic framework to add new mldev based test cases
+to verify functionality and measure the performance of inference execution on DPDK ML devices.
+
+
+Application and Options
+-----------------------
+
+The application has a number of command line options:
+
+.. code-block:: console
+
+ dpdk-test-mldev [EAL Options] -- [application options]
+
+EAL Options
+~~~~~~~~~~~
+
+The following are the EAL command-line options that can be used with the ``dpdk-test-mldev``
+application. See the DPDK Getting Started Guides for more information on these options.
+
+* ``-c <COREMASK>`` or ``-l <CORELIST>``
+
+ Set the hexadecimal bitmask of the cores to run on. The corelist is a list of cores to use.
+
+* ``-a <PCI_ID>``
+
+ Attach a PCI based ML device. Specific to drivers using a PCI based ML devices.
+
+* ``--vdev <driver>``
+
+ Add a virtual mldev device. Specific to drivers using a ML virtual device.
+
+
+Application Options
+~~~~~~~~~~~~~~~~~~~
+
+The following are the command-line options supported by the test application.
+
+* ``--test <name>``
+
+ ML tests are divided into two groups, Model and Device tests and Inference tests. Test
+ name one of the following supported tests.
+
+ **ML Device Tests** ::
+
+ device_ops
+
+ **ML Model Tests** ::
+
+ model_ops
+
+ **ML Inference Tests** ::
+
+ inference_ordered
+ inference_interleave
+
+* ``--dev_id <n>``
+
+ Set the device id of the ML device to be used for the test. Default value is `0`.
+
+* ``--socket_id <n>``
+
+ Set the socket id of the application resources. Default value is `SOCKET_ID_ANY`.
+
+* ``--debug``
+
+ Enable the tests to run in debug mode.
+
+* ``--models <model_list>``
+
+ Set the list of model files to be used for the tests. Application expects the
+ ``model_list`` in comma separated form (i.e. ``--models model_A.bin,model_B.bin``).
+ Maximum number of models supported by the test is ``8``.
+
+* ``--filelist <file_list>``
+
+ Set the list of model, input, output and reference files to be used for the tests.
+ Application expects the ``file_list`` to be in comma separated form
+ (i.e. ``--filelist <model,input,output>[,reference]``).
+
+ Multiple filelist entries can be specified when running the tests with multiple models.
+ Both quantized and dequantized outputs are written to the disk. Dequantized output file
+ would have the name specified by the user through ``--filelist`` option. A suffix ``.q``
+ is appended to quantized output filename. Maximum number of filelist entries supported
+ by the test is ``8``.
+
+* ``--repetitions <n>``
+
+ Set the number of inference repetitions to be executed in the test per each model. Default
+ value is `1`.
+
+* ``--burst_size <n>``
+
+ Set the burst size to be used when enqueuing / dequeuing inferences. Default value is `1`.
+
+* ``--queue_pairs <n>``
+
+ Set the number of queue-pairs to be used for inference enqueue and dequeue operations.
+ Default value is `1`.
+
+* ``--queue_size <n>``
+
+ Set the size of queue-pair to be created for inference enqueue / dequeue operations.
+ Queue size would translate into `rte_ml_dev_qp_conf::nb_desc` field during queue-pair
+ creation. Default value is `1`.
+
+* ``--batches <n>``
+
+ Set the number batches in the input file provided for inference run. When not specified
+ the test would assume the number of batches is equal to the batch size of the model.
+
+* ``--tolerance <n>``
+
+ Set the tolerance value in percentage to be used for output validation. Default value
+ is `0`.
+
+* ``--stats``
+
+ Enable reporting device extended stats.
+
+
+ML Device Tests
+-------------------------
+
+ML device tests are functional tests to validate ML device APIs. Device tests validate the ML device
+handling APIs configure, close, start and stop APIs.
+
+
+Application Options
+~~~~~~~~~~~~~~~~~~~
+
+Supported command line options for the `model_ops` test are following::
+
+ --debug
+ --test
+ --dev_id
+ --socket_id
+ --queue_pairs
+ --queue_size
+
+
+DEVICE_OPS Test
+~~~~~~~~~~~~~~~
+
+Device ops test validates the device configuration and reconfiguration support. The test configures
+ML device based on the option ``--queue_pairs`` and ``--queue_size`` specified by the user, and
+later reconfigures the ML device with the number of queue pairs and queue size based the maximum
+specified through the device info.
+
+
+Example
+^^^^^^^
+
+Command to run device_ops test:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=device_ops
+
+
+Command to run device_ops test with user options:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=device_ops --queue_pairs <M> --queue_size <N>
+
+
+ML Model Tests
+-------------------------
+
+Model tests are functional tests to validate ML model APIs. Model tests validate the functioning
+of APIs to load, start, stop and unload ML models.
+
+
+Application Options
+~~~~~~~~~~~~~~~~~~~
+
+Supported command line options for the `model_ops` test are following::
+
+ --debug
+ --test
+ --dev_id
+ --socket_id
+ --models
+
+
+List of model files to be used for the `model_ops` test can be specified through the option
+``--models <model_list>`` as a comma separated list. Maximum number of models supported in
+the test is `8`.
+
+.. Note::
+
+ * The ``--models <model_list>`` is a mandatory option for running this test.
+ * Options not supported by the test are ignored if specified.
+
+
+MODEL_OPS Test
+~~~~~~~~~~~~~~
+
+The test is a collection of multiple sub-tests, each with a different order of slow-path
+operations when handling with `N` number of models.
+
+
+**Sub-test A:** executes the sequence of load / start / stop / unload for a model in order,
+followed by next model.
+.. _figure_mldev_model_ops_subtest_a:
+
+.. figure:: img/mldev_model_ops_subtest_a.*
+
+ Execution sequence of model_ops subtest A.
+
+
+**Sub-test B:** executes load for all models, followed by a start for all models. Upon successful
+start of all models, stop is invoked for all models followed by unload.
+.. _figure_mldev_model_ops_subtest_b:
+
+.. figure:: img/mldev_model_ops_subtest_b.*
+
+ Execution sequence of model_ops subtest B.
+
+
+**Sub-test C:** loads all models, followed by a start and stop of all models in order. Upon
+completion of stop, unload is invoked for all models.
+.. _figure_mldev_model_ops_subtest_c:
+
+.. figure:: img/mldev_model_ops_subtest_c.*
+
+ Execution sequence of model_ops subtest C.
+
+
+**Sub-test D:** executes load and start for all models available. Upon successful start of all
+models, stop and stop is executed for the models.
+.. _figure_mldev_model_ops_subtest_d:
+
+.. figure:: img/mldev_model_ops_subtest_d.*
+
+ Execution sequence of model_ops subtest D.
+
+
+Example
+^^^^^^^
+
+Command to run model_ops test:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=model_ops --models model_1.bin,model_2.bin,model_3.bin, model_4.bin
+
+
+ML Inference Tests
+------------------
+
+Inference tests are a set of tests to validate end-to-end inference execution on ML device.
+These tests executes the full sequence of operations required to run inferences with one or
+multiple models.
+
+Application Options
+~~~~~~~~~~~~~~~~~~~
+
+Supported command line options for inference tests are following::
+
+ --debug
+ --test
+ --dev_id
+ --socket_id
+ --filelist
+ --repetitions
+ --burst_size
+ --queue_pairs
+ --queue_size
+ --batches
+ --tolerance
+ --stats
+
+
+List of files to be used for the inference tests can be specified through the option
+``--filelist <file_list>`` as a comma separated list. A filelist entry would be of the format
+``--filelist <model_file,input_file,output_file>[,reference_file]`` and is used to specify the
+list of files required to test with a single model. Multiple filelist entries are supported by
+the test, one entry per model. Maximum number of file entries supported by the test is `8`.
+
+When ``--burst_size <num>`` option is specified for the test, enqueue and dequeue burst would
+try to enqueue or dequeue ``num`` number of inferences per each call respectively.
+
+In the inference test, a pair of lcores are mapped to each queue pair. Minimum number of lcores
+required for the tests is equal to ``(queue_pairs * 2 + 1)``.
+
+Output validation of inference would be enabled only when a reference file is specified through
+the ``--filelist`` option. Application would additionally consider the tolerance value provided
+through ``--tolerance`` option during validation. When the tolerance values is 0, CRC32 hash of
+inference output and reference output are compared. When the tolerance is non-zero, element wise
+comparison of output is performed. Validation is considered as successful only when all the
+elements of the output tensor are with in the tolerance range specified.
+
+When ``--debug`` option is specified, tests are run in debug mode.
+
+Enabling ``--stats`` would print the extended stats supported by the driver.
+
+.. Note::
+
+ * The ``--filelist <file_list>`` is a mandatory option for running inference tests.
+ * Options not supported by the tests are ignored if specified.
+ * Element wise comparison is not supported when the output dtype is either fp8, fp16
+ or bfloat16. This is applicable only when the tolerance is greater than zero and for
+ pre-quantized models only.
+
+
+INFERENCE_ORDERED Test
+~~~~~~~~~~~~~~~~~~~~~~
+
+This is a functional test for validating the end-to-end inference execution on ML device. This
+test configures ML device and queue pairs as per the queue-pair related options (queue_pairs and
+queue_size) specified by the user. Upon successful configuration of the device and queue pairs,
+the first model specified through the filelist is loaded to the device and inferences are enqueued
+by a pool of worker threads to the ML device. Total number of inferences enqueued for the model
+are equal to the repetitions specified. A dedicated pool of worker threads would dequeue the
+inferences from the device. The model is unloaded upon completion of all inferences for the model.
+The test would continue loading and executing inference requests for all models specified
+through ``filelist`` option in an ordered manner.
+
+.. _figure_mldev_inference_ordered:
+
+.. figure:: img/mldev_inference_ordered.*
+
+ Execution of inference_ordered on single model.
+
+
+Example
+^^^^^^^
+
+Example command to run inference_ordered test:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_ordered --filelist model.bin,input.bin,output.bin
+
+Example command to run inference_ordered with output validation using tolerance of `1%``:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_ordered --filelist model.bin,input.bin,output.bin,reference.bin \
+ --tolerance 1.0
+
+Example command to run inference_ordered test with multiple queue-pairs and queue size:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_ordered --filelist model.bin,input.bin,output.bin \
+ --queue_pairs 4 --queue_size 16
+
+Example command to run inference_ordered test with a specific burst size:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_ordered --filelist model.bin,input.bin,output.bin \
+ --burst_size 12
+
+
+INFERENCE_INTERLEAVE Test
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This is a stress test for validating the end-to-end inference execution on ML device. The test
+configures the ML device and queue pairs as per the queue-pair related options (queue_pairs
+and queue_size) specified by the user. Upon successful configuration of the device and queue
+pairs, all models specified through the filelist are loaded to the device. Inferences for multiple
+models are enqueued by a pool of worker threads in parallel. Inference execution by the device is
+interleaved between multiple models. Total number of inferences enqueued for a model are equal to
+the repetitions specified. An additional pool of threads would dequeue the inferences from the
+device. Models would be unloaded upon completion of inferences for all models loaded.
+
+
+.. _figure_mldev_inference_interleave:
+
+.. figure:: img/mldev_inference_interleave.*
+
+ Execution of inference_interleave on single model.
+
+
+Example
+^^^^^^^
+
+Example command to run inference_interleave test:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_interleave --filelist model.bin,input.bin,output.bin
+
+
+Example command to run inference_interleave test with multiple models:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_interleave --filelist model_A.bin,input_A.bin,output_A.bin \
+ --filelist model_B.bin,input_B.bin,output_B.bin
+
+
+Example command to run inference_interleave test with multiple models ad output validation
+using tolerance of `2.0%``:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_interleave \
+ --filelist model_A.bin,input_A.bin,output_A.bin,reference_A.bin \
+ --filelist model_B.bin,input_B.bin,output_B.bin,reference_B.bin \
+ --tolerance 2.0
+
+Example command to run inference_interleave test with multiple queue-pairs and queue size
+and burst size:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_interleave --filelist model.bin,input.bin,output.bin \
+ --queue_pairs 8 --queue_size 12 --burst_size 16
+
+
+Debug mode
+----------
+
+ML tests can be executed in debug mode by enabling the option ``--debug``. Execution of tests in
+debug mode would enable additional prints.
+
+When a validation failure is observed, output from that buffer is written to the disk, with the
+filenames having similar convention when the test has passed. Additionally index of the buffer
+would be appended to the filenames.
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread
* [PATCH v1 02/12] app/mldev: add common test functions
2022-11-29 7:07 [PATCH v1 00/12] implement mldev test application Srikanth Yalavarthi
2022-11-29 7:07 ` [PATCH v1 01/12] app/mldev: implement test framework for mldev Srikanth Yalavarthi
@ 2022-11-29 7:07 ` Srikanth Yalavarthi
2022-11-29 7:07 ` [PATCH v1 03/12] app/mldev: add test case to validate device ops Srikanth Yalavarthi
` (13 subsequent siblings)
15 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2022-11-29 7:07 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj
Added common functions used by all tests. Common code
includes functions to check capabilities, options, and
handle ML devices.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/meson.build | 1 +
app/test-mldev/test_common.c | 139 +++++++++++++++++++++++++++++++++++
app/test-mldev/test_common.h | 27 +++++++
3 files changed, 167 insertions(+)
create mode 100644 app/test-mldev/test_common.c
create mode 100644 app/test-mldev/test_common.h
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
index 8ca2e1a1c1..964bb9ddc4 100644
--- a/app/test-mldev/meson.build
+++ b/app/test-mldev/meson.build
@@ -12,6 +12,7 @@ sources = files(
'ml_options.c',
'ml_test.c',
'parser.c',
+ 'test_common.c',
)
deps += ['mldev']
diff --git a/app/test-mldev/test_common.c b/app/test-mldev/test_common.c
new file mode 100644
index 0000000000..b6b32904e4
--- /dev/null
+++ b/app/test-mldev/test_common.c
@@ -0,0 +1,139 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <errno.h>
+#include <stdbool.h>
+#include <stdint.h>
+
+#include <rte_common.h>
+#include <rte_memory.h>
+#include <rte_mldev.h>
+
+#include "ml_common.h"
+#include "ml_options.h"
+#include "test_common.h"
+
+bool
+ml_test_cap_check(struct ml_options *opt)
+{
+ struct rte_ml_dev_info dev_info;
+
+ rte_ml_dev_info_get(opt->dev_id, &dev_info);
+ if (dev_info.max_models == 0) {
+ ml_err("Not enough mldev models supported = %d", dev_info.max_models);
+ return false;
+ }
+
+ return true;
+}
+
+int
+ml_test_opt_check(struct ml_options *opt)
+{
+ uint16_t dev_count;
+ int socket_id;
+
+ RTE_SET_USED(opt);
+
+ dev_count = rte_ml_dev_count();
+ if (dev_count == 0) {
+ ml_err("No ML devices found");
+ return -ENODEV;
+ }
+
+ if (opt->dev_id >= dev_count) {
+ ml_err("Invalid option dev_id = %d", opt->dev_id);
+ return -EINVAL;
+ }
+
+ socket_id = rte_ml_dev_socket_id(opt->dev_id);
+ if (!((opt->socket_id != SOCKET_ID_ANY) || (opt->socket_id != socket_id))) {
+ ml_err("Invalid option, socket_id = %d\n", opt->socket_id);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+void
+ml_test_opt_dump(struct ml_options *opt)
+{
+ ml_options_dump(opt);
+}
+
+int
+ml_test_device_configure(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_common *t = ml_test_priv(test);
+ struct rte_ml_dev_config dev_config;
+ int ret;
+
+ ret = rte_ml_dev_info_get(opt->dev_id, &t->dev_info);
+ if (ret != 0) {
+ ml_err("Failed to get mldev info, dev_id = %d\n", opt->dev_id);
+ return ret;
+ }
+
+ /* configure device */
+ dev_config.socket_id = opt->socket_id;
+ dev_config.nb_models = t->dev_info.max_models;
+ dev_config.nb_queue_pairs = t->dev_info.max_queue_pairs;
+ ret = rte_ml_dev_configure(opt->dev_id, &dev_config);
+ if (ret != 0) {
+ ml_err("Failed to configure ml device, dev_id = %d\n", opt->dev_id);
+ return ret;
+ }
+
+ return 0;
+}
+
+int
+ml_test_device_close(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_common *t = ml_test_priv(test);
+ int ret = 0;
+
+ RTE_SET_USED(t);
+
+ /* close device */
+ ret = rte_ml_dev_close(opt->dev_id);
+ if (ret != 0)
+ ml_err("Failed to close ML device, dev_id = %d\n", opt->dev_id);
+
+ return ret;
+}
+
+int
+ml_test_device_start(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_common *t = ml_test_priv(test);
+ int ret;
+
+ RTE_SET_USED(t);
+
+ /* start device */
+ ret = rte_ml_dev_start(opt->dev_id);
+ if (ret != 0) {
+ ml_err("Failed to start ml device, dev_id = %d\n", opt->dev_id);
+ return ret;
+ }
+
+ return 0;
+}
+
+int
+ml_test_device_stop(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_common *t = ml_test_priv(test);
+ int ret = 0;
+
+ RTE_SET_USED(t);
+
+ /* stop device */
+ ret = rte_ml_dev_stop(opt->dev_id);
+ if (ret != 0)
+ ml_err("Failed to stop ML device, dev_id = %d\n", opt->dev_id);
+
+ return ret;
+}
diff --git a/app/test-mldev/test_common.h b/app/test-mldev/test_common.h
new file mode 100644
index 0000000000..05a2e43e2f
--- /dev/null
+++ b/app/test-mldev/test_common.h
@@ -0,0 +1,27 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_TEST_COMMON_
+#define _ML_TEST_COMMON_
+
+#include <rte_mldev.h>
+
+#include "ml_options.h"
+#include "ml_test.h"
+
+struct test_common {
+ struct ml_options *opt;
+ enum ml_test_result result;
+ struct rte_ml_dev_info dev_info;
+};
+
+bool ml_test_cap_check(struct ml_options *opt);
+int ml_test_opt_check(struct ml_options *opt);
+void ml_test_opt_dump(struct ml_options *opt);
+int ml_test_device_configure(struct ml_test *test, struct ml_options *opt);
+int ml_test_device_close(struct ml_test *test, struct ml_options *opt);
+int ml_test_device_start(struct ml_test *test, struct ml_options *opt);
+int ml_test_device_stop(struct ml_test *test, struct ml_options *opt);
+
+#endif /* _ML_TEST_COMMON_ */
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v1 03/12] app/mldev: add test case to validate device ops
2022-11-29 7:07 [PATCH v1 00/12] implement mldev test application Srikanth Yalavarthi
2022-11-29 7:07 ` [PATCH v1 01/12] app/mldev: implement test framework for mldev Srikanth Yalavarthi
2022-11-29 7:07 ` [PATCH v1 02/12] app/mldev: add common test functions Srikanth Yalavarthi
@ 2022-11-29 7:07 ` Srikanth Yalavarthi
2022-11-29 7:07 ` [PATCH v1 04/12] app/mldev: add test case to validate model ops Srikanth Yalavarthi
` (12 subsequent siblings)
15 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2022-11-29 7:07 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj
Added test case to validate device handling operations. Device ops
test is a collection of multiple sub-tests. Enabled sub-test to
validate device reconfiguration. Set device_ops as the default test.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/meson.build | 1 +
app/test-mldev/ml_options.c | 5 +-
app/test-mldev/test_device_ops.c | 234 +++++++++++++++++++++++++++++++
app/test-mldev/test_device_ops.h | 17 +++
4 files changed, 255 insertions(+), 2 deletions(-)
create mode 100644 app/test-mldev/test_device_ops.c
create mode 100644 app/test-mldev/test_device_ops.h
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
index 964bb9ddc4..60ea23d142 100644
--- a/app/test-mldev/meson.build
+++ b/app/test-mldev/meson.build
@@ -13,6 +13,7 @@ sources = files(
'ml_test.c',
'parser.c',
'test_common.c',
+ 'test_device_ops.c',
)
deps += ['mldev']
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 8fd7760e36..2e5f11bca2 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -24,7 +24,7 @@ void
ml_options_default(struct ml_options *opt)
{
memset(opt, 0, sizeof(*opt));
- strlcpy(opt->test_name, "ml_test", ML_TEST_NAME_MAX_LEN);
+ strlcpy(opt->test_name, "device_ops", ML_TEST_NAME_MAX_LEN);
opt->dev_id = 0;
opt->socket_id = SOCKET_ID_ANY;
opt->debug = false;
@@ -66,7 +66,8 @@ ml_parse_socket_id(struct ml_options *opt, const char *arg)
static void
ml_dump_test_options(const char *testname)
{
- RTE_SET_USED(testname);
+ if (strcmp(testname, "device_ops") == 0)
+ printf("\n");
}
static void
diff --git a/app/test-mldev/test_device_ops.c b/app/test-mldev/test_device_ops.c
new file mode 100644
index 0000000000..4cafcf41a6
--- /dev/null
+++ b/app/test-mldev/test_device_ops.c
@@ -0,0 +1,234 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <errno.h>
+#include <stdbool.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+
+#include <rte_common.h>
+#include <rte_malloc.h>
+#include <rte_mldev.h>
+
+#include "ml_common.h"
+#include "ml_options.h"
+#include "test_device_ops.h"
+
+static bool
+test_device_cap_check(struct ml_options *opt)
+{
+ if (!ml_test_cap_check(opt))
+ return false;
+
+ return true;
+}
+
+static int
+test_device_opt_check(struct ml_options *opt)
+{
+ int ret;
+
+ /* check common opts */
+ ret = ml_test_opt_check(opt);
+ if (ret != 0)
+ return ret;
+
+ return 0;
+}
+
+static void
+test_device_opt_dump(struct ml_options *opt)
+{
+ /* dump common opts */
+ ml_test_opt_dump(opt);
+}
+
+static int
+test_device_setup(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_device *t;
+ void *test_device;
+ int ret = 0;
+
+ /* allocate for test structure */
+ test_device = rte_zmalloc_socket(test->name, sizeof(struct test_device),
+ RTE_CACHE_LINE_SIZE, opt->socket_id);
+ if (test_device == NULL) {
+ ml_err("failed to allocate memory for test_model");
+ ret = -ENOMEM;
+ goto error;
+ }
+ test->test_priv = test_device;
+ t = ml_test_priv(test);
+
+ t->cmn.result = ML_TEST_FAILED;
+ t->cmn.opt = opt;
+
+ /* get device info */
+ ret = rte_ml_dev_info_get(opt->dev_id, &t->cmn.dev_info);
+ if (ret < 0) {
+ ml_err("failed to get device info");
+ goto error;
+ }
+
+ return 0;
+
+error:
+ if (test_device != NULL)
+ rte_free(test_device);
+
+ return ret;
+}
+
+static void
+test_device_destroy(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_device *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+ if (t != NULL)
+ rte_free(t);
+}
+
+static int
+test_device_reconfigure(struct ml_test *test, struct ml_options *opt)
+{
+ struct rte_ml_dev_config dev_config;
+ struct rte_ml_dev_qp_conf qp_conf;
+ struct test_device *t;
+ uint16_t qp_id = 0;
+ int ret = 0;
+
+ t = ml_test_priv(test);
+
+ /* configure with default options */
+ ret = ml_test_device_configure(test, opt);
+ if (ret != 0)
+ return ret;
+
+ /* setup one queue pair with nb_desc = 1 */
+ qp_conf.nb_desc = 1;
+ qp_conf.cb = NULL;
+
+ ret = rte_ml_dev_queue_pair_setup(opt->dev_id, qp_id, &qp_conf, opt->socket_id);
+ if (ret != 0) {
+ ml_err("Failed to setup ML device queue-pair, dev_id = %d, qp_id = %u\n",
+ opt->dev_id, qp_id);
+ goto error;
+ }
+
+ /* start device */
+ ret = ml_test_device_start(test, opt);
+ if (ret != 0)
+ goto error;
+
+ /* stop device */
+ ret = ml_test_device_stop(test, opt);
+ if (ret != 0) {
+ ml_err("Failed to stop device");
+ goto error;
+ }
+
+ /* reconfigure device based on dev_info */
+ dev_config.socket_id = opt->socket_id;
+ dev_config.nb_models = t->cmn.dev_info.max_models;
+ dev_config.nb_queue_pairs = t->cmn.dev_info.max_queue_pairs;
+ ret = rte_ml_dev_configure(opt->dev_id, &dev_config);
+ if (ret != 0) {
+ ml_err("Failed to reconfigure ML device, dev_id = %d\n", opt->dev_id);
+ return ret;
+ }
+
+ /* setup queue pairs */
+ for (qp_id = 0; qp_id < t->cmn.dev_info.max_queue_pairs; qp_id++) {
+ qp_conf.nb_desc = t->cmn.dev_info.max_desc;
+ qp_conf.cb = NULL;
+
+ ret = rte_ml_dev_queue_pair_setup(opt->dev_id, qp_id, &qp_conf, opt->socket_id);
+ if (ret != 0) {
+ ml_err("Failed to setup ML device queue-pair, dev_id = %d, qp_id = %u\n",
+ opt->dev_id, qp_id);
+ goto error;
+ }
+ }
+
+ /* start device */
+ ret = ml_test_device_start(test, opt);
+ if (ret != 0)
+ goto error;
+
+ /* stop device */
+ ret = ml_test_device_stop(test, opt);
+ if (ret != 0)
+ goto error;
+
+ /* close device */
+ ret = ml_test_device_close(test, opt);
+ if (ret != 0)
+ return ret;
+
+ return 0;
+
+error:
+ ml_test_device_close(test, opt);
+
+ return ret;
+}
+
+static int
+test_device_driver(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_device *t;
+ int ret = 0;
+
+ t = ml_test_priv(test);
+
+ /* sub-test: device reconfigure */
+ ret = test_device_reconfigure(test, opt);
+ if (ret != 0) {
+ printf("\n");
+ printf("Model Device Reconfigure Test: " CLRED "%s" CLNRM "\n", "Failed");
+ goto error;
+ } else {
+ printf("\n");
+ printf("Model Device Reconfigure Test: " CLYEL "%s" CLNRM "\n", "Passed");
+ }
+
+ printf("\n");
+
+ t->cmn.result = ML_TEST_SUCCESS;
+
+ return 0;
+
+error:
+ t->cmn.result = ML_TEST_FAILED;
+ return -1;
+}
+
+static int
+test_device_result(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_device *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+
+ return t->cmn.result;
+}
+
+static const struct ml_test_ops device_ops = {
+ .cap_check = test_device_cap_check,
+ .opt_check = test_device_opt_check,
+ .opt_dump = test_device_opt_dump,
+ .test_setup = test_device_setup,
+ .test_destroy = test_device_destroy,
+ .test_driver = test_device_driver,
+ .test_result = test_device_result,
+};
+
+ML_TEST_REGISTER(device_ops);
diff --git a/app/test-mldev/test_device_ops.h b/app/test-mldev/test_device_ops.h
new file mode 100644
index 0000000000..115b1072a2
--- /dev/null
+++ b/app/test-mldev/test_device_ops.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_TEST_DEVICE_OPS_
+#define _ML_TEST_DEVICE_OPS_
+
+#include <rte_common.h>
+
+#include "test_common.h"
+
+struct test_device {
+ /* common data */
+ struct test_common cmn;
+} __rte_cache_aligned;
+
+#endif /* _ML_TEST_DEVICE_OPS_ */
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v1 04/12] app/mldev: add test case to validate model ops
2022-11-29 7:07 [PATCH v1 00/12] implement mldev test application Srikanth Yalavarthi
` (2 preceding siblings ...)
2022-11-29 7:07 ` [PATCH v1 03/12] app/mldev: add test case to validate device ops Srikanth Yalavarthi
@ 2022-11-29 7:07 ` Srikanth Yalavarthi
2022-11-29 7:07 ` [PATCH v1 05/12] app/mldev: add ordered inference test case Srikanth Yalavarthi
` (11 subsequent siblings)
15 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2022-11-29 7:07 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj
Added test case to validate model operations. Model ops test
is a collection of sub-tests. Each sub-test invokes the model
operations in a specific order.
Sub-test A: (load -> start -> stop -> unload) x n
Sub-test B: load x n -> start x n -> stop x n -> unload x n
Sub-test C: load x n + (start + stop) x n + unload x n
Sub-test D: (load + start) x n -> (stop + unload) x n
Added internal functions to handle model load, start, stop and
unload. List of models to be used for testing can be specified
through application argument "--models"
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/meson.build | 2 +
app/test-mldev/ml_options.c | 45 ++-
app/test-mldev/ml_options.h | 9 +
app/test-mldev/test_model_common.c | 162 +++++++++++
app/test-mldev/test_model_common.h | 37 +++
app/test-mldev/test_model_ops.c | 433 +++++++++++++++++++++++++++++
app/test-mldev/test_model_ops.h | 21 ++
7 files changed, 706 insertions(+), 3 deletions(-)
create mode 100644 app/test-mldev/test_model_common.c
create mode 100644 app/test-mldev/test_model_common.h
create mode 100644 app/test-mldev/test_model_ops.c
create mode 100644 app/test-mldev/test_model_ops.h
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
index 60ea23d142..b09e1ccc8a 100644
--- a/app/test-mldev/meson.build
+++ b/app/test-mldev/meson.build
@@ -14,6 +14,8 @@ sources = files(
'parser.c',
'test_common.c',
'test_device_ops.c',
+ 'test_model_common.c',
+ 'test_model_ops.c',
)
deps += ['mldev']
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 2e5f11bca2..8e40a33ed0 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -4,6 +4,7 @@
#include <errno.h>
#include <getopt.h>
+#include <linux/limits.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
@@ -27,6 +28,7 @@ ml_options_default(struct ml_options *opt)
strlcpy(opt->test_name, "device_ops", ML_TEST_NAME_MAX_LEN);
opt->dev_id = 0;
opt->socket_id = SOCKET_ID_ANY;
+ opt->nb_filelist = 0;
opt->debug = false;
}
@@ -63,11 +65,47 @@ ml_parse_socket_id(struct ml_options *opt, const char *arg)
return 0;
}
+static int
+ml_parse_models(struct ml_options *opt, const char *arg)
+{
+ const char *delim = ",";
+ char models[PATH_MAX];
+ char *token;
+ int ret = 0;
+
+ strlcpy(models, arg, PATH_MAX);
+
+ token = strtok(models, delim);
+ while (token != NULL) {
+ strlcpy(opt->filelist[opt->nb_filelist].model, token, PATH_MAX);
+ opt->nb_filelist++;
+
+ if (opt->nb_filelist >= ML_TEST_MAX_MODELS) {
+ ml_err("Exceeded model count, max = %d\n", ML_TEST_MAX_MODELS);
+ ret = -EINVAL;
+ break;
+ }
+ token = strtok(NULL, delim);
+ }
+
+ if (opt->nb_filelist == 0) {
+ ml_err("Models list is empty. Need atleast one model for the test");
+ ret = -EINVAL;
+ }
+
+ return ret;
+}
+
static void
ml_dump_test_options(const char *testname)
{
if (strcmp(testname, "device_ops") == 0)
printf("\n");
+
+ if (strcmp(testname, "model_ops") == 0) {
+ printf("\t\t--models : comma separated list of models\n");
+ printf("\n");
+ }
}
static void
@@ -85,9 +123,9 @@ print_usage(char *program)
ml_test_dump_names(ml_dump_test_options);
}
-static struct option lgopts[] = {{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0},
- {ML_SOCKET_ID, 1, 0, 0}, {ML_DEBUG, 0, 0, 0},
- {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
+static struct option lgopts[] = {
+ {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0}, {ML_MODELS, 1, 0, 0},
+ {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
static int
ml_opts_parse_long(int opt_idx, struct ml_options *opt)
@@ -98,6 +136,7 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
{ML_TEST, ml_parse_test_name},
{ML_DEVICE_ID, ml_parse_dev_id},
{ML_SOCKET_ID, ml_parse_socket_id},
+ {ML_MODELS, ml_parse_models},
};
for (i = 0; i < RTE_DIM(parsermap); i++) {
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index 05311a9a47..8faf3b5deb 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -5,22 +5,31 @@
#ifndef _ML_OPTIONS_
#define _ML_OPTIONS_
+#include <linux/limits.h>
#include <stdbool.h>
#include <stdint.h>
#define ML_TEST_NAME_MAX_LEN 32
+#define ML_TEST_MAX_MODELS 8
/* Options names */
#define ML_TEST ("test")
#define ML_DEVICE_ID ("dev_id")
#define ML_SOCKET_ID ("socket_id")
+#define ML_MODELS ("models")
#define ML_DEBUG ("debug")
#define ML_HELP ("help")
+struct ml_filelist {
+ char model[PATH_MAX];
+};
+
struct ml_options {
char test_name[ML_TEST_NAME_MAX_LEN];
int16_t dev_id;
int socket_id;
+ struct ml_filelist filelist[ML_TEST_MAX_MODELS];
+ uint8_t nb_filelist;
bool debug;
};
diff --git a/app/test-mldev/test_model_common.c b/app/test-mldev/test_model_common.c
new file mode 100644
index 0000000000..5368be17fe
--- /dev/null
+++ b/app/test-mldev/test_model_common.c
@@ -0,0 +1,162 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <errno.h>
+#include <stdio.h>
+
+#include <rte_common.h>
+#include <rte_malloc.h>
+#include <rte_mldev.h>
+
+#include "ml_common.h"
+#include "ml_options.h"
+#include "ml_test.h"
+#include "test_common.h"
+#include "test_model_common.h"
+
+int
+ml_model_load(struct ml_test *test, struct ml_options *opt, struct ml_model *model, int16_t fid)
+{
+ struct test_common *t = ml_test_priv(test);
+ struct rte_ml_model_params model_params;
+ FILE *fp;
+ int ret;
+
+ if (model->state == MODEL_LOADED)
+ return 0;
+
+ if (model->state != MODEL_INITIAL)
+ return -EINVAL;
+
+ /* read model binary */
+ fp = fopen(opt->filelist[fid].model, "r");
+ if (fp == NULL) {
+ ml_err("Failed to open model file : %s\n", opt->filelist[fid].model);
+ return -1;
+ }
+
+ fseek(fp, 0, SEEK_END);
+ model_params.size = ftell(fp);
+ fseek(fp, 0, SEEK_SET);
+
+ model_params.addr = rte_malloc_socket("ml_model", model_params.size,
+ t->dev_info.min_align_size, opt->socket_id);
+ if (model_params.addr == NULL) {
+ ml_err("Failed to allocate memory for model: %s\n", opt->filelist[fid].model);
+ fclose(fp);
+ return -ENOMEM;
+ }
+
+ if (fread(model_params.addr, 1, model_params.size, fp) != model_params.size) {
+ ml_err("Failed to read model file : %s\n", opt->filelist[fid].model);
+ rte_free(model_params.addr);
+ fclose(fp);
+ return -1;
+ }
+ fclose(fp);
+
+ /* load model to device */
+ ret = rte_ml_model_load(opt->dev_id, &model_params, &model->id);
+ if (ret != 0) {
+ ml_err("Failed to load model : %s\n", opt->filelist[fid].model);
+ model->state = MODEL_ERROR;
+ rte_free(model_params.addr);
+ return ret;
+ }
+
+ /* release mz */
+ rte_free(model_params.addr);
+
+ /* get model info */
+ ret = rte_ml_model_info_get(opt->dev_id, model->id, &model->info);
+ if (ret != 0) {
+ ml_err("Failed to get model info : %s\n", opt->filelist[fid].model);
+ return ret;
+ }
+
+ model->state = MODEL_LOADED;
+
+ return 0;
+}
+
+int
+ml_model_unload(struct ml_test *test, struct ml_options *opt, struct ml_model *model, int16_t fid)
+{
+ struct test_common *t = ml_test_priv(test);
+ int ret;
+
+ RTE_SET_USED(t);
+
+ if (model->state == MODEL_INITIAL)
+ return 0;
+
+ if (model->state != MODEL_LOADED)
+ return -EINVAL;
+
+ /* unload model */
+ ret = rte_ml_model_unload(opt->dev_id, model->id);
+ if (ret != 0) {
+ ml_err("Failed to unload model: %s\n", opt->filelist[fid].model);
+ model->state = MODEL_ERROR;
+ return ret;
+ }
+
+ model->state = MODEL_INITIAL;
+
+ return 0;
+}
+
+int
+ml_model_start(struct ml_test *test, struct ml_options *opt, struct ml_model *model, int16_t fid)
+{
+ struct test_common *t = ml_test_priv(test);
+ int ret;
+
+ RTE_SET_USED(t);
+
+ if (model->state == MODEL_STARTED)
+ return 0;
+
+ if (model->state != MODEL_LOADED)
+ return -EINVAL;
+
+ /* start model */
+ ret = rte_ml_model_start(opt->dev_id, model->id);
+ if (ret != 0) {
+ ml_err("Failed to start model : %s\n", opt->filelist[fid].model);
+ model->state = MODEL_ERROR;
+ return ret;
+ }
+
+ model->state = MODEL_STARTED;
+
+ return 0;
+}
+
+int
+ml_model_stop(struct ml_test *test, struct ml_options *opt, struct ml_model *model, int16_t fid)
+{
+ struct test_common *t = ml_test_priv(test);
+ int ret;
+
+ RTE_SET_USED(t);
+
+ if (model->state == MODEL_LOADED)
+ return 0;
+
+ if (model->state != MODEL_STARTED)
+ return -EINVAL;
+
+ /* stop model */
+ ret = rte_ml_model_stop(opt->dev_id, model->id);
+ if (ret != 0) {
+ ml_err("Failed to stop model: %s\n", opt->filelist[fid].model);
+ model->state = MODEL_ERROR;
+ return ret;
+ }
+
+ model->state = MODEL_LOADED;
+
+ return 0;
+}
diff --git a/app/test-mldev/test_model_common.h b/app/test-mldev/test_model_common.h
new file mode 100644
index 0000000000..302e4eb45f
--- /dev/null
+++ b/app/test-mldev/test_model_common.h
@@ -0,0 +1,37 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_TEST_MODEL_COMMON_
+#define _ML_TEST_MODEL_COMMON_
+
+#include <stdint.h>
+
+#include <rte_mldev.h>
+
+#include "ml_options.h"
+#include "ml_test.h"
+
+enum model_state {
+ MODEL_INITIAL,
+ MODEL_LOADED,
+ MODEL_STARTED,
+ MODEL_ERROR,
+};
+
+struct ml_model {
+ int16_t id;
+ struct rte_ml_model_info info;
+ enum model_state state;
+};
+
+int ml_model_load(struct ml_test *test, struct ml_options *opt, struct ml_model *model,
+ int16_t fid);
+int ml_model_unload(struct ml_test *test, struct ml_options *opt, struct ml_model *model,
+ int16_t fid);
+int ml_model_start(struct ml_test *test, struct ml_options *opt, struct ml_model *model,
+ int16_t fid);
+int ml_model_stop(struct ml_test *test, struct ml_options *opt, struct ml_model *model,
+ int16_t fid);
+
+#endif /* _ML_TEST_MODEL_COMMON_ */
diff --git a/app/test-mldev/test_model_ops.c b/app/test-mldev/test_model_ops.c
new file mode 100644
index 0000000000..69c9df8ed6
--- /dev/null
+++ b/app/test-mldev/test_model_ops.c
@@ -0,0 +1,433 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <stdint.h>
+#include <stdio.h>
+#include <unistd.h>
+
+#include <rte_common.h>
+#include <rte_malloc.h>
+#include <rte_mldev.h>
+
+#include "ml_common.h"
+#include "ml_options.h"
+#include "ml_test.h"
+#include "test_model_ops.h"
+
+static bool
+test_model_ops_cap_check(struct ml_options *opt)
+{
+ if (!ml_test_cap_check(opt))
+ return false;
+
+ return true;
+}
+
+static int
+test_model_ops_opt_check(struct ml_options *opt)
+{
+ uint32_t i;
+ int ret;
+
+ /* check common opts */
+ ret = ml_test_opt_check(opt);
+ if (ret != 0)
+ return ret;
+
+ /* check model file availability */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ if (access(opt->filelist[i].model, F_OK) == -1) {
+ ml_err("Model file not available: id = %u, file = %s", i,
+ opt->filelist[i].model);
+ return -ENOENT;
+ }
+ }
+
+ return 0;
+}
+
+static void
+test_model_ops_opt_dump(struct ml_options *opt)
+{
+ uint32_t i;
+
+ /* dump common opts */
+ ml_test_opt_dump(opt);
+
+ /* dump test specific opts */
+ ml_dump_begin("models");
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_dump_list("model", i, opt->filelist[i].model);
+ ml_dump_end;
+}
+
+static int
+test_model_ops_setup(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+ void *test_model_ops;
+ int ret = 0;
+ uint32_t i;
+
+ /* allocate model ops test structure */
+ test_model_ops = rte_zmalloc_socket(test->name, sizeof(struct test_model_ops),
+ RTE_CACHE_LINE_SIZE, opt->socket_id);
+ if (test_model_ops == NULL) {
+ ml_err("Failed to allocate memory for test_model");
+ ret = -ENOMEM;
+ goto error;
+ }
+ test->test_priv = test_model_ops;
+ t = ml_test_priv(test);
+
+ t->cmn.result = ML_TEST_FAILED;
+ t->cmn.opt = opt;
+
+ /* get device info */
+ ret = rte_ml_dev_info_get(opt->dev_id, &t->cmn.dev_info);
+ if (ret < 0) {
+ ml_err("Failed to get device info");
+ goto error;
+ }
+
+ /* set model initial state */
+ for (i = 0; i < opt->nb_filelist; i++)
+ t->model[i].state = MODEL_INITIAL;
+
+ return 0;
+
+error:
+ if (test_model_ops != NULL)
+ rte_free(test_model_ops);
+
+ return ret;
+}
+
+static void
+test_model_ops_destroy(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+ if (t != NULL)
+ rte_free(t);
+}
+
+static int
+test_model_ops_mldev_setup(struct ml_test *test, struct ml_options *opt)
+{
+ int ret;
+
+ ret = ml_test_device_configure(test, opt);
+ if (ret != 0)
+ return ret;
+
+ ret = ml_test_device_start(test, opt);
+ if (ret != 0)
+ goto error;
+
+ return 0;
+
+error:
+ ml_test_device_close(test, opt);
+
+ return ret;
+}
+
+static int
+test_model_ops_mldev_destroy(struct ml_test *test, struct ml_options *opt)
+{
+ int ret;
+
+ ret = ml_test_device_stop(test, opt);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_test_device_close(test, opt);
+ if (ret != 0)
+ return ret;
+
+ return 0;
+
+error:
+ ml_test_device_close(test, opt);
+
+ return ret;
+}
+
+/* Sub-test A: (load -> start -> stop -> unload) x n */
+static int
+test_model_ops_subtest_a(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+ int ret = 0;
+ uint32_t i;
+
+ t = ml_test_priv(test);
+
+ /* load + start + stop + unload */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_load(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_start(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_stop(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_unload(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+error:
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_stop(test, opt, &t->model[i], i);
+
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_unload(test, opt, &t->model[i], i);
+
+ return ret;
+}
+
+/* Sub-test B: load x n -> start x n -> stop x n -> unload x n */
+static int
+test_model_ops_subtest_b(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+ int ret = 0;
+ uint32_t i;
+
+ t = ml_test_priv(test);
+
+ /* load */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_load(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* start */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_start(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* stop */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_stop(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* unload */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_unload(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ return 0;
+
+error:
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_stop(test, opt, &t->model[i], i);
+
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_unload(test, opt, &t->model[i], i);
+
+ return ret;
+}
+
+/* Sub-test C: load x n + (start + stop) x n + unload x n */
+static int
+test_model_ops_subtest_c(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+ int ret = 0;
+ uint32_t i;
+
+ t = ml_test_priv(test);
+
+ /* load */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_load(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* start + stop */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_start(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_stop(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* unload */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_unload(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ return 0;
+
+error:
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_stop(test, opt, &t->model[i], i);
+
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_unload(test, opt, &t->model[i], i);
+
+ return ret;
+}
+
+/* Sub-test D: (load + start) x n -> (stop + unload) x n */
+static int
+test_model_ops_subtest_d(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+ int ret = 0;
+ uint32_t i;
+
+ t = ml_test_priv(test);
+
+ /* load + start */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_load(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_start(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* stop + unload */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_stop(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_unload(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ return 0;
+
+error:
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_stop(test, opt, &t->model[i], i);
+
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_unload(test, opt, &t->model[i], i);
+
+ return ret;
+}
+
+static int
+test_model_ops_driver(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+ int ret = 0;
+
+ t = ml_test_priv(test);
+
+ /* device setup */
+ ret = test_model_ops_mldev_setup(test, opt);
+ if (ret != 0)
+ return ret;
+
+ printf("\n");
+
+ /* sub-test A */
+ ret = test_model_ops_subtest_a(test, opt);
+ if (ret != 0) {
+ printf("Model Ops Sub-test A: " CLRED "%s" CLNRM "\n", "Failed");
+ goto error;
+ } else {
+ printf("Model Ops Sub-test A: " CLYEL "%s" CLNRM "\n", "Passed");
+ }
+
+ /* sub-test B */
+ ret = test_model_ops_subtest_b(test, opt);
+ if (ret != 0) {
+ printf("Model Ops Sub-test B: " CLRED "%s" CLNRM "\n", "Failed");
+ goto error;
+ } else {
+ printf("Model Ops Sub-test B: " CLYEL "%s" CLNRM "\n", "Passed");
+ }
+
+ /* sub-test C */
+ ret = test_model_ops_subtest_c(test, opt);
+ if (ret != 0) {
+ printf("Model Ops Sub-test C: " CLRED "%s" CLNRM "\n", "Failed");
+ goto error;
+ } else {
+ printf("Model Ops Sub-test C: " CLYEL "%s" CLNRM "\n", "Passed");
+ }
+
+ /* sub-test D */
+ ret = test_model_ops_subtest_d(test, opt);
+ if (ret != 0) {
+ printf("Model Ops Sub-test D: " CLRED "%s" CLNRM "\n", "Failed");
+ goto error;
+ } else {
+ printf("Model Ops Sub-test D: " CLYEL "%s" CLNRM "\n", "Passed");
+ }
+
+ printf("\n");
+
+ /* device destroy */
+ ret = test_model_ops_mldev_destroy(test, opt);
+ if (ret != 0)
+ return ret;
+
+ t->cmn.result = ML_TEST_SUCCESS;
+
+ return 0;
+
+error:
+ test_model_ops_mldev_destroy(test, opt);
+
+ t->cmn.result = ML_TEST_FAILED;
+
+ return ret;
+}
+
+static int
+test_model_ops_result(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+
+ return t->cmn.result;
+}
+
+static const struct ml_test_ops model_ops = {
+ .cap_check = test_model_ops_cap_check,
+ .opt_check = test_model_ops_opt_check,
+ .opt_dump = test_model_ops_opt_dump,
+ .test_setup = test_model_ops_setup,
+ .test_destroy = test_model_ops_destroy,
+ .test_driver = test_model_ops_driver,
+ .test_result = test_model_ops_result,
+};
+
+ML_TEST_REGISTER(model_ops);
diff --git a/app/test-mldev/test_model_ops.h b/app/test-mldev/test_model_ops.h
new file mode 100644
index 0000000000..9dd8402390
--- /dev/null
+++ b/app/test-mldev/test_model_ops.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_TEST_MODEL_OPS_
+#define _ML_TEST_MODEL_OPS_
+
+#include <rte_common.h>
+
+#include "test_common.h"
+#include "test_model_common.h"
+
+struct test_model_ops {
+ /* common data */
+ struct test_common cmn;
+
+ /* test specific data */
+ struct ml_model model[ML_TEST_MAX_MODELS];
+} __rte_cache_aligned;
+
+#endif /* _ML_TEST_MODEL_OPS_ */
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v1 05/12] app/mldev: add ordered inference test case
2022-11-29 7:07 [PATCH v1 00/12] implement mldev test application Srikanth Yalavarthi
` (3 preceding siblings ...)
2022-11-29 7:07 ` [PATCH v1 04/12] app/mldev: add test case to validate model ops Srikanth Yalavarthi
@ 2022-11-29 7:07 ` Srikanth Yalavarthi
2022-11-29 7:07 ` [PATCH v1 06/12] app/mldev: add test case to interleave inferences Srikanth Yalavarthi
` (10 subsequent siblings)
15 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2022-11-29 7:07 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj
Added an ordered test case to execute inferences with single
or multiple models. In this test case inference requests for
a model are enqueued after completion of all requests for
the previous model. Test supports inference repetitions.
Operations sequence when testing with N models and R reps,
(load -> start -> (enqueue + dequeue) x R -> stop -> unload) x N
Test case can be executed by selecting "inference_ordered" test
and repetitions can be specified through "--repetitions" argument.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/meson.build | 2 +
app/test-mldev/ml_options.c | 73 ++-
app/test-mldev/ml_options.h | 17 +-
app/test-mldev/test_inference_common.c | 565 ++++++++++++++++++++++++
app/test-mldev/test_inference_common.h | 65 +++
app/test-mldev/test_inference_ordered.c | 119 +++++
app/test-mldev/test_model_common.h | 10 +
7 files changed, 839 insertions(+), 12 deletions(-)
create mode 100644 app/test-mldev/test_inference_common.c
create mode 100644 app/test-mldev/test_inference_common.h
create mode 100644 app/test-mldev/test_inference_ordered.c
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
index b09e1ccc8a..475d76d126 100644
--- a/app/test-mldev/meson.build
+++ b/app/test-mldev/meson.build
@@ -16,6 +16,8 @@ sources = files(
'test_device_ops.c',
'test_model_common.c',
'test_model_ops.c',
+ 'test_inference_common.c',
+ 'test_inference_ordered.c',
)
deps += ['mldev']
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 8e40a33ed0..59a5d16584 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -29,6 +29,7 @@ ml_options_default(struct ml_options *opt)
opt->dev_id = 0;
opt->socket_id = SOCKET_ID_ANY;
opt->nb_filelist = 0;
+ opt->repetitions = 1;
opt->debug = false;
}
@@ -96,6 +97,60 @@ ml_parse_models(struct ml_options *opt, const char *arg)
return ret;
}
+static int
+ml_parse_filelist(struct ml_options *opt, const char *arg)
+{
+ const char *delim = ",";
+ char filelist[PATH_MAX];
+ char *token;
+
+ if (opt->nb_filelist >= ML_TEST_MAX_MODELS) {
+ ml_err("Exceeded filelist count, max = %d\n", ML_TEST_MAX_MODELS);
+ return -1;
+ }
+
+ strlcpy(filelist, arg, PATH_MAX);
+
+ /* model */
+ token = strtok(filelist, delim);
+ if (token == NULL) {
+ ml_err("Invalid filelist, model not specified = %s\n", arg);
+ return -EINVAL;
+ }
+ strlcpy(opt->filelist[opt->nb_filelist].model, token, PATH_MAX);
+
+ /* input */
+ token = strtok(NULL, delim);
+ if (token == NULL) {
+ ml_err("Invalid filelist, input not specified = %s\n", arg);
+ return -EINVAL;
+ }
+ strlcpy(opt->filelist[opt->nb_filelist].input, token, PATH_MAX);
+
+ /* output */
+ token = strtok(NULL, delim);
+ if (token == NULL) {
+ ml_err("Invalid filelist, output not specified = %s\n", arg);
+ return -EINVAL;
+ }
+ strlcpy(opt->filelist[opt->nb_filelist].output, token, PATH_MAX);
+
+ opt->nb_filelist++;
+
+ if (opt->nb_filelist == 0) {
+ ml_err("Empty filelist. Need atleast one filelist entry for the test.");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int
+ml_parse_repetitions(struct ml_options *opt, const char *arg)
+{
+ return parser_read_uint64(&opt->repetitions, arg);
+}
+
static void
ml_dump_test_options(const char *testname)
{
@@ -106,6 +161,12 @@ ml_dump_test_options(const char *testname)
printf("\t\t--models : comma separated list of models\n");
printf("\n");
}
+
+ if (strcmp(testname, "inference_ordered") == 0) {
+ printf("\t\t--filelist : comma separated list of model, input and output\n"
+ "\t\t--repetitions : number of inference repetitions\n");
+ printf("\n");
+ }
}
static void
@@ -124,8 +185,9 @@ print_usage(char *program)
}
static struct option lgopts[] = {
- {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0}, {ML_MODELS, 1, 0, 0},
- {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
+ {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
+ {ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
+ {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
static int
ml_opts_parse_long(int opt_idx, struct ml_options *opt)
@@ -133,10 +195,9 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
unsigned int i;
struct long_opt_parser parsermap[] = {
- {ML_TEST, ml_parse_test_name},
- {ML_DEVICE_ID, ml_parse_dev_id},
- {ML_SOCKET_ID, ml_parse_socket_id},
- {ML_MODELS, ml_parse_models},
+ {ML_TEST, ml_parse_test_name}, {ML_DEVICE_ID, ml_parse_dev_id},
+ {ML_SOCKET_ID, ml_parse_socket_id}, {ML_MODELS, ml_parse_models},
+ {ML_FILELIST, ml_parse_filelist}, {ML_REPETITIONS, ml_parse_repetitions},
};
for (i = 0; i < RTE_DIM(parsermap); i++) {
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index 8faf3b5deb..ad8aee5964 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -13,15 +13,19 @@
#define ML_TEST_MAX_MODELS 8
/* Options names */
-#define ML_TEST ("test")
-#define ML_DEVICE_ID ("dev_id")
-#define ML_SOCKET_ID ("socket_id")
-#define ML_MODELS ("models")
-#define ML_DEBUG ("debug")
-#define ML_HELP ("help")
+#define ML_TEST ("test")
+#define ML_DEVICE_ID ("dev_id")
+#define ML_SOCKET_ID ("socket_id")
+#define ML_MODELS ("models")
+#define ML_FILELIST ("filelist")
+#define ML_REPETITIONS ("repetitions")
+#define ML_DEBUG ("debug")
+#define ML_HELP ("help")
struct ml_filelist {
char model[PATH_MAX];
+ char input[PATH_MAX];
+ char output[PATH_MAX];
};
struct ml_options {
@@ -30,6 +34,7 @@ struct ml_options {
int socket_id;
struct ml_filelist filelist[ML_TEST_MAX_MODELS];
uint8_t nb_filelist;
+ uint64_t repetitions;
bool debug;
};
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
new file mode 100644
index 0000000000..e5e300ffdc
--- /dev/null
+++ b/app/test-mldev/test_inference_common.c
@@ -0,0 +1,565 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <errno.h>
+#include <stdbool.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <unistd.h>
+
+#include <rte_common.h>
+#include <rte_launch.h>
+#include <rte_lcore.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_mldev.h>
+
+#include "ml_common.h"
+#include "ml_options.h"
+#include "ml_test.h"
+#include "test_common.h"
+#include "test_inference_common.h"
+
+/* Enqueue inference requests with burst size equal to 1 */
+static int
+ml_enqueue_single(void *arg)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)arg);
+ struct ml_request *req = NULL;
+ struct rte_ml_op *op = NULL;
+ struct ml_core_args *args;
+ uint64_t model_enq = 0;
+ uint32_t burst_enq;
+ uint32_t lcore_id;
+ int16_t fid;
+ int ret;
+
+ lcore_id = rte_lcore_id();
+ args = &t->args[lcore_id];
+ model_enq = 0;
+
+ if (args->nb_reqs == 0)
+ return 0;
+
+next_rep:
+ fid = args->start_fid;
+
+next_model:
+ ret = rte_mempool_get(t->op_pool, (void **)&op);
+ if (ret != 0)
+ goto next_model;
+
+retry:
+ ret = rte_mempool_get(t->model[fid].io_pool, (void **)&req);
+ if (ret != 0)
+ goto retry;
+
+ op->model_id = t->model[fid].id;
+ op->nb_batches = t->model[fid].info.batch_size;
+ op->mempool = t->op_pool;
+
+ op->input.addr = req->input;
+ op->input.length = t->model[fid].inp_qsize;
+ op->input.next = NULL;
+
+ op->output.addr = req->output;
+ op->output.length = t->model[fid].out_qsize;
+ op->output.next = NULL;
+
+ op->user_ptr = req;
+ req->niters++;
+ req->fid = fid;
+
+enqueue_req:
+ burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, 0, &op, 1);
+ if (burst_enq == 0)
+ goto enqueue_req;
+
+ fid++;
+ if (likely(fid <= args->end_fid))
+ goto next_model;
+
+ model_enq++;
+ if (likely(model_enq < args->nb_reqs))
+ goto next_rep;
+
+ return 0;
+}
+
+/* Dequeue inference requests with burst size equal to 1 */
+static int
+ml_dequeue_single(void *arg)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)arg);
+ struct rte_ml_op_error error;
+ struct rte_ml_op *op = NULL;
+ struct ml_core_args *args;
+ struct ml_request *req;
+ uint64_t total_deq = 0;
+ uint8_t nb_filelist;
+ uint32_t burst_deq;
+ uint32_t lcore_id;
+
+ lcore_id = rte_lcore_id();
+ args = &t->args[lcore_id];
+ nb_filelist = args->end_fid - args->start_fid + 1;
+
+ if (args->nb_reqs == 0)
+ return 0;
+
+dequeue_req:
+ burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, 0, &op, 1);
+
+ if (likely(burst_deq == 1)) {
+ total_deq += burst_deq;
+ if (unlikely(op->status == RTE_ML_OP_STATUS_ERROR)) {
+ rte_ml_op_error_get(t->cmn.opt->dev_id, op, &error);
+ ml_err("error_code = 0x%016lx, error_message = %s\n", error.errcode,
+ error.message);
+ }
+ req = (struct ml_request *)op->user_ptr;
+ rte_mempool_put(t->model[req->fid].io_pool, req);
+ rte_mempool_put(t->op_pool, op);
+ }
+
+ if (likely(total_deq < args->nb_reqs * nb_filelist))
+ goto dequeue_req;
+
+ return 0;
+}
+
+bool
+test_inference_cap_check(struct ml_options *opt)
+{
+ struct rte_ml_dev_info dev_info;
+
+ if (!ml_test_cap_check(opt))
+ return false;
+
+ rte_ml_dev_info_get(opt->dev_id, &dev_info);
+ if (opt->nb_filelist > dev_info.max_models) {
+ ml_err("Insufficient capabilities: Filelist count exceeded device limit, count = %u (max limit = %u)",
+ opt->nb_filelist, dev_info.max_models);
+ return false;
+ }
+
+ return true;
+}
+
+int
+test_inference_opt_check(struct ml_options *opt)
+{
+ uint32_t i;
+ int ret;
+
+ /* check common opts */
+ ret = ml_test_opt_check(opt);
+ if (ret != 0)
+ return ret;
+
+ /* check file availability */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ if (access(opt->filelist[i].model, F_OK) == -1) {
+ ml_err("Model file not accessible: id = %u, file = %s", i,
+ opt->filelist[i].model);
+ return -ENOENT;
+ }
+
+ if (access(opt->filelist[i].input, F_OK) == -1) {
+ ml_err("Input file not accessible: id = %u, file = %s", i,
+ opt->filelist[i].input);
+ return -ENOENT;
+ }
+ }
+
+ if (opt->repetitions == 0) {
+ ml_err("Invalid option, repetitions = %" PRIu64 "\n", opt->repetitions);
+ return -EINVAL;
+ }
+
+ /* check number of available lcores. */
+ if (rte_lcore_count() < 3) {
+ ml_err("Insufficient lcores = %u\n", rte_lcore_count());
+ ml_err("Minimum lcores required to create %u queue-pairs = %u\n", 1, 3);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+void
+test_inference_opt_dump(struct ml_options *opt)
+{
+ uint32_t i;
+
+ /* dump common opts */
+ ml_test_opt_dump(opt);
+
+ /* dump test opts */
+ ml_dump("repetitions", "%" PRIu64, opt->repetitions);
+
+ ml_dump_begin("filelist");
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ml_dump_list("model", i, opt->filelist[i].model);
+ ml_dump_list("input", i, opt->filelist[i].input);
+ ml_dump_list("output", i, opt->filelist[i].output);
+ }
+ ml_dump_end;
+}
+
+int
+test_inference_setup(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t;
+ void *test_inference;
+ int ret = 0;
+ uint32_t i;
+
+ test_inference = rte_zmalloc_socket(test->name, sizeof(struct test_inference),
+ RTE_CACHE_LINE_SIZE, opt->socket_id);
+ if (test_inference == NULL) {
+ ml_err("failed to allocate memory for test_model");
+ ret = -ENOMEM;
+ goto error;
+ }
+ test->test_priv = test_inference;
+ t = ml_test_priv(test);
+
+ t->nb_used = 0;
+ t->cmn.result = ML_TEST_FAILED;
+ t->cmn.opt = opt;
+
+ /* get device info */
+ ret = rte_ml_dev_info_get(opt->dev_id, &t->cmn.dev_info);
+ if (ret < 0) {
+ ml_err("failed to get device info");
+ goto error;
+ }
+
+ t->enqueue = ml_enqueue_single;
+ t->dequeue = ml_dequeue_single;
+
+ /* set model initial state */
+ for (i = 0; i < opt->nb_filelist; i++)
+ t->model[i].state = MODEL_INITIAL;
+
+ return 0;
+
+error:
+ if (test_inference != NULL)
+ rte_free(test_inference);
+
+ return ret;
+}
+
+void
+test_inference_destroy(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+ if (t != NULL)
+ rte_free(t);
+}
+
+int
+ml_inference_mldev_setup(struct ml_test *test, struct ml_options *opt)
+{
+ struct rte_ml_dev_qp_conf qp_conf;
+ struct test_inference *t;
+ int ret;
+
+ t = ml_test_priv(test);
+
+ ret = ml_test_device_configure(test, opt);
+ if (ret != 0)
+ return ret;
+
+ /* setup queue pairs */
+ qp_conf.nb_desc = t->cmn.dev_info.max_desc;
+ qp_conf.cb = NULL;
+
+ ret = rte_ml_dev_queue_pair_setup(opt->dev_id, 0, &qp_conf, opt->socket_id);
+ if (ret != 0) {
+ ml_err("Failed to setup ml device queue-pair, dev_id = %d, qp_id = %u\n",
+ opt->dev_id, 0);
+ goto error;
+ }
+
+ ret = ml_test_device_start(test, opt);
+ if (ret != 0)
+ goto error;
+
+ return 0;
+
+error:
+ ml_test_device_close(test, opt);
+
+ return ret;
+}
+
+int
+ml_inference_mldev_destroy(struct ml_test *test, struct ml_options *opt)
+{
+ int ret;
+
+ ret = ml_test_device_stop(test, opt);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_test_device_close(test, opt);
+ if (ret != 0)
+ return ret;
+
+ return 0;
+
+error:
+ ml_test_device_close(test, opt);
+
+ return ret;
+}
+
+/* Callback for IO pool create. This function would compute the fields of ml_request
+ * structure and prepare the quantized input data.
+ */
+static void
+ml_request_initialize(struct rte_mempool *mp, void *opaque, void *obj, unsigned int obj_idx)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)opaque);
+ struct ml_request *req = (struct ml_request *)obj;
+
+ RTE_SET_USED(mp);
+ RTE_SET_USED(obj_idx);
+
+ req->input = RTE_PTR_ADD(
+ obj, RTE_ALIGN_CEIL(sizeof(struct ml_request), t->cmn.dev_info.min_align_size));
+ req->output = RTE_PTR_ADD(req->input, RTE_ALIGN_CEIL(t->model[t->fid].inp_qsize,
+ t->cmn.dev_info.min_align_size));
+ req->niters = 0;
+
+ /* quantize data */
+ rte_ml_io_quantize(t->cmn.opt->dev_id, t->model[t->fid].id,
+ t->model[t->fid].info.batch_size, t->model[t->fid].input, req->input);
+}
+
+int
+ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, int16_t fid)
+{
+ struct test_inference *t = ml_test_priv(test);
+ char mz_name[RTE_MEMZONE_NAMESIZE];
+ char mp_name[RTE_MEMPOOL_NAMESIZE];
+ const struct rte_memzone *mz;
+ uint64_t nb_buffers;
+ uint32_t buff_size;
+ uint32_t mz_size;
+ uint32_t fsize;
+ FILE *fp;
+ int ret;
+
+ /* get input buffer size */
+ ret = rte_ml_io_input_size_get(opt->dev_id, t->model[fid].id, t->model[fid].info.batch_size,
+ &t->model[fid].inp_qsize, &t->model[fid].inp_dsize);
+ if (ret != 0) {
+ ml_err("Failed to get input size, model : %s\n", opt->filelist[fid].model);
+ return ret;
+ }
+
+ /* get output buffer size */
+ ret = rte_ml_io_output_size_get(opt->dev_id, t->model[fid].id,
+ t->model[fid].info.batch_size, &t->model[fid].out_qsize,
+ &t->model[fid].out_dsize);
+ if (ret != 0) {
+ ml_err("Failed to get input size, model : %s\n", opt->filelist[fid].model);
+ return ret;
+ }
+
+ /* allocate buffer for user data */
+ mz_size = t->model[fid].inp_dsize + t->model[fid].out_dsize;
+ sprintf(mz_name, "ml_user_data_%d", fid);
+ mz = rte_memzone_reserve(mz_name, mz_size, opt->socket_id, 0);
+ if (mz == NULL) {
+ ml_err("Memzone allocation failed for ml_user_data\n");
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ t->model[fid].input = mz->addr;
+ t->model[fid].output = RTE_PTR_ADD(t->model[fid].input, t->model[fid].inp_dsize);
+
+ /* load input file */
+ fp = fopen(opt->filelist[fid].input, "r");
+ if (fp == NULL) {
+ ml_err("Failed to open input file : %s\n", opt->filelist[fid].input);
+ ret = -errno;
+ goto error;
+ }
+
+ fseek(fp, 0, SEEK_END);
+ fsize = ftell(fp);
+ fseek(fp, 0, SEEK_SET);
+ if (fsize != t->model[fid].inp_dsize) {
+ ml_err("Invalid input file, size = %u (expected size = %" PRIu64 ")\n", fsize,
+ t->model[fid].inp_dsize);
+ ret = -EINVAL;
+ fclose(fp);
+ goto error;
+ }
+
+ if (fread(t->model[fid].input, 1, t->model[fid].inp_dsize, fp) != t->model[fid].inp_dsize) {
+ ml_err("Failed to read input file : %s\n", opt->filelist[fid].input);
+ ret = -errno;
+ fclose(fp);
+ goto error;
+ }
+ fclose(fp);
+
+ /* create mempool for quantized input and output buffers. ml_request_initialize is
+ * used as a callback for object creation.
+ */
+ buff_size = RTE_ALIGN_CEIL(sizeof(struct ml_request), t->cmn.dev_info.min_align_size) +
+ RTE_ALIGN_CEIL(t->model[fid].inp_qsize, t->cmn.dev_info.min_align_size) +
+ RTE_ALIGN_CEIL(t->model[fid].out_qsize, t->cmn.dev_info.min_align_size);
+ nb_buffers = RTE_MIN((uint64_t)ML_TEST_MAX_POOL_SIZE, opt->repetitions);
+
+ t->fid = fid;
+ sprintf(mp_name, "ml_io_pool_%d", fid);
+ t->model[fid].io_pool = rte_mempool_create(mp_name, nb_buffers, buff_size, 0, 0, NULL, NULL,
+ ml_request_initialize, test, opt->socket_id, 0);
+ if (t->model[fid].io_pool == NULL) {
+ ml_err("Failed to create io pool : %s\n", "ml_io_pool");
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ return 0;
+
+error:
+ if (mz != NULL)
+ rte_memzone_free(mz);
+
+ if (t->model[fid].io_pool != NULL) {
+ rte_mempool_free(t->model[fid].io_pool);
+ t->model[fid].io_pool = NULL;
+ }
+
+ return ret;
+}
+
+void
+ml_inference_iomem_destroy(struct ml_test *test, struct ml_options *opt, int16_t fid)
+{
+ char mz_name[RTE_MEMZONE_NAMESIZE];
+ char mp_name[RTE_MEMPOOL_NAMESIZE];
+ const struct rte_memzone *mz;
+ struct rte_mempool *mp;
+
+ RTE_SET_USED(test);
+ RTE_SET_USED(opt);
+
+ /* release user data memzone */
+ sprintf(mz_name, "ml_user_data_%d", fid);
+ mz = rte_memzone_lookup(mz_name);
+ if (mz != NULL)
+ rte_memzone_free(mz);
+
+ /* destroy io pool */
+ sprintf(mp_name, "ml_io_pool_%d", fid);
+ mp = rte_mempool_lookup(mp_name);
+ if (mp != NULL)
+ rte_mempool_free(mp);
+}
+
+int
+ml_inference_mem_setup(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t = ml_test_priv(test);
+
+ /* create op pool */
+ t->op_pool = rte_ml_op_pool_create("ml_test_op_pool", ML_TEST_MAX_POOL_SIZE, 0, 0,
+ opt->socket_id);
+ if (t->op_pool == NULL) {
+ ml_err("Failed to create op pool : %s\n", "ml_op_pool");
+ return -ENOMEM;
+ }
+
+ return 0;
+}
+
+void
+ml_inference_mem_destroy(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t = ml_test_priv(test);
+
+ RTE_SET_USED(opt);
+
+ /* release op pool */
+ if (t->op_pool != NULL)
+ rte_mempool_free(t->op_pool);
+}
+
+/* Callback for mempool object iteration. This call would dequantize ouput data. */
+static void
+ml_request_finish(struct rte_mempool *mp, void *opaque, void *obj, unsigned int obj_idx)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)opaque);
+ struct ml_request *req = (struct ml_request *)obj;
+ struct ml_model *model = &t->model[req->fid];
+
+ RTE_SET_USED(mp);
+ RTE_SET_USED(obj_idx);
+
+ if (req->niters == 0)
+ return;
+
+ t->nb_used++;
+ rte_ml_io_dequantize(t->cmn.opt->dev_id, model->id, t->model[req->fid].info.batch_size,
+ req->output, model->output);
+}
+
+int
+ml_inference_result(struct ml_test *test, struct ml_options *opt, int16_t fid)
+{
+ struct test_inference *t = ml_test_priv(test);
+
+ RTE_SET_USED(opt);
+
+ rte_mempool_obj_iter(t->model[fid].io_pool, ml_request_finish, test);
+
+ if (t->nb_used > 0)
+ t->cmn.result = ML_TEST_SUCCESS;
+ else
+ t->cmn.result = ML_TEST_FAILED;
+
+ return t->cmn.result;
+}
+
+int
+ml_inference_launch_cores(struct ml_test *test, struct ml_options *opt, int16_t start_fid,
+ int16_t end_fid)
+{
+ struct test_inference *t = ml_test_priv(test);
+ uint32_t lcore_id;
+ uint32_t id = 0;
+
+ RTE_LCORE_FOREACH_WORKER(lcore_id)
+ {
+ if (id == 2)
+ break;
+
+ t->args[lcore_id].nb_reqs = opt->repetitions;
+ t->args[lcore_id].start_fid = start_fid;
+ t->args[lcore_id].end_fid = end_fid;
+
+ if (id % 2 == 0)
+ rte_eal_remote_launch(t->enqueue, test, lcore_id);
+ else
+ rte_eal_remote_launch(t->dequeue, test, lcore_id);
+
+ id++;
+ }
+
+ return 0;
+}
diff --git a/app/test-mldev/test_inference_common.h b/app/test-mldev/test_inference_common.h
new file mode 100644
index 0000000000..91007954b4
--- /dev/null
+++ b/app/test-mldev/test_inference_common.h
@@ -0,0 +1,65 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_TEST_INFERENCE_COMMON_
+#define _ML_TEST_INFERENCE_COMMON_
+
+#include <stdbool.h>
+#include <stdint.h>
+
+#include <rte_common.h>
+#include <rte_mempool.h>
+
+#include "ml_options.h"
+#include "ml_test.h"
+#include "test_common.h"
+#include "test_model_common.h"
+
+struct ml_request {
+ void *input;
+ void *output;
+ int16_t fid;
+ uint64_t niters;
+};
+
+struct ml_core_args {
+ uint64_t nb_reqs;
+ int16_t start_fid;
+ int16_t end_fid;
+};
+
+struct test_inference {
+ /* common data */
+ struct test_common cmn;
+
+ /* test specific data */
+ struct ml_model model[ML_TEST_MAX_MODELS];
+ struct rte_mempool *op_pool;
+
+ uint64_t nb_used;
+ int16_t fid;
+
+ int (*enqueue)(void *arg);
+ int (*dequeue)(void *arg);
+
+ struct ml_core_args args[RTE_MAX_LCORE];
+} __rte_cache_aligned;
+
+bool test_inference_cap_check(struct ml_options *opt);
+int test_inference_opt_check(struct ml_options *opt);
+void test_inference_opt_dump(struct ml_options *opt);
+int test_inference_setup(struct ml_test *test, struct ml_options *opt);
+void test_inference_destroy(struct ml_test *test, struct ml_options *opt);
+
+int ml_inference_mldev_setup(struct ml_test *test, struct ml_options *opt);
+int ml_inference_mldev_destroy(struct ml_test *test, struct ml_options *opt);
+int ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, int16_t fid);
+void ml_inference_iomem_destroy(struct ml_test *test, struct ml_options *opt, int16_t fid);
+int ml_inference_mem_setup(struct ml_test *test, struct ml_options *opt);
+void ml_inference_mem_destroy(struct ml_test *test, struct ml_options *opt);
+int ml_inference_result(struct ml_test *test, struct ml_options *opt, int16_t fid);
+int ml_inference_launch_cores(struct ml_test *test, struct ml_options *opt, int16_t start_fid,
+ int16_t end_fid);
+
+#endif /* _ML_TEST_INFERENCE_COMMON_ */
diff --git a/app/test-mldev/test_inference_ordered.c b/app/test-mldev/test_inference_ordered.c
new file mode 100644
index 0000000000..84e6bf9109
--- /dev/null
+++ b/app/test-mldev/test_inference_ordered.c
@@ -0,0 +1,119 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <stdint.h>
+
+#include <rte_common.h>
+#include <rte_launch.h>
+
+#include "ml_common.h"
+#include "ml_test.h"
+#include "test_inference_common.h"
+#include "test_model_common.h"
+
+static int
+test_inference_ordered_driver(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t;
+ int16_t fid = 0;
+ int ret = 0;
+
+ t = ml_test_priv(test);
+
+ ret = ml_inference_mldev_setup(test, opt);
+ if (ret != 0)
+ return ret;
+
+ ret = ml_inference_mem_setup(test, opt);
+ if (ret != 0)
+ return ret;
+
+next_model:
+ /* load model */
+ ret = ml_model_load(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ /* start model */
+ ret = ml_model_start(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_inference_iomem_setup(test, opt, fid);
+ if (ret != 0)
+ goto error;
+
+ /* launch inferences for one model using available queue pairs */
+ ret = ml_inference_launch_cores(test, opt, fid, fid);
+ if (ret != 0) {
+ ml_err("failed to launch cores");
+ goto error;
+ }
+
+ rte_eal_mp_wait_lcore();
+
+ ret = ml_inference_result(test, opt, fid);
+ if (ret != ML_TEST_SUCCESS)
+ goto error;
+
+ ml_inference_iomem_destroy(test, opt, fid);
+
+ /* stop model */
+ ret = ml_model_stop(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ /* unload model */
+ ret = ml_model_unload(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ fid++;
+ if (fid < opt->nb_filelist)
+ goto next_model;
+
+ ml_inference_mem_destroy(test, opt);
+
+ ret = ml_inference_mldev_destroy(test, opt);
+ if (ret != 0)
+ return ret;
+
+ t->cmn.result = ML_TEST_SUCCESS;
+
+ return 0;
+
+error:
+ ml_inference_iomem_destroy(test, opt, fid);
+ ml_inference_mem_destroy(test, opt);
+ ml_model_stop(test, opt, &t->model[fid], fid);
+ ml_model_unload(test, opt, &t->model[fid], fid);
+
+ t->cmn.result = ML_TEST_FAILED;
+
+ return ret;
+}
+
+static int
+test_inference_ordered_result(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+
+ return t->cmn.result;
+}
+
+static const struct ml_test_ops inference_ordered = {
+ .cap_check = test_inference_cap_check,
+ .opt_check = test_inference_opt_check,
+ .opt_dump = test_inference_opt_dump,
+ .test_setup = test_inference_setup,
+ .test_destroy = test_inference_destroy,
+ .test_driver = test_inference_ordered_driver,
+ .test_result = test_inference_ordered_result,
+};
+
+ML_TEST_REGISTER(inference_ordered);
diff --git a/app/test-mldev/test_model_common.h b/app/test-mldev/test_model_common.h
index 302e4eb45f..c45ae80853 100644
--- a/app/test-mldev/test_model_common.h
+++ b/app/test-mldev/test_model_common.h
@@ -23,6 +23,16 @@ struct ml_model {
int16_t id;
struct rte_ml_model_info info;
enum model_state state;
+
+ uint64_t inp_dsize;
+ uint64_t inp_qsize;
+ uint64_t out_dsize;
+ uint64_t out_qsize;
+
+ uint8_t *input;
+ uint8_t *output;
+
+ struct rte_mempool *io_pool;
};
int ml_model_load(struct ml_test *test, struct ml_options *opt, struct ml_model *model,
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v1 06/12] app/mldev: add test case to interleave inferences
2022-11-29 7:07 [PATCH v1 00/12] implement mldev test application Srikanth Yalavarthi
` (4 preceding siblings ...)
2022-11-29 7:07 ` [PATCH v1 05/12] app/mldev: add ordered inference test case Srikanth Yalavarthi
@ 2022-11-29 7:07 ` Srikanth Yalavarthi
2022-11-29 7:07 ` [PATCH v1 07/12] app/mldev: enable support for burst inferences Srikanth Yalavarthi
` (9 subsequent siblings)
15 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2022-11-29 7:07 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj
Added test case to interleave inference requests from multiple
models. Interleaving would load and start all models and launch
inference requests for the models using available queue-pairs
Operations sequence when testing with N models and R reps,
(load + start) x N -> (enqueue + dequeue) x N x R ...
-> (stop + unload) x N
Test can be executed by selecting "inference_interleave" test.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/meson.build | 1 +
app/test-mldev/ml_options.c | 3 +-
app/test-mldev/test_inference_common.c | 12 +--
app/test-mldev/test_inference_common.h | 4 +-
app/test-mldev/test_inference_interleave.c | 118 +++++++++++++++++++++
5 files changed, 129 insertions(+), 9 deletions(-)
create mode 100644 app/test-mldev/test_inference_interleave.c
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
index 475d76d126..41d22fb22c 100644
--- a/app/test-mldev/meson.build
+++ b/app/test-mldev/meson.build
@@ -18,6 +18,7 @@ sources = files(
'test_model_ops.c',
'test_inference_common.c',
'test_inference_ordered.c',
+ 'test_inference_interleave.c',
)
deps += ['mldev']
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 59a5d16584..9a006ff7c8 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -162,7 +162,8 @@ ml_dump_test_options(const char *testname)
printf("\n");
}
- if (strcmp(testname, "inference_ordered") == 0) {
+ if ((strcmp(testname, "inference_ordered") == 0) ||
+ (strcmp(testname, "inference_interleave") == 0)) {
printf("\t\t--filelist : comma separated list of model, input and output\n"
"\t\t--repetitions : number of inference repetitions\n");
printf("\n");
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
index e5e300ffdc..1e0e30637f 100644
--- a/app/test-mldev/test_inference_common.c
+++ b/app/test-mldev/test_inference_common.c
@@ -115,7 +115,7 @@ ml_dequeue_single(void *arg)
total_deq += burst_deq;
if (unlikely(op->status == RTE_ML_OP_STATUS_ERROR)) {
rte_ml_op_error_get(t->cmn.opt->dev_id, op, &error);
- ml_err("error_code = 0x%016lx, error_message = %s\n", error.errcode,
+ ml_err("error_code = 0x%" PRIx64 ", error_message = %s\n", error.errcode,
error.message);
}
req = (struct ml_request *)op->user_ptr;
@@ -334,10 +334,10 @@ ml_request_initialize(struct rte_mempool *mp, void *opaque, void *obj, unsigned
RTE_SET_USED(mp);
RTE_SET_USED(obj_idx);
- req->input = RTE_PTR_ADD(
- obj, RTE_ALIGN_CEIL(sizeof(struct ml_request), t->cmn.dev_info.min_align_size));
- req->output = RTE_PTR_ADD(req->input, RTE_ALIGN_CEIL(t->model[t->fid].inp_qsize,
- t->cmn.dev_info.min_align_size));
+ req->input = (uint8_t *)obj +
+ RTE_ALIGN_CEIL(sizeof(struct ml_request), t->cmn.dev_info.min_align_size);
+ req->output = req->input +
+ RTE_ALIGN_CEIL(t->model[t->fid].inp_qsize, t->cmn.dev_info.min_align_size);
req->niters = 0;
/* quantize data */
@@ -387,7 +387,7 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, int16_t f
}
t->model[fid].input = mz->addr;
- t->model[fid].output = RTE_PTR_ADD(t->model[fid].input, t->model[fid].inp_dsize);
+ t->model[fid].output = t->model[fid].input + t->model[fid].inp_dsize;
/* load input file */
fp = fopen(opt->filelist[fid].input, "r");
diff --git a/app/test-mldev/test_inference_common.h b/app/test-mldev/test_inference_common.h
index 91007954b4..b058abada4 100644
--- a/app/test-mldev/test_inference_common.h
+++ b/app/test-mldev/test_inference_common.h
@@ -17,8 +17,8 @@
#include "test_model_common.h"
struct ml_request {
- void *input;
- void *output;
+ uint8_t *input;
+ uint8_t *output;
int16_t fid;
uint64_t niters;
};
diff --git a/app/test-mldev/test_inference_interleave.c b/app/test-mldev/test_inference_interleave.c
new file mode 100644
index 0000000000..74ad0c597f
--- /dev/null
+++ b/app/test-mldev/test_inference_interleave.c
@@ -0,0 +1,118 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <stdint.h>
+
+#include <rte_common.h>
+#include <rte_launch.h>
+
+#include "ml_common.h"
+#include "ml_test.h"
+#include "test_inference_common.h"
+#include "test_model_common.h"
+
+static int
+test_inference_interleave_driver(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t;
+ int16_t fid = 0;
+ int ret = 0;
+
+ t = ml_test_priv(test);
+
+ ret = ml_inference_mldev_setup(test, opt);
+ if (ret != 0)
+ return ret;
+
+ ret = ml_inference_mem_setup(test, opt);
+ if (ret != 0)
+ return ret;
+
+ /* load and start all models */
+ for (fid = 0; fid < opt->nb_filelist; fid++) {
+ ret = ml_model_load(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_start(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_inference_iomem_setup(test, opt, fid);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* launch inference requests */
+ ret = ml_inference_launch_cores(test, opt, 0, opt->nb_filelist - 1);
+ if (ret != 0) {
+ ml_err("failed to launch cores");
+ goto error;
+ }
+
+ rte_eal_mp_wait_lcore();
+
+ /* stop and unload all models */
+ for (fid = 0; fid < opt->nb_filelist; fid++) {
+ ret = ml_inference_result(test, opt, fid);
+ if (ret != ML_TEST_SUCCESS)
+ goto error;
+
+ ml_inference_iomem_destroy(test, opt, fid);
+
+ ret = ml_model_stop(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_unload(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+ }
+
+ ml_inference_mem_destroy(test, opt);
+
+ ret = ml_inference_mldev_destroy(test, opt);
+ if (ret != 0)
+ return ret;
+
+ t->cmn.result = ML_TEST_SUCCESS;
+
+ return 0;
+
+error:
+ ml_inference_mem_destroy(test, opt);
+ for (fid = 0; fid < opt->nb_filelist; fid++) {
+ ml_inference_iomem_destroy(test, opt, fid);
+ ml_model_stop(test, opt, &t->model[fid], fid);
+ ml_model_unload(test, opt, &t->model[fid], fid);
+ }
+
+ t->cmn.result = ML_TEST_FAILED;
+
+ return ret;
+}
+
+static int
+test_inference_interleave_result(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+
+ return t->cmn.result;
+}
+
+static const struct ml_test_ops inference_interleave = {
+ .cap_check = test_inference_cap_check,
+ .opt_check = test_inference_opt_check,
+ .opt_dump = test_inference_opt_dump,
+ .test_setup = test_inference_setup,
+ .test_destroy = test_inference_destroy,
+ .test_driver = test_inference_interleave_driver,
+ .test_result = test_inference_interleave_result,
+};
+
+ML_TEST_REGISTER(inference_interleave);
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v1 07/12] app/mldev: enable support for burst inferences
2022-11-29 7:07 [PATCH v1 00/12] implement mldev test application Srikanth Yalavarthi
` (5 preceding siblings ...)
2022-11-29 7:07 ` [PATCH v1 06/12] app/mldev: add test case to interleave inferences Srikanth Yalavarthi
@ 2022-11-29 7:07 ` Srikanth Yalavarthi
2022-11-29 7:07 ` [PATCH v1 08/12] app/mldev: enable support for queue pairs and size Srikanth Yalavarthi
` (8 subsequent siblings)
15 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2022-11-29 7:07 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj
Added 'burst_size' support for inference tests. Burst size
controls the number of inference requests handled during
the burst enqueue and dequeue operations of the test case.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/ml_options.c | 26 ++--
app/test-mldev/ml_options.h | 2 +
app/test-mldev/test_inference_common.c | 159 ++++++++++++++++++++++++-
app/test-mldev/test_inference_common.h | 4 +
4 files changed, 181 insertions(+), 10 deletions(-)
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 9a006ff7c8..957218af3c 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -30,6 +30,7 @@ ml_options_default(struct ml_options *opt)
opt->socket_id = SOCKET_ID_ANY;
opt->nb_filelist = 0;
opt->repetitions = 1;
+ opt->burst_size = 1;
opt->debug = false;
}
@@ -151,6 +152,12 @@ ml_parse_repetitions(struct ml_options *opt, const char *arg)
return parser_read_uint64(&opt->repetitions, arg);
}
+static int
+ml_parse_burst_size(struct ml_options *opt, const char *arg)
+{
+ return parser_read_uint16(&opt->burst_size, arg);
+}
+
static void
ml_dump_test_options(const char *testname)
{
@@ -165,7 +172,8 @@ ml_dump_test_options(const char *testname)
if ((strcmp(testname, "inference_ordered") == 0) ||
(strcmp(testname, "inference_interleave") == 0)) {
printf("\t\t--filelist : comma separated list of model, input and output\n"
- "\t\t--repetitions : number of inference repetitions\n");
+ "\t\t--repetitions : number of inference repetitions\n"
+ "\t\t--burst_size : inference burst size\n");
printf("\n");
}
}
@@ -185,10 +193,11 @@ print_usage(char *program)
ml_test_dump_names(ml_dump_test_options);
}
-static struct option lgopts[] = {
- {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
- {ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
- {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
+static struct option lgopts[] = {{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0},
+ {ML_SOCKET_ID, 1, 0, 0}, {ML_MODELS, 1, 0, 0},
+ {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
+ {ML_BURST_SIZE, 1, 0, 0}, {ML_DEBUG, 0, 0, 0},
+ {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
static int
ml_opts_parse_long(int opt_idx, struct ml_options *opt)
@@ -196,9 +205,10 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
unsigned int i;
struct long_opt_parser parsermap[] = {
- {ML_TEST, ml_parse_test_name}, {ML_DEVICE_ID, ml_parse_dev_id},
- {ML_SOCKET_ID, ml_parse_socket_id}, {ML_MODELS, ml_parse_models},
- {ML_FILELIST, ml_parse_filelist}, {ML_REPETITIONS, ml_parse_repetitions},
+ {ML_TEST, ml_parse_test_name}, {ML_DEVICE_ID, ml_parse_dev_id},
+ {ML_SOCKET_ID, ml_parse_socket_id}, {ML_MODELS, ml_parse_models},
+ {ML_FILELIST, ml_parse_filelist}, {ML_REPETITIONS, ml_parse_repetitions},
+ {ML_BURST_SIZE, ml_parse_burst_size},
};
for (i = 0; i < RTE_DIM(parsermap); i++) {
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index ad8aee5964..305b39629a 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -19,6 +19,7 @@
#define ML_MODELS ("models")
#define ML_FILELIST ("filelist")
#define ML_REPETITIONS ("repetitions")
+#define ML_BURST_SIZE ("burst_size")
#define ML_DEBUG ("debug")
#define ML_HELP ("help")
@@ -35,6 +36,7 @@ struct ml_options {
struct ml_filelist filelist[ML_TEST_MAX_MODELS];
uint8_t nb_filelist;
uint64_t repetitions;
+ uint16_t burst_size;
bool debug;
};
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
index 1e0e30637f..ea8106c4ec 100644
--- a/app/test-mldev/test_inference_common.c
+++ b/app/test-mldev/test_inference_common.c
@@ -129,6 +129,131 @@ ml_dequeue_single(void *arg)
return 0;
}
+/* Enqueue inference requests with burst size greater than 1 */
+static int
+ml_enqueue_burst(void *arg)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)arg);
+ struct ml_core_args *args;
+ uint16_t ops_count;
+ uint64_t model_enq;
+ uint16_t burst_enq;
+ uint32_t lcore_id;
+ uint16_t pending;
+ uint16_t idx;
+ int16_t fid;
+ uint16_t i;
+ int ret;
+
+ lcore_id = rte_lcore_id();
+ args = &t->args[lcore_id];
+ model_enq = 0;
+
+ if (args->nb_reqs == 0)
+ return 0;
+
+next_rep:
+ fid = args->start_fid;
+
+next_model:
+ ops_count = RTE_MIN(t->cmn.opt->burst_size, args->nb_reqs - model_enq);
+ ret = rte_mempool_get_bulk(t->op_pool, (void **)args->enq_ops, ops_count);
+ if (ret != 0)
+ goto next_model;
+
+retry:
+ ret = rte_mempool_get_bulk(t->model[fid].io_pool, (void **)args->reqs, ops_count);
+ if (ret != 0)
+ goto retry;
+
+ for (i = 0; i < ops_count; i++) {
+ args->enq_ops[i]->model_id = t->model[fid].id;
+ args->enq_ops[i]->nb_batches = t->model[fid].info.batch_size;
+ args->enq_ops[i]->mempool = t->op_pool;
+
+ args->enq_ops[i]->input.addr = args->reqs[i]->input;
+ args->enq_ops[i]->input.length = t->model[fid].inp_qsize;
+ args->enq_ops[i]->input.next = NULL;
+
+ args->enq_ops[i]->output.addr = args->reqs[i]->output;
+ args->enq_ops[i]->output.length = t->model[fid].out_qsize;
+ args->enq_ops[i]->output.next = NULL;
+
+ args->enq_ops[i]->user_ptr = args->reqs[i];
+ args->reqs[i]->niters++;
+ args->reqs[i]->fid = fid;
+ }
+
+ idx = 0;
+ pending = ops_count;
+
+enqueue_reqs:
+ burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, 0, &args->enq_ops[idx], pending);
+ pending = pending - burst_enq;
+
+ if (pending > 0) {
+ idx = idx + burst_enq;
+ goto enqueue_reqs;
+ }
+
+ fid++;
+ if (fid <= args->end_fid)
+ goto next_model;
+
+ model_enq = model_enq + ops_count;
+ if (model_enq < args->nb_reqs)
+ goto next_rep;
+
+ return 0;
+}
+
+/* Dequeue inference requests with burst size greater than 1 */
+static int
+ml_dequeue_burst(void *arg)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)arg);
+ struct rte_ml_op_error error;
+ struct ml_core_args *args;
+ struct ml_request *req;
+ uint64_t total_deq = 0;
+ uint16_t burst_deq = 0;
+ uint8_t nb_filelist;
+ uint32_t lcore_id;
+ uint32_t i;
+
+ lcore_id = rte_lcore_id();
+ args = &t->args[lcore_id];
+ nb_filelist = args->end_fid - args->start_fid + 1;
+
+ if (args->nb_reqs == 0)
+ return 0;
+
+dequeue_burst:
+ burst_deq =
+ rte_ml_dequeue_burst(t->cmn.opt->dev_id, 0, args->deq_ops, t->cmn.opt->burst_size);
+
+ if (likely(burst_deq > 0)) {
+ total_deq += burst_deq;
+
+ for (i = 0; i < burst_deq; i++) {
+ if (unlikely(args->deq_ops[i]->status == RTE_ML_OP_STATUS_ERROR)) {
+ rte_ml_op_error_get(t->cmn.opt->dev_id, args->deq_ops[i], &error);
+ ml_err("error_code = 0x%" PRIx64 ", error_message = %s\n",
+ error.errcode, error.message);
+ }
+ req = (struct ml_request *)args->deq_ops[i]->user_ptr;
+ if (req != NULL)
+ rte_mempool_put(t->model[req->fid].io_pool, req);
+ }
+ rte_mempool_put_bulk(t->op_pool, (void *)args->deq_ops, burst_deq);
+ }
+
+ if (total_deq < args->nb_reqs * nb_filelist)
+ goto dequeue_burst;
+
+ return 0;
+}
+
bool
test_inference_cap_check(struct ml_options *opt)
{
@@ -178,6 +303,17 @@ test_inference_opt_check(struct ml_options *opt)
return -EINVAL;
}
+ if (opt->burst_size == 0) {
+ ml_err("Invalid option, burst_size = %u\n", opt->burst_size);
+ return -EINVAL;
+ }
+
+ if (opt->burst_size > ML_TEST_MAX_POOL_SIZE) {
+ ml_err("Invalid option, burst_size = %u (> max supported = %d)\n", opt->burst_size,
+ ML_TEST_MAX_POOL_SIZE);
+ return -EINVAL;
+ }
+
/* check number of available lcores. */
if (rte_lcore_count() < 3) {
ml_err("Insufficient lcores = %u\n", rte_lcore_count());
@@ -198,6 +334,7 @@ test_inference_opt_dump(struct ml_options *opt)
/* dump test opts */
ml_dump("repetitions", "%" PRIu64, opt->repetitions);
+ ml_dump("burst_size", "%u", opt->burst_size);
ml_dump_begin("filelist");
for (i = 0; i < opt->nb_filelist; i++) {
@@ -213,6 +350,7 @@ test_inference_setup(struct ml_test *test, struct ml_options *opt)
{
struct test_inference *t;
void *test_inference;
+ uint32_t lcore_id;
int ret = 0;
uint32_t i;
@@ -237,13 +375,30 @@ test_inference_setup(struct ml_test *test, struct ml_options *opt)
goto error;
}
- t->enqueue = ml_enqueue_single;
- t->dequeue = ml_dequeue_single;
+ if (opt->burst_size == 1) {
+ t->enqueue = ml_enqueue_single;
+ t->dequeue = ml_dequeue_single;
+ } else {
+ t->enqueue = ml_enqueue_burst;
+ t->dequeue = ml_dequeue_burst;
+ }
/* set model initial state */
for (i = 0; i < opt->nb_filelist; i++)
t->model[i].state = MODEL_INITIAL;
+ for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) {
+ t->args[lcore_id].enq_ops = rte_zmalloc_socket(
+ "ml_test_enq_ops", opt->burst_size * sizeof(struct rte_ml_op *),
+ RTE_CACHE_LINE_SIZE, opt->socket_id);
+ t->args[lcore_id].deq_ops = rte_zmalloc_socket(
+ "ml_test_deq_ops", opt->burst_size * sizeof(struct rte_ml_op *),
+ RTE_CACHE_LINE_SIZE, opt->socket_id);
+ t->args[lcore_id].reqs = rte_zmalloc_socket(
+ "ml_test_requests", opt->burst_size * sizeof(struct ml_request *),
+ RTE_CACHE_LINE_SIZE, opt->socket_id);
+ }
+
return 0;
error:
diff --git a/app/test-mldev/test_inference_common.h b/app/test-mldev/test_inference_common.h
index b058abada4..75d588308b 100644
--- a/app/test-mldev/test_inference_common.h
+++ b/app/test-mldev/test_inference_common.h
@@ -27,6 +27,10 @@ struct ml_core_args {
uint64_t nb_reqs;
int16_t start_fid;
int16_t end_fid;
+
+ struct rte_ml_op **enq_ops;
+ struct rte_ml_op **deq_ops;
+ struct ml_request **reqs;
};
struct test_inference {
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v1 08/12] app/mldev: enable support for queue pairs and size
2022-11-29 7:07 [PATCH v1 00/12] implement mldev test application Srikanth Yalavarthi
` (6 preceding siblings ...)
2022-11-29 7:07 ` [PATCH v1 07/12] app/mldev: enable support for burst inferences Srikanth Yalavarthi
@ 2022-11-29 7:07 ` Srikanth Yalavarthi
2022-11-29 7:07 ` [PATCH v1 09/12] app/mldev: enable support for inference batches Srikanth Yalavarthi
` (7 subsequent siblings)
15 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2022-11-29 7:07 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj
Added support to create multiple queue-pairs per device to
enqueue and dequeue inference requests. Number of queue pairs
to be created can be specified through "--queue_pairs" option.
Support is also enabled to control the number of descriptors
per each queue pair through "--queue_size" option. Inference
requests for a model are distributed across all available
queue-pairs.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/ml_options.c | 40 ++++++++++---
app/test-mldev/ml_options.h | 4 ++
app/test-mldev/test_common.c | 2 +-
app/test-mldev/test_inference_common.c | 79 +++++++++++++++++++++-----
app/test-mldev/test_inference_common.h | 1 +
5 files changed, 102 insertions(+), 24 deletions(-)
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 957218af3c..27b628c8b3 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -31,6 +31,8 @@ ml_options_default(struct ml_options *opt)
opt->nb_filelist = 0;
opt->repetitions = 1;
opt->burst_size = 1;
+ opt->queue_pairs = 1;
+ opt->queue_size = 1;
opt->debug = false;
}
@@ -158,11 +160,30 @@ ml_parse_burst_size(struct ml_options *opt, const char *arg)
return parser_read_uint16(&opt->burst_size, arg);
}
+static int
+ml_parse_queue_pairs(struct ml_options *opt, const char *arg)
+{
+ int ret;
+
+ ret = parser_read_uint16(&opt->queue_pairs, arg);
+
+ return ret;
+}
+
+static int
+ml_parse_queue_size(struct ml_options *opt, const char *arg)
+{
+ return parser_read_uint16(&opt->queue_size, arg);
+}
+
static void
ml_dump_test_options(const char *testname)
{
- if (strcmp(testname, "device_ops") == 0)
+ if (strcmp(testname, "device_ops") == 0) {
+ printf("\t\t--queue_pairs : number of queue pairs to create\n"
+ "\t\t--queue_size : size fo queue-pair\n");
printf("\n");
+ }
if (strcmp(testname, "model_ops") == 0) {
printf("\t\t--models : comma separated list of models\n");
@@ -173,7 +194,9 @@ ml_dump_test_options(const char *testname)
(strcmp(testname, "inference_interleave") == 0)) {
printf("\t\t--filelist : comma separated list of model, input and output\n"
"\t\t--repetitions : number of inference repetitions\n"
- "\t\t--burst_size : inference burst size\n");
+ "\t\t--burst_size : inference burst size\n"
+ "\t\t--queue_pairs : number of queue pairs to create\n"
+ "\t\t--queue_size : size fo queue-pair\n");
printf("\n");
}
}
@@ -193,11 +216,11 @@ print_usage(char *program)
ml_test_dump_names(ml_dump_test_options);
}
-static struct option lgopts[] = {{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0},
- {ML_SOCKET_ID, 1, 0, 0}, {ML_MODELS, 1, 0, 0},
- {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
- {ML_BURST_SIZE, 1, 0, 0}, {ML_DEBUG, 0, 0, 0},
- {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
+static struct option lgopts[] = {
+ {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
+ {ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
+ {ML_BURST_SIZE, 1, 0, 0}, {ML_QUEUE_PAIRS, 1, 0, 0}, {ML_QUEUE_SIZE, 1, 0, 0},
+ {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
static int
ml_opts_parse_long(int opt_idx, struct ml_options *opt)
@@ -208,7 +231,8 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
{ML_TEST, ml_parse_test_name}, {ML_DEVICE_ID, ml_parse_dev_id},
{ML_SOCKET_ID, ml_parse_socket_id}, {ML_MODELS, ml_parse_models},
{ML_FILELIST, ml_parse_filelist}, {ML_REPETITIONS, ml_parse_repetitions},
- {ML_BURST_SIZE, ml_parse_burst_size},
+ {ML_BURST_SIZE, ml_parse_burst_size}, {ML_QUEUE_PAIRS, ml_parse_queue_pairs},
+ {ML_QUEUE_SIZE, ml_parse_queue_size},
};
for (i = 0; i < RTE_DIM(parsermap); i++) {
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index 305b39629a..6bfef1b979 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -20,6 +20,8 @@
#define ML_FILELIST ("filelist")
#define ML_REPETITIONS ("repetitions")
#define ML_BURST_SIZE ("burst_size")
+#define ML_QUEUE_PAIRS ("queue_pairs")
+#define ML_QUEUE_SIZE ("queue_size")
#define ML_DEBUG ("debug")
#define ML_HELP ("help")
@@ -37,6 +39,8 @@ struct ml_options {
uint8_t nb_filelist;
uint64_t repetitions;
uint16_t burst_size;
+ uint16_t queue_pairs;
+ uint16_t queue_size;
bool debug;
};
diff --git a/app/test-mldev/test_common.c b/app/test-mldev/test_common.c
index b6b32904e4..22e6acb3b6 100644
--- a/app/test-mldev/test_common.c
+++ b/app/test-mldev/test_common.c
@@ -78,7 +78,7 @@ ml_test_device_configure(struct ml_test *test, struct ml_options *opt)
/* configure device */
dev_config.socket_id = opt->socket_id;
dev_config.nb_models = t->dev_info.max_models;
- dev_config.nb_queue_pairs = t->dev_info.max_queue_pairs;
+ dev_config.nb_queue_pairs = opt->queue_pairs;
ret = rte_ml_dev_configure(opt->dev_id, &dev_config);
if (ret != 0) {
ml_err("Failed to configure ml device, dev_id = %d\n", opt->dev_id);
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
index ea8106c4ec..a1111d9119 100644
--- a/app/test-mldev/test_inference_common.c
+++ b/app/test-mldev/test_inference_common.c
@@ -72,7 +72,7 @@ ml_enqueue_single(void *arg)
req->fid = fid;
enqueue_req:
- burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, 0, &op, 1);
+ burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, args->qp_id, &op, 1);
if (burst_enq == 0)
goto enqueue_req;
@@ -109,7 +109,7 @@ ml_dequeue_single(void *arg)
return 0;
dequeue_req:
- burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, 0, &op, 1);
+ burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, args->qp_id, &op, 1);
if (likely(burst_deq == 1)) {
total_deq += burst_deq;
@@ -188,7 +188,8 @@ ml_enqueue_burst(void *arg)
pending = ops_count;
enqueue_reqs:
- burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, 0, &args->enq_ops[idx], pending);
+ burst_enq =
+ rte_ml_enqueue_burst(t->cmn.opt->dev_id, args->qp_id, &args->enq_ops[idx], pending);
pending = pending - burst_enq;
if (pending > 0) {
@@ -229,8 +230,8 @@ ml_dequeue_burst(void *arg)
return 0;
dequeue_burst:
- burst_deq =
- rte_ml_dequeue_burst(t->cmn.opt->dev_id, 0, args->deq_ops, t->cmn.opt->burst_size);
+ burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, args->qp_id, args->deq_ops,
+ t->cmn.opt->burst_size);
if (likely(burst_deq > 0)) {
total_deq += burst_deq;
@@ -263,6 +264,19 @@ test_inference_cap_check(struct ml_options *opt)
return false;
rte_ml_dev_info_get(opt->dev_id, &dev_info);
+
+ if (opt->queue_pairs > dev_info.max_queue_pairs) {
+ ml_err("Insufficient capabilities: queue_pairs = %u, max_queue_pairs = %u",
+ opt->queue_pairs, dev_info.max_queue_pairs);
+ return false;
+ }
+
+ if (opt->queue_size > dev_info.max_desc) {
+ ml_err("Insufficient capabilities: queue_size = %u, max_desc = %u", opt->queue_size,
+ dev_info.max_desc);
+ return false;
+ }
+
if (opt->nb_filelist > dev_info.max_models) {
ml_err("Insufficient capabilities: Filelist count exceeded device limit, count = %u (max limit = %u)",
opt->nb_filelist, dev_info.max_models);
@@ -314,10 +328,21 @@ test_inference_opt_check(struct ml_options *opt)
return -EINVAL;
}
+ if (opt->queue_pairs == 0) {
+ ml_err("Invalid option, queue_pairs = %u\n", opt->queue_pairs);
+ return -EINVAL;
+ }
+
+ if (opt->queue_size == 0) {
+ ml_err("Invalid option, queue_size = %u\n", opt->queue_size);
+ return -EINVAL;
+ }
+
/* check number of available lcores. */
- if (rte_lcore_count() < 3) {
+ if (rte_lcore_count() < (uint32_t)(opt->queue_pairs * 2 + 1)) {
ml_err("Insufficient lcores = %u\n", rte_lcore_count());
- ml_err("Minimum lcores required to create %u queue-pairs = %u\n", 1, 3);
+ ml_err("Minimum lcores required to create %u queue-pairs = %u\n", opt->queue_pairs,
+ (opt->queue_pairs * 2 + 1));
return -EINVAL;
}
@@ -335,6 +360,8 @@ test_inference_opt_dump(struct ml_options *opt)
/* dump test opts */
ml_dump("repetitions", "%" PRIu64, opt->repetitions);
ml_dump("burst_size", "%u", opt->burst_size);
+ ml_dump("queue_pairs", "%u", opt->queue_pairs);
+ ml_dump("queue_size", "%u", opt->queue_size);
ml_dump_begin("filelist");
for (i = 0; i < opt->nb_filelist; i++) {
@@ -425,23 +452,31 @@ ml_inference_mldev_setup(struct ml_test *test, struct ml_options *opt)
{
struct rte_ml_dev_qp_conf qp_conf;
struct test_inference *t;
+ uint16_t qp_id;
int ret;
t = ml_test_priv(test);
+ RTE_SET_USED(t);
+
ret = ml_test_device_configure(test, opt);
if (ret != 0)
return ret;
/* setup queue pairs */
- qp_conf.nb_desc = t->cmn.dev_info.max_desc;
+ qp_conf.nb_desc = opt->queue_size;
qp_conf.cb = NULL;
- ret = rte_ml_dev_queue_pair_setup(opt->dev_id, 0, &qp_conf, opt->socket_id);
- if (ret != 0) {
- ml_err("Failed to setup ml device queue-pair, dev_id = %d, qp_id = %u\n",
- opt->dev_id, 0);
- goto error;
+ for (qp_id = 0; qp_id < opt->queue_pairs; qp_id++) {
+ qp_conf.nb_desc = opt->queue_size;
+ qp_conf.cb = NULL;
+
+ ret = rte_ml_dev_queue_pair_setup(opt->dev_id, qp_id, &qp_conf, opt->socket_id);
+ if (ret != 0) {
+ ml_err("Failed to setup ml device queue-pair, dev_id = %d, qp_id = %u\n",
+ opt->dev_id, qp_id);
+ return ret;
+ }
}
ret = ml_test_device_start(test, opt);
@@ -697,14 +732,28 @@ ml_inference_launch_cores(struct ml_test *test, struct ml_options *opt, int16_t
{
struct test_inference *t = ml_test_priv(test);
uint32_t lcore_id;
+ uint32_t nb_reqs;
uint32_t id = 0;
+ uint32_t qp_id;
+
+ nb_reqs = opt->repetitions / opt->queue_pairs;
RTE_LCORE_FOREACH_WORKER(lcore_id)
{
- if (id == 2)
+ if (id >= opt->queue_pairs * 2)
break;
- t->args[lcore_id].nb_reqs = opt->repetitions;
+ qp_id = id / 2;
+ t->args[lcore_id].qp_id = qp_id;
+ t->args[lcore_id].nb_reqs = nb_reqs;
+ if (qp_id == 0)
+ t->args[lcore_id].nb_reqs += opt->repetitions - nb_reqs * opt->queue_pairs;
+
+ if (t->args[lcore_id].nb_reqs == 0) {
+ id++;
+ break;
+ }
+
t->args[lcore_id].start_fid = start_fid;
t->args[lcore_id].end_fid = end_fid;
diff --git a/app/test-mldev/test_inference_common.h b/app/test-mldev/test_inference_common.h
index 75d588308b..1bac2dcfa0 100644
--- a/app/test-mldev/test_inference_common.h
+++ b/app/test-mldev/test_inference_common.h
@@ -27,6 +27,7 @@ struct ml_core_args {
uint64_t nb_reqs;
int16_t start_fid;
int16_t end_fid;
+ uint32_t qp_id;
struct rte_ml_op **enq_ops;
struct rte_ml_op **deq_ops;
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v1 09/12] app/mldev: enable support for inference batches
2022-11-29 7:07 [PATCH v1 00/12] implement mldev test application Srikanth Yalavarthi
` (7 preceding siblings ...)
2022-11-29 7:07 ` [PATCH v1 08/12] app/mldev: enable support for queue pairs and size Srikanth Yalavarthi
@ 2022-11-29 7:07 ` Srikanth Yalavarthi
2022-11-29 7:07 ` [PATCH v1 10/12] app/mldev: enable support for inference validation Srikanth Yalavarthi
` (6 subsequent siblings)
15 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2022-11-29 7:07 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj
Enabled support to execute multiple batches of inferences
per each enqueue request. Input and reference for the test
should be appropriately provided for multi-batch run. Number
of batches can be specified through "--batches" option.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/ml_options.c | 15 ++++++++++++---
app/test-mldev/ml_options.h | 2 ++
app/test-mldev/test_inference_common.c | 22 +++++++++++++---------
app/test-mldev/test_model_common.c | 6 ++++++
app/test-mldev/test_model_common.h | 1 +
5 files changed, 34 insertions(+), 12 deletions(-)
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 27b628c8b3..a27a919a0d 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -33,6 +33,7 @@ ml_options_default(struct ml_options *opt)
opt->burst_size = 1;
opt->queue_pairs = 1;
opt->queue_size = 1;
+ opt->batches = 0;
opt->debug = false;
}
@@ -176,6 +177,12 @@ ml_parse_queue_size(struct ml_options *opt, const char *arg)
return parser_read_uint16(&opt->queue_size, arg);
}
+static int
+ml_parse_batches(struct ml_options *opt, const char *arg)
+{
+ return parser_read_uint16(&opt->batches, arg);
+}
+
static void
ml_dump_test_options(const char *testname)
{
@@ -196,7 +203,8 @@ ml_dump_test_options(const char *testname)
"\t\t--repetitions : number of inference repetitions\n"
"\t\t--burst_size : inference burst size\n"
"\t\t--queue_pairs : number of queue pairs to create\n"
- "\t\t--queue_size : size fo queue-pair\n");
+ "\t\t--queue_size : size fo queue-pair\n"
+ "\t\t--batches : number of batches of input\n");
printf("\n");
}
}
@@ -220,7 +228,8 @@ static struct option lgopts[] = {
{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
{ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
{ML_BURST_SIZE, 1, 0, 0}, {ML_QUEUE_PAIRS, 1, 0, 0}, {ML_QUEUE_SIZE, 1, 0, 0},
- {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
+ {ML_BATCHES, 1, 0, 0}, {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0},
+ {NULL, 0, 0, 0}};
static int
ml_opts_parse_long(int opt_idx, struct ml_options *opt)
@@ -232,7 +241,7 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
{ML_SOCKET_ID, ml_parse_socket_id}, {ML_MODELS, ml_parse_models},
{ML_FILELIST, ml_parse_filelist}, {ML_REPETITIONS, ml_parse_repetitions},
{ML_BURST_SIZE, ml_parse_burst_size}, {ML_QUEUE_PAIRS, ml_parse_queue_pairs},
- {ML_QUEUE_SIZE, ml_parse_queue_size},
+ {ML_QUEUE_SIZE, ml_parse_queue_size}, {ML_BATCHES, ml_parse_batches},
};
for (i = 0; i < RTE_DIM(parsermap); i++) {
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index 6bfef1b979..d23e842895 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -22,6 +22,7 @@
#define ML_BURST_SIZE ("burst_size")
#define ML_QUEUE_PAIRS ("queue_pairs")
#define ML_QUEUE_SIZE ("queue_size")
+#define ML_BATCHES ("batches")
#define ML_DEBUG ("debug")
#define ML_HELP ("help")
@@ -41,6 +42,7 @@ struct ml_options {
uint16_t burst_size;
uint16_t queue_pairs;
uint16_t queue_size;
+ uint16_t batches;
bool debug;
};
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
index a1111d9119..fc7e162514 100644
--- a/app/test-mldev/test_inference_common.c
+++ b/app/test-mldev/test_inference_common.c
@@ -56,7 +56,7 @@ ml_enqueue_single(void *arg)
goto retry;
op->model_id = t->model[fid].id;
- op->nb_batches = t->model[fid].info.batch_size;
+ op->nb_batches = t->model[fid].nb_batches;
op->mempool = t->op_pool;
op->input.addr = req->input;
@@ -168,7 +168,7 @@ ml_enqueue_burst(void *arg)
for (i = 0; i < ops_count; i++) {
args->enq_ops[i]->model_id = t->model[fid].id;
- args->enq_ops[i]->nb_batches = t->model[fid].info.batch_size;
+ args->enq_ops[i]->nb_batches = t->model[fid].nb_batches;
args->enq_ops[i]->mempool = t->op_pool;
args->enq_ops[i]->input.addr = args->reqs[i]->input;
@@ -363,6 +363,11 @@ test_inference_opt_dump(struct ml_options *opt)
ml_dump("queue_pairs", "%u", opt->queue_pairs);
ml_dump("queue_size", "%u", opt->queue_size);
+ if (opt->batches == 0)
+ ml_dump("batches", "%u (default)", opt->batches);
+ else
+ ml_dump("batches", "%u", opt->batches);
+
ml_dump_begin("filelist");
for (i = 0; i < opt->nb_filelist; i++) {
ml_dump_list("model", i, opt->filelist[i].model);
@@ -531,8 +536,8 @@ ml_request_initialize(struct rte_mempool *mp, void *opaque, void *obj, unsigned
req->niters = 0;
/* quantize data */
- rte_ml_io_quantize(t->cmn.opt->dev_id, t->model[t->fid].id,
- t->model[t->fid].info.batch_size, t->model[t->fid].input, req->input);
+ rte_ml_io_quantize(t->cmn.opt->dev_id, t->model[t->fid].id, t->model[t->fid].nb_batches,
+ t->model[t->fid].input, req->input);
}
int
@@ -550,7 +555,7 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, int16_t f
int ret;
/* get input buffer size */
- ret = rte_ml_io_input_size_get(opt->dev_id, t->model[fid].id, t->model[fid].info.batch_size,
+ ret = rte_ml_io_input_size_get(opt->dev_id, t->model[fid].id, t->model[fid].nb_batches,
&t->model[fid].inp_qsize, &t->model[fid].inp_dsize);
if (ret != 0) {
ml_err("Failed to get input size, model : %s\n", opt->filelist[fid].model);
@@ -558,9 +563,8 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, int16_t f
}
/* get output buffer size */
- ret = rte_ml_io_output_size_get(opt->dev_id, t->model[fid].id,
- t->model[fid].info.batch_size, &t->model[fid].out_qsize,
- &t->model[fid].out_dsize);
+ ret = rte_ml_io_output_size_get(opt->dev_id, t->model[fid].id, t->model[fid].nb_batches,
+ &t->model[fid].out_qsize, &t->model[fid].out_dsize);
if (ret != 0) {
ml_err("Failed to get input size, model : %s\n", opt->filelist[fid].model);
return ret;
@@ -705,7 +709,7 @@ ml_request_finish(struct rte_mempool *mp, void *opaque, void *obj, unsigned int
return;
t->nb_used++;
- rte_ml_io_dequantize(t->cmn.opt->dev_id, model->id, t->model[req->fid].info.batch_size,
+ rte_ml_io_dequantize(t->cmn.opt->dev_id, model->id, t->model[req->fid].nb_batches,
req->output, model->output);
}
diff --git a/app/test-mldev/test_model_common.c b/app/test-mldev/test_model_common.c
index 5368be17fe..51260c0789 100644
--- a/app/test-mldev/test_model_common.c
+++ b/app/test-mldev/test_model_common.c
@@ -75,6 +75,12 @@ ml_model_load(struct ml_test *test, struct ml_options *opt, struct ml_model *mod
return ret;
}
+ /* Update number of batches */
+ if (opt->batches == 0)
+ model->nb_batches = model->info.batch_size;
+ else
+ model->nb_batches = opt->batches;
+
model->state = MODEL_LOADED;
return 0;
diff --git a/app/test-mldev/test_model_common.h b/app/test-mldev/test_model_common.h
index c45ae80853..dfbf568f0b 100644
--- a/app/test-mldev/test_model_common.h
+++ b/app/test-mldev/test_model_common.h
@@ -33,6 +33,7 @@ struct ml_model {
uint8_t *output;
struct rte_mempool *io_pool;
+ uint32_t nb_batches;
};
int ml_model_load(struct ml_test *test, struct ml_options *opt, struct ml_model *model,
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v1 10/12] app/mldev: enable support for inference validation
2022-11-29 7:07 [PATCH v1 00/12] implement mldev test application Srikanth Yalavarthi
` (8 preceding siblings ...)
2022-11-29 7:07 ` [PATCH v1 09/12] app/mldev: enable support for inference batches Srikanth Yalavarthi
@ 2022-11-29 7:07 ` Srikanth Yalavarthi
2022-11-29 7:07 ` [PATCH v1 11/12] app/mldev: enable reporting stats in mldev app Srikanth Yalavarthi
` (5 subsequent siblings)
15 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2022-11-29 7:07 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj
Enabled support to validate inference output with reference
output provided by the user. Validation would be successful
only when the inference outputs are within the 'tolerance'
specified through command line option "--tolerance".
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/meson.build | 2 +-
app/test-mldev/ml_options.c | 36 +++-
app/test-mldev/ml_options.h | 3 +
app/test-mldev/test_inference_common.c | 218 ++++++++++++++++++++++++-
app/test-mldev/test_inference_common.h | 1 +
app/test-mldev/test_model_common.h | 1 +
6 files changed, 250 insertions(+), 11 deletions(-)
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
index 41d22fb22c..15db534dc2 100644
--- a/app/test-mldev/meson.build
+++ b/app/test-mldev/meson.build
@@ -21,4 +21,4 @@ sources = files(
'test_inference_interleave.c',
)
-deps += ['mldev']
+deps += ['mldev', 'hash']
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index a27a919a0d..4087ab52db 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -5,6 +5,7 @@
#include <errno.h>
#include <getopt.h>
#include <linux/limits.h>
+#include <math.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
@@ -34,6 +35,7 @@ ml_options_default(struct ml_options *opt)
opt->queue_pairs = 1;
opt->queue_size = 1;
opt->batches = 0;
+ opt->tolerance = 0.0;
opt->debug = false;
}
@@ -139,6 +141,13 @@ ml_parse_filelist(struct ml_options *opt, const char *arg)
}
strlcpy(opt->filelist[opt->nb_filelist].output, token, PATH_MAX);
+ /* reference - optional */
+ token = strtok(NULL, delim);
+ if (token != NULL)
+ strlcpy(opt->filelist[opt->nb_filelist].reference, token, PATH_MAX);
+ else
+ memset(opt->filelist[opt->nb_filelist].reference, 0, PATH_MAX);
+
opt->nb_filelist++;
if (opt->nb_filelist == 0) {
@@ -183,6 +192,14 @@ ml_parse_batches(struct ml_options *opt, const char *arg)
return parser_read_uint16(&opt->batches, arg);
}
+static int
+ml_parse_tolerance(struct ml_options *opt, const char *arg)
+{
+ opt->tolerance = fabs(atof(arg));
+
+ return 0;
+}
+
static void
ml_dump_test_options(const char *testname)
{
@@ -199,12 +216,13 @@ ml_dump_test_options(const char *testname)
if ((strcmp(testname, "inference_ordered") == 0) ||
(strcmp(testname, "inference_interleave") == 0)) {
- printf("\t\t--filelist : comma separated list of model, input and output\n"
+ printf("\t\t--filelist : comma separated list of model, input, output and reference\n"
"\t\t--repetitions : number of inference repetitions\n"
"\t\t--burst_size : inference burst size\n"
"\t\t--queue_pairs : number of queue pairs to create\n"
"\t\t--queue_size : size fo queue-pair\n"
- "\t\t--batches : number of batches of input\n");
+ "\t\t--batches : number of batches of input\n"
+ "\t\t--tolerance : maximum tolerance (%%) for output validation\n");
printf("\n");
}
}
@@ -224,12 +242,13 @@ print_usage(char *program)
ml_test_dump_names(ml_dump_test_options);
}
-static struct option lgopts[] = {
- {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
- {ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
- {ML_BURST_SIZE, 1, 0, 0}, {ML_QUEUE_PAIRS, 1, 0, 0}, {ML_QUEUE_SIZE, 1, 0, 0},
- {ML_BATCHES, 1, 0, 0}, {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0},
- {NULL, 0, 0, 0}};
+static struct option lgopts[] = {{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0},
+ {ML_SOCKET_ID, 1, 0, 0}, {ML_MODELS, 1, 0, 0},
+ {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
+ {ML_BURST_SIZE, 1, 0, 0}, {ML_QUEUE_PAIRS, 1, 0, 0},
+ {ML_QUEUE_SIZE, 1, 0, 0}, {ML_BATCHES, 1, 0, 0},
+ {ML_TOLERANCE, 1, 0, 0}, {ML_DEBUG, 0, 0, 0},
+ {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
static int
ml_opts_parse_long(int opt_idx, struct ml_options *opt)
@@ -242,6 +261,7 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
{ML_FILELIST, ml_parse_filelist}, {ML_REPETITIONS, ml_parse_repetitions},
{ML_BURST_SIZE, ml_parse_burst_size}, {ML_QUEUE_PAIRS, ml_parse_queue_pairs},
{ML_QUEUE_SIZE, ml_parse_queue_size}, {ML_BATCHES, ml_parse_batches},
+ {ML_TOLERANCE, ml_parse_tolerance},
};
for (i = 0; i < RTE_DIM(parsermap); i++) {
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index d23e842895..79ac54de98 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -23,6 +23,7 @@
#define ML_QUEUE_PAIRS ("queue_pairs")
#define ML_QUEUE_SIZE ("queue_size")
#define ML_BATCHES ("batches")
+#define ML_TOLERANCE ("tolerance")
#define ML_DEBUG ("debug")
#define ML_HELP ("help")
@@ -30,6 +31,7 @@ struct ml_filelist {
char model[PATH_MAX];
char input[PATH_MAX];
char output[PATH_MAX];
+ char reference[PATH_MAX];
};
struct ml_options {
@@ -43,6 +45,7 @@ struct ml_options {
uint16_t queue_pairs;
uint16_t queue_size;
uint16_t batches;
+ float tolerance;
bool debug;
};
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
index fc7e162514..cdd1667c71 100644
--- a/app/test-mldev/test_inference_common.c
+++ b/app/test-mldev/test_inference_common.c
@@ -3,12 +3,15 @@
*/
#include <errno.h>
+#include <linux/limits.h>
#include <stdbool.h>
#include <stdint.h>
#include <stdio.h>
+#include <string.h>
#include <unistd.h>
#include <rte_common.h>
+#include <rte_hash_crc.h>
#include <rte_launch.h>
#include <rte_lcore.h>
#include <rte_malloc.h>
@@ -21,6 +24,27 @@
#include "test_common.h"
#include "test_inference_common.h"
+#define ML_TEST_READ_TYPE(buffer, type) (*((type *)buffer))
+
+#define ML_TEST_CHECK_OUTPUT(output, reference, tolerance) \
+ (((float)output - (float)reference) <= (((float)reference * tolerance) / 100.0))
+
+#define ML_OPEN_WRITE_GET_ERR(name, buffer, size, err) \
+ do { \
+ FILE *fp = fopen(name, "w+"); \
+ if (fp == NULL) { \
+ ml_err("Unable to create file: %s, error: %s", name, strerror(errno)); \
+ err = true; \
+ } else { \
+ if (fwrite(buffer, 1, size, fp) != size) { \
+ ml_err("Error writing output, file: %s, error: %s", name, \
+ strerror(errno)); \
+ err = true; \
+ } \
+ fclose(fp); \
+ } \
+ } while (0)
+
/* Enqueue inference requests with burst size equal to 1 */
static int
ml_enqueue_single(void *arg)
@@ -362,6 +386,7 @@ test_inference_opt_dump(struct ml_options *opt)
ml_dump("burst_size", "%u", opt->burst_size);
ml_dump("queue_pairs", "%u", opt->queue_pairs);
ml_dump("queue_size", "%u", opt->queue_size);
+ ml_dump("tolerance", "%-7.3f", opt->tolerance);
if (opt->batches == 0)
ml_dump("batches", "%u (default)", opt->batches);
@@ -373,6 +398,8 @@ test_inference_opt_dump(struct ml_options *opt)
ml_dump_list("model", i, opt->filelist[i].model);
ml_dump_list("input", i, opt->filelist[i].input);
ml_dump_list("output", i, opt->filelist[i].output);
+ if (strcmp(opt->filelist[i].reference, "\0") != 0)
+ ml_dump_list("reference", i, opt->filelist[i].reference);
}
ml_dump_end;
}
@@ -397,6 +424,7 @@ test_inference_setup(struct ml_test *test, struct ml_options *opt)
t = ml_test_priv(test);
t->nb_used = 0;
+ t->nb_valid = 0;
t->cmn.result = ML_TEST_FAILED;
t->cmn.opt = opt;
@@ -572,6 +600,9 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, int16_t f
/* allocate buffer for user data */
mz_size = t->model[fid].inp_dsize + t->model[fid].out_dsize;
+ if (strcmp(opt->filelist[fid].reference, "\0") != 0)
+ mz_size += t->model[fid].out_dsize;
+
sprintf(mz_name, "ml_user_data_%d", fid);
mz = rte_memzone_reserve(mz_name, mz_size, opt->socket_id, 0);
if (mz == NULL) {
@@ -582,6 +613,10 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, int16_t f
t->model[fid].input = mz->addr;
t->model[fid].output = t->model[fid].input + t->model[fid].inp_dsize;
+ if (strcmp(opt->filelist[fid].reference, "\0") != 0)
+ t->model[fid].reference = t->model[fid].output + t->model[fid].out_dsize;
+ else
+ t->model[fid].reference = NULL;
/* load input file */
fp = fopen(opt->filelist[fid].input, "r");
@@ -610,6 +645,27 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, int16_t f
}
fclose(fp);
+ /* load reference file */
+ if (t->model[fid].reference != NULL) {
+ fp = fopen(opt->filelist[fid].reference, "r");
+ if (fp == NULL) {
+ ml_err("Failed to open reference file : %s\n",
+ opt->filelist[fid].reference);
+ ret = -errno;
+ goto error;
+ }
+
+ if (fread(t->model[fid].reference, 1, t->model[fid].out_dsize, fp) !=
+ t->model[fid].out_dsize) {
+ ml_err("Failed to read reference file : %s\n",
+ opt->filelist[fid].reference);
+ ret = -errno;
+ fclose(fp);
+ goto error;
+ }
+ fclose(fp);
+ }
+
/* create mempool for quantized input and output buffers. ml_request_initialize is
* used as a callback for object creation.
*/
@@ -694,6 +750,121 @@ ml_inference_mem_destroy(struct ml_test *test, struct ml_options *opt)
rte_mempool_free(t->op_pool);
}
+static bool
+ml_inference_validation(struct ml_test *test, struct ml_request *req)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)test);
+ struct ml_model *model;
+ uint32_t nb_elements;
+ uint8_t *reference;
+ uint8_t *output;
+ bool match;
+ uint32_t i;
+ uint32_t j;
+
+ model = &t->model[req->fid];
+
+ /* compare crc when tolerance is 0 */
+ if (t->cmn.opt->tolerance == 0.0) {
+ match = (rte_hash_crc(model->output, model->out_dsize, 0) ==
+ rte_hash_crc(model->reference, model->out_dsize, 0));
+ } else {
+ output = model->output;
+ reference = model->reference;
+
+ i = 0;
+next_output:
+ nb_elements =
+ model->info.output_info[i].shape.w * model->info.output_info[i].shape.x *
+ model->info.output_info[i].shape.y * model->info.output_info[i].shape.z;
+ j = 0;
+next_element:
+ match = false;
+ switch (model->info.output_info[i].dtype) {
+ case RTE_ML_IO_TYPE_INT8:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, int8_t),
+ ML_TEST_READ_TYPE(reference, int8_t),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(int8_t);
+ reference += sizeof(int8_t);
+ break;
+ case RTE_ML_IO_TYPE_UINT8:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, uint8_t),
+ ML_TEST_READ_TYPE(reference, uint8_t),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(float);
+ reference += sizeof(float);
+ break;
+ case RTE_ML_IO_TYPE_INT16:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, int16_t),
+ ML_TEST_READ_TYPE(reference, int16_t),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(int16_t);
+ reference += sizeof(int16_t);
+ break;
+ case RTE_ML_IO_TYPE_UINT16:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, uint16_t),
+ ML_TEST_READ_TYPE(reference, uint16_t),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(uint16_t);
+ reference += sizeof(uint16_t);
+ break;
+ case RTE_ML_IO_TYPE_INT32:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, int32_t),
+ ML_TEST_READ_TYPE(reference, int32_t),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(int32_t);
+ reference += sizeof(int32_t);
+ break;
+ case RTE_ML_IO_TYPE_UINT32:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, uint32_t),
+ ML_TEST_READ_TYPE(reference, uint32_t),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(uint32_t);
+ reference += sizeof(uint32_t);
+ break;
+ case RTE_ML_IO_TYPE_FP32:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, float),
+ ML_TEST_READ_TYPE(reference, float),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(float);
+ reference += sizeof(float);
+ break;
+ default: /* other types, fp8, fp16, bfloat16 */
+ match = true;
+ }
+
+ if (!match)
+ goto done;
+ j++;
+ if (j < nb_elements)
+ goto next_element;
+
+ i++;
+ if (i < model->info.nb_outputs)
+ goto next_output;
+ }
+done:
+ if (match)
+ t->nb_valid++;
+
+ return match;
+}
+
/* Callback for mempool object iteration. This call would dequantize ouput data. */
static void
ml_request_finish(struct rte_mempool *mp, void *opaque, void *obj, unsigned int obj_idx)
@@ -701,9 +872,10 @@ ml_request_finish(struct rte_mempool *mp, void *opaque, void *obj, unsigned int
struct test_inference *t = ml_test_priv((struct ml_test *)opaque);
struct ml_request *req = (struct ml_request *)obj;
struct ml_model *model = &t->model[req->fid];
+ char str[PATH_MAX];
+ bool error = false;
RTE_SET_USED(mp);
- RTE_SET_USED(obj_idx);
if (req->niters == 0)
return;
@@ -711,6 +883,48 @@ ml_request_finish(struct rte_mempool *mp, void *opaque, void *obj, unsigned int
t->nb_used++;
rte_ml_io_dequantize(t->cmn.opt->dev_id, model->id, t->model[req->fid].nb_batches,
req->output, model->output);
+
+ if (model->reference == NULL) {
+ t->nb_valid++;
+ goto dump_output_pass;
+ }
+
+ if (!ml_inference_validation(opaque, req))
+ goto dump_output_fail;
+ else
+ goto dump_output_pass;
+
+dump_output_pass:
+ if (obj_idx == 0) {
+ /* write quantized output */
+ snprintf(str, PATH_MAX, "%s.q", t->cmn.opt->filelist[req->fid].output);
+ ML_OPEN_WRITE_GET_ERR(str, req->output, model->out_qsize, error);
+ if (error)
+ return;
+
+ /* write dequantized output */
+ snprintf(str, PATH_MAX, "%s", t->cmn.opt->filelist[req->fid].output);
+ ML_OPEN_WRITE_GET_ERR(str, model->output, model->out_dsize, error);
+ if (error)
+ return;
+ }
+
+ return;
+
+dump_output_fail:
+ if (t->cmn.opt->debug) {
+ /* dump quantized output buffer */
+ snprintf(str, PATH_MAX, "%s.q.%d", t->cmn.opt->filelist[req->fid].output, obj_idx);
+ ML_OPEN_WRITE_GET_ERR(str, req->output, model->out_qsize, error);
+ if (error)
+ return;
+
+ /* dump dequantized output buffer */
+ snprintf(str, PATH_MAX, "%s.%d", t->cmn.opt->filelist[req->fid].output, obj_idx);
+ ML_OPEN_WRITE_GET_ERR(str, model->output, model->out_dsize, error);
+ if (error)
+ return;
+ }
}
int
@@ -722,7 +936,7 @@ ml_inference_result(struct ml_test *test, struct ml_options *opt, int16_t fid)
rte_mempool_obj_iter(t->model[fid].io_pool, ml_request_finish, test);
- if (t->nb_used > 0)
+ if (t->nb_used == t->nb_valid)
t->cmn.result = ML_TEST_SUCCESS;
else
t->cmn.result = ML_TEST_FAILED;
diff --git a/app/test-mldev/test_inference_common.h b/app/test-mldev/test_inference_common.h
index 1bac2dcfa0..3f2b042360 100644
--- a/app/test-mldev/test_inference_common.h
+++ b/app/test-mldev/test_inference_common.h
@@ -43,6 +43,7 @@ struct test_inference {
struct rte_mempool *op_pool;
uint64_t nb_used;
+ uint64_t nb_valid;
int16_t fid;
int (*enqueue)(void *arg);
diff --git a/app/test-mldev/test_model_common.h b/app/test-mldev/test_model_common.h
index dfbf568f0b..ce12cbfecc 100644
--- a/app/test-mldev/test_model_common.h
+++ b/app/test-mldev/test_model_common.h
@@ -31,6 +31,7 @@ struct ml_model {
uint8_t *input;
uint8_t *output;
+ uint8_t *reference;
struct rte_mempool *io_pool;
uint32_t nb_batches;
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v1 11/12] app/mldev: enable reporting stats in mldev app
2022-11-29 7:07 [PATCH v1 00/12] implement mldev test application Srikanth Yalavarthi
` (9 preceding siblings ...)
2022-11-29 7:07 ` [PATCH v1 10/12] app/mldev: enable support for inference validation Srikanth Yalavarthi
@ 2022-11-29 7:07 ` Srikanth Yalavarthi
2022-11-29 7:07 ` [PATCH v1 12/12] app/mldev: add documentation for mldev test cases Srikanth Yalavarthi
` (4 subsequent siblings)
15 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2022-11-29 7:07 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj
Enable reporting driver xstats and inference end-to-end
latency and throughput in mldev inference tests. Reporting
of stats can be enabled using "--stats" option.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/ml_options.c | 22 ++--
app/test-mldev/ml_options.h | 2 +
app/test-mldev/test_inference_common.c | 139 +++++++++++++++++++++
app/test-mldev/test_inference_common.h | 8 ++
app/test-mldev/test_inference_interleave.c | 4 +
app/test-mldev/test_inference_ordered.c | 1 +
6 files changed, 168 insertions(+), 8 deletions(-)
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 4087ab52db..91052d4593 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -36,6 +36,7 @@ ml_options_default(struct ml_options *opt)
opt->queue_size = 1;
opt->batches = 0;
opt->tolerance = 0.0;
+ opt->stats = false;
opt->debug = false;
}
@@ -222,7 +223,8 @@ ml_dump_test_options(const char *testname)
"\t\t--queue_pairs : number of queue pairs to create\n"
"\t\t--queue_size : size fo queue-pair\n"
"\t\t--batches : number of batches of input\n"
- "\t\t--tolerance : maximum tolerance (%%) for output validation\n");
+ "\t\t--tolerance : maximum tolerance (%%) for output validation\n"
+ "\t\t--stats : enable reporting performance statistics\n");
printf("\n");
}
}
@@ -242,13 +244,12 @@ print_usage(char *program)
ml_test_dump_names(ml_dump_test_options);
}
-static struct option lgopts[] = {{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0},
- {ML_SOCKET_ID, 1, 0, 0}, {ML_MODELS, 1, 0, 0},
- {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
- {ML_BURST_SIZE, 1, 0, 0}, {ML_QUEUE_PAIRS, 1, 0, 0},
- {ML_QUEUE_SIZE, 1, 0, 0}, {ML_BATCHES, 1, 0, 0},
- {ML_TOLERANCE, 1, 0, 0}, {ML_DEBUG, 0, 0, 0},
- {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
+static struct option lgopts[] = {
+ {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
+ {ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
+ {ML_BURST_SIZE, 1, 0, 0}, {ML_QUEUE_PAIRS, 1, 0, 0}, {ML_QUEUE_SIZE, 1, 0, 0},
+ {ML_BATCHES, 1, 0, 0}, {ML_TOLERANCE, 1, 0, 0}, {ML_STATS, 0, 0, 0},
+ {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
static int
ml_opts_parse_long(int opt_idx, struct ml_options *opt)
@@ -283,6 +284,11 @@ ml_options_parse(struct ml_options *opt, int argc, char **argv)
while ((opts = getopt_long(argc, argv, "", lgopts, &opt_idx)) != EOF) {
switch (opts) {
case 0: /* parse long options */
+ if (!strcmp(lgopts[opt_idx].name, "stats")) {
+ opt->stats = true;
+ break;
+ }
+
if (!strcmp(lgopts[opt_idx].name, "debug")) {
opt->debug = true;
break;
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index 79ac54de98..a375ae6750 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -24,6 +24,7 @@
#define ML_QUEUE_SIZE ("queue_size")
#define ML_BATCHES ("batches")
#define ML_TOLERANCE ("tolerance")
+#define ML_STATS ("stats")
#define ML_DEBUG ("debug")
#define ML_HELP ("help")
@@ -46,6 +47,7 @@ struct ml_options {
uint16_t queue_size;
uint16_t batches;
float tolerance;
+ bool stats;
bool debug;
};
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
index cdd1667c71..d3f0211852 100644
--- a/app/test-mldev/test_inference_common.c
+++ b/app/test-mldev/test_inference_common.c
@@ -11,6 +11,7 @@
#include <unistd.h>
#include <rte_common.h>
+#include <rte_cycles.h>
#include <rte_hash_crc.h>
#include <rte_launch.h>
#include <rte_lcore.h>
@@ -45,6 +46,17 @@
} \
} while (0)
+static void
+print_line(uint16_t len)
+{
+ uint16_t i;
+
+ for (i = 0; i < len; i++)
+ printf("-");
+
+ printf("\n");
+}
+
/* Enqueue inference requests with burst size equal to 1 */
static int
ml_enqueue_single(void *arg)
@@ -54,6 +66,7 @@ ml_enqueue_single(void *arg)
struct rte_ml_op *op = NULL;
struct ml_core_args *args;
uint64_t model_enq = 0;
+ uint64_t start_cycle;
uint32_t burst_enq;
uint32_t lcore_id;
int16_t fid;
@@ -61,6 +74,7 @@ ml_enqueue_single(void *arg)
lcore_id = rte_lcore_id();
args = &t->args[lcore_id];
+ args->start_cycles = 0;
model_enq = 0;
if (args->nb_reqs == 0)
@@ -96,10 +110,12 @@ ml_enqueue_single(void *arg)
req->fid = fid;
enqueue_req:
+ start_cycle = rte_get_tsc_cycles();
burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, args->qp_id, &op, 1);
if (burst_enq == 0)
goto enqueue_req;
+ args->start_cycles += start_cycle;
fid++;
if (likely(fid <= args->end_fid))
goto next_model;
@@ -123,10 +139,12 @@ ml_dequeue_single(void *arg)
uint64_t total_deq = 0;
uint8_t nb_filelist;
uint32_t burst_deq;
+ uint64_t end_cycle;
uint32_t lcore_id;
lcore_id = rte_lcore_id();
args = &t->args[lcore_id];
+ args->end_cycles = 0;
nb_filelist = args->end_fid - args->start_fid + 1;
if (args->nb_reqs == 0)
@@ -134,9 +152,11 @@ ml_dequeue_single(void *arg)
dequeue_req:
burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, args->qp_id, &op, 1);
+ end_cycle = rte_get_tsc_cycles();
if (likely(burst_deq == 1)) {
total_deq += burst_deq;
+ args->end_cycles += end_cycle;
if (unlikely(op->status == RTE_ML_OP_STATUS_ERROR)) {
rte_ml_op_error_get(t->cmn.opt->dev_id, op, &error);
ml_err("error_code = 0x%" PRIx64 ", error_message = %s\n", error.errcode,
@@ -159,6 +179,7 @@ ml_enqueue_burst(void *arg)
{
struct test_inference *t = ml_test_priv((struct ml_test *)arg);
struct ml_core_args *args;
+ uint64_t start_cycle;
uint16_t ops_count;
uint64_t model_enq;
uint16_t burst_enq;
@@ -171,6 +192,7 @@ ml_enqueue_burst(void *arg)
lcore_id = rte_lcore_id();
args = &t->args[lcore_id];
+ args->start_cycles = 0;
model_enq = 0;
if (args->nb_reqs == 0)
@@ -212,8 +234,10 @@ ml_enqueue_burst(void *arg)
pending = ops_count;
enqueue_reqs:
+ start_cycle = rte_get_tsc_cycles();
burst_enq =
rte_ml_enqueue_burst(t->cmn.opt->dev_id, args->qp_id, &args->enq_ops[idx], pending);
+ args->start_cycles += burst_enq * start_cycle;
pending = pending - burst_enq;
if (pending > 0) {
@@ -243,11 +267,13 @@ ml_dequeue_burst(void *arg)
uint64_t total_deq = 0;
uint16_t burst_deq = 0;
uint8_t nb_filelist;
+ uint64_t end_cycle;
uint32_t lcore_id;
uint32_t i;
lcore_id = rte_lcore_id();
args = &t->args[lcore_id];
+ args->end_cycles = 0;
nb_filelist = args->end_fid - args->start_fid + 1;
if (args->nb_reqs == 0)
@@ -256,9 +282,11 @@ ml_dequeue_burst(void *arg)
dequeue_burst:
burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, args->qp_id, args->deq_ops,
t->cmn.opt->burst_size);
+ end_cycle = rte_get_tsc_cycles();
if (likely(burst_deq > 0)) {
total_deq += burst_deq;
+ args->end_cycles += burst_deq * end_cycle;
for (i = 0; i < burst_deq; i++) {
if (unlikely(args->deq_ops[i]->status == RTE_ML_OP_STATUS_ERROR)) {
@@ -387,6 +415,7 @@ test_inference_opt_dump(struct ml_options *opt)
ml_dump("queue_pairs", "%u", opt->queue_pairs);
ml_dump("queue_size", "%u", opt->queue_size);
ml_dump("tolerance", "%-7.3f", opt->tolerance);
+ ml_dump("stats", "%s", (opt->stats ? "true" : "false"));
if (opt->batches == 0)
ml_dump("batches", "%u (default)", opt->batches);
@@ -459,6 +488,11 @@ test_inference_setup(struct ml_test *test, struct ml_options *opt)
RTE_CACHE_LINE_SIZE, opt->socket_id);
}
+ for (i = 0; i < RTE_MAX_LCORE; i++) {
+ t->args[i].start_cycles = 0;
+ t->args[i].end_cycles = 0;
+ }
+
return 0;
error:
@@ -985,3 +1019,108 @@ ml_inference_launch_cores(struct ml_test *test, struct ml_options *opt, int16_t
return 0;
}
+
+int
+ml_inference_stats_get(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t = ml_test_priv(test);
+ uint64_t total_cycles = 0;
+ uint32_t nb_filelist;
+ uint64_t throughput;
+ uint64_t avg_e2e;
+ uint32_t qp_id;
+ uint64_t freq;
+ int ret;
+ int i;
+
+ if (!opt->stats)
+ return 0;
+
+ /* get xstats size */
+ t->xstats_size = rte_ml_dev_xstats_names_get(opt->dev_id, NULL, 0);
+ if (t->xstats_size >= 0) {
+ /* allocate for xstats_map and values */
+ t->xstats_map = rte_malloc(
+ "ml_xstats_map", t->xstats_size * sizeof(struct rte_ml_dev_xstats_map), 0);
+ if (t->xstats_map == NULL) {
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ t->xstats_values =
+ rte_malloc("ml_xstats_values", t->xstats_size * sizeof(uint64_t), 0);
+ if (t->xstats_values == NULL) {
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ ret = rte_ml_dev_xstats_names_get(opt->dev_id, t->xstats_map, t->xstats_size);
+ if (ret != t->xstats_size) {
+ printf("Unable to get xstats names, ret = %d\n", ret);
+ ret = -1;
+ goto error;
+ }
+
+ for (i = 0; i < t->xstats_size; i++)
+ rte_ml_dev_xstats_get(opt->dev_id, &t->xstats_map[i].id,
+ &t->xstats_values[i], 1);
+ }
+
+ /* print xstats*/
+ printf("\n");
+ print_line(80);
+ printf(" ML Device Extended Statistics\n");
+ print_line(80);
+ for (i = 0; i < t->xstats_size; i++)
+ printf(" %-64s = %" PRIu64 "\n", t->xstats_map[i].name, t->xstats_values[i]);
+ print_line(80);
+
+ /* release buffers */
+ if (t->xstats_map)
+ rte_free(t->xstats_map);
+
+ if (t->xstats_values)
+ rte_free(t->xstats_values);
+
+ /* print end-to-end stats */
+ freq = rte_get_tsc_hz();
+ for (qp_id = 0; qp_id < RTE_MAX_LCORE; qp_id++)
+ total_cycles += t->args[qp_id].end_cycles - t->args[qp_id].start_cycles;
+ avg_e2e = total_cycles / opt->repetitions;
+
+ if (freq == 0) {
+ avg_e2e = total_cycles / opt->repetitions;
+ printf(" %-64s = %" PRIu64 "\n", "Average End-to-End Latency (cycles)", avg_e2e);
+ } else {
+ avg_e2e = (total_cycles * NS_PER_S) / (opt->repetitions * freq);
+ printf(" %-64s = %" PRIu64 "\n", "Average End-to-End Latency (ns)", avg_e2e);
+ }
+
+ if (strcmp(opt->test_name, "inference_ordered") == 0)
+ nb_filelist = 1;
+ else
+ nb_filelist = t->cmn.opt->nb_filelist;
+
+ if (freq == 0) {
+ throughput = (nb_filelist * t->cmn.opt->repetitions * 1000000) / total_cycles;
+ printf(" %-64s = %" PRIu64 "\n", "Average Throughput (inferences / million cycles)",
+ throughput);
+ } else {
+ throughput = (nb_filelist * t->cmn.opt->repetitions * freq) / total_cycles;
+ printf(" %-64s = %" PRIu64 "\n", "Average Throughput (inferences / second)",
+ throughput);
+ }
+
+ print_line(80);
+
+ return 0;
+
+error:
+ if (t->xstats_map)
+ rte_free(t->xstats_map);
+
+ if (t->xstats_values)
+ rte_free(t->xstats_values);
+
+ return ret;
+}
diff --git a/app/test-mldev/test_inference_common.h b/app/test-mldev/test_inference_common.h
index 3f2b042360..bb2920cc30 100644
--- a/app/test-mldev/test_inference_common.h
+++ b/app/test-mldev/test_inference_common.h
@@ -32,6 +32,9 @@ struct ml_core_args {
struct rte_ml_op **enq_ops;
struct rte_ml_op **deq_ops;
struct ml_request **reqs;
+
+ uint64_t start_cycles;
+ uint64_t end_cycles;
};
struct test_inference {
@@ -50,6 +53,10 @@ struct test_inference {
int (*dequeue)(void *arg);
struct ml_core_args args[RTE_MAX_LCORE];
+
+ struct rte_ml_dev_xstats_map *xstats_map;
+ uint64_t *xstats_values;
+ int xstats_size;
} __rte_cache_aligned;
bool test_inference_cap_check(struct ml_options *opt);
@@ -67,5 +74,6 @@ void ml_inference_mem_destroy(struct ml_test *test, struct ml_options *opt);
int ml_inference_result(struct ml_test *test, struct ml_options *opt, int16_t fid);
int ml_inference_launch_cores(struct ml_test *test, struct ml_options *opt, int16_t start_fid,
int16_t end_fid);
+int ml_inference_stats_get(struct ml_test *test, struct ml_options *opt);
#endif /* _ML_TEST_INFERENCE_COMMON_ */
diff --git a/app/test-mldev/test_inference_interleave.c b/app/test-mldev/test_inference_interleave.c
index 74ad0c597f..d86838c3fa 100644
--- a/app/test-mldev/test_inference_interleave.c
+++ b/app/test-mldev/test_inference_interleave.c
@@ -60,7 +60,11 @@ test_inference_interleave_driver(struct ml_test *test, struct ml_options *opt)
goto error;
ml_inference_iomem_destroy(test, opt, fid);
+ }
+
+ ml_inference_stats_get(test, opt);
+ for (fid = 0; fid < opt->nb_filelist; fid++) {
ret = ml_model_stop(test, opt, &t->model[fid], fid);
if (ret != 0)
goto error;
diff --git a/app/test-mldev/test_inference_ordered.c b/app/test-mldev/test_inference_ordered.c
index 84e6bf9109..3826121a65 100644
--- a/app/test-mldev/test_inference_ordered.c
+++ b/app/test-mldev/test_inference_ordered.c
@@ -58,6 +58,7 @@ test_inference_ordered_driver(struct ml_test *test, struct ml_options *opt)
goto error;
ml_inference_iomem_destroy(test, opt, fid);
+ ml_inference_stats_get(test, opt);
/* stop model */
ret = ml_model_stop(test, opt, &t->model[fid], fid);
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v1 12/12] app/mldev: add documentation for mldev test cases
2022-11-29 7:07 [PATCH v1 00/12] implement mldev test application Srikanth Yalavarthi
` (10 preceding siblings ...)
2022-11-29 7:07 ` [PATCH v1 11/12] app/mldev: enable reporting stats in mldev app Srikanth Yalavarthi
@ 2022-11-29 7:07 ` Srikanth Yalavarthi
2023-02-07 15:49 ` [PATCH v4 00/12] Implementation of mldev test application Srikanth Yalavarthi
` (3 subsequent siblings)
15 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2022-11-29 7:07 UTC (permalink / raw)
To: Thomas Monjalon, Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj
Added documentation specific to mldev test cases. Added details
about all test cases and option supported by individual tests.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
MAINTAINERS | 1 +
.../tools/img/mldev_inference_interleave.svg | 667 ++++++++++++++++++
.../tools/img/mldev_inference_ordered.svg | 526 ++++++++++++++
.../tools/img/mldev_model_ops_subtest_a.svg | 418 +++++++++++
.../tools/img/mldev_model_ops_subtest_b.svg | 421 +++++++++++
.../tools/img/mldev_model_ops_subtest_c.svg | 364 ++++++++++
.../tools/img/mldev_model_ops_subtest_d.svg | 422 +++++++++++
doc/guides/tools/index.rst | 1 +
doc/guides/tools/testmldev.rst | 441 ++++++++++++
9 files changed, 3261 insertions(+)
create mode 100644 doc/guides/tools/img/mldev_inference_interleave.svg
create mode 100644 doc/guides/tools/img/mldev_inference_ordered.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_a.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_b.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_c.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_d.svg
create mode 100644 doc/guides/tools/testmldev.rst
diff --git a/MAINTAINERS b/MAINTAINERS
index 1edea42fad..1cddd6ead2 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -540,6 +540,7 @@ M: Srikanth Yalavarthi <syalavarthi@marvell.com>
F: lib/mldev/
F: app/test-mldev/
F: doc/guides/prog_guide/mldev.rst
+F: doc/guides/tools/testmldev.rst
Memory Pool Drivers
diff --git a/doc/guides/tools/img/mldev_inference_interleave.svg b/doc/guides/tools/img/mldev_inference_interleave.svg
new file mode 100644
index 0000000000..517c53d294
--- /dev/null
+++ b/doc/guides/tools/img/mldev_inference_interleave.svg
@@ -0,0 +1,667 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="224mm"
+ height="150mm"
+ viewBox="0 0 224 150"
+ version="1.1"
+ id="svg5369"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="inference_interleave.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns:xlink="http://www.w3.org/1999/xlink"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg">
+ <sodipodi:namedview
+ id="namedview5371"
+ pagecolor="#ffffff"
+ bordercolor="#000000"
+ borderopacity="0.25"
+ inkscape:showpageshadow="2"
+ inkscape:pageopacity="0.0"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1"
+ inkscape:document-units="mm"
+ showgrid="false"
+ inkscape:zoom="0.74564394"
+ inkscape:cx="415.07747"
+ inkscape:cy="348.6919"
+ inkscape:window-width="1920"
+ inkscape:window-height="1017"
+ inkscape:window-x="1912"
+ inkscape:window-y="-8"
+ inkscape:window-maximized="1"
+ inkscape:current-layer="layer1" />
+ <defs
+ id="defs5366">
+ <linearGradient
+ inkscape:collect="always"
+ id="linearGradient19189">
+ <stop
+ style="stop-color:#ffeeaa;stop-opacity:1;"
+ offset="0"
+ id="stop19185" />
+ <stop
+ style="stop-color:#ffeeaa;stop-opacity:0;"
+ offset="1"
+ id="stop19187" />
+ </linearGradient>
+ <marker
+ style="overflow:visible"
+ id="TriangleStart"
+ refX="4"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="TriangleStart"
+ markerWidth="5.3244081"
+ markerHeight="6.155385"
+ viewBox="0 0 5.3244081 6.1553851"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.5)"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:context-stroke;stroke-width:1pt"
+ d="M 5.77,0 -2.88,5 V -5 Z"
+ id="path135" />
+ </marker>
+ <marker
+ style="overflow:visible"
+ id="RoundedArrow"
+ refX="5"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="RoundedArrow"
+ markerWidth="6.1347523"
+ markerHeight="5.9304948"
+ viewBox="0 0 6.1347524 5.9304951"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.7)"
+ d="m -0.21114562,-4.1055728 6.42229122,3.21114561 a 1,1 90 0 1 0,1.78885438 L -0.21114562,4.1055728 A 1.236068,1.236068 31.717474 0 1 -2,3 v -6 a 1.236068,1.236068 148.28253 0 1 1.78885438,-1.1055728 z"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:none"
+ id="path1367" />
+ </marker>
+ <linearGradient
+ inkscape:collect="always"
+ xlink:href="#linearGradient19189"
+ id="linearGradient19191"
+ x1="12.169352"
+ y1="105"
+ x2="284.83066"
+ y2="105"
+ gradientUnits="userSpaceOnUse"
+ gradientTransform="matrix(0.79055445,0,0,0.74078976,29.505892,28.991272)" />
+ </defs>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1">
+ <g
+ id="g1477"
+ transform="translate(-34.903236,-31.774189)">
+ <rect
+ style="fill:url(#linearGradient19191);fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.396267;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect16635"
+ width="215.18147"
+ height="139.60078"
+ x="39.3125"
+ y="36.973797"
+ ry="2.2354064" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-169.3954"
+ y="214.97237"
+ id="text5181"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan5179"
+ style="font-size:5.64444px;stroke-width:0.75"
+ x="-169.3954"
+ y="214.97237">test: inference_interleave</tspan></text>
+ <path
+ style="display:inline;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 138.05133,87.886263 17.45982,-10e-7"
+ id="path1912"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1724-0"
+ inkscape:connection-end="#rect1679" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 191.51115,87.886262 17.45981,10e-7"
+ id="path1914"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1679"
+ inkscape:connection-end="#rect1724" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 138.05133,109.78102 17.45982,-1e-5"
+ id="path1916"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1724-4-8"
+ inkscape:connection-end="#rect1679-4" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 191.51115,109.78101 17.45981,1e-5"
+ id="path1918"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1679-4"
+ inkscape:connection-end="#rect1724-4" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 138.05133,131.67576 17.45982,-1e-5"
+ id="path1920"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1724-6-7"
+ inkscape:connection-end="#rect1679-8" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 191.51115,131.67575 17.45981,1e-5"
+ id="path1922"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1679-8"
+ inkscape:connection-end="#rect1724-6" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.75;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;marker-start:url(#RoundedArrow);marker-end:url(#RoundedArrow)"
+ d="m 173.51116,60.08164 0,12.907336"
+ id="path1933"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1811"
+ inkscape:connection-end="#rect1924" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.368668;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.737336, 0.737336;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924"
+ width="46.97015"
+ height="73.58287"
+ x="150.02565"
+ y="72.988976"
+ ry="2.4685853" />
+ <text
+ xml:space="preserve"
+ style="font-size:6.35px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-84.44075"
+ y="173.5065"
+ id="text4097"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4095"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-84.44075"
+ y="173.5065">Queue</tspan><tspan
+ sodipodi:role="line"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-92.37825"
+ y="173.5065"
+ id="tspan4099">Pair 0</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:6.35px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-128.2318"
+ y="173.5065"
+ id="text4097-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4095-6"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-128.2318"
+ y="173.5065">Queue</tspan><tspan
+ sodipodi:role="line"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-136.1693"
+ y="173.5065"
+ id="tspan4099-4">Pair 2</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1679"
+ width="36"
+ height="18"
+ x="155.51115"
+ y="78.886261"
+ ry="3" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1679-8"
+ width="36"
+ height="18"
+ x="155.51115"
+ y="122.67575"
+ ry="3" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1679-4"
+ width="36"
+ height="18"
+ x="155.51115"
+ y="100.78101"
+ ry="3" />
+ <text
+ xml:space="preserve"
+ style="font-size:6.35px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-106.33705"
+ y="173.5065"
+ id="text4097-8"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4095-4"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-106.33705"
+ y="173.5065">Queue</tspan><tspan
+ sodipodi:role="line"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-114.27455"
+ y="173.5065"
+ id="tspan4099-5">Pair 1</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.388863;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1811"
+ width="58.909527"
+ height="18.812746"
+ x="144.0564"
+ y="41.268894"
+ ry="2.2255962" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-47.017281"
+ y="173.49187"
+ id="text4156"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4154"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';stroke-width:0.75"
+ x="-47.017281"
+ y="173.49187">Machine Learning</tspan><tspan
+ sodipodi:role="line"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';stroke-width:0.75"
+ x="-54.07283"
+ y="173.49187"
+ id="tspan4158">Hardware Engine</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.368668;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.737336, 0.737336;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924-0"
+ width="46.97015"
+ height="73.58287"
+ x="98.42067"
+ y="72.988976"
+ ry="2.4685853" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-88.32518"
+ y="122.24379"
+ id="text3708"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-88.32518"
+ y="122.24379">lcore 1</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-132.10504"
+ y="121.83865"
+ id="text3708-8"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-7"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-132.10504"
+ y="121.83865">lcore 5</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-0"
+ width="32.290321"
+ height="11.709678"
+ x="105.76101"
+ y="82.031425"
+ ry="3.0161259" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-6-7"
+ width="32.290321"
+ height="11.709678"
+ x="105.76101"
+ y="125.82092"
+ ry="3.0161259"
+ inkscape:connector-avoid="true" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-4-8"
+ width="32.290321"
+ height="11.709678"
+ x="105.76101"
+ y="103.92618"
+ ry="3.0161259"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-110.21718"
+ y="121.85381"
+ id="text3708-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-87"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-110.21718"
+ y="121.85381">lcore 3</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-150.82878"
+ y="121.79179"
+ id="text4542"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4540"
+ style="font-size:5.64444px;stroke-width:0.75"
+ x="-150.82878"
+ y="121.79179">Enqueue Workers</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.368668;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.737336, 0.737336;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924-7"
+ width="46.97015"
+ height="73.58287"
+ x="201.63062"
+ y="72.988976"
+ ry="2.4685853" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-88.32518"
+ y="225.08443"
+ id="text3708-9"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-9"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-88.32518"
+ y="225.08443">lcore 2</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-110.21167"
+ y="225.07202"
+ id="text3708-7"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-8"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-110.21167"
+ y="225.07202">lcore 4</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724"
+ width="32.290321"
+ height="11.709678"
+ x="208.97096"
+ y="82.031425"
+ ry="3.0161259" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-4"
+ width="32.290321"
+ height="11.709678"
+ x="208.97096"
+ y="103.92618"
+ ry="3.0161259" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-6"
+ width="32.290321"
+ height="11.709678"
+ x="208.97096"
+ y="125.82092"
+ ry="3.0161259" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-132.1133"
+ y="225.06514"
+ id="text3708-78"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-0"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-132.1133"
+ y="225.06514">lcore 6</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-150.82878"
+ y="225.00725"
+ id="text4542-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4540-7"
+ style="font-size:5.64444px;stroke-width:0.75"
+ x="-150.82878"
+ y="225.00725">Dequeue Workers</tspan></text>
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 79.368074,78.962117 26.440446,5.546991"
+ id="path6677"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-0"
+ inkscape:connection-start="#rect6252" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="M 78.978262,84.050781 112.13805,103.92618"
+ id="path6679"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-4-8"
+ inkscape:connection-start="#rect6252" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 73.959073,84.25738 42.026977,41.56354"
+ id="path6681"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-6-7"
+ inkscape:connection-start="#rect6252" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="M 79.368074,95.959838 105.76913,90.949016"
+ id="path6683"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-0"
+ inkscape:connection-start="#rect6252-8" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 79.368074,101.28215 26.416716,5.27791"
+ id="path7830"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-4-8"
+ inkscape:connection-start="#rect6252-8" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 79.069199,106.4283 32.903161,19.39262"
+ id="path7862"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-6-7"
+ inkscape:connection-start="#rect6252-8" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="M 79.069201,113.13371 111.97235,93.741103"
+ id="path7900"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-0"
+ inkscape:connection-start="#rect6252-2" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 79.368074,118.27987 26.416716,-5.2779"
+ id="path7932"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-4-8"
+ inkscape:connection-start="#rect6252-2" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 79.368074,123.60218 26.401056,5.01083"
+ id="path7998"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-6-7"
+ inkscape:connection-start="#rect6252-2" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="M 73.959074,135.30464 115.98605,93.741103"
+ id="path8000"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-0"
+ inkscape:connection-start="#rect6252-6" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="M 78.978264,135.51124 112.13805,115.63586"
+ id="path8002"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-4-8"
+ inkscape:connection-start="#rect6252-6" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 79.368074,140.59991 26.440446,-5.54699"
+ id="path8004"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-6-7"
+ inkscape:connection-start="#rect6252-6" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-76.53363"
+ y="65.63237"
+ id="text3708-84"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-4"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-76.53363"
+ y="65.63237">Model 0</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-98.991623"
+ y="66.015465"
+ id="text3708-84-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-4-6"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-98.991623"
+ y="66.015465">Model 1</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-121.44823"
+ y="65.646149"
+ id="text3708-84-9"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-4-1"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-121.44823"
+ y="65.646149">Model 2</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-143.9021"
+ y="65.625481"
+ id="text3708-84-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-4-8"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-143.9021"
+ y="65.625481">Model 3</tspan></text>
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect6252"
+ width="27.32258"
+ height="16.32258"
+ x="52.045494"
+ y="67.934799"
+ ry="1" />
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect6252-8"
+ width="27.32258"
+ height="16.32258"
+ x="52.045494"
+ y="90.391411"
+ ry="1" />
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect6252-2"
+ width="27.32258"
+ height="16.32258"
+ x="52.045494"
+ y="112.84802"
+ ry="1" />
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect6252-6"
+ width="27.32258"
+ height="16.32258"
+ x="52.045494"
+ y="135.30464"
+ ry="1" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.399886;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.799773, 0.799773;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924-1-3"
+ width="43.035683"
+ height="94.487152"
+ x="44.188942"
+ y="62.536991"
+ ry="3.1694498" />
+ <g
+ id="g1010"
+ transform="translate(21.464467,-15.875004)">
+ <text
+ xml:space="preserve"
+ style="font-weight:bold;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:'Arial Bold';text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-185.88483"
+ y="86.542366"
+ id="text15571"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan15569"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.5;stroke-opacity:1"
+ x="-185.88483"
+ y="86.542366">nb_worker_threads = 2 * MIN(nb_queue_pairs, (lcore_count - 1) / 2)</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-weight:bold;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:'Arial Bold';text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-178.43243"
+ y="90.903854"
+ id="text15571-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan15569-9"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.5;stroke-opacity:1"
+ x="-178.43243"
+ y="90.903854">inferences_per_queue_pair = nb_models * (repetitions / nb_queue_pairs)</tspan></text>
+ </g>
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/tools/img/mldev_inference_ordered.svg b/doc/guides/tools/img/mldev_inference_ordered.svg
new file mode 100644
index 0000000000..9d2b2c9246
--- /dev/null
+++ b/doc/guides/tools/img/mldev_inference_ordered.svg
@@ -0,0 +1,526 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="243mm"
+ height="144mm"
+ viewBox="0 0 243 144"
+ version="1.1"
+ id="svg5"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="inference_ordered.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns:xlink="http://www.w3.org/1999/xlink"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg">
+ <sodipodi:namedview
+ id="namedview7"
+ pagecolor="#ffffff"
+ bordercolor="#000000"
+ borderopacity="0.25"
+ inkscape:showpageshadow="2"
+ inkscape:pageopacity="0.0"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1"
+ inkscape:document-units="mm"
+ showgrid="false"
+ inkscape:zoom="0.74564394"
+ inkscape:cx="488.83922"
+ inkscape:cy="234.69647"
+ inkscape:window-width="1920"
+ inkscape:window-height="1017"
+ inkscape:window-x="1912"
+ inkscape:window-y="-8"
+ inkscape:window-maximized="1"
+ inkscape:current-layer="layer1" />
+ <defs
+ id="defs2">
+ <marker
+ style="overflow:visible"
+ id="RoundedArrow"
+ refX="5"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="RoundedArrow"
+ markerWidth="6.1347523"
+ markerHeight="5.9304948"
+ viewBox="0 0 6.1347524 5.9304951"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.7)"
+ d="m -0.21114562,-4.1055728 6.42229122,3.21114561 a 1,1 90 0 1 0,1.78885438 L -0.21114562,4.1055728 A 1.236068,1.236068 31.717474 0 1 -2,3 v -6 a 1.236068,1.236068 148.28253 0 1 1.78885438,-1.1055728 z"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:none"
+ id="path1367" />
+ </marker>
+ <linearGradient
+ inkscape:collect="always"
+ id="linearGradient31002">
+ <stop
+ style="stop-color:#fff6d5;stop-opacity:1;"
+ offset="0"
+ id="stop30998" />
+ <stop
+ style="stop-color:#fff6d5;stop-opacity:0;"
+ offset="1"
+ id="stop31000" />
+ </linearGradient>
+ <marker
+ style="overflow:visible"
+ id="TriangleStart"
+ refX="4"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="TriangleStart"
+ markerWidth="5.3244081"
+ markerHeight="6.155385"
+ viewBox="0 0 5.3244081 6.1553851"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.5)"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:context-stroke;stroke-width:1pt"
+ d="M 5.77,0 -2.88,5 V -5 Z"
+ id="path135" />
+ </marker>
+ <linearGradient
+ inkscape:collect="always"
+ xlink:href="#linearGradient31002"
+ id="linearGradient31004"
+ x1="19.620968"
+ y1="102.90323"
+ x2="279.1532"
+ y2="102.90323"
+ gradientUnits="userSpaceOnUse"
+ gradientTransform="matrix(0.89215122,0,0,0.73190287,13.449912,42.668706)" />
+ </defs>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1">
+ <g
+ id="g1340"
+ transform="translate(-25.225796,-45.983871)">
+ <rect
+ style="fill:url(#linearGradient31004);fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.404032;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect27876"
+ width="231.09595"
+ height="132.45081"
+ x="31.177822"
+ y="51.758469"
+ ry="3.5071263" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1813"
+ width="38.408459"
+ height="45.86002"
+ x="34.901794"
+ y="99.14959"
+ ry="5.2246051"
+ inkscape:connector-avoid="true" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 73.310253,115.94935 36.498807,-11.6509"
+ id="path1906"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1813"
+ inkscape:connection-end="#rect1724-0" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 73.310253,122.0796 36.117817,1e-5"
+ id="path1908"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1813"
+ inkscape:connection-end="#rect1724-4-8" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="M 73.310253,128.20983 109.80905,139.8607"
+ id="path1910"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1813"
+ inkscape:connection-end="#rect1724-6-7" />
+ <path
+ style="display:inline;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 141.71839,99.266314 19.42262,-10e-7"
+ id="path1912"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1724-0"
+ inkscape:connection-end="#rect1679" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 197.14101,99.266313 19.42259,10e-7"
+ id="path1914"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1679"
+ inkscape:connection-end="#rect1724" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 141.71839,122.07961 19.42262,-1e-5"
+ id="path1916"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1724-4-8"
+ inkscape:connection-end="#rect1679-4" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 197.14101,122.0796 19.42259,1e-5"
+ id="path1918"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1679-4"
+ inkscape:connection-end="#rect1724-4" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 141.71839,144.89282 19.42262,0"
+ id="path1920"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1724-6-7"
+ inkscape:connection-end="#rect1679-8" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 197.14101,144.89282 19.42259,0"
+ id="path1922"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1679-8"
+ inkscape:connection-end="#rect1724-6" />
+ <text
+ xml:space="preserve"
+ style="font-size:8.46667px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-121.09793"
+ y="54.031597"
+ id="text4093"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4091"
+ style="font-size:8.46667px;stroke-width:0.75"
+ x="-121.09793"
+ y="54.031597">Model X</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:6.35px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-118.63563"
+ y="179.13635"
+ id="text4097"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4095"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-118.63563"
+ y="179.13635">Queue</tspan><tspan
+ sodipodi:role="line"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-126.57313"
+ y="179.13635"
+ id="tspan4099">Pair 1</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:6.35px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-141.44887"
+ y="179.13635"
+ id="text4097-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4095-6"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-141.44887"
+ y="179.13635">Queue</tspan><tspan
+ sodipodi:role="line"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-149.38637"
+ y="179.13635"
+ id="tspan4099-4">Pair 2</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.354849;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.709699, 0.709699;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924"
+ width="44.145252"
+ height="72.532341"
+ x="157.06865"
+ y="85.813438"
+ ry="4.31247" />
+ <g
+ id="g1224">
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1679"
+ width="36"
+ height="18"
+ x="161.14101"
+ y="90.266312"
+ ry="3" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1679-8"
+ width="36"
+ height="18"
+ x="161.14101"
+ y="135.89282"
+ ry="3" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1679-4"
+ width="36"
+ height="18"
+ x="161.14101"
+ y="113.07959"
+ ry="3" />
+ </g>
+ <text
+ xml:space="preserve"
+ style="font-size:6.35px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-95.820801"
+ y="179.13635"
+ id="text4097-8"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4095-4"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-95.820801"
+ y="179.13635">Queue</tspan><tspan
+ sodipodi:role="line"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-103.7583"
+ y="179.13635"
+ id="tspan4099-5">Pair 0</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.317648;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1811"
+ width="44.196934"
+ height="16.731901"
+ x="157.04254"
+ y="56.49292"
+ ry="2.761292" />
+ <text
+ xml:space="preserve"
+ style="font-size:3.5859px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.317649;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.952945, 0.317649;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-60.009941"
+ y="186.38451"
+ id="text4156"
+ transform="matrix(0,-1.040508,0.96106903,0,0,0)"><tspan
+ sodipodi:role="line"
+ id="tspan4154"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-family:Arial;-inkscape-font-specification:'Arial Bold';stroke-width:0.317648"
+ x="-60.009941"
+ y="186.38451">Machine Learning</tspan><tspan
+ sodipodi:role="line"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-family:Arial;-inkscape-font-specification:'Arial Bold';stroke-width:0.317648"
+ x="-64.492317"
+ y="186.38451"
+ id="tspan4158">Hardware Engine</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-99.705231"
+ y="125.91087"
+ id="text3708"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-99.705231"
+ y="125.91087">lcore 1</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-145.3221"
+ y="125.50572"
+ id="text3708-8"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-7"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-145.3221"
+ y="125.50572">lcore 5</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-122.51577"
+ y="125.52089"
+ id="text3708-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-87"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-122.51577"
+ y="125.52089">lcore 3</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-162.06549"
+ y="125.4589"
+ id="text4542"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4540"
+ style="font-size:5.64444px;stroke-width:0.75"
+ x="-162.06549"
+ y="125.4589">Enqueue Workers</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-99.705231"
+ y="232.67706"
+ id="text3708-9"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-9"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-99.705231"
+ y="232.67706">lcore 2</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-122.51025"
+ y="232.66466"
+ id="text3708-7"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-8"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-122.51025"
+ y="232.66466">lcore 4</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-145.33035"
+ y="232.65778"
+ id="text3708-78"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-0"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-145.33035"
+ y="232.65778">lcore 6</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-162.06549"
+ y="232.59988"
+ id="text4542-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4540-7"
+ style="font-size:5.64444px;stroke-width:0.75"
+ x="-162.06549"
+ y="232.59988">Dequeue Workers</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:6.35px;font-family:Arial;-inkscape-font-specification:'Arial Bold';text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-177.01665"
+ y="220.07283"
+ id="text5181"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan5179"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-177.01665"
+ y="220.07283">test: inference_ordered</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-weight:bold;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:'Arial Bold';text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-178.63324"
+ y="98.67057"
+ id="text15571"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan15569"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.5;stroke-opacity:1"
+ x="-178.63324"
+ y="98.67057">nb_worker_threads = 2 * MIN(nb_queue_pairs, (lcore_count - 1) / 2)</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-weight:bold;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:'Arial Bold';text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-171.18085"
+ y="89.26754"
+ id="text15571-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan15569-9"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.5;stroke-opacity:1"
+ x="-171.18085"
+ y="89.26754">inferences_per_queue_pair = repetitions / nb_queue_pairs</tspan></text>
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.75;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#RoundedArrow);marker-end:url(#RoundedArrow)"
+ d="m 179.14101,85.813438 0,-12.588618"
+ id="path31090"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1924" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724"
+ width="32.290321"
+ height="11.709678"
+ x="216.5636"
+ y="93.411476"
+ ry="2" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-4"
+ width="32.290321"
+ height="11.709678"
+ x="216.5636"
+ y="116.22478"
+ ry="2" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-6"
+ width="32.290321"
+ height="11.709678"
+ x="216.5636"
+ y="139.03798"
+ ry="2" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.354849;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.709699, 0.709699;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924-6"
+ width="44.145252"
+ height="72.532341"
+ x="210.6364"
+ y="85.813477"
+ ry="4.31247" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-0"
+ width="32.290321"
+ height="11.709678"
+ x="109.42807"
+ y="93.411476"
+ ry="2" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-6-7"
+ width="32.290321"
+ height="11.709678"
+ x="109.42807"
+ y="139.03798"
+ ry="2"
+ inkscape:connector-avoid="true" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-4-8"
+ width="32.290321"
+ height="11.709678"
+ x="109.42807"
+ y="116.22478"
+ ry="2"
+ inkscape:connector-avoid="true" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.354849;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.709699, 0.709699;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924-6-3"
+ width="44.145252"
+ height="72.532341"
+ x="103.50092"
+ y="85.813477"
+ ry="4.31247" />
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/tools/img/mldev_model_ops_subtest_a.svg b/doc/guides/tools/img/mldev_model_ops_subtest_a.svg
new file mode 100644
index 0000000000..cce5c3be7c
--- /dev/null
+++ b/doc/guides/tools/img/mldev_model_ops_subtest_a.svg
@@ -0,0 +1,418 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="275mm"
+ height="135mm"
+ viewBox="0 0 275 135"
+ version="1.1"
+ id="svg5"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="mldev_model_ops_subtest_d.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg">
+ <sodipodi:namedview
+ id="namedview7"
+ pagecolor="#ffffff"
+ bordercolor="#000000"
+ borderopacity="0.25"
+ inkscape:showpageshadow="2"
+ inkscape:pageopacity="0.0"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1"
+ inkscape:document-units="mm"
+ showgrid="false"
+ inkscape:zoom="0.72515432"
+ inkscape:cx="372.33454"
+ inkscape:cy="401.29389"
+ inkscape:window-width="1920"
+ inkscape:window-height="986"
+ inkscape:window-x="-11"
+ inkscape:window-y="-11"
+ inkscape:window-maximized="1"
+ inkscape:current-layer="layer1" />
+ <defs
+ id="defs2">
+ <marker
+ style="overflow:visible"
+ id="RoundedArrow"
+ refX="6"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="RoundedArrow"
+ markerWidth="6.1347523"
+ markerHeight="5.9304948"
+ viewBox="0 0 6.1347524 5.9304951"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.7)"
+ d="m -0.21114562,-4.1055728 6.42229122,3.21114561 a 1,1 90 0 1 0,1.78885438 L -0.21114562,4.1055728 A 1.236068,1.236068 31.717474 0 1 -2,3 v -6 a 1.236068,1.236068 148.28253 0 1 1.78885438,-1.1055728 z"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:none"
+ id="path1367" />
+ </marker>
+ </defs>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1">
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 62.203489,25.425124 H 80.823463"
+ id="path1916"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234"
+ inkscape:connection-end="#rect234-07" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 128.19001,25.425124 h 18.6197"
+ id="path1918"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-07"
+ inkscape:connection-end="#rect234-6" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 194.17681,25.425125 h 18.61942"
+ id="path1922"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-6"
+ inkscape:connection-end="#rect234-0-6-6" />
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.633453;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234"
+ width="47.366547"
+ height="17.366549"
+ x="14.836943"
+ y="16.74185"
+ ry="1.4985789"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-25.888845"
+ y="38.514706"
+ id="text290"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-25.888845"
+ y="38.514706">Model 0 / Load</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.633453;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-07"
+ width="47.366547"
+ height="17.366549"
+ x="80.823463"
+ y="16.74185"
+ ry="1.4985789"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-25.888845"
+ y="104.31795"
+ id="text290-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-11"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-25.888845"
+ y="104.31795">Model 0 / Start</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.6329;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-6"
+ width="47.3671"
+ height="17.33604"
+ x="146.80971"
+ y="16.757105"
+ ry="1.4959463" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-25.362436"
+ y="170.39679"
+ id="text290-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-15"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-25.362436"
+ y="170.39679">Model 0 / Stop</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6"
+ width="47.3671"
+ height="17.33604"
+ x="212.79623"
+ y="16.757105"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-25.888849"
+ y="236.47427"
+ id="text290-3-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-25.888849"
+ y="236.47427">Model 0 / Unload</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 236.47978,34.093145 V 44.957249 H 38.520216 v 10.84885"
+ id="path1924"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6"
+ inkscape:connection-end="#rect234-0-6-6-3" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 62.203766,64.474119 H 80.823189"
+ id="path1926"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-3"
+ inkscape:connection-end="#rect234-0-6-6-2" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 128.19029,64.474119 h 18.61942"
+ id="path1928"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-2"
+ inkscape:connection-end="#rect234-0-6-1" />
+ <rect
+ style="fill:#fff6d5;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-3"
+ width="47.3671"
+ height="17.33604"
+ x="14.836666"
+ y="55.806099"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-64.937843"
+ y="38.541786"
+ id="text290-3-2-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-64.937843"
+ y="38.541786">Model 1 / Load</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-2"
+ width="47.3671"
+ height="17.33604"
+ x="80.823189"
+ y="55.806099"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-64.937843"
+ y="104.31795"
+ id="text290-3-2-2-28"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-2"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-64.937843"
+ y="104.31795">Model 1 / Start</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6"
+ width="47.3671"
+ height="17.33604"
+ x="212.79623"
+ y="55.806099"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-64.937843"
+ y="236.47427"
+ id="text290-3-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-64.937843"
+ y="236.47427">Model 1 / Unload</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-1"
+ width="47.3671"
+ height="17.33604"
+ x="146.80971"
+ y="55.806099"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-64.41143"
+ y="170.39679"
+ id="text290-3-2-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-8"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-64.41143"
+ y="170.39679">Model 1 / Stop</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 194.17681,64.474119 h 18.61942"
+ id="path1930"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6"
+ inkscape:connection-start="#rect234-0-6-1" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 236.47978,73.142139 V 83.990988 H 38.520216 V 94.85511"
+ id="path1932"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6-7"
+ inkscape:connection-start="#rect234-0-6" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 62.203766,103.52313 h 9.309711 v 1.3e-4 h 9.309712"
+ id="path1934"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-7" />
+ <rect
+ style="fill:#fff6d5;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-7"
+ width="47.3671"
+ height="17.33604"
+ x="14.836666"
+ y="94.85511"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.98687"
+ y="38.541786"
+ id="text290-3-2-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.98687"
+ y="38.541786">Model N / Load</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0"
+ width="47.3671"
+ height="17.33604"
+ x="80.823189"
+ y="94.855164"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.9869"
+ y="104.31795"
+ id="text290-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.9869"
+ y="104.31795">Model N / Start</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-9"
+ width="47.3671"
+ height="17.33604"
+ x="146.80971"
+ y="94.855164"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.4605"
+ y="170.39679"
+ id="text290-3-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.4605"
+ y="170.39679">Model N / Stop</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.633452;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-5"
+ width="47.366547"
+ height="17.366549"
+ x="212.79651"
+ y="94.839836"
+ ry="1.4985789"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.98682"
+ y="236.47427"
+ id="text290-3-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-3"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.98682"
+ y="236.47427">Model N / Unload</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 128.19029,103.52326 h 18.61942"
+ id="path1936"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 194.17681,103.52326 h 9.30985 v -1.5e-4 h 9.30985"
+ id="path1938"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-5" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#fff6d5;fill-rule:evenodd;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-120.71075"
+ y="220.77164"
+ id="text392"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan390"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';fill:#000000;stroke-width:0.5"
+ x="-120.71075"
+ y="220.77164">mldev: model_ops / subtest D</tspan></text>
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.448724;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect2164"
+ width="259.55127"
+ height="119.55128"
+ x="7.7243652"
+ y="7.7243614"
+ ry="1.17494" />
+ </g>
+</svg>
diff --git a/doc/guides/tools/img/mldev_model_ops_subtest_b.svg b/doc/guides/tools/img/mldev_model_ops_subtest_b.svg
new file mode 100644
index 0000000000..53a49a2823
--- /dev/null
+++ b/doc/guides/tools/img/mldev_model_ops_subtest_b.svg
@@ -0,0 +1,421 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="275mm"
+ height="135mm"
+ viewBox="0 0 275 135"
+ version="1.1"
+ id="svg5"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="mldev_model_ops_subtest_a.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg">
+ <sodipodi:namedview
+ id="namedview7"
+ pagecolor="#ffffff"
+ bordercolor="#000000"
+ borderopacity="0.25"
+ inkscape:showpageshadow="2"
+ inkscape:pageopacity="0.0"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1"
+ inkscape:document-units="mm"
+ showgrid="false"
+ inkscape:zoom="0.72515432"
+ inkscape:cx="277.87189"
+ inkscape:cy="401.29389"
+ inkscape:window-width="1920"
+ inkscape:window-height="986"
+ inkscape:window-x="-11"
+ inkscape:window-y="-11"
+ inkscape:window-maximized="1"
+ inkscape:current-layer="layer1" />
+ <defs
+ id="defs2">
+ <marker
+ style="overflow:visible"
+ id="RoundedArrow"
+ refX="6"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="RoundedArrow"
+ markerWidth="6.1347523"
+ markerHeight="5.9304948"
+ viewBox="0 0 6.1347524 5.9304951"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.7)"
+ d="m -0.21114562,-4.1055728 6.42229122,3.21114561 a 1,1 90 0 1 0,1.78885438 L -0.21114562,4.1055728 A 1.236068,1.236068 31.717474 0 1 -2,3 v -6 a 1.236068,1.236068 148.28253 0 1 1.78885438,-1.1055728 z"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:none"
+ id="path1367" />
+ </marker>
+ </defs>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1">
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.671403;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234"
+ width="53.328598"
+ height="17.328598"
+ x="15.144302"
+ y="12.878438"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-22.00646"
+ y="41.803089"
+ id="text290"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-22.00646"
+ y="41.803089">Model 0 / Load</tspan></text>
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.671403;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-07"
+ width="53.328598"
+ height="17.328598"
+ x="87.552399"
+ y="12.878438"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-22.00646"
+ y="114.21132"
+ id="text290-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-11"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-22.00646"
+ y="114.21132">Model 1 / Load</tspan></text>
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.671403;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-6"
+ width="53.328598"
+ height="17.328598"
+ x="206.5271"
+ y="12.878438"
+ ry="1.4953041" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-22.00646"
+ y="233.18588"
+ id="text290-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-15"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-22.00646"
+ y="233.18588">Model N / Load</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 68.4729,21.542737 H 87.552399"
+ id="path1916"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234"
+ inkscape:connection-end="#rect234-07" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 140.881,21.542737 h 65.6461"
+ id="path1918"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-07"
+ inkscape:connection-end="#rect234-6" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 233.1914,30.207036 0,4.918771 H 41.808601 v 4.918772"
+ id="path1922"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-6"
+ inkscape:connection-end="#rect234-0-6-6" />
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6"
+ width="53.328598"
+ height="17.328598"
+ x="15.144302"
+ y="40.044579"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-49.1726"
+ y="41.619808"
+ id="text290-3-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-49.1726"
+ y="41.619808">Model 0 / Start</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-3"
+ width="53.328598"
+ height="17.328598"
+ x="87.552399"
+ y="40.044579"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-49.1726"
+ y="114.02805"
+ id="text290-3-2-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-49.1726"
+ y="114.02805">Model 1 / Start</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-2"
+ width="53.328598"
+ height="17.328598"
+ x="206.5271"
+ y="40.044579"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-49.1726"
+ y="233.00261"
+ id="text290-3-2-2-28"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-2"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-49.1726"
+ y="233.00261">Model N / Start</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 68.4729,48.708878 H 87.552399"
+ id="path1924"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6"
+ inkscape:connection-end="#rect234-0-6-6-3" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 140.881,48.708878 h 65.6461"
+ id="path1926"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-3"
+ inkscape:connection-end="#rect234-0-6-6-2" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 233.1914,57.373177 V 62.29195 H 41.808599 v 4.918774"
+ id="path1928"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-2"
+ inkscape:connection-end="#rect234-0-6-1" />
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6"
+ width="53.328598"
+ height="17.328598"
+ x="87.552399"
+ y="67.210724"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-75.812294"
+ y="114.12037"
+ id="text290-3-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-75.812294"
+ y="114.12037">Model 1 / Stop</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-1"
+ width="53.328598"
+ height="17.328598"
+ x="15.1443"
+ y="67.210724"
+ ry="1.4953041" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-75.812294"
+ y="41.712139"
+ id="text290-3-2-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-8"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-75.812294"
+ y="41.712139">Model 0 / Stop</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-7"
+ width="53.328598"
+ height="17.328598"
+ x="206.5271"
+ y="67.210724"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-75.812294"
+ y="233.09494"
+ id="text290-3-2-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-75.812294"
+ y="233.09494">Model N / Stop</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 68.472898,75.875023 H 87.552399"
+ id="path1930"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6"
+ inkscape:connection-start="#rect234-0-6-1" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 140.881,75.875023 h 65.6461"
+ id="path1932"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6-7"
+ inkscape:connection-start="#rect234-0-6" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 233.1914,84.539322 0,4.91877 H 41.808602 v 4.91877"
+ id="path1934"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0"
+ inkscape:connection-start="#rect234-0-6-7" />
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0"
+ width="53.328598"
+ height="17.328598"
+ x="15.144303"
+ y="94.376862"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.50489"
+ y="41.803085"
+ id="text290-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.50489"
+ y="41.803085">Model 0 / Unload</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-9"
+ width="53.328598"
+ height="17.328598"
+ x="87.552399"
+ y="94.376862"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.50489"
+ y="114.2113"
+ id="text290-3-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.50489"
+ y="114.2113">Model 1 / Unload</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-5"
+ width="53.328598"
+ height="17.328598"
+ x="206.5271"
+ y="94.376862"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.50489"
+ y="233.18588"
+ id="text290-3-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-3"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.50489"
+ y="233.18588">Model N / Unload</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 68.472901,103.04116 H 87.552399"
+ id="path1936"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-9"
+ inkscape:connection-start="#rect234-0" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 140.881,103.04116 h 65.6461"
+ id="path1938"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-5"
+ inkscape:connection-start="#rect234-0-9" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#fff6d5;fill-rule:evenodd;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-119.78458"
+ y="220.32892"
+ id="text392"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan390"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';fill:#000000;stroke-width:0.5"
+ x="-119.78458"
+ y="220.32892">mldev: model_ops / subtest A</tspan></text>
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.442854;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect3967"
+ width="259.55716"
+ height="119.55714"
+ x="7.7214203"
+ y="7.7214317"
+ ry="1.1806604" />
+ </g>
+</svg>
diff --git a/doc/guides/tools/img/mldev_model_ops_subtest_c.svg b/doc/guides/tools/img/mldev_model_ops_subtest_c.svg
new file mode 100644
index 0000000000..320d4978e3
--- /dev/null
+++ b/doc/guides/tools/img/mldev_model_ops_subtest_c.svg
@@ -0,0 +1,364 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="275mm"
+ height="135mm"
+ viewBox="0 0 275 135"
+ version="1.1"
+ id="svg5"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="mldev_model_ops_subtest_c.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg">
+ <sodipodi:namedview
+ id="namedview7"
+ pagecolor="#ffffff"
+ bordercolor="#000000"
+ borderopacity="0.25"
+ inkscape:showpageshadow="2"
+ inkscape:pageopacity="0.0"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1"
+ inkscape:document-units="mm"
+ showgrid="false"
+ inkscape:zoom="0.72515432"
+ inkscape:cx="479.89785"
+ inkscape:cy="401.29389"
+ inkscape:window-width="1920"
+ inkscape:window-height="986"
+ inkscape:window-x="-11"
+ inkscape:window-y="-11"
+ inkscape:window-maximized="1"
+ inkscape:current-layer="layer1" />
+ <defs
+ id="defs2">
+ <marker
+ style="overflow:visible"
+ id="RoundedArrow"
+ refX="6"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="RoundedArrow"
+ markerWidth="6.1347523"
+ markerHeight="5.9304948"
+ viewBox="0 0 6.1347524 5.9304951"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.7)"
+ d="m -0.21114562,-4.1055728 6.42229122,3.21114561 a 1,1 90 0 1 0,1.78885438 L -0.21114562,4.1055728 A 1.236068,1.236068 31.717474 0 1 -2,3 v -6 a 1.236068,1.236068 148.28253 0 1 1.78885438,-1.1055728 z"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:none"
+ id="path1367" />
+ </marker>
+ </defs>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1">
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.671403;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234"
+ width="53.328598"
+ height="17.328598"
+ x="12.6443"
+ y="13.208546"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-22.336561"
+ y="39.303089"
+ id="text290"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-22.336561"
+ y="39.303089">Model 0 / Load</tspan></text>
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.671403;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-07"
+ width="53.328598"
+ height="17.328598"
+ x="85.052399"
+ y="13.208546"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-22.336561"
+ y="111.71132"
+ id="text290-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-11"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-22.336561"
+ y="111.71132">Model 1 / Load</tspan></text>
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.671403;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-6"
+ width="53.328598"
+ height="17.328598"
+ x="204.0271"
+ y="13.208546"
+ ry="1.4953041" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-22.336561"
+ y="230.68588"
+ id="text290-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-15"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-22.336561"
+ y="230.68588">Model N / Load</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 65.972898,21.872845 H 85.052399"
+ id="path1916"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234"
+ inkscape:connection-end="#rect234-07" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 138.381,21.872845 h 65.6461"
+ id="path1918"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-07"
+ inkscape:connection-end="#rect234-6" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 230.6914,30.537144 V 36.33787 H 39.308599 v 5.800726"
+ id="path1922"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-6"
+ inkscape:connection-end="#rect234-0-6-6" />
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6"
+ width="53.328598"
+ height="17.328598"
+ x="12.6443"
+ y="42.138596"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-51.266617"
+ y="39.119808"
+ id="text290-3-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-51.266617"
+ y="39.119808">Model 0 / Start</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-3"
+ width="53.328598"
+ height="17.328598"
+ x="85.052399"
+ y="42.138596"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-51.266617"
+ y="111.52805"
+ id="text290-3-2-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-51.266617"
+ y="111.52805">Model 0 / Stop</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 65.972898,50.802895 H 85.052399"
+ id="path1924"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6"
+ inkscape:connection-end="#rect234-0-6-6-3" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 184.94759,79.732941 H 204.0271"
+ id="path1930"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6"
+ inkscape:connection-start="#rect234-0-6-1" />
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0"
+ width="53.328598"
+ height="17.328598"
+ x="12.644301"
+ y="99.998688"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-109.12671"
+ y="39.303085"
+ id="text290-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-109.12671"
+ y="39.303085">Model 0 / Unload</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-9"
+ width="53.328598"
+ height="17.328598"
+ x="85.052399"
+ y="99.998688"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-109.12671"
+ y="111.7113"
+ id="text290-3-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-109.12671"
+ y="111.7113">Model 1 / Unload</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-5"
+ width="53.328598"
+ height="17.328598"
+ x="204.0271"
+ y="99.998688"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-109.12671"
+ y="230.68588"
+ id="text290-3-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-3"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-109.12671"
+ y="230.68588">Model N / Unload</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 65.972899,108.66299 h 19.0795"
+ id="path1936"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-9"
+ inkscape:connection-start="#rect234-0" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 138.381,108.66299 h 65.6461"
+ id="path1938"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-5"
+ inkscape:connection-start="#rect234-0-9" />
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6"
+ width="53.328598"
+ height="17.328598"
+ x="204.0271"
+ y="71.068642"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-79.670258"
+ y="230.59494"
+ id="text290-3-2-2-28"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-2"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-79.670258"
+ y="230.59494">Model N / Stop</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-1"
+ width="53.328598"
+ height="17.328598"
+ x="131.61899"
+ y="71.068642"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-80.196663"
+ y="158.0945"
+ id="text290-3-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-80.196663"
+ y="158.0945">Model N / Start</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 138.381,50.802895 h 19.90229 v 20.265747"
+ id="path1158"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-3"
+ inkscape:connection-end="#rect234-0-6-1" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 230.6914,88.39724 v 5.800724 H 39.3086 v 5.800724"
+ id="path1160"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6"
+ inkscape:connection-end="#rect234-0" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#fff6d5;fill-rule:evenodd;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-123.28999"
+ y="217.99152"
+ id="text392"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan390"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';fill:#000000;stroke-width:0.5"
+ x="-123.28999"
+ y="217.99152">mldev: model_ops / subtest C</tspan></text>
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.441855;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect3015"
+ width="259.55814"
+ height="119.55814"
+ x="7.720932"
+ y="7.7209282"
+ ry="1.1638433" />
+ </g>
+</svg>
diff --git a/doc/guides/tools/img/mldev_model_ops_subtest_d.svg b/doc/guides/tools/img/mldev_model_ops_subtest_d.svg
new file mode 100644
index 0000000000..80c1798d99
--- /dev/null
+++ b/doc/guides/tools/img/mldev_model_ops_subtest_d.svg
@@ -0,0 +1,422 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="275mm"
+ height="135mm"
+ viewBox="0 0 275 135"
+ version="1.1"
+ id="svg5"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="mldev_model_ops_subtest_b.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg">
+ <sodipodi:namedview
+ id="namedview7"
+ pagecolor="#ffffff"
+ bordercolor="#000000"
+ borderopacity="0.25"
+ inkscape:showpageshadow="2"
+ inkscape:pageopacity="0.0"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1"
+ inkscape:document-units="mm"
+ showgrid="false"
+ inkscape:zoom="0.72515432"
+ inkscape:cx="324.06895"
+ inkscape:cy="295.1096"
+ inkscape:window-width="1920"
+ inkscape:window-height="986"
+ inkscape:window-x="-11"
+ inkscape:window-y="-11"
+ inkscape:window-maximized="1"
+ inkscape:current-layer="layer1" />
+ <defs
+ id="defs2">
+ <marker
+ style="overflow:visible"
+ id="RoundedArrow"
+ refX="6"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="RoundedArrow"
+ markerWidth="6.1347523"
+ markerHeight="5.9304948"
+ viewBox="0 0 6.1347524 5.9304951"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.7)"
+ d="m -0.21114562,-4.1055728 6.42229122,3.21114561 a 1,1 90 0 1 0,1.78885438 L -0.21114562,4.1055728 A 1.236068,1.236068 31.717474 0 1 -2,3 v -6 a 1.236068,1.236068 148.28253 0 1 1.78885438,-1.1055728 z"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:none"
+ id="path1367" />
+ </marker>
+ </defs>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1">
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.633453;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234"
+ width="47.366547"
+ height="17.366549"
+ x="14.864025"
+ y="14.24185"
+ ry="1.4985789"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-23.388845"
+ y="38.541786"
+ id="text290"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-23.388845"
+ y="38.541786">Model 0 / Load</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.633453;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-07"
+ width="47.366547"
+ height="17.366549"
+ x="81.026939"
+ y="14.24185"
+ ry="1.4985789"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-23.388845"
+ y="104.52142"
+ id="text290-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-11"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-23.388845"
+ y="104.52142">Model 0 / Start</tspan></text>
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.6329;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-6"
+ width="47.3671"
+ height="17.33604"
+ x="147.18958"
+ y="14.257105"
+ ry="1.4959463" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-23.388849"
+ y="170.86761"
+ id="text290-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-15"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-23.388849"
+ y="170.86761">Model 1 / Load</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 62.230572,22.925124 H 81.026939"
+ id="path1916"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234"
+ inkscape:connection-end="#rect234-07" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 128.39349,22.925124 h 18.79609"
+ id="path1918"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-07"
+ inkscape:connection-end="#rect234-6" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 194.55667,22.925125 18.79581,0"
+ id="path1922"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-6"
+ inkscape:connection-end="#rect234-0-6-6" />
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6"
+ width="47.3671"
+ height="17.33604"
+ x="213.35248"
+ y="14.257105"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-23.388849"
+ y="236.84723"
+ id="text290-3-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-23.388849"
+ y="236.84723">Model 1 / Start</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 237.03603,31.593145 0,5.328675 H 38.547297 v 5.313421"
+ id="path1924"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6"
+ inkscape:connection-end="#rect234-0-6-6-3" />
+ <rect
+ style="fill:#fff6d5;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-3"
+ width="47.3671"
+ height="17.33604"
+ x="14.863747"
+ y="42.235241"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-51.366989"
+ y="38.541786"
+ id="text290-3-2-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-51.366989"
+ y="38.541786">Model N / Load</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-2"
+ width="47.3671"
+ height="17.33604"
+ x="81.026657"
+ y="42.235241"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-51.366989"
+ y="104.52142"
+ id="text290-3-2-2-28"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-2"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-51.366989"
+ y="104.52142">Model N / Start</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 62.230846,50.903261 H 81.026657"
+ id="path1926"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-3"
+ inkscape:connection-end="#rect234-0-6-6-2" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 128.39376,50.903261 h 42.47937 v 19.294584"
+ id="path1928"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-2"
+ inkscape:connection-end="#rect234-0-6-1" />
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6"
+ width="47.3671"
+ height="17.33604"
+ x="213.35248"
+ y="70.197845"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-79.32959"
+ y="237.03052"
+ id="text290-3-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-79.32959"
+ y="237.03052">Model N / Unload</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-1"
+ width="47.3671"
+ height="17.33604"
+ x="147.18958"
+ y="70.197845"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-78.803177"
+ y="170.77666"
+ id="text290-3-2-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-8"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-78.803177"
+ y="170.77666">Model N / Stop</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 194.55667,78.865866 h 18.79581"
+ id="path1930"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6"
+ inkscape:connection-start="#rect234-0-6-1" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 237.03603,87.533886 v 5.313417 H 38.547297 v 5.328677"
+ id="path1932"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6-7"
+ inkscape:connection-start="#rect234-0-6" />
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-7"
+ width="47.3671"
+ height="17.33604"
+ x="14.863747"
+ y="98.17598"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-106.78132"
+ y="38.450832"
+ id="text290-3-2-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-106.78132"
+ y="38.450832">Model 1 / Stop</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 62.230846,106.844 H 81.026657"
+ id="path1934"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0"
+ inkscape:connection-start="#rect234-0-6-7" />
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0"
+ width="47.3671"
+ height="17.33604"
+ x="81.026657"
+ y="98.17598"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-107.30773"
+ y="104.7047"
+ id="text290-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-107.30773"
+ y="104.7047">Model 1 / Unload</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-9"
+ width="47.3671"
+ height="17.33604"
+ x="147.18958"
+ y="98.17598"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-106.78133"
+ y="170.77666"
+ id="text290-3-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-106.78133"
+ y="170.77666">Model 0 / Stop</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.633452;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-5"
+ width="47.366547"
+ height="17.366549"
+ x="213.35277"
+ y="98.160721"
+ ry="1.4985789"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-107.30773"
+ y="237.03053"
+ id="text290-3-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-3"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-107.30773"
+ y="237.03053">Model 0 / Unload</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 128.39376,106.844 h 18.79582"
+ id="path1936"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-9"
+ inkscape:connection-start="#rect234-0" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 194.55667,106.844 9.39805,0 v 0 h 9.39805"
+ id="path1938"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-5"
+ inkscape:connection-start="#rect234-0-9" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#fff6d5;fill-rule:evenodd;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-122.00021"
+ y="222.26495"
+ id="text392"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan390"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';fill:#000000;stroke-width:0.5"
+ x="-122.00021"
+ y="222.26495">mldev: model_ops / subest B</tspan></text>
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.462341;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect3475"
+ width="259.53769"
+ height="119.53766"
+ x="7.7311554"
+ y="7.7311707"
+ ry="1.2186421" />
+ </g>
+</svg>
diff --git a/doc/guides/tools/index.rst b/doc/guides/tools/index.rst
index f1f5b94c8c..6f84fc31ff 100644
--- a/doc/guides/tools/index.rst
+++ b/doc/guides/tools/index.rst
@@ -21,4 +21,5 @@ DPDK Tools User Guides
comp_perf
testeventdev
testregex
+ testmldev
dts
diff --git a/doc/guides/tools/testmldev.rst b/doc/guides/tools/testmldev.rst
new file mode 100644
index 0000000000..845c2d9381
--- /dev/null
+++ b/doc/guides/tools/testmldev.rst
@@ -0,0 +1,441 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright (c) 2022 Marvell.
+
+dpdk-test-mldev Application
+===========================
+
+The ``dpdk-test-mldev`` tool is a Data Plane Development Kit (DPDK) application that allows testing
+various mldev use cases. This application has a generic framework to add new mldev based test cases
+to verify functionality and measure the performance of inference execution on DPDK ML devices.
+
+
+Application and Options
+-----------------------
+
+The application has a number of command line options:
+
+.. code-block:: console
+
+ dpdk-test-mldev [EAL Options] -- [application options]
+
+EAL Options
+~~~~~~~~~~~
+
+The following are the EAL command-line options that can be used with the ``dpdk-test-mldev``
+application. See the DPDK Getting Started Guides for more information on these options.
+
+* ``-c <COREMASK>`` or ``-l <CORELIST>``
+
+ Set the hexadecimal bitmask of the cores to run on. The corelist is a list of cores to use.
+
+* ``-a <PCI_ID>``
+
+ Attach a PCI based ML device. Specific to drivers using a PCI based ML devices.
+
+* ``--vdev <driver>``
+
+ Add a virtual mldev device. Specific to drivers using a ML virtual device.
+
+
+Application Options
+~~~~~~~~~~~~~~~~~~~
+
+The following are the command-line options supported by the test application.
+
+* ``--test <name>``
+
+ ML tests are divided into two groups, Model and Device tests and Inference tests. Test
+ name one of the following supported tests.
+
+ **ML Device Tests** ::
+
+ device_ops
+
+ **ML Model Tests** ::
+
+ model_ops
+
+ **ML Inference Tests** ::
+
+ inference_ordered
+ inference_interleave
+
+* ``--dev_id <n>``
+
+ Set the device id of the ML device to be used for the test. Default value is `0`.
+
+* ``--socket_id <n>``
+
+ Set the socket id of the application resources. Default value is `SOCKET_ID_ANY`.
+
+* ``--debug``
+
+ Enable the tests to run in debug mode.
+
+* ``--models <model_list>``
+
+ Set the list of model files to be used for the tests. Application expects the
+ ``model_list`` in comma separated form (i.e. ``--models model_A.bin,model_B.bin``).
+ Maximum number of models supported by the test is ``8``.
+
+* ``--filelist <file_list>``
+
+ Set the list of model, input, output and reference files to be used for the tests.
+ Application expects the ``file_list`` to be in comma separated form
+ (i.e. ``--filelist <model,input,output>[,reference]``).
+
+ Multiple filelist entries can be specified when running the tests with multiple models.
+ Both quantized and dequantized outputs are written to the disk. Dequantized output file
+ would have the name specified by the user through ``--filelist`` option. A suffix ``.q``
+ is appended to quantized output filename. Maximum number of filelist entries supported
+ by the test is ``8``.
+
+* ``--repetitions <n>``
+
+ Set the number of inference repetitions to be executed in the test per each model. Default
+ value is `1`.
+
+* ``--burst_size <n>``
+
+ Set the burst size to be used when enqueuing / dequeuing inferences. Default value is `1`.
+
+* ``--queue_pairs <n>``
+
+ Set the number of queue-pairs to be used for inference enqueue and dequeue operations.
+ Default value is `1`.
+
+* ``--queue_size <n>``
+
+ Set the size of queue-pair to be created for inference enqueue / dequeue operations.
+ Queue size would translate into `rte_ml_dev_qp_conf::nb_desc` field during queue-pair
+ creation. Default value is `1`.
+
+* ``--batches <n>``
+
+ Set the number batches in the input file provided for inference run. When not specified
+ the test would assume the number of batches is equal to the batch size of the model.
+
+* ``--tolerance <n>``
+
+ Set the tolerance value in percentage to be used for output validation. Default value
+ is `0`.
+
+* ``--stats``
+
+ Enable reporting device extended stats.
+
+
+ML Device Tests
+-------------------------
+
+ML device tests are functional tests to validate ML device APIs. Device tests validate the ML device
+handling APIs configure, close, start and stop APIs.
+
+
+Application Options
+~~~~~~~~~~~~~~~~~~~
+
+Supported command line options for the `model_ops` test are following::
+
+ --debug
+ --test
+ --dev_id
+ --socket_id
+ --queue_pairs
+ --queue_size
+
+
+DEVICE_OPS Test
+~~~~~~~~~~~~~~~
+
+Device ops test validates the device configuration and reconfiguration support. The test configures
+ML device based on the option ``--queue_pairs`` and ``--queue_size`` specified by the user, and
+later reconfigures the ML device with the number of queue pairs and queue size based the maximum
+specified through the device info.
+
+
+Example
+^^^^^^^
+
+Command to run device_ops test:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=device_ops
+
+
+Command to run device_ops test with user options:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=device_ops --queue_pairs <M> --queue_size <N>
+
+
+ML Model Tests
+-------------------------
+
+Model tests are functional tests to validate ML model APIs. Model tests validate the functioning
+of APIs to load, start, stop and unload ML models.
+
+
+Application Options
+~~~~~~~~~~~~~~~~~~~
+
+Supported command line options for the `model_ops` test are following::
+
+ --debug
+ --test
+ --dev_id
+ --socket_id
+ --models
+
+
+List of model files to be used for the `model_ops` test can be specified through the option
+``--models <model_list>`` as a comma separated list. Maximum number of models supported in
+the test is `8`.
+
+.. Note::
+
+ * The ``--models <model_list>`` is a mandatory option for running this test.
+ * Options not supported by the test are ignored if specified.
+
+
+MODEL_OPS Test
+~~~~~~~~~~~~~~
+
+The test is a collection of multiple sub-tests, each with a different order of slow-path
+operations when handling with `N` number of models.
+
+
+**Sub-test A:** executes the sequence of load / start / stop / unload for a model in order,
+followed by next model.
+.. _figure_mldev_model_ops_subtest_a:
+
+.. figure:: img/mldev_model_ops_subtest_a.*
+
+ Execution sequence of model_ops subtest A.
+
+
+**Sub-test B:** executes load for all models, followed by a start for all models. Upon successful
+start of all models, stop is invoked for all models followed by unload.
+.. _figure_mldev_model_ops_subtest_b:
+
+.. figure:: img/mldev_model_ops_subtest_b.*
+
+ Execution sequence of model_ops subtest B.
+
+
+**Sub-test C:** loads all models, followed by a start and stop of all models in order. Upon
+completion of stop, unload is invoked for all models.
+.. _figure_mldev_model_ops_subtest_c:
+
+.. figure:: img/mldev_model_ops_subtest_c.*
+
+ Execution sequence of model_ops subtest C.
+
+
+**Sub-test D:** executes load and start for all models available. Upon successful start of all
+models, stop and stop is executed for the models.
+.. _figure_mldev_model_ops_subtest_d:
+
+.. figure:: img/mldev_model_ops_subtest_d.*
+
+ Execution sequence of model_ops subtest D.
+
+
+Example
+^^^^^^^
+
+Command to run model_ops test:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=model_ops --models model_1.bin,model_2.bin,model_3.bin, model_4.bin
+
+
+ML Inference Tests
+------------------
+
+Inference tests are a set of tests to validate end-to-end inference execution on ML device.
+These tests executes the full sequence of operations required to run inferences with one or
+multiple models.
+
+Application Options
+~~~~~~~~~~~~~~~~~~~
+
+Supported command line options for inference tests are following::
+
+ --debug
+ --test
+ --dev_id
+ --socket_id
+ --filelist
+ --repetitions
+ --burst_size
+ --queue_pairs
+ --queue_size
+ --batches
+ --tolerance
+ --stats
+
+
+List of files to be used for the inference tests can be specified through the option
+``--filelist <file_list>`` as a comma separated list. A filelist entry would be of the format
+``--filelist <model_file,input_file,output_file>[,reference_file]`` and is used to specify the
+list of files required to test with a single model. Multiple filelist entries are supported by
+the test, one entry per model. Maximum number of file entries supported by the test is `8`.
+
+When ``--burst_size <num>`` option is specified for the test, enqueue and dequeue burst would
+try to enqueue or dequeue ``num`` number of inferences per each call respectively.
+
+In the inference test, a pair of lcores are mapped to each queue pair. Minimum number of lcores
+required for the tests is equal to ``(queue_pairs * 2 + 1)``.
+
+Output validation of inference would be enabled only when a reference file is specified through
+the ``--filelist`` option. Application would additionally consider the tolerance value provided
+through ``--tolerance`` option during validation. When the tolerance values is 0, CRC32 hash of
+inference output and reference output are compared. When the tolerance is non-zero, element wise
+comparison of output is performed. Validation is considered as successful only when all the
+elements of the output tensor are with in the tolerance range specified.
+
+When ``--debug`` option is specified, tests are run in debug mode.
+
+Enabling ``--stats`` would print the extended stats supported by the driver.
+
+.. Note::
+
+ * The ``--filelist <file_list>`` is a mandatory option for running inference tests.
+ * Options not supported by the tests are ignored if specified.
+ * Element wise comparison is not supported when the output dtype is either fp8, fp16
+ or bfloat16. This is applicable only when the tolerance is greater than zero and for
+ pre-quantized models only.
+
+
+INFERENCE_ORDERED Test
+~~~~~~~~~~~~~~~~~~~~~~
+
+This is a functional test for validating the end-to-end inference execution on ML device. This
+test configures ML device and queue pairs as per the queue-pair related options (queue_pairs and
+queue_size) specified by the user. Upon successful configuration of the device and queue pairs,
+the first model specified through the filelist is loaded to the device and inferences are enqueued
+by a pool of worker threads to the ML device. Total number of inferences enqueued for the model
+are equal to the repetitions specified. A dedicated pool of worker threads would dequeue the
+inferences from the device. The model is unloaded upon completion of all inferences for the model.
+The test would continue loading and executing inference requests for all models specified
+through ``filelist`` option in an ordered manner.
+
+.. _figure_mldev_inference_ordered:
+
+.. figure:: img/mldev_inference_ordered.*
+
+ Execution of inference_ordered on single model.
+
+
+Example
+^^^^^^^
+
+Example command to run inference_ordered test:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_ordered --filelist model.bin,input.bin,output.bin
+
+Example command to run inference_ordered with output validation using tolerance of `1%``:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_ordered --filelist model.bin,input.bin,output.bin,reference.bin \
+ --tolerance 1.0
+
+Example command to run inference_ordered test with multiple queue-pairs and queue size:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_ordered --filelist model.bin,input.bin,output.bin \
+ --queue_pairs 4 --queue_size 16
+
+Example command to run inference_ordered test with a specific burst size:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_ordered --filelist model.bin,input.bin,output.bin \
+ --burst_size 12
+
+
+INFERENCE_INTERLEAVE Test
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This is a stress test for validating the end-to-end inference execution on ML device. The test
+configures the ML device and queue pairs as per the queue-pair related options (queue_pairs
+and queue_size) specified by the user. Upon successful configuration of the device and queue
+pairs, all models specified through the filelist are loaded to the device. Inferences for multiple
+models are enqueued by a pool of worker threads in parallel. Inference execution by the device is
+interleaved between multiple models. Total number of inferences enqueued for a model are equal to
+the repetitions specified. An additional pool of threads would dequeue the inferences from the
+device. Models would be unloaded upon completion of inferences for all models loaded.
+
+
+.. _figure_mldev_inference_interleave:
+
+.. figure:: img/mldev_inference_interleave.*
+
+ Execution of inference_interleave on single model.
+
+
+Example
+^^^^^^^
+
+Example command to run inference_interleave test:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_interleave --filelist model.bin,input.bin,output.bin
+
+
+Example command to run inference_interleave test with multiple models:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_interleave --filelist model_A.bin,input_A.bin,output_A.bin \
+ --filelist model_B.bin,input_B.bin,output_B.bin
+
+
+Example command to run inference_interleave test with multiple models ad output validation
+using tolerance of `2.0%``:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_interleave \
+ --filelist model_A.bin,input_A.bin,output_A.bin,reference_A.bin \
+ --filelist model_B.bin,input_B.bin,output_B.bin,reference_B.bin \
+ --tolerance 2.0
+
+Example command to run inference_interleave test with multiple queue-pairs and queue size
+and burst size:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_interleave --filelist model.bin,input.bin,output.bin \
+ --queue_pairs 8 --queue_size 12 --burst_size 16
+
+
+Debug mode
+----------
+
+ML tests can be executed in debug mode by enabling the option ``--debug``. Execution of tests in
+debug mode would enable additional prints.
+
+When a validation failure is observed, output from that buffer is written to the disk, with the
+filenames having similar convention when the test has passed. Additionally index of the buffer
+would be appended to the filenames.
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v4 00/12] Implementation of mldev test application
2022-11-29 7:07 [PATCH v1 00/12] implement mldev test application Srikanth Yalavarthi
` (11 preceding siblings ...)
2022-11-29 7:07 ` [PATCH v1 12/12] app/mldev: add documentation for mldev test cases Srikanth Yalavarthi
@ 2023-02-07 15:49 ` Srikanth Yalavarthi
2023-02-07 15:49 ` [PATCH v4 01/12] app/mldev: implement test framework for mldev Srikanth Yalavarthi
` (11 more replies)
2023-03-10 8:09 ` [PATCH v5 00/12] Implementation of mldev test application Srikanth Yalavarthi
` (2 subsequent siblings)
15 siblings, 12 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-02-07 15:49 UTC (permalink / raw)
Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla,
Srikanth Yalavarthi
Machine learning device APIs test application
=============================================
This series of patches introduces a test application for machine
learning device APIs. A test framework is implemented with multiple
test enabled, to validate the device, model and fast-path functions.
New tests can be added using the test framework.
List of tests supported
-----------------------
1) device_ops: Test case to validate device re-configuration
2) model_ops: Collection of 4 sub-tests to validate model slow APIs.
Each sub-test would invoke the slow path model APIs (load / start /
stop / unload) in different order.
3) inference_ordered: Test case to validate execution of end-to-end
inferences on ML device with one active model at a time. This test
can execute inference requests for multiple models, with inferences
for a model executed after completion of inferences for a previously
loaded model.
4) inference_interleave: Test case to validate end-to-end inferences
with multiple active models concurrently. This case would work as a
stress test to validate ML device.
Options supported the by tests include burst size for enqueuing and
dequeuing inference requests, enabling multiple queue pairs with a
user specified value for queue size. Support is also enabled for
batch inferencing, output validation and statistics.
v4:
* Updated model_id as uint16_t
* Updated license info in SVG files
* Updated release notes
v3:
* Code rebase
Srikanth Yalavarthi (12):
app/mldev: implement test framework for mldev
app/mldev: add common test functions
app/mldev: add test case to validate device ops
app/mldev: add test case to validate model ops
app/mldev: add ordered inference test case
app/mldev: add test case to interleave inferences
app/mldev: enable support for burst inferences
app/mldev: enable support for queue pairs and size
app/mldev: enable support for inference batches
app/mldev: enable support for inference validation
app/mldev: enable reporting stats in mldev app
app/mldev: add documentation for mldev test cases
MAINTAINERS | 2 +
app/meson.build | 1 +
app/test-mldev/meson.build | 24 +
app/test-mldev/ml_common.h | 29 +
app/test-mldev/ml_main.c | 118 ++
app/test-mldev/ml_options.c | 331 +++++
app/test-mldev/ml_options.h | 58 +
app/test-mldev/ml_test.c | 45 +
app/test-mldev/ml_test.h | 75 ++
app/test-mldev/parser.c | 380 ++++++
app/test-mldev/parser.h | 55 +
app/test-mldev/test_common.c | 139 +++
app/test-mldev/test_common.h | 27 +
app/test-mldev/test_device_ops.c | 234 ++++
app/test-mldev/test_device_ops.h | 17 +
app/test-mldev/test_inference_common.c | 1111 +++++++++++++++++
app/test-mldev/test_inference_common.h | 79 ++
app/test-mldev/test_inference_interleave.c | 122 ++
app/test-mldev/test_inference_ordered.c | 120 ++
app/test-mldev/test_model_common.c | 168 +++
app/test-mldev/test_model_common.h | 49 +
app/test-mldev/test_model_ops.c | 433 +++++++
app/test-mldev/test_model_ops.h | 21 +
doc/guides/rel_notes/release_23_03.rst | 8 +
.../tools/img/mldev_inference_interleave.svg | 669 ++++++++++
.../tools/img/mldev_inference_ordered.svg | 528 ++++++++
.../tools/img/mldev_model_ops_subtest_a.svg | 420 +++++++
.../tools/img/mldev_model_ops_subtest_b.svg | 423 +++++++
.../tools/img/mldev_model_ops_subtest_c.svg | 366 ++++++
.../tools/img/mldev_model_ops_subtest_d.svg | 424 +++++++
doc/guides/tools/index.rst | 1 +
doc/guides/tools/testmldev.rst | 441 +++++++
32 files changed, 6918 insertions(+)
create mode 100644 app/test-mldev/meson.build
create mode 100644 app/test-mldev/ml_common.h
create mode 100644 app/test-mldev/ml_main.c
create mode 100644 app/test-mldev/ml_options.c
create mode 100644 app/test-mldev/ml_options.h
create mode 100644 app/test-mldev/ml_test.c
create mode 100644 app/test-mldev/ml_test.h
create mode 100644 app/test-mldev/parser.c
create mode 100644 app/test-mldev/parser.h
create mode 100644 app/test-mldev/test_common.c
create mode 100644 app/test-mldev/test_common.h
create mode 100644 app/test-mldev/test_device_ops.c
create mode 100644 app/test-mldev/test_device_ops.h
create mode 100644 app/test-mldev/test_inference_common.c
create mode 100644 app/test-mldev/test_inference_common.h
create mode 100644 app/test-mldev/test_inference_interleave.c
create mode 100644 app/test-mldev/test_inference_ordered.c
create mode 100644 app/test-mldev/test_model_common.c
create mode 100644 app/test-mldev/test_model_common.h
create mode 100644 app/test-mldev/test_model_ops.c
create mode 100644 app/test-mldev/test_model_ops.h
create mode 100644 doc/guides/tools/img/mldev_inference_interleave.svg
create mode 100644 doc/guides/tools/img/mldev_inference_ordered.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_a.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_b.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_c.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_d.svg
create mode 100644 doc/guides/tools/testmldev.rst
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v4 01/12] app/mldev: implement test framework for mldev
2023-02-07 15:49 ` [PATCH v4 00/12] Implementation of mldev test application Srikanth Yalavarthi
@ 2023-02-07 15:49 ` Srikanth Yalavarthi
2023-02-14 4:55 ` Shivah Shankar Shankar Narayan Rao
2023-03-03 8:15 ` Anup Prabhu
2023-02-07 15:49 ` [PATCH v4 02/12] app/mldev: add common test functions Srikanth Yalavarthi
` (10 subsequent siblings)
11 siblings, 2 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-02-07 15:49 UTC (permalink / raw)
To: Thomas Monjalon, Srikanth Yalavarthi
Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Implemented framework for mldev test application. New test cases
can be added using the framework. Support is also enabled to add
options specific to the test cases. User can launch the tests by
specifying the name of test as part of launch arguments.
Code to parse command line arguments is imported from
test-eventdev, with support to parse additional data types.
Common arguments supported include:
test : name of the test application to run
dev_id : device id of the ML device
socket_id : socket_id of application resources
debug : enable debugging
help : print help
Sample launch command:
./dpdk-test-mldev -- --test <testname> --dev_id <dev_id> \
--socket_id <socket_id>
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
Depends-on: series-25753 ("mldev: introduce machine learning device library")
MAINTAINERS | 1 +
app/meson.build | 1 +
app/test-mldev/meson.build | 17 ++
app/test-mldev/ml_common.h | 29 ++
app/test-mldev/ml_main.c | 118 ++++++++
app/test-mldev/ml_options.c | 160 +++++++++++
app/test-mldev/ml_options.h | 31 ++
app/test-mldev/ml_test.c | 45 +++
app/test-mldev/ml_test.h | 75 +++++
app/test-mldev/parser.c | 380 +++++++++++++++++++++++++
app/test-mldev/parser.h | 55 ++++
doc/guides/rel_notes/release_23_03.rst | 8 +
12 files changed, 920 insertions(+)
create mode 100644 app/test-mldev/meson.build
create mode 100644 app/test-mldev/ml_common.h
create mode 100644 app/test-mldev/ml_main.c
create mode 100644 app/test-mldev/ml_options.c
create mode 100644 app/test-mldev/ml_options.h
create mode 100644 app/test-mldev/ml_test.c
create mode 100644 app/test-mldev/ml_test.h
create mode 100644 app/test-mldev/parser.c
create mode 100644 app/test-mldev/parser.h
diff --git a/MAINTAINERS b/MAINTAINERS
index fa91900a20..9aae343556 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -541,6 +541,7 @@ F: doc/guides/prog_guide/rawdev.rst
ML device API - EXPERIMENTAL
M: Srikanth Yalavarthi <syalavarthi@marvell.com>
F: lib/mldev/
+F: app/test-mldev/
F: doc/guides/prog_guide/mldev.rst
diff --git a/app/meson.build b/app/meson.build
index e32ea4bd5c..74d2420f67 100644
--- a/app/meson.build
+++ b/app/meson.build
@@ -23,6 +23,7 @@ apps = [
'test-fib',
'test-flow-perf',
'test-gpudev',
+ 'test-mldev',
'test-pipeline',
'test-pmd',
'test-regex',
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
new file mode 100644
index 0000000000..8ca2e1a1c1
--- /dev/null
+++ b/app/test-mldev/meson.build
@@ -0,0 +1,17 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright (c) 2022 Marvell.
+
+if is_windows
+ build = false
+ reason = 'not supported on Windows'
+ subdir_done()
+endif
+
+sources = files(
+ 'ml_main.c',
+ 'ml_options.c',
+ 'ml_test.c',
+ 'parser.c',
+)
+
+deps += ['mldev']
diff --git a/app/test-mldev/ml_common.h b/app/test-mldev/ml_common.h
new file mode 100644
index 0000000000..065180b619
--- /dev/null
+++ b/app/test-mldev/ml_common.h
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_COMMON_
+#define _ML_COMMON_
+
+#include <stdio.h>
+
+#define CLNRM "\x1b[0m"
+#define CLRED "\x1b[31m"
+#define CLGRN "\x1b[32m"
+#define CLYEL "\x1b[33m"
+
+#define ML_STR_FMT 20
+
+#define ml_err(fmt, args...) fprintf(stderr, CLRED "error: %s() " fmt CLNRM "\n", __func__, ##args)
+
+#define ml_info(fmt, args...) fprintf(stdout, CLYEL "" fmt CLNRM "\n", ##args)
+
+#define ml_dump(str, fmt, val...) printf("\t%-*s : " fmt "\n", ML_STR_FMT, str, ##val)
+
+#define ml_dump_begin(str) printf("\t%-*s :\n\t{\n", ML_STR_FMT, str)
+
+#define ml_dump_list(str, id, val) printf("\t%*s[%2u] : %s\n", ML_STR_FMT - 4, str, id, val)
+
+#define ml_dump_end printf("\b\t}\n\n")
+
+#endif /* _ML_COMMON_*/
diff --git a/app/test-mldev/ml_main.c b/app/test-mldev/ml_main.c
new file mode 100644
index 0000000000..d6652cd7b7
--- /dev/null
+++ b/app/test-mldev/ml_main.c
@@ -0,0 +1,118 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+
+#include <rte_debug.h>
+#include <rte_eal.h>
+#include <rte_mldev.h>
+
+#include "ml_common.h"
+#include "ml_options.h"
+#include "ml_test.h"
+
+struct ml_options opt;
+struct ml_test *test;
+
+int
+main(int argc, char **argv)
+{
+ uint16_t mldevs;
+ int ret;
+
+ ret = rte_eal_init(argc, argv);
+ if (ret < 0)
+ rte_panic("invalid EAL arguments\n");
+ argc -= ret;
+ argv += ret;
+
+ mldevs = rte_ml_dev_count();
+ if (!mldevs)
+ rte_panic("no mldev devices found\n");
+
+ /* set default values for options */
+ ml_options_default(&opt);
+
+ /* parse the command line arguments */
+ ret = ml_options_parse(&opt, argc, argv);
+ if (ret) {
+ ml_err("parsing one or more user options failed");
+ goto error;
+ }
+
+ /* get test struct from name */
+ test = ml_test_get(opt.test_name);
+ if (test == NULL) {
+ ml_err("failed to find requested test: %s", opt.test_name);
+ goto error;
+ }
+
+ if (test->ops.test_result == NULL) {
+ ml_err("%s: ops.test_result not found", opt.test_name);
+ goto error;
+ }
+
+ /* check test options */
+ if (test->ops.opt_check) {
+ if (test->ops.opt_check(&opt)) {
+ ml_err("invalid command line argument");
+ goto error;
+ }
+ }
+
+ /* check the device capability */
+ if (test->ops.cap_check) {
+ if (test->ops.cap_check(&opt) == false) {
+ ml_info("unsupported test: %s", opt.test_name);
+ ret = ML_TEST_UNSUPPORTED;
+ goto no_cap;
+ }
+ }
+
+ /* dump options */
+ if (opt.debug) {
+ if (test->ops.opt_dump)
+ test->ops.opt_dump(&opt);
+ }
+
+ /* test specific setup */
+ if (test->ops.test_setup) {
+ if (test->ops.test_setup(test, &opt)) {
+ ml_err("failed to setup test: %s", opt.test_name);
+ goto error;
+ }
+ }
+
+ /* test driver */
+ if (test->ops.test_driver)
+ test->ops.test_driver(test, &opt);
+
+ /* get result */
+ if (test->ops.test_result)
+ ret = test->ops.test_result(test, &opt);
+
+ if (test->ops.test_destroy)
+ test->ops.test_destroy(test, &opt);
+
+no_cap:
+ if (ret == ML_TEST_SUCCESS) {
+ printf("Result: " CLGRN "%s" CLNRM "\n", "Success");
+ } else if (ret == ML_TEST_FAILED) {
+ printf("Result: " CLRED "%s" CLNRM "\n", "Failed");
+ return EXIT_FAILURE;
+ } else if (ret == ML_TEST_UNSUPPORTED) {
+ printf("Result: " CLYEL "%s" CLNRM "\n", "Unsupported");
+ }
+
+ rte_eal_cleanup();
+
+ return 0;
+
+error:
+ rte_eal_cleanup();
+
+ return EXIT_FAILURE;
+}
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
new file mode 100644
index 0000000000..8fd7760e36
--- /dev/null
+++ b/app/test-mldev/ml_options.c
@@ -0,0 +1,160 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <errno.h>
+#include <getopt.h>
+#include <stdbool.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include <rte_memory.h>
+#include <rte_mldev.h>
+#include <rte_string_fns.h>
+
+#include "ml_common.h"
+#include "ml_options.h"
+#include "ml_test.h"
+#include "parser.h"
+
+typedef int (*option_parser_t)(struct ml_options *opt, const char *arg);
+
+void
+ml_options_default(struct ml_options *opt)
+{
+ memset(opt, 0, sizeof(*opt));
+ strlcpy(opt->test_name, "ml_test", ML_TEST_NAME_MAX_LEN);
+ opt->dev_id = 0;
+ opt->socket_id = SOCKET_ID_ANY;
+ opt->debug = false;
+}
+
+struct long_opt_parser {
+ const char *lgopt_name;
+ option_parser_t parser_fn;
+};
+
+static int
+ml_parse_test_name(struct ml_options *opt, const char *arg)
+{
+ strlcpy(opt->test_name, arg, ML_TEST_NAME_MAX_LEN);
+ return 0;
+}
+
+static int
+ml_parse_dev_id(struct ml_options *opt, const char *arg)
+{
+ int ret;
+
+ ret = parser_read_int16(&opt->dev_id, arg);
+
+ if (ret < 0)
+ return -EINVAL;
+
+ return ret;
+}
+
+static int
+ml_parse_socket_id(struct ml_options *opt, const char *arg)
+{
+ opt->socket_id = atoi(arg);
+
+ return 0;
+}
+
+static void
+ml_dump_test_options(const char *testname)
+{
+ RTE_SET_USED(testname);
+}
+
+static void
+print_usage(char *program)
+{
+ printf("\nusage : %s [EAL options] -- [application options]\n", program);
+ printf("application options:\n");
+ printf("\t--test : name of the test application to run\n"
+ "\t--dev_id : device id of the ML device\n"
+ "\t--socket_id : socket_id of application resources\n"
+ "\t--debug : enable debug mode\n"
+ "\t--help : print help\n");
+ printf("\n");
+ printf("available tests and test specific application options:\n");
+ ml_test_dump_names(ml_dump_test_options);
+}
+
+static struct option lgopts[] = {{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0},
+ {ML_SOCKET_ID, 1, 0, 0}, {ML_DEBUG, 0, 0, 0},
+ {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
+
+static int
+ml_opts_parse_long(int opt_idx, struct ml_options *opt)
+{
+ unsigned int i;
+
+ struct long_opt_parser parsermap[] = {
+ {ML_TEST, ml_parse_test_name},
+ {ML_DEVICE_ID, ml_parse_dev_id},
+ {ML_SOCKET_ID, ml_parse_socket_id},
+ };
+
+ for (i = 0; i < RTE_DIM(parsermap); i++) {
+ if (strncmp(lgopts[opt_idx].name, parsermap[i].lgopt_name,
+ strlen(lgopts[opt_idx].name)) == 0)
+ return parsermap[i].parser_fn(opt, optarg);
+ }
+
+ return -EINVAL;
+}
+
+int
+ml_options_parse(struct ml_options *opt, int argc, char **argv)
+{
+ int opt_idx;
+ int retval;
+ int opts;
+
+ while ((opts = getopt_long(argc, argv, "", lgopts, &opt_idx)) != EOF) {
+ switch (opts) {
+ case 0: /* parse long options */
+ if (!strcmp(lgopts[opt_idx].name, "debug")) {
+ opt->debug = true;
+ break;
+ }
+
+ if (!strcmp(lgopts[opt_idx].name, "help")) {
+ print_usage(argv[0]);
+ exit(EXIT_SUCCESS);
+ }
+
+ retval = ml_opts_parse_long(opt_idx, opt);
+ if (retval != 0)
+ return retval;
+ break;
+ default:
+ return -EINVAL;
+ }
+ }
+
+ return 0;
+}
+
+void
+ml_options_dump(struct ml_options *opt)
+{
+ struct rte_ml_dev_info dev_info;
+
+ rte_ml_dev_info_get(opt->dev_id, &dev_info);
+
+ ml_dump("driver", "%s", dev_info.driver_name);
+ ml_dump("test", "%s", opt->test_name);
+ ml_dump("dev_id", "%d", opt->dev_id);
+
+ if (opt->socket_id == SOCKET_ID_ANY)
+ ml_dump("socket_id", "%d (SOCKET_ID_ANY)", opt->socket_id);
+ else
+ ml_dump("socket_id", "%d", opt->socket_id);
+
+ ml_dump("debug", "%s", (opt->debug ? "true" : "false"));
+}
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
new file mode 100644
index 0000000000..05311a9a47
--- /dev/null
+++ b/app/test-mldev/ml_options.h
@@ -0,0 +1,31 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_OPTIONS_
+#define _ML_OPTIONS_
+
+#include <stdbool.h>
+#include <stdint.h>
+
+#define ML_TEST_NAME_MAX_LEN 32
+
+/* Options names */
+#define ML_TEST ("test")
+#define ML_DEVICE_ID ("dev_id")
+#define ML_SOCKET_ID ("socket_id")
+#define ML_DEBUG ("debug")
+#define ML_HELP ("help")
+
+struct ml_options {
+ char test_name[ML_TEST_NAME_MAX_LEN];
+ int16_t dev_id;
+ int socket_id;
+ bool debug;
+};
+
+void ml_options_default(struct ml_options *opt);
+int ml_options_parse(struct ml_options *opt, int argc, char **argv);
+void ml_options_dump(struct ml_options *opt);
+
+#endif /* _ML_OPTIONS_ */
diff --git a/app/test-mldev/ml_test.c b/app/test-mldev/ml_test.c
new file mode 100644
index 0000000000..2304712764
--- /dev/null
+++ b/app/test-mldev/ml_test.c
@@ -0,0 +1,45 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <stdio.h>
+#include <string.h>
+#include <sys/queue.h>
+
+#include "ml_test.h"
+
+static STAILQ_HEAD(, ml_test_entry) head = STAILQ_HEAD_INITIALIZER(head);
+
+void
+ml_test_register(struct ml_test_entry *entry)
+{
+ STAILQ_INSERT_TAIL(&head, entry, next);
+}
+
+struct ml_test *
+ml_test_get(const char *name)
+{
+ struct ml_test_entry *entry;
+
+ if (!name)
+ return NULL;
+
+ STAILQ_FOREACH(entry, &head, next)
+ if (!strncmp(entry->test.name, name, strlen(name)))
+ return &entry->test;
+
+ return NULL;
+}
+
+void
+ml_test_dump_names(void (*f)(const char *name))
+{
+ struct ml_test_entry *entry;
+
+ STAILQ_FOREACH(entry, &head, next)
+ {
+ if (entry->test.name)
+ printf("\t %s\n", entry->test.name);
+ f(entry->test.name);
+ }
+}
diff --git a/app/test-mldev/ml_test.h b/app/test-mldev/ml_test.h
new file mode 100644
index 0000000000..4a1430ec1b
--- /dev/null
+++ b/app/test-mldev/ml_test.h
@@ -0,0 +1,75 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_TEST_
+#define _ML_TEST_
+
+#include <stdbool.h>
+#include <string.h>
+#include <sys/queue.h>
+
+#include <rte_common.h>
+
+#include "ml_options.h"
+
+#define ML_TEST_MAX_POOL_SIZE 256
+
+enum ml_test_result {
+ ML_TEST_SUCCESS,
+ ML_TEST_FAILED,
+ ML_TEST_UNSUPPORTED,
+};
+
+struct ml_test;
+
+typedef bool (*ml_test_capability_check_t)(struct ml_options *opt);
+typedef int (*ml_test_options_check_t)(struct ml_options *opt);
+typedef void (*ml_test_options_dump_t)(struct ml_options *opt);
+typedef int (*ml_test_setup_t)(struct ml_test *test, struct ml_options *opt);
+typedef void (*ml_test_destroy_t)(struct ml_test *test, struct ml_options *opt);
+typedef int (*ml_test_driver_t)(struct ml_test *test, struct ml_options *opt);
+typedef int (*ml_test_result_t)(struct ml_test *test, struct ml_options *opt);
+
+struct ml_test_ops {
+ ml_test_capability_check_t cap_check;
+ ml_test_options_check_t opt_check;
+ ml_test_options_dump_t opt_dump;
+ ml_test_setup_t test_setup;
+ ml_test_destroy_t test_destroy;
+ ml_test_driver_t test_driver;
+ ml_test_result_t test_result;
+};
+
+struct ml_test {
+ const char *name;
+ void *test_priv;
+ struct ml_test_ops ops;
+};
+
+struct ml_test_entry {
+ struct ml_test test;
+
+ STAILQ_ENTRY(ml_test_entry) next;
+};
+
+static inline void *
+ml_test_priv(struct ml_test *test)
+{
+ return test->test_priv;
+}
+
+struct ml_test *ml_test_get(const char *name);
+void ml_test_register(struct ml_test_entry *test);
+void ml_test_dump_names(void (*f)(const char *));
+
+#define ML_TEST_REGISTER(nm) \
+ static struct ml_test_entry _ml_test_entry_##nm; \
+ RTE_INIT(ml_test_##nm) \
+ { \
+ _ml_test_entry_##nm.test.name = RTE_STR(nm); \
+ memcpy(&_ml_test_entry_##nm.test.ops, &nm, sizeof(struct ml_test_ops)); \
+ ml_test_register(&_ml_test_entry_##nm); \
+ }
+
+#endif /* _ML_TEST_ */
diff --git a/app/test-mldev/parser.c b/app/test-mldev/parser.c
new file mode 100644
index 0000000000..0b7fb63fe5
--- /dev/null
+++ b/app/test-mldev/parser.c
@@ -0,0 +1,380 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2016 Intel Corporation.
+ * Copyright (c) 2017 Cavium, Inc.
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <ctype.h>
+#include <errno.h>
+#include <stdbool.h>
+#include <stdint.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include <rte_common.h>
+
+#include "parser.h"
+
+static uint32_t
+get_hex_val(char c)
+{
+ switch (c) {
+ case '0':
+ case '1':
+ case '2':
+ case '3':
+ case '4':
+ case '5':
+ case '6':
+ case '7':
+ case '8':
+ case '9':
+ return c - '0';
+ case 'A':
+ case 'B':
+ case 'C':
+ case 'D':
+ case 'E':
+ case 'F':
+ return c - 'A' + 10;
+ case 'a':
+ case 'b':
+ case 'c':
+ case 'd':
+ case 'e':
+ case 'f':
+ return c - 'a' + 10;
+ default:
+ return 0;
+ }
+}
+
+int
+parser_read_arg_bool(const char *p)
+{
+ p = skip_white_spaces(p);
+ int result = -EINVAL;
+
+ if (((p[0] == 'y') && (p[1] == 'e') && (p[2] == 's')) ||
+ ((p[0] == 'Y') && (p[1] == 'E') && (p[2] == 'S'))) {
+ p += 3;
+ result = 1;
+ }
+
+ if (((p[0] == 'o') && (p[1] == 'n')) || ((p[0] == 'O') && (p[1] == 'N'))) {
+ p += 2;
+ result = 1;
+ }
+
+ if (((p[0] == 'n') && (p[1] == 'o')) || ((p[0] == 'N') && (p[1] == 'O'))) {
+ p += 2;
+ result = 0;
+ }
+
+ if (((p[0] == 'o') && (p[1] == 'f') && (p[2] == 'f')) ||
+ ((p[0] == 'O') && (p[1] == 'F') && (p[2] == 'F'))) {
+ p += 3;
+ result = 0;
+ }
+
+ p = skip_white_spaces(p);
+
+ if (p[0] != '\0')
+ return -EINVAL;
+
+ return result;
+}
+
+int
+parser_read_uint64(uint64_t *value, const char *p)
+{
+ char *next;
+ uint64_t val;
+
+ p = skip_white_spaces(p);
+ if (!isdigit(*p))
+ return -EINVAL;
+
+ val = strtoul(p, &next, 10);
+ if (p == next)
+ return -EINVAL;
+
+ p = next;
+ switch (*p) {
+ case 'T':
+ val *= 1024ULL;
+ /* fall through */
+ case 'G':
+ val *= 1024ULL;
+ /* fall through */
+ case 'M':
+ val *= 1024ULL;
+ /* fall through */
+ case 'k':
+ case 'K':
+ val *= 1024ULL;
+ p++;
+ break;
+ }
+
+ p = skip_white_spaces(p);
+ if (*p != '\0')
+ return -EINVAL;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_int32(int32_t *value, const char *p)
+{
+ char *next;
+ int32_t val;
+
+ p = skip_white_spaces(p);
+ if (!isdigit(*p))
+ return -EINVAL;
+
+ val = strtol(p, &next, 10);
+ if (p == next)
+ return -EINVAL;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_int16(int16_t *value, const char *p)
+{
+ char *next;
+ int16_t val;
+
+ p = skip_white_spaces(p);
+ if (!isdigit(*p))
+ return -EINVAL;
+
+ val = strtol(p, &next, 10);
+ if (p == next)
+ return -EINVAL;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint64_hex(uint64_t *value, const char *p)
+{
+ char *next;
+ uint64_t val;
+
+ p = skip_white_spaces(p);
+
+ val = strtoul(p, &next, 16);
+ if (p == next)
+ return -EINVAL;
+
+ p = skip_white_spaces(next);
+ if (*p != '\0')
+ return -EINVAL;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint32(uint32_t *value, const char *p)
+{
+ uint64_t val = 0;
+ int ret = parser_read_uint64(&val, p);
+
+ if (ret < 0)
+ return ret;
+
+ if (val > UINT32_MAX)
+ return -ERANGE;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint32_hex(uint32_t *value, const char *p)
+{
+ uint64_t val = 0;
+ int ret = parser_read_uint64_hex(&val, p);
+
+ if (ret < 0)
+ return ret;
+
+ if (val > UINT32_MAX)
+ return -ERANGE;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint16(uint16_t *value, const char *p)
+{
+ uint64_t val = 0;
+ int ret = parser_read_uint64(&val, p);
+
+ if (ret < 0)
+ return ret;
+
+ if (val > UINT16_MAX)
+ return -ERANGE;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint16_hex(uint16_t *value, const char *p)
+{
+ uint64_t val = 0;
+ int ret = parser_read_uint64_hex(&val, p);
+
+ if (ret < 0)
+ return ret;
+
+ if (val > UINT16_MAX)
+ return -ERANGE;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint8(uint8_t *value, const char *p)
+{
+ uint64_t val = 0;
+ int ret = parser_read_uint64(&val, p);
+
+ if (ret < 0)
+ return ret;
+
+ if (val > UINT8_MAX)
+ return -ERANGE;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint8_hex(uint8_t *value, const char *p)
+{
+ uint64_t val = 0;
+ int ret = parser_read_uint64_hex(&val, p);
+
+ if (ret < 0)
+ return ret;
+
+ if (val > UINT8_MAX)
+ return -ERANGE;
+
+ *value = val;
+ return 0;
+}
+
+int
+parse_tokenize_string(char *string, char *tokens[], uint32_t *n_tokens)
+{
+ uint32_t i;
+
+ if ((string == NULL) || (tokens == NULL) || (*n_tokens < 1))
+ return -EINVAL;
+
+ for (i = 0; i < *n_tokens; i++) {
+ tokens[i] = strtok_r(string, PARSE_DELIMITER, &string);
+ if (tokens[i] == NULL)
+ break;
+ }
+
+ if ((i == *n_tokens) && (strtok_r(string, PARSE_DELIMITER, &string) != NULL))
+ return -E2BIG;
+
+ *n_tokens = i;
+ return 0;
+}
+
+int
+parse_hex_string(char *src, uint8_t *dst, uint32_t *size)
+{
+ char *c;
+ uint32_t len, i;
+
+ /* Check input parameters */
+ if ((src == NULL) || (dst == NULL) || (size == NULL) || (*size == 0))
+ return -1;
+
+ len = strlen(src);
+ if (((len & 3) != 0) || (len > (*size) * 2))
+ return -1;
+ *size = len / 2;
+
+ for (c = src; *c != 0; c++) {
+ if ((((*c) >= '0') && ((*c) <= '9')) || (((*c) >= 'A') && ((*c) <= 'F')) ||
+ (((*c) >= 'a') && ((*c) <= 'f')))
+ continue;
+
+ return -1;
+ }
+
+ /* Convert chars to bytes */
+ for (i = 0; i < *size; i++)
+ dst[i] = get_hex_val(src[2 * i]) * 16 + get_hex_val(src[2 * i + 1]);
+
+ return 0;
+}
+
+int
+parse_lcores_list(bool lcores[], int lcores_num, const char *corelist)
+{
+ int i, idx = 0;
+ int min, max;
+ char *end = NULL;
+
+ if (corelist == NULL)
+ return -1;
+ while (isblank(*corelist))
+ corelist++;
+ i = strlen(corelist);
+ while ((i > 0) && isblank(corelist[i - 1]))
+ i--;
+
+ /* Get list of lcores */
+ min = RTE_MAX_LCORE;
+ do {
+ while (isblank(*corelist))
+ corelist++;
+ if (*corelist == '\0')
+ return -1;
+ idx = strtoul(corelist, &end, 10);
+ if (idx < 0 || idx > lcores_num)
+ return -1;
+
+ if (end == NULL)
+ return -1;
+ while (isblank(*end))
+ end++;
+ if (*end == '-') {
+ min = idx;
+ } else if ((*end == ',') || (*end == '\0')) {
+ max = idx;
+ if (min == RTE_MAX_LCORE)
+ min = idx;
+ for (idx = min; idx <= max; idx++) {
+ if (lcores[idx] == 1)
+ return -E2BIG;
+ lcores[idx] = 1;
+ }
+
+ min = RTE_MAX_LCORE;
+ } else
+ return -1;
+ corelist = end + 1;
+ } while (*end != '\0');
+
+ return 0;
+}
diff --git a/app/test-mldev/parser.h b/app/test-mldev/parser.h
new file mode 100644
index 0000000000..f0d5e79e4b
--- /dev/null
+++ b/app/test-mldev/parser.h
@@ -0,0 +1,55 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2010-2016 Intel Corporation.
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef __INCLUDE_PARSER_H__
+#define __INCLUDE_PARSER_H__
+
+#include <ctype.h>
+#include <stdint.h>
+#include <stdlib.h>
+
+#define PARSE_DELIMITER " \f\n\r\t\v"
+
+#define skip_white_spaces(pos) \
+ ({ \
+ __typeof__(pos) _p = (pos); \
+ for (; isspace(*_p); _p++) \
+ ; \
+ _p; \
+ })
+
+static inline size_t
+skip_digits(const char *src)
+{
+ size_t i;
+
+ for (i = 0; isdigit(src[i]); i++)
+ ;
+
+ return i;
+}
+
+int parser_read_arg_bool(const char *p);
+
+int parser_read_uint64(uint64_t *value, const char *p);
+int parser_read_uint32(uint32_t *value, const char *p);
+int parser_read_uint16(uint16_t *value, const char *p);
+int parser_read_uint8(uint8_t *value, const char *p);
+
+int parser_read_uint64_hex(uint64_t *value, const char *p);
+int parser_read_uint32_hex(uint32_t *value, const char *p);
+int parser_read_uint16_hex(uint16_t *value, const char *p);
+int parser_read_uint8_hex(uint8_t *value, const char *p);
+
+int parser_read_int32(int32_t *value, const char *p);
+int parser_read_int16(int16_t *value, const char *p);
+
+int parse_hex_string(char *src, uint8_t *dst, uint32_t *size);
+
+int parse_tokenize_string(char *string, char *tokens[], uint32_t *n_tokens);
+
+int parse_lcores_list(bool lcores[], int lcores_num, const char *corelist);
+
+#endif /* __INCLUDE_PARSER_H__ */
diff --git a/doc/guides/rel_notes/release_23_03.rst b/doc/guides/rel_notes/release_23_03.rst
index 6086055a52..cd1ac98abe 100644
--- a/doc/guides/rel_notes/release_23_03.rst
+++ b/doc/guides/rel_notes/release_23_03.rst
@@ -87,6 +87,14 @@ New Features
* Added a machine learning inference device framework for management and provision of
hardware and software machine learning inference devices.
+* **Added test application for machine learning inference device library.**
+
+ * Added test application for mldev library with support for multiple test cases.
+ * Test case for device operations.
+ * Test case for model operations.
+ * Test case for inferences from multiple models in ordered mode.
+ * Test case for inferences from multiple models.in interleaving mode.
+
Removed Items
-------------
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* RE: [PATCH v4 01/12] app/mldev: implement test framework for mldev
2023-02-07 15:49 ` [PATCH v4 01/12] app/mldev: implement test framework for mldev Srikanth Yalavarthi
@ 2023-02-14 4:55 ` Shivah Shankar Shankar Narayan Rao
2023-03-03 8:15 ` Anup Prabhu
1 sibling, 0 replies; 122+ messages in thread
From: Shivah Shankar Shankar Narayan Rao @ 2023-02-14 4:55 UTC (permalink / raw)
To: Srikanth Yalavarthi, Thomas Monjalon, Srikanth Yalavarthi
Cc: dev, Jerin Jacob Kollanukkaran, Anup Prabhu, Prince Takkar,
Parijat Shukla
[-- Attachment #1: Type: text/plain, Size: 1440 bytes --]
> -----Original Message-----
> From: Srikanth Yalavarthi <syalavarthi@marvell.com>
> Sent: Tuesday, February 7, 2023 9:20 PM
> To: Thomas Monjalon <thomas@monjalon.net>; Srikanth Yalavarthi
> <syalavarthi@marvell.com>
> Cc: dev@dpdk.org; Shivah Shankar Shankar Narayan Rao
> <sshankarnara@marvell.com>; Jerin Jacob Kollanukkaran
> <jerinj@marvell.com>; Anup Prabhu <aprabhu@marvell.com>; Prince Takkar
> <ptakkar@marvell.com>; Parijat Shukla <pshukla@marvell.com>
> Subject: [PATCH v4 01/12] app/mldev: implement test framework for mldev
>
> Implemented framework for mldev test application. New test cases can be
> added using the framework. Support is also enabled to add options specific
> to the test cases. User can launch the tests by specifying the name of test as
> part of launch arguments.
>
> Code to parse command line arguments is imported from test-eventdev,
> with support to parse additional data types.
>
> Common arguments supported include:
>
> test : name of the test application to run
> dev_id : device id of the ML device
> socket_id : socket_id of application resources
> debug : enable debugging
> help : print help
>
> Sample launch command:
> ./dpdk-test-mldev -- --test <testname> --dev_id <dev_id> \ --socket_id
> <socket_id>
>
> Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
Acked-by: Shivah Shankar S <sshankarnara@marvell.com>
[-- Attachment #2: winmail.dat --]
[-- Type: application/ms-tnef, Size: 23843 bytes --]
^ permalink raw reply [flat|nested] 122+ messages in thread
* RE: [PATCH v4 01/12] app/mldev: implement test framework for mldev
2023-02-07 15:49 ` [PATCH v4 01/12] app/mldev: implement test framework for mldev Srikanth Yalavarthi
2023-02-14 4:55 ` Shivah Shankar Shankar Narayan Rao
@ 2023-03-03 8:15 ` Anup Prabhu
1 sibling, 0 replies; 122+ messages in thread
From: Anup Prabhu @ 2023-03-03 8:15 UTC (permalink / raw)
To: Srikanth Yalavarthi, Thomas Monjalon, Srikanth Yalavarthi
Cc: dev, Shivah Shankar Shankar Narayan Rao,
Jerin Jacob Kollanukkaran, Prince Takkar, Parijat Shukla
[-- Attachment #1: Type: text/plain, Size: 2239 bytes --]
> -----Original Message-----
> From: Srikanth Yalavarthi <syalavarthi@marvell.com>
> Sent: Tuesday, February 7, 2023 9:20 PM
> To: Thomas Monjalon <thomas@monjalon.net>; Srikanth Yalavarthi
> <syalavarthi@marvell.com>
> Cc: dev@dpdk.org; Shivah Shankar Shankar Narayan Rao
> <sshankarnara@marvell.com>; Jerin Jacob Kollanukkaran
> <jerinj@marvell.com>; Anup Prabhu <aprabhu@marvell.com>; Prince Takkar
> <ptakkar@marvell.com>; Parijat Shukla <pshukla@marvell.com>
> Subject: [PATCH v4 01/12] app/mldev: implement test framework for mldev
>
> Implemented framework for mldev test application. New test cases can be
> added using the framework. Support is also enabled to add options specific
> to the test cases. User can launch the tests by specifying the name of test as
> part of launch arguments.
>
> Code to parse command line arguments is imported from test-eventdev,
> with support to parse additional data types.
>
> Common arguments supported include:
>
> test : name of the test application to run
> dev_id : device id of the ML device
> socket_id : socket_id of application resources
> debug : enable debugging
> help : print help
>
> Sample launch command:
> ./dpdk-test-mldev -- --test <testname> --dev_id <dev_id> \ --socket_id
> <socket_id>
>
> Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
> ---
> Depends-on: series-25753 ("mldev: introduce machine learning device
> library")
>
> MAINTAINERS | 1 +
> app/meson.build | 1 +
> app/test-mldev/meson.build | 17 ++
> app/test-mldev/ml_common.h | 29 ++
> app/test-mldev/ml_main.c | 118 ++++++++
> app/test-mldev/ml_options.c | 160 +++++++++++
> app/test-mldev/ml_options.h | 31 ++
> app/test-mldev/ml_test.c | 45 +++
> app/test-mldev/ml_test.h | 75 +++++
> app/test-mldev/parser.c | 380 +++++++++++++++++++++++++
> app/test-mldev/parser.h | 55 ++++
> doc/guides/rel_notes/release_23_03.rst | 8 +
> 12 files changed, 920 insertions(+)
>
Acked-by: Anup Prabhu <aprabhu@marvell.com>
[-- Attachment #2: winmail.dat --]
[-- Type: application/ms-tnef, Size: 23531 bytes --]
^ permalink raw reply [flat|nested] 122+ messages in thread
* [PATCH v4 02/12] app/mldev: add common test functions
2023-02-07 15:49 ` [PATCH v4 00/12] Implementation of mldev test application Srikanth Yalavarthi
2023-02-07 15:49 ` [PATCH v4 01/12] app/mldev: implement test framework for mldev Srikanth Yalavarthi
@ 2023-02-07 15:49 ` Srikanth Yalavarthi
2023-02-23 9:03 ` Anup Prabhu
2023-02-07 15:49 ` [PATCH v4 03/12] app/mldev: add test case to validate device ops Srikanth Yalavarthi
` (9 subsequent siblings)
11 siblings, 1 reply; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-02-07 15:49 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Added common functions used by all tests. Common code
includes functions to check capabilities, options, and
handle ML devices.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/meson.build | 1 +
app/test-mldev/test_common.c | 139 +++++++++++++++++++++++++++++++++++
app/test-mldev/test_common.h | 27 +++++++
3 files changed, 167 insertions(+)
create mode 100644 app/test-mldev/test_common.c
create mode 100644 app/test-mldev/test_common.h
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
index 8ca2e1a1c1..964bb9ddc4 100644
--- a/app/test-mldev/meson.build
+++ b/app/test-mldev/meson.build
@@ -12,6 +12,7 @@ sources = files(
'ml_options.c',
'ml_test.c',
'parser.c',
+ 'test_common.c',
)
deps += ['mldev']
diff --git a/app/test-mldev/test_common.c b/app/test-mldev/test_common.c
new file mode 100644
index 0000000000..4a9a4ec4a2
--- /dev/null
+++ b/app/test-mldev/test_common.c
@@ -0,0 +1,139 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <errno.h>
+#include <stdbool.h>
+#include <stdint.h>
+
+#include <rte_common.h>
+#include <rte_memory.h>
+#include <rte_mldev.h>
+
+#include "ml_common.h"
+#include "ml_options.h"
+#include "test_common.h"
+
+bool
+ml_test_cap_check(struct ml_options *opt)
+{
+ struct rte_ml_dev_info dev_info;
+
+ rte_ml_dev_info_get(opt->dev_id, &dev_info);
+ if (dev_info.max_models == 0) {
+ ml_err("Not enough mldev models supported = %u", dev_info.max_models);
+ return false;
+ }
+
+ return true;
+}
+
+int
+ml_test_opt_check(struct ml_options *opt)
+{
+ uint16_t dev_count;
+ int socket_id;
+
+ RTE_SET_USED(opt);
+
+ dev_count = rte_ml_dev_count();
+ if (dev_count == 0) {
+ ml_err("No ML devices found");
+ return -ENODEV;
+ }
+
+ if (opt->dev_id >= dev_count) {
+ ml_err("Invalid option dev_id = %d", opt->dev_id);
+ return -EINVAL;
+ }
+
+ socket_id = rte_ml_dev_socket_id(opt->dev_id);
+ if (!((opt->socket_id != SOCKET_ID_ANY) || (opt->socket_id != socket_id))) {
+ ml_err("Invalid option, socket_id = %d\n", opt->socket_id);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+void
+ml_test_opt_dump(struct ml_options *opt)
+{
+ ml_options_dump(opt);
+}
+
+int
+ml_test_device_configure(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_common *t = ml_test_priv(test);
+ struct rte_ml_dev_config dev_config;
+ int ret;
+
+ ret = rte_ml_dev_info_get(opt->dev_id, &t->dev_info);
+ if (ret != 0) {
+ ml_err("Failed to get mldev info, dev_id = %d\n", opt->dev_id);
+ return ret;
+ }
+
+ /* configure device */
+ dev_config.socket_id = opt->socket_id;
+ dev_config.nb_models = t->dev_info.max_models;
+ dev_config.nb_queue_pairs = t->dev_info.max_queue_pairs;
+ ret = rte_ml_dev_configure(opt->dev_id, &dev_config);
+ if (ret != 0) {
+ ml_err("Failed to configure ml device, dev_id = %d\n", opt->dev_id);
+ return ret;
+ }
+
+ return 0;
+}
+
+int
+ml_test_device_close(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_common *t = ml_test_priv(test);
+ int ret = 0;
+
+ RTE_SET_USED(t);
+
+ /* close device */
+ ret = rte_ml_dev_close(opt->dev_id);
+ if (ret != 0)
+ ml_err("Failed to close ML device, dev_id = %d\n", opt->dev_id);
+
+ return ret;
+}
+
+int
+ml_test_device_start(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_common *t = ml_test_priv(test);
+ int ret;
+
+ RTE_SET_USED(t);
+
+ /* start device */
+ ret = rte_ml_dev_start(opt->dev_id);
+ if (ret != 0) {
+ ml_err("Failed to start ml device, dev_id = %d\n", opt->dev_id);
+ return ret;
+ }
+
+ return 0;
+}
+
+int
+ml_test_device_stop(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_common *t = ml_test_priv(test);
+ int ret = 0;
+
+ RTE_SET_USED(t);
+
+ /* stop device */
+ ret = rte_ml_dev_stop(opt->dev_id);
+ if (ret != 0)
+ ml_err("Failed to stop ML device, dev_id = %d\n", opt->dev_id);
+
+ return ret;
+}
diff --git a/app/test-mldev/test_common.h b/app/test-mldev/test_common.h
new file mode 100644
index 0000000000..05a2e43e2f
--- /dev/null
+++ b/app/test-mldev/test_common.h
@@ -0,0 +1,27 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_TEST_COMMON_
+#define _ML_TEST_COMMON_
+
+#include <rte_mldev.h>
+
+#include "ml_options.h"
+#include "ml_test.h"
+
+struct test_common {
+ struct ml_options *opt;
+ enum ml_test_result result;
+ struct rte_ml_dev_info dev_info;
+};
+
+bool ml_test_cap_check(struct ml_options *opt);
+int ml_test_opt_check(struct ml_options *opt);
+void ml_test_opt_dump(struct ml_options *opt);
+int ml_test_device_configure(struct ml_test *test, struct ml_options *opt);
+int ml_test_device_close(struct ml_test *test, struct ml_options *opt);
+int ml_test_device_start(struct ml_test *test, struct ml_options *opt);
+int ml_test_device_stop(struct ml_test *test, struct ml_options *opt);
+
+#endif /* _ML_TEST_COMMON_ */
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* RE: [PATCH v4 02/12] app/mldev: add common test functions
2023-02-07 15:49 ` [PATCH v4 02/12] app/mldev: add common test functions Srikanth Yalavarthi
@ 2023-02-23 9:03 ` Anup Prabhu
0 siblings, 0 replies; 122+ messages in thread
From: Anup Prabhu @ 2023-02-23 9:03 UTC (permalink / raw)
To: Srikanth Yalavarthi, Srikanth Yalavarthi
Cc: dev, Shivah Shankar Shankar Narayan Rao,
Jerin Jacob Kollanukkaran, Prince Takkar, Parijat Shukla
[-- Attachment #1: Type: text/plain, Size: 969 bytes --]
> -----Original Message-----
> From: Srikanth Yalavarthi <syalavarthi@marvell.com>
> Sent: Tuesday, February 7, 2023 9:20 PM
> To: Srikanth Yalavarthi <syalavarthi@marvell.com>
> Cc: dev@dpdk.org; Shivah Shankar Shankar Narayan Rao
> <sshankarnara@marvell.com>; Jerin Jacob Kollanukkaran
> <jerinj@marvell.com>; Anup Prabhu <aprabhu@marvell.com>; Prince Takkar
> <ptakkar@marvell.com>; Parijat Shukla <pshukla@marvell.com>
> Subject: [PATCH v4 02/12] app/mldev: add common test functions
>
> Added common functions used by all tests. Common code includes functions
> to check capabilities, options, and handle ML devices.
>
> Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
> ---
> app/test-mldev/meson.build | 1 +
> app/test-mldev/test_common.c | 139
> +++++++++++++++++++++++++++++++++++
> app/test-mldev/test_common.h | 27 +++++++
> 3 files changed, 167 insertions(+)
>
Acked-by: Anup Prabhu <aprabhu@marvell.com>
[-- Attachment #2: winmail.dat --]
[-- Type: application/ms-tnef, Size: 22910 bytes --]
^ permalink raw reply [flat|nested] 122+ messages in thread
* [PATCH v4 03/12] app/mldev: add test case to validate device ops
2023-02-07 15:49 ` [PATCH v4 00/12] Implementation of mldev test application Srikanth Yalavarthi
2023-02-07 15:49 ` [PATCH v4 01/12] app/mldev: implement test framework for mldev Srikanth Yalavarthi
2023-02-07 15:49 ` [PATCH v4 02/12] app/mldev: add common test functions Srikanth Yalavarthi
@ 2023-02-07 15:49 ` Srikanth Yalavarthi
2023-03-01 5:35 ` Anup Prabhu
2023-02-07 15:49 ` [PATCH v4 04/12] app/mldev: add test case to validate model ops Srikanth Yalavarthi
` (8 subsequent siblings)
11 siblings, 1 reply; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-02-07 15:49 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Added test case to validate device handling operations. Device ops
test is a collection of multiple sub-tests. Enabled sub-test to
validate device reconfiguration. Set device_ops as the default test.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/meson.build | 1 +
app/test-mldev/ml_options.c | 5 +-
app/test-mldev/test_device_ops.c | 234 +++++++++++++++++++++++++++++++
app/test-mldev/test_device_ops.h | 17 +++
4 files changed, 255 insertions(+), 2 deletions(-)
create mode 100644 app/test-mldev/test_device_ops.c
create mode 100644 app/test-mldev/test_device_ops.h
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
index 964bb9ddc4..60ea23d142 100644
--- a/app/test-mldev/meson.build
+++ b/app/test-mldev/meson.build
@@ -13,6 +13,7 @@ sources = files(
'ml_test.c',
'parser.c',
'test_common.c',
+ 'test_device_ops.c',
)
deps += ['mldev']
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 8fd7760e36..2e5f11bca2 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -24,7 +24,7 @@ void
ml_options_default(struct ml_options *opt)
{
memset(opt, 0, sizeof(*opt));
- strlcpy(opt->test_name, "ml_test", ML_TEST_NAME_MAX_LEN);
+ strlcpy(opt->test_name, "device_ops", ML_TEST_NAME_MAX_LEN);
opt->dev_id = 0;
opt->socket_id = SOCKET_ID_ANY;
opt->debug = false;
@@ -66,7 +66,8 @@ ml_parse_socket_id(struct ml_options *opt, const char *arg)
static void
ml_dump_test_options(const char *testname)
{
- RTE_SET_USED(testname);
+ if (strcmp(testname, "device_ops") == 0)
+ printf("\n");
}
static void
diff --git a/app/test-mldev/test_device_ops.c b/app/test-mldev/test_device_ops.c
new file mode 100644
index 0000000000..4cafcf41a6
--- /dev/null
+++ b/app/test-mldev/test_device_ops.c
@@ -0,0 +1,234 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <errno.h>
+#include <stdbool.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+
+#include <rte_common.h>
+#include <rte_malloc.h>
+#include <rte_mldev.h>
+
+#include "ml_common.h"
+#include "ml_options.h"
+#include "test_device_ops.h"
+
+static bool
+test_device_cap_check(struct ml_options *opt)
+{
+ if (!ml_test_cap_check(opt))
+ return false;
+
+ return true;
+}
+
+static int
+test_device_opt_check(struct ml_options *opt)
+{
+ int ret;
+
+ /* check common opts */
+ ret = ml_test_opt_check(opt);
+ if (ret != 0)
+ return ret;
+
+ return 0;
+}
+
+static void
+test_device_opt_dump(struct ml_options *opt)
+{
+ /* dump common opts */
+ ml_test_opt_dump(opt);
+}
+
+static int
+test_device_setup(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_device *t;
+ void *test_device;
+ int ret = 0;
+
+ /* allocate for test structure */
+ test_device = rte_zmalloc_socket(test->name, sizeof(struct test_device),
+ RTE_CACHE_LINE_SIZE, opt->socket_id);
+ if (test_device == NULL) {
+ ml_err("failed to allocate memory for test_model");
+ ret = -ENOMEM;
+ goto error;
+ }
+ test->test_priv = test_device;
+ t = ml_test_priv(test);
+
+ t->cmn.result = ML_TEST_FAILED;
+ t->cmn.opt = opt;
+
+ /* get device info */
+ ret = rte_ml_dev_info_get(opt->dev_id, &t->cmn.dev_info);
+ if (ret < 0) {
+ ml_err("failed to get device info");
+ goto error;
+ }
+
+ return 0;
+
+error:
+ if (test_device != NULL)
+ rte_free(test_device);
+
+ return ret;
+}
+
+static void
+test_device_destroy(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_device *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+ if (t != NULL)
+ rte_free(t);
+}
+
+static int
+test_device_reconfigure(struct ml_test *test, struct ml_options *opt)
+{
+ struct rte_ml_dev_config dev_config;
+ struct rte_ml_dev_qp_conf qp_conf;
+ struct test_device *t;
+ uint16_t qp_id = 0;
+ int ret = 0;
+
+ t = ml_test_priv(test);
+
+ /* configure with default options */
+ ret = ml_test_device_configure(test, opt);
+ if (ret != 0)
+ return ret;
+
+ /* setup one queue pair with nb_desc = 1 */
+ qp_conf.nb_desc = 1;
+ qp_conf.cb = NULL;
+
+ ret = rte_ml_dev_queue_pair_setup(opt->dev_id, qp_id, &qp_conf, opt->socket_id);
+ if (ret != 0) {
+ ml_err("Failed to setup ML device queue-pair, dev_id = %d, qp_id = %u\n",
+ opt->dev_id, qp_id);
+ goto error;
+ }
+
+ /* start device */
+ ret = ml_test_device_start(test, opt);
+ if (ret != 0)
+ goto error;
+
+ /* stop device */
+ ret = ml_test_device_stop(test, opt);
+ if (ret != 0) {
+ ml_err("Failed to stop device");
+ goto error;
+ }
+
+ /* reconfigure device based on dev_info */
+ dev_config.socket_id = opt->socket_id;
+ dev_config.nb_models = t->cmn.dev_info.max_models;
+ dev_config.nb_queue_pairs = t->cmn.dev_info.max_queue_pairs;
+ ret = rte_ml_dev_configure(opt->dev_id, &dev_config);
+ if (ret != 0) {
+ ml_err("Failed to reconfigure ML device, dev_id = %d\n", opt->dev_id);
+ return ret;
+ }
+
+ /* setup queue pairs */
+ for (qp_id = 0; qp_id < t->cmn.dev_info.max_queue_pairs; qp_id++) {
+ qp_conf.nb_desc = t->cmn.dev_info.max_desc;
+ qp_conf.cb = NULL;
+
+ ret = rte_ml_dev_queue_pair_setup(opt->dev_id, qp_id, &qp_conf, opt->socket_id);
+ if (ret != 0) {
+ ml_err("Failed to setup ML device queue-pair, dev_id = %d, qp_id = %u\n",
+ opt->dev_id, qp_id);
+ goto error;
+ }
+ }
+
+ /* start device */
+ ret = ml_test_device_start(test, opt);
+ if (ret != 0)
+ goto error;
+
+ /* stop device */
+ ret = ml_test_device_stop(test, opt);
+ if (ret != 0)
+ goto error;
+
+ /* close device */
+ ret = ml_test_device_close(test, opt);
+ if (ret != 0)
+ return ret;
+
+ return 0;
+
+error:
+ ml_test_device_close(test, opt);
+
+ return ret;
+}
+
+static int
+test_device_driver(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_device *t;
+ int ret = 0;
+
+ t = ml_test_priv(test);
+
+ /* sub-test: device reconfigure */
+ ret = test_device_reconfigure(test, opt);
+ if (ret != 0) {
+ printf("\n");
+ printf("Model Device Reconfigure Test: " CLRED "%s" CLNRM "\n", "Failed");
+ goto error;
+ } else {
+ printf("\n");
+ printf("Model Device Reconfigure Test: " CLYEL "%s" CLNRM "\n", "Passed");
+ }
+
+ printf("\n");
+
+ t->cmn.result = ML_TEST_SUCCESS;
+
+ return 0;
+
+error:
+ t->cmn.result = ML_TEST_FAILED;
+ return -1;
+}
+
+static int
+test_device_result(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_device *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+
+ return t->cmn.result;
+}
+
+static const struct ml_test_ops device_ops = {
+ .cap_check = test_device_cap_check,
+ .opt_check = test_device_opt_check,
+ .opt_dump = test_device_opt_dump,
+ .test_setup = test_device_setup,
+ .test_destroy = test_device_destroy,
+ .test_driver = test_device_driver,
+ .test_result = test_device_result,
+};
+
+ML_TEST_REGISTER(device_ops);
diff --git a/app/test-mldev/test_device_ops.h b/app/test-mldev/test_device_ops.h
new file mode 100644
index 0000000000..115b1072a2
--- /dev/null
+++ b/app/test-mldev/test_device_ops.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_TEST_DEVICE_OPS_
+#define _ML_TEST_DEVICE_OPS_
+
+#include <rte_common.h>
+
+#include "test_common.h"
+
+struct test_device {
+ /* common data */
+ struct test_common cmn;
+} __rte_cache_aligned;
+
+#endif /* _ML_TEST_DEVICE_OPS_ */
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* RE: [PATCH v4 03/12] app/mldev: add test case to validate device ops
2023-02-07 15:49 ` [PATCH v4 03/12] app/mldev: add test case to validate device ops Srikanth Yalavarthi
@ 2023-03-01 5:35 ` Anup Prabhu
0 siblings, 0 replies; 122+ messages in thread
From: Anup Prabhu @ 2023-03-01 5:35 UTC (permalink / raw)
To: Srikanth Yalavarthi, Srikanth Yalavarthi
Cc: dev, Shivah Shankar Shankar Narayan Rao,
Jerin Jacob Kollanukkaran, Prince Takkar, Parijat Shukla
[-- Attachment #1: Type: text/plain, Size: 1142 bytes --]
> -----Original Message-----
> From: Srikanth Yalavarthi <syalavarthi@marvell.com>
> Sent: Tuesday, February 7, 2023 9:20 PM
> To: Srikanth Yalavarthi <syalavarthi@marvell.com>
> Cc: dev@dpdk.org; Shivah Shankar Shankar Narayan Rao
> <sshankarnara@marvell.com>; Jerin Jacob Kollanukkaran
> <jerinj@marvell.com>; Anup Prabhu <aprabhu@marvell.com>; Prince Takkar
> <ptakkar@marvell.com>; Parijat Shukla <pshukla@marvell.com>
> Subject: [PATCH v4 03/12] app/mldev: add test case to validate device ops
>
> Added test case to validate device handling operations. Device ops test is a
> collection of multiple sub-tests. Enabled sub-test to validate device
> reconfiguration. Set device_ops as the default test.
>
> Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
> ---
> app/test-mldev/meson.build | 1 +
> app/test-mldev/ml_options.c | 5 +-
> app/test-mldev/test_device_ops.c | 234
> +++++++++++++++++++++++++++++++ app/test-mldev/test_device_ops.h
> | 17 +++
> 4 files changed, 255 insertions(+), 2 deletions(-) create mode 100644
>
Acked-by: Anup Prabhu <aprabhu@marvell.com>
[-- Attachment #2: winmail.dat --]
[-- Type: application/ms-tnef, Size: 23069 bytes --]
^ permalink raw reply [flat|nested] 122+ messages in thread
* [PATCH v4 04/12] app/mldev: add test case to validate model ops
2023-02-07 15:49 ` [PATCH v4 00/12] Implementation of mldev test application Srikanth Yalavarthi
` (2 preceding siblings ...)
2023-02-07 15:49 ` [PATCH v4 03/12] app/mldev: add test case to validate device ops Srikanth Yalavarthi
@ 2023-02-07 15:49 ` Srikanth Yalavarthi
2023-03-02 2:58 ` Anup Prabhu
2023-03-09 18:42 ` Thomas Monjalon
2023-02-07 15:49 ` [PATCH v4 05/12] app/mldev: add ordered inference test case Srikanth Yalavarthi
` (7 subsequent siblings)
11 siblings, 2 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-02-07 15:49 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Added test case to validate model operations. Model ops test
is a collection of sub-tests. Each sub-test invokes the model
operations in a specific order.
Sub-test A: (load -> start -> stop -> unload) x n
Sub-test B: load x n -> start x n -> stop x n -> unload x n
Sub-test C: load x n + (start + stop) x n + unload x n
Sub-test D: (load + start) x n -> (stop + unload) x n
Added internal functions to handle model load, start, stop and
unload. List of models to be used for testing can be specified
through application argument "--models"
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/meson.build | 2 +
app/test-mldev/ml_options.c | 45 ++-
app/test-mldev/ml_options.h | 9 +
app/test-mldev/test_model_common.c | 162 +++++++++++
app/test-mldev/test_model_common.h | 37 +++
app/test-mldev/test_model_ops.c | 433 +++++++++++++++++++++++++++++
app/test-mldev/test_model_ops.h | 21 ++
7 files changed, 706 insertions(+), 3 deletions(-)
create mode 100644 app/test-mldev/test_model_common.c
create mode 100644 app/test-mldev/test_model_common.h
create mode 100644 app/test-mldev/test_model_ops.c
create mode 100644 app/test-mldev/test_model_ops.h
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
index 60ea23d142..b09e1ccc8a 100644
--- a/app/test-mldev/meson.build
+++ b/app/test-mldev/meson.build
@@ -14,6 +14,8 @@ sources = files(
'parser.c',
'test_common.c',
'test_device_ops.c',
+ 'test_model_common.c',
+ 'test_model_ops.c',
)
deps += ['mldev']
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 2e5f11bca2..15043c0992 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -4,6 +4,7 @@
#include <errno.h>
#include <getopt.h>
+#include <linux/limits.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
@@ -27,6 +28,7 @@ ml_options_default(struct ml_options *opt)
strlcpy(opt->test_name, "device_ops", ML_TEST_NAME_MAX_LEN);
opt->dev_id = 0;
opt->socket_id = SOCKET_ID_ANY;
+ opt->nb_filelist = 0;
opt->debug = false;
}
@@ -63,11 +65,47 @@ ml_parse_socket_id(struct ml_options *opt, const char *arg)
return 0;
}
+static int
+ml_parse_models(struct ml_options *opt, const char *arg)
+{
+ const char *delim = ",";
+ char models[PATH_MAX];
+ char *token;
+ int ret = 0;
+
+ strlcpy(models, arg, PATH_MAX);
+
+ token = strtok(models, delim);
+ while (token != NULL) {
+ strlcpy(opt->filelist[opt->nb_filelist].model, token, PATH_MAX);
+ opt->nb_filelist++;
+
+ if (opt->nb_filelist >= ML_TEST_MAX_MODELS) {
+ ml_err("Exceeded model count, max = %d\n", ML_TEST_MAX_MODELS);
+ ret = -EINVAL;
+ break;
+ }
+ token = strtok(NULL, delim);
+ }
+
+ if (opt->nb_filelist == 0) {
+ ml_err("Models list is empty. Need at least one model for the test");
+ ret = -EINVAL;
+ }
+
+ return ret;
+}
+
static void
ml_dump_test_options(const char *testname)
{
if (strcmp(testname, "device_ops") == 0)
printf("\n");
+
+ if (strcmp(testname, "model_ops") == 0) {
+ printf("\t\t--models : comma separated list of models\n");
+ printf("\n");
+ }
}
static void
@@ -85,9 +123,9 @@ print_usage(char *program)
ml_test_dump_names(ml_dump_test_options);
}
-static struct option lgopts[] = {{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0},
- {ML_SOCKET_ID, 1, 0, 0}, {ML_DEBUG, 0, 0, 0},
- {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
+static struct option lgopts[] = {
+ {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0}, {ML_MODELS, 1, 0, 0},
+ {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
static int
ml_opts_parse_long(int opt_idx, struct ml_options *opt)
@@ -98,6 +136,7 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
{ML_TEST, ml_parse_test_name},
{ML_DEVICE_ID, ml_parse_dev_id},
{ML_SOCKET_ID, ml_parse_socket_id},
+ {ML_MODELS, ml_parse_models},
};
for (i = 0; i < RTE_DIM(parsermap); i++) {
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index 05311a9a47..8faf3b5deb 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -5,22 +5,31 @@
#ifndef _ML_OPTIONS_
#define _ML_OPTIONS_
+#include <linux/limits.h>
#include <stdbool.h>
#include <stdint.h>
#define ML_TEST_NAME_MAX_LEN 32
+#define ML_TEST_MAX_MODELS 8
/* Options names */
#define ML_TEST ("test")
#define ML_DEVICE_ID ("dev_id")
#define ML_SOCKET_ID ("socket_id")
+#define ML_MODELS ("models")
#define ML_DEBUG ("debug")
#define ML_HELP ("help")
+struct ml_filelist {
+ char model[PATH_MAX];
+};
+
struct ml_options {
char test_name[ML_TEST_NAME_MAX_LEN];
int16_t dev_id;
int socket_id;
+ struct ml_filelist filelist[ML_TEST_MAX_MODELS];
+ uint8_t nb_filelist;
bool debug;
};
diff --git a/app/test-mldev/test_model_common.c b/app/test-mldev/test_model_common.c
new file mode 100644
index 0000000000..6316430813
--- /dev/null
+++ b/app/test-mldev/test_model_common.c
@@ -0,0 +1,162 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <errno.h>
+#include <stdio.h>
+
+#include <rte_common.h>
+#include <rte_malloc.h>
+#include <rte_mldev.h>
+
+#include "ml_common.h"
+#include "ml_options.h"
+#include "ml_test.h"
+#include "test_common.h"
+#include "test_model_common.h"
+
+int
+ml_model_load(struct ml_test *test, struct ml_options *opt, struct ml_model *model, uint16_t fid)
+{
+ struct test_common *t = ml_test_priv(test);
+ struct rte_ml_model_params model_params;
+ FILE *fp;
+ int ret;
+
+ if (model->state == MODEL_LOADED)
+ return 0;
+
+ if (model->state != MODEL_INITIAL)
+ return -EINVAL;
+
+ /* read model binary */
+ fp = fopen(opt->filelist[fid].model, "r");
+ if (fp == NULL) {
+ ml_err("Failed to open model file : %s\n", opt->filelist[fid].model);
+ return -1;
+ }
+
+ fseek(fp, 0, SEEK_END);
+ model_params.size = ftell(fp);
+ fseek(fp, 0, SEEK_SET);
+
+ model_params.addr = rte_malloc_socket("ml_model", model_params.size,
+ t->dev_info.min_align_size, opt->socket_id);
+ if (model_params.addr == NULL) {
+ ml_err("Failed to allocate memory for model: %s\n", opt->filelist[fid].model);
+ fclose(fp);
+ return -ENOMEM;
+ }
+
+ if (fread(model_params.addr, 1, model_params.size, fp) != model_params.size) {
+ ml_err("Failed to read model file : %s\n", opt->filelist[fid].model);
+ rte_free(model_params.addr);
+ fclose(fp);
+ return -1;
+ }
+ fclose(fp);
+
+ /* load model to device */
+ ret = rte_ml_model_load(opt->dev_id, &model_params, &model->id);
+ if (ret != 0) {
+ ml_err("Failed to load model : %s\n", opt->filelist[fid].model);
+ model->state = MODEL_ERROR;
+ rte_free(model_params.addr);
+ return ret;
+ }
+
+ /* release mz */
+ rte_free(model_params.addr);
+
+ /* get model info */
+ ret = rte_ml_model_info_get(opt->dev_id, model->id, &model->info);
+ if (ret != 0) {
+ ml_err("Failed to get model info : %s\n", opt->filelist[fid].model);
+ return ret;
+ }
+
+ model->state = MODEL_LOADED;
+
+ return 0;
+}
+
+int
+ml_model_unload(struct ml_test *test, struct ml_options *opt, struct ml_model *model, uint16_t fid)
+{
+ struct test_common *t = ml_test_priv(test);
+ int ret;
+
+ RTE_SET_USED(t);
+
+ if (model->state == MODEL_INITIAL)
+ return 0;
+
+ if (model->state != MODEL_LOADED)
+ return -EINVAL;
+
+ /* unload model */
+ ret = rte_ml_model_unload(opt->dev_id, model->id);
+ if (ret != 0) {
+ ml_err("Failed to unload model: %s\n", opt->filelist[fid].model);
+ model->state = MODEL_ERROR;
+ return ret;
+ }
+
+ model->state = MODEL_INITIAL;
+
+ return 0;
+}
+
+int
+ml_model_start(struct ml_test *test, struct ml_options *opt, struct ml_model *model, uint16_t fid)
+{
+ struct test_common *t = ml_test_priv(test);
+ int ret;
+
+ RTE_SET_USED(t);
+
+ if (model->state == MODEL_STARTED)
+ return 0;
+
+ if (model->state != MODEL_LOADED)
+ return -EINVAL;
+
+ /* start model */
+ ret = rte_ml_model_start(opt->dev_id, model->id);
+ if (ret != 0) {
+ ml_err("Failed to start model : %s\n", opt->filelist[fid].model);
+ model->state = MODEL_ERROR;
+ return ret;
+ }
+
+ model->state = MODEL_STARTED;
+
+ return 0;
+}
+
+int
+ml_model_stop(struct ml_test *test, struct ml_options *opt, struct ml_model *model, uint16_t fid)
+{
+ struct test_common *t = ml_test_priv(test);
+ int ret;
+
+ RTE_SET_USED(t);
+
+ if (model->state == MODEL_LOADED)
+ return 0;
+
+ if (model->state != MODEL_STARTED)
+ return -EINVAL;
+
+ /* stop model */
+ ret = rte_ml_model_stop(opt->dev_id, model->id);
+ if (ret != 0) {
+ ml_err("Failed to stop model: %s\n", opt->filelist[fid].model);
+ model->state = MODEL_ERROR;
+ return ret;
+ }
+
+ model->state = MODEL_LOADED;
+
+ return 0;
+}
diff --git a/app/test-mldev/test_model_common.h b/app/test-mldev/test_model_common.h
new file mode 100644
index 0000000000..cdd1215cbc
--- /dev/null
+++ b/app/test-mldev/test_model_common.h
@@ -0,0 +1,37 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_TEST_MODEL_COMMON_
+#define _ML_TEST_MODEL_COMMON_
+
+#include <stdint.h>
+
+#include <rte_mldev.h>
+
+#include "ml_options.h"
+#include "ml_test.h"
+
+enum model_state {
+ MODEL_INITIAL,
+ MODEL_LOADED,
+ MODEL_STARTED,
+ MODEL_ERROR,
+};
+
+struct ml_model {
+ uint16_t id;
+ struct rte_ml_model_info info;
+ enum model_state state;
+};
+
+int ml_model_load(struct ml_test *test, struct ml_options *opt, struct ml_model *model,
+ uint16_t fid);
+int ml_model_unload(struct ml_test *test, struct ml_options *opt, struct ml_model *model,
+ uint16_t fid);
+int ml_model_start(struct ml_test *test, struct ml_options *opt, struct ml_model *model,
+ uint16_t fid);
+int ml_model_stop(struct ml_test *test, struct ml_options *opt, struct ml_model *model,
+ uint16_t fid);
+
+#endif /* _ML_TEST_MODEL_COMMON_ */
diff --git a/app/test-mldev/test_model_ops.c b/app/test-mldev/test_model_ops.c
new file mode 100644
index 0000000000..69c9df8ed6
--- /dev/null
+++ b/app/test-mldev/test_model_ops.c
@@ -0,0 +1,433 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <stdint.h>
+#include <stdio.h>
+#include <unistd.h>
+
+#include <rte_common.h>
+#include <rte_malloc.h>
+#include <rte_mldev.h>
+
+#include "ml_common.h"
+#include "ml_options.h"
+#include "ml_test.h"
+#include "test_model_ops.h"
+
+static bool
+test_model_ops_cap_check(struct ml_options *opt)
+{
+ if (!ml_test_cap_check(opt))
+ return false;
+
+ return true;
+}
+
+static int
+test_model_ops_opt_check(struct ml_options *opt)
+{
+ uint32_t i;
+ int ret;
+
+ /* check common opts */
+ ret = ml_test_opt_check(opt);
+ if (ret != 0)
+ return ret;
+
+ /* check model file availability */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ if (access(opt->filelist[i].model, F_OK) == -1) {
+ ml_err("Model file not available: id = %u, file = %s", i,
+ opt->filelist[i].model);
+ return -ENOENT;
+ }
+ }
+
+ return 0;
+}
+
+static void
+test_model_ops_opt_dump(struct ml_options *opt)
+{
+ uint32_t i;
+
+ /* dump common opts */
+ ml_test_opt_dump(opt);
+
+ /* dump test specific opts */
+ ml_dump_begin("models");
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_dump_list("model", i, opt->filelist[i].model);
+ ml_dump_end;
+}
+
+static int
+test_model_ops_setup(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+ void *test_model_ops;
+ int ret = 0;
+ uint32_t i;
+
+ /* allocate model ops test structure */
+ test_model_ops = rte_zmalloc_socket(test->name, sizeof(struct test_model_ops),
+ RTE_CACHE_LINE_SIZE, opt->socket_id);
+ if (test_model_ops == NULL) {
+ ml_err("Failed to allocate memory for test_model");
+ ret = -ENOMEM;
+ goto error;
+ }
+ test->test_priv = test_model_ops;
+ t = ml_test_priv(test);
+
+ t->cmn.result = ML_TEST_FAILED;
+ t->cmn.opt = opt;
+
+ /* get device info */
+ ret = rte_ml_dev_info_get(opt->dev_id, &t->cmn.dev_info);
+ if (ret < 0) {
+ ml_err("Failed to get device info");
+ goto error;
+ }
+
+ /* set model initial state */
+ for (i = 0; i < opt->nb_filelist; i++)
+ t->model[i].state = MODEL_INITIAL;
+
+ return 0;
+
+error:
+ if (test_model_ops != NULL)
+ rte_free(test_model_ops);
+
+ return ret;
+}
+
+static void
+test_model_ops_destroy(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+ if (t != NULL)
+ rte_free(t);
+}
+
+static int
+test_model_ops_mldev_setup(struct ml_test *test, struct ml_options *opt)
+{
+ int ret;
+
+ ret = ml_test_device_configure(test, opt);
+ if (ret != 0)
+ return ret;
+
+ ret = ml_test_device_start(test, opt);
+ if (ret != 0)
+ goto error;
+
+ return 0;
+
+error:
+ ml_test_device_close(test, opt);
+
+ return ret;
+}
+
+static int
+test_model_ops_mldev_destroy(struct ml_test *test, struct ml_options *opt)
+{
+ int ret;
+
+ ret = ml_test_device_stop(test, opt);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_test_device_close(test, opt);
+ if (ret != 0)
+ return ret;
+
+ return 0;
+
+error:
+ ml_test_device_close(test, opt);
+
+ return ret;
+}
+
+/* Sub-test A: (load -> start -> stop -> unload) x n */
+static int
+test_model_ops_subtest_a(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+ int ret = 0;
+ uint32_t i;
+
+ t = ml_test_priv(test);
+
+ /* load + start + stop + unload */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_load(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_start(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_stop(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_unload(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+error:
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_stop(test, opt, &t->model[i], i);
+
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_unload(test, opt, &t->model[i], i);
+
+ return ret;
+}
+
+/* Sub-test B: load x n -> start x n -> stop x n -> unload x n */
+static int
+test_model_ops_subtest_b(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+ int ret = 0;
+ uint32_t i;
+
+ t = ml_test_priv(test);
+
+ /* load */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_load(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* start */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_start(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* stop */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_stop(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* unload */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_unload(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ return 0;
+
+error:
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_stop(test, opt, &t->model[i], i);
+
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_unload(test, opt, &t->model[i], i);
+
+ return ret;
+}
+
+/* Sub-test C: load x n + (start + stop) x n + unload x n */
+static int
+test_model_ops_subtest_c(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+ int ret = 0;
+ uint32_t i;
+
+ t = ml_test_priv(test);
+
+ /* load */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_load(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* start + stop */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_start(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_stop(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* unload */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_unload(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ return 0;
+
+error:
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_stop(test, opt, &t->model[i], i);
+
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_unload(test, opt, &t->model[i], i);
+
+ return ret;
+}
+
+/* Sub-test D: (load + start) x n -> (stop + unload) x n */
+static int
+test_model_ops_subtest_d(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+ int ret = 0;
+ uint32_t i;
+
+ t = ml_test_priv(test);
+
+ /* load + start */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_load(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_start(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* stop + unload */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_stop(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_unload(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ return 0;
+
+error:
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_stop(test, opt, &t->model[i], i);
+
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_unload(test, opt, &t->model[i], i);
+
+ return ret;
+}
+
+static int
+test_model_ops_driver(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+ int ret = 0;
+
+ t = ml_test_priv(test);
+
+ /* device setup */
+ ret = test_model_ops_mldev_setup(test, opt);
+ if (ret != 0)
+ return ret;
+
+ printf("\n");
+
+ /* sub-test A */
+ ret = test_model_ops_subtest_a(test, opt);
+ if (ret != 0) {
+ printf("Model Ops Sub-test A: " CLRED "%s" CLNRM "\n", "Failed");
+ goto error;
+ } else {
+ printf("Model Ops Sub-test A: " CLYEL "%s" CLNRM "\n", "Passed");
+ }
+
+ /* sub-test B */
+ ret = test_model_ops_subtest_b(test, opt);
+ if (ret != 0) {
+ printf("Model Ops Sub-test B: " CLRED "%s" CLNRM "\n", "Failed");
+ goto error;
+ } else {
+ printf("Model Ops Sub-test B: " CLYEL "%s" CLNRM "\n", "Passed");
+ }
+
+ /* sub-test C */
+ ret = test_model_ops_subtest_c(test, opt);
+ if (ret != 0) {
+ printf("Model Ops Sub-test C: " CLRED "%s" CLNRM "\n", "Failed");
+ goto error;
+ } else {
+ printf("Model Ops Sub-test C: " CLYEL "%s" CLNRM "\n", "Passed");
+ }
+
+ /* sub-test D */
+ ret = test_model_ops_subtest_d(test, opt);
+ if (ret != 0) {
+ printf("Model Ops Sub-test D: " CLRED "%s" CLNRM "\n", "Failed");
+ goto error;
+ } else {
+ printf("Model Ops Sub-test D: " CLYEL "%s" CLNRM "\n", "Passed");
+ }
+
+ printf("\n");
+
+ /* device destroy */
+ ret = test_model_ops_mldev_destroy(test, opt);
+ if (ret != 0)
+ return ret;
+
+ t->cmn.result = ML_TEST_SUCCESS;
+
+ return 0;
+
+error:
+ test_model_ops_mldev_destroy(test, opt);
+
+ t->cmn.result = ML_TEST_FAILED;
+
+ return ret;
+}
+
+static int
+test_model_ops_result(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+
+ return t->cmn.result;
+}
+
+static const struct ml_test_ops model_ops = {
+ .cap_check = test_model_ops_cap_check,
+ .opt_check = test_model_ops_opt_check,
+ .opt_dump = test_model_ops_opt_dump,
+ .test_setup = test_model_ops_setup,
+ .test_destroy = test_model_ops_destroy,
+ .test_driver = test_model_ops_driver,
+ .test_result = test_model_ops_result,
+};
+
+ML_TEST_REGISTER(model_ops);
diff --git a/app/test-mldev/test_model_ops.h b/app/test-mldev/test_model_ops.h
new file mode 100644
index 0000000000..9dd8402390
--- /dev/null
+++ b/app/test-mldev/test_model_ops.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_TEST_MODEL_OPS_
+#define _ML_TEST_MODEL_OPS_
+
+#include <rte_common.h>
+
+#include "test_common.h"
+#include "test_model_common.h"
+
+struct test_model_ops {
+ /* common data */
+ struct test_common cmn;
+
+ /* test specific data */
+ struct ml_model model[ML_TEST_MAX_MODELS];
+} __rte_cache_aligned;
+
+#endif /* _ML_TEST_MODEL_OPS_ */
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* RE: [PATCH v4 04/12] app/mldev: add test case to validate model ops
2023-02-07 15:49 ` [PATCH v4 04/12] app/mldev: add test case to validate model ops Srikanth Yalavarthi
@ 2023-03-02 2:58 ` Anup Prabhu
2023-03-09 18:42 ` Thomas Monjalon
1 sibling, 0 replies; 122+ messages in thread
From: Anup Prabhu @ 2023-03-02 2:58 UTC (permalink / raw)
To: Srikanth Yalavarthi, Srikanth Yalavarthi
Cc: dev, Shivah Shankar Shankar Narayan Rao,
Jerin Jacob Kollanukkaran, Prince Takkar, Parijat Shukla
[-- Attachment #1: Type: text/plain, Size: 1871 bytes --]
> -----Original Message-----
> From: Srikanth Yalavarthi <syalavarthi@marvell.com>
> Sent: Tuesday, February 7, 2023 9:20 PM
> To: Srikanth Yalavarthi <syalavarthi@marvell.com>
> Cc: dev@dpdk.org; Shivah Shankar Shankar Narayan Rao
> <sshankarnara@marvell.com>; Jerin Jacob Kollanukkaran
> <jerinj@marvell.com>; Anup Prabhu <aprabhu@marvell.com>; Prince Takkar
> <ptakkar@marvell.com>; Parijat Shukla <pshukla@marvell.com>
> Subject: [PATCH v4 04/12] app/mldev: add test case to validate model ops
>
> Added test case to validate model operations. Model ops test is a collection
> of sub-tests. Each sub-test invokes the model operations in a specific order.
>
> Sub-test A: (load -> start -> stop -> unload) x n Sub-test B: load x n -> start x n
> -> stop x n -> unload x n Sub-test C: load x n + (start + stop) x n + unload x n
> Sub-test D: (load + start) x n -> (stop + unload) x n
>
> Added internal functions to handle model load, start, stop and unload. List of
> models to be used for testing can be specified through application argument
> "--models"
>
> Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
> ---
> app/test-mldev/meson.build | 2 +
> app/test-mldev/ml_options.c | 45 ++-
> app/test-mldev/ml_options.h | 9 +
> app/test-mldev/test_model_common.c | 162 +++++++++++ app/test-
> mldev/test_model_common.h | 37 +++
> app/test-mldev/test_model_ops.c | 433
> +++++++++++++++++++++++++++++
> app/test-mldev/test_model_ops.h | 21 ++
> 7 files changed, 706 insertions(+), 3 deletions(-) create mode 100644
> app/test-mldev/test_model_common.c
> create mode 100644 app/test-mldev/test_model_common.h
> create mode 100644 app/test-mldev/test_model_ops.c create mode
> 100644 app/test-mldev/test_model_ops.h
>
Acked-by: Anup Prabhu <aprabhu@marvell.com>
[-- Attachment #2: winmail.dat --]
[-- Type: application/ms-tnef, Size: 23288 bytes --]
^ permalink raw reply [flat|nested] 122+ messages in thread
* Re: [PATCH v4 04/12] app/mldev: add test case to validate model ops
2023-02-07 15:49 ` [PATCH v4 04/12] app/mldev: add test case to validate model ops Srikanth Yalavarthi
2023-03-02 2:58 ` Anup Prabhu
@ 2023-03-09 18:42 ` Thomas Monjalon
2023-03-10 2:55 ` [EXT] " Srikanth Yalavarthi
1 sibling, 1 reply; 122+ messages in thread
From: Thomas Monjalon @ 2023-03-09 18:42 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
07/02/2023 16:49, Srikanth Yalavarthi:
> --- a/app/test-mldev/ml_options.c
> +++ b/app/test-mldev/ml_options.c
> @@ -4,6 +4,7 @@
>
> #include <errno.h>
> #include <getopt.h>
> +#include <linux/limits.h>
It breaks on non-Linux OS of course.
include <limits.h> is enough and more portable.
I will fix while merging.
In general, I think you have too many includes.
You could reduce them by using devtools/process-iwyu.py
^ permalink raw reply [flat|nested] 122+ messages in thread
* RE: [EXT] Re: [PATCH v4 04/12] app/mldev: add test case to validate model ops
2023-03-09 18:42 ` Thomas Monjalon
@ 2023-03-10 2:55 ` Srikanth Yalavarthi
0 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-10 2:55 UTC (permalink / raw)
To: Thomas Monjalon
Cc: dev, Shivah Shankar Shankar Narayan Rao,
Jerin Jacob Kollanukkaran, Anup Prabhu, Prince Takkar,
Parijat Shukla, Srikanth Yalavarthi
> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: 10 March 2023 00:13
> To: Srikanth Yalavarthi <syalavarthi@marvell.com>
> Cc: dev@dpdk.org; Shivah Shankar Shankar Narayan Rao
> <sshankarnara@marvell.com>; Jerin Jacob Kollanukkaran
> <jerinj@marvell.com>; Anup Prabhu <aprabhu@marvell.com>; Prince Takkar
> <ptakkar@marvell.com>; Parijat Shukla <pshukla@marvell.com>
> Subject: [EXT] Re: [PATCH v4 04/12] app/mldev: add test case to validate
> model ops
>
> External Email
>
> ----------------------------------------------------------------------
> 07/02/2023 16:49, Srikanth Yalavarthi:
> > --- a/app/test-mldev/ml_options.c
> > +++ b/app/test-mldev/ml_options.c
> > @@ -4,6 +4,7 @@
> >
> > #include <errno.h>
> > #include <getopt.h>
> > +#include <linux/limits.h>
>
> It breaks on non-Linux OS of course.
> include <limits.h> is enough and more portable.
>
> I will fix while merging.
Currently ML is not supported on windows. Hence used <linux/limits.h>. Will be removed in version 5.
>
> In general, I think you have too many includes.
> You could reduce them by using devtools/process-iwyu.py
>
Will do a cleanup on header includes in version 5.
^ permalink raw reply [flat|nested] 122+ messages in thread
* [PATCH v4 05/12] app/mldev: add ordered inference test case
2023-02-07 15:49 ` [PATCH v4 00/12] Implementation of mldev test application Srikanth Yalavarthi
` (3 preceding siblings ...)
2023-02-07 15:49 ` [PATCH v4 04/12] app/mldev: add test case to validate model ops Srikanth Yalavarthi
@ 2023-02-07 15:49 ` Srikanth Yalavarthi
2023-02-27 6:11 ` Anup Prabhu
2023-03-09 20:06 ` Thomas Monjalon
2023-02-07 15:49 ` [PATCH v4 06/12] app/mldev: add test case to interleave inferences Srikanth Yalavarthi
` (6 subsequent siblings)
11 siblings, 2 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-02-07 15:49 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Added an ordered test case to execute inferences with single
or multiple models. In this test case inference requests for
a model are enqueued after completion of all requests for
the previous model. Test supports inference repetitions.
Operations sequence when testing with N models and R reps,
(load -> start -> (enqueue + dequeue) x R -> stop -> unload) x N
Test case can be executed by selecting "inference_ordered" test
and repetitions can be specified through "--repetitions" argument.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/meson.build | 2 +
app/test-mldev/ml_options.c | 73 ++-
app/test-mldev/ml_options.h | 17 +-
app/test-mldev/test_inference_common.c | 565 ++++++++++++++++++++++++
app/test-mldev/test_inference_common.h | 65 +++
app/test-mldev/test_inference_ordered.c | 119 +++++
app/test-mldev/test_model_common.h | 10 +
7 files changed, 839 insertions(+), 12 deletions(-)
create mode 100644 app/test-mldev/test_inference_common.c
create mode 100644 app/test-mldev/test_inference_common.h
create mode 100644 app/test-mldev/test_inference_ordered.c
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
index b09e1ccc8a..475d76d126 100644
--- a/app/test-mldev/meson.build
+++ b/app/test-mldev/meson.build
@@ -16,6 +16,8 @@ sources = files(
'test_device_ops.c',
'test_model_common.c',
'test_model_ops.c',
+ 'test_inference_common.c',
+ 'test_inference_ordered.c',
)
deps += ['mldev']
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 15043c0992..10dad18fff 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -29,6 +29,7 @@ ml_options_default(struct ml_options *opt)
opt->dev_id = 0;
opt->socket_id = SOCKET_ID_ANY;
opt->nb_filelist = 0;
+ opt->repetitions = 1;
opt->debug = false;
}
@@ -96,6 +97,60 @@ ml_parse_models(struct ml_options *opt, const char *arg)
return ret;
}
+static int
+ml_parse_filelist(struct ml_options *opt, const char *arg)
+{
+ const char *delim = ",";
+ char filelist[PATH_MAX];
+ char *token;
+
+ if (opt->nb_filelist >= ML_TEST_MAX_MODELS) {
+ ml_err("Exceeded filelist count, max = %d\n", ML_TEST_MAX_MODELS);
+ return -1;
+ }
+
+ strlcpy(filelist, arg, PATH_MAX);
+
+ /* model */
+ token = strtok(filelist, delim);
+ if (token == NULL) {
+ ml_err("Invalid filelist, model not specified = %s\n", arg);
+ return -EINVAL;
+ }
+ strlcpy(opt->filelist[opt->nb_filelist].model, token, PATH_MAX);
+
+ /* input */
+ token = strtok(NULL, delim);
+ if (token == NULL) {
+ ml_err("Invalid filelist, input not specified = %s\n", arg);
+ return -EINVAL;
+ }
+ strlcpy(opt->filelist[opt->nb_filelist].input, token, PATH_MAX);
+
+ /* output */
+ token = strtok(NULL, delim);
+ if (token == NULL) {
+ ml_err("Invalid filelist, output not specified = %s\n", arg);
+ return -EINVAL;
+ }
+ strlcpy(opt->filelist[opt->nb_filelist].output, token, PATH_MAX);
+
+ opt->nb_filelist++;
+
+ if (opt->nb_filelist == 0) {
+ ml_err("Empty filelist. Need at least one filelist entry for the test.");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int
+ml_parse_repetitions(struct ml_options *opt, const char *arg)
+{
+ return parser_read_uint64(&opt->repetitions, arg);
+}
+
static void
ml_dump_test_options(const char *testname)
{
@@ -106,6 +161,12 @@ ml_dump_test_options(const char *testname)
printf("\t\t--models : comma separated list of models\n");
printf("\n");
}
+
+ if (strcmp(testname, "inference_ordered") == 0) {
+ printf("\t\t--filelist : comma separated list of model, input and output\n"
+ "\t\t--repetitions : number of inference repetitions\n");
+ printf("\n");
+ }
}
static void
@@ -124,8 +185,9 @@ print_usage(char *program)
}
static struct option lgopts[] = {
- {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0}, {ML_MODELS, 1, 0, 0},
- {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
+ {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
+ {ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
+ {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
static int
ml_opts_parse_long(int opt_idx, struct ml_options *opt)
@@ -133,10 +195,9 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
unsigned int i;
struct long_opt_parser parsermap[] = {
- {ML_TEST, ml_parse_test_name},
- {ML_DEVICE_ID, ml_parse_dev_id},
- {ML_SOCKET_ID, ml_parse_socket_id},
- {ML_MODELS, ml_parse_models},
+ {ML_TEST, ml_parse_test_name}, {ML_DEVICE_ID, ml_parse_dev_id},
+ {ML_SOCKET_ID, ml_parse_socket_id}, {ML_MODELS, ml_parse_models},
+ {ML_FILELIST, ml_parse_filelist}, {ML_REPETITIONS, ml_parse_repetitions},
};
for (i = 0; i < RTE_DIM(parsermap); i++) {
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index 8faf3b5deb..ad8aee5964 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -13,15 +13,19 @@
#define ML_TEST_MAX_MODELS 8
/* Options names */
-#define ML_TEST ("test")
-#define ML_DEVICE_ID ("dev_id")
-#define ML_SOCKET_ID ("socket_id")
-#define ML_MODELS ("models")
-#define ML_DEBUG ("debug")
-#define ML_HELP ("help")
+#define ML_TEST ("test")
+#define ML_DEVICE_ID ("dev_id")
+#define ML_SOCKET_ID ("socket_id")
+#define ML_MODELS ("models")
+#define ML_FILELIST ("filelist")
+#define ML_REPETITIONS ("repetitions")
+#define ML_DEBUG ("debug")
+#define ML_HELP ("help")
struct ml_filelist {
char model[PATH_MAX];
+ char input[PATH_MAX];
+ char output[PATH_MAX];
};
struct ml_options {
@@ -30,6 +34,7 @@ struct ml_options {
int socket_id;
struct ml_filelist filelist[ML_TEST_MAX_MODELS];
uint8_t nb_filelist;
+ uint64_t repetitions;
bool debug;
};
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
new file mode 100644
index 0000000000..ff25c056a0
--- /dev/null
+++ b/app/test-mldev/test_inference_common.c
@@ -0,0 +1,565 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <errno.h>
+#include <stdbool.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <unistd.h>
+
+#include <rte_common.h>
+#include <rte_launch.h>
+#include <rte_lcore.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_mldev.h>
+
+#include "ml_common.h"
+#include "ml_options.h"
+#include "ml_test.h"
+#include "test_common.h"
+#include "test_inference_common.h"
+
+/* Enqueue inference requests with burst size equal to 1 */
+static int
+ml_enqueue_single(void *arg)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)arg);
+ struct ml_request *req = NULL;
+ struct rte_ml_op *op = NULL;
+ struct ml_core_args *args;
+ uint64_t model_enq = 0;
+ uint32_t burst_enq;
+ uint32_t lcore_id;
+ uint16_t fid;
+ int ret;
+
+ lcore_id = rte_lcore_id();
+ args = &t->args[lcore_id];
+ model_enq = 0;
+
+ if (args->nb_reqs == 0)
+ return 0;
+
+next_rep:
+ fid = args->start_fid;
+
+next_model:
+ ret = rte_mempool_get(t->op_pool, (void **)&op);
+ if (ret != 0)
+ goto next_model;
+
+retry:
+ ret = rte_mempool_get(t->model[fid].io_pool, (void **)&req);
+ if (ret != 0)
+ goto retry;
+
+ op->model_id = t->model[fid].id;
+ op->nb_batches = t->model[fid].info.batch_size;
+ op->mempool = t->op_pool;
+
+ op->input.addr = req->input;
+ op->input.length = t->model[fid].inp_qsize;
+ op->input.next = NULL;
+
+ op->output.addr = req->output;
+ op->output.length = t->model[fid].out_qsize;
+ op->output.next = NULL;
+
+ op->user_ptr = req;
+ req->niters++;
+ req->fid = fid;
+
+enqueue_req:
+ burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, 0, &op, 1);
+ if (burst_enq == 0)
+ goto enqueue_req;
+
+ fid++;
+ if (likely(fid <= args->end_fid))
+ goto next_model;
+
+ model_enq++;
+ if (likely(model_enq < args->nb_reqs))
+ goto next_rep;
+
+ return 0;
+}
+
+/* Dequeue inference requests with burst size equal to 1 */
+static int
+ml_dequeue_single(void *arg)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)arg);
+ struct rte_ml_op_error error;
+ struct rte_ml_op *op = NULL;
+ struct ml_core_args *args;
+ struct ml_request *req;
+ uint64_t total_deq = 0;
+ uint8_t nb_filelist;
+ uint32_t burst_deq;
+ uint32_t lcore_id;
+
+ lcore_id = rte_lcore_id();
+ args = &t->args[lcore_id];
+ nb_filelist = args->end_fid - args->start_fid + 1;
+
+ if (args->nb_reqs == 0)
+ return 0;
+
+dequeue_req:
+ burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, 0, &op, 1);
+
+ if (likely(burst_deq == 1)) {
+ total_deq += burst_deq;
+ if (unlikely(op->status == RTE_ML_OP_STATUS_ERROR)) {
+ rte_ml_op_error_get(t->cmn.opt->dev_id, op, &error);
+ ml_err("error_code = 0x%016lx, error_message = %s\n", error.errcode,
+ error.message);
+ }
+ req = (struct ml_request *)op->user_ptr;
+ rte_mempool_put(t->model[req->fid].io_pool, req);
+ rte_mempool_put(t->op_pool, op);
+ }
+
+ if (likely(total_deq < args->nb_reqs * nb_filelist))
+ goto dequeue_req;
+
+ return 0;
+}
+
+bool
+test_inference_cap_check(struct ml_options *opt)
+{
+ struct rte_ml_dev_info dev_info;
+
+ if (!ml_test_cap_check(opt))
+ return false;
+
+ rte_ml_dev_info_get(opt->dev_id, &dev_info);
+ if (opt->nb_filelist > dev_info.max_models) {
+ ml_err("Insufficient capabilities: Filelist count exceeded device limit, count = %u (max limit = %u)",
+ opt->nb_filelist, dev_info.max_models);
+ return false;
+ }
+
+ return true;
+}
+
+int
+test_inference_opt_check(struct ml_options *opt)
+{
+ uint32_t i;
+ int ret;
+
+ /* check common opts */
+ ret = ml_test_opt_check(opt);
+ if (ret != 0)
+ return ret;
+
+ /* check file availability */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ if (access(opt->filelist[i].model, F_OK) == -1) {
+ ml_err("Model file not accessible: id = %u, file = %s", i,
+ opt->filelist[i].model);
+ return -ENOENT;
+ }
+
+ if (access(opt->filelist[i].input, F_OK) == -1) {
+ ml_err("Input file not accessible: id = %u, file = %s", i,
+ opt->filelist[i].input);
+ return -ENOENT;
+ }
+ }
+
+ if (opt->repetitions == 0) {
+ ml_err("Invalid option, repetitions = %" PRIu64 "\n", opt->repetitions);
+ return -EINVAL;
+ }
+
+ /* check number of available lcores. */
+ if (rte_lcore_count() < 3) {
+ ml_err("Insufficient lcores = %u\n", rte_lcore_count());
+ ml_err("Minimum lcores required to create %u queue-pairs = %u\n", 1, 3);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+void
+test_inference_opt_dump(struct ml_options *opt)
+{
+ uint32_t i;
+
+ /* dump common opts */
+ ml_test_opt_dump(opt);
+
+ /* dump test opts */
+ ml_dump("repetitions", "%" PRIu64, opt->repetitions);
+
+ ml_dump_begin("filelist");
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ml_dump_list("model", i, opt->filelist[i].model);
+ ml_dump_list("input", i, opt->filelist[i].input);
+ ml_dump_list("output", i, opt->filelist[i].output);
+ }
+ ml_dump_end;
+}
+
+int
+test_inference_setup(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t;
+ void *test_inference;
+ int ret = 0;
+ uint32_t i;
+
+ test_inference = rte_zmalloc_socket(test->name, sizeof(struct test_inference),
+ RTE_CACHE_LINE_SIZE, opt->socket_id);
+ if (test_inference == NULL) {
+ ml_err("failed to allocate memory for test_model");
+ ret = -ENOMEM;
+ goto error;
+ }
+ test->test_priv = test_inference;
+ t = ml_test_priv(test);
+
+ t->nb_used = 0;
+ t->cmn.result = ML_TEST_FAILED;
+ t->cmn.opt = opt;
+
+ /* get device info */
+ ret = rte_ml_dev_info_get(opt->dev_id, &t->cmn.dev_info);
+ if (ret < 0) {
+ ml_err("failed to get device info");
+ goto error;
+ }
+
+ t->enqueue = ml_enqueue_single;
+ t->dequeue = ml_dequeue_single;
+
+ /* set model initial state */
+ for (i = 0; i < opt->nb_filelist; i++)
+ t->model[i].state = MODEL_INITIAL;
+
+ return 0;
+
+error:
+ if (test_inference != NULL)
+ rte_free(test_inference);
+
+ return ret;
+}
+
+void
+test_inference_destroy(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+ if (t != NULL)
+ rte_free(t);
+}
+
+int
+ml_inference_mldev_setup(struct ml_test *test, struct ml_options *opt)
+{
+ struct rte_ml_dev_qp_conf qp_conf;
+ struct test_inference *t;
+ int ret;
+
+ t = ml_test_priv(test);
+
+ ret = ml_test_device_configure(test, opt);
+ if (ret != 0)
+ return ret;
+
+ /* setup queue pairs */
+ qp_conf.nb_desc = t->cmn.dev_info.max_desc;
+ qp_conf.cb = NULL;
+
+ ret = rte_ml_dev_queue_pair_setup(opt->dev_id, 0, &qp_conf, opt->socket_id);
+ if (ret != 0) {
+ ml_err("Failed to setup ml device queue-pair, dev_id = %d, qp_id = %u\n",
+ opt->dev_id, 0);
+ goto error;
+ }
+
+ ret = ml_test_device_start(test, opt);
+ if (ret != 0)
+ goto error;
+
+ return 0;
+
+error:
+ ml_test_device_close(test, opt);
+
+ return ret;
+}
+
+int
+ml_inference_mldev_destroy(struct ml_test *test, struct ml_options *opt)
+{
+ int ret;
+
+ ret = ml_test_device_stop(test, opt);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_test_device_close(test, opt);
+ if (ret != 0)
+ return ret;
+
+ return 0;
+
+error:
+ ml_test_device_close(test, opt);
+
+ return ret;
+}
+
+/* Callback for IO pool create. This function would compute the fields of ml_request
+ * structure and prepare the quantized input data.
+ */
+static void
+ml_request_initialize(struct rte_mempool *mp, void *opaque, void *obj, unsigned int obj_idx)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)opaque);
+ struct ml_request *req = (struct ml_request *)obj;
+
+ RTE_SET_USED(mp);
+ RTE_SET_USED(obj_idx);
+
+ req->input = RTE_PTR_ADD(
+ obj, RTE_ALIGN_CEIL(sizeof(struct ml_request), t->cmn.dev_info.min_align_size));
+ req->output = RTE_PTR_ADD(req->input, RTE_ALIGN_CEIL(t->model[t->fid].inp_qsize,
+ t->cmn.dev_info.min_align_size));
+ req->niters = 0;
+
+ /* quantize data */
+ rte_ml_io_quantize(t->cmn.opt->dev_id, t->model[t->fid].id,
+ t->model[t->fid].info.batch_size, t->model[t->fid].input, req->input);
+}
+
+int
+ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, uint16_t fid)
+{
+ struct test_inference *t = ml_test_priv(test);
+ char mz_name[RTE_MEMZONE_NAMESIZE];
+ char mp_name[RTE_MEMPOOL_NAMESIZE];
+ const struct rte_memzone *mz;
+ uint64_t nb_buffers;
+ uint32_t buff_size;
+ uint32_t mz_size;
+ uint32_t fsize;
+ FILE *fp;
+ int ret;
+
+ /* get input buffer size */
+ ret = rte_ml_io_input_size_get(opt->dev_id, t->model[fid].id, t->model[fid].info.batch_size,
+ &t->model[fid].inp_qsize, &t->model[fid].inp_dsize);
+ if (ret != 0) {
+ ml_err("Failed to get input size, model : %s\n", opt->filelist[fid].model);
+ return ret;
+ }
+
+ /* get output buffer size */
+ ret = rte_ml_io_output_size_get(opt->dev_id, t->model[fid].id,
+ t->model[fid].info.batch_size, &t->model[fid].out_qsize,
+ &t->model[fid].out_dsize);
+ if (ret != 0) {
+ ml_err("Failed to get input size, model : %s\n", opt->filelist[fid].model);
+ return ret;
+ }
+
+ /* allocate buffer for user data */
+ mz_size = t->model[fid].inp_dsize + t->model[fid].out_dsize;
+ sprintf(mz_name, "ml_user_data_%d", fid);
+ mz = rte_memzone_reserve(mz_name, mz_size, opt->socket_id, 0);
+ if (mz == NULL) {
+ ml_err("Memzone allocation failed for ml_user_data\n");
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ t->model[fid].input = mz->addr;
+ t->model[fid].output = RTE_PTR_ADD(t->model[fid].input, t->model[fid].inp_dsize);
+
+ /* load input file */
+ fp = fopen(opt->filelist[fid].input, "r");
+ if (fp == NULL) {
+ ml_err("Failed to open input file : %s\n", opt->filelist[fid].input);
+ ret = -errno;
+ goto error;
+ }
+
+ fseek(fp, 0, SEEK_END);
+ fsize = ftell(fp);
+ fseek(fp, 0, SEEK_SET);
+ if (fsize != t->model[fid].inp_dsize) {
+ ml_err("Invalid input file, size = %u (expected size = %" PRIu64 ")\n", fsize,
+ t->model[fid].inp_dsize);
+ ret = -EINVAL;
+ fclose(fp);
+ goto error;
+ }
+
+ if (fread(t->model[fid].input, 1, t->model[fid].inp_dsize, fp) != t->model[fid].inp_dsize) {
+ ml_err("Failed to read input file : %s\n", opt->filelist[fid].input);
+ ret = -errno;
+ fclose(fp);
+ goto error;
+ }
+ fclose(fp);
+
+ /* create mempool for quantized input and output buffers. ml_request_initialize is
+ * used as a callback for object creation.
+ */
+ buff_size = RTE_ALIGN_CEIL(sizeof(struct ml_request), t->cmn.dev_info.min_align_size) +
+ RTE_ALIGN_CEIL(t->model[fid].inp_qsize, t->cmn.dev_info.min_align_size) +
+ RTE_ALIGN_CEIL(t->model[fid].out_qsize, t->cmn.dev_info.min_align_size);
+ nb_buffers = RTE_MIN((uint64_t)ML_TEST_MAX_POOL_SIZE, opt->repetitions);
+
+ t->fid = fid;
+ sprintf(mp_name, "ml_io_pool_%d", fid);
+ t->model[fid].io_pool = rte_mempool_create(mp_name, nb_buffers, buff_size, 0, 0, NULL, NULL,
+ ml_request_initialize, test, opt->socket_id, 0);
+ if (t->model[fid].io_pool == NULL) {
+ ml_err("Failed to create io pool : %s\n", "ml_io_pool");
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ return 0;
+
+error:
+ if (mz != NULL)
+ rte_memzone_free(mz);
+
+ if (t->model[fid].io_pool != NULL) {
+ rte_mempool_free(t->model[fid].io_pool);
+ t->model[fid].io_pool = NULL;
+ }
+
+ return ret;
+}
+
+void
+ml_inference_iomem_destroy(struct ml_test *test, struct ml_options *opt, uint16_t fid)
+{
+ char mz_name[RTE_MEMZONE_NAMESIZE];
+ char mp_name[RTE_MEMPOOL_NAMESIZE];
+ const struct rte_memzone *mz;
+ struct rte_mempool *mp;
+
+ RTE_SET_USED(test);
+ RTE_SET_USED(opt);
+
+ /* release user data memzone */
+ sprintf(mz_name, "ml_user_data_%d", fid);
+ mz = rte_memzone_lookup(mz_name);
+ if (mz != NULL)
+ rte_memzone_free(mz);
+
+ /* destroy io pool */
+ sprintf(mp_name, "ml_io_pool_%d", fid);
+ mp = rte_mempool_lookup(mp_name);
+ if (mp != NULL)
+ rte_mempool_free(mp);
+}
+
+int
+ml_inference_mem_setup(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t = ml_test_priv(test);
+
+ /* create op pool */
+ t->op_pool = rte_ml_op_pool_create("ml_test_op_pool", ML_TEST_MAX_POOL_SIZE, 0, 0,
+ opt->socket_id);
+ if (t->op_pool == NULL) {
+ ml_err("Failed to create op pool : %s\n", "ml_op_pool");
+ return -ENOMEM;
+ }
+
+ return 0;
+}
+
+void
+ml_inference_mem_destroy(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t = ml_test_priv(test);
+
+ RTE_SET_USED(opt);
+
+ /* release op pool */
+ if (t->op_pool != NULL)
+ rte_mempool_free(t->op_pool);
+}
+
+/* Callback for mempool object iteration. This call would dequantize output data. */
+static void
+ml_request_finish(struct rte_mempool *mp, void *opaque, void *obj, unsigned int obj_idx)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)opaque);
+ struct ml_request *req = (struct ml_request *)obj;
+ struct ml_model *model = &t->model[req->fid];
+
+ RTE_SET_USED(mp);
+ RTE_SET_USED(obj_idx);
+
+ if (req->niters == 0)
+ return;
+
+ t->nb_used++;
+ rte_ml_io_dequantize(t->cmn.opt->dev_id, model->id, t->model[req->fid].info.batch_size,
+ req->output, model->output);
+}
+
+int
+ml_inference_result(struct ml_test *test, struct ml_options *opt, uint16_t fid)
+{
+ struct test_inference *t = ml_test_priv(test);
+
+ RTE_SET_USED(opt);
+
+ rte_mempool_obj_iter(t->model[fid].io_pool, ml_request_finish, test);
+
+ if (t->nb_used > 0)
+ t->cmn.result = ML_TEST_SUCCESS;
+ else
+ t->cmn.result = ML_TEST_FAILED;
+
+ return t->cmn.result;
+}
+
+int
+ml_inference_launch_cores(struct ml_test *test, struct ml_options *opt, uint16_t start_fid,
+ uint16_t end_fid)
+{
+ struct test_inference *t = ml_test_priv(test);
+ uint32_t lcore_id;
+ uint32_t id = 0;
+
+ RTE_LCORE_FOREACH_WORKER(lcore_id)
+ {
+ if (id == 2)
+ break;
+
+ t->args[lcore_id].nb_reqs = opt->repetitions;
+ t->args[lcore_id].start_fid = start_fid;
+ t->args[lcore_id].end_fid = end_fid;
+
+ if (id % 2 == 0)
+ rte_eal_remote_launch(t->enqueue, test, lcore_id);
+ else
+ rte_eal_remote_launch(t->dequeue, test, lcore_id);
+
+ id++;
+ }
+
+ return 0;
+}
diff --git a/app/test-mldev/test_inference_common.h b/app/test-mldev/test_inference_common.h
new file mode 100644
index 0000000000..99baad5bfd
--- /dev/null
+++ b/app/test-mldev/test_inference_common.h
@@ -0,0 +1,65 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_TEST_INFERENCE_COMMON_
+#define _ML_TEST_INFERENCE_COMMON_
+
+#include <stdbool.h>
+#include <stdint.h>
+
+#include <rte_common.h>
+#include <rte_mempool.h>
+
+#include "ml_options.h"
+#include "ml_test.h"
+#include "test_common.h"
+#include "test_model_common.h"
+
+struct ml_request {
+ uint8_t *input;
+ uint8_t *output;
+ uint16_t fid;
+ uint64_t niters;
+};
+
+struct ml_core_args {
+ uint64_t nb_reqs;
+ uint16_t start_fid;
+ uint16_t end_fid;
+};
+
+struct test_inference {
+ /* common data */
+ struct test_common cmn;
+
+ /* test specific data */
+ struct ml_model model[ML_TEST_MAX_MODELS];
+ struct rte_mempool *op_pool;
+
+ uint64_t nb_used;
+ uint16_t fid;
+
+ int (*enqueue)(void *arg);
+ int (*dequeue)(void *arg);
+
+ struct ml_core_args args[RTE_MAX_LCORE];
+} __rte_cache_aligned;
+
+bool test_inference_cap_check(struct ml_options *opt);
+int test_inference_opt_check(struct ml_options *opt);
+void test_inference_opt_dump(struct ml_options *opt);
+int test_inference_setup(struct ml_test *test, struct ml_options *opt);
+void test_inference_destroy(struct ml_test *test, struct ml_options *opt);
+
+int ml_inference_mldev_setup(struct ml_test *test, struct ml_options *opt);
+int ml_inference_mldev_destroy(struct ml_test *test, struct ml_options *opt);
+int ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, uint16_t fid);
+void ml_inference_iomem_destroy(struct ml_test *test, struct ml_options *opt, uint16_t fid);
+int ml_inference_mem_setup(struct ml_test *test, struct ml_options *opt);
+void ml_inference_mem_destroy(struct ml_test *test, struct ml_options *opt);
+int ml_inference_result(struct ml_test *test, struct ml_options *opt, uint16_t fid);
+int ml_inference_launch_cores(struct ml_test *test, struct ml_options *opt, uint16_t start_fid,
+ uint16_t end_fid);
+
+#endif /* _ML_TEST_INFERENCE_COMMON_ */
diff --git a/app/test-mldev/test_inference_ordered.c b/app/test-mldev/test_inference_ordered.c
new file mode 100644
index 0000000000..ee9c7bf14e
--- /dev/null
+++ b/app/test-mldev/test_inference_ordered.c
@@ -0,0 +1,119 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <stdint.h>
+
+#include <rte_common.h>
+#include <rte_launch.h>
+
+#include "ml_common.h"
+#include "ml_test.h"
+#include "test_inference_common.h"
+#include "test_model_common.h"
+
+static int
+test_inference_ordered_driver(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t;
+ uint16_t fid = 0;
+ int ret = 0;
+
+ t = ml_test_priv(test);
+
+ ret = ml_inference_mldev_setup(test, opt);
+ if (ret != 0)
+ return ret;
+
+ ret = ml_inference_mem_setup(test, opt);
+ if (ret != 0)
+ return ret;
+
+next_model:
+ /* load model */
+ ret = ml_model_load(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ /* start model */
+ ret = ml_model_start(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_inference_iomem_setup(test, opt, fid);
+ if (ret != 0)
+ goto error;
+
+ /* launch inferences for one model using available queue pairs */
+ ret = ml_inference_launch_cores(test, opt, fid, fid);
+ if (ret != 0) {
+ ml_err("failed to launch cores");
+ goto error;
+ }
+
+ rte_eal_mp_wait_lcore();
+
+ ret = ml_inference_result(test, opt, fid);
+ if (ret != ML_TEST_SUCCESS)
+ goto error;
+
+ ml_inference_iomem_destroy(test, opt, fid);
+
+ /* stop model */
+ ret = ml_model_stop(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ /* unload model */
+ ret = ml_model_unload(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ fid++;
+ if (fid < opt->nb_filelist)
+ goto next_model;
+
+ ml_inference_mem_destroy(test, opt);
+
+ ret = ml_inference_mldev_destroy(test, opt);
+ if (ret != 0)
+ return ret;
+
+ t->cmn.result = ML_TEST_SUCCESS;
+
+ return 0;
+
+error:
+ ml_inference_iomem_destroy(test, opt, fid);
+ ml_inference_mem_destroy(test, opt);
+ ml_model_stop(test, opt, &t->model[fid], fid);
+ ml_model_unload(test, opt, &t->model[fid], fid);
+
+ t->cmn.result = ML_TEST_FAILED;
+
+ return ret;
+}
+
+static int
+test_inference_ordered_result(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+
+ return t->cmn.result;
+}
+
+static const struct ml_test_ops inference_ordered = {
+ .cap_check = test_inference_cap_check,
+ .opt_check = test_inference_opt_check,
+ .opt_dump = test_inference_opt_dump,
+ .test_setup = test_inference_setup,
+ .test_destroy = test_inference_destroy,
+ .test_driver = test_inference_ordered_driver,
+ .test_result = test_inference_ordered_result,
+};
+
+ML_TEST_REGISTER(inference_ordered);
diff --git a/app/test-mldev/test_model_common.h b/app/test-mldev/test_model_common.h
index cdd1215cbc..6eb70a352a 100644
--- a/app/test-mldev/test_model_common.h
+++ b/app/test-mldev/test_model_common.h
@@ -23,6 +23,16 @@ struct ml_model {
uint16_t id;
struct rte_ml_model_info info;
enum model_state state;
+
+ uint64_t inp_dsize;
+ uint64_t inp_qsize;
+ uint64_t out_dsize;
+ uint64_t out_qsize;
+
+ uint8_t *input;
+ uint8_t *output;
+
+ struct rte_mempool *io_pool;
};
int ml_model_load(struct ml_test *test, struct ml_options *opt, struct ml_model *model,
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* RE: [PATCH v4 05/12] app/mldev: add ordered inference test case
2023-02-07 15:49 ` [PATCH v4 05/12] app/mldev: add ordered inference test case Srikanth Yalavarthi
@ 2023-02-27 6:11 ` Anup Prabhu
2023-03-09 20:06 ` Thomas Monjalon
1 sibling, 0 replies; 122+ messages in thread
From: Anup Prabhu @ 2023-02-27 6:11 UTC (permalink / raw)
To: Srikanth Yalavarthi, Srikanth Yalavarthi
Cc: dev, Shivah Shankar Shankar Narayan Rao,
Jerin Jacob Kollanukkaran, Prince Takkar, Parijat Shukla
[-- Attachment #1: Type: text/plain, Size: 1635 bytes --]
> -----Original Message-----
> From: Srikanth Yalavarthi <syalavarthi@marvell.com>
> Sent: Tuesday, February 7, 2023 9:20 PM
> To: Srikanth Yalavarthi <syalavarthi@marvell.com>
> Cc: dev@dpdk.org; Shivah Shankar Shankar Narayan Rao
> <sshankarnara@marvell.com>; Jerin Jacob Kollanukkaran
> <jerinj@marvell.com>; Anup Prabhu <aprabhu@marvell.com>; Prince Takkar
> <ptakkar@marvell.com>; Parijat Shukla <pshukla@marvell.com>
> Subject: [PATCH v4 05/12] app/mldev: add ordered inference test case
>
> Added an ordered test case to execute inferences with single or multiple
> models. In this test case inference requests for a model are enqueued after
> completion of all requests for the previous model. Test supports inference
> repetitions.
>
> Operations sequence when testing with N models and R reps,
>
> (load -> start -> (enqueue + dequeue) x R -> stop -> unload) x N
>
> Test case can be executed by selecting "inference_ordered" test and
> repetitions can be specified through "--repetitions" argument.
>
> Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
> ---
> app/test-mldev/meson.build | 2 +
> app/test-mldev/ml_options.c | 73 ++-
> app/test-mldev/ml_options.h | 17 +-
> app/test-mldev/test_inference_common.c | 565
> ++++++++++++++++++++++++ app/test-mldev/test_inference_common.h
> | 65 +++ app/test-mldev/test_inference_ordered.c | 119 +++++
> app/test-mldev/test_model_common.h | 10 +
> 7 files changed, 839 insertions(+), 12 deletions(-) create mode 100644
>
Acked-by: Anup Prabhu <aprabhu@marvell.com>
[-- Attachment #2: winmail.dat --]
[-- Type: application/ms-tnef, Size: 23264 bytes --]
^ permalink raw reply [flat|nested] 122+ messages in thread
* Re: [PATCH v4 05/12] app/mldev: add ordered inference test case
2023-02-07 15:49 ` [PATCH v4 05/12] app/mldev: add ordered inference test case Srikanth Yalavarthi
2023-02-27 6:11 ` Anup Prabhu
@ 2023-03-09 20:06 ` Thomas Monjalon
2023-03-10 8:13 ` [EXT] " Srikanth Yalavarthi
1 sibling, 1 reply; 122+ messages in thread
From: Thomas Monjalon @ 2023-03-09 20:06 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
07/02/2023 16:49, Srikanth Yalavarthi:
> static struct option lgopts[] = {
> - {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0}, {ML_MODELS, 1, 0, 0},
> - {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
> + {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
> + {ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
> + {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
>
> static int
> ml_opts_parse_long(int opt_idx, struct ml_options *opt)
> @@ -133,10 +195,9 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
> unsigned int i;
>
> struct long_opt_parser parsermap[] = {
> - {ML_TEST, ml_parse_test_name},
> - {ML_DEVICE_ID, ml_parse_dev_id},
> - {ML_SOCKET_ID, ml_parse_socket_id},
> - {ML_MODELS, ml_parse_models},
> + {ML_TEST, ml_parse_test_name}, {ML_DEVICE_ID, ml_parse_dev_id},
> + {ML_SOCKET_ID, ml_parse_socket_id}, {ML_MODELS, ml_parse_models},
> + {ML_FILELIST, ml_parse_filelist}, {ML_REPETITIONS, ml_parse_repetitions},
[...]
> /* Options names */
> -#define ML_TEST ("test")
> -#define ML_DEVICE_ID ("dev_id")
> -#define ML_SOCKET_ID ("socket_id")
> -#define ML_MODELS ("models")
> -#define ML_DEBUG ("debug")
> -#define ML_HELP ("help")
> +#define ML_TEST ("test")
> +#define ML_DEVICE_ID ("dev_id")
> +#define ML_SOCKET_ID ("socket_id")
> +#define ML_MODELS ("models")
> +#define ML_FILELIST ("filelist")
> +#define ML_REPETITIONS ("repetitions")
> +#define ML_DEBUG ("debug")
> +#define ML_HELP ("help")
It seems you have issues with alignment.
Please try to do the right alignment in the initial patch, using only spaces.
> + ml_err("error_code = 0x%016lx, error_message = %s\n", error.errcode,
> + error.message);
errcode is 64-bit, you cannot use %lx with 32-bit compiler.
PRIx64 is OK
[...]
> + req->output = RTE_PTR_ADD(req->input, RTE_ALIGN_CEIL(t->model[t->fid].inp_qsize,
> + t->cmn.dev_info.min_align_size));
[...]
> + t->model[fid].output = RTE_PTR_ADD(t->model[fid].input, t->model[fid].inp_dsize);
inp_qsize and inp_dsize are defined as 64-bit fields.
Is it really necessary to have such big sizes?
It cannot compile on 32-bit systems.
The workaround is to cast to uint32_t or uintptr_t.
^ permalink raw reply [flat|nested] 122+ messages in thread* RE: [EXT] Re: [PATCH v4 05/12] app/mldev: add ordered inference test case
2023-03-09 20:06 ` Thomas Monjalon
@ 2023-03-10 8:13 ` Srikanth Yalavarthi
0 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-10 8:13 UTC (permalink / raw)
To: Thomas Monjalon
Cc: dev, Shivah Shankar Shankar Narayan Rao,
Jerin Jacob Kollanukkaran, Anup Prabhu, Prince Takkar,
Parijat Shukla, Srikanth Yalavarthi
> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: 10 March 2023 01:37
> To: Srikanth Yalavarthi <syalavarthi@marvell.com>
> Cc: dev@dpdk.org; Shivah Shankar Shankar Narayan Rao
> <sshankarnara@marvell.com>; Jerin Jacob Kollanukkaran
> <jerinj@marvell.com>; Anup Prabhu <aprabhu@marvell.com>; Prince Takkar
> <ptakkar@marvell.com>; Parijat Shukla <pshukla@marvell.com>; Srikanth
> Yalavarthi <syalavarthi@marvell.com>
> Subject: [EXT] Re: [PATCH v4 05/12] app/mldev: add ordered inference test
> case
>
> External Email
>
> ----------------------------------------------------------------------
> 07/02/2023 16:49, Srikanth Yalavarthi:
> > static struct option lgopts[] = {
> > - {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
> {ML_MODELS, 1, 0, 0},
> > - {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
> > + {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
> > + {ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0,
> 0},
> > + {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
> >
> > static int
> > ml_opts_parse_long(int opt_idx, struct ml_options *opt) @@ -133,10
> > +195,9 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
> > unsigned int i;
> >
> > struct long_opt_parser parsermap[] = {
> > - {ML_TEST, ml_parse_test_name},
> > - {ML_DEVICE_ID, ml_parse_dev_id},
> > - {ML_SOCKET_ID, ml_parse_socket_id},
> > - {ML_MODELS, ml_parse_models},
> > + {ML_TEST, ml_parse_test_name}, {ML_DEVICE_ID,
> ml_parse_dev_id},
> > + {ML_SOCKET_ID, ml_parse_socket_id}, {ML_MODELS,
> ml_parse_models},
> > + {ML_FILELIST, ml_parse_filelist}, {ML_REPETITIONS,
> ml_parse_repetitions},
> [...]
> > /* Options names */
> > -#define ML_TEST ("test")
> > -#define ML_DEVICE_ID ("dev_id")
> > -#define ML_SOCKET_ID ("socket_id")
> > -#define ML_MODELS ("models")
> > -#define ML_DEBUG ("debug")
> > -#define ML_HELP ("help")
> > +#define ML_TEST ("test")
> > +#define ML_DEVICE_ID ("dev_id")
> > +#define ML_SOCKET_ID ("socket_id")
> > +#define ML_MODELS ("models")
> > +#define ML_FILELIST ("filelist")
> > +#define ML_REPETITIONS ("repetitions")
> > +#define ML_DEBUG ("debug")
> > +#define ML_HELP ("help")
>
> It seems you have issues with alignment.
> Please try to do the right alignment in the initial patch, using only spaces.
Alignment in all patches is done using spaces only. Individual patches were alse passing checkpatch.
I think the issue may be due to adding new defines over multiple patches.
>
> > + ml_err("error_code = 0x%016lx, error_message =
> %s\n", error.errcode,
> > + error.message);
>
> errcode is 64-bit, you cannot use %lx with 32-bit compiler.
> PRIx64 is OK
Fixed. Replaced with PRIx64
>
> [...]
> > + req->output = RTE_PTR_ADD(req->input, RTE_ALIGN_CEIL(t-
> >model[t->fid].inp_qsize,
> > + t-
> >cmn.dev_info.min_align_size));
> [...]
> > + t->model[fid].output = RTE_PTR_ADD(t->model[fid].input,
> > +t->model[fid].inp_dsize);
>
> inp_qsize and inp_dsize are defined as 64-bit fields.
> Is it really necessary to have such big sizes?
> It cannot compile on 32-bit systems.
> The workaround is to cast to uint32_t or uintptr_t.
We were defining based on types expected by mldev spec.
Updated the variables to uint32_t. Ran meson build tests including 32b build test. Tests are passing on version 5 patchseries.
>
>
^ permalink raw reply [flat|nested] 122+ messages in thread
* [PATCH v4 06/12] app/mldev: add test case to interleave inferences
2023-02-07 15:49 ` [PATCH v4 00/12] Implementation of mldev test application Srikanth Yalavarthi
` (4 preceding siblings ...)
2023-02-07 15:49 ` [PATCH v4 05/12] app/mldev: add ordered inference test case Srikanth Yalavarthi
@ 2023-02-07 15:49 ` Srikanth Yalavarthi
2023-02-20 6:31 ` Anup Prabhu
2023-03-09 20:15 ` Thomas Monjalon
2023-02-07 15:49 ` [PATCH v4 07/12] app/mldev: enable support for burst inferences Srikanth Yalavarthi
` (5 subsequent siblings)
11 siblings, 2 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-02-07 15:49 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Added test case to interleave inference requests from multiple
models. Interleaving would load and start all models and launch
inference requests for the models using available queue-pairs
Operations sequence when testing with N models and R reps,
(load + start) x N -> (enqueue + dequeue) x N x R ...
-> (stop + unload) x N
Test can be executed by selecting "inference_interleave" test.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/meson.build | 1 +
app/test-mldev/ml_options.c | 3 +-
app/test-mldev/test_inference_common.c | 12 +--
app/test-mldev/test_inference_interleave.c | 118 +++++++++++++++++++++
4 files changed, 127 insertions(+), 7 deletions(-)
create mode 100644 app/test-mldev/test_inference_interleave.c
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
index 475d76d126..41d22fb22c 100644
--- a/app/test-mldev/meson.build
+++ b/app/test-mldev/meson.build
@@ -18,6 +18,7 @@ sources = files(
'test_model_ops.c',
'test_inference_common.c',
'test_inference_ordered.c',
+ 'test_inference_interleave.c',
)
deps += ['mldev']
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 10dad18fff..01ea050ee7 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -162,7 +162,8 @@ ml_dump_test_options(const char *testname)
printf("\n");
}
- if (strcmp(testname, "inference_ordered") == 0) {
+ if ((strcmp(testname, "inference_ordered") == 0) ||
+ (strcmp(testname, "inference_interleave") == 0)) {
printf("\t\t--filelist : comma separated list of model, input and output\n"
"\t\t--repetitions : number of inference repetitions\n");
printf("\n");
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
index ff25c056a0..ae0f4489f7 100644
--- a/app/test-mldev/test_inference_common.c
+++ b/app/test-mldev/test_inference_common.c
@@ -115,7 +115,7 @@ ml_dequeue_single(void *arg)
total_deq += burst_deq;
if (unlikely(op->status == RTE_ML_OP_STATUS_ERROR)) {
rte_ml_op_error_get(t->cmn.opt->dev_id, op, &error);
- ml_err("error_code = 0x%016lx, error_message = %s\n", error.errcode,
+ ml_err("error_code = 0x%" PRIx64 ", error_message = %s\n", error.errcode,
error.message);
}
req = (struct ml_request *)op->user_ptr;
@@ -334,10 +334,10 @@ ml_request_initialize(struct rte_mempool *mp, void *opaque, void *obj, unsigned
RTE_SET_USED(mp);
RTE_SET_USED(obj_idx);
- req->input = RTE_PTR_ADD(
- obj, RTE_ALIGN_CEIL(sizeof(struct ml_request), t->cmn.dev_info.min_align_size));
- req->output = RTE_PTR_ADD(req->input, RTE_ALIGN_CEIL(t->model[t->fid].inp_qsize,
- t->cmn.dev_info.min_align_size));
+ req->input = (uint8_t *)obj +
+ RTE_ALIGN_CEIL(sizeof(struct ml_request), t->cmn.dev_info.min_align_size);
+ req->output = req->input +
+ RTE_ALIGN_CEIL(t->model[t->fid].inp_qsize, t->cmn.dev_info.min_align_size);
req->niters = 0;
/* quantize data */
@@ -387,7 +387,7 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, uint16_t
}
t->model[fid].input = mz->addr;
- t->model[fid].output = RTE_PTR_ADD(t->model[fid].input, t->model[fid].inp_dsize);
+ t->model[fid].output = t->model[fid].input + t->model[fid].inp_dsize;
/* load input file */
fp = fopen(opt->filelist[fid].input, "r");
diff --git a/app/test-mldev/test_inference_interleave.c b/app/test-mldev/test_inference_interleave.c
new file mode 100644
index 0000000000..eca67f0e62
--- /dev/null
+++ b/app/test-mldev/test_inference_interleave.c
@@ -0,0 +1,118 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <stdint.h>
+
+#include <rte_common.h>
+#include <rte_launch.h>
+
+#include "ml_common.h"
+#include "ml_test.h"
+#include "test_inference_common.h"
+#include "test_model_common.h"
+
+static int
+test_inference_interleave_driver(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t;
+ uint16_t fid = 0;
+ int ret = 0;
+
+ t = ml_test_priv(test);
+
+ ret = ml_inference_mldev_setup(test, opt);
+ if (ret != 0)
+ return ret;
+
+ ret = ml_inference_mem_setup(test, opt);
+ if (ret != 0)
+ return ret;
+
+ /* load and start all models */
+ for (fid = 0; fid < opt->nb_filelist; fid++) {
+ ret = ml_model_load(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_start(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_inference_iomem_setup(test, opt, fid);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* launch inference requests */
+ ret = ml_inference_launch_cores(test, opt, 0, opt->nb_filelist - 1);
+ if (ret != 0) {
+ ml_err("failed to launch cores");
+ goto error;
+ }
+
+ rte_eal_mp_wait_lcore();
+
+ /* stop and unload all models */
+ for (fid = 0; fid < opt->nb_filelist; fid++) {
+ ret = ml_inference_result(test, opt, fid);
+ if (ret != ML_TEST_SUCCESS)
+ goto error;
+
+ ml_inference_iomem_destroy(test, opt, fid);
+
+ ret = ml_model_stop(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_unload(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+ }
+
+ ml_inference_mem_destroy(test, opt);
+
+ ret = ml_inference_mldev_destroy(test, opt);
+ if (ret != 0)
+ return ret;
+
+ t->cmn.result = ML_TEST_SUCCESS;
+
+ return 0;
+
+error:
+ ml_inference_mem_destroy(test, opt);
+ for (fid = 0; fid < opt->nb_filelist; fid++) {
+ ml_inference_iomem_destroy(test, opt, fid);
+ ml_model_stop(test, opt, &t->model[fid], fid);
+ ml_model_unload(test, opt, &t->model[fid], fid);
+ }
+
+ t->cmn.result = ML_TEST_FAILED;
+
+ return ret;
+}
+
+static int
+test_inference_interleave_result(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+
+ return t->cmn.result;
+}
+
+static const struct ml_test_ops inference_interleave = {
+ .cap_check = test_inference_cap_check,
+ .opt_check = test_inference_opt_check,
+ .opt_dump = test_inference_opt_dump,
+ .test_setup = test_inference_setup,
+ .test_destroy = test_inference_destroy,
+ .test_driver = test_inference_interleave_driver,
+ .test_result = test_inference_interleave_result,
+};
+
+ML_TEST_REGISTER(inference_interleave);
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* RE: [PATCH v4 06/12] app/mldev: add test case to interleave inferences
2023-02-07 15:49 ` [PATCH v4 06/12] app/mldev: add test case to interleave inferences Srikanth Yalavarthi
@ 2023-02-20 6:31 ` Anup Prabhu
2023-03-09 20:15 ` Thomas Monjalon
1 sibling, 0 replies; 122+ messages in thread
From: Anup Prabhu @ 2023-02-20 6:31 UTC (permalink / raw)
To: Srikanth Yalavarthi, Srikanth Yalavarthi
Cc: dev, Shivah Shankar Shankar Narayan Rao,
Jerin Jacob Kollanukkaran, Prince Takkar, Parijat Shukla
[-- Attachment #1: Type: text/plain, Size: 1387 bytes --]
> -----Original Message-----
> From: Srikanth Yalavarthi <syalavarthi@marvell.com>
> Sent: Tuesday, February 7, 2023 9:20 PM
> To: Srikanth Yalavarthi <syalavarthi@marvell.com>
> Cc: dev@dpdk.org; Shivah Shankar Shankar Narayan Rao
> <sshankarnara@marvell.com>; Jerin Jacob Kollanukkaran
> <jerinj@marvell.com>; Anup Prabhu <aprabhu@marvell.com>; Prince Takkar
> <ptakkar@marvell.com>; Parijat Shukla <pshukla@marvell.com>
> Subject: [PATCH v4 06/12] app/mldev: add test case to interleave inferences
>
> Added test case to interleave inference requests from multiple models.
> Interleaving would load and start all models and launch inference requests
> for the models using available queue-pairs
>
> Operations sequence when testing with N models and R reps,
>
> (load + start) x N -> (enqueue + dequeue) x N x R ...
> -> (stop + unload) x N
>
> Test can be executed by selecting "inference_interleave" test.
>
> Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
> ---
> app/test-mldev/meson.build | 1 +
> app/test-mldev/ml_options.c | 3 +-
> app/test-mldev/test_inference_common.c | 12 +--
> app/test-mldev/test_inference_interleave.c | 118
> +++++++++++++++++++++
> 4 files changed, 127 insertions(+), 7 deletions(-) create mode 100644
>
Acked-by: Anup Prabhu <aprabhu@marvell.com>
[-- Attachment #2: winmail.dat --]
[-- Type: application/ms-tnef, Size: 23187 bytes --]
^ permalink raw reply [flat|nested] 122+ messages in thread
* Re: [PATCH v4 06/12] app/mldev: add test case to interleave inferences
2023-02-07 15:49 ` [PATCH v4 06/12] app/mldev: add test case to interleave inferences Srikanth Yalavarthi
2023-02-20 6:31 ` Anup Prabhu
@ 2023-03-09 20:15 ` Thomas Monjalon
2023-03-10 8:14 ` [EXT] " Srikanth Yalavarthi
1 sibling, 1 reply; 122+ messages in thread
From: Thomas Monjalon @ 2023-03-09 20:15 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
07/02/2023 16:49, Srikanth Yalavarthi:
> - ml_err("error_code = 0x%016lx, error_message = %s\n", error.errcode,
> + ml_err("error_code = 0x%" PRIx64 ", error_message = %s\n", error.errcode,
> error.message);
> }
> req = (struct ml_request *)op->user_ptr;
> @@ -334,10 +334,10 @@ ml_request_initialize(struct rte_mempool *mp, void *opaque, void *obj, unsigned
> RTE_SET_USED(mp);
> RTE_SET_USED(obj_idx);
>
> - req->input = RTE_PTR_ADD(
> - obj, RTE_ALIGN_CEIL(sizeof(struct ml_request), t->cmn.dev_info.min_align_size));
> - req->output = RTE_PTR_ADD(req->input, RTE_ALIGN_CEIL(t->model[t->fid].inp_qsize,
> - t->cmn.dev_info.min_align_size));
> + req->input = (uint8_t *)obj +
> + RTE_ALIGN_CEIL(sizeof(struct ml_request), t->cmn.dev_info.min_align_size);
> + req->output = req->input +
> + RTE_ALIGN_CEIL(t->model[t->fid].inp_qsize, t->cmn.dev_info.min_align_size);
> req->niters = 0;
>
> /* quantize data */
> @@ -387,7 +387,7 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, uint16_t
> }
>
> t->model[fid].input = mz->addr;
> - t->model[fid].output = RTE_PTR_ADD(t->model[fid].input, t->model[fid].inp_dsize);
> + t->model[fid].output = t->model[fid].input + t->model[fid].inp_dsize;
So you are fixing your previous patch in an unrelated patch?
Really I cannot merge this mess in this state.
Please provide cleaner patches in v5.
^ permalink raw reply [flat|nested] 122+ messages in thread* RE: [EXT] Re: [PATCH v4 06/12] app/mldev: add test case to interleave inferences
2023-03-09 20:15 ` Thomas Monjalon
@ 2023-03-10 8:14 ` Srikanth Yalavarthi
0 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-10 8:14 UTC (permalink / raw)
To: Thomas Monjalon
Cc: dev, Shivah Shankar Shankar Narayan Rao,
Jerin Jacob Kollanukkaran, Anup Prabhu, Prince Takkar,
Parijat Shukla, Srikanth Yalavarthi
> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: 10 March 2023 01:46
> To: Srikanth Yalavarthi <syalavarthi@marvell.com>
> Cc: dev@dpdk.org; Shivah Shankar Shankar Narayan Rao
> <sshankarnara@marvell.com>; Jerin Jacob Kollanukkaran
> <jerinj@marvell.com>; Anup Prabhu <aprabhu@marvell.com>; Prince Takkar
> <ptakkar@marvell.com>; Parijat Shukla <pshukla@marvell.com>; Srikanth
> Yalavarthi <syalavarthi@marvell.com>
> Subject: [EXT] Re: [PATCH v4 06/12] app/mldev: add test case to interleave
> inferences
>
> External Email
>
> ----------------------------------------------------------------------
> 07/02/2023 16:49, Srikanth Yalavarthi:
> > - ml_err("error_code = 0x%016lx, error_message =
> %s\n", error.errcode,
> > + ml_err("error_code = 0x%" PRIx64 ", error_message
> = %s\n",
> > +error.errcode,
> > error.message);
> > }
> > req = (struct ml_request *)op->user_ptr; @@ -334,10
> +334,10 @@
> > ml_request_initialize(struct rte_mempool *mp, void *opaque, void *obj,
> unsigned
> > RTE_SET_USED(mp);
> > RTE_SET_USED(obj_idx);
> >
> > - req->input = RTE_PTR_ADD(
> > - obj, RTE_ALIGN_CEIL(sizeof(struct ml_request), t-
> >cmn.dev_info.min_align_size));
> > - req->output = RTE_PTR_ADD(req->input, RTE_ALIGN_CEIL(t-
> >model[t->fid].inp_qsize,
> > - t-
> >cmn.dev_info.min_align_size));
> > + req->input = (uint8_t *)obj +
> > + RTE_ALIGN_CEIL(sizeof(struct ml_request), t-
> >cmn.dev_info.min_align_size);
> > + req->output = req->input +
> > + RTE_ALIGN_CEIL(t->model[t->fid].inp_qsize,
> > +t->cmn.dev_info.min_align_size);
> > req->niters = 0;
> >
> > /* quantize data */
> > @@ -387,7 +387,7 @@ ml_inference_iomem_setup(struct ml_test *test,
> struct ml_options *opt, uint16_t
> > }
> >
> > t->model[fid].input = mz->addr;
> > - t->model[fid].output = RTE_PTR_ADD(t->model[fid].input, t-
> >model[fid].inp_dsize);
> > + t->model[fid].output = t->model[fid].input +
> > +t->model[fid].inp_dsize;
>
> So you are fixing your previous patch in an unrelated patch?
>
This is because of the datatypes being. Fixed all issues in version 5 patchseries. Patches are clean now.
> Really I cannot merge this mess in this state.
> Please provide cleaner patches in v5.
Done. Pushed patchseries v5.
>
^ permalink raw reply [flat|nested] 122+ messages in thread
* [PATCH v4 07/12] app/mldev: enable support for burst inferences
2023-02-07 15:49 ` [PATCH v4 00/12] Implementation of mldev test application Srikanth Yalavarthi
` (5 preceding siblings ...)
2023-02-07 15:49 ` [PATCH v4 06/12] app/mldev: add test case to interleave inferences Srikanth Yalavarthi
@ 2023-02-07 15:49 ` Srikanth Yalavarthi
2023-02-20 10:11 ` Anup Prabhu
2023-02-07 15:49 ` [PATCH v4 08/12] app/mldev: enable support for queue pairs and size Srikanth Yalavarthi
` (4 subsequent siblings)
11 siblings, 1 reply; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-02-07 15:49 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Added 'burst_size' support for inference tests. Burst size
controls the number of inference requests handled during
the burst enqueue and dequeue operations of the test case.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/ml_options.c | 26 ++--
app/test-mldev/ml_options.h | 2 +
app/test-mldev/test_inference_common.c | 159 ++++++++++++++++++++++++-
app/test-mldev/test_inference_common.h | 4 +
4 files changed, 181 insertions(+), 10 deletions(-)
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 01ea050ee7..1990939200 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -30,6 +30,7 @@ ml_options_default(struct ml_options *opt)
opt->socket_id = SOCKET_ID_ANY;
opt->nb_filelist = 0;
opt->repetitions = 1;
+ opt->burst_size = 1;
opt->debug = false;
}
@@ -151,6 +152,12 @@ ml_parse_repetitions(struct ml_options *opt, const char *arg)
return parser_read_uint64(&opt->repetitions, arg);
}
+static int
+ml_parse_burst_size(struct ml_options *opt, const char *arg)
+{
+ return parser_read_uint16(&opt->burst_size, arg);
+}
+
static void
ml_dump_test_options(const char *testname)
{
@@ -165,7 +172,8 @@ ml_dump_test_options(const char *testname)
if ((strcmp(testname, "inference_ordered") == 0) ||
(strcmp(testname, "inference_interleave") == 0)) {
printf("\t\t--filelist : comma separated list of model, input and output\n"
- "\t\t--repetitions : number of inference repetitions\n");
+ "\t\t--repetitions : number of inference repetitions\n"
+ "\t\t--burst_size : inference burst size\n");
printf("\n");
}
}
@@ -185,10 +193,11 @@ print_usage(char *program)
ml_test_dump_names(ml_dump_test_options);
}
-static struct option lgopts[] = {
- {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
- {ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
- {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
+static struct option lgopts[] = {{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0},
+ {ML_SOCKET_ID, 1, 0, 0}, {ML_MODELS, 1, 0, 0},
+ {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
+ {ML_BURST_SIZE, 1, 0, 0}, {ML_DEBUG, 0, 0, 0},
+ {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
static int
ml_opts_parse_long(int opt_idx, struct ml_options *opt)
@@ -196,9 +205,10 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
unsigned int i;
struct long_opt_parser parsermap[] = {
- {ML_TEST, ml_parse_test_name}, {ML_DEVICE_ID, ml_parse_dev_id},
- {ML_SOCKET_ID, ml_parse_socket_id}, {ML_MODELS, ml_parse_models},
- {ML_FILELIST, ml_parse_filelist}, {ML_REPETITIONS, ml_parse_repetitions},
+ {ML_TEST, ml_parse_test_name}, {ML_DEVICE_ID, ml_parse_dev_id},
+ {ML_SOCKET_ID, ml_parse_socket_id}, {ML_MODELS, ml_parse_models},
+ {ML_FILELIST, ml_parse_filelist}, {ML_REPETITIONS, ml_parse_repetitions},
+ {ML_BURST_SIZE, ml_parse_burst_size},
};
for (i = 0; i < RTE_DIM(parsermap); i++) {
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index ad8aee5964..305b39629a 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -19,6 +19,7 @@
#define ML_MODELS ("models")
#define ML_FILELIST ("filelist")
#define ML_REPETITIONS ("repetitions")
+#define ML_BURST_SIZE ("burst_size")
#define ML_DEBUG ("debug")
#define ML_HELP ("help")
@@ -35,6 +36,7 @@ struct ml_options {
struct ml_filelist filelist[ML_TEST_MAX_MODELS];
uint8_t nb_filelist;
uint64_t repetitions;
+ uint16_t burst_size;
bool debug;
};
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
index ae0f4489f7..95d7f8d17d 100644
--- a/app/test-mldev/test_inference_common.c
+++ b/app/test-mldev/test_inference_common.c
@@ -129,6 +129,131 @@ ml_dequeue_single(void *arg)
return 0;
}
+/* Enqueue inference requests with burst size greater than 1 */
+static int
+ml_enqueue_burst(void *arg)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)arg);
+ struct ml_core_args *args;
+ uint16_t ops_count;
+ uint64_t model_enq;
+ uint16_t burst_enq;
+ uint32_t lcore_id;
+ uint16_t pending;
+ uint16_t idx;
+ uint16_t fid;
+ uint16_t i;
+ int ret;
+
+ lcore_id = rte_lcore_id();
+ args = &t->args[lcore_id];
+ model_enq = 0;
+
+ if (args->nb_reqs == 0)
+ return 0;
+
+next_rep:
+ fid = args->start_fid;
+
+next_model:
+ ops_count = RTE_MIN(t->cmn.opt->burst_size, args->nb_reqs - model_enq);
+ ret = rte_mempool_get_bulk(t->op_pool, (void **)args->enq_ops, ops_count);
+ if (ret != 0)
+ goto next_model;
+
+retry:
+ ret = rte_mempool_get_bulk(t->model[fid].io_pool, (void **)args->reqs, ops_count);
+ if (ret != 0)
+ goto retry;
+
+ for (i = 0; i < ops_count; i++) {
+ args->enq_ops[i]->model_id = t->model[fid].id;
+ args->enq_ops[i]->nb_batches = t->model[fid].info.batch_size;
+ args->enq_ops[i]->mempool = t->op_pool;
+
+ args->enq_ops[i]->input.addr = args->reqs[i]->input;
+ args->enq_ops[i]->input.length = t->model[fid].inp_qsize;
+ args->enq_ops[i]->input.next = NULL;
+
+ args->enq_ops[i]->output.addr = args->reqs[i]->output;
+ args->enq_ops[i]->output.length = t->model[fid].out_qsize;
+ args->enq_ops[i]->output.next = NULL;
+
+ args->enq_ops[i]->user_ptr = args->reqs[i];
+ args->reqs[i]->niters++;
+ args->reqs[i]->fid = fid;
+ }
+
+ idx = 0;
+ pending = ops_count;
+
+enqueue_reqs:
+ burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, 0, &args->enq_ops[idx], pending);
+ pending = pending - burst_enq;
+
+ if (pending > 0) {
+ idx = idx + burst_enq;
+ goto enqueue_reqs;
+ }
+
+ fid++;
+ if (fid <= args->end_fid)
+ goto next_model;
+
+ model_enq = model_enq + ops_count;
+ if (model_enq < args->nb_reqs)
+ goto next_rep;
+
+ return 0;
+}
+
+/* Dequeue inference requests with burst size greater than 1 */
+static int
+ml_dequeue_burst(void *arg)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)arg);
+ struct rte_ml_op_error error;
+ struct ml_core_args *args;
+ struct ml_request *req;
+ uint64_t total_deq = 0;
+ uint16_t burst_deq = 0;
+ uint8_t nb_filelist;
+ uint32_t lcore_id;
+ uint32_t i;
+
+ lcore_id = rte_lcore_id();
+ args = &t->args[lcore_id];
+ nb_filelist = args->end_fid - args->start_fid + 1;
+
+ if (args->nb_reqs == 0)
+ return 0;
+
+dequeue_burst:
+ burst_deq =
+ rte_ml_dequeue_burst(t->cmn.opt->dev_id, 0, args->deq_ops, t->cmn.opt->burst_size);
+
+ if (likely(burst_deq > 0)) {
+ total_deq += burst_deq;
+
+ for (i = 0; i < burst_deq; i++) {
+ if (unlikely(args->deq_ops[i]->status == RTE_ML_OP_STATUS_ERROR)) {
+ rte_ml_op_error_get(t->cmn.opt->dev_id, args->deq_ops[i], &error);
+ ml_err("error_code = 0x%" PRIx64 ", error_message = %s\n",
+ error.errcode, error.message);
+ }
+ req = (struct ml_request *)args->deq_ops[i]->user_ptr;
+ if (req != NULL)
+ rte_mempool_put(t->model[req->fid].io_pool, req);
+ }
+ rte_mempool_put_bulk(t->op_pool, (void *)args->deq_ops, burst_deq);
+ }
+
+ if (total_deq < args->nb_reqs * nb_filelist)
+ goto dequeue_burst;
+
+ return 0;
+}
+
bool
test_inference_cap_check(struct ml_options *opt)
{
@@ -178,6 +303,17 @@ test_inference_opt_check(struct ml_options *opt)
return -EINVAL;
}
+ if (opt->burst_size == 0) {
+ ml_err("Invalid option, burst_size = %u\n", opt->burst_size);
+ return -EINVAL;
+ }
+
+ if (opt->burst_size > ML_TEST_MAX_POOL_SIZE) {
+ ml_err("Invalid option, burst_size = %u (> max supported = %d)\n", opt->burst_size,
+ ML_TEST_MAX_POOL_SIZE);
+ return -EINVAL;
+ }
+
/* check number of available lcores. */
if (rte_lcore_count() < 3) {
ml_err("Insufficient lcores = %u\n", rte_lcore_count());
@@ -198,6 +334,7 @@ test_inference_opt_dump(struct ml_options *opt)
/* dump test opts */
ml_dump("repetitions", "%" PRIu64, opt->repetitions);
+ ml_dump("burst_size", "%u", opt->burst_size);
ml_dump_begin("filelist");
for (i = 0; i < opt->nb_filelist; i++) {
@@ -213,6 +350,7 @@ test_inference_setup(struct ml_test *test, struct ml_options *opt)
{
struct test_inference *t;
void *test_inference;
+ uint32_t lcore_id;
int ret = 0;
uint32_t i;
@@ -237,13 +375,30 @@ test_inference_setup(struct ml_test *test, struct ml_options *opt)
goto error;
}
- t->enqueue = ml_enqueue_single;
- t->dequeue = ml_dequeue_single;
+ if (opt->burst_size == 1) {
+ t->enqueue = ml_enqueue_single;
+ t->dequeue = ml_dequeue_single;
+ } else {
+ t->enqueue = ml_enqueue_burst;
+ t->dequeue = ml_dequeue_burst;
+ }
/* set model initial state */
for (i = 0; i < opt->nb_filelist; i++)
t->model[i].state = MODEL_INITIAL;
+ for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) {
+ t->args[lcore_id].enq_ops = rte_zmalloc_socket(
+ "ml_test_enq_ops", opt->burst_size * sizeof(struct rte_ml_op *),
+ RTE_CACHE_LINE_SIZE, opt->socket_id);
+ t->args[lcore_id].deq_ops = rte_zmalloc_socket(
+ "ml_test_deq_ops", opt->burst_size * sizeof(struct rte_ml_op *),
+ RTE_CACHE_LINE_SIZE, opt->socket_id);
+ t->args[lcore_id].reqs = rte_zmalloc_socket(
+ "ml_test_requests", opt->burst_size * sizeof(struct ml_request *),
+ RTE_CACHE_LINE_SIZE, opt->socket_id);
+ }
+
return 0;
error:
diff --git a/app/test-mldev/test_inference_common.h b/app/test-mldev/test_inference_common.h
index 99baad5bfd..7bbd68aefa 100644
--- a/app/test-mldev/test_inference_common.h
+++ b/app/test-mldev/test_inference_common.h
@@ -27,6 +27,10 @@ struct ml_core_args {
uint64_t nb_reqs;
uint16_t start_fid;
uint16_t end_fid;
+
+ struct rte_ml_op **enq_ops;
+ struct rte_ml_op **deq_ops;
+ struct ml_request **reqs;
};
struct test_inference {
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* RE: [PATCH v4 07/12] app/mldev: enable support for burst inferences
2023-02-07 15:49 ` [PATCH v4 07/12] app/mldev: enable support for burst inferences Srikanth Yalavarthi
@ 2023-02-20 10:11 ` Anup Prabhu
0 siblings, 0 replies; 122+ messages in thread
From: Anup Prabhu @ 2023-02-20 10:11 UTC (permalink / raw)
To: Srikanth Yalavarthi, Srikanth Yalavarthi
Cc: dev, Shivah Shankar Shankar Narayan Rao,
Jerin Jacob Kollanukkaran, Prince Takkar, Parijat Shukla
[-- Attachment #1: Type: text/plain, Size: 1113 bytes --]
> -----Original Message-----
> From: Srikanth Yalavarthi <syalavarthi@marvell.com>
> Sent: Tuesday, February 7, 2023 9:20 PM
> To: Srikanth Yalavarthi <syalavarthi@marvell.com>
> Cc: dev@dpdk.org; Shivah Shankar Shankar Narayan Rao
> <sshankarnara@marvell.com>; Jerin Jacob Kollanukkaran
> <jerinj@marvell.com>; Anup Prabhu <aprabhu@marvell.com>; Prince Takkar
> <ptakkar@marvell.com>; Parijat Shukla <pshukla@marvell.com>
> Subject: [PATCH v4 07/12] app/mldev: enable support for burst inferences
>
> Added 'burst_size' support for inference tests. Burst size controls the
> number of inference requests handled during the burst enqueue and
> dequeue operations of the test case.
>
> Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
> ---
> app/test-mldev/ml_options.c | 26 ++--
> app/test-mldev/ml_options.h | 2 +
> app/test-mldev/test_inference_common.c | 159
> ++++++++++++++++++++++++-
> app/test-mldev/test_inference_common.h | 4 +
> 4 files changed, 181 insertions(+), 10 deletions(-)
>
Acked-by: Anup Prabhu <aprabhu@marvell.com>
[-- Attachment #2: winmail.dat --]
[-- Type: application/ms-tnef, Size: 23020 bytes --]
^ permalink raw reply [flat|nested] 122+ messages in thread
* [PATCH v4 08/12] app/mldev: enable support for queue pairs and size
2023-02-07 15:49 ` [PATCH v4 00/12] Implementation of mldev test application Srikanth Yalavarthi
` (6 preceding siblings ...)
2023-02-07 15:49 ` [PATCH v4 07/12] app/mldev: enable support for burst inferences Srikanth Yalavarthi
@ 2023-02-07 15:49 ` Srikanth Yalavarthi
2023-03-02 8:15 ` Anup Prabhu
2023-02-07 15:49 ` [PATCH v4 09/12] app/mldev: enable support for inference batches Srikanth Yalavarthi
` (3 subsequent siblings)
11 siblings, 1 reply; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-02-07 15:49 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Added support to create multiple queue-pairs per device to
enqueue and dequeue inference requests. Number of queue pairs
to be created can be specified through "--queue_pairs" option.
Support is also enabled to control the number of descriptors
per each queue pair through "--queue_size" option. Inference
requests for a model are distributed across all available
queue-pairs.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/ml_options.c | 40 ++++++++++---
app/test-mldev/ml_options.h | 4 ++
app/test-mldev/test_common.c | 2 +-
app/test-mldev/test_inference_common.c | 79 +++++++++++++++++++++-----
app/test-mldev/test_inference_common.h | 1 +
5 files changed, 102 insertions(+), 24 deletions(-)
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 1990939200..d5182a1701 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -31,6 +31,8 @@ ml_options_default(struct ml_options *opt)
opt->nb_filelist = 0;
opt->repetitions = 1;
opt->burst_size = 1;
+ opt->queue_pairs = 1;
+ opt->queue_size = 1;
opt->debug = false;
}
@@ -158,11 +160,30 @@ ml_parse_burst_size(struct ml_options *opt, const char *arg)
return parser_read_uint16(&opt->burst_size, arg);
}
+static int
+ml_parse_queue_pairs(struct ml_options *opt, const char *arg)
+{
+ int ret;
+
+ ret = parser_read_uint16(&opt->queue_pairs, arg);
+
+ return ret;
+}
+
+static int
+ml_parse_queue_size(struct ml_options *opt, const char *arg)
+{
+ return parser_read_uint16(&opt->queue_size, arg);
+}
+
static void
ml_dump_test_options(const char *testname)
{
- if (strcmp(testname, "device_ops") == 0)
+ if (strcmp(testname, "device_ops") == 0) {
+ printf("\t\t--queue_pairs : number of queue pairs to create\n"
+ "\t\t--queue_size : size fo queue-pair\n");
printf("\n");
+ }
if (strcmp(testname, "model_ops") == 0) {
printf("\t\t--models : comma separated list of models\n");
@@ -173,7 +194,9 @@ ml_dump_test_options(const char *testname)
(strcmp(testname, "inference_interleave") == 0)) {
printf("\t\t--filelist : comma separated list of model, input and output\n"
"\t\t--repetitions : number of inference repetitions\n"
- "\t\t--burst_size : inference burst size\n");
+ "\t\t--burst_size : inference burst size\n"
+ "\t\t--queue_pairs : number of queue pairs to create\n"
+ "\t\t--queue_size : size fo queue-pair\n");
printf("\n");
}
}
@@ -193,11 +216,11 @@ print_usage(char *program)
ml_test_dump_names(ml_dump_test_options);
}
-static struct option lgopts[] = {{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0},
- {ML_SOCKET_ID, 1, 0, 0}, {ML_MODELS, 1, 0, 0},
- {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
- {ML_BURST_SIZE, 1, 0, 0}, {ML_DEBUG, 0, 0, 0},
- {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
+static struct option lgopts[] = {
+ {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
+ {ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
+ {ML_BURST_SIZE, 1, 0, 0}, {ML_QUEUE_PAIRS, 1, 0, 0}, {ML_QUEUE_SIZE, 1, 0, 0},
+ {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
static int
ml_opts_parse_long(int opt_idx, struct ml_options *opt)
@@ -208,7 +231,8 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
{ML_TEST, ml_parse_test_name}, {ML_DEVICE_ID, ml_parse_dev_id},
{ML_SOCKET_ID, ml_parse_socket_id}, {ML_MODELS, ml_parse_models},
{ML_FILELIST, ml_parse_filelist}, {ML_REPETITIONS, ml_parse_repetitions},
- {ML_BURST_SIZE, ml_parse_burst_size},
+ {ML_BURST_SIZE, ml_parse_burst_size}, {ML_QUEUE_PAIRS, ml_parse_queue_pairs},
+ {ML_QUEUE_SIZE, ml_parse_queue_size},
};
for (i = 0; i < RTE_DIM(parsermap); i++) {
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index 305b39629a..6bfef1b979 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -20,6 +20,8 @@
#define ML_FILELIST ("filelist")
#define ML_REPETITIONS ("repetitions")
#define ML_BURST_SIZE ("burst_size")
+#define ML_QUEUE_PAIRS ("queue_pairs")
+#define ML_QUEUE_SIZE ("queue_size")
#define ML_DEBUG ("debug")
#define ML_HELP ("help")
@@ -37,6 +39,8 @@ struct ml_options {
uint8_t nb_filelist;
uint64_t repetitions;
uint16_t burst_size;
+ uint16_t queue_pairs;
+ uint16_t queue_size;
bool debug;
};
diff --git a/app/test-mldev/test_common.c b/app/test-mldev/test_common.c
index 4a9a4ec4a2..2cb64f207d 100644
--- a/app/test-mldev/test_common.c
+++ b/app/test-mldev/test_common.c
@@ -78,7 +78,7 @@ ml_test_device_configure(struct ml_test *test, struct ml_options *opt)
/* configure device */
dev_config.socket_id = opt->socket_id;
dev_config.nb_models = t->dev_info.max_models;
- dev_config.nb_queue_pairs = t->dev_info.max_queue_pairs;
+ dev_config.nb_queue_pairs = opt->queue_pairs;
ret = rte_ml_dev_configure(opt->dev_id, &dev_config);
if (ret != 0) {
ml_err("Failed to configure ml device, dev_id = %d\n", opt->dev_id);
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
index 95d7f8d17d..d4a00a5399 100644
--- a/app/test-mldev/test_inference_common.c
+++ b/app/test-mldev/test_inference_common.c
@@ -72,7 +72,7 @@ ml_enqueue_single(void *arg)
req->fid = fid;
enqueue_req:
- burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, 0, &op, 1);
+ burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, args->qp_id, &op, 1);
if (burst_enq == 0)
goto enqueue_req;
@@ -109,7 +109,7 @@ ml_dequeue_single(void *arg)
return 0;
dequeue_req:
- burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, 0, &op, 1);
+ burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, args->qp_id, &op, 1);
if (likely(burst_deq == 1)) {
total_deq += burst_deq;
@@ -188,7 +188,8 @@ ml_enqueue_burst(void *arg)
pending = ops_count;
enqueue_reqs:
- burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, 0, &args->enq_ops[idx], pending);
+ burst_enq =
+ rte_ml_enqueue_burst(t->cmn.opt->dev_id, args->qp_id, &args->enq_ops[idx], pending);
pending = pending - burst_enq;
if (pending > 0) {
@@ -229,8 +230,8 @@ ml_dequeue_burst(void *arg)
return 0;
dequeue_burst:
- burst_deq =
- rte_ml_dequeue_burst(t->cmn.opt->dev_id, 0, args->deq_ops, t->cmn.opt->burst_size);
+ burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, args->qp_id, args->deq_ops,
+ t->cmn.opt->burst_size);
if (likely(burst_deq > 0)) {
total_deq += burst_deq;
@@ -263,6 +264,19 @@ test_inference_cap_check(struct ml_options *opt)
return false;
rte_ml_dev_info_get(opt->dev_id, &dev_info);
+
+ if (opt->queue_pairs > dev_info.max_queue_pairs) {
+ ml_err("Insufficient capabilities: queue_pairs = %u, max_queue_pairs = %u",
+ opt->queue_pairs, dev_info.max_queue_pairs);
+ return false;
+ }
+
+ if (opt->queue_size > dev_info.max_desc) {
+ ml_err("Insufficient capabilities: queue_size = %u, max_desc = %u", opt->queue_size,
+ dev_info.max_desc);
+ return false;
+ }
+
if (opt->nb_filelist > dev_info.max_models) {
ml_err("Insufficient capabilities: Filelist count exceeded device limit, count = %u (max limit = %u)",
opt->nb_filelist, dev_info.max_models);
@@ -314,10 +328,21 @@ test_inference_opt_check(struct ml_options *opt)
return -EINVAL;
}
+ if (opt->queue_pairs == 0) {
+ ml_err("Invalid option, queue_pairs = %u\n", opt->queue_pairs);
+ return -EINVAL;
+ }
+
+ if (opt->queue_size == 0) {
+ ml_err("Invalid option, queue_size = %u\n", opt->queue_size);
+ return -EINVAL;
+ }
+
/* check number of available lcores. */
- if (rte_lcore_count() < 3) {
+ if (rte_lcore_count() < (uint32_t)(opt->queue_pairs * 2 + 1)) {
ml_err("Insufficient lcores = %u\n", rte_lcore_count());
- ml_err("Minimum lcores required to create %u queue-pairs = %u\n", 1, 3);
+ ml_err("Minimum lcores required to create %u queue-pairs = %u\n", opt->queue_pairs,
+ (opt->queue_pairs * 2 + 1));
return -EINVAL;
}
@@ -335,6 +360,8 @@ test_inference_opt_dump(struct ml_options *opt)
/* dump test opts */
ml_dump("repetitions", "%" PRIu64, opt->repetitions);
ml_dump("burst_size", "%u", opt->burst_size);
+ ml_dump("queue_pairs", "%u", opt->queue_pairs);
+ ml_dump("queue_size", "%u", opt->queue_size);
ml_dump_begin("filelist");
for (i = 0; i < opt->nb_filelist; i++) {
@@ -425,23 +452,31 @@ ml_inference_mldev_setup(struct ml_test *test, struct ml_options *opt)
{
struct rte_ml_dev_qp_conf qp_conf;
struct test_inference *t;
+ uint16_t qp_id;
int ret;
t = ml_test_priv(test);
+ RTE_SET_USED(t);
+
ret = ml_test_device_configure(test, opt);
if (ret != 0)
return ret;
/* setup queue pairs */
- qp_conf.nb_desc = t->cmn.dev_info.max_desc;
+ qp_conf.nb_desc = opt->queue_size;
qp_conf.cb = NULL;
- ret = rte_ml_dev_queue_pair_setup(opt->dev_id, 0, &qp_conf, opt->socket_id);
- if (ret != 0) {
- ml_err("Failed to setup ml device queue-pair, dev_id = %d, qp_id = %u\n",
- opt->dev_id, 0);
- goto error;
+ for (qp_id = 0; qp_id < opt->queue_pairs; qp_id++) {
+ qp_conf.nb_desc = opt->queue_size;
+ qp_conf.cb = NULL;
+
+ ret = rte_ml_dev_queue_pair_setup(opt->dev_id, qp_id, &qp_conf, opt->socket_id);
+ if (ret != 0) {
+ ml_err("Failed to setup ml device queue-pair, dev_id = %d, qp_id = %u\n",
+ opt->dev_id, qp_id);
+ return ret;
+ }
}
ret = ml_test_device_start(test, opt);
@@ -697,14 +732,28 @@ ml_inference_launch_cores(struct ml_test *test, struct ml_options *opt, uint16_t
{
struct test_inference *t = ml_test_priv(test);
uint32_t lcore_id;
+ uint32_t nb_reqs;
uint32_t id = 0;
+ uint32_t qp_id;
+
+ nb_reqs = opt->repetitions / opt->queue_pairs;
RTE_LCORE_FOREACH_WORKER(lcore_id)
{
- if (id == 2)
+ if (id >= opt->queue_pairs * 2)
break;
- t->args[lcore_id].nb_reqs = opt->repetitions;
+ qp_id = id / 2;
+ t->args[lcore_id].qp_id = qp_id;
+ t->args[lcore_id].nb_reqs = nb_reqs;
+ if (qp_id == 0)
+ t->args[lcore_id].nb_reqs += opt->repetitions - nb_reqs * opt->queue_pairs;
+
+ if (t->args[lcore_id].nb_reqs == 0) {
+ id++;
+ break;
+ }
+
t->args[lcore_id].start_fid = start_fid;
t->args[lcore_id].end_fid = end_fid;
diff --git a/app/test-mldev/test_inference_common.h b/app/test-mldev/test_inference_common.h
index 7bbd68aefa..2c08d768f2 100644
--- a/app/test-mldev/test_inference_common.h
+++ b/app/test-mldev/test_inference_common.h
@@ -27,6 +27,7 @@ struct ml_core_args {
uint64_t nb_reqs;
uint16_t start_fid;
uint16_t end_fid;
+ uint32_t qp_id;
struct rte_ml_op **enq_ops;
struct rte_ml_op **deq_ops;
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* RE: [PATCH v4 08/12] app/mldev: enable support for queue pairs and size
2023-02-07 15:49 ` [PATCH v4 08/12] app/mldev: enable support for queue pairs and size Srikanth Yalavarthi
@ 2023-03-02 8:15 ` Anup Prabhu
0 siblings, 0 replies; 122+ messages in thread
From: Anup Prabhu @ 2023-03-02 8:15 UTC (permalink / raw)
To: Srikanth Yalavarthi, Srikanth Yalavarthi
Cc: dev, Shivah Shankar Shankar Narayan Rao,
Jerin Jacob Kollanukkaran, Prince Takkar, Parijat Shukla
[-- Attachment #1: Type: text/plain, Size: 1391 bytes --]
> -----Original Message-----
> From: Srikanth Yalavarthi <syalavarthi@marvell.com>
> Sent: Tuesday, February 7, 2023 9:20 PM
> To: Srikanth Yalavarthi <syalavarthi@marvell.com>
> Cc: dev@dpdk.org; Shivah Shankar Shankar Narayan Rao
> <sshankarnara@marvell.com>; Jerin Jacob Kollanukkaran
> <jerinj@marvell.com>; Anup Prabhu <aprabhu@marvell.com>; Prince Takkar
> <ptakkar@marvell.com>; Parijat Shukla <pshukla@marvell.com>
> Subject: [PATCH v4 08/12] app/mldev: enable support for queue pairs and
> size
>
> Added support to create multiple queue-pairs per device to enqueue and
> dequeue inference requests. Number of queue pairs to be created can be
> specified through "--queue_pairs" option.
> Support is also enabled to control the number of descriptors per each queue
> pair through "--queue_size" option. Inference requests for a model are
> distributed across all available queue-pairs.
>
> Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
> ---
> app/test-mldev/ml_options.c | 40 ++++++++++---
> app/test-mldev/ml_options.h | 4 ++
> app/test-mldev/test_common.c | 2 +-
> app/test-mldev/test_inference_common.c | 79
> +++++++++++++++++++++----- app/test-mldev/test_inference_common.h
> | 1 +
> 5 files changed, 102 insertions(+), 24 deletions(-)
>
Acked-by: Anup Prabhu <aprabhu@marvell.com>
[-- Attachment #2: winmail.dat --]
[-- Type: application/ms-tnef, Size: 23116 bytes --]
^ permalink raw reply [flat|nested] 122+ messages in thread
* [PATCH v4 09/12] app/mldev: enable support for inference batches
2023-02-07 15:49 ` [PATCH v4 00/12] Implementation of mldev test application Srikanth Yalavarthi
` (7 preceding siblings ...)
2023-02-07 15:49 ` [PATCH v4 08/12] app/mldev: enable support for queue pairs and size Srikanth Yalavarthi
@ 2023-02-07 15:49 ` Srikanth Yalavarthi
2023-02-27 3:46 ` Anup Prabhu
2023-02-07 15:49 ` [PATCH v4 10/12] app/mldev: enable support for inference validation Srikanth Yalavarthi
` (2 subsequent siblings)
11 siblings, 1 reply; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-02-07 15:49 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Enabled support to execute multiple batches of inferences
per each enqueue request. Input and reference for the test
should be appropriately provided for multi-batch run. Number
of batches can be specified through "--batches" option.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/ml_options.c | 15 ++++++++++++---
app/test-mldev/ml_options.h | 2 ++
app/test-mldev/test_inference_common.c | 22 +++++++++++++---------
app/test-mldev/test_model_common.c | 6 ++++++
app/test-mldev/test_model_common.h | 1 +
5 files changed, 34 insertions(+), 12 deletions(-)
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index d5182a1701..331ec1704c 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -33,6 +33,7 @@ ml_options_default(struct ml_options *opt)
opt->burst_size = 1;
opt->queue_pairs = 1;
opt->queue_size = 1;
+ opt->batches = 0;
opt->debug = false;
}
@@ -176,6 +177,12 @@ ml_parse_queue_size(struct ml_options *opt, const char *arg)
return parser_read_uint16(&opt->queue_size, arg);
}
+static int
+ml_parse_batches(struct ml_options *opt, const char *arg)
+{
+ return parser_read_uint16(&opt->batches, arg);
+}
+
static void
ml_dump_test_options(const char *testname)
{
@@ -196,7 +203,8 @@ ml_dump_test_options(const char *testname)
"\t\t--repetitions : number of inference repetitions\n"
"\t\t--burst_size : inference burst size\n"
"\t\t--queue_pairs : number of queue pairs to create\n"
- "\t\t--queue_size : size fo queue-pair\n");
+ "\t\t--queue_size : size fo queue-pair\n"
+ "\t\t--batches : number of batches of input\n");
printf("\n");
}
}
@@ -220,7 +228,8 @@ static struct option lgopts[] = {
{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
{ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
{ML_BURST_SIZE, 1, 0, 0}, {ML_QUEUE_PAIRS, 1, 0, 0}, {ML_QUEUE_SIZE, 1, 0, 0},
- {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
+ {ML_BATCHES, 1, 0, 0}, {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0},
+ {NULL, 0, 0, 0}};
static int
ml_opts_parse_long(int opt_idx, struct ml_options *opt)
@@ -232,7 +241,7 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
{ML_SOCKET_ID, ml_parse_socket_id}, {ML_MODELS, ml_parse_models},
{ML_FILELIST, ml_parse_filelist}, {ML_REPETITIONS, ml_parse_repetitions},
{ML_BURST_SIZE, ml_parse_burst_size}, {ML_QUEUE_PAIRS, ml_parse_queue_pairs},
- {ML_QUEUE_SIZE, ml_parse_queue_size},
+ {ML_QUEUE_SIZE, ml_parse_queue_size}, {ML_BATCHES, ml_parse_batches},
};
for (i = 0; i < RTE_DIM(parsermap); i++) {
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index 6bfef1b979..d23e842895 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -22,6 +22,7 @@
#define ML_BURST_SIZE ("burst_size")
#define ML_QUEUE_PAIRS ("queue_pairs")
#define ML_QUEUE_SIZE ("queue_size")
+#define ML_BATCHES ("batches")
#define ML_DEBUG ("debug")
#define ML_HELP ("help")
@@ -41,6 +42,7 @@ struct ml_options {
uint16_t burst_size;
uint16_t queue_pairs;
uint16_t queue_size;
+ uint16_t batches;
bool debug;
};
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
index d4a00a5399..03873be69f 100644
--- a/app/test-mldev/test_inference_common.c
+++ b/app/test-mldev/test_inference_common.c
@@ -56,7 +56,7 @@ ml_enqueue_single(void *arg)
goto retry;
op->model_id = t->model[fid].id;
- op->nb_batches = t->model[fid].info.batch_size;
+ op->nb_batches = t->model[fid].nb_batches;
op->mempool = t->op_pool;
op->input.addr = req->input;
@@ -168,7 +168,7 @@ ml_enqueue_burst(void *arg)
for (i = 0; i < ops_count; i++) {
args->enq_ops[i]->model_id = t->model[fid].id;
- args->enq_ops[i]->nb_batches = t->model[fid].info.batch_size;
+ args->enq_ops[i]->nb_batches = t->model[fid].nb_batches;
args->enq_ops[i]->mempool = t->op_pool;
args->enq_ops[i]->input.addr = args->reqs[i]->input;
@@ -363,6 +363,11 @@ test_inference_opt_dump(struct ml_options *opt)
ml_dump("queue_pairs", "%u", opt->queue_pairs);
ml_dump("queue_size", "%u", opt->queue_size);
+ if (opt->batches == 0)
+ ml_dump("batches", "%u (default)", opt->batches);
+ else
+ ml_dump("batches", "%u", opt->batches);
+
ml_dump_begin("filelist");
for (i = 0; i < opt->nb_filelist; i++) {
ml_dump_list("model", i, opt->filelist[i].model);
@@ -531,8 +536,8 @@ ml_request_initialize(struct rte_mempool *mp, void *opaque, void *obj, unsigned
req->niters = 0;
/* quantize data */
- rte_ml_io_quantize(t->cmn.opt->dev_id, t->model[t->fid].id,
- t->model[t->fid].info.batch_size, t->model[t->fid].input, req->input);
+ rte_ml_io_quantize(t->cmn.opt->dev_id, t->model[t->fid].id, t->model[t->fid].nb_batches,
+ t->model[t->fid].input, req->input);
}
int
@@ -550,7 +555,7 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, uint16_t
int ret;
/* get input buffer size */
- ret = rte_ml_io_input_size_get(opt->dev_id, t->model[fid].id, t->model[fid].info.batch_size,
+ ret = rte_ml_io_input_size_get(opt->dev_id, t->model[fid].id, t->model[fid].nb_batches,
&t->model[fid].inp_qsize, &t->model[fid].inp_dsize);
if (ret != 0) {
ml_err("Failed to get input size, model : %s\n", opt->filelist[fid].model);
@@ -558,9 +563,8 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, uint16_t
}
/* get output buffer size */
- ret = rte_ml_io_output_size_get(opt->dev_id, t->model[fid].id,
- t->model[fid].info.batch_size, &t->model[fid].out_qsize,
- &t->model[fid].out_dsize);
+ ret = rte_ml_io_output_size_get(opt->dev_id, t->model[fid].id, t->model[fid].nb_batches,
+ &t->model[fid].out_qsize, &t->model[fid].out_dsize);
if (ret != 0) {
ml_err("Failed to get input size, model : %s\n", opt->filelist[fid].model);
return ret;
@@ -705,7 +709,7 @@ ml_request_finish(struct rte_mempool *mp, void *opaque, void *obj, unsigned int
return;
t->nb_used++;
- rte_ml_io_dequantize(t->cmn.opt->dev_id, model->id, t->model[req->fid].info.batch_size,
+ rte_ml_io_dequantize(t->cmn.opt->dev_id, model->id, t->model[req->fid].nb_batches,
req->output, model->output);
}
diff --git a/app/test-mldev/test_model_common.c b/app/test-mldev/test_model_common.c
index 6316430813..5e319e3118 100644
--- a/app/test-mldev/test_model_common.c
+++ b/app/test-mldev/test_model_common.c
@@ -75,6 +75,12 @@ ml_model_load(struct ml_test *test, struct ml_options *opt, struct ml_model *mod
return ret;
}
+ /* Update number of batches */
+ if (opt->batches == 0)
+ model->nb_batches = model->info.batch_size;
+ else
+ model->nb_batches = opt->batches;
+
model->state = MODEL_LOADED;
return 0;
diff --git a/app/test-mldev/test_model_common.h b/app/test-mldev/test_model_common.h
index 6eb70a352a..bd55bc7334 100644
--- a/app/test-mldev/test_model_common.h
+++ b/app/test-mldev/test_model_common.h
@@ -33,6 +33,7 @@ struct ml_model {
uint8_t *output;
struct rte_mempool *io_pool;
+ uint32_t nb_batches;
};
int ml_model_load(struct ml_test *test, struct ml_options *opt, struct ml_model *model,
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* RE: [PATCH v4 09/12] app/mldev: enable support for inference batches
2023-02-07 15:49 ` [PATCH v4 09/12] app/mldev: enable support for inference batches Srikanth Yalavarthi
@ 2023-02-27 3:46 ` Anup Prabhu
0 siblings, 0 replies; 122+ messages in thread
From: Anup Prabhu @ 2023-02-27 3:46 UTC (permalink / raw)
To: Srikanth Yalavarthi, Srikanth Yalavarthi
Cc: dev, Shivah Shankar Shankar Narayan Rao,
Jerin Jacob Kollanukkaran, Prince Takkar, Parijat Shukla
[-- Attachment #1: Type: text/plain, Size: 1235 bytes --]
> -----Original Message-----
> From: Srikanth Yalavarthi <syalavarthi@marvell.com>
> Sent: Tuesday, February 7, 2023 9:20 PM
> To: Srikanth Yalavarthi <syalavarthi@marvell.com>
> Cc: dev@dpdk.org; Shivah Shankar Shankar Narayan Rao
> <sshankarnara@marvell.com>; Jerin Jacob Kollanukkaran
> <jerinj@marvell.com>; Anup Prabhu <aprabhu@marvell.com>; Prince Takkar
> <ptakkar@marvell.com>; Parijat Shukla <pshukla@marvell.com>
> Subject: [PATCH v4 09/12] app/mldev: enable support for inference batches
>
> Enabled support to execute multiple batches of inferences per each
> enqueue request. Input and reference for the test should be appropriately
> provided for multi-batch run. Number of batches can be specified through "--
> batches" option.
>
> Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
> ---
> app/test-mldev/ml_options.c | 15 ++++++++++++---
> app/test-mldev/ml_options.h | 2 ++
> app/test-mldev/test_inference_common.c | 22 +++++++++++++---------
> app/test-mldev/test_model_common.c | 6 ++++++
> app/test-mldev/test_model_common.h | 1 +
> 5 files changed, 34 insertions(+), 12 deletions(-)
>
Acked-by: Anup Prabhu <aprabhu@marvell.com>
[-- Attachment #2: winmail.dat --]
[-- Type: application/ms-tnef, Size: 23113 bytes --]
^ permalink raw reply [flat|nested] 122+ messages in thread
* [PATCH v4 10/12] app/mldev: enable support for inference validation
2023-02-07 15:49 ` [PATCH v4 00/12] Implementation of mldev test application Srikanth Yalavarthi
` (8 preceding siblings ...)
2023-02-07 15:49 ` [PATCH v4 09/12] app/mldev: enable support for inference batches Srikanth Yalavarthi
@ 2023-02-07 15:49 ` Srikanth Yalavarthi
2023-02-16 12:23 ` Anup Prabhu
2023-02-07 15:49 ` [PATCH v4 11/12] app/mldev: enable reporting stats in mldev app Srikanth Yalavarthi
2023-02-07 15:49 ` [PATCH v4 12/12] app/mldev: add documentation for mldev test cases Srikanth Yalavarthi
11 siblings, 1 reply; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-02-07 15:49 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Enabled support to validate inference output with reference
output provided by the user. Validation would be successful
only when the inference outputs are within the 'tolerance'
specified through command line option "--tolerance".
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/meson.build | 2 +-
app/test-mldev/ml_options.c | 36 +++-
app/test-mldev/ml_options.h | 3 +
app/test-mldev/test_inference_common.c | 218 ++++++++++++++++++++++++-
app/test-mldev/test_inference_common.h | 1 +
app/test-mldev/test_model_common.h | 1 +
6 files changed, 250 insertions(+), 11 deletions(-)
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
index 41d22fb22c..15db534dc2 100644
--- a/app/test-mldev/meson.build
+++ b/app/test-mldev/meson.build
@@ -21,4 +21,4 @@ sources = files(
'test_inference_interleave.c',
)
-deps += ['mldev']
+deps += ['mldev', 'hash']
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 331ec1704c..092303903f 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -5,6 +5,7 @@
#include <errno.h>
#include <getopt.h>
#include <linux/limits.h>
+#include <math.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
@@ -34,6 +35,7 @@ ml_options_default(struct ml_options *opt)
opt->queue_pairs = 1;
opt->queue_size = 1;
opt->batches = 0;
+ opt->tolerance = 0.0;
opt->debug = false;
}
@@ -139,6 +141,13 @@ ml_parse_filelist(struct ml_options *opt, const char *arg)
}
strlcpy(opt->filelist[opt->nb_filelist].output, token, PATH_MAX);
+ /* reference - optional */
+ token = strtok(NULL, delim);
+ if (token != NULL)
+ strlcpy(opt->filelist[opt->nb_filelist].reference, token, PATH_MAX);
+ else
+ memset(opt->filelist[opt->nb_filelist].reference, 0, PATH_MAX);
+
opt->nb_filelist++;
if (opt->nb_filelist == 0) {
@@ -183,6 +192,14 @@ ml_parse_batches(struct ml_options *opt, const char *arg)
return parser_read_uint16(&opt->batches, arg);
}
+static int
+ml_parse_tolerance(struct ml_options *opt, const char *arg)
+{
+ opt->tolerance = fabs(atof(arg));
+
+ return 0;
+}
+
static void
ml_dump_test_options(const char *testname)
{
@@ -199,12 +216,13 @@ ml_dump_test_options(const char *testname)
if ((strcmp(testname, "inference_ordered") == 0) ||
(strcmp(testname, "inference_interleave") == 0)) {
- printf("\t\t--filelist : comma separated list of model, input and output\n"
+ printf("\t\t--filelist : comma separated list of model, input, output and reference\n"
"\t\t--repetitions : number of inference repetitions\n"
"\t\t--burst_size : inference burst size\n"
"\t\t--queue_pairs : number of queue pairs to create\n"
"\t\t--queue_size : size fo queue-pair\n"
- "\t\t--batches : number of batches of input\n");
+ "\t\t--batches : number of batches of input\n"
+ "\t\t--tolerance : maximum tolerance (%%) for output validation\n");
printf("\n");
}
}
@@ -224,12 +242,13 @@ print_usage(char *program)
ml_test_dump_names(ml_dump_test_options);
}
-static struct option lgopts[] = {
- {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
- {ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
- {ML_BURST_SIZE, 1, 0, 0}, {ML_QUEUE_PAIRS, 1, 0, 0}, {ML_QUEUE_SIZE, 1, 0, 0},
- {ML_BATCHES, 1, 0, 0}, {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0},
- {NULL, 0, 0, 0}};
+static struct option lgopts[] = {{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0},
+ {ML_SOCKET_ID, 1, 0, 0}, {ML_MODELS, 1, 0, 0},
+ {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
+ {ML_BURST_SIZE, 1, 0, 0}, {ML_QUEUE_PAIRS, 1, 0, 0},
+ {ML_QUEUE_SIZE, 1, 0, 0}, {ML_BATCHES, 1, 0, 0},
+ {ML_TOLERANCE, 1, 0, 0}, {ML_DEBUG, 0, 0, 0},
+ {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
static int
ml_opts_parse_long(int opt_idx, struct ml_options *opt)
@@ -242,6 +261,7 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
{ML_FILELIST, ml_parse_filelist}, {ML_REPETITIONS, ml_parse_repetitions},
{ML_BURST_SIZE, ml_parse_burst_size}, {ML_QUEUE_PAIRS, ml_parse_queue_pairs},
{ML_QUEUE_SIZE, ml_parse_queue_size}, {ML_BATCHES, ml_parse_batches},
+ {ML_TOLERANCE, ml_parse_tolerance},
};
for (i = 0; i < RTE_DIM(parsermap); i++) {
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index d23e842895..79ac54de98 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -23,6 +23,7 @@
#define ML_QUEUE_PAIRS ("queue_pairs")
#define ML_QUEUE_SIZE ("queue_size")
#define ML_BATCHES ("batches")
+#define ML_TOLERANCE ("tolerance")
#define ML_DEBUG ("debug")
#define ML_HELP ("help")
@@ -30,6 +31,7 @@ struct ml_filelist {
char model[PATH_MAX];
char input[PATH_MAX];
char output[PATH_MAX];
+ char reference[PATH_MAX];
};
struct ml_options {
@@ -43,6 +45,7 @@ struct ml_options {
uint16_t queue_pairs;
uint16_t queue_size;
uint16_t batches;
+ float tolerance;
bool debug;
};
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
index 03873be69f..3fce750e45 100644
--- a/app/test-mldev/test_inference_common.c
+++ b/app/test-mldev/test_inference_common.c
@@ -3,12 +3,15 @@
*/
#include <errno.h>
+#include <linux/limits.h>
#include <stdbool.h>
#include <stdint.h>
#include <stdio.h>
+#include <string.h>
#include <unistd.h>
#include <rte_common.h>
+#include <rte_hash_crc.h>
#include <rte_launch.h>
#include <rte_lcore.h>
#include <rte_malloc.h>
@@ -21,6 +24,27 @@
#include "test_common.h"
#include "test_inference_common.h"
+#define ML_TEST_READ_TYPE(buffer, type) (*((type *)buffer))
+
+#define ML_TEST_CHECK_OUTPUT(output, reference, tolerance) \
+ (((float)output - (float)reference) <= (((float)reference * tolerance) / 100.0))
+
+#define ML_OPEN_WRITE_GET_ERR(name, buffer, size, err) \
+ do { \
+ FILE *fp = fopen(name, "w+"); \
+ if (fp == NULL) { \
+ ml_err("Unable to create file: %s, error: %s", name, strerror(errno)); \
+ err = true; \
+ } else { \
+ if (fwrite(buffer, 1, size, fp) != size) { \
+ ml_err("Error writing output, file: %s, error: %s", name, \
+ strerror(errno)); \
+ err = true; \
+ } \
+ fclose(fp); \
+ } \
+ } while (0)
+
/* Enqueue inference requests with burst size equal to 1 */
static int
ml_enqueue_single(void *arg)
@@ -362,6 +386,7 @@ test_inference_opt_dump(struct ml_options *opt)
ml_dump("burst_size", "%u", opt->burst_size);
ml_dump("queue_pairs", "%u", opt->queue_pairs);
ml_dump("queue_size", "%u", opt->queue_size);
+ ml_dump("tolerance", "%-7.3f", opt->tolerance);
if (opt->batches == 0)
ml_dump("batches", "%u (default)", opt->batches);
@@ -373,6 +398,8 @@ test_inference_opt_dump(struct ml_options *opt)
ml_dump_list("model", i, opt->filelist[i].model);
ml_dump_list("input", i, opt->filelist[i].input);
ml_dump_list("output", i, opt->filelist[i].output);
+ if (strcmp(opt->filelist[i].reference, "\0") != 0)
+ ml_dump_list("reference", i, opt->filelist[i].reference);
}
ml_dump_end;
}
@@ -397,6 +424,7 @@ test_inference_setup(struct ml_test *test, struct ml_options *opt)
t = ml_test_priv(test);
t->nb_used = 0;
+ t->nb_valid = 0;
t->cmn.result = ML_TEST_FAILED;
t->cmn.opt = opt;
@@ -572,6 +600,9 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, uint16_t
/* allocate buffer for user data */
mz_size = t->model[fid].inp_dsize + t->model[fid].out_dsize;
+ if (strcmp(opt->filelist[fid].reference, "\0") != 0)
+ mz_size += t->model[fid].out_dsize;
+
sprintf(mz_name, "ml_user_data_%d", fid);
mz = rte_memzone_reserve(mz_name, mz_size, opt->socket_id, 0);
if (mz == NULL) {
@@ -582,6 +613,10 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, uint16_t
t->model[fid].input = mz->addr;
t->model[fid].output = t->model[fid].input + t->model[fid].inp_dsize;
+ if (strcmp(opt->filelist[fid].reference, "\0") != 0)
+ t->model[fid].reference = t->model[fid].output + t->model[fid].out_dsize;
+ else
+ t->model[fid].reference = NULL;
/* load input file */
fp = fopen(opt->filelist[fid].input, "r");
@@ -610,6 +645,27 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, uint16_t
}
fclose(fp);
+ /* load reference file */
+ if (t->model[fid].reference != NULL) {
+ fp = fopen(opt->filelist[fid].reference, "r");
+ if (fp == NULL) {
+ ml_err("Failed to open reference file : %s\n",
+ opt->filelist[fid].reference);
+ ret = -errno;
+ goto error;
+ }
+
+ if (fread(t->model[fid].reference, 1, t->model[fid].out_dsize, fp) !=
+ t->model[fid].out_dsize) {
+ ml_err("Failed to read reference file : %s\n",
+ opt->filelist[fid].reference);
+ ret = -errno;
+ fclose(fp);
+ goto error;
+ }
+ fclose(fp);
+ }
+
/* create mempool for quantized input and output buffers. ml_request_initialize is
* used as a callback for object creation.
*/
@@ -694,6 +750,121 @@ ml_inference_mem_destroy(struct ml_test *test, struct ml_options *opt)
rte_mempool_free(t->op_pool);
}
+static bool
+ml_inference_validation(struct ml_test *test, struct ml_request *req)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)test);
+ struct ml_model *model;
+ uint32_t nb_elements;
+ uint8_t *reference;
+ uint8_t *output;
+ bool match;
+ uint32_t i;
+ uint32_t j;
+
+ model = &t->model[req->fid];
+
+ /* compare crc when tolerance is 0 */
+ if (t->cmn.opt->tolerance == 0.0) {
+ match = (rte_hash_crc(model->output, model->out_dsize, 0) ==
+ rte_hash_crc(model->reference, model->out_dsize, 0));
+ } else {
+ output = model->output;
+ reference = model->reference;
+
+ i = 0;
+next_output:
+ nb_elements =
+ model->info.output_info[i].shape.w * model->info.output_info[i].shape.x *
+ model->info.output_info[i].shape.y * model->info.output_info[i].shape.z;
+ j = 0;
+next_element:
+ match = false;
+ switch (model->info.output_info[i].dtype) {
+ case RTE_ML_IO_TYPE_INT8:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, int8_t),
+ ML_TEST_READ_TYPE(reference, int8_t),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(int8_t);
+ reference += sizeof(int8_t);
+ break;
+ case RTE_ML_IO_TYPE_UINT8:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, uint8_t),
+ ML_TEST_READ_TYPE(reference, uint8_t),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(float);
+ reference += sizeof(float);
+ break;
+ case RTE_ML_IO_TYPE_INT16:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, int16_t),
+ ML_TEST_READ_TYPE(reference, int16_t),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(int16_t);
+ reference += sizeof(int16_t);
+ break;
+ case RTE_ML_IO_TYPE_UINT16:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, uint16_t),
+ ML_TEST_READ_TYPE(reference, uint16_t),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(uint16_t);
+ reference += sizeof(uint16_t);
+ break;
+ case RTE_ML_IO_TYPE_INT32:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, int32_t),
+ ML_TEST_READ_TYPE(reference, int32_t),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(int32_t);
+ reference += sizeof(int32_t);
+ break;
+ case RTE_ML_IO_TYPE_UINT32:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, uint32_t),
+ ML_TEST_READ_TYPE(reference, uint32_t),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(uint32_t);
+ reference += sizeof(uint32_t);
+ break;
+ case RTE_ML_IO_TYPE_FP32:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, float),
+ ML_TEST_READ_TYPE(reference, float),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(float);
+ reference += sizeof(float);
+ break;
+ default: /* other types, fp8, fp16, bfloat16 */
+ match = true;
+ }
+
+ if (!match)
+ goto done;
+ j++;
+ if (j < nb_elements)
+ goto next_element;
+
+ i++;
+ if (i < model->info.nb_outputs)
+ goto next_output;
+ }
+done:
+ if (match)
+ t->nb_valid++;
+
+ return match;
+}
+
/* Callback for mempool object iteration. This call would dequantize output data. */
static void
ml_request_finish(struct rte_mempool *mp, void *opaque, void *obj, unsigned int obj_idx)
@@ -701,9 +872,10 @@ ml_request_finish(struct rte_mempool *mp, void *opaque, void *obj, unsigned int
struct test_inference *t = ml_test_priv((struct ml_test *)opaque);
struct ml_request *req = (struct ml_request *)obj;
struct ml_model *model = &t->model[req->fid];
+ char str[PATH_MAX];
+ bool error = false;
RTE_SET_USED(mp);
- RTE_SET_USED(obj_idx);
if (req->niters == 0)
return;
@@ -711,6 +883,48 @@ ml_request_finish(struct rte_mempool *mp, void *opaque, void *obj, unsigned int
t->nb_used++;
rte_ml_io_dequantize(t->cmn.opt->dev_id, model->id, t->model[req->fid].nb_batches,
req->output, model->output);
+
+ if (model->reference == NULL) {
+ t->nb_valid++;
+ goto dump_output_pass;
+ }
+
+ if (!ml_inference_validation(opaque, req))
+ goto dump_output_fail;
+ else
+ goto dump_output_pass;
+
+dump_output_pass:
+ if (obj_idx == 0) {
+ /* write quantized output */
+ snprintf(str, PATH_MAX, "%s.q", t->cmn.opt->filelist[req->fid].output);
+ ML_OPEN_WRITE_GET_ERR(str, req->output, model->out_qsize, error);
+ if (error)
+ return;
+
+ /* write dequantized output */
+ snprintf(str, PATH_MAX, "%s", t->cmn.opt->filelist[req->fid].output);
+ ML_OPEN_WRITE_GET_ERR(str, model->output, model->out_dsize, error);
+ if (error)
+ return;
+ }
+
+ return;
+
+dump_output_fail:
+ if (t->cmn.opt->debug) {
+ /* dump quantized output buffer */
+ snprintf(str, PATH_MAX, "%s.q.%d", t->cmn.opt->filelist[req->fid].output, obj_idx);
+ ML_OPEN_WRITE_GET_ERR(str, req->output, model->out_qsize, error);
+ if (error)
+ return;
+
+ /* dump dequantized output buffer */
+ snprintf(str, PATH_MAX, "%s.%d", t->cmn.opt->filelist[req->fid].output, obj_idx);
+ ML_OPEN_WRITE_GET_ERR(str, model->output, model->out_dsize, error);
+ if (error)
+ return;
+ }
}
int
@@ -722,7 +936,7 @@ ml_inference_result(struct ml_test *test, struct ml_options *opt, uint16_t fid)
rte_mempool_obj_iter(t->model[fid].io_pool, ml_request_finish, test);
- if (t->nb_used > 0)
+ if (t->nb_used == t->nb_valid)
t->cmn.result = ML_TEST_SUCCESS;
else
t->cmn.result = ML_TEST_FAILED;
diff --git a/app/test-mldev/test_inference_common.h b/app/test-mldev/test_inference_common.h
index 2c08d768f2..e8e0b319e0 100644
--- a/app/test-mldev/test_inference_common.h
+++ b/app/test-mldev/test_inference_common.h
@@ -43,6 +43,7 @@ struct test_inference {
struct rte_mempool *op_pool;
uint64_t nb_used;
+ uint64_t nb_valid;
uint16_t fid;
int (*enqueue)(void *arg);
diff --git a/app/test-mldev/test_model_common.h b/app/test-mldev/test_model_common.h
index bd55bc7334..e78bc491af 100644
--- a/app/test-mldev/test_model_common.h
+++ b/app/test-mldev/test_model_common.h
@@ -31,6 +31,7 @@ struct ml_model {
uint8_t *input;
uint8_t *output;
+ uint8_t *reference;
struct rte_mempool *io_pool;
uint32_t nb_batches;
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* RE: [PATCH v4 10/12] app/mldev: enable support for inference validation
2023-02-07 15:49 ` [PATCH v4 10/12] app/mldev: enable support for inference validation Srikanth Yalavarthi
@ 2023-02-16 12:23 ` Anup Prabhu
0 siblings, 0 replies; 122+ messages in thread
From: Anup Prabhu @ 2023-02-16 12:23 UTC (permalink / raw)
To: Srikanth Yalavarthi, Srikanth Yalavarthi
Cc: dev, Shivah Shankar Shankar Narayan Rao,
Jerin Jacob Kollanukkaran, Prince Takkar, Parijat Shukla
[-- Attachment #1: Type: text/plain, Size: 1283 bytes --]
> -----Original Message-----
> From: Srikanth Yalavarthi <syalavarthi@marvell.com>
> Sent: Tuesday, February 7, 2023 9:20 PM
> To: Srikanth Yalavarthi <syalavarthi@marvell.com>
> Cc: dev@dpdk.org; Shivah Shankar Shankar Narayan Rao
> <sshankarnara@marvell.com>; Jerin Jacob Kollanukkaran
> <jerinj@marvell.com>; Anup Prabhu <aprabhu@marvell.com>; Prince Takkar
> <ptakkar@marvell.com>; Parijat Shukla <pshukla@marvell.com>
> Subject: [PATCH v4 10/12] app/mldev: enable support for inference
> validation
>
> Enabled support to validate inference output with reference output
> provided by the user. Validation would be successful only when the
> inference outputs are within the 'tolerance'
> specified through command line option "--tolerance".
>
> Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
> ---
> app/test-mldev/meson.build | 2 +-
> app/test-mldev/ml_options.c | 36 +++-
> app/test-mldev/ml_options.h | 3 +
> app/test-mldev/test_inference_common.c | 218
> ++++++++++++++++++++++++-
> app/test-mldev/test_inference_common.h | 1 +
> app/test-mldev/test_model_common.h | 1 +
> 6 files changed, 250 insertions(+), 11 deletions(-)
>
Acked-by: Anup Prabhu <aprabhu@marvell.com>
[-- Attachment #2: winmail.dat --]
[-- Type: application/ms-tnef, Size: 23084 bytes --]
^ permalink raw reply [flat|nested] 122+ messages in thread
* [PATCH v4 11/12] app/mldev: enable reporting stats in mldev app
2023-02-07 15:49 ` [PATCH v4 00/12] Implementation of mldev test application Srikanth Yalavarthi
` (9 preceding siblings ...)
2023-02-07 15:49 ` [PATCH v4 10/12] app/mldev: enable support for inference validation Srikanth Yalavarthi
@ 2023-02-07 15:49 ` Srikanth Yalavarthi
2023-02-16 4:21 ` Anup Prabhu
2023-02-07 15:49 ` [PATCH v4 12/12] app/mldev: add documentation for mldev test cases Srikanth Yalavarthi
11 siblings, 1 reply; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-02-07 15:49 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Enable reporting driver xstats and inference end-to-end
latency and throughput in mldev inference tests. Reporting
of stats can be enabled using "--stats" option.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/ml_options.c | 22 ++--
app/test-mldev/ml_options.h | 2 +
app/test-mldev/test_inference_common.c | 139 +++++++++++++++++++++
app/test-mldev/test_inference_common.h | 8 ++
app/test-mldev/test_inference_interleave.c | 4 +
app/test-mldev/test_inference_ordered.c | 1 +
6 files changed, 168 insertions(+), 8 deletions(-)
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 092303903f..0e7877eed3 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -36,6 +36,7 @@ ml_options_default(struct ml_options *opt)
opt->queue_size = 1;
opt->batches = 0;
opt->tolerance = 0.0;
+ opt->stats = false;
opt->debug = false;
}
@@ -222,7 +223,8 @@ ml_dump_test_options(const char *testname)
"\t\t--queue_pairs : number of queue pairs to create\n"
"\t\t--queue_size : size fo queue-pair\n"
"\t\t--batches : number of batches of input\n"
- "\t\t--tolerance : maximum tolerance (%%) for output validation\n");
+ "\t\t--tolerance : maximum tolerance (%%) for output validation\n"
+ "\t\t--stats : enable reporting performance statistics\n");
printf("\n");
}
}
@@ -242,13 +244,12 @@ print_usage(char *program)
ml_test_dump_names(ml_dump_test_options);
}
-static struct option lgopts[] = {{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0},
- {ML_SOCKET_ID, 1, 0, 0}, {ML_MODELS, 1, 0, 0},
- {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
- {ML_BURST_SIZE, 1, 0, 0}, {ML_QUEUE_PAIRS, 1, 0, 0},
- {ML_QUEUE_SIZE, 1, 0, 0}, {ML_BATCHES, 1, 0, 0},
- {ML_TOLERANCE, 1, 0, 0}, {ML_DEBUG, 0, 0, 0},
- {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
+static struct option lgopts[] = {
+ {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
+ {ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
+ {ML_BURST_SIZE, 1, 0, 0}, {ML_QUEUE_PAIRS, 1, 0, 0}, {ML_QUEUE_SIZE, 1, 0, 0},
+ {ML_BATCHES, 1, 0, 0}, {ML_TOLERANCE, 1, 0, 0}, {ML_STATS, 0, 0, 0},
+ {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
static int
ml_opts_parse_long(int opt_idx, struct ml_options *opt)
@@ -283,6 +284,11 @@ ml_options_parse(struct ml_options *opt, int argc, char **argv)
while ((opts = getopt_long(argc, argv, "", lgopts, &opt_idx)) != EOF) {
switch (opts) {
case 0: /* parse long options */
+ if (!strcmp(lgopts[opt_idx].name, "stats")) {
+ opt->stats = true;
+ break;
+ }
+
if (!strcmp(lgopts[opt_idx].name, "debug")) {
opt->debug = true;
break;
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index 79ac54de98..a375ae6750 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -24,6 +24,7 @@
#define ML_QUEUE_SIZE ("queue_size")
#define ML_BATCHES ("batches")
#define ML_TOLERANCE ("tolerance")
+#define ML_STATS ("stats")
#define ML_DEBUG ("debug")
#define ML_HELP ("help")
@@ -46,6 +47,7 @@ struct ml_options {
uint16_t queue_size;
uint16_t batches;
float tolerance;
+ bool stats;
bool debug;
};
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
index 3fce750e45..b68687378d 100644
--- a/app/test-mldev/test_inference_common.c
+++ b/app/test-mldev/test_inference_common.c
@@ -11,6 +11,7 @@
#include <unistd.h>
#include <rte_common.h>
+#include <rte_cycles.h>
#include <rte_hash_crc.h>
#include <rte_launch.h>
#include <rte_lcore.h>
@@ -45,6 +46,17 @@
} \
} while (0)
+static void
+print_line(uint16_t len)
+{
+ uint16_t i;
+
+ for (i = 0; i < len; i++)
+ printf("-");
+
+ printf("\n");
+}
+
/* Enqueue inference requests with burst size equal to 1 */
static int
ml_enqueue_single(void *arg)
@@ -54,6 +66,7 @@ ml_enqueue_single(void *arg)
struct rte_ml_op *op = NULL;
struct ml_core_args *args;
uint64_t model_enq = 0;
+ uint64_t start_cycle;
uint32_t burst_enq;
uint32_t lcore_id;
uint16_t fid;
@@ -61,6 +74,7 @@ ml_enqueue_single(void *arg)
lcore_id = rte_lcore_id();
args = &t->args[lcore_id];
+ args->start_cycles = 0;
model_enq = 0;
if (args->nb_reqs == 0)
@@ -96,10 +110,12 @@ ml_enqueue_single(void *arg)
req->fid = fid;
enqueue_req:
+ start_cycle = rte_get_tsc_cycles();
burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, args->qp_id, &op, 1);
if (burst_enq == 0)
goto enqueue_req;
+ args->start_cycles += start_cycle;
fid++;
if (likely(fid <= args->end_fid))
goto next_model;
@@ -123,10 +139,12 @@ ml_dequeue_single(void *arg)
uint64_t total_deq = 0;
uint8_t nb_filelist;
uint32_t burst_deq;
+ uint64_t end_cycle;
uint32_t lcore_id;
lcore_id = rte_lcore_id();
args = &t->args[lcore_id];
+ args->end_cycles = 0;
nb_filelist = args->end_fid - args->start_fid + 1;
if (args->nb_reqs == 0)
@@ -134,9 +152,11 @@ ml_dequeue_single(void *arg)
dequeue_req:
burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, args->qp_id, &op, 1);
+ end_cycle = rte_get_tsc_cycles();
if (likely(burst_deq == 1)) {
total_deq += burst_deq;
+ args->end_cycles += end_cycle;
if (unlikely(op->status == RTE_ML_OP_STATUS_ERROR)) {
rte_ml_op_error_get(t->cmn.opt->dev_id, op, &error);
ml_err("error_code = 0x%" PRIx64 ", error_message = %s\n", error.errcode,
@@ -159,6 +179,7 @@ ml_enqueue_burst(void *arg)
{
struct test_inference *t = ml_test_priv((struct ml_test *)arg);
struct ml_core_args *args;
+ uint64_t start_cycle;
uint16_t ops_count;
uint64_t model_enq;
uint16_t burst_enq;
@@ -171,6 +192,7 @@ ml_enqueue_burst(void *arg)
lcore_id = rte_lcore_id();
args = &t->args[lcore_id];
+ args->start_cycles = 0;
model_enq = 0;
if (args->nb_reqs == 0)
@@ -212,8 +234,10 @@ ml_enqueue_burst(void *arg)
pending = ops_count;
enqueue_reqs:
+ start_cycle = rte_get_tsc_cycles();
burst_enq =
rte_ml_enqueue_burst(t->cmn.opt->dev_id, args->qp_id, &args->enq_ops[idx], pending);
+ args->start_cycles += burst_enq * start_cycle;
pending = pending - burst_enq;
if (pending > 0) {
@@ -243,11 +267,13 @@ ml_dequeue_burst(void *arg)
uint64_t total_deq = 0;
uint16_t burst_deq = 0;
uint8_t nb_filelist;
+ uint64_t end_cycle;
uint32_t lcore_id;
uint32_t i;
lcore_id = rte_lcore_id();
args = &t->args[lcore_id];
+ args->end_cycles = 0;
nb_filelist = args->end_fid - args->start_fid + 1;
if (args->nb_reqs == 0)
@@ -256,9 +282,11 @@ ml_dequeue_burst(void *arg)
dequeue_burst:
burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, args->qp_id, args->deq_ops,
t->cmn.opt->burst_size);
+ end_cycle = rte_get_tsc_cycles();
if (likely(burst_deq > 0)) {
total_deq += burst_deq;
+ args->end_cycles += burst_deq * end_cycle;
for (i = 0; i < burst_deq; i++) {
if (unlikely(args->deq_ops[i]->status == RTE_ML_OP_STATUS_ERROR)) {
@@ -387,6 +415,7 @@ test_inference_opt_dump(struct ml_options *opt)
ml_dump("queue_pairs", "%u", opt->queue_pairs);
ml_dump("queue_size", "%u", opt->queue_size);
ml_dump("tolerance", "%-7.3f", opt->tolerance);
+ ml_dump("stats", "%s", (opt->stats ? "true" : "false"));
if (opt->batches == 0)
ml_dump("batches", "%u (default)", opt->batches);
@@ -459,6 +488,11 @@ test_inference_setup(struct ml_test *test, struct ml_options *opt)
RTE_CACHE_LINE_SIZE, opt->socket_id);
}
+ for (i = 0; i < RTE_MAX_LCORE; i++) {
+ t->args[i].start_cycles = 0;
+ t->args[i].end_cycles = 0;
+ }
+
return 0;
error:
@@ -985,3 +1019,108 @@ ml_inference_launch_cores(struct ml_test *test, struct ml_options *opt, uint16_t
return 0;
}
+
+int
+ml_inference_stats_get(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t = ml_test_priv(test);
+ uint64_t total_cycles = 0;
+ uint32_t nb_filelist;
+ uint64_t throughput;
+ uint64_t avg_e2e;
+ uint32_t qp_id;
+ uint64_t freq;
+ int ret;
+ int i;
+
+ if (!opt->stats)
+ return 0;
+
+ /* get xstats size */
+ t->xstats_size = rte_ml_dev_xstats_names_get(opt->dev_id, NULL, 0);
+ if (t->xstats_size >= 0) {
+ /* allocate for xstats_map and values */
+ t->xstats_map = rte_malloc(
+ "ml_xstats_map", t->xstats_size * sizeof(struct rte_ml_dev_xstats_map), 0);
+ if (t->xstats_map == NULL) {
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ t->xstats_values =
+ rte_malloc("ml_xstats_values", t->xstats_size * sizeof(uint64_t), 0);
+ if (t->xstats_values == NULL) {
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ ret = rte_ml_dev_xstats_names_get(opt->dev_id, t->xstats_map, t->xstats_size);
+ if (ret != t->xstats_size) {
+ printf("Unable to get xstats names, ret = %d\n", ret);
+ ret = -1;
+ goto error;
+ }
+
+ for (i = 0; i < t->xstats_size; i++)
+ rte_ml_dev_xstats_get(opt->dev_id, &t->xstats_map[i].id,
+ &t->xstats_values[i], 1);
+ }
+
+ /* print xstats*/
+ printf("\n");
+ print_line(80);
+ printf(" ML Device Extended Statistics\n");
+ print_line(80);
+ for (i = 0; i < t->xstats_size; i++)
+ printf(" %-64s = %" PRIu64 "\n", t->xstats_map[i].name, t->xstats_values[i]);
+ print_line(80);
+
+ /* release buffers */
+ if (t->xstats_map)
+ rte_free(t->xstats_map);
+
+ if (t->xstats_values)
+ rte_free(t->xstats_values);
+
+ /* print end-to-end stats */
+ freq = rte_get_tsc_hz();
+ for (qp_id = 0; qp_id < RTE_MAX_LCORE; qp_id++)
+ total_cycles += t->args[qp_id].end_cycles - t->args[qp_id].start_cycles;
+ avg_e2e = total_cycles / opt->repetitions;
+
+ if (freq == 0) {
+ avg_e2e = total_cycles / opt->repetitions;
+ printf(" %-64s = %" PRIu64 "\n", "Average End-to-End Latency (cycles)", avg_e2e);
+ } else {
+ avg_e2e = (total_cycles * NS_PER_S) / (opt->repetitions * freq);
+ printf(" %-64s = %" PRIu64 "\n", "Average End-to-End Latency (ns)", avg_e2e);
+ }
+
+ if (strcmp(opt->test_name, "inference_ordered") == 0)
+ nb_filelist = 1;
+ else
+ nb_filelist = t->cmn.opt->nb_filelist;
+
+ if (freq == 0) {
+ throughput = (nb_filelist * t->cmn.opt->repetitions * 1000000) / total_cycles;
+ printf(" %-64s = %" PRIu64 "\n", "Average Throughput (inferences / million cycles)",
+ throughput);
+ } else {
+ throughput = (nb_filelist * t->cmn.opt->repetitions * freq) / total_cycles;
+ printf(" %-64s = %" PRIu64 "\n", "Average Throughput (inferences / second)",
+ throughput);
+ }
+
+ print_line(80);
+
+ return 0;
+
+error:
+ if (t->xstats_map)
+ rte_free(t->xstats_map);
+
+ if (t->xstats_values)
+ rte_free(t->xstats_values);
+
+ return ret;
+}
diff --git a/app/test-mldev/test_inference_common.h b/app/test-mldev/test_inference_common.h
index e8e0b319e0..c20f6ecd89 100644
--- a/app/test-mldev/test_inference_common.h
+++ b/app/test-mldev/test_inference_common.h
@@ -32,6 +32,9 @@ struct ml_core_args {
struct rte_ml_op **enq_ops;
struct rte_ml_op **deq_ops;
struct ml_request **reqs;
+
+ uint64_t start_cycles;
+ uint64_t end_cycles;
};
struct test_inference {
@@ -50,6 +53,10 @@ struct test_inference {
int (*dequeue)(void *arg);
struct ml_core_args args[RTE_MAX_LCORE];
+
+ struct rte_ml_dev_xstats_map *xstats_map;
+ uint64_t *xstats_values;
+ int xstats_size;
} __rte_cache_aligned;
bool test_inference_cap_check(struct ml_options *opt);
@@ -67,5 +74,6 @@ void ml_inference_mem_destroy(struct ml_test *test, struct ml_options *opt);
int ml_inference_result(struct ml_test *test, struct ml_options *opt, uint16_t fid);
int ml_inference_launch_cores(struct ml_test *test, struct ml_options *opt, uint16_t start_fid,
uint16_t end_fid);
+int ml_inference_stats_get(struct ml_test *test, struct ml_options *opt);
#endif /* _ML_TEST_INFERENCE_COMMON_ */
diff --git a/app/test-mldev/test_inference_interleave.c b/app/test-mldev/test_inference_interleave.c
index eca67f0e62..5cb4eb81a0 100644
--- a/app/test-mldev/test_inference_interleave.c
+++ b/app/test-mldev/test_inference_interleave.c
@@ -60,7 +60,11 @@ test_inference_interleave_driver(struct ml_test *test, struct ml_options *opt)
goto error;
ml_inference_iomem_destroy(test, opt, fid);
+ }
+
+ ml_inference_stats_get(test, opt);
+ for (fid = 0; fid < opt->nb_filelist; fid++) {
ret = ml_model_stop(test, opt, &t->model[fid], fid);
if (ret != 0)
goto error;
diff --git a/app/test-mldev/test_inference_ordered.c b/app/test-mldev/test_inference_ordered.c
index ee9c7bf14e..68670aad77 100644
--- a/app/test-mldev/test_inference_ordered.c
+++ b/app/test-mldev/test_inference_ordered.c
@@ -58,6 +58,7 @@ test_inference_ordered_driver(struct ml_test *test, struct ml_options *opt)
goto error;
ml_inference_iomem_destroy(test, opt, fid);
+ ml_inference_stats_get(test, opt);
/* stop model */
ret = ml_model_stop(test, opt, &t->model[fid], fid);
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* RE: [PATCH v4 11/12] app/mldev: enable reporting stats in mldev app
2023-02-07 15:49 ` [PATCH v4 11/12] app/mldev: enable reporting stats in mldev app Srikanth Yalavarthi
@ 2023-02-16 4:21 ` Anup Prabhu
0 siblings, 0 replies; 122+ messages in thread
From: Anup Prabhu @ 2023-02-16 4:21 UTC (permalink / raw)
To: Srikanth Yalavarthi, Srikanth Yalavarthi
Cc: dev, Shivah Shankar Shankar Narayan Rao,
Jerin Jacob Kollanukkaran, Prince Takkar, Parijat Shukla
[-- Attachment #1: Type: text/plain, Size: 1223 bytes --]
> -----Original Message-----
> From: Srikanth Yalavarthi <syalavarthi@marvell.com>
> Sent: Tuesday, February 7, 2023 9:20 PM
> To: Srikanth Yalavarthi <syalavarthi@marvell.com>
> Cc: dev@dpdk.org; Shivah Shankar Shankar Narayan Rao
> <sshankarnara@marvell.com>; Jerin Jacob Kollanukkaran
> <jerinj@marvell.com>; Anup Prabhu <aprabhu@marvell.com>; Prince Takkar
> <ptakkar@marvell.com>; Parijat Shukla <pshukla@marvell.com>
> Subject: [PATCH v4 11/12] app/mldev: enable reporting stats in mldev app
>
> Enable reporting driver xstats and inference end-to-end latency and
> throughput in mldev inference tests. Reporting of stats can be enabled using
> "--stats" option.
>
> Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
> ---
> app/test-mldev/ml_options.c | 22 ++--
> app/test-mldev/ml_options.h | 2 +
> app/test-mldev/test_inference_common.c | 139
> +++++++++++++++++++++
> app/test-mldev/test_inference_common.h | 8 ++
> app/test-mldev/test_inference_interleave.c | 4 +
> app/test-mldev/test_inference_ordered.c | 1 +
> 6 files changed, 168 insertions(+), 8 deletions(-)
>
Acked-by: Anup Prabhu <aprabhu@marvell.com>
[-- Attachment #2: winmail.dat --]
[-- Type: application/ms-tnef, Size: 23052 bytes --]
^ permalink raw reply [flat|nested] 122+ messages in thread
* [PATCH v4 12/12] app/mldev: add documentation for mldev test cases
2023-02-07 15:49 ` [PATCH v4 00/12] Implementation of mldev test application Srikanth Yalavarthi
` (10 preceding siblings ...)
2023-02-07 15:49 ` [PATCH v4 11/12] app/mldev: enable reporting stats in mldev app Srikanth Yalavarthi
@ 2023-02-07 15:49 ` Srikanth Yalavarthi
2023-02-15 12:26 ` Shivah Shankar Shankar Narayan Rao
2023-03-03 6:07 ` Anup Prabhu
11 siblings, 2 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-02-07 15:49 UTC (permalink / raw)
To: Thomas Monjalon, Srikanth Yalavarthi
Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Added documentation specific to mldev test cases. Added details
about all test cases and option supported by individual tests.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
MAINTAINERS | 1 +
app/test-mldev/test_inference_common.c | 27 +-
.../tools/img/mldev_inference_interleave.svg | 669 ++++++++++++++++++
.../tools/img/mldev_inference_ordered.svg | 528 ++++++++++++++
.../tools/img/mldev_model_ops_subtest_a.svg | 420 +++++++++++
.../tools/img/mldev_model_ops_subtest_b.svg | 423 +++++++++++
.../tools/img/mldev_model_ops_subtest_c.svg | 366 ++++++++++
.../tools/img/mldev_model_ops_subtest_d.svg | 424 +++++++++++
doc/guides/tools/index.rst | 1 +
doc/guides/tools/testmldev.rst | 441 ++++++++++++
10 files changed, 3279 insertions(+), 21 deletions(-)
create mode 100644 doc/guides/tools/img/mldev_inference_interleave.svg
create mode 100644 doc/guides/tools/img/mldev_inference_ordered.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_a.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_b.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_c.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_d.svg
create mode 100644 doc/guides/tools/testmldev.rst
diff --git a/MAINTAINERS b/MAINTAINERS
index 9aae343556..f1b1915053 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -543,6 +543,7 @@ M: Srikanth Yalavarthi <syalavarthi@marvell.com>
F: lib/mldev/
F: app/test-mldev/
F: doc/guides/prog_guide/mldev.rst
+F: doc/guides/tools/testmldev.rst
Memory Pool Drivers
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
index b68687378d..7ce47766a1 100644
--- a/app/test-mldev/test_inference_common.c
+++ b/app/test-mldev/test_inference_common.c
@@ -1029,7 +1029,6 @@ ml_inference_stats_get(struct ml_test *test, struct ml_options *opt)
uint64_t throughput;
uint64_t avg_e2e;
uint32_t qp_id;
- uint64_t freq;
int ret;
int i;
@@ -1083,34 +1082,20 @@ ml_inference_stats_get(struct ml_test *test, struct ml_options *opt)
rte_free(t->xstats_values);
/* print end-to-end stats */
- freq = rte_get_tsc_hz();
for (qp_id = 0; qp_id < RTE_MAX_LCORE; qp_id++)
total_cycles += t->args[qp_id].end_cycles - t->args[qp_id].start_cycles;
- avg_e2e = total_cycles / opt->repetitions;
- if (freq == 0) {
- avg_e2e = total_cycles / opt->repetitions;
- printf(" %-64s = %" PRIu64 "\n", "Average End-to-End Latency (cycles)", avg_e2e);
- } else {
- avg_e2e = (total_cycles * NS_PER_S) / (opt->repetitions * freq);
- printf(" %-64s = %" PRIu64 "\n", "Average End-to-End Latency (ns)", avg_e2e);
- }
+ avg_e2e = ((total_cycles / opt->repetitions) * NS_PER_S) / rte_get_tsc_hz();
+ printf(" %-64s = %" PRIu64 "\n", "Average End-to-End Latency (ns)", avg_e2e);
+ /* print inference throughput */
if (strcmp(opt->test_name, "inference_ordered") == 0)
nb_filelist = 1;
else
- nb_filelist = t->cmn.opt->nb_filelist;
-
- if (freq == 0) {
- throughput = (nb_filelist * t->cmn.opt->repetitions * 1000000) / total_cycles;
- printf(" %-64s = %" PRIu64 "\n", "Average Throughput (inferences / million cycles)",
- throughput);
- } else {
- throughput = (nb_filelist * t->cmn.opt->repetitions * freq) / total_cycles;
- printf(" %-64s = %" PRIu64 "\n", "Average Throughput (inferences / second)",
- throughput);
- }
+ nb_filelist = opt->nb_filelist;
+ throughput = (nb_filelist * NS_PER_S) / avg_e2e;
+ printf(" %-64s = %" PRIu64 "\n", "Average Throughput (inferences / second)", throughput);
print_line(80);
return 0;
diff --git a/doc/guides/tools/img/mldev_inference_interleave.svg b/doc/guides/tools/img/mldev_inference_interleave.svg
new file mode 100644
index 0000000000..3a741ea627
--- /dev/null
+++ b/doc/guides/tools/img/mldev_inference_interleave.svg
@@ -0,0 +1,669 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- SPDX-License-Identifier: BSD-3-Clause -->
+<!-- Copyright (c) 2022 Marvell. -->
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="224mm"
+ height="150mm"
+ viewBox="0 0 224 150"
+ version="1.1"
+ id="svg5369"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="inference_interleave.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns:xlink="http://www.w3.org/1999/xlink"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg">
+ <sodipodi:namedview
+ id="namedview5371"
+ pagecolor="#ffffff"
+ bordercolor="#000000"
+ borderopacity="0.25"
+ inkscape:showpageshadow="2"
+ inkscape:pageopacity="0.0"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1"
+ inkscape:document-units="mm"
+ showgrid="false"
+ inkscape:zoom="0.74564394"
+ inkscape:cx="415.07747"
+ inkscape:cy="348.6919"
+ inkscape:window-width="1920"
+ inkscape:window-height="1017"
+ inkscape:window-x="1912"
+ inkscape:window-y="-8"
+ inkscape:window-maximized="1"
+ inkscape:current-layer="layer1" />
+ <defs
+ id="defs5366">
+ <linearGradient
+ inkscape:collect="always"
+ id="linearGradient19189">
+ <stop
+ style="stop-color:#ffeeaa;stop-opacity:1;"
+ offset="0"
+ id="stop19185" />
+ <stop
+ style="stop-color:#ffeeaa;stop-opacity:0;"
+ offset="1"
+ id="stop19187" />
+ </linearGradient>
+ <marker
+ style="overflow:visible"
+ id="TriangleStart"
+ refX="4"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="TriangleStart"
+ markerWidth="5.3244081"
+ markerHeight="6.155385"
+ viewBox="0 0 5.3244081 6.1553851"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.5)"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:context-stroke;stroke-width:1pt"
+ d="M 5.77,0 -2.88,5 V -5 Z"
+ id="path135" />
+ </marker>
+ <marker
+ style="overflow:visible"
+ id="RoundedArrow"
+ refX="5"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="RoundedArrow"
+ markerWidth="6.1347523"
+ markerHeight="5.9304948"
+ viewBox="0 0 6.1347524 5.9304951"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.7)"
+ d="m -0.21114562,-4.1055728 6.42229122,3.21114561 a 1,1 90 0 1 0,1.78885438 L -0.21114562,4.1055728 A 1.236068,1.236068 31.717474 0 1 -2,3 v -6 a 1.236068,1.236068 148.28253 0 1 1.78885438,-1.1055728 z"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:none"
+ id="path1367" />
+ </marker>
+ <linearGradient
+ inkscape:collect="always"
+ xlink:href="#linearGradient19189"
+ id="linearGradient19191"
+ x1="12.169352"
+ y1="105"
+ x2="284.83066"
+ y2="105"
+ gradientUnits="userSpaceOnUse"
+ gradientTransform="matrix(0.79055445,0,0,0.74078976,29.505892,28.991272)" />
+ </defs>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1">
+ <g
+ id="g1477"
+ transform="translate(-34.903236,-31.774189)">
+ <rect
+ style="fill:url(#linearGradient19191);fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.396267;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect16635"
+ width="215.18147"
+ height="139.60078"
+ x="39.3125"
+ y="36.973797"
+ ry="2.2354064" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-169.3954"
+ y="214.97237"
+ id="text5181"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan5179"
+ style="font-size:5.64444px;stroke-width:0.75"
+ x="-169.3954"
+ y="214.97237">test: inference_interleave</tspan></text>
+ <path
+ style="display:inline;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 138.05133,87.886263 17.45982,-10e-7"
+ id="path1912"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1724-0"
+ inkscape:connection-end="#rect1679" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 191.51115,87.886262 17.45981,10e-7"
+ id="path1914"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1679"
+ inkscape:connection-end="#rect1724" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 138.05133,109.78102 17.45982,-1e-5"
+ id="path1916"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1724-4-8"
+ inkscape:connection-end="#rect1679-4" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 191.51115,109.78101 17.45981,1e-5"
+ id="path1918"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1679-4"
+ inkscape:connection-end="#rect1724-4" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 138.05133,131.67576 17.45982,-1e-5"
+ id="path1920"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1724-6-7"
+ inkscape:connection-end="#rect1679-8" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 191.51115,131.67575 17.45981,1e-5"
+ id="path1922"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1679-8"
+ inkscape:connection-end="#rect1724-6" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.75;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;marker-start:url(#RoundedArrow);marker-end:url(#RoundedArrow)"
+ d="m 173.51116,60.08164 0,12.907336"
+ id="path1933"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1811"
+ inkscape:connection-end="#rect1924" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.368668;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.737336, 0.737336;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924"
+ width="46.97015"
+ height="73.58287"
+ x="150.02565"
+ y="72.988976"
+ ry="2.4685853" />
+ <text
+ xml:space="preserve"
+ style="font-size:6.35px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-84.44075"
+ y="173.5065"
+ id="text4097"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4095"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-84.44075"
+ y="173.5065">Queue</tspan><tspan
+ sodipodi:role="line"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-92.37825"
+ y="173.5065"
+ id="tspan4099">Pair 0</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:6.35px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-128.2318"
+ y="173.5065"
+ id="text4097-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4095-6"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-128.2318"
+ y="173.5065">Queue</tspan><tspan
+ sodipodi:role="line"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-136.1693"
+ y="173.5065"
+ id="tspan4099-4">Pair 2</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1679"
+ width="36"
+ height="18"
+ x="155.51115"
+ y="78.886261"
+ ry="3" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1679-8"
+ width="36"
+ height="18"
+ x="155.51115"
+ y="122.67575"
+ ry="3" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1679-4"
+ width="36"
+ height="18"
+ x="155.51115"
+ y="100.78101"
+ ry="3" />
+ <text
+ xml:space="preserve"
+ style="font-size:6.35px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-106.33705"
+ y="173.5065"
+ id="text4097-8"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4095-4"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-106.33705"
+ y="173.5065">Queue</tspan><tspan
+ sodipodi:role="line"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-114.27455"
+ y="173.5065"
+ id="tspan4099-5">Pair 1</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.388863;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1811"
+ width="58.909527"
+ height="18.812746"
+ x="144.0564"
+ y="41.268894"
+ ry="2.2255962" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-47.017281"
+ y="173.49187"
+ id="text4156"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4154"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';stroke-width:0.75"
+ x="-47.017281"
+ y="173.49187">Machine Learning</tspan><tspan
+ sodipodi:role="line"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';stroke-width:0.75"
+ x="-54.07283"
+ y="173.49187"
+ id="tspan4158">Hardware Engine</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.368668;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.737336, 0.737336;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924-0"
+ width="46.97015"
+ height="73.58287"
+ x="98.42067"
+ y="72.988976"
+ ry="2.4685853" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-88.32518"
+ y="122.24379"
+ id="text3708"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-88.32518"
+ y="122.24379">lcore 1</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-132.10504"
+ y="121.83865"
+ id="text3708-8"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-7"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-132.10504"
+ y="121.83865">lcore 5</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-0"
+ width="32.290321"
+ height="11.709678"
+ x="105.76101"
+ y="82.031425"
+ ry="3.0161259" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-6-7"
+ width="32.290321"
+ height="11.709678"
+ x="105.76101"
+ y="125.82092"
+ ry="3.0161259"
+ inkscape:connector-avoid="true" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-4-8"
+ width="32.290321"
+ height="11.709678"
+ x="105.76101"
+ y="103.92618"
+ ry="3.0161259"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-110.21718"
+ y="121.85381"
+ id="text3708-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-87"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-110.21718"
+ y="121.85381">lcore 3</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-150.82878"
+ y="121.79179"
+ id="text4542"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4540"
+ style="font-size:5.64444px;stroke-width:0.75"
+ x="-150.82878"
+ y="121.79179">Enqueue Workers</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.368668;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.737336, 0.737336;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924-7"
+ width="46.97015"
+ height="73.58287"
+ x="201.63062"
+ y="72.988976"
+ ry="2.4685853" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-88.32518"
+ y="225.08443"
+ id="text3708-9"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-9"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-88.32518"
+ y="225.08443">lcore 2</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-110.21167"
+ y="225.07202"
+ id="text3708-7"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-8"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-110.21167"
+ y="225.07202">lcore 4</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724"
+ width="32.290321"
+ height="11.709678"
+ x="208.97096"
+ y="82.031425"
+ ry="3.0161259" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-4"
+ width="32.290321"
+ height="11.709678"
+ x="208.97096"
+ y="103.92618"
+ ry="3.0161259" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-6"
+ width="32.290321"
+ height="11.709678"
+ x="208.97096"
+ y="125.82092"
+ ry="3.0161259" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-132.1133"
+ y="225.06514"
+ id="text3708-78"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-0"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-132.1133"
+ y="225.06514">lcore 6</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-150.82878"
+ y="225.00725"
+ id="text4542-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4540-7"
+ style="font-size:5.64444px;stroke-width:0.75"
+ x="-150.82878"
+ y="225.00725">Dequeue Workers</tspan></text>
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 79.368074,78.962117 26.440446,5.546991"
+ id="path6677"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-0"
+ inkscape:connection-start="#rect6252" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="M 78.978262,84.050781 112.13805,103.92618"
+ id="path6679"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-4-8"
+ inkscape:connection-start="#rect6252" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 73.959073,84.25738 42.026977,41.56354"
+ id="path6681"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-6-7"
+ inkscape:connection-start="#rect6252" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="M 79.368074,95.959838 105.76913,90.949016"
+ id="path6683"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-0"
+ inkscape:connection-start="#rect6252-8" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 79.368074,101.28215 26.416716,5.27791"
+ id="path7830"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-4-8"
+ inkscape:connection-start="#rect6252-8" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 79.069199,106.4283 32.903161,19.39262"
+ id="path7862"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-6-7"
+ inkscape:connection-start="#rect6252-8" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="M 79.069201,113.13371 111.97235,93.741103"
+ id="path7900"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-0"
+ inkscape:connection-start="#rect6252-2" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 79.368074,118.27987 26.416716,-5.2779"
+ id="path7932"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-4-8"
+ inkscape:connection-start="#rect6252-2" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 79.368074,123.60218 26.401056,5.01083"
+ id="path7998"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-6-7"
+ inkscape:connection-start="#rect6252-2" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="M 73.959074,135.30464 115.98605,93.741103"
+ id="path8000"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-0"
+ inkscape:connection-start="#rect6252-6" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="M 78.978264,135.51124 112.13805,115.63586"
+ id="path8002"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-4-8"
+ inkscape:connection-start="#rect6252-6" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 79.368074,140.59991 26.440446,-5.54699"
+ id="path8004"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-6-7"
+ inkscape:connection-start="#rect6252-6" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-76.53363"
+ y="65.63237"
+ id="text3708-84"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-4"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-76.53363"
+ y="65.63237">Model 0</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-98.991623"
+ y="66.015465"
+ id="text3708-84-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-4-6"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-98.991623"
+ y="66.015465">Model 1</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-121.44823"
+ y="65.646149"
+ id="text3708-84-9"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-4-1"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-121.44823"
+ y="65.646149">Model 2</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-143.9021"
+ y="65.625481"
+ id="text3708-84-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-4-8"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-143.9021"
+ y="65.625481">Model 3</tspan></text>
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect6252"
+ width="27.32258"
+ height="16.32258"
+ x="52.045494"
+ y="67.934799"
+ ry="1" />
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect6252-8"
+ width="27.32258"
+ height="16.32258"
+ x="52.045494"
+ y="90.391411"
+ ry="1" />
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect6252-2"
+ width="27.32258"
+ height="16.32258"
+ x="52.045494"
+ y="112.84802"
+ ry="1" />
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect6252-6"
+ width="27.32258"
+ height="16.32258"
+ x="52.045494"
+ y="135.30464"
+ ry="1" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.399886;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.799773, 0.799773;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924-1-3"
+ width="43.035683"
+ height="94.487152"
+ x="44.188942"
+ y="62.536991"
+ ry="3.1694498" />
+ <g
+ id="g1010"
+ transform="translate(21.464467,-15.875004)">
+ <text
+ xml:space="preserve"
+ style="font-weight:bold;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:'Arial Bold';text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-185.88483"
+ y="86.542366"
+ id="text15571"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan15569"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.5;stroke-opacity:1"
+ x="-185.88483"
+ y="86.542366">nb_worker_threads = 2 * MIN(nb_queue_pairs, (lcore_count - 1) / 2)</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-weight:bold;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:'Arial Bold';text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-178.43243"
+ y="90.903854"
+ id="text15571-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan15569-9"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.5;stroke-opacity:1"
+ x="-178.43243"
+ y="90.903854">inferences_per_queue_pair = nb_models * (repetitions / nb_queue_pairs)</tspan></text>
+ </g>
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/tools/img/mldev_inference_ordered.svg b/doc/guides/tools/img/mldev_inference_ordered.svg
new file mode 100644
index 0000000000..12fa6acaec
--- /dev/null
+++ b/doc/guides/tools/img/mldev_inference_ordered.svg
@@ -0,0 +1,528 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- SPDX-License-Identifier: BSD-3-Clause -->
+<!-- Copyright (c) 2022 Marvell. -->
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="243mm"
+ height="144mm"
+ viewBox="0 0 243 144"
+ version="1.1"
+ id="svg5"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="inference_ordered.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns:xlink="http://www.w3.org/1999/xlink"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg">
+ <sodipodi:namedview
+ id="namedview7"
+ pagecolor="#ffffff"
+ bordercolor="#000000"
+ borderopacity="0.25"
+ inkscape:showpageshadow="2"
+ inkscape:pageopacity="0.0"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1"
+ inkscape:document-units="mm"
+ showgrid="false"
+ inkscape:zoom="0.74564394"
+ inkscape:cx="488.83922"
+ inkscape:cy="234.69647"
+ inkscape:window-width="1920"
+ inkscape:window-height="1017"
+ inkscape:window-x="1912"
+ inkscape:window-y="-8"
+ inkscape:window-maximized="1"
+ inkscape:current-layer="layer1" />
+ <defs
+ id="defs2">
+ <marker
+ style="overflow:visible"
+ id="RoundedArrow"
+ refX="5"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="RoundedArrow"
+ markerWidth="6.1347523"
+ markerHeight="5.9304948"
+ viewBox="0 0 6.1347524 5.9304951"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.7)"
+ d="m -0.21114562,-4.1055728 6.42229122,3.21114561 a 1,1 90 0 1 0,1.78885438 L -0.21114562,4.1055728 A 1.236068,1.236068 31.717474 0 1 -2,3 v -6 a 1.236068,1.236068 148.28253 0 1 1.78885438,-1.1055728 z"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:none"
+ id="path1367" />
+ </marker>
+ <linearGradient
+ inkscape:collect="always"
+ id="linearGradient31002">
+ <stop
+ style="stop-color:#fff6d5;stop-opacity:1;"
+ offset="0"
+ id="stop30998" />
+ <stop
+ style="stop-color:#fff6d5;stop-opacity:0;"
+ offset="1"
+ id="stop31000" />
+ </linearGradient>
+ <marker
+ style="overflow:visible"
+ id="TriangleStart"
+ refX="4"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="TriangleStart"
+ markerWidth="5.3244081"
+ markerHeight="6.155385"
+ viewBox="0 0 5.3244081 6.1553851"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.5)"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:context-stroke;stroke-width:1pt"
+ d="M 5.77,0 -2.88,5 V -5 Z"
+ id="path135" />
+ </marker>
+ <linearGradient
+ inkscape:collect="always"
+ xlink:href="#linearGradient31002"
+ id="linearGradient31004"
+ x1="19.620968"
+ y1="102.90323"
+ x2="279.1532"
+ y2="102.90323"
+ gradientUnits="userSpaceOnUse"
+ gradientTransform="matrix(0.89215122,0,0,0.73190287,13.449912,42.668706)" />
+ </defs>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1">
+ <g
+ id="g1340"
+ transform="translate(-25.225796,-45.983871)">
+ <rect
+ style="fill:url(#linearGradient31004);fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.404032;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect27876"
+ width="231.09595"
+ height="132.45081"
+ x="31.177822"
+ y="51.758469"
+ ry="3.5071263" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1813"
+ width="38.408459"
+ height="45.86002"
+ x="34.901794"
+ y="99.14959"
+ ry="5.2246051"
+ inkscape:connector-avoid="true" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 73.310253,115.94935 36.498807,-11.6509"
+ id="path1906"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1813"
+ inkscape:connection-end="#rect1724-0" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 73.310253,122.0796 36.117817,1e-5"
+ id="path1908"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1813"
+ inkscape:connection-end="#rect1724-4-8" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="M 73.310253,128.20983 109.80905,139.8607"
+ id="path1910"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1813"
+ inkscape:connection-end="#rect1724-6-7" />
+ <path
+ style="display:inline;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 141.71839,99.266314 19.42262,-10e-7"
+ id="path1912"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1724-0"
+ inkscape:connection-end="#rect1679" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 197.14101,99.266313 19.42259,10e-7"
+ id="path1914"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1679"
+ inkscape:connection-end="#rect1724" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 141.71839,122.07961 19.42262,-1e-5"
+ id="path1916"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1724-4-8"
+ inkscape:connection-end="#rect1679-4" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 197.14101,122.0796 19.42259,1e-5"
+ id="path1918"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1679-4"
+ inkscape:connection-end="#rect1724-4" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 141.71839,144.89282 19.42262,0"
+ id="path1920"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1724-6-7"
+ inkscape:connection-end="#rect1679-8" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 197.14101,144.89282 19.42259,0"
+ id="path1922"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1679-8"
+ inkscape:connection-end="#rect1724-6" />
+ <text
+ xml:space="preserve"
+ style="font-size:8.46667px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-121.09793"
+ y="54.031597"
+ id="text4093"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4091"
+ style="font-size:8.46667px;stroke-width:0.75"
+ x="-121.09793"
+ y="54.031597">Model X</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:6.35px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-118.63563"
+ y="179.13635"
+ id="text4097"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4095"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-118.63563"
+ y="179.13635">Queue</tspan><tspan
+ sodipodi:role="line"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-126.57313"
+ y="179.13635"
+ id="tspan4099">Pair 1</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:6.35px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-141.44887"
+ y="179.13635"
+ id="text4097-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4095-6"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-141.44887"
+ y="179.13635">Queue</tspan><tspan
+ sodipodi:role="line"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-149.38637"
+ y="179.13635"
+ id="tspan4099-4">Pair 2</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.354849;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.709699, 0.709699;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924"
+ width="44.145252"
+ height="72.532341"
+ x="157.06865"
+ y="85.813438"
+ ry="4.31247" />
+ <g
+ id="g1224">
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1679"
+ width="36"
+ height="18"
+ x="161.14101"
+ y="90.266312"
+ ry="3" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1679-8"
+ width="36"
+ height="18"
+ x="161.14101"
+ y="135.89282"
+ ry="3" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1679-4"
+ width="36"
+ height="18"
+ x="161.14101"
+ y="113.07959"
+ ry="3" />
+ </g>
+ <text
+ xml:space="preserve"
+ style="font-size:6.35px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-95.820801"
+ y="179.13635"
+ id="text4097-8"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4095-4"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-95.820801"
+ y="179.13635">Queue</tspan><tspan
+ sodipodi:role="line"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-103.7583"
+ y="179.13635"
+ id="tspan4099-5">Pair 0</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.317648;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1811"
+ width="44.196934"
+ height="16.731901"
+ x="157.04254"
+ y="56.49292"
+ ry="2.761292" />
+ <text
+ xml:space="preserve"
+ style="font-size:3.5859px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.317649;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.952945, 0.317649;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-60.009941"
+ y="186.38451"
+ id="text4156"
+ transform="matrix(0,-1.040508,0.96106903,0,0,0)"><tspan
+ sodipodi:role="line"
+ id="tspan4154"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-family:Arial;-inkscape-font-specification:'Arial Bold';stroke-width:0.317648"
+ x="-60.009941"
+ y="186.38451">Machine Learning</tspan><tspan
+ sodipodi:role="line"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-family:Arial;-inkscape-font-specification:'Arial Bold';stroke-width:0.317648"
+ x="-64.492317"
+ y="186.38451"
+ id="tspan4158">Hardware Engine</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-99.705231"
+ y="125.91087"
+ id="text3708"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-99.705231"
+ y="125.91087">lcore 1</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-145.3221"
+ y="125.50572"
+ id="text3708-8"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-7"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-145.3221"
+ y="125.50572">lcore 5</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-122.51577"
+ y="125.52089"
+ id="text3708-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-87"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-122.51577"
+ y="125.52089">lcore 3</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-162.06549"
+ y="125.4589"
+ id="text4542"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4540"
+ style="font-size:5.64444px;stroke-width:0.75"
+ x="-162.06549"
+ y="125.4589">Enqueue Workers</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-99.705231"
+ y="232.67706"
+ id="text3708-9"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-9"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-99.705231"
+ y="232.67706">lcore 2</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-122.51025"
+ y="232.66466"
+ id="text3708-7"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-8"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-122.51025"
+ y="232.66466">lcore 4</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-145.33035"
+ y="232.65778"
+ id="text3708-78"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-0"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-145.33035"
+ y="232.65778">lcore 6</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-162.06549"
+ y="232.59988"
+ id="text4542-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4540-7"
+ style="font-size:5.64444px;stroke-width:0.75"
+ x="-162.06549"
+ y="232.59988">Dequeue Workers</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:6.35px;font-family:Arial;-inkscape-font-specification:'Arial Bold';text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-177.01665"
+ y="220.07283"
+ id="text5181"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan5179"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-177.01665"
+ y="220.07283">test: inference_ordered</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-weight:bold;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:'Arial Bold';text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-178.63324"
+ y="98.67057"
+ id="text15571"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan15569"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.5;stroke-opacity:1"
+ x="-178.63324"
+ y="98.67057">nb_worker_threads = 2 * MIN(nb_queue_pairs, (lcore_count - 1) / 2)</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-weight:bold;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:'Arial Bold';text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-171.18085"
+ y="89.26754"
+ id="text15571-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan15569-9"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.5;stroke-opacity:1"
+ x="-171.18085"
+ y="89.26754">inferences_per_queue_pair = repetitions / nb_queue_pairs</tspan></text>
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.75;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#RoundedArrow);marker-end:url(#RoundedArrow)"
+ d="m 179.14101,85.813438 0,-12.588618"
+ id="path31090"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1924" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724"
+ width="32.290321"
+ height="11.709678"
+ x="216.5636"
+ y="93.411476"
+ ry="2" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-4"
+ width="32.290321"
+ height="11.709678"
+ x="216.5636"
+ y="116.22478"
+ ry="2" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-6"
+ width="32.290321"
+ height="11.709678"
+ x="216.5636"
+ y="139.03798"
+ ry="2" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.354849;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.709699, 0.709699;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924-6"
+ width="44.145252"
+ height="72.532341"
+ x="210.6364"
+ y="85.813477"
+ ry="4.31247" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-0"
+ width="32.290321"
+ height="11.709678"
+ x="109.42807"
+ y="93.411476"
+ ry="2" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-6-7"
+ width="32.290321"
+ height="11.709678"
+ x="109.42807"
+ y="139.03798"
+ ry="2"
+ inkscape:connector-avoid="true" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-4-8"
+ width="32.290321"
+ height="11.709678"
+ x="109.42807"
+ y="116.22478"
+ ry="2"
+ inkscape:connector-avoid="true" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.354849;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.709699, 0.709699;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924-6-3"
+ width="44.145252"
+ height="72.532341"
+ x="103.50092"
+ y="85.813477"
+ ry="4.31247" />
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/tools/img/mldev_model_ops_subtest_a.svg b/doc/guides/tools/img/mldev_model_ops_subtest_a.svg
new file mode 100644
index 0000000000..ed12cc5a05
--- /dev/null
+++ b/doc/guides/tools/img/mldev_model_ops_subtest_a.svg
@@ -0,0 +1,420 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- SPDX-License-Identifier: BSD-3-Clause -->
+<!-- Copyright (c) 2022 Marvell. -->
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="275mm"
+ height="135mm"
+ viewBox="0 0 275 135"
+ version="1.1"
+ id="svg5"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="mldev_model_ops_subtest_d.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg">
+ <sodipodi:namedview
+ id="namedview7"
+ pagecolor="#ffffff"
+ bordercolor="#000000"
+ borderopacity="0.25"
+ inkscape:showpageshadow="2"
+ inkscape:pageopacity="0.0"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1"
+ inkscape:document-units="mm"
+ showgrid="false"
+ inkscape:zoom="0.72515432"
+ inkscape:cx="372.33454"
+ inkscape:cy="401.29389"
+ inkscape:window-width="1920"
+ inkscape:window-height="986"
+ inkscape:window-x="-11"
+ inkscape:window-y="-11"
+ inkscape:window-maximized="1"
+ inkscape:current-layer="layer1" />
+ <defs
+ id="defs2">
+ <marker
+ style="overflow:visible"
+ id="RoundedArrow"
+ refX="6"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="RoundedArrow"
+ markerWidth="6.1347523"
+ markerHeight="5.9304948"
+ viewBox="0 0 6.1347524 5.9304951"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.7)"
+ d="m -0.21114562,-4.1055728 6.42229122,3.21114561 a 1,1 90 0 1 0,1.78885438 L -0.21114562,4.1055728 A 1.236068,1.236068 31.717474 0 1 -2,3 v -6 a 1.236068,1.236068 148.28253 0 1 1.78885438,-1.1055728 z"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:none"
+ id="path1367" />
+ </marker>
+ </defs>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1">
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 62.203489,25.425124 H 80.823463"
+ id="path1916"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234"
+ inkscape:connection-end="#rect234-07" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 128.19001,25.425124 h 18.6197"
+ id="path1918"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-07"
+ inkscape:connection-end="#rect234-6" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 194.17681,25.425125 h 18.61942"
+ id="path1922"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-6"
+ inkscape:connection-end="#rect234-0-6-6" />
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.633453;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234"
+ width="47.366547"
+ height="17.366549"
+ x="14.836943"
+ y="16.74185"
+ ry="1.4985789"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-25.888845"
+ y="38.514706"
+ id="text290"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-25.888845"
+ y="38.514706">Model 0 / Load</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.633453;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-07"
+ width="47.366547"
+ height="17.366549"
+ x="80.823463"
+ y="16.74185"
+ ry="1.4985789"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-25.888845"
+ y="104.31795"
+ id="text290-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-11"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-25.888845"
+ y="104.31795">Model 0 / Start</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.6329;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-6"
+ width="47.3671"
+ height="17.33604"
+ x="146.80971"
+ y="16.757105"
+ ry="1.4959463" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-25.362436"
+ y="170.39679"
+ id="text290-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-15"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-25.362436"
+ y="170.39679">Model 0 / Stop</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6"
+ width="47.3671"
+ height="17.33604"
+ x="212.79623"
+ y="16.757105"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-25.888849"
+ y="236.47427"
+ id="text290-3-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-25.888849"
+ y="236.47427">Model 0 / Unload</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 236.47978,34.093145 V 44.957249 H 38.520216 v 10.84885"
+ id="path1924"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6"
+ inkscape:connection-end="#rect234-0-6-6-3" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 62.203766,64.474119 H 80.823189"
+ id="path1926"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-3"
+ inkscape:connection-end="#rect234-0-6-6-2" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 128.19029,64.474119 h 18.61942"
+ id="path1928"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-2"
+ inkscape:connection-end="#rect234-0-6-1" />
+ <rect
+ style="fill:#fff6d5;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-3"
+ width="47.3671"
+ height="17.33604"
+ x="14.836666"
+ y="55.806099"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-64.937843"
+ y="38.541786"
+ id="text290-3-2-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-64.937843"
+ y="38.541786">Model 1 / Load</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-2"
+ width="47.3671"
+ height="17.33604"
+ x="80.823189"
+ y="55.806099"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-64.937843"
+ y="104.31795"
+ id="text290-3-2-2-28"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-2"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-64.937843"
+ y="104.31795">Model 1 / Start</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6"
+ width="47.3671"
+ height="17.33604"
+ x="212.79623"
+ y="55.806099"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-64.937843"
+ y="236.47427"
+ id="text290-3-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-64.937843"
+ y="236.47427">Model 1 / Unload</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-1"
+ width="47.3671"
+ height="17.33604"
+ x="146.80971"
+ y="55.806099"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-64.41143"
+ y="170.39679"
+ id="text290-3-2-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-8"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-64.41143"
+ y="170.39679">Model 1 / Stop</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 194.17681,64.474119 h 18.61942"
+ id="path1930"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6"
+ inkscape:connection-start="#rect234-0-6-1" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 236.47978,73.142139 V 83.990988 H 38.520216 V 94.85511"
+ id="path1932"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6-7"
+ inkscape:connection-start="#rect234-0-6" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 62.203766,103.52313 h 9.309711 v 1.3e-4 h 9.309712"
+ id="path1934"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-7" />
+ <rect
+ style="fill:#fff6d5;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-7"
+ width="47.3671"
+ height="17.33604"
+ x="14.836666"
+ y="94.85511"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.98687"
+ y="38.541786"
+ id="text290-3-2-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.98687"
+ y="38.541786">Model N / Load</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0"
+ width="47.3671"
+ height="17.33604"
+ x="80.823189"
+ y="94.855164"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.9869"
+ y="104.31795"
+ id="text290-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.9869"
+ y="104.31795">Model N / Start</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-9"
+ width="47.3671"
+ height="17.33604"
+ x="146.80971"
+ y="94.855164"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.4605"
+ y="170.39679"
+ id="text290-3-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.4605"
+ y="170.39679">Model N / Stop</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.633452;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-5"
+ width="47.366547"
+ height="17.366549"
+ x="212.79651"
+ y="94.839836"
+ ry="1.4985789"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.98682"
+ y="236.47427"
+ id="text290-3-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-3"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.98682"
+ y="236.47427">Model N / Unload</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 128.19029,103.52326 h 18.61942"
+ id="path1936"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 194.17681,103.52326 h 9.30985 v -1.5e-4 h 9.30985"
+ id="path1938"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-5" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#fff6d5;fill-rule:evenodd;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-120.71075"
+ y="220.77164"
+ id="text392"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan390"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';fill:#000000;stroke-width:0.5"
+ x="-120.71075"
+ y="220.77164">mldev: model_ops / subtest D</tspan></text>
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.448724;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect2164"
+ width="259.55127"
+ height="119.55128"
+ x="7.7243652"
+ y="7.7243614"
+ ry="1.17494" />
+ </g>
+</svg>
diff --git a/doc/guides/tools/img/mldev_model_ops_subtest_b.svg b/doc/guides/tools/img/mldev_model_ops_subtest_b.svg
new file mode 100644
index 0000000000..173a2c6c05
--- /dev/null
+++ b/doc/guides/tools/img/mldev_model_ops_subtest_b.svg
@@ -0,0 +1,423 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- SPDX-License-Identifier: BSD-3-Clause -->
+<!-- Copyright (c) 2022 Marvell. -->
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="275mm"
+ height="135mm"
+ viewBox="0 0 275 135"
+ version="1.1"
+ id="svg5"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="mldev_model_ops_subtest_a.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg">
+ <sodipodi:namedview
+ id="namedview7"
+ pagecolor="#ffffff"
+ bordercolor="#000000"
+ borderopacity="0.25"
+ inkscape:showpageshadow="2"
+ inkscape:pageopacity="0.0"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1"
+ inkscape:document-units="mm"
+ showgrid="false"
+ inkscape:zoom="0.72515432"
+ inkscape:cx="277.87189"
+ inkscape:cy="401.29389"
+ inkscape:window-width="1920"
+ inkscape:window-height="986"
+ inkscape:window-x="-11"
+ inkscape:window-y="-11"
+ inkscape:window-maximized="1"
+ inkscape:current-layer="layer1" />
+ <defs
+ id="defs2">
+ <marker
+ style="overflow:visible"
+ id="RoundedArrow"
+ refX="6"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="RoundedArrow"
+ markerWidth="6.1347523"
+ markerHeight="5.9304948"
+ viewBox="0 0 6.1347524 5.9304951"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.7)"
+ d="m -0.21114562,-4.1055728 6.42229122,3.21114561 a 1,1 90 0 1 0,1.78885438 L -0.21114562,4.1055728 A 1.236068,1.236068 31.717474 0 1 -2,3 v -6 a 1.236068,1.236068 148.28253 0 1 1.78885438,-1.1055728 z"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:none"
+ id="path1367" />
+ </marker>
+ </defs>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1">
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.671403;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234"
+ width="53.328598"
+ height="17.328598"
+ x="15.144302"
+ y="12.878438"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-22.00646"
+ y="41.803089"
+ id="text290"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-22.00646"
+ y="41.803089">Model 0 / Load</tspan></text>
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.671403;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-07"
+ width="53.328598"
+ height="17.328598"
+ x="87.552399"
+ y="12.878438"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-22.00646"
+ y="114.21132"
+ id="text290-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-11"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-22.00646"
+ y="114.21132">Model 1 / Load</tspan></text>
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.671403;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-6"
+ width="53.328598"
+ height="17.328598"
+ x="206.5271"
+ y="12.878438"
+ ry="1.4953041" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-22.00646"
+ y="233.18588"
+ id="text290-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-15"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-22.00646"
+ y="233.18588">Model N / Load</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 68.4729,21.542737 H 87.552399"
+ id="path1916"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234"
+ inkscape:connection-end="#rect234-07" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 140.881,21.542737 h 65.6461"
+ id="path1918"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-07"
+ inkscape:connection-end="#rect234-6" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 233.1914,30.207036 0,4.918771 H 41.808601 v 4.918772"
+ id="path1922"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-6"
+ inkscape:connection-end="#rect234-0-6-6" />
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6"
+ width="53.328598"
+ height="17.328598"
+ x="15.144302"
+ y="40.044579"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-49.1726"
+ y="41.619808"
+ id="text290-3-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-49.1726"
+ y="41.619808">Model 0 / Start</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-3"
+ width="53.328598"
+ height="17.328598"
+ x="87.552399"
+ y="40.044579"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-49.1726"
+ y="114.02805"
+ id="text290-3-2-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-49.1726"
+ y="114.02805">Model 1 / Start</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-2"
+ width="53.328598"
+ height="17.328598"
+ x="206.5271"
+ y="40.044579"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-49.1726"
+ y="233.00261"
+ id="text290-3-2-2-28"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-2"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-49.1726"
+ y="233.00261">Model N / Start</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 68.4729,48.708878 H 87.552399"
+ id="path1924"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6"
+ inkscape:connection-end="#rect234-0-6-6-3" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 140.881,48.708878 h 65.6461"
+ id="path1926"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-3"
+ inkscape:connection-end="#rect234-0-6-6-2" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 233.1914,57.373177 V 62.29195 H 41.808599 v 4.918774"
+ id="path1928"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-2"
+ inkscape:connection-end="#rect234-0-6-1" />
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6"
+ width="53.328598"
+ height="17.328598"
+ x="87.552399"
+ y="67.210724"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-75.812294"
+ y="114.12037"
+ id="text290-3-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-75.812294"
+ y="114.12037">Model 1 / Stop</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-1"
+ width="53.328598"
+ height="17.328598"
+ x="15.1443"
+ y="67.210724"
+ ry="1.4953041" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-75.812294"
+ y="41.712139"
+ id="text290-3-2-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-8"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-75.812294"
+ y="41.712139">Model 0 / Stop</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-7"
+ width="53.328598"
+ height="17.328598"
+ x="206.5271"
+ y="67.210724"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-75.812294"
+ y="233.09494"
+ id="text290-3-2-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-75.812294"
+ y="233.09494">Model N / Stop</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 68.472898,75.875023 H 87.552399"
+ id="path1930"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6"
+ inkscape:connection-start="#rect234-0-6-1" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 140.881,75.875023 h 65.6461"
+ id="path1932"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6-7"
+ inkscape:connection-start="#rect234-0-6" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 233.1914,84.539322 0,4.91877 H 41.808602 v 4.91877"
+ id="path1934"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0"
+ inkscape:connection-start="#rect234-0-6-7" />
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0"
+ width="53.328598"
+ height="17.328598"
+ x="15.144303"
+ y="94.376862"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.50489"
+ y="41.803085"
+ id="text290-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.50489"
+ y="41.803085">Model 0 / Unload</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-9"
+ width="53.328598"
+ height="17.328598"
+ x="87.552399"
+ y="94.376862"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.50489"
+ y="114.2113"
+ id="text290-3-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.50489"
+ y="114.2113">Model 1 / Unload</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-5"
+ width="53.328598"
+ height="17.328598"
+ x="206.5271"
+ y="94.376862"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.50489"
+ y="233.18588"
+ id="text290-3-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-3"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.50489"
+ y="233.18588">Model N / Unload</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 68.472901,103.04116 H 87.552399"
+ id="path1936"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-9"
+ inkscape:connection-start="#rect234-0" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 140.881,103.04116 h 65.6461"
+ id="path1938"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-5"
+ inkscape:connection-start="#rect234-0-9" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#fff6d5;fill-rule:evenodd;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-119.78458"
+ y="220.32892"
+ id="text392"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan390"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';fill:#000000;stroke-width:0.5"
+ x="-119.78458"
+ y="220.32892">mldev: model_ops / subtest A</tspan></text>
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.442854;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect3967"
+ width="259.55716"
+ height="119.55714"
+ x="7.7214203"
+ y="7.7214317"
+ ry="1.1806604" />
+ </g>
+</svg>
diff --git a/doc/guides/tools/img/mldev_model_ops_subtest_c.svg b/doc/guides/tools/img/mldev_model_ops_subtest_c.svg
new file mode 100644
index 0000000000..f66f146d05
--- /dev/null
+++ b/doc/guides/tools/img/mldev_model_ops_subtest_c.svg
@@ -0,0 +1,366 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- SPDX-License-Identifier: BSD-3-Clause -->
+<!-- Copyright (c) 2022 Marvell. -->
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="275mm"
+ height="135mm"
+ viewBox="0 0 275 135"
+ version="1.1"
+ id="svg5"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="mldev_model_ops_subtest_c.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg">
+ <sodipodi:namedview
+ id="namedview7"
+ pagecolor="#ffffff"
+ bordercolor="#000000"
+ borderopacity="0.25"
+ inkscape:showpageshadow="2"
+ inkscape:pageopacity="0.0"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1"
+ inkscape:document-units="mm"
+ showgrid="false"
+ inkscape:zoom="0.72515432"
+ inkscape:cx="479.89785"
+ inkscape:cy="401.29389"
+ inkscape:window-width="1920"
+ inkscape:window-height="986"
+ inkscape:window-x="-11"
+ inkscape:window-y="-11"
+ inkscape:window-maximized="1"
+ inkscape:current-layer="layer1" />
+ <defs
+ id="defs2">
+ <marker
+ style="overflow:visible"
+ id="RoundedArrow"
+ refX="6"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="RoundedArrow"
+ markerWidth="6.1347523"
+ markerHeight="5.9304948"
+ viewBox="0 0 6.1347524 5.9304951"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.7)"
+ d="m -0.21114562,-4.1055728 6.42229122,3.21114561 a 1,1 90 0 1 0,1.78885438 L -0.21114562,4.1055728 A 1.236068,1.236068 31.717474 0 1 -2,3 v -6 a 1.236068,1.236068 148.28253 0 1 1.78885438,-1.1055728 z"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:none"
+ id="path1367" />
+ </marker>
+ </defs>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1">
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.671403;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234"
+ width="53.328598"
+ height="17.328598"
+ x="12.6443"
+ y="13.208546"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-22.336561"
+ y="39.303089"
+ id="text290"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-22.336561"
+ y="39.303089">Model 0 / Load</tspan></text>
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.671403;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-07"
+ width="53.328598"
+ height="17.328598"
+ x="85.052399"
+ y="13.208546"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-22.336561"
+ y="111.71132"
+ id="text290-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-11"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-22.336561"
+ y="111.71132">Model 1 / Load</tspan></text>
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.671403;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-6"
+ width="53.328598"
+ height="17.328598"
+ x="204.0271"
+ y="13.208546"
+ ry="1.4953041" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-22.336561"
+ y="230.68588"
+ id="text290-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-15"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-22.336561"
+ y="230.68588">Model N / Load</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 65.972898,21.872845 H 85.052399"
+ id="path1916"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234"
+ inkscape:connection-end="#rect234-07" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 138.381,21.872845 h 65.6461"
+ id="path1918"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-07"
+ inkscape:connection-end="#rect234-6" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 230.6914,30.537144 V 36.33787 H 39.308599 v 5.800726"
+ id="path1922"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-6"
+ inkscape:connection-end="#rect234-0-6-6" />
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6"
+ width="53.328598"
+ height="17.328598"
+ x="12.6443"
+ y="42.138596"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-51.266617"
+ y="39.119808"
+ id="text290-3-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-51.266617"
+ y="39.119808">Model 0 / Start</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-3"
+ width="53.328598"
+ height="17.328598"
+ x="85.052399"
+ y="42.138596"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-51.266617"
+ y="111.52805"
+ id="text290-3-2-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-51.266617"
+ y="111.52805">Model 0 / Stop</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 65.972898,50.802895 H 85.052399"
+ id="path1924"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6"
+ inkscape:connection-end="#rect234-0-6-6-3" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 184.94759,79.732941 H 204.0271"
+ id="path1930"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6"
+ inkscape:connection-start="#rect234-0-6-1" />
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0"
+ width="53.328598"
+ height="17.328598"
+ x="12.644301"
+ y="99.998688"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-109.12671"
+ y="39.303085"
+ id="text290-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-109.12671"
+ y="39.303085">Model 0 / Unload</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-9"
+ width="53.328598"
+ height="17.328598"
+ x="85.052399"
+ y="99.998688"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-109.12671"
+ y="111.7113"
+ id="text290-3-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-109.12671"
+ y="111.7113">Model 1 / Unload</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-5"
+ width="53.328598"
+ height="17.328598"
+ x="204.0271"
+ y="99.998688"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-109.12671"
+ y="230.68588"
+ id="text290-3-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-3"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-109.12671"
+ y="230.68588">Model N / Unload</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 65.972899,108.66299 h 19.0795"
+ id="path1936"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-9"
+ inkscape:connection-start="#rect234-0" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 138.381,108.66299 h 65.6461"
+ id="path1938"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-5"
+ inkscape:connection-start="#rect234-0-9" />
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6"
+ width="53.328598"
+ height="17.328598"
+ x="204.0271"
+ y="71.068642"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-79.670258"
+ y="230.59494"
+ id="text290-3-2-2-28"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-2"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-79.670258"
+ y="230.59494">Model N / Stop</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-1"
+ width="53.328598"
+ height="17.328598"
+ x="131.61899"
+ y="71.068642"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-80.196663"
+ y="158.0945"
+ id="text290-3-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-80.196663"
+ y="158.0945">Model N / Start</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 138.381,50.802895 h 19.90229 v 20.265747"
+ id="path1158"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-3"
+ inkscape:connection-end="#rect234-0-6-1" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 230.6914,88.39724 v 5.800724 H 39.3086 v 5.800724"
+ id="path1160"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6"
+ inkscape:connection-end="#rect234-0" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#fff6d5;fill-rule:evenodd;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-123.28999"
+ y="217.99152"
+ id="text392"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan390"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';fill:#000000;stroke-width:0.5"
+ x="-123.28999"
+ y="217.99152">mldev: model_ops / subtest C</tspan></text>
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.441855;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect3015"
+ width="259.55814"
+ height="119.55814"
+ x="7.720932"
+ y="7.7209282"
+ ry="1.1638433" />
+ </g>
+</svg>
diff --git a/doc/guides/tools/img/mldev_model_ops_subtest_d.svg b/doc/guides/tools/img/mldev_model_ops_subtest_d.svg
new file mode 100644
index 0000000000..3e2b89ad25
--- /dev/null
+++ b/doc/guides/tools/img/mldev_model_ops_subtest_d.svg
@@ -0,0 +1,424 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- SPDX-License-Identifier: BSD-3-Clause -->
+<!-- Copyright (c) 2022 Marvell. -->
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="275mm"
+ height="135mm"
+ viewBox="0 0 275 135"
+ version="1.1"
+ id="svg5"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="mldev_model_ops_subtest_b.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg">
+ <sodipodi:namedview
+ id="namedview7"
+ pagecolor="#ffffff"
+ bordercolor="#000000"
+ borderopacity="0.25"
+ inkscape:showpageshadow="2"
+ inkscape:pageopacity="0.0"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1"
+ inkscape:document-units="mm"
+ showgrid="false"
+ inkscape:zoom="0.72515432"
+ inkscape:cx="324.06895"
+ inkscape:cy="295.1096"
+ inkscape:window-width="1920"
+ inkscape:window-height="986"
+ inkscape:window-x="-11"
+ inkscape:window-y="-11"
+ inkscape:window-maximized="1"
+ inkscape:current-layer="layer1" />
+ <defs
+ id="defs2">
+ <marker
+ style="overflow:visible"
+ id="RoundedArrow"
+ refX="6"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="RoundedArrow"
+ markerWidth="6.1347523"
+ markerHeight="5.9304948"
+ viewBox="0 0 6.1347524 5.9304951"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.7)"
+ d="m -0.21114562,-4.1055728 6.42229122,3.21114561 a 1,1 90 0 1 0,1.78885438 L -0.21114562,4.1055728 A 1.236068,1.236068 31.717474 0 1 -2,3 v -6 a 1.236068,1.236068 148.28253 0 1 1.78885438,-1.1055728 z"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:none"
+ id="path1367" />
+ </marker>
+ </defs>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1">
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.633453;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234"
+ width="47.366547"
+ height="17.366549"
+ x="14.864025"
+ y="14.24185"
+ ry="1.4985789"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-23.388845"
+ y="38.541786"
+ id="text290"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-23.388845"
+ y="38.541786">Model 0 / Load</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.633453;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-07"
+ width="47.366547"
+ height="17.366549"
+ x="81.026939"
+ y="14.24185"
+ ry="1.4985789"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-23.388845"
+ y="104.52142"
+ id="text290-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-11"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-23.388845"
+ y="104.52142">Model 0 / Start</tspan></text>
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.6329;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-6"
+ width="47.3671"
+ height="17.33604"
+ x="147.18958"
+ y="14.257105"
+ ry="1.4959463" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-23.388849"
+ y="170.86761"
+ id="text290-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-15"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-23.388849"
+ y="170.86761">Model 1 / Load</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 62.230572,22.925124 H 81.026939"
+ id="path1916"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234"
+ inkscape:connection-end="#rect234-07" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 128.39349,22.925124 h 18.79609"
+ id="path1918"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-07"
+ inkscape:connection-end="#rect234-6" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 194.55667,22.925125 18.79581,0"
+ id="path1922"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-6"
+ inkscape:connection-end="#rect234-0-6-6" />
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6"
+ width="47.3671"
+ height="17.33604"
+ x="213.35248"
+ y="14.257105"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-23.388849"
+ y="236.84723"
+ id="text290-3-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-23.388849"
+ y="236.84723">Model 1 / Start</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 237.03603,31.593145 0,5.328675 H 38.547297 v 5.313421"
+ id="path1924"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6"
+ inkscape:connection-end="#rect234-0-6-6-3" />
+ <rect
+ style="fill:#fff6d5;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-3"
+ width="47.3671"
+ height="17.33604"
+ x="14.863747"
+ y="42.235241"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-51.366989"
+ y="38.541786"
+ id="text290-3-2-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-51.366989"
+ y="38.541786">Model N / Load</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-2"
+ width="47.3671"
+ height="17.33604"
+ x="81.026657"
+ y="42.235241"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-51.366989"
+ y="104.52142"
+ id="text290-3-2-2-28"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-2"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-51.366989"
+ y="104.52142">Model N / Start</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 62.230846,50.903261 H 81.026657"
+ id="path1926"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-3"
+ inkscape:connection-end="#rect234-0-6-6-2" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 128.39376,50.903261 h 42.47937 v 19.294584"
+ id="path1928"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-2"
+ inkscape:connection-end="#rect234-0-6-1" />
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6"
+ width="47.3671"
+ height="17.33604"
+ x="213.35248"
+ y="70.197845"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-79.32959"
+ y="237.03052"
+ id="text290-3-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-79.32959"
+ y="237.03052">Model N / Unload</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-1"
+ width="47.3671"
+ height="17.33604"
+ x="147.18958"
+ y="70.197845"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-78.803177"
+ y="170.77666"
+ id="text290-3-2-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-8"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-78.803177"
+ y="170.77666">Model N / Stop</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 194.55667,78.865866 h 18.79581"
+ id="path1930"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6"
+ inkscape:connection-start="#rect234-0-6-1" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 237.03603,87.533886 v 5.313417 H 38.547297 v 5.328677"
+ id="path1932"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6-7"
+ inkscape:connection-start="#rect234-0-6" />
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-7"
+ width="47.3671"
+ height="17.33604"
+ x="14.863747"
+ y="98.17598"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-106.78132"
+ y="38.450832"
+ id="text290-3-2-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-106.78132"
+ y="38.450832">Model 1 / Stop</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 62.230846,106.844 H 81.026657"
+ id="path1934"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0"
+ inkscape:connection-start="#rect234-0-6-7" />
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0"
+ width="47.3671"
+ height="17.33604"
+ x="81.026657"
+ y="98.17598"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-107.30773"
+ y="104.7047"
+ id="text290-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-107.30773"
+ y="104.7047">Model 1 / Unload</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-9"
+ width="47.3671"
+ height="17.33604"
+ x="147.18958"
+ y="98.17598"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-106.78133"
+ y="170.77666"
+ id="text290-3-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-106.78133"
+ y="170.77666">Model 0 / Stop</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.633452;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-5"
+ width="47.366547"
+ height="17.366549"
+ x="213.35277"
+ y="98.160721"
+ ry="1.4985789"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-107.30773"
+ y="237.03053"
+ id="text290-3-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-3"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-107.30773"
+ y="237.03053">Model 0 / Unload</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 128.39376,106.844 h 18.79582"
+ id="path1936"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-9"
+ inkscape:connection-start="#rect234-0" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 194.55667,106.844 9.39805,0 v 0 h 9.39805"
+ id="path1938"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-5"
+ inkscape:connection-start="#rect234-0-9" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#fff6d5;fill-rule:evenodd;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-122.00021"
+ y="222.26495"
+ id="text392"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan390"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';fill:#000000;stroke-width:0.5"
+ x="-122.00021"
+ y="222.26495">mldev: model_ops / subest B</tspan></text>
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.462341;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect3475"
+ width="259.53769"
+ height="119.53766"
+ x="7.7311554"
+ y="7.7311707"
+ ry="1.2186421" />
+ </g>
+</svg>
diff --git a/doc/guides/tools/index.rst b/doc/guides/tools/index.rst
index f1f5b94c8c..6f84fc31ff 100644
--- a/doc/guides/tools/index.rst
+++ b/doc/guides/tools/index.rst
@@ -21,4 +21,5 @@ DPDK Tools User Guides
comp_perf
testeventdev
testregex
+ testmldev
dts
diff --git a/doc/guides/tools/testmldev.rst b/doc/guides/tools/testmldev.rst
new file mode 100644
index 0000000000..845c2d9381
--- /dev/null
+++ b/doc/guides/tools/testmldev.rst
@@ -0,0 +1,441 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright (c) 2022 Marvell.
+
+dpdk-test-mldev Application
+===========================
+
+The ``dpdk-test-mldev`` tool is a Data Plane Development Kit (DPDK) application that allows testing
+various mldev use cases. This application has a generic framework to add new mldev based test cases
+to verify functionality and measure the performance of inference execution on DPDK ML devices.
+
+
+Application and Options
+-----------------------
+
+The application has a number of command line options:
+
+.. code-block:: console
+
+ dpdk-test-mldev [EAL Options] -- [application options]
+
+EAL Options
+~~~~~~~~~~~
+
+The following are the EAL command-line options that can be used with the ``dpdk-test-mldev``
+application. See the DPDK Getting Started Guides for more information on these options.
+
+* ``-c <COREMASK>`` or ``-l <CORELIST>``
+
+ Set the hexadecimal bitmask of the cores to run on. The corelist is a list of cores to use.
+
+* ``-a <PCI_ID>``
+
+ Attach a PCI based ML device. Specific to drivers using a PCI based ML devices.
+
+* ``--vdev <driver>``
+
+ Add a virtual mldev device. Specific to drivers using a ML virtual device.
+
+
+Application Options
+~~~~~~~~~~~~~~~~~~~
+
+The following are the command-line options supported by the test application.
+
+* ``--test <name>``
+
+ ML tests are divided into two groups, Model and Device tests and Inference tests. Test
+ name one of the following supported tests.
+
+ **ML Device Tests** ::
+
+ device_ops
+
+ **ML Model Tests** ::
+
+ model_ops
+
+ **ML Inference Tests** ::
+
+ inference_ordered
+ inference_interleave
+
+* ``--dev_id <n>``
+
+ Set the device id of the ML device to be used for the test. Default value is `0`.
+
+* ``--socket_id <n>``
+
+ Set the socket id of the application resources. Default value is `SOCKET_ID_ANY`.
+
+* ``--debug``
+
+ Enable the tests to run in debug mode.
+
+* ``--models <model_list>``
+
+ Set the list of model files to be used for the tests. Application expects the
+ ``model_list`` in comma separated form (i.e. ``--models model_A.bin,model_B.bin``).
+ Maximum number of models supported by the test is ``8``.
+
+* ``--filelist <file_list>``
+
+ Set the list of model, input, output and reference files to be used for the tests.
+ Application expects the ``file_list`` to be in comma separated form
+ (i.e. ``--filelist <model,input,output>[,reference]``).
+
+ Multiple filelist entries can be specified when running the tests with multiple models.
+ Both quantized and dequantized outputs are written to the disk. Dequantized output file
+ would have the name specified by the user through ``--filelist`` option. A suffix ``.q``
+ is appended to quantized output filename. Maximum number of filelist entries supported
+ by the test is ``8``.
+
+* ``--repetitions <n>``
+
+ Set the number of inference repetitions to be executed in the test per each model. Default
+ value is `1`.
+
+* ``--burst_size <n>``
+
+ Set the burst size to be used when enqueuing / dequeuing inferences. Default value is `1`.
+
+* ``--queue_pairs <n>``
+
+ Set the number of queue-pairs to be used for inference enqueue and dequeue operations.
+ Default value is `1`.
+
+* ``--queue_size <n>``
+
+ Set the size of queue-pair to be created for inference enqueue / dequeue operations.
+ Queue size would translate into `rte_ml_dev_qp_conf::nb_desc` field during queue-pair
+ creation. Default value is `1`.
+
+* ``--batches <n>``
+
+ Set the number batches in the input file provided for inference run. When not specified
+ the test would assume the number of batches is equal to the batch size of the model.
+
+* ``--tolerance <n>``
+
+ Set the tolerance value in percentage to be used for output validation. Default value
+ is `0`.
+
+* ``--stats``
+
+ Enable reporting device extended stats.
+
+
+ML Device Tests
+-------------------------
+
+ML device tests are functional tests to validate ML device APIs. Device tests validate the ML device
+handling APIs configure, close, start and stop APIs.
+
+
+Application Options
+~~~~~~~~~~~~~~~~~~~
+
+Supported command line options for the `model_ops` test are following::
+
+ --debug
+ --test
+ --dev_id
+ --socket_id
+ --queue_pairs
+ --queue_size
+
+
+DEVICE_OPS Test
+~~~~~~~~~~~~~~~
+
+Device ops test validates the device configuration and reconfiguration support. The test configures
+ML device based on the option ``--queue_pairs`` and ``--queue_size`` specified by the user, and
+later reconfigures the ML device with the number of queue pairs and queue size based the maximum
+specified through the device info.
+
+
+Example
+^^^^^^^
+
+Command to run device_ops test:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=device_ops
+
+
+Command to run device_ops test with user options:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=device_ops --queue_pairs <M> --queue_size <N>
+
+
+ML Model Tests
+-------------------------
+
+Model tests are functional tests to validate ML model APIs. Model tests validate the functioning
+of APIs to load, start, stop and unload ML models.
+
+
+Application Options
+~~~~~~~~~~~~~~~~~~~
+
+Supported command line options for the `model_ops` test are following::
+
+ --debug
+ --test
+ --dev_id
+ --socket_id
+ --models
+
+
+List of model files to be used for the `model_ops` test can be specified through the option
+``--models <model_list>`` as a comma separated list. Maximum number of models supported in
+the test is `8`.
+
+.. Note::
+
+ * The ``--models <model_list>`` is a mandatory option for running this test.
+ * Options not supported by the test are ignored if specified.
+
+
+MODEL_OPS Test
+~~~~~~~~~~~~~~
+
+The test is a collection of multiple sub-tests, each with a different order of slow-path
+operations when handling with `N` number of models.
+
+
+**Sub-test A:** executes the sequence of load / start / stop / unload for a model in order,
+followed by next model.
+.. _figure_mldev_model_ops_subtest_a:
+
+.. figure:: img/mldev_model_ops_subtest_a.*
+
+ Execution sequence of model_ops subtest A.
+
+
+**Sub-test B:** executes load for all models, followed by a start for all models. Upon successful
+start of all models, stop is invoked for all models followed by unload.
+.. _figure_mldev_model_ops_subtest_b:
+
+.. figure:: img/mldev_model_ops_subtest_b.*
+
+ Execution sequence of model_ops subtest B.
+
+
+**Sub-test C:** loads all models, followed by a start and stop of all models in order. Upon
+completion of stop, unload is invoked for all models.
+.. _figure_mldev_model_ops_subtest_c:
+
+.. figure:: img/mldev_model_ops_subtest_c.*
+
+ Execution sequence of model_ops subtest C.
+
+
+**Sub-test D:** executes load and start for all models available. Upon successful start of all
+models, stop and stop is executed for the models.
+.. _figure_mldev_model_ops_subtest_d:
+
+.. figure:: img/mldev_model_ops_subtest_d.*
+
+ Execution sequence of model_ops subtest D.
+
+
+Example
+^^^^^^^
+
+Command to run model_ops test:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=model_ops --models model_1.bin,model_2.bin,model_3.bin, model_4.bin
+
+
+ML Inference Tests
+------------------
+
+Inference tests are a set of tests to validate end-to-end inference execution on ML device.
+These tests executes the full sequence of operations required to run inferences with one or
+multiple models.
+
+Application Options
+~~~~~~~~~~~~~~~~~~~
+
+Supported command line options for inference tests are following::
+
+ --debug
+ --test
+ --dev_id
+ --socket_id
+ --filelist
+ --repetitions
+ --burst_size
+ --queue_pairs
+ --queue_size
+ --batches
+ --tolerance
+ --stats
+
+
+List of files to be used for the inference tests can be specified through the option
+``--filelist <file_list>`` as a comma separated list. A filelist entry would be of the format
+``--filelist <model_file,input_file,output_file>[,reference_file]`` and is used to specify the
+list of files required to test with a single model. Multiple filelist entries are supported by
+the test, one entry per model. Maximum number of file entries supported by the test is `8`.
+
+When ``--burst_size <num>`` option is specified for the test, enqueue and dequeue burst would
+try to enqueue or dequeue ``num`` number of inferences per each call respectively.
+
+In the inference test, a pair of lcores are mapped to each queue pair. Minimum number of lcores
+required for the tests is equal to ``(queue_pairs * 2 + 1)``.
+
+Output validation of inference would be enabled only when a reference file is specified through
+the ``--filelist`` option. Application would additionally consider the tolerance value provided
+through ``--tolerance`` option during validation. When the tolerance values is 0, CRC32 hash of
+inference output and reference output are compared. When the tolerance is non-zero, element wise
+comparison of output is performed. Validation is considered as successful only when all the
+elements of the output tensor are with in the tolerance range specified.
+
+When ``--debug`` option is specified, tests are run in debug mode.
+
+Enabling ``--stats`` would print the extended stats supported by the driver.
+
+.. Note::
+
+ * The ``--filelist <file_list>`` is a mandatory option for running inference tests.
+ * Options not supported by the tests are ignored if specified.
+ * Element wise comparison is not supported when the output dtype is either fp8, fp16
+ or bfloat16. This is applicable only when the tolerance is greater than zero and for
+ pre-quantized models only.
+
+
+INFERENCE_ORDERED Test
+~~~~~~~~~~~~~~~~~~~~~~
+
+This is a functional test for validating the end-to-end inference execution on ML device. This
+test configures ML device and queue pairs as per the queue-pair related options (queue_pairs and
+queue_size) specified by the user. Upon successful configuration of the device and queue pairs,
+the first model specified through the filelist is loaded to the device and inferences are enqueued
+by a pool of worker threads to the ML device. Total number of inferences enqueued for the model
+are equal to the repetitions specified. A dedicated pool of worker threads would dequeue the
+inferences from the device. The model is unloaded upon completion of all inferences for the model.
+The test would continue loading and executing inference requests for all models specified
+through ``filelist`` option in an ordered manner.
+
+.. _figure_mldev_inference_ordered:
+
+.. figure:: img/mldev_inference_ordered.*
+
+ Execution of inference_ordered on single model.
+
+
+Example
+^^^^^^^
+
+Example command to run inference_ordered test:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_ordered --filelist model.bin,input.bin,output.bin
+
+Example command to run inference_ordered with output validation using tolerance of `1%``:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_ordered --filelist model.bin,input.bin,output.bin,reference.bin \
+ --tolerance 1.0
+
+Example command to run inference_ordered test with multiple queue-pairs and queue size:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_ordered --filelist model.bin,input.bin,output.bin \
+ --queue_pairs 4 --queue_size 16
+
+Example command to run inference_ordered test with a specific burst size:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_ordered --filelist model.bin,input.bin,output.bin \
+ --burst_size 12
+
+
+INFERENCE_INTERLEAVE Test
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This is a stress test for validating the end-to-end inference execution on ML device. The test
+configures the ML device and queue pairs as per the queue-pair related options (queue_pairs
+and queue_size) specified by the user. Upon successful configuration of the device and queue
+pairs, all models specified through the filelist are loaded to the device. Inferences for multiple
+models are enqueued by a pool of worker threads in parallel. Inference execution by the device is
+interleaved between multiple models. Total number of inferences enqueued for a model are equal to
+the repetitions specified. An additional pool of threads would dequeue the inferences from the
+device. Models would be unloaded upon completion of inferences for all models loaded.
+
+
+.. _figure_mldev_inference_interleave:
+
+.. figure:: img/mldev_inference_interleave.*
+
+ Execution of inference_interleave on single model.
+
+
+Example
+^^^^^^^
+
+Example command to run inference_interleave test:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_interleave --filelist model.bin,input.bin,output.bin
+
+
+Example command to run inference_interleave test with multiple models:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_interleave --filelist model_A.bin,input_A.bin,output_A.bin \
+ --filelist model_B.bin,input_B.bin,output_B.bin
+
+
+Example command to run inference_interleave test with multiple models ad output validation
+using tolerance of `2.0%``:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_interleave \
+ --filelist model_A.bin,input_A.bin,output_A.bin,reference_A.bin \
+ --filelist model_B.bin,input_B.bin,output_B.bin,reference_B.bin \
+ --tolerance 2.0
+
+Example command to run inference_interleave test with multiple queue-pairs and queue size
+and burst size:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_interleave --filelist model.bin,input.bin,output.bin \
+ --queue_pairs 8 --queue_size 12 --burst_size 16
+
+
+Debug mode
+----------
+
+ML tests can be executed in debug mode by enabling the option ``--debug``. Execution of tests in
+debug mode would enable additional prints.
+
+When a validation failure is observed, output from that buffer is written to the disk, with the
+filenames having similar convention when the test has passed. Additionally index of the buffer
+would be appended to the filenames.
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* RE: [PATCH v4 12/12] app/mldev: add documentation for mldev test cases
2023-02-07 15:49 ` [PATCH v4 12/12] app/mldev: add documentation for mldev test cases Srikanth Yalavarthi
@ 2023-02-15 12:26 ` Shivah Shankar Shankar Narayan Rao
2023-03-03 6:07 ` Anup Prabhu
1 sibling, 0 replies; 122+ messages in thread
From: Shivah Shankar Shankar Narayan Rao @ 2023-02-15 12:26 UTC (permalink / raw)
To: Srikanth Yalavarthi, Thomas Monjalon, Srikanth Yalavarthi
Cc: dev, Jerin Jacob Kollanukkaran, Anup Prabhu, Prince Takkar,
Parijat Shukla
[-- Attachment #1: Type: text/plain, Size: 815 bytes --]
> -----Original Message-----
> From: Srikanth Yalavarthi <syalavarthi@marvell.com>
> Sent: Tuesday, February 7, 2023 9:20 PM
> To: Thomas Monjalon <thomas@monjalon.net>; Srikanth Yalavarthi
> <syalavarthi@marvell.com>
> Cc: dev@dpdk.org; Shivah Shankar Shankar Narayan Rao
> <sshankarnara@marvell.com>; Jerin Jacob Kollanukkaran
> <jerinj@marvell.com>; Anup Prabhu <aprabhu@marvell.com>; Prince Takkar
> <ptakkar@marvell.com>; Parijat Shukla <pshukla@marvell.com>
> Subject: [PATCH v4 12/12] app/mldev: add documentation for mldev test
> cases
>
> Added documentation specific to mldev test cases. Added details
> about all test cases and option supported by individual tests.
>
> Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
Acked-by: Shivah Shankar S <sshankarnara@marvell.com>
[-- Attachment #2: winmail.dat --]
[-- Type: application/ms-tnef, Size: 23611 bytes --]
^ permalink raw reply [flat|nested] 122+ messages in thread
* RE: [PATCH v4 12/12] app/mldev: add documentation for mldev test cases
2023-02-07 15:49 ` [PATCH v4 12/12] app/mldev: add documentation for mldev test cases Srikanth Yalavarthi
2023-02-15 12:26 ` Shivah Shankar Shankar Narayan Rao
@ 2023-03-03 6:07 ` Anup Prabhu
1 sibling, 0 replies; 122+ messages in thread
From: Anup Prabhu @ 2023-03-03 6:07 UTC (permalink / raw)
To: Srikanth Yalavarthi, Thomas Monjalon, Srikanth Yalavarthi
Cc: dev, Shivah Shankar Shankar Narayan Rao,
Jerin Jacob Kollanukkaran, Prince Takkar, Parijat Shukla
[-- Attachment #1: Type: text/plain, Size: 1539 bytes --]
> -----Original Message-----
> From: Srikanth Yalavarthi <syalavarthi@marvell.com>
> Sent: Tuesday, February 7, 2023 9:20 PM
> To: Thomas Monjalon <thomas@monjalon.net>; Srikanth Yalavarthi
> <syalavarthi@marvell.com>
> Cc: dev@dpdk.org; Shivah Shankar Shankar Narayan Rao
> <sshankarnara@marvell.com>; Jerin Jacob Kollanukkaran
> <jerinj@marvell.com>; Anup Prabhu <aprabhu@marvell.com>; Prince Takkar
> <ptakkar@marvell.com>; Parijat Shukla <pshukla@marvell.com>
> Subject: [PATCH v4 12/12] app/mldev: add documentation for mldev test
> cases
>
> Added documentation specific to mldev test cases. Added details
> about all test cases and option supported by individual tests.
>
> Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
> ---
> MAINTAINERS | 1 +
> app/test-mldev/test_inference_common.c | 27 +-
> .../tools/img/mldev_inference_interleave.svg | 669 ++++++++++++++++++
> .../tools/img/mldev_inference_ordered.svg | 528 ++++++++++++++
> .../tools/img/mldev_model_ops_subtest_a.svg | 420 +++++++++++
> .../tools/img/mldev_model_ops_subtest_b.svg | 423 +++++++++++
> .../tools/img/mldev_model_ops_subtest_c.svg | 366 ++++++++++
> .../tools/img/mldev_model_ops_subtest_d.svg | 424 +++++++++++
> doc/guides/tools/index.rst | 1 +
> doc/guides/tools/testmldev.rst | 441 ++++++++++++
> 10 files changed, 3279 insertions(+), 21 deletions(-)
>
Acked-by: Anup Prabhu <aprabhu@marvell.com>
[-- Attachment #2: winmail.dat --]
[-- Type: application/ms-tnef, Size: 23175 bytes --]
^ permalink raw reply [flat|nested] 122+ messages in thread
* [PATCH v5 00/12] Implementation of mldev test application
2022-11-29 7:07 [PATCH v1 00/12] implement mldev test application Srikanth Yalavarthi
` (12 preceding siblings ...)
2023-02-07 15:49 ` [PATCH v4 00/12] Implementation of mldev test application Srikanth Yalavarthi
@ 2023-03-10 8:09 ` Srikanth Yalavarthi
2023-03-10 8:09 ` [PATCH v5 01/12] app/mldev: implement test framework for mldev Srikanth Yalavarthi
` (11 more replies)
2023-03-11 15:08 ` [PATCH v6 00/12] Implementation of mldev test application Srikanth Yalavarthi
2023-03-16 21:14 ` [PATCH v7 00/11] Implementation of mldev test application Srikanth Yalavarthi
15 siblings, 12 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-10 8:09 UTC (permalink / raw)
Cc: dev, syalavarthi, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Machine learning device APIs test application
=============================================
This series of patches introduces a test application for machine
learning device APIs. A test framework is implemented with multiple
test enabled, to validate the device, model and fast-path functions.
New tests can be added using the test framework.
List of tests supported
-----------------------
1) device_ops: Test case to validate device re-configuration
2) model_ops: Collection of 4 sub-tests to validate model slow APIs.
Each sub-test would invoke the slow path model APIs (load / start /
stop / unload) in different order.
3) inference_ordered: Test case to validate execution of end-to-end
inferences on ML device with one active model at a time. This test
can execute inference requests for multiple models, with inferences
for a model executed after completion of inferences for a previously
loaded model.
4) inference_interleave: Test case to validate end-to-end inferences
with multiple active models concurrently. This case would work as a
stress test to validate ML device.
Options supported the by tests include burst size for enqueuing and
dequeuing inference requests, enabling multiple queue pairs with a
user specified value for queue size. Support is also enabled for
batch inferencing, output validation and statistics.
v5:
* Cleanup of header includes
* Addressed issues with unclean patches
* Changed input/output size variables to uint32_t
* Rebase over main branch
v4:
* Updated model_id as uint16_t
* Updated license info in SVG files
* Updated release notes
v3:
* Code rebase
Srikanth Yalavarthi (12):
app/mldev: implement test framework for mldev
app/mldev: add common test functions
app/mldev: add test case to validate device ops
app/mldev: add test case to validate model ops
app/mldev: add ordered inference test case
app/mldev: add test case to interleave inferences
app/mldev: enable support for burst inferences
app/mldev: enable support for queue pairs and size
app/mldev: enable support for inference batches
app/mldev: enable support for inference validation
app/mldev: enable reporting stats in mldev app
app/mldev: add documentation for mldev test cases
MAINTAINERS | 2 +
app/meson.build | 1 +
app/test-mldev/meson.build | 24 +
app/test-mldev/ml_common.h | 29 +
app/test-mldev/ml_main.c | 113 ++
app/test-mldev/ml_options.c | 325 +++++
app/test-mldev/ml_options.h | 57 +
app/test-mldev/ml_test.c | 41 +
app/test-mldev/ml_test.h | 76 ++
app/test-mldev/parser.c | 380 ++++++
app/test-mldev/parser.h | 55 +
app/test-mldev/test_common.c | 136 ++
app/test-mldev/test_common.h | 27 +
app/test-mldev/test_device_ops.c | 228 ++++
app/test-mldev/test_device_ops.h | 17 +
app/test-mldev/test_inference_common.c | 1122 +++++++++++++++++
app/test-mldev/test_inference_common.h | 74 ++
app/test-mldev/test_inference_interleave.c | 118 ++
app/test-mldev/test_inference_ordered.c | 116 ++
app/test-mldev/test_model_common.c | 164 +++
app/test-mldev/test_model_common.h | 46 +
app/test-mldev/test_model_ops.c | 428 +++++++
app/test-mldev/test_model_ops.h | 20 +
doc/guides/rel_notes/release_23_03.rst | 8 +
.../tools/img/mldev_inference_interleave.svg | 669 ++++++++++
.../tools/img/mldev_inference_ordered.svg | 528 ++++++++
.../tools/img/mldev_model_ops_subtest_a.svg | 420 ++++++
.../tools/img/mldev_model_ops_subtest_b.svg | 423 +++++++
.../tools/img/mldev_model_ops_subtest_c.svg | 366 ++++++
.../tools/img/mldev_model_ops_subtest_d.svg | 424 +++++++
doc/guides/tools/index.rst | 1 +
doc/guides/tools/testmldev.rst | 441 +++++++
32 files changed, 6879 insertions(+)
create mode 100644 app/test-mldev/meson.build
create mode 100644 app/test-mldev/ml_common.h
create mode 100644 app/test-mldev/ml_main.c
create mode 100644 app/test-mldev/ml_options.c
create mode 100644 app/test-mldev/ml_options.h
create mode 100644 app/test-mldev/ml_test.c
create mode 100644 app/test-mldev/ml_test.h
create mode 100644 app/test-mldev/parser.c
create mode 100644 app/test-mldev/parser.h
create mode 100644 app/test-mldev/test_common.c
create mode 100644 app/test-mldev/test_common.h
create mode 100644 app/test-mldev/test_device_ops.c
create mode 100644 app/test-mldev/test_device_ops.h
create mode 100644 app/test-mldev/test_inference_common.c
create mode 100644 app/test-mldev/test_inference_common.h
create mode 100644 app/test-mldev/test_inference_interleave.c
create mode 100644 app/test-mldev/test_inference_ordered.c
create mode 100644 app/test-mldev/test_model_common.c
create mode 100644 app/test-mldev/test_model_common.h
create mode 100644 app/test-mldev/test_model_ops.c
create mode 100644 app/test-mldev/test_model_ops.h
create mode 100644 doc/guides/tools/img/mldev_inference_interleave.svg
create mode 100644 doc/guides/tools/img/mldev_inference_ordered.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_a.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_b.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_c.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_d.svg
create mode 100644 doc/guides/tools/testmldev.rst
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v5 01/12] app/mldev: implement test framework for mldev
2023-03-10 8:09 ` [PATCH v5 00/12] Implementation of mldev test application Srikanth Yalavarthi
@ 2023-03-10 8:09 ` Srikanth Yalavarthi
2023-03-10 8:09 ` [PATCH v5 02/12] app/mldev: add common test functions Srikanth Yalavarthi
` (10 subsequent siblings)
11 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-10 8:09 UTC (permalink / raw)
To: Thomas Monjalon, Srikanth Yalavarthi
Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Implemented framework for mldev test application. New test cases
can be added using the framework. Support is also enabled to add
options specific to the test cases. User can launch the tests by
specifying the name of test as part of launch arguments.
Code to parse command line arguments is imported from
test-eventdev, with support to parse additional data types.
Common arguments supported include:
test : name of the test application to run
dev_id : device id of the ML device
socket_id : socket_id of application resources
debug : enable debugging
help : print help
Sample launch command:
./dpdk-test-mldev -- --test <testname> --dev_id <dev_id> \
--socket_id <socket_id>
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
MAINTAINERS | 1 +
app/meson.build | 1 +
app/test-mldev/meson.build | 17 ++
app/test-mldev/ml_common.h | 29 ++
app/test-mldev/ml_main.c | 113 ++++++++
app/test-mldev/ml_options.c | 155 ++++++++++
app/test-mldev/ml_options.h | 31 ++
app/test-mldev/ml_test.c | 41 +++
app/test-mldev/ml_test.h | 76 +++++
app/test-mldev/parser.c | 380 +++++++++++++++++++++++++
app/test-mldev/parser.h | 55 ++++
doc/guides/rel_notes/release_23_03.rst | 8 +
12 files changed, 907 insertions(+)
create mode 100644 app/test-mldev/meson.build
create mode 100644 app/test-mldev/ml_common.h
create mode 100644 app/test-mldev/ml_main.c
create mode 100644 app/test-mldev/ml_options.c
create mode 100644 app/test-mldev/ml_options.h
create mode 100644 app/test-mldev/ml_test.c
create mode 100644 app/test-mldev/ml_test.h
create mode 100644 app/test-mldev/parser.c
create mode 100644 app/test-mldev/parser.h
diff --git a/MAINTAINERS b/MAINTAINERS
index 7c6fea7c28..1914c4d614 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -480,6 +480,7 @@ Machine Learning device API - EXPERIMENTAL
M: Srikanth Yalavarthi <syalavarthi@marvell.com>
F: lib/mldev/
F: doc/guides/prog_guide/mldev.rst
+F: app/test-mldev
DMA device API - EXPERIMENTAL
M: Chengwen Feng <fengchengwen@huawei.com>
diff --git a/app/meson.build b/app/meson.build
index e32ea4bd5c..74d2420f67 100644
--- a/app/meson.build
+++ b/app/meson.build
@@ -23,6 +23,7 @@ apps = [
'test-fib',
'test-flow-perf',
'test-gpudev',
+ 'test-mldev',
'test-pipeline',
'test-pmd',
'test-regex',
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
new file mode 100644
index 0000000000..8ca2e1a1c1
--- /dev/null
+++ b/app/test-mldev/meson.build
@@ -0,0 +1,17 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright (c) 2022 Marvell.
+
+if is_windows
+ build = false
+ reason = 'not supported on Windows'
+ subdir_done()
+endif
+
+sources = files(
+ 'ml_main.c',
+ 'ml_options.c',
+ 'ml_test.c',
+ 'parser.c',
+)
+
+deps += ['mldev']
diff --git a/app/test-mldev/ml_common.h b/app/test-mldev/ml_common.h
new file mode 100644
index 0000000000..065180b619
--- /dev/null
+++ b/app/test-mldev/ml_common.h
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_COMMON_
+#define _ML_COMMON_
+
+#include <stdio.h>
+
+#define CLNRM "\x1b[0m"
+#define CLRED "\x1b[31m"
+#define CLGRN "\x1b[32m"
+#define CLYEL "\x1b[33m"
+
+#define ML_STR_FMT 20
+
+#define ml_err(fmt, args...) fprintf(stderr, CLRED "error: %s() " fmt CLNRM "\n", __func__, ##args)
+
+#define ml_info(fmt, args...) fprintf(stdout, CLYEL "" fmt CLNRM "\n", ##args)
+
+#define ml_dump(str, fmt, val...) printf("\t%-*s : " fmt "\n", ML_STR_FMT, str, ##val)
+
+#define ml_dump_begin(str) printf("\t%-*s :\n\t{\n", ML_STR_FMT, str)
+
+#define ml_dump_list(str, id, val) printf("\t%*s[%2u] : %s\n", ML_STR_FMT - 4, str, id, val)
+
+#define ml_dump_end printf("\b\t}\n\n")
+
+#endif /* _ML_COMMON_*/
diff --git a/app/test-mldev/ml_main.c b/app/test-mldev/ml_main.c
new file mode 100644
index 0000000000..940e609c9a
--- /dev/null
+++ b/app/test-mldev/ml_main.c
@@ -0,0 +1,113 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <rte_debug.h>
+#include <rte_eal.h>
+#include <rte_mldev.h>
+
+#include "ml_common.h"
+#include "ml_test.h"
+
+struct ml_options opt;
+struct ml_test *test;
+
+int
+main(int argc, char **argv)
+{
+ uint16_t mldevs;
+ int ret;
+
+ ret = rte_eal_init(argc, argv);
+ if (ret < 0)
+ rte_panic("invalid EAL arguments\n");
+ argc -= ret;
+ argv += ret;
+
+ mldevs = rte_ml_dev_count();
+ if (!mldevs)
+ rte_panic("no mldev devices found\n");
+
+ /* set default values for options */
+ ml_options_default(&opt);
+
+ /* parse the command line arguments */
+ ret = ml_options_parse(&opt, argc, argv);
+ if (ret) {
+ ml_err("parsing one or more user options failed");
+ goto error;
+ }
+
+ /* get test struct from name */
+ test = ml_test_get(opt.test_name);
+ if (test == NULL) {
+ ml_err("failed to find requested test: %s", opt.test_name);
+ goto error;
+ }
+
+ if (test->ops.test_result == NULL) {
+ ml_err("%s: ops.test_result not found", opt.test_name);
+ goto error;
+ }
+
+ /* check test options */
+ if (test->ops.opt_check) {
+ if (test->ops.opt_check(&opt)) {
+ ml_err("invalid command line argument");
+ goto error;
+ }
+ }
+
+ /* check the device capability */
+ if (test->ops.cap_check) {
+ if (test->ops.cap_check(&opt) == false) {
+ ml_info("unsupported test: %s", opt.test_name);
+ ret = ML_TEST_UNSUPPORTED;
+ goto no_cap;
+ }
+ }
+
+ /* dump options */
+ if (opt.debug) {
+ if (test->ops.opt_dump)
+ test->ops.opt_dump(&opt);
+ }
+
+ /* test specific setup */
+ if (test->ops.test_setup) {
+ if (test->ops.test_setup(test, &opt)) {
+ ml_err("failed to setup test: %s", opt.test_name);
+ goto error;
+ }
+ }
+
+ /* test driver */
+ if (test->ops.test_driver)
+ test->ops.test_driver(test, &opt);
+
+ /* get result */
+ if (test->ops.test_result)
+ ret = test->ops.test_result(test, &opt);
+
+ if (test->ops.test_destroy)
+ test->ops.test_destroy(test, &opt);
+
+no_cap:
+ if (ret == ML_TEST_SUCCESS) {
+ printf("Result: " CLGRN "%s" CLNRM "\n", "Success");
+ } else if (ret == ML_TEST_FAILED) {
+ printf("Result: " CLRED "%s" CLNRM "\n", "Failed");
+ return EXIT_FAILURE;
+ } else if (ret == ML_TEST_UNSUPPORTED) {
+ printf("Result: " CLYEL "%s" CLNRM "\n", "Unsupported");
+ }
+
+ rte_eal_cleanup();
+
+ return 0;
+
+error:
+ rte_eal_cleanup();
+
+ return EXIT_FAILURE;
+}
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
new file mode 100644
index 0000000000..521e9a2f97
--- /dev/null
+++ b/app/test-mldev/ml_options.c
@@ -0,0 +1,155 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <errno.h>
+#include <getopt.h>
+
+#include <rte_memory.h>
+#include <rte_mldev.h>
+#include <rte_string_fns.h>
+
+#include "ml_common.h"
+#include "ml_test.h"
+#include "parser.h"
+
+typedef int (*option_parser_t)(struct ml_options *opt, const char *arg);
+
+void
+ml_options_default(struct ml_options *opt)
+{
+ memset(opt, 0, sizeof(*opt));
+ strlcpy(opt->test_name, "ml_test", ML_TEST_NAME_MAX_LEN);
+ opt->dev_id = 0;
+ opt->socket_id = SOCKET_ID_ANY;
+ opt->debug = false;
+}
+
+struct long_opt_parser {
+ const char *lgopt_name;
+ option_parser_t parser_fn;
+};
+
+static int
+ml_parse_test_name(struct ml_options *opt, const char *arg)
+{
+ strlcpy(opt->test_name, arg, ML_TEST_NAME_MAX_LEN);
+ return 0;
+}
+
+static int
+ml_parse_dev_id(struct ml_options *opt, const char *arg)
+{
+ int ret;
+
+ ret = parser_read_int16(&opt->dev_id, arg);
+
+ if (ret < 0)
+ return -EINVAL;
+
+ return ret;
+}
+
+static int
+ml_parse_socket_id(struct ml_options *opt, const char *arg)
+{
+ opt->socket_id = atoi(arg);
+
+ return 0;
+}
+
+static void
+ml_dump_test_options(const char *testname)
+{
+ RTE_SET_USED(testname);
+}
+
+static void
+print_usage(char *program)
+{
+ printf("\nusage : %s [EAL options] -- [application options]\n", program);
+ printf("application options:\n");
+ printf("\t--test : name of the test application to run\n"
+ "\t--dev_id : device id of the ML device\n"
+ "\t--socket_id : socket_id of application resources\n"
+ "\t--debug : enable debug mode\n"
+ "\t--help : print help\n");
+ printf("\n");
+ printf("available tests and test specific application options:\n");
+ ml_test_dump_names(ml_dump_test_options);
+}
+
+static struct option lgopts[] = {{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0},
+ {ML_SOCKET_ID, 1, 0, 0}, {ML_DEBUG, 0, 0, 0},
+ {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
+
+static int
+ml_opts_parse_long(int opt_idx, struct ml_options *opt)
+{
+ unsigned int i;
+
+ struct long_opt_parser parsermap[] = {
+ {ML_TEST, ml_parse_test_name},
+ {ML_DEVICE_ID, ml_parse_dev_id},
+ {ML_SOCKET_ID, ml_parse_socket_id},
+ };
+
+ for (i = 0; i < RTE_DIM(parsermap); i++) {
+ if (strncmp(lgopts[opt_idx].name, parsermap[i].lgopt_name,
+ strlen(lgopts[opt_idx].name)) == 0)
+ return parsermap[i].parser_fn(opt, optarg);
+ }
+
+ return -EINVAL;
+}
+
+int
+ml_options_parse(struct ml_options *opt, int argc, char **argv)
+{
+ int opt_idx;
+ int retval;
+ int opts;
+
+ while ((opts = getopt_long(argc, argv, "", lgopts, &opt_idx)) != EOF) {
+ switch (opts) {
+ case 0: /* parse long options */
+ if (!strcmp(lgopts[opt_idx].name, "debug")) {
+ opt->debug = true;
+ break;
+ }
+
+ if (!strcmp(lgopts[opt_idx].name, "help")) {
+ print_usage(argv[0]);
+ exit(EXIT_SUCCESS);
+ }
+
+ retval = ml_opts_parse_long(opt_idx, opt);
+ if (retval != 0)
+ return retval;
+ break;
+ default:
+ return -EINVAL;
+ }
+ }
+
+ return 0;
+}
+
+void
+ml_options_dump(struct ml_options *opt)
+{
+ struct rte_ml_dev_info dev_info;
+
+ rte_ml_dev_info_get(opt->dev_id, &dev_info);
+
+ ml_dump("driver", "%s", dev_info.driver_name);
+ ml_dump("test", "%s", opt->test_name);
+ ml_dump("dev_id", "%d", opt->dev_id);
+
+ if (opt->socket_id == SOCKET_ID_ANY)
+ ml_dump("socket_id", "%d (SOCKET_ID_ANY)", opt->socket_id);
+ else
+ ml_dump("socket_id", "%d", opt->socket_id);
+
+ ml_dump("debug", "%s", (opt->debug ? "true" : "false"));
+}
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
new file mode 100644
index 0000000000..05311a9a47
--- /dev/null
+++ b/app/test-mldev/ml_options.h
@@ -0,0 +1,31 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_OPTIONS_
+#define _ML_OPTIONS_
+
+#include <stdbool.h>
+#include <stdint.h>
+
+#define ML_TEST_NAME_MAX_LEN 32
+
+/* Options names */
+#define ML_TEST ("test")
+#define ML_DEVICE_ID ("dev_id")
+#define ML_SOCKET_ID ("socket_id")
+#define ML_DEBUG ("debug")
+#define ML_HELP ("help")
+
+struct ml_options {
+ char test_name[ML_TEST_NAME_MAX_LEN];
+ int16_t dev_id;
+ int socket_id;
+ bool debug;
+};
+
+void ml_options_default(struct ml_options *opt);
+int ml_options_parse(struct ml_options *opt, int argc, char **argv);
+void ml_options_dump(struct ml_options *opt);
+
+#endif /* _ML_OPTIONS_ */
diff --git a/app/test-mldev/ml_test.c b/app/test-mldev/ml_test.c
new file mode 100644
index 0000000000..dcdeabe0bd
--- /dev/null
+++ b/app/test-mldev/ml_test.c
@@ -0,0 +1,41 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include "ml_test.h"
+
+static STAILQ_HEAD(, ml_test_entry) head = STAILQ_HEAD_INITIALIZER(head);
+
+void
+ml_test_register(struct ml_test_entry *entry)
+{
+ STAILQ_INSERT_TAIL(&head, entry, next);
+}
+
+struct ml_test *
+ml_test_get(const char *name)
+{
+ struct ml_test_entry *entry;
+
+ if (!name)
+ return NULL;
+
+ STAILQ_FOREACH(entry, &head, next)
+ if (!strncmp(entry->test.name, name, strlen(name)))
+ return &entry->test;
+
+ return NULL;
+}
+
+void
+ml_test_dump_names(void (*f)(const char *name))
+{
+ struct ml_test_entry *entry;
+
+ STAILQ_FOREACH(entry, &head, next)
+ {
+ if (entry->test.name)
+ printf("\t %s\n", entry->test.name);
+ f(entry->test.name);
+ }
+}
diff --git a/app/test-mldev/ml_test.h b/app/test-mldev/ml_test.h
new file mode 100644
index 0000000000..f7e6bf2f44
--- /dev/null
+++ b/app/test-mldev/ml_test.h
@@ -0,0 +1,76 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_TEST_
+#define _ML_TEST_
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/queue.h>
+
+#include <rte_common.h>
+
+#include "ml_options.h"
+
+#define ML_TEST_MAX_POOL_SIZE 256
+
+enum ml_test_result {
+ ML_TEST_SUCCESS,
+ ML_TEST_FAILED,
+ ML_TEST_UNSUPPORTED,
+};
+
+struct ml_test;
+
+typedef bool (*ml_test_capability_check_t)(struct ml_options *opt);
+typedef int (*ml_test_options_check_t)(struct ml_options *opt);
+typedef void (*ml_test_options_dump_t)(struct ml_options *opt);
+typedef int (*ml_test_setup_t)(struct ml_test *test, struct ml_options *opt);
+typedef void (*ml_test_destroy_t)(struct ml_test *test, struct ml_options *opt);
+typedef int (*ml_test_driver_t)(struct ml_test *test, struct ml_options *opt);
+typedef int (*ml_test_result_t)(struct ml_test *test, struct ml_options *opt);
+
+struct ml_test_ops {
+ ml_test_capability_check_t cap_check;
+ ml_test_options_check_t opt_check;
+ ml_test_options_dump_t opt_dump;
+ ml_test_setup_t test_setup;
+ ml_test_destroy_t test_destroy;
+ ml_test_driver_t test_driver;
+ ml_test_result_t test_result;
+};
+
+struct ml_test {
+ const char *name;
+ void *test_priv;
+ struct ml_test_ops ops;
+};
+
+struct ml_test_entry {
+ struct ml_test test;
+
+ STAILQ_ENTRY(ml_test_entry) next;
+};
+
+static inline void *
+ml_test_priv(struct ml_test *test)
+{
+ return test->test_priv;
+}
+
+struct ml_test *ml_test_get(const char *name);
+void ml_test_register(struct ml_test_entry *test);
+void ml_test_dump_names(void (*f)(const char *));
+
+#define ML_TEST_REGISTER(nm) \
+ static struct ml_test_entry _ml_test_entry_##nm; \
+ RTE_INIT(ml_test_##nm) \
+ { \
+ _ml_test_entry_##nm.test.name = RTE_STR(nm); \
+ memcpy(&_ml_test_entry_##nm.test.ops, &nm, sizeof(struct ml_test_ops)); \
+ ml_test_register(&_ml_test_entry_##nm); \
+ }
+
+#endif /* _ML_TEST_ */
diff --git a/app/test-mldev/parser.c b/app/test-mldev/parser.c
new file mode 100644
index 0000000000..0b7fb63fe5
--- /dev/null
+++ b/app/test-mldev/parser.c
@@ -0,0 +1,380 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2016 Intel Corporation.
+ * Copyright (c) 2017 Cavium, Inc.
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <ctype.h>
+#include <errno.h>
+#include <stdbool.h>
+#include <stdint.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include <rte_common.h>
+
+#include "parser.h"
+
+static uint32_t
+get_hex_val(char c)
+{
+ switch (c) {
+ case '0':
+ case '1':
+ case '2':
+ case '3':
+ case '4':
+ case '5':
+ case '6':
+ case '7':
+ case '8':
+ case '9':
+ return c - '0';
+ case 'A':
+ case 'B':
+ case 'C':
+ case 'D':
+ case 'E':
+ case 'F':
+ return c - 'A' + 10;
+ case 'a':
+ case 'b':
+ case 'c':
+ case 'd':
+ case 'e':
+ case 'f':
+ return c - 'a' + 10;
+ default:
+ return 0;
+ }
+}
+
+int
+parser_read_arg_bool(const char *p)
+{
+ p = skip_white_spaces(p);
+ int result = -EINVAL;
+
+ if (((p[0] == 'y') && (p[1] == 'e') && (p[2] == 's')) ||
+ ((p[0] == 'Y') && (p[1] == 'E') && (p[2] == 'S'))) {
+ p += 3;
+ result = 1;
+ }
+
+ if (((p[0] == 'o') && (p[1] == 'n')) || ((p[0] == 'O') && (p[1] == 'N'))) {
+ p += 2;
+ result = 1;
+ }
+
+ if (((p[0] == 'n') && (p[1] == 'o')) || ((p[0] == 'N') && (p[1] == 'O'))) {
+ p += 2;
+ result = 0;
+ }
+
+ if (((p[0] == 'o') && (p[1] == 'f') && (p[2] == 'f')) ||
+ ((p[0] == 'O') && (p[1] == 'F') && (p[2] == 'F'))) {
+ p += 3;
+ result = 0;
+ }
+
+ p = skip_white_spaces(p);
+
+ if (p[0] != '\0')
+ return -EINVAL;
+
+ return result;
+}
+
+int
+parser_read_uint64(uint64_t *value, const char *p)
+{
+ char *next;
+ uint64_t val;
+
+ p = skip_white_spaces(p);
+ if (!isdigit(*p))
+ return -EINVAL;
+
+ val = strtoul(p, &next, 10);
+ if (p == next)
+ return -EINVAL;
+
+ p = next;
+ switch (*p) {
+ case 'T':
+ val *= 1024ULL;
+ /* fall through */
+ case 'G':
+ val *= 1024ULL;
+ /* fall through */
+ case 'M':
+ val *= 1024ULL;
+ /* fall through */
+ case 'k':
+ case 'K':
+ val *= 1024ULL;
+ p++;
+ break;
+ }
+
+ p = skip_white_spaces(p);
+ if (*p != '\0')
+ return -EINVAL;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_int32(int32_t *value, const char *p)
+{
+ char *next;
+ int32_t val;
+
+ p = skip_white_spaces(p);
+ if (!isdigit(*p))
+ return -EINVAL;
+
+ val = strtol(p, &next, 10);
+ if (p == next)
+ return -EINVAL;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_int16(int16_t *value, const char *p)
+{
+ char *next;
+ int16_t val;
+
+ p = skip_white_spaces(p);
+ if (!isdigit(*p))
+ return -EINVAL;
+
+ val = strtol(p, &next, 10);
+ if (p == next)
+ return -EINVAL;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint64_hex(uint64_t *value, const char *p)
+{
+ char *next;
+ uint64_t val;
+
+ p = skip_white_spaces(p);
+
+ val = strtoul(p, &next, 16);
+ if (p == next)
+ return -EINVAL;
+
+ p = skip_white_spaces(next);
+ if (*p != '\0')
+ return -EINVAL;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint32(uint32_t *value, const char *p)
+{
+ uint64_t val = 0;
+ int ret = parser_read_uint64(&val, p);
+
+ if (ret < 0)
+ return ret;
+
+ if (val > UINT32_MAX)
+ return -ERANGE;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint32_hex(uint32_t *value, const char *p)
+{
+ uint64_t val = 0;
+ int ret = parser_read_uint64_hex(&val, p);
+
+ if (ret < 0)
+ return ret;
+
+ if (val > UINT32_MAX)
+ return -ERANGE;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint16(uint16_t *value, const char *p)
+{
+ uint64_t val = 0;
+ int ret = parser_read_uint64(&val, p);
+
+ if (ret < 0)
+ return ret;
+
+ if (val > UINT16_MAX)
+ return -ERANGE;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint16_hex(uint16_t *value, const char *p)
+{
+ uint64_t val = 0;
+ int ret = parser_read_uint64_hex(&val, p);
+
+ if (ret < 0)
+ return ret;
+
+ if (val > UINT16_MAX)
+ return -ERANGE;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint8(uint8_t *value, const char *p)
+{
+ uint64_t val = 0;
+ int ret = parser_read_uint64(&val, p);
+
+ if (ret < 0)
+ return ret;
+
+ if (val > UINT8_MAX)
+ return -ERANGE;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint8_hex(uint8_t *value, const char *p)
+{
+ uint64_t val = 0;
+ int ret = parser_read_uint64_hex(&val, p);
+
+ if (ret < 0)
+ return ret;
+
+ if (val > UINT8_MAX)
+ return -ERANGE;
+
+ *value = val;
+ return 0;
+}
+
+int
+parse_tokenize_string(char *string, char *tokens[], uint32_t *n_tokens)
+{
+ uint32_t i;
+
+ if ((string == NULL) || (tokens == NULL) || (*n_tokens < 1))
+ return -EINVAL;
+
+ for (i = 0; i < *n_tokens; i++) {
+ tokens[i] = strtok_r(string, PARSE_DELIMITER, &string);
+ if (tokens[i] == NULL)
+ break;
+ }
+
+ if ((i == *n_tokens) && (strtok_r(string, PARSE_DELIMITER, &string) != NULL))
+ return -E2BIG;
+
+ *n_tokens = i;
+ return 0;
+}
+
+int
+parse_hex_string(char *src, uint8_t *dst, uint32_t *size)
+{
+ char *c;
+ uint32_t len, i;
+
+ /* Check input parameters */
+ if ((src == NULL) || (dst == NULL) || (size == NULL) || (*size == 0))
+ return -1;
+
+ len = strlen(src);
+ if (((len & 3) != 0) || (len > (*size) * 2))
+ return -1;
+ *size = len / 2;
+
+ for (c = src; *c != 0; c++) {
+ if ((((*c) >= '0') && ((*c) <= '9')) || (((*c) >= 'A') && ((*c) <= 'F')) ||
+ (((*c) >= 'a') && ((*c) <= 'f')))
+ continue;
+
+ return -1;
+ }
+
+ /* Convert chars to bytes */
+ for (i = 0; i < *size; i++)
+ dst[i] = get_hex_val(src[2 * i]) * 16 + get_hex_val(src[2 * i + 1]);
+
+ return 0;
+}
+
+int
+parse_lcores_list(bool lcores[], int lcores_num, const char *corelist)
+{
+ int i, idx = 0;
+ int min, max;
+ char *end = NULL;
+
+ if (corelist == NULL)
+ return -1;
+ while (isblank(*corelist))
+ corelist++;
+ i = strlen(corelist);
+ while ((i > 0) && isblank(corelist[i - 1]))
+ i--;
+
+ /* Get list of lcores */
+ min = RTE_MAX_LCORE;
+ do {
+ while (isblank(*corelist))
+ corelist++;
+ if (*corelist == '\0')
+ return -1;
+ idx = strtoul(corelist, &end, 10);
+ if (idx < 0 || idx > lcores_num)
+ return -1;
+
+ if (end == NULL)
+ return -1;
+ while (isblank(*end))
+ end++;
+ if (*end == '-') {
+ min = idx;
+ } else if ((*end == ',') || (*end == '\0')) {
+ max = idx;
+ if (min == RTE_MAX_LCORE)
+ min = idx;
+ for (idx = min; idx <= max; idx++) {
+ if (lcores[idx] == 1)
+ return -E2BIG;
+ lcores[idx] = 1;
+ }
+
+ min = RTE_MAX_LCORE;
+ } else
+ return -1;
+ corelist = end + 1;
+ } while (*end != '\0');
+
+ return 0;
+}
diff --git a/app/test-mldev/parser.h b/app/test-mldev/parser.h
new file mode 100644
index 0000000000..f0d5e79e4b
--- /dev/null
+++ b/app/test-mldev/parser.h
@@ -0,0 +1,55 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2010-2016 Intel Corporation.
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef __INCLUDE_PARSER_H__
+#define __INCLUDE_PARSER_H__
+
+#include <ctype.h>
+#include <stdint.h>
+#include <stdlib.h>
+
+#define PARSE_DELIMITER " \f\n\r\t\v"
+
+#define skip_white_spaces(pos) \
+ ({ \
+ __typeof__(pos) _p = (pos); \
+ for (; isspace(*_p); _p++) \
+ ; \
+ _p; \
+ })
+
+static inline size_t
+skip_digits(const char *src)
+{
+ size_t i;
+
+ for (i = 0; isdigit(src[i]); i++)
+ ;
+
+ return i;
+}
+
+int parser_read_arg_bool(const char *p);
+
+int parser_read_uint64(uint64_t *value, const char *p);
+int parser_read_uint32(uint32_t *value, const char *p);
+int parser_read_uint16(uint16_t *value, const char *p);
+int parser_read_uint8(uint8_t *value, const char *p);
+
+int parser_read_uint64_hex(uint64_t *value, const char *p);
+int parser_read_uint32_hex(uint32_t *value, const char *p);
+int parser_read_uint16_hex(uint16_t *value, const char *p);
+int parser_read_uint8_hex(uint8_t *value, const char *p);
+
+int parser_read_int32(int32_t *value, const char *p);
+int parser_read_int16(int16_t *value, const char *p);
+
+int parse_hex_string(char *src, uint8_t *dst, uint32_t *size);
+
+int parse_tokenize_string(char *string, char *tokens[], uint32_t *n_tokens);
+
+int parse_lcores_list(bool lcores[], int lcores_num, const char *corelist);
+
+#endif /* __INCLUDE_PARSER_H__ */
diff --git a/doc/guides/rel_notes/release_23_03.rst b/doc/guides/rel_notes/release_23_03.rst
index d5b9f721dd..8186545082 100644
--- a/doc/guides/rel_notes/release_23_03.rst
+++ b/doc/guides/rel_notes/release_23_03.rst
@@ -216,6 +216,14 @@ New Features
* Added support to capture packets at each graph node with packet metadata and
node name.
+* **Added test application for machine learning inference device library.**
+
+ * Added test application for mldev library with support for multiple test cases.
+ * Test case for device operations.
+ * Test case for model operations.
+ * Test case for inferences from multiple models in ordered mode.
+ * Test case for inferences from multiple models.in interleaving mode.
+
Removed Items
-------------
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v5 02/12] app/mldev: add common test functions
2023-03-10 8:09 ` [PATCH v5 00/12] Implementation of mldev test application Srikanth Yalavarthi
2023-03-10 8:09 ` [PATCH v5 01/12] app/mldev: implement test framework for mldev Srikanth Yalavarthi
@ 2023-03-10 8:09 ` Srikanth Yalavarthi
2023-03-10 8:09 ` [PATCH v5 03/12] app/mldev: add test case to validate device ops Srikanth Yalavarthi
` (9 subsequent siblings)
11 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-10 8:09 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Added common functions used by all tests. Common code
includes functions to check capabilities, options, and
handle ML devices.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/meson.build | 1 +
app/test-mldev/test_common.c | 136 +++++++++++++++++++++++++++++++++++
app/test-mldev/test_common.h | 27 +++++++
3 files changed, 164 insertions(+)
create mode 100644 app/test-mldev/test_common.c
create mode 100644 app/test-mldev/test_common.h
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
index 8ca2e1a1c1..964bb9ddc4 100644
--- a/app/test-mldev/meson.build
+++ b/app/test-mldev/meson.build
@@ -12,6 +12,7 @@ sources = files(
'ml_options.c',
'ml_test.c',
'parser.c',
+ 'test_common.c',
)
deps += ['mldev']
diff --git a/app/test-mldev/test_common.c b/app/test-mldev/test_common.c
new file mode 100644
index 0000000000..8c4da4609a
--- /dev/null
+++ b/app/test-mldev/test_common.c
@@ -0,0 +1,136 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <errno.h>
+
+#include <rte_common.h>
+#include <rte_memory.h>
+#include <rte_mldev.h>
+
+#include "ml_common.h"
+#include "test_common.h"
+
+bool
+ml_test_cap_check(struct ml_options *opt)
+{
+ struct rte_ml_dev_info dev_info;
+
+ rte_ml_dev_info_get(opt->dev_id, &dev_info);
+ if (dev_info.max_models == 0) {
+ ml_err("Not enough mldev models supported = %u", dev_info.max_models);
+ return false;
+ }
+
+ return true;
+}
+
+int
+ml_test_opt_check(struct ml_options *opt)
+{
+ uint16_t dev_count;
+ int socket_id;
+
+ RTE_SET_USED(opt);
+
+ dev_count = rte_ml_dev_count();
+ if (dev_count == 0) {
+ ml_err("No ML devices found");
+ return -ENODEV;
+ }
+
+ if (opt->dev_id >= dev_count) {
+ ml_err("Invalid option dev_id = %d", opt->dev_id);
+ return -EINVAL;
+ }
+
+ socket_id = rte_ml_dev_socket_id(opt->dev_id);
+ if (!((opt->socket_id != SOCKET_ID_ANY) || (opt->socket_id != socket_id))) {
+ ml_err("Invalid option, socket_id = %d\n", opt->socket_id);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+void
+ml_test_opt_dump(struct ml_options *opt)
+{
+ ml_options_dump(opt);
+}
+
+int
+ml_test_device_configure(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_common *t = ml_test_priv(test);
+ struct rte_ml_dev_config dev_config;
+ int ret;
+
+ ret = rte_ml_dev_info_get(opt->dev_id, &t->dev_info);
+ if (ret != 0) {
+ ml_err("Failed to get mldev info, dev_id = %d\n", opt->dev_id);
+ return ret;
+ }
+
+ /* configure device */
+ dev_config.socket_id = opt->socket_id;
+ dev_config.nb_models = t->dev_info.max_models;
+ dev_config.nb_queue_pairs = t->dev_info.max_queue_pairs;
+ ret = rte_ml_dev_configure(opt->dev_id, &dev_config);
+ if (ret != 0) {
+ ml_err("Failed to configure ml device, dev_id = %d\n", opt->dev_id);
+ return ret;
+ }
+
+ return 0;
+}
+
+int
+ml_test_device_close(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_common *t = ml_test_priv(test);
+ int ret = 0;
+
+ RTE_SET_USED(t);
+
+ /* close device */
+ ret = rte_ml_dev_close(opt->dev_id);
+ if (ret != 0)
+ ml_err("Failed to close ML device, dev_id = %d\n", opt->dev_id);
+
+ return ret;
+}
+
+int
+ml_test_device_start(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_common *t = ml_test_priv(test);
+ int ret;
+
+ RTE_SET_USED(t);
+
+ /* start device */
+ ret = rte_ml_dev_start(opt->dev_id);
+ if (ret != 0) {
+ ml_err("Failed to start ml device, dev_id = %d\n", opt->dev_id);
+ return ret;
+ }
+
+ return 0;
+}
+
+int
+ml_test_device_stop(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_common *t = ml_test_priv(test);
+ int ret = 0;
+
+ RTE_SET_USED(t);
+
+ /* stop device */
+ ret = rte_ml_dev_stop(opt->dev_id);
+ if (ret != 0)
+ ml_err("Failed to stop ML device, dev_id = %d\n", opt->dev_id);
+
+ return ret;
+}
diff --git a/app/test-mldev/test_common.h b/app/test-mldev/test_common.h
new file mode 100644
index 0000000000..cb286adcd6
--- /dev/null
+++ b/app/test-mldev/test_common.h
@@ -0,0 +1,27 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_TEST_COMMON_
+#define _ML_TEST_COMMON_
+
+#include <rte_mldev.h>
+
+#include "ml_common.h"
+#include "ml_test.h"
+
+struct test_common {
+ struct ml_options *opt;
+ enum ml_test_result result;
+ struct rte_ml_dev_info dev_info;
+};
+
+bool ml_test_cap_check(struct ml_options *opt);
+int ml_test_opt_check(struct ml_options *opt);
+void ml_test_opt_dump(struct ml_options *opt);
+int ml_test_device_configure(struct ml_test *test, struct ml_options *opt);
+int ml_test_device_close(struct ml_test *test, struct ml_options *opt);
+int ml_test_device_start(struct ml_test *test, struct ml_options *opt);
+int ml_test_device_stop(struct ml_test *test, struct ml_options *opt);
+
+#endif /* _ML_TEST_COMMON_ */
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v5 03/12] app/mldev: add test case to validate device ops
2023-03-10 8:09 ` [PATCH v5 00/12] Implementation of mldev test application Srikanth Yalavarthi
2023-03-10 8:09 ` [PATCH v5 01/12] app/mldev: implement test framework for mldev Srikanth Yalavarthi
2023-03-10 8:09 ` [PATCH v5 02/12] app/mldev: add common test functions Srikanth Yalavarthi
@ 2023-03-10 8:09 ` Srikanth Yalavarthi
2023-03-10 8:09 ` [PATCH v5 04/12] app/mldev: add test case to validate model ops Srikanth Yalavarthi
` (8 subsequent siblings)
11 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-10 8:09 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Added test case to validate device handling operations. Device ops
test is a collection of multiple sub-tests. Enabled sub-test to
validate device reconfiguration. Set device_ops as the default test.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/meson.build | 1 +
app/test-mldev/ml_options.c | 5 +-
app/test-mldev/ml_options.h | 2 +
app/test-mldev/test_device_ops.c | 228 +++++++++++++++++++++++++++++++
app/test-mldev/test_device_ops.h | 17 +++
5 files changed, 251 insertions(+), 2 deletions(-)
create mode 100644 app/test-mldev/test_device_ops.c
create mode 100644 app/test-mldev/test_device_ops.h
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
index 964bb9ddc4..60ea23d142 100644
--- a/app/test-mldev/meson.build
+++ b/app/test-mldev/meson.build
@@ -13,6 +13,7 @@ sources = files(
'ml_test.c',
'parser.c',
'test_common.c',
+ 'test_device_ops.c',
)
deps += ['mldev']
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 521e9a2f97..b0837dde14 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -19,7 +19,7 @@ void
ml_options_default(struct ml_options *opt)
{
memset(opt, 0, sizeof(*opt));
- strlcpy(opt->test_name, "ml_test", ML_TEST_NAME_MAX_LEN);
+ strlcpy(opt->test_name, "device_ops", ML_TEST_NAME_MAX_LEN);
opt->dev_id = 0;
opt->socket_id = SOCKET_ID_ANY;
opt->debug = false;
@@ -61,7 +61,8 @@ ml_parse_socket_id(struct ml_options *opt, const char *arg)
static void
ml_dump_test_options(const char *testname)
{
- RTE_SET_USED(testname);
+ if (strcmp(testname, "device_ops") == 0)
+ printf("\n");
}
static void
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index 05311a9a47..6899cc2f88 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -9,6 +9,7 @@
#include <stdint.h>
#define ML_TEST_NAME_MAX_LEN 32
+#define ML_TEST_MAX_MODELS 8
/* Options names */
#define ML_TEST ("test")
@@ -21,6 +22,7 @@ struct ml_options {
char test_name[ML_TEST_NAME_MAX_LEN];
int16_t dev_id;
int socket_id;
+ uint8_t nb_filelist;
bool debug;
};
diff --git a/app/test-mldev/test_device_ops.c b/app/test-mldev/test_device_ops.c
new file mode 100644
index 0000000000..be0c5f0780
--- /dev/null
+++ b/app/test-mldev/test_device_ops.c
@@ -0,0 +1,228 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <errno.h>
+
+#include <rte_common.h>
+#include <rte_malloc.h>
+#include <rte_mldev.h>
+
+#include "test_device_ops.h"
+
+static bool
+test_device_cap_check(struct ml_options *opt)
+{
+ if (!ml_test_cap_check(opt))
+ return false;
+
+ return true;
+}
+
+static int
+test_device_opt_check(struct ml_options *opt)
+{
+ int ret;
+
+ /* check common opts */
+ ret = ml_test_opt_check(opt);
+ if (ret != 0)
+ return ret;
+
+ return 0;
+}
+
+static void
+test_device_opt_dump(struct ml_options *opt)
+{
+ /* dump common opts */
+ ml_test_opt_dump(opt);
+}
+
+static int
+test_device_setup(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_device *t;
+ void *test_device;
+ int ret = 0;
+
+ /* allocate for test structure */
+ test_device = rte_zmalloc_socket(test->name, sizeof(struct test_device),
+ RTE_CACHE_LINE_SIZE, opt->socket_id);
+ if (test_device == NULL) {
+ ml_err("failed to allocate memory for test_model");
+ ret = -ENOMEM;
+ goto error;
+ }
+ test->test_priv = test_device;
+ t = ml_test_priv(test);
+
+ t->cmn.result = ML_TEST_FAILED;
+ t->cmn.opt = opt;
+
+ /* get device info */
+ ret = rte_ml_dev_info_get(opt->dev_id, &t->cmn.dev_info);
+ if (ret < 0) {
+ ml_err("failed to get device info");
+ goto error;
+ }
+
+ return 0;
+
+error:
+ if (test_device != NULL)
+ rte_free(test_device);
+
+ return ret;
+}
+
+static void
+test_device_destroy(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_device *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+ if (t != NULL)
+ rte_free(t);
+}
+
+static int
+test_device_reconfigure(struct ml_test *test, struct ml_options *opt)
+{
+ struct rte_ml_dev_config dev_config;
+ struct rte_ml_dev_qp_conf qp_conf;
+ struct test_device *t;
+ uint16_t qp_id = 0;
+ int ret = 0;
+
+ t = ml_test_priv(test);
+
+ /* configure with default options */
+ ret = ml_test_device_configure(test, opt);
+ if (ret != 0)
+ return ret;
+
+ /* setup one queue pair with nb_desc = 1 */
+ qp_conf.nb_desc = 1;
+ qp_conf.cb = NULL;
+
+ ret = rte_ml_dev_queue_pair_setup(opt->dev_id, qp_id, &qp_conf, opt->socket_id);
+ if (ret != 0) {
+ ml_err("Failed to setup ML device queue-pair, dev_id = %d, qp_id = %u\n",
+ opt->dev_id, qp_id);
+ goto error;
+ }
+
+ /* start device */
+ ret = ml_test_device_start(test, opt);
+ if (ret != 0)
+ goto error;
+
+ /* stop device */
+ ret = ml_test_device_stop(test, opt);
+ if (ret != 0) {
+ ml_err("Failed to stop device");
+ goto error;
+ }
+
+ /* reconfigure device based on dev_info */
+ dev_config.socket_id = opt->socket_id;
+ dev_config.nb_models = t->cmn.dev_info.max_models;
+ dev_config.nb_queue_pairs = t->cmn.dev_info.max_queue_pairs;
+ ret = rte_ml_dev_configure(opt->dev_id, &dev_config);
+ if (ret != 0) {
+ ml_err("Failed to reconfigure ML device, dev_id = %d\n", opt->dev_id);
+ return ret;
+ }
+
+ /* setup queue pairs */
+ for (qp_id = 0; qp_id < t->cmn.dev_info.max_queue_pairs; qp_id++) {
+ qp_conf.nb_desc = t->cmn.dev_info.max_desc;
+ qp_conf.cb = NULL;
+
+ ret = rte_ml_dev_queue_pair_setup(opt->dev_id, qp_id, &qp_conf, opt->socket_id);
+ if (ret != 0) {
+ ml_err("Failed to setup ML device queue-pair, dev_id = %d, qp_id = %u\n",
+ opt->dev_id, qp_id);
+ goto error;
+ }
+ }
+
+ /* start device */
+ ret = ml_test_device_start(test, opt);
+ if (ret != 0)
+ goto error;
+
+ /* stop device */
+ ret = ml_test_device_stop(test, opt);
+ if (ret != 0)
+ goto error;
+
+ /* close device */
+ ret = ml_test_device_close(test, opt);
+ if (ret != 0)
+ return ret;
+
+ return 0;
+
+error:
+ ml_test_device_close(test, opt);
+
+ return ret;
+}
+
+static int
+test_device_driver(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_device *t;
+ int ret = 0;
+
+ t = ml_test_priv(test);
+
+ /* sub-test: device reconfigure */
+ ret = test_device_reconfigure(test, opt);
+ if (ret != 0) {
+ printf("\n");
+ printf("Model Device Reconfigure Test: " CLRED "%s" CLNRM "\n", "Failed");
+ goto error;
+ } else {
+ printf("\n");
+ printf("Model Device Reconfigure Test: " CLYEL "%s" CLNRM "\n", "Passed");
+ }
+
+ printf("\n");
+
+ t->cmn.result = ML_TEST_SUCCESS;
+
+ return 0;
+
+error:
+ t->cmn.result = ML_TEST_FAILED;
+ return -1;
+}
+
+static int
+test_device_result(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_device *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+
+ return t->cmn.result;
+}
+
+static const struct ml_test_ops device_ops = {
+ .cap_check = test_device_cap_check,
+ .opt_check = test_device_opt_check,
+ .opt_dump = test_device_opt_dump,
+ .test_setup = test_device_setup,
+ .test_destroy = test_device_destroy,
+ .test_driver = test_device_driver,
+ .test_result = test_device_result,
+};
+
+ML_TEST_REGISTER(device_ops);
diff --git a/app/test-mldev/test_device_ops.h b/app/test-mldev/test_device_ops.h
new file mode 100644
index 0000000000..115b1072a2
--- /dev/null
+++ b/app/test-mldev/test_device_ops.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_TEST_DEVICE_OPS_
+#define _ML_TEST_DEVICE_OPS_
+
+#include <rte_common.h>
+
+#include "test_common.h"
+
+struct test_device {
+ /* common data */
+ struct test_common cmn;
+} __rte_cache_aligned;
+
+#endif /* _ML_TEST_DEVICE_OPS_ */
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v5 04/12] app/mldev: add test case to validate model ops
2023-03-10 8:09 ` [PATCH v5 00/12] Implementation of mldev test application Srikanth Yalavarthi
` (2 preceding siblings ...)
2023-03-10 8:09 ` [PATCH v5 03/12] app/mldev: add test case to validate device ops Srikanth Yalavarthi
@ 2023-03-10 8:09 ` Srikanth Yalavarthi
2023-03-10 8:09 ` [PATCH v5 05/12] app/mldev: add ordered inference test case Srikanth Yalavarthi
` (7 subsequent siblings)
11 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-10 8:09 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Added test case to validate model operations. Model ops test
is a collection of sub-tests. Each sub-test invokes the model
operations in a specific order.
Sub-test A: (load -> start -> stop -> unload) x n
Sub-test B: load x n -> start x n -> stop x n -> unload x n
Sub-test C: load x n + (start + stop) x n + unload x n
Sub-test D: (load + start) x n -> (stop + unload) x n
Added internal functions to handle model load, start, stop and
unload. List of models to be used for testing can be specified
through application argument "--models"
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/meson.build | 2 +
app/test-mldev/ml_options.c | 44 ++-
app/test-mldev/ml_options.h | 6 +
app/test-mldev/test_model_common.c | 158 +++++++++++
app/test-mldev/test_model_common.h | 34 +++
app/test-mldev/test_model_ops.c | 428 +++++++++++++++++++++++++++++
app/test-mldev/test_model_ops.h | 20 ++
7 files changed, 689 insertions(+), 3 deletions(-)
create mode 100644 app/test-mldev/test_model_common.c
create mode 100644 app/test-mldev/test_model_common.h
create mode 100644 app/test-mldev/test_model_ops.c
create mode 100644 app/test-mldev/test_model_ops.h
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
index 60ea23d142..b09e1ccc8a 100644
--- a/app/test-mldev/meson.build
+++ b/app/test-mldev/meson.build
@@ -14,6 +14,8 @@ sources = files(
'parser.c',
'test_common.c',
'test_device_ops.c',
+ 'test_model_common.c',
+ 'test_model_ops.c',
)
deps += ['mldev']
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index b0837dde14..91259e1427 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -22,6 +22,7 @@ ml_options_default(struct ml_options *opt)
strlcpy(opt->test_name, "device_ops", ML_TEST_NAME_MAX_LEN);
opt->dev_id = 0;
opt->socket_id = SOCKET_ID_ANY;
+ opt->nb_filelist = 0;
opt->debug = false;
}
@@ -58,11 +59,47 @@ ml_parse_socket_id(struct ml_options *opt, const char *arg)
return 0;
}
+static int
+ml_parse_models(struct ml_options *opt, const char *arg)
+{
+ const char *delim = ",";
+ char models[PATH_MAX];
+ char *token;
+ int ret = 0;
+
+ strlcpy(models, arg, PATH_MAX);
+
+ token = strtok(models, delim);
+ while (token != NULL) {
+ strlcpy(opt->filelist[opt->nb_filelist].model, token, PATH_MAX);
+ opt->nb_filelist++;
+
+ if (opt->nb_filelist >= ML_TEST_MAX_MODELS) {
+ ml_err("Exceeded model count, max = %d\n", ML_TEST_MAX_MODELS);
+ ret = -EINVAL;
+ break;
+ }
+ token = strtok(NULL, delim);
+ }
+
+ if (opt->nb_filelist == 0) {
+ ml_err("Models list is empty. Need at least one model for the test");
+ ret = -EINVAL;
+ }
+
+ return ret;
+}
+
static void
ml_dump_test_options(const char *testname)
{
if (strcmp(testname, "device_ops") == 0)
printf("\n");
+
+ if (strcmp(testname, "model_ops") == 0) {
+ printf("\t\t--models : comma separated list of models\n");
+ printf("\n");
+ }
}
static void
@@ -80,9 +117,9 @@ print_usage(char *program)
ml_test_dump_names(ml_dump_test_options);
}
-static struct option lgopts[] = {{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0},
- {ML_SOCKET_ID, 1, 0, 0}, {ML_DEBUG, 0, 0, 0},
- {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
+static struct option lgopts[] = {
+ {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0}, {ML_MODELS, 1, 0, 0},
+ {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
static int
ml_opts_parse_long(int opt_idx, struct ml_options *opt)
@@ -93,6 +130,7 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
{ML_TEST, ml_parse_test_name},
{ML_DEVICE_ID, ml_parse_dev_id},
{ML_SOCKET_ID, ml_parse_socket_id},
+ {ML_MODELS, ml_parse_models},
};
for (i = 0; i < RTE_DIM(parsermap); i++) {
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index 6899cc2f88..61e938d2e2 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -15,13 +15,19 @@
#define ML_TEST ("test")
#define ML_DEVICE_ID ("dev_id")
#define ML_SOCKET_ID ("socket_id")
+#define ML_MODELS ("models")
#define ML_DEBUG ("debug")
#define ML_HELP ("help")
+struct ml_filelist {
+ char model[PATH_MAX];
+};
+
struct ml_options {
char test_name[ML_TEST_NAME_MAX_LEN];
int16_t dev_id;
int socket_id;
+ struct ml_filelist filelist[ML_TEST_MAX_MODELS];
uint8_t nb_filelist;
bool debug;
};
diff --git a/app/test-mldev/test_model_common.c b/app/test-mldev/test_model_common.c
new file mode 100644
index 0000000000..b94d46154d
--- /dev/null
+++ b/app/test-mldev/test_model_common.c
@@ -0,0 +1,158 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <errno.h>
+
+#include <rte_common.h>
+#include <rte_malloc.h>
+#include <rte_mldev.h>
+
+#include "ml_common.h"
+#include "test_model_common.h"
+
+int
+ml_model_load(struct ml_test *test, struct ml_options *opt, struct ml_model *model, uint16_t fid)
+{
+ struct test_common *t = ml_test_priv(test);
+ struct rte_ml_model_params model_params;
+ FILE *fp;
+ int ret;
+
+ if (model->state == MODEL_LOADED)
+ return 0;
+
+ if (model->state != MODEL_INITIAL)
+ return -EINVAL;
+
+ /* read model binary */
+ fp = fopen(opt->filelist[fid].model, "r");
+ if (fp == NULL) {
+ ml_err("Failed to open model file : %s\n", opt->filelist[fid].model);
+ return -1;
+ }
+
+ fseek(fp, 0, SEEK_END);
+ model_params.size = ftell(fp);
+ fseek(fp, 0, SEEK_SET);
+
+ model_params.addr = rte_malloc_socket("ml_model", model_params.size,
+ t->dev_info.min_align_size, opt->socket_id);
+ if (model_params.addr == NULL) {
+ ml_err("Failed to allocate memory for model: %s\n", opt->filelist[fid].model);
+ fclose(fp);
+ return -ENOMEM;
+ }
+
+ if (fread(model_params.addr, 1, model_params.size, fp) != model_params.size) {
+ ml_err("Failed to read model file : %s\n", opt->filelist[fid].model);
+ rte_free(model_params.addr);
+ fclose(fp);
+ return -1;
+ }
+ fclose(fp);
+
+ /* load model to device */
+ ret = rte_ml_model_load(opt->dev_id, &model_params, &model->id);
+ if (ret != 0) {
+ ml_err("Failed to load model : %s\n", opt->filelist[fid].model);
+ model->state = MODEL_ERROR;
+ rte_free(model_params.addr);
+ return ret;
+ }
+
+ /* release mz */
+ rte_free(model_params.addr);
+
+ /* get model info */
+ ret = rte_ml_model_info_get(opt->dev_id, model->id, &model->info);
+ if (ret != 0) {
+ ml_err("Failed to get model info : %s\n", opt->filelist[fid].model);
+ return ret;
+ }
+
+ model->state = MODEL_LOADED;
+
+ return 0;
+}
+
+int
+ml_model_unload(struct ml_test *test, struct ml_options *opt, struct ml_model *model, uint16_t fid)
+{
+ struct test_common *t = ml_test_priv(test);
+ int ret;
+
+ RTE_SET_USED(t);
+
+ if (model->state == MODEL_INITIAL)
+ return 0;
+
+ if (model->state != MODEL_LOADED)
+ return -EINVAL;
+
+ /* unload model */
+ ret = rte_ml_model_unload(opt->dev_id, model->id);
+ if (ret != 0) {
+ ml_err("Failed to unload model: %s\n", opt->filelist[fid].model);
+ model->state = MODEL_ERROR;
+ return ret;
+ }
+
+ model->state = MODEL_INITIAL;
+
+ return 0;
+}
+
+int
+ml_model_start(struct ml_test *test, struct ml_options *opt, struct ml_model *model, uint16_t fid)
+{
+ struct test_common *t = ml_test_priv(test);
+ int ret;
+
+ RTE_SET_USED(t);
+
+ if (model->state == MODEL_STARTED)
+ return 0;
+
+ if (model->state != MODEL_LOADED)
+ return -EINVAL;
+
+ /* start model */
+ ret = rte_ml_model_start(opt->dev_id, model->id);
+ if (ret != 0) {
+ ml_err("Failed to start model : %s\n", opt->filelist[fid].model);
+ model->state = MODEL_ERROR;
+ return ret;
+ }
+
+ model->state = MODEL_STARTED;
+
+ return 0;
+}
+
+int
+ml_model_stop(struct ml_test *test, struct ml_options *opt, struct ml_model *model, uint16_t fid)
+{
+ struct test_common *t = ml_test_priv(test);
+ int ret;
+
+ RTE_SET_USED(t);
+
+ if (model->state == MODEL_LOADED)
+ return 0;
+
+ if (model->state != MODEL_STARTED)
+ return -EINVAL;
+
+ /* stop model */
+ ret = rte_ml_model_stop(opt->dev_id, model->id);
+ if (ret != 0) {
+ ml_err("Failed to stop model: %s\n", opt->filelist[fid].model);
+ model->state = MODEL_ERROR;
+ return ret;
+ }
+
+ model->state = MODEL_LOADED;
+
+ return 0;
+}
diff --git a/app/test-mldev/test_model_common.h b/app/test-mldev/test_model_common.h
new file mode 100644
index 0000000000..74aec0a797
--- /dev/null
+++ b/app/test-mldev/test_model_common.h
@@ -0,0 +1,34 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_TEST_MODEL_COMMON_
+#define _ML_TEST_MODEL_COMMON_
+
+#include <rte_mldev.h>
+
+#include "test_common.h"
+
+enum model_state {
+ MODEL_INITIAL,
+ MODEL_LOADED,
+ MODEL_STARTED,
+ MODEL_ERROR,
+};
+
+struct ml_model {
+ uint16_t id;
+ struct rte_ml_model_info info;
+ enum model_state state;
+};
+
+int ml_model_load(struct ml_test *test, struct ml_options *opt, struct ml_model *model,
+ uint16_t fid);
+int ml_model_unload(struct ml_test *test, struct ml_options *opt, struct ml_model *model,
+ uint16_t fid);
+int ml_model_start(struct ml_test *test, struct ml_options *opt, struct ml_model *model,
+ uint16_t fid);
+int ml_model_stop(struct ml_test *test, struct ml_options *opt, struct ml_model *model,
+ uint16_t fid);
+
+#endif /* _ML_TEST_MODEL_COMMON_ */
diff --git a/app/test-mldev/test_model_ops.c b/app/test-mldev/test_model_ops.c
new file mode 100644
index 0000000000..0202b31190
--- /dev/null
+++ b/app/test-mldev/test_model_ops.c
@@ -0,0 +1,428 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <unistd.h>
+
+#include <rte_common.h>
+#include <rte_malloc.h>
+#include <rte_mldev.h>
+
+#include "test_model_ops.h"
+
+static bool
+test_model_ops_cap_check(struct ml_options *opt)
+{
+ if (!ml_test_cap_check(opt))
+ return false;
+
+ return true;
+}
+
+static int
+test_model_ops_opt_check(struct ml_options *opt)
+{
+ uint32_t i;
+ int ret;
+
+ /* check common opts */
+ ret = ml_test_opt_check(opt);
+ if (ret != 0)
+ return ret;
+
+ /* check model file availability */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ if (access(opt->filelist[i].model, F_OK) == -1) {
+ ml_err("Model file not available: id = %u, file = %s", i,
+ opt->filelist[i].model);
+ return -ENOENT;
+ }
+ }
+
+ return 0;
+}
+
+static void
+test_model_ops_opt_dump(struct ml_options *opt)
+{
+ uint32_t i;
+
+ /* dump common opts */
+ ml_test_opt_dump(opt);
+
+ /* dump test specific opts */
+ ml_dump_begin("models");
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_dump_list("model", i, opt->filelist[i].model);
+ ml_dump_end;
+}
+
+static int
+test_model_ops_setup(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+ void *test_model_ops;
+ int ret = 0;
+ uint32_t i;
+
+ /* allocate model ops test structure */
+ test_model_ops = rte_zmalloc_socket(test->name, sizeof(struct test_model_ops),
+ RTE_CACHE_LINE_SIZE, opt->socket_id);
+ if (test_model_ops == NULL) {
+ ml_err("Failed to allocate memory for test_model");
+ ret = -ENOMEM;
+ goto error;
+ }
+ test->test_priv = test_model_ops;
+ t = ml_test_priv(test);
+
+ t->cmn.result = ML_TEST_FAILED;
+ t->cmn.opt = opt;
+
+ /* get device info */
+ ret = rte_ml_dev_info_get(opt->dev_id, &t->cmn.dev_info);
+ if (ret < 0) {
+ ml_err("Failed to get device info");
+ goto error;
+ }
+
+ /* set model initial state */
+ for (i = 0; i < opt->nb_filelist; i++)
+ t->model[i].state = MODEL_INITIAL;
+
+ return 0;
+
+error:
+ if (test_model_ops != NULL)
+ rte_free(test_model_ops);
+
+ return ret;
+}
+
+static void
+test_model_ops_destroy(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+ if (t != NULL)
+ rte_free(t);
+}
+
+static int
+test_model_ops_mldev_setup(struct ml_test *test, struct ml_options *opt)
+{
+ int ret;
+
+ ret = ml_test_device_configure(test, opt);
+ if (ret != 0)
+ return ret;
+
+ ret = ml_test_device_start(test, opt);
+ if (ret != 0)
+ goto error;
+
+ return 0;
+
+error:
+ ml_test_device_close(test, opt);
+
+ return ret;
+}
+
+static int
+test_model_ops_mldev_destroy(struct ml_test *test, struct ml_options *opt)
+{
+ int ret;
+
+ ret = ml_test_device_stop(test, opt);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_test_device_close(test, opt);
+ if (ret != 0)
+ return ret;
+
+ return 0;
+
+error:
+ ml_test_device_close(test, opt);
+
+ return ret;
+}
+
+/* Sub-test A: (load -> start -> stop -> unload) x n */
+static int
+test_model_ops_subtest_a(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+ int ret = 0;
+ uint32_t i;
+
+ t = ml_test_priv(test);
+
+ /* load + start + stop + unload */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_load(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_start(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_stop(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_unload(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+error:
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_stop(test, opt, &t->model[i], i);
+
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_unload(test, opt, &t->model[i], i);
+
+ return ret;
+}
+
+/* Sub-test B: load x n -> start x n -> stop x n -> unload x n */
+static int
+test_model_ops_subtest_b(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+ int ret = 0;
+ uint32_t i;
+
+ t = ml_test_priv(test);
+
+ /* load */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_load(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* start */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_start(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* stop */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_stop(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* unload */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_unload(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ return 0;
+
+error:
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_stop(test, opt, &t->model[i], i);
+
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_unload(test, opt, &t->model[i], i);
+
+ return ret;
+}
+
+/* Sub-test C: load x n + (start + stop) x n + unload x n */
+static int
+test_model_ops_subtest_c(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+ int ret = 0;
+ uint32_t i;
+
+ t = ml_test_priv(test);
+
+ /* load */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_load(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* start + stop */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_start(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_stop(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* unload */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_unload(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ return 0;
+
+error:
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_stop(test, opt, &t->model[i], i);
+
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_unload(test, opt, &t->model[i], i);
+
+ return ret;
+}
+
+/* Sub-test D: (load + start) x n -> (stop + unload) x n */
+static int
+test_model_ops_subtest_d(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+ int ret = 0;
+ uint32_t i;
+
+ t = ml_test_priv(test);
+
+ /* load + start */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_load(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_start(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* stop + unload */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_stop(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_unload(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ return 0;
+
+error:
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_stop(test, opt, &t->model[i], i);
+
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_unload(test, opt, &t->model[i], i);
+
+ return ret;
+}
+
+static int
+test_model_ops_driver(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+ int ret = 0;
+
+ t = ml_test_priv(test);
+
+ /* device setup */
+ ret = test_model_ops_mldev_setup(test, opt);
+ if (ret != 0)
+ return ret;
+
+ printf("\n");
+
+ /* sub-test A */
+ ret = test_model_ops_subtest_a(test, opt);
+ if (ret != 0) {
+ printf("Model Ops Sub-test A: " CLRED "%s" CLNRM "\n", "Failed");
+ goto error;
+ } else {
+ printf("Model Ops Sub-test A: " CLYEL "%s" CLNRM "\n", "Passed");
+ }
+
+ /* sub-test B */
+ ret = test_model_ops_subtest_b(test, opt);
+ if (ret != 0) {
+ printf("Model Ops Sub-test B: " CLRED "%s" CLNRM "\n", "Failed");
+ goto error;
+ } else {
+ printf("Model Ops Sub-test B: " CLYEL "%s" CLNRM "\n", "Passed");
+ }
+
+ /* sub-test C */
+ ret = test_model_ops_subtest_c(test, opt);
+ if (ret != 0) {
+ printf("Model Ops Sub-test C: " CLRED "%s" CLNRM "\n", "Failed");
+ goto error;
+ } else {
+ printf("Model Ops Sub-test C: " CLYEL "%s" CLNRM "\n", "Passed");
+ }
+
+ /* sub-test D */
+ ret = test_model_ops_subtest_d(test, opt);
+ if (ret != 0) {
+ printf("Model Ops Sub-test D: " CLRED "%s" CLNRM "\n", "Failed");
+ goto error;
+ } else {
+ printf("Model Ops Sub-test D: " CLYEL "%s" CLNRM "\n", "Passed");
+ }
+
+ printf("\n");
+
+ /* device destroy */
+ ret = test_model_ops_mldev_destroy(test, opt);
+ if (ret != 0)
+ return ret;
+
+ t->cmn.result = ML_TEST_SUCCESS;
+
+ return 0;
+
+error:
+ test_model_ops_mldev_destroy(test, opt);
+
+ t->cmn.result = ML_TEST_FAILED;
+
+ return ret;
+}
+
+static int
+test_model_ops_result(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+
+ return t->cmn.result;
+}
+
+static const struct ml_test_ops model_ops = {
+ .cap_check = test_model_ops_cap_check,
+ .opt_check = test_model_ops_opt_check,
+ .opt_dump = test_model_ops_opt_dump,
+ .test_setup = test_model_ops_setup,
+ .test_destroy = test_model_ops_destroy,
+ .test_driver = test_model_ops_driver,
+ .test_result = test_model_ops_result,
+};
+
+ML_TEST_REGISTER(model_ops);
diff --git a/app/test-mldev/test_model_ops.h b/app/test-mldev/test_model_ops.h
new file mode 100644
index 0000000000..c21f12b788
--- /dev/null
+++ b/app/test-mldev/test_model_ops.h
@@ -0,0 +1,20 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_TEST_MODEL_OPS_
+#define _ML_TEST_MODEL_OPS_
+
+#include <rte_common.h>
+
+#include "test_model_common.h"
+
+struct test_model_ops {
+ /* common data */
+ struct test_common cmn;
+
+ /* test specific data */
+ struct ml_model model[ML_TEST_MAX_MODELS];
+} __rte_cache_aligned;
+
+#endif /* _ML_TEST_MODEL_OPS_ */
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v5 05/12] app/mldev: add ordered inference test case
2023-03-10 8:09 ` [PATCH v5 00/12] Implementation of mldev test application Srikanth Yalavarthi
` (3 preceding siblings ...)
2023-03-10 8:09 ` [PATCH v5 04/12] app/mldev: add test case to validate model ops Srikanth Yalavarthi
@ 2023-03-10 8:09 ` Srikanth Yalavarthi
2023-03-10 8:09 ` [PATCH v5 06/12] app/mldev: add test case to interleave inferences Srikanth Yalavarthi
` (6 subsequent siblings)
11 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-10 8:09 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Added an ordered test case to execute inferences with single
or multiple models. In this test case inference requests for
a model are enqueued after completion of all requests for
the previous model. Test supports inference repetitions.
Operations sequence when testing with N models and R reps,
(load -> start -> (enqueue + dequeue) x R -> stop -> unload) x N
Test case can be executed by selecting "inference_ordered" test
and repetitions can be specified through "--repetitions" argument.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/meson.build | 2 +
app/test-mldev/ml_options.c | 73 ++-
app/test-mldev/ml_options.h | 17 +-
app/test-mldev/test_inference_common.c | 560 ++++++++++++++++++++++++
app/test-mldev/test_inference_common.h | 60 +++
app/test-mldev/test_inference_ordered.c | 115 +++++
app/test-mldev/test_model_common.h | 10 +
7 files changed, 825 insertions(+), 12 deletions(-)
create mode 100644 app/test-mldev/test_inference_common.c
create mode 100644 app/test-mldev/test_inference_common.h
create mode 100644 app/test-mldev/test_inference_ordered.c
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
index b09e1ccc8a..475d76d126 100644
--- a/app/test-mldev/meson.build
+++ b/app/test-mldev/meson.build
@@ -16,6 +16,8 @@ sources = files(
'test_device_ops.c',
'test_model_common.c',
'test_model_ops.c',
+ 'test_inference_common.c',
+ 'test_inference_ordered.c',
)
deps += ['mldev']
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 91259e1427..b7215a7e88 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -23,6 +23,7 @@ ml_options_default(struct ml_options *opt)
opt->dev_id = 0;
opt->socket_id = SOCKET_ID_ANY;
opt->nb_filelist = 0;
+ opt->repetitions = 1;
opt->debug = false;
}
@@ -90,6 +91,60 @@ ml_parse_models(struct ml_options *opt, const char *arg)
return ret;
}
+static int
+ml_parse_filelist(struct ml_options *opt, const char *arg)
+{
+ const char *delim = ",";
+ char filelist[PATH_MAX];
+ char *token;
+
+ if (opt->nb_filelist >= ML_TEST_MAX_MODELS) {
+ ml_err("Exceeded filelist count, max = %d\n", ML_TEST_MAX_MODELS);
+ return -1;
+ }
+
+ strlcpy(filelist, arg, PATH_MAX);
+
+ /* model */
+ token = strtok(filelist, delim);
+ if (token == NULL) {
+ ml_err("Invalid filelist, model not specified = %s\n", arg);
+ return -EINVAL;
+ }
+ strlcpy(opt->filelist[opt->nb_filelist].model, token, PATH_MAX);
+
+ /* input */
+ token = strtok(NULL, delim);
+ if (token == NULL) {
+ ml_err("Invalid filelist, input not specified = %s\n", arg);
+ return -EINVAL;
+ }
+ strlcpy(opt->filelist[opt->nb_filelist].input, token, PATH_MAX);
+
+ /* output */
+ token = strtok(NULL, delim);
+ if (token == NULL) {
+ ml_err("Invalid filelist, output not specified = %s\n", arg);
+ return -EINVAL;
+ }
+ strlcpy(opt->filelist[opt->nb_filelist].output, token, PATH_MAX);
+
+ opt->nb_filelist++;
+
+ if (opt->nb_filelist == 0) {
+ ml_err("Empty filelist. Need at least one filelist entry for the test.");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int
+ml_parse_repetitions(struct ml_options *opt, const char *arg)
+{
+ return parser_read_uint64(&opt->repetitions, arg);
+}
+
static void
ml_dump_test_options(const char *testname)
{
@@ -100,6 +155,12 @@ ml_dump_test_options(const char *testname)
printf("\t\t--models : comma separated list of models\n");
printf("\n");
}
+
+ if (strcmp(testname, "inference_ordered") == 0) {
+ printf("\t\t--filelist : comma separated list of model, input and output\n"
+ "\t\t--repetitions : number of inference repetitions\n");
+ printf("\n");
+ }
}
static void
@@ -118,8 +179,9 @@ print_usage(char *program)
}
static struct option lgopts[] = {
- {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0}, {ML_MODELS, 1, 0, 0},
- {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
+ {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
+ {ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
+ {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
static int
ml_opts_parse_long(int opt_idx, struct ml_options *opt)
@@ -127,10 +189,9 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
unsigned int i;
struct long_opt_parser parsermap[] = {
- {ML_TEST, ml_parse_test_name},
- {ML_DEVICE_ID, ml_parse_dev_id},
- {ML_SOCKET_ID, ml_parse_socket_id},
- {ML_MODELS, ml_parse_models},
+ {ML_TEST, ml_parse_test_name}, {ML_DEVICE_ID, ml_parse_dev_id},
+ {ML_SOCKET_ID, ml_parse_socket_id}, {ML_MODELS, ml_parse_models},
+ {ML_FILELIST, ml_parse_filelist}, {ML_REPETITIONS, ml_parse_repetitions},
};
for (i = 0; i < RTE_DIM(parsermap); i++) {
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index 61e938d2e2..6a13f97a30 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -12,15 +12,19 @@
#define ML_TEST_MAX_MODELS 8
/* Options names */
-#define ML_TEST ("test")
-#define ML_DEVICE_ID ("dev_id")
-#define ML_SOCKET_ID ("socket_id")
-#define ML_MODELS ("models")
-#define ML_DEBUG ("debug")
-#define ML_HELP ("help")
+#define ML_TEST ("test")
+#define ML_DEVICE_ID ("dev_id")
+#define ML_SOCKET_ID ("socket_id")
+#define ML_MODELS ("models")
+#define ML_FILELIST ("filelist")
+#define ML_REPETITIONS ("repetitions")
+#define ML_DEBUG ("debug")
+#define ML_HELP ("help")
struct ml_filelist {
char model[PATH_MAX];
+ char input[PATH_MAX];
+ char output[PATH_MAX];
};
struct ml_options {
@@ -29,6 +33,7 @@ struct ml_options {
int socket_id;
struct ml_filelist filelist[ML_TEST_MAX_MODELS];
uint8_t nb_filelist;
+ uint64_t repetitions;
bool debug;
};
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
new file mode 100644
index 0000000000..ea06c0c1e4
--- /dev/null
+++ b/app/test-mldev/test_inference_common.c
@@ -0,0 +1,560 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <errno.h>
+#include <unistd.h>
+
+#include <rte_common.h>
+#include <rte_launch.h>
+#include <rte_lcore.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_mldev.h>
+
+#include "ml_common.h"
+#include "test_inference_common.h"
+
+/* Enqueue inference requests with burst size equal to 1 */
+static int
+ml_enqueue_single(void *arg)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)arg);
+ struct ml_request *req = NULL;
+ struct rte_ml_op *op = NULL;
+ struct ml_core_args *args;
+ uint64_t model_enq = 0;
+ uint32_t burst_enq;
+ uint32_t lcore_id;
+ uint16_t fid;
+ int ret;
+
+ lcore_id = rte_lcore_id();
+ args = &t->args[lcore_id];
+ model_enq = 0;
+
+ if (args->nb_reqs == 0)
+ return 0;
+
+next_rep:
+ fid = args->start_fid;
+
+next_model:
+ ret = rte_mempool_get(t->op_pool, (void **)&op);
+ if (ret != 0)
+ goto next_model;
+
+retry:
+ ret = rte_mempool_get(t->model[fid].io_pool, (void **)&req);
+ if (ret != 0)
+ goto retry;
+
+ op->model_id = t->model[fid].id;
+ op->nb_batches = t->model[fid].info.batch_size;
+ op->mempool = t->op_pool;
+
+ op->input.addr = req->input;
+ op->input.length = t->model[fid].inp_qsize;
+ op->input.next = NULL;
+
+ op->output.addr = req->output;
+ op->output.length = t->model[fid].out_qsize;
+ op->output.next = NULL;
+
+ op->user_ptr = req;
+ req->niters++;
+ req->fid = fid;
+
+enqueue_req:
+ burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, 0, &op, 1);
+ if (burst_enq == 0)
+ goto enqueue_req;
+
+ fid++;
+ if (likely(fid <= args->end_fid))
+ goto next_model;
+
+ model_enq++;
+ if (likely(model_enq < args->nb_reqs))
+ goto next_rep;
+
+ return 0;
+}
+
+/* Dequeue inference requests with burst size equal to 1 */
+static int
+ml_dequeue_single(void *arg)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)arg);
+ struct rte_ml_op_error error;
+ struct rte_ml_op *op = NULL;
+ struct ml_core_args *args;
+ struct ml_request *req;
+ uint64_t total_deq = 0;
+ uint8_t nb_filelist;
+ uint32_t burst_deq;
+ uint32_t lcore_id;
+
+ lcore_id = rte_lcore_id();
+ args = &t->args[lcore_id];
+ nb_filelist = args->end_fid - args->start_fid + 1;
+
+ if (args->nb_reqs == 0)
+ return 0;
+
+dequeue_req:
+ burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, 0, &op, 1);
+
+ if (likely(burst_deq == 1)) {
+ total_deq += burst_deq;
+ if (unlikely(op->status == RTE_ML_OP_STATUS_ERROR)) {
+ rte_ml_op_error_get(t->cmn.opt->dev_id, op, &error);
+ ml_err("error_code = 0x%" PRIx64 ", error_message = %s\n", error.errcode,
+ error.message);
+ }
+ req = (struct ml_request *)op->user_ptr;
+ rte_mempool_put(t->model[req->fid].io_pool, req);
+ rte_mempool_put(t->op_pool, op);
+ }
+
+ if (likely(total_deq < args->nb_reqs * nb_filelist))
+ goto dequeue_req;
+
+ return 0;
+}
+
+bool
+test_inference_cap_check(struct ml_options *opt)
+{
+ struct rte_ml_dev_info dev_info;
+
+ if (!ml_test_cap_check(opt))
+ return false;
+
+ rte_ml_dev_info_get(opt->dev_id, &dev_info);
+ if (opt->nb_filelist > dev_info.max_models) {
+ ml_err("Insufficient capabilities: Filelist count exceeded device limit, count = %u (max limit = %u)",
+ opt->nb_filelist, dev_info.max_models);
+ return false;
+ }
+
+ return true;
+}
+
+int
+test_inference_opt_check(struct ml_options *opt)
+{
+ uint32_t i;
+ int ret;
+
+ /* check common opts */
+ ret = ml_test_opt_check(opt);
+ if (ret != 0)
+ return ret;
+
+ /* check file availability */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ if (access(opt->filelist[i].model, F_OK) == -1) {
+ ml_err("Model file not accessible: id = %u, file = %s", i,
+ opt->filelist[i].model);
+ return -ENOENT;
+ }
+
+ if (access(opt->filelist[i].input, F_OK) == -1) {
+ ml_err("Input file not accessible: id = %u, file = %s", i,
+ opt->filelist[i].input);
+ return -ENOENT;
+ }
+ }
+
+ if (opt->repetitions == 0) {
+ ml_err("Invalid option, repetitions = %" PRIu64 "\n", opt->repetitions);
+ return -EINVAL;
+ }
+
+ /* check number of available lcores. */
+ if (rte_lcore_count() < 3) {
+ ml_err("Insufficient lcores = %u\n", rte_lcore_count());
+ ml_err("Minimum lcores required to create %u queue-pairs = %u\n", 1, 3);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+void
+test_inference_opt_dump(struct ml_options *opt)
+{
+ uint32_t i;
+
+ /* dump common opts */
+ ml_test_opt_dump(opt);
+
+ /* dump test opts */
+ ml_dump("repetitions", "%" PRIu64, opt->repetitions);
+
+ ml_dump_begin("filelist");
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ml_dump_list("model", i, opt->filelist[i].model);
+ ml_dump_list("input", i, opt->filelist[i].input);
+ ml_dump_list("output", i, opt->filelist[i].output);
+ }
+ ml_dump_end;
+}
+
+int
+test_inference_setup(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t;
+ void *test_inference;
+ int ret = 0;
+ uint32_t i;
+
+ test_inference = rte_zmalloc_socket(test->name, sizeof(struct test_inference),
+ RTE_CACHE_LINE_SIZE, opt->socket_id);
+ if (test_inference == NULL) {
+ ml_err("failed to allocate memory for test_model");
+ ret = -ENOMEM;
+ goto error;
+ }
+ test->test_priv = test_inference;
+ t = ml_test_priv(test);
+
+ t->nb_used = 0;
+ t->cmn.result = ML_TEST_FAILED;
+ t->cmn.opt = opt;
+
+ /* get device info */
+ ret = rte_ml_dev_info_get(opt->dev_id, &t->cmn.dev_info);
+ if (ret < 0) {
+ ml_err("failed to get device info");
+ goto error;
+ }
+
+ t->enqueue = ml_enqueue_single;
+ t->dequeue = ml_dequeue_single;
+
+ /* set model initial state */
+ for (i = 0; i < opt->nb_filelist; i++)
+ t->model[i].state = MODEL_INITIAL;
+
+ return 0;
+
+error:
+ if (test_inference != NULL)
+ rte_free(test_inference);
+
+ return ret;
+}
+
+void
+test_inference_destroy(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+ if (t != NULL)
+ rte_free(t);
+}
+
+int
+ml_inference_mldev_setup(struct ml_test *test, struct ml_options *opt)
+{
+ struct rte_ml_dev_qp_conf qp_conf;
+ struct test_inference *t;
+ int ret;
+
+ t = ml_test_priv(test);
+
+ ret = ml_test_device_configure(test, opt);
+ if (ret != 0)
+ return ret;
+
+ /* setup queue pairs */
+ qp_conf.nb_desc = t->cmn.dev_info.max_desc;
+ qp_conf.cb = NULL;
+
+ ret = rte_ml_dev_queue_pair_setup(opt->dev_id, 0, &qp_conf, opt->socket_id);
+ if (ret != 0) {
+ ml_err("Failed to setup ml device queue-pair, dev_id = %d, qp_id = %u\n",
+ opt->dev_id, 0);
+ goto error;
+ }
+
+ ret = ml_test_device_start(test, opt);
+ if (ret != 0)
+ goto error;
+
+ return 0;
+
+error:
+ ml_test_device_close(test, opt);
+
+ return ret;
+}
+
+int
+ml_inference_mldev_destroy(struct ml_test *test, struct ml_options *opt)
+{
+ int ret;
+
+ ret = ml_test_device_stop(test, opt);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_test_device_close(test, opt);
+ if (ret != 0)
+ return ret;
+
+ return 0;
+
+error:
+ ml_test_device_close(test, opt);
+
+ return ret;
+}
+
+/* Callback for IO pool create. This function would compute the fields of ml_request
+ * structure and prepare the quantized input data.
+ */
+static void
+ml_request_initialize(struct rte_mempool *mp, void *opaque, void *obj, unsigned int obj_idx)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)opaque);
+ struct ml_request *req = (struct ml_request *)obj;
+
+ RTE_SET_USED(mp);
+ RTE_SET_USED(obj_idx);
+
+ req->input = RTE_PTR_ADD(
+ obj, RTE_ALIGN_CEIL(sizeof(struct ml_request), t->cmn.dev_info.min_align_size));
+ req->output = RTE_PTR_ADD(req->input, RTE_ALIGN_CEIL(t->model[t->fid].inp_qsize,
+ t->cmn.dev_info.min_align_size));
+ req->niters = 0;
+
+ /* quantize data */
+ rte_ml_io_quantize(t->cmn.opt->dev_id, t->model[t->fid].id,
+ t->model[t->fid].info.batch_size, t->model[t->fid].input, req->input);
+}
+
+int
+ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, uint16_t fid)
+{
+ struct test_inference *t = ml_test_priv(test);
+ char mz_name[RTE_MEMZONE_NAMESIZE];
+ char mp_name[RTE_MEMPOOL_NAMESIZE];
+ const struct rte_memzone *mz;
+ uint64_t nb_buffers;
+ uint32_t buff_size;
+ uint32_t mz_size;
+ uint32_t fsize;
+ FILE *fp;
+ int ret;
+
+ /* get input buffer size */
+ ret = rte_ml_io_input_size_get(opt->dev_id, t->model[fid].id, t->model[fid].info.batch_size,
+ (uint64_t *)&t->model[fid].inp_qsize,
+ (uint64_t *)&t->model[fid].inp_dsize);
+ if (ret != 0) {
+ ml_err("Failed to get input size, model : %s\n", opt->filelist[fid].model);
+ return ret;
+ }
+
+ /* get output buffer size */
+ ret = rte_ml_io_output_size_get(
+ opt->dev_id, t->model[fid].id, t->model[fid].info.batch_size,
+ (uint64_t *)&t->model[fid].out_qsize, (uint64_t *)&t->model[fid].out_dsize);
+ if (ret != 0) {
+ ml_err("Failed to get input size, model : %s\n", opt->filelist[fid].model);
+ return ret;
+ }
+
+ /* allocate buffer for user data */
+ mz_size = t->model[fid].inp_dsize + t->model[fid].out_dsize;
+ sprintf(mz_name, "ml_user_data_%d", fid);
+ mz = rte_memzone_reserve(mz_name, mz_size, opt->socket_id, 0);
+ if (mz == NULL) {
+ ml_err("Memzone allocation failed for ml_user_data\n");
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ t->model[fid].input = mz->addr;
+ t->model[fid].output = RTE_PTR_ADD(t->model[fid].input, t->model[fid].inp_dsize);
+
+ /* load input file */
+ fp = fopen(opt->filelist[fid].input, "r");
+ if (fp == NULL) {
+ ml_err("Failed to open input file : %s\n", opt->filelist[fid].input);
+ ret = -errno;
+ goto error;
+ }
+
+ fseek(fp, 0, SEEK_END);
+ fsize = ftell(fp);
+ fseek(fp, 0, SEEK_SET);
+ if (fsize != t->model[fid].inp_dsize) {
+ ml_err("Invalid input file, size = %u (expected size = %" PRIu32 ")\n", fsize,
+ t->model[fid].inp_dsize);
+ ret = -EINVAL;
+ fclose(fp);
+ goto error;
+ }
+
+ if (fread(t->model[fid].input, 1, t->model[fid].inp_dsize, fp) != t->model[fid].inp_dsize) {
+ ml_err("Failed to read input file : %s\n", opt->filelist[fid].input);
+ ret = -errno;
+ fclose(fp);
+ goto error;
+ }
+ fclose(fp);
+
+ /* create mempool for quantized input and output buffers. ml_request_initialize is
+ * used as a callback for object creation.
+ */
+ buff_size = RTE_ALIGN_CEIL(sizeof(struct ml_request), t->cmn.dev_info.min_align_size) +
+ RTE_ALIGN_CEIL(t->model[fid].inp_qsize, t->cmn.dev_info.min_align_size) +
+ RTE_ALIGN_CEIL(t->model[fid].out_qsize, t->cmn.dev_info.min_align_size);
+ nb_buffers = RTE_MIN((uint64_t)ML_TEST_MAX_POOL_SIZE, opt->repetitions);
+
+ t->fid = fid;
+ sprintf(mp_name, "ml_io_pool_%d", fid);
+ t->model[fid].io_pool = rte_mempool_create(mp_name, nb_buffers, buff_size, 0, 0, NULL, NULL,
+ ml_request_initialize, test, opt->socket_id, 0);
+ if (t->model[fid].io_pool == NULL) {
+ ml_err("Failed to create io pool : %s\n", "ml_io_pool");
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ return 0;
+
+error:
+ if (mz != NULL)
+ rte_memzone_free(mz);
+
+ if (t->model[fid].io_pool != NULL) {
+ rte_mempool_free(t->model[fid].io_pool);
+ t->model[fid].io_pool = NULL;
+ }
+
+ return ret;
+}
+
+void
+ml_inference_iomem_destroy(struct ml_test *test, struct ml_options *opt, uint16_t fid)
+{
+ char mz_name[RTE_MEMZONE_NAMESIZE];
+ char mp_name[RTE_MEMPOOL_NAMESIZE];
+ const struct rte_memzone *mz;
+ struct rte_mempool *mp;
+
+ RTE_SET_USED(test);
+ RTE_SET_USED(opt);
+
+ /* release user data memzone */
+ sprintf(mz_name, "ml_user_data_%d", fid);
+ mz = rte_memzone_lookup(mz_name);
+ if (mz != NULL)
+ rte_memzone_free(mz);
+
+ /* destroy io pool */
+ sprintf(mp_name, "ml_io_pool_%d", fid);
+ mp = rte_mempool_lookup(mp_name);
+ if (mp != NULL)
+ rte_mempool_free(mp);
+}
+
+int
+ml_inference_mem_setup(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t = ml_test_priv(test);
+
+ /* create op pool */
+ t->op_pool = rte_ml_op_pool_create("ml_test_op_pool", ML_TEST_MAX_POOL_SIZE, 0, 0,
+ opt->socket_id);
+ if (t->op_pool == NULL) {
+ ml_err("Failed to create op pool : %s\n", "ml_op_pool");
+ return -ENOMEM;
+ }
+
+ return 0;
+}
+
+void
+ml_inference_mem_destroy(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t = ml_test_priv(test);
+
+ RTE_SET_USED(opt);
+
+ /* release op pool */
+ if (t->op_pool != NULL)
+ rte_mempool_free(t->op_pool);
+}
+
+/* Callback for mempool object iteration. This call would dequantize output data. */
+static void
+ml_request_finish(struct rte_mempool *mp, void *opaque, void *obj, unsigned int obj_idx)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)opaque);
+ struct ml_request *req = (struct ml_request *)obj;
+ struct ml_model *model = &t->model[req->fid];
+
+ RTE_SET_USED(mp);
+ RTE_SET_USED(obj_idx);
+
+ if (req->niters == 0)
+ return;
+
+ t->nb_used++;
+ rte_ml_io_dequantize(t->cmn.opt->dev_id, model->id, t->model[req->fid].info.batch_size,
+ req->output, model->output);
+}
+
+int
+ml_inference_result(struct ml_test *test, struct ml_options *opt, uint16_t fid)
+{
+ struct test_inference *t = ml_test_priv(test);
+
+ RTE_SET_USED(opt);
+
+ rte_mempool_obj_iter(t->model[fid].io_pool, ml_request_finish, test);
+
+ if (t->nb_used > 0)
+ t->cmn.result = ML_TEST_SUCCESS;
+ else
+ t->cmn.result = ML_TEST_FAILED;
+
+ return t->cmn.result;
+}
+
+int
+ml_inference_launch_cores(struct ml_test *test, struct ml_options *opt, uint16_t start_fid,
+ uint16_t end_fid)
+{
+ struct test_inference *t = ml_test_priv(test);
+ uint32_t lcore_id;
+ uint32_t id = 0;
+
+ RTE_LCORE_FOREACH_WORKER(lcore_id)
+ {
+ if (id == 2)
+ break;
+
+ t->args[lcore_id].nb_reqs = opt->repetitions;
+ t->args[lcore_id].start_fid = start_fid;
+ t->args[lcore_id].end_fid = end_fid;
+
+ if (id % 2 == 0)
+ rte_eal_remote_launch(t->enqueue, test, lcore_id);
+ else
+ rte_eal_remote_launch(t->dequeue, test, lcore_id);
+
+ id++;
+ }
+
+ return 0;
+}
diff --git a/app/test-mldev/test_inference_common.h b/app/test-mldev/test_inference_common.h
new file mode 100644
index 0000000000..aee59d11a6
--- /dev/null
+++ b/app/test-mldev/test_inference_common.h
@@ -0,0 +1,60 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_TEST_INFERENCE_COMMON_
+#define _ML_TEST_INFERENCE_COMMON_
+
+#include <rte_common.h>
+#include <rte_mempool.h>
+#include <rte_mldev.h>
+
+#include "test_model_common.h"
+
+struct ml_request {
+ uint8_t *input;
+ uint8_t *output;
+ uint16_t fid;
+ uint64_t niters;
+};
+
+struct ml_core_args {
+ uint64_t nb_reqs;
+ uint16_t start_fid;
+ uint16_t end_fid;
+};
+
+struct test_inference {
+ /* common data */
+ struct test_common cmn;
+
+ /* test specific data */
+ struct ml_model model[ML_TEST_MAX_MODELS];
+ struct rte_mempool *op_pool;
+
+ uint64_t nb_used;
+ uint16_t fid;
+
+ int (*enqueue)(void *arg);
+ int (*dequeue)(void *arg);
+
+ struct ml_core_args args[RTE_MAX_LCORE];
+} __rte_cache_aligned;
+
+bool test_inference_cap_check(struct ml_options *opt);
+int test_inference_opt_check(struct ml_options *opt);
+void test_inference_opt_dump(struct ml_options *opt);
+int test_inference_setup(struct ml_test *test, struct ml_options *opt);
+void test_inference_destroy(struct ml_test *test, struct ml_options *opt);
+
+int ml_inference_mldev_setup(struct ml_test *test, struct ml_options *opt);
+int ml_inference_mldev_destroy(struct ml_test *test, struct ml_options *opt);
+int ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, uint16_t fid);
+void ml_inference_iomem_destroy(struct ml_test *test, struct ml_options *opt, uint16_t fid);
+int ml_inference_mem_setup(struct ml_test *test, struct ml_options *opt);
+void ml_inference_mem_destroy(struct ml_test *test, struct ml_options *opt);
+int ml_inference_result(struct ml_test *test, struct ml_options *opt, uint16_t fid);
+int ml_inference_launch_cores(struct ml_test *test, struct ml_options *opt, uint16_t start_fid,
+ uint16_t end_fid);
+
+#endif /* _ML_TEST_INFERENCE_COMMON_ */
diff --git a/app/test-mldev/test_inference_ordered.c b/app/test-mldev/test_inference_ordered.c
new file mode 100644
index 0000000000..1cd91dc3d3
--- /dev/null
+++ b/app/test-mldev/test_inference_ordered.c
@@ -0,0 +1,115 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <rte_common.h>
+#include <rte_launch.h>
+
+#include "ml_common.h"
+#include "test_inference_common.h"
+
+static int
+test_inference_ordered_driver(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t;
+ uint16_t fid = 0;
+ int ret = 0;
+
+ t = ml_test_priv(test);
+
+ ret = ml_inference_mldev_setup(test, opt);
+ if (ret != 0)
+ return ret;
+
+ ret = ml_inference_mem_setup(test, opt);
+ if (ret != 0)
+ return ret;
+
+next_model:
+ /* load model */
+ ret = ml_model_load(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ /* start model */
+ ret = ml_model_start(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_inference_iomem_setup(test, opt, fid);
+ if (ret != 0)
+ goto error;
+
+ /* launch inferences for one model using available queue pairs */
+ ret = ml_inference_launch_cores(test, opt, fid, fid);
+ if (ret != 0) {
+ ml_err("failed to launch cores");
+ goto error;
+ }
+
+ rte_eal_mp_wait_lcore();
+
+ ret = ml_inference_result(test, opt, fid);
+ if (ret != ML_TEST_SUCCESS)
+ goto error;
+
+ ml_inference_iomem_destroy(test, opt, fid);
+
+ /* stop model */
+ ret = ml_model_stop(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ /* unload model */
+ ret = ml_model_unload(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ fid++;
+ if (fid < opt->nb_filelist)
+ goto next_model;
+
+ ml_inference_mem_destroy(test, opt);
+
+ ret = ml_inference_mldev_destroy(test, opt);
+ if (ret != 0)
+ return ret;
+
+ t->cmn.result = ML_TEST_SUCCESS;
+
+ return 0;
+
+error:
+ ml_inference_iomem_destroy(test, opt, fid);
+ ml_inference_mem_destroy(test, opt);
+ ml_model_stop(test, opt, &t->model[fid], fid);
+ ml_model_unload(test, opt, &t->model[fid], fid);
+
+ t->cmn.result = ML_TEST_FAILED;
+
+ return ret;
+}
+
+static int
+test_inference_ordered_result(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+
+ return t->cmn.result;
+}
+
+static const struct ml_test_ops inference_ordered = {
+ .cap_check = test_inference_cap_check,
+ .opt_check = test_inference_opt_check,
+ .opt_dump = test_inference_opt_dump,
+ .test_setup = test_inference_setup,
+ .test_destroy = test_inference_destroy,
+ .test_driver = test_inference_ordered_driver,
+ .test_result = test_inference_ordered_result,
+};
+
+ML_TEST_REGISTER(inference_ordered);
diff --git a/app/test-mldev/test_model_common.h b/app/test-mldev/test_model_common.h
index 74aec0a797..1c89ef83aa 100644
--- a/app/test-mldev/test_model_common.h
+++ b/app/test-mldev/test_model_common.h
@@ -20,6 +20,16 @@ struct ml_model {
uint16_t id;
struct rte_ml_model_info info;
enum model_state state;
+
+ uint32_t inp_dsize;
+ uint32_t inp_qsize;
+ uint32_t out_dsize;
+ uint32_t out_qsize;
+
+ uint8_t *input;
+ uint8_t *output;
+
+ struct rte_mempool *io_pool;
};
int ml_model_load(struct ml_test *test, struct ml_options *opt, struct ml_model *model,
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v5 06/12] app/mldev: add test case to interleave inferences
2023-03-10 8:09 ` [PATCH v5 00/12] Implementation of mldev test application Srikanth Yalavarthi
` (4 preceding siblings ...)
2023-03-10 8:09 ` [PATCH v5 05/12] app/mldev: add ordered inference test case Srikanth Yalavarthi
@ 2023-03-10 8:09 ` Srikanth Yalavarthi
2023-03-10 8:09 ` [PATCH v5 07/12] app/mldev: enable support for burst inferences Srikanth Yalavarthi
` (5 subsequent siblings)
11 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-10 8:09 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Added test case to interleave inference requests from multiple
models. Interleaving would load and start all models and launch
inference requests for the models using available queue-pairs
Operations sequence when testing with N models and R reps,
(load + start) x N -> (enqueue + dequeue) x N x R ...
-> (stop + unload) x N
Test can be executed by selecting "inference_interleave" test.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/meson.build | 1 +
app/test-mldev/ml_options.c | 3 +-
app/test-mldev/test_inference_interleave.c | 114 +++++++++++++++++++++
3 files changed, 117 insertions(+), 1 deletion(-)
create mode 100644 app/test-mldev/test_inference_interleave.c
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
index 475d76d126..41d22fb22c 100644
--- a/app/test-mldev/meson.build
+++ b/app/test-mldev/meson.build
@@ -18,6 +18,7 @@ sources = files(
'test_model_ops.c',
'test_inference_common.c',
'test_inference_ordered.c',
+ 'test_inference_interleave.c',
)
deps += ['mldev']
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index b7215a7e88..f9e3ce8e6f 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -156,7 +156,8 @@ ml_dump_test_options(const char *testname)
printf("\n");
}
- if (strcmp(testname, "inference_ordered") == 0) {
+ if ((strcmp(testname, "inference_ordered") == 0) ||
+ (strcmp(testname, "inference_interleave") == 0)) {
printf("\t\t--filelist : comma separated list of model, input and output\n"
"\t\t--repetitions : number of inference repetitions\n");
printf("\n");
diff --git a/app/test-mldev/test_inference_interleave.c b/app/test-mldev/test_inference_interleave.c
new file mode 100644
index 0000000000..9cf4cfa197
--- /dev/null
+++ b/app/test-mldev/test_inference_interleave.c
@@ -0,0 +1,114 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <rte_common.h>
+#include <rte_launch.h>
+
+#include "ml_common.h"
+#include "test_inference_common.h"
+
+static int
+test_inference_interleave_driver(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t;
+ uint16_t fid = 0;
+ int ret = 0;
+
+ t = ml_test_priv(test);
+
+ ret = ml_inference_mldev_setup(test, opt);
+ if (ret != 0)
+ return ret;
+
+ ret = ml_inference_mem_setup(test, opt);
+ if (ret != 0)
+ return ret;
+
+ /* load and start all models */
+ for (fid = 0; fid < opt->nb_filelist; fid++) {
+ ret = ml_model_load(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_start(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_inference_iomem_setup(test, opt, fid);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* launch inference requests */
+ ret = ml_inference_launch_cores(test, opt, 0, opt->nb_filelist - 1);
+ if (ret != 0) {
+ ml_err("failed to launch cores");
+ goto error;
+ }
+
+ rte_eal_mp_wait_lcore();
+
+ /* stop and unload all models */
+ for (fid = 0; fid < opt->nb_filelist; fid++) {
+ ret = ml_inference_result(test, opt, fid);
+ if (ret != ML_TEST_SUCCESS)
+ goto error;
+
+ ml_inference_iomem_destroy(test, opt, fid);
+
+ ret = ml_model_stop(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_unload(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+ }
+
+ ml_inference_mem_destroy(test, opt);
+
+ ret = ml_inference_mldev_destroy(test, opt);
+ if (ret != 0)
+ return ret;
+
+ t->cmn.result = ML_TEST_SUCCESS;
+
+ return 0;
+
+error:
+ ml_inference_mem_destroy(test, opt);
+ for (fid = 0; fid < opt->nb_filelist; fid++) {
+ ml_inference_iomem_destroy(test, opt, fid);
+ ml_model_stop(test, opt, &t->model[fid], fid);
+ ml_model_unload(test, opt, &t->model[fid], fid);
+ }
+
+ t->cmn.result = ML_TEST_FAILED;
+
+ return ret;
+}
+
+static int
+test_inference_interleave_result(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+
+ return t->cmn.result;
+}
+
+static const struct ml_test_ops inference_interleave = {
+ .cap_check = test_inference_cap_check,
+ .opt_check = test_inference_opt_check,
+ .opt_dump = test_inference_opt_dump,
+ .test_setup = test_inference_setup,
+ .test_destroy = test_inference_destroy,
+ .test_driver = test_inference_interleave_driver,
+ .test_result = test_inference_interleave_result,
+};
+
+ML_TEST_REGISTER(inference_interleave);
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v5 07/12] app/mldev: enable support for burst inferences
2023-03-10 8:09 ` [PATCH v5 00/12] Implementation of mldev test application Srikanth Yalavarthi
` (5 preceding siblings ...)
2023-03-10 8:09 ` [PATCH v5 06/12] app/mldev: add test case to interleave inferences Srikanth Yalavarthi
@ 2023-03-10 8:09 ` Srikanth Yalavarthi
2023-03-10 8:09 ` [PATCH v5 08/12] app/mldev: enable support for queue pairs and size Srikanth Yalavarthi
` (4 subsequent siblings)
11 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-10 8:09 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Added 'burst_size' support for inference tests. Burst size
controls the number of inference requests handled during
the burst enqueue and dequeue operations of the test case.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/ml_options.c | 26 ++--
app/test-mldev/ml_options.h | 2 +
app/test-mldev/test_inference_common.c | 159 ++++++++++++++++++++++++-
app/test-mldev/test_inference_common.h | 4 +
4 files changed, 181 insertions(+), 10 deletions(-)
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index f9e3ce8e6f..7fbf9c5678 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -24,6 +24,7 @@ ml_options_default(struct ml_options *opt)
opt->socket_id = SOCKET_ID_ANY;
opt->nb_filelist = 0;
opt->repetitions = 1;
+ opt->burst_size = 1;
opt->debug = false;
}
@@ -145,6 +146,12 @@ ml_parse_repetitions(struct ml_options *opt, const char *arg)
return parser_read_uint64(&opt->repetitions, arg);
}
+static int
+ml_parse_burst_size(struct ml_options *opt, const char *arg)
+{
+ return parser_read_uint16(&opt->burst_size, arg);
+}
+
static void
ml_dump_test_options(const char *testname)
{
@@ -159,7 +166,8 @@ ml_dump_test_options(const char *testname)
if ((strcmp(testname, "inference_ordered") == 0) ||
(strcmp(testname, "inference_interleave") == 0)) {
printf("\t\t--filelist : comma separated list of model, input and output\n"
- "\t\t--repetitions : number of inference repetitions\n");
+ "\t\t--repetitions : number of inference repetitions\n"
+ "\t\t--burst_size : inference burst size\n");
printf("\n");
}
}
@@ -179,10 +187,11 @@ print_usage(char *program)
ml_test_dump_names(ml_dump_test_options);
}
-static struct option lgopts[] = {
- {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
- {ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
- {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
+static struct option lgopts[] = {{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0},
+ {ML_SOCKET_ID, 1, 0, 0}, {ML_MODELS, 1, 0, 0},
+ {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
+ {ML_BURST_SIZE, 1, 0, 0}, {ML_DEBUG, 0, 0, 0},
+ {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
static int
ml_opts_parse_long(int opt_idx, struct ml_options *opt)
@@ -190,9 +199,10 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
unsigned int i;
struct long_opt_parser parsermap[] = {
- {ML_TEST, ml_parse_test_name}, {ML_DEVICE_ID, ml_parse_dev_id},
- {ML_SOCKET_ID, ml_parse_socket_id}, {ML_MODELS, ml_parse_models},
- {ML_FILELIST, ml_parse_filelist}, {ML_REPETITIONS, ml_parse_repetitions},
+ {ML_TEST, ml_parse_test_name}, {ML_DEVICE_ID, ml_parse_dev_id},
+ {ML_SOCKET_ID, ml_parse_socket_id}, {ML_MODELS, ml_parse_models},
+ {ML_FILELIST, ml_parse_filelist}, {ML_REPETITIONS, ml_parse_repetitions},
+ {ML_BURST_SIZE, ml_parse_burst_size},
};
for (i = 0; i < RTE_DIM(parsermap); i++) {
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index 6a13f97a30..00342d8a0c 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -18,6 +18,7 @@
#define ML_MODELS ("models")
#define ML_FILELIST ("filelist")
#define ML_REPETITIONS ("repetitions")
+#define ML_BURST_SIZE ("burst_size")
#define ML_DEBUG ("debug")
#define ML_HELP ("help")
@@ -34,6 +35,7 @@ struct ml_options {
struct ml_filelist filelist[ML_TEST_MAX_MODELS];
uint8_t nb_filelist;
uint64_t repetitions;
+ uint16_t burst_size;
bool debug;
};
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
index ea06c0c1e4..530872def6 100644
--- a/app/test-mldev/test_inference_common.c
+++ b/app/test-mldev/test_inference_common.c
@@ -123,6 +123,131 @@ ml_dequeue_single(void *arg)
return 0;
}
+/* Enqueue inference requests with burst size greater than 1 */
+static int
+ml_enqueue_burst(void *arg)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)arg);
+ struct ml_core_args *args;
+ uint16_t ops_count;
+ uint64_t model_enq;
+ uint16_t burst_enq;
+ uint32_t lcore_id;
+ uint16_t pending;
+ uint16_t idx;
+ uint16_t fid;
+ uint16_t i;
+ int ret;
+
+ lcore_id = rte_lcore_id();
+ args = &t->args[lcore_id];
+ model_enq = 0;
+
+ if (args->nb_reqs == 0)
+ return 0;
+
+next_rep:
+ fid = args->start_fid;
+
+next_model:
+ ops_count = RTE_MIN(t->cmn.opt->burst_size, args->nb_reqs - model_enq);
+ ret = rte_mempool_get_bulk(t->op_pool, (void **)args->enq_ops, ops_count);
+ if (ret != 0)
+ goto next_model;
+
+retry:
+ ret = rte_mempool_get_bulk(t->model[fid].io_pool, (void **)args->reqs, ops_count);
+ if (ret != 0)
+ goto retry;
+
+ for (i = 0; i < ops_count; i++) {
+ args->enq_ops[i]->model_id = t->model[fid].id;
+ args->enq_ops[i]->nb_batches = t->model[fid].info.batch_size;
+ args->enq_ops[i]->mempool = t->op_pool;
+
+ args->enq_ops[i]->input.addr = args->reqs[i]->input;
+ args->enq_ops[i]->input.length = t->model[fid].inp_qsize;
+ args->enq_ops[i]->input.next = NULL;
+
+ args->enq_ops[i]->output.addr = args->reqs[i]->output;
+ args->enq_ops[i]->output.length = t->model[fid].out_qsize;
+ args->enq_ops[i]->output.next = NULL;
+
+ args->enq_ops[i]->user_ptr = args->reqs[i];
+ args->reqs[i]->niters++;
+ args->reqs[i]->fid = fid;
+ }
+
+ idx = 0;
+ pending = ops_count;
+
+enqueue_reqs:
+ burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, 0, &args->enq_ops[idx], pending);
+ pending = pending - burst_enq;
+
+ if (pending > 0) {
+ idx = idx + burst_enq;
+ goto enqueue_reqs;
+ }
+
+ fid++;
+ if (fid <= args->end_fid)
+ goto next_model;
+
+ model_enq = model_enq + ops_count;
+ if (model_enq < args->nb_reqs)
+ goto next_rep;
+
+ return 0;
+}
+
+/* Dequeue inference requests with burst size greater than 1 */
+static int
+ml_dequeue_burst(void *arg)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)arg);
+ struct rte_ml_op_error error;
+ struct ml_core_args *args;
+ struct ml_request *req;
+ uint64_t total_deq = 0;
+ uint16_t burst_deq = 0;
+ uint8_t nb_filelist;
+ uint32_t lcore_id;
+ uint32_t i;
+
+ lcore_id = rte_lcore_id();
+ args = &t->args[lcore_id];
+ nb_filelist = args->end_fid - args->start_fid + 1;
+
+ if (args->nb_reqs == 0)
+ return 0;
+
+dequeue_burst:
+ burst_deq =
+ rte_ml_dequeue_burst(t->cmn.opt->dev_id, 0, args->deq_ops, t->cmn.opt->burst_size);
+
+ if (likely(burst_deq > 0)) {
+ total_deq += burst_deq;
+
+ for (i = 0; i < burst_deq; i++) {
+ if (unlikely(args->deq_ops[i]->status == RTE_ML_OP_STATUS_ERROR)) {
+ rte_ml_op_error_get(t->cmn.opt->dev_id, args->deq_ops[i], &error);
+ ml_err("error_code = 0x%" PRIx64 ", error_message = %s\n",
+ error.errcode, error.message);
+ }
+ req = (struct ml_request *)args->deq_ops[i]->user_ptr;
+ if (req != NULL)
+ rte_mempool_put(t->model[req->fid].io_pool, req);
+ }
+ rte_mempool_put_bulk(t->op_pool, (void *)args->deq_ops, burst_deq);
+ }
+
+ if (total_deq < args->nb_reqs * nb_filelist)
+ goto dequeue_burst;
+
+ return 0;
+}
+
bool
test_inference_cap_check(struct ml_options *opt)
{
@@ -172,6 +297,17 @@ test_inference_opt_check(struct ml_options *opt)
return -EINVAL;
}
+ if (opt->burst_size == 0) {
+ ml_err("Invalid option, burst_size = %u\n", opt->burst_size);
+ return -EINVAL;
+ }
+
+ if (opt->burst_size > ML_TEST_MAX_POOL_SIZE) {
+ ml_err("Invalid option, burst_size = %u (> max supported = %d)\n", opt->burst_size,
+ ML_TEST_MAX_POOL_SIZE);
+ return -EINVAL;
+ }
+
/* check number of available lcores. */
if (rte_lcore_count() < 3) {
ml_err("Insufficient lcores = %u\n", rte_lcore_count());
@@ -192,6 +328,7 @@ test_inference_opt_dump(struct ml_options *opt)
/* dump test opts */
ml_dump("repetitions", "%" PRIu64, opt->repetitions);
+ ml_dump("burst_size", "%u", opt->burst_size);
ml_dump_begin("filelist");
for (i = 0; i < opt->nb_filelist; i++) {
@@ -207,6 +344,7 @@ test_inference_setup(struct ml_test *test, struct ml_options *opt)
{
struct test_inference *t;
void *test_inference;
+ uint32_t lcore_id;
int ret = 0;
uint32_t i;
@@ -231,13 +369,30 @@ test_inference_setup(struct ml_test *test, struct ml_options *opt)
goto error;
}
- t->enqueue = ml_enqueue_single;
- t->dequeue = ml_dequeue_single;
+ if (opt->burst_size == 1) {
+ t->enqueue = ml_enqueue_single;
+ t->dequeue = ml_dequeue_single;
+ } else {
+ t->enqueue = ml_enqueue_burst;
+ t->dequeue = ml_dequeue_burst;
+ }
/* set model initial state */
for (i = 0; i < opt->nb_filelist; i++)
t->model[i].state = MODEL_INITIAL;
+ for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) {
+ t->args[lcore_id].enq_ops = rte_zmalloc_socket(
+ "ml_test_enq_ops", opt->burst_size * sizeof(struct rte_ml_op *),
+ RTE_CACHE_LINE_SIZE, opt->socket_id);
+ t->args[lcore_id].deq_ops = rte_zmalloc_socket(
+ "ml_test_deq_ops", opt->burst_size * sizeof(struct rte_ml_op *),
+ RTE_CACHE_LINE_SIZE, opt->socket_id);
+ t->args[lcore_id].reqs = rte_zmalloc_socket(
+ "ml_test_requests", opt->burst_size * sizeof(struct ml_request *),
+ RTE_CACHE_LINE_SIZE, opt->socket_id);
+ }
+
return 0;
error:
diff --git a/app/test-mldev/test_inference_common.h b/app/test-mldev/test_inference_common.h
index aee59d11a6..d0307a6432 100644
--- a/app/test-mldev/test_inference_common.h
+++ b/app/test-mldev/test_inference_common.h
@@ -22,6 +22,10 @@ struct ml_core_args {
uint64_t nb_reqs;
uint16_t start_fid;
uint16_t end_fid;
+
+ struct rte_ml_op **enq_ops;
+ struct rte_ml_op **deq_ops;
+ struct ml_request **reqs;
};
struct test_inference {
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v5 08/12] app/mldev: enable support for queue pairs and size
2023-03-10 8:09 ` [PATCH v5 00/12] Implementation of mldev test application Srikanth Yalavarthi
` (6 preceding siblings ...)
2023-03-10 8:09 ` [PATCH v5 07/12] app/mldev: enable support for burst inferences Srikanth Yalavarthi
@ 2023-03-10 8:09 ` Srikanth Yalavarthi
2023-03-10 8:09 ` [PATCH v5 09/12] app/mldev: enable support for inference batches Srikanth Yalavarthi
` (3 subsequent siblings)
11 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-10 8:09 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Added support to create multiple queue-pairs per device to
enqueue and dequeue inference requests. Number of queue pairs
to be created can be specified through "--queue_pairs" option.
Support is also enabled to control the number of descriptors
per each queue pair through "--queue_size" option. Inference
requests for a model are distributed across all available
queue-pairs.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/ml_options.c | 40 ++++++++++---
app/test-mldev/ml_options.h | 4 ++
app/test-mldev/test_common.c | 2 +-
app/test-mldev/test_inference_common.c | 79 +++++++++++++++++++++-----
app/test-mldev/test_inference_common.h | 1 +
5 files changed, 102 insertions(+), 24 deletions(-)
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 7fbf9c5678..c81dec6e30 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -25,6 +25,8 @@ ml_options_default(struct ml_options *opt)
opt->nb_filelist = 0;
opt->repetitions = 1;
opt->burst_size = 1;
+ opt->queue_pairs = 1;
+ opt->queue_size = 1;
opt->debug = false;
}
@@ -152,11 +154,30 @@ ml_parse_burst_size(struct ml_options *opt, const char *arg)
return parser_read_uint16(&opt->burst_size, arg);
}
+static int
+ml_parse_queue_pairs(struct ml_options *opt, const char *arg)
+{
+ int ret;
+
+ ret = parser_read_uint16(&opt->queue_pairs, arg);
+
+ return ret;
+}
+
+static int
+ml_parse_queue_size(struct ml_options *opt, const char *arg)
+{
+ return parser_read_uint16(&opt->queue_size, arg);
+}
+
static void
ml_dump_test_options(const char *testname)
{
- if (strcmp(testname, "device_ops") == 0)
+ if (strcmp(testname, "device_ops") == 0) {
+ printf("\t\t--queue_pairs : number of queue pairs to create\n"
+ "\t\t--queue_size : size fo queue-pair\n");
printf("\n");
+ }
if (strcmp(testname, "model_ops") == 0) {
printf("\t\t--models : comma separated list of models\n");
@@ -167,7 +188,9 @@ ml_dump_test_options(const char *testname)
(strcmp(testname, "inference_interleave") == 0)) {
printf("\t\t--filelist : comma separated list of model, input and output\n"
"\t\t--repetitions : number of inference repetitions\n"
- "\t\t--burst_size : inference burst size\n");
+ "\t\t--burst_size : inference burst size\n"
+ "\t\t--queue_pairs : number of queue pairs to create\n"
+ "\t\t--queue_size : size fo queue-pair\n");
printf("\n");
}
}
@@ -187,11 +210,11 @@ print_usage(char *program)
ml_test_dump_names(ml_dump_test_options);
}
-static struct option lgopts[] = {{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0},
- {ML_SOCKET_ID, 1, 0, 0}, {ML_MODELS, 1, 0, 0},
- {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
- {ML_BURST_SIZE, 1, 0, 0}, {ML_DEBUG, 0, 0, 0},
- {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
+static struct option lgopts[] = {
+ {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
+ {ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
+ {ML_BURST_SIZE, 1, 0, 0}, {ML_QUEUE_PAIRS, 1, 0, 0}, {ML_QUEUE_SIZE, 1, 0, 0},
+ {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
static int
ml_opts_parse_long(int opt_idx, struct ml_options *opt)
@@ -202,7 +225,8 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
{ML_TEST, ml_parse_test_name}, {ML_DEVICE_ID, ml_parse_dev_id},
{ML_SOCKET_ID, ml_parse_socket_id}, {ML_MODELS, ml_parse_models},
{ML_FILELIST, ml_parse_filelist}, {ML_REPETITIONS, ml_parse_repetitions},
- {ML_BURST_SIZE, ml_parse_burst_size},
+ {ML_BURST_SIZE, ml_parse_burst_size}, {ML_QUEUE_PAIRS, ml_parse_queue_pairs},
+ {ML_QUEUE_SIZE, ml_parse_queue_size},
};
for (i = 0; i < RTE_DIM(parsermap); i++) {
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index 00342d8a0c..c4018ee9d1 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -19,6 +19,8 @@
#define ML_FILELIST ("filelist")
#define ML_REPETITIONS ("repetitions")
#define ML_BURST_SIZE ("burst_size")
+#define ML_QUEUE_PAIRS ("queue_pairs")
+#define ML_QUEUE_SIZE ("queue_size")
#define ML_DEBUG ("debug")
#define ML_HELP ("help")
@@ -36,6 +38,8 @@ struct ml_options {
uint8_t nb_filelist;
uint64_t repetitions;
uint16_t burst_size;
+ uint16_t queue_pairs;
+ uint16_t queue_size;
bool debug;
};
diff --git a/app/test-mldev/test_common.c b/app/test-mldev/test_common.c
index 8c4da4609a..016b31c6ba 100644
--- a/app/test-mldev/test_common.c
+++ b/app/test-mldev/test_common.c
@@ -75,7 +75,7 @@ ml_test_device_configure(struct ml_test *test, struct ml_options *opt)
/* configure device */
dev_config.socket_id = opt->socket_id;
dev_config.nb_models = t->dev_info.max_models;
- dev_config.nb_queue_pairs = t->dev_info.max_queue_pairs;
+ dev_config.nb_queue_pairs = opt->queue_pairs;
ret = rte_ml_dev_configure(opt->dev_id, &dev_config);
if (ret != 0) {
ml_err("Failed to configure ml device, dev_id = %d\n", opt->dev_id);
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
index 530872def6..7637d0833e 100644
--- a/app/test-mldev/test_inference_common.c
+++ b/app/test-mldev/test_inference_common.c
@@ -66,7 +66,7 @@ ml_enqueue_single(void *arg)
req->fid = fid;
enqueue_req:
- burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, 0, &op, 1);
+ burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, args->qp_id, &op, 1);
if (burst_enq == 0)
goto enqueue_req;
@@ -103,7 +103,7 @@ ml_dequeue_single(void *arg)
return 0;
dequeue_req:
- burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, 0, &op, 1);
+ burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, args->qp_id, &op, 1);
if (likely(burst_deq == 1)) {
total_deq += burst_deq;
@@ -182,7 +182,8 @@ ml_enqueue_burst(void *arg)
pending = ops_count;
enqueue_reqs:
- burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, 0, &args->enq_ops[idx], pending);
+ burst_enq =
+ rte_ml_enqueue_burst(t->cmn.opt->dev_id, args->qp_id, &args->enq_ops[idx], pending);
pending = pending - burst_enq;
if (pending > 0) {
@@ -223,8 +224,8 @@ ml_dequeue_burst(void *arg)
return 0;
dequeue_burst:
- burst_deq =
- rte_ml_dequeue_burst(t->cmn.opt->dev_id, 0, args->deq_ops, t->cmn.opt->burst_size);
+ burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, args->qp_id, args->deq_ops,
+ t->cmn.opt->burst_size);
if (likely(burst_deq > 0)) {
total_deq += burst_deq;
@@ -257,6 +258,19 @@ test_inference_cap_check(struct ml_options *opt)
return false;
rte_ml_dev_info_get(opt->dev_id, &dev_info);
+
+ if (opt->queue_pairs > dev_info.max_queue_pairs) {
+ ml_err("Insufficient capabilities: queue_pairs = %u, max_queue_pairs = %u",
+ opt->queue_pairs, dev_info.max_queue_pairs);
+ return false;
+ }
+
+ if (opt->queue_size > dev_info.max_desc) {
+ ml_err("Insufficient capabilities: queue_size = %u, max_desc = %u", opt->queue_size,
+ dev_info.max_desc);
+ return false;
+ }
+
if (opt->nb_filelist > dev_info.max_models) {
ml_err("Insufficient capabilities: Filelist count exceeded device limit, count = %u (max limit = %u)",
opt->nb_filelist, dev_info.max_models);
@@ -308,10 +322,21 @@ test_inference_opt_check(struct ml_options *opt)
return -EINVAL;
}
+ if (opt->queue_pairs == 0) {
+ ml_err("Invalid option, queue_pairs = %u\n", opt->queue_pairs);
+ return -EINVAL;
+ }
+
+ if (opt->queue_size == 0) {
+ ml_err("Invalid option, queue_size = %u\n", opt->queue_size);
+ return -EINVAL;
+ }
+
/* check number of available lcores. */
- if (rte_lcore_count() < 3) {
+ if (rte_lcore_count() < (uint32_t)(opt->queue_pairs * 2 + 1)) {
ml_err("Insufficient lcores = %u\n", rte_lcore_count());
- ml_err("Minimum lcores required to create %u queue-pairs = %u\n", 1, 3);
+ ml_err("Minimum lcores required to create %u queue-pairs = %u\n", opt->queue_pairs,
+ (opt->queue_pairs * 2 + 1));
return -EINVAL;
}
@@ -329,6 +354,8 @@ test_inference_opt_dump(struct ml_options *opt)
/* dump test opts */
ml_dump("repetitions", "%" PRIu64, opt->repetitions);
ml_dump("burst_size", "%u", opt->burst_size);
+ ml_dump("queue_pairs", "%u", opt->queue_pairs);
+ ml_dump("queue_size", "%u", opt->queue_size);
ml_dump_begin("filelist");
for (i = 0; i < opt->nb_filelist; i++) {
@@ -419,23 +446,31 @@ ml_inference_mldev_setup(struct ml_test *test, struct ml_options *opt)
{
struct rte_ml_dev_qp_conf qp_conf;
struct test_inference *t;
+ uint16_t qp_id;
int ret;
t = ml_test_priv(test);
+ RTE_SET_USED(t);
+
ret = ml_test_device_configure(test, opt);
if (ret != 0)
return ret;
/* setup queue pairs */
- qp_conf.nb_desc = t->cmn.dev_info.max_desc;
+ qp_conf.nb_desc = opt->queue_size;
qp_conf.cb = NULL;
- ret = rte_ml_dev_queue_pair_setup(opt->dev_id, 0, &qp_conf, opt->socket_id);
- if (ret != 0) {
- ml_err("Failed to setup ml device queue-pair, dev_id = %d, qp_id = %u\n",
- opt->dev_id, 0);
- goto error;
+ for (qp_id = 0; qp_id < opt->queue_pairs; qp_id++) {
+ qp_conf.nb_desc = opt->queue_size;
+ qp_conf.cb = NULL;
+
+ ret = rte_ml_dev_queue_pair_setup(opt->dev_id, qp_id, &qp_conf, opt->socket_id);
+ if (ret != 0) {
+ ml_err("Failed to setup ml device queue-pair, dev_id = %d, qp_id = %u\n",
+ opt->dev_id, qp_id);
+ return ret;
+ }
}
ret = ml_test_device_start(test, opt);
@@ -692,14 +727,28 @@ ml_inference_launch_cores(struct ml_test *test, struct ml_options *opt, uint16_t
{
struct test_inference *t = ml_test_priv(test);
uint32_t lcore_id;
+ uint32_t nb_reqs;
uint32_t id = 0;
+ uint32_t qp_id;
+
+ nb_reqs = opt->repetitions / opt->queue_pairs;
RTE_LCORE_FOREACH_WORKER(lcore_id)
{
- if (id == 2)
+ if (id >= opt->queue_pairs * 2)
break;
- t->args[lcore_id].nb_reqs = opt->repetitions;
+ qp_id = id / 2;
+ t->args[lcore_id].qp_id = qp_id;
+ t->args[lcore_id].nb_reqs = nb_reqs;
+ if (qp_id == 0)
+ t->args[lcore_id].nb_reqs += opt->repetitions - nb_reqs * opt->queue_pairs;
+
+ if (t->args[lcore_id].nb_reqs == 0) {
+ id++;
+ break;
+ }
+
t->args[lcore_id].start_fid = start_fid;
t->args[lcore_id].end_fid = end_fid;
diff --git a/app/test-mldev/test_inference_common.h b/app/test-mldev/test_inference_common.h
index d0307a6432..b85fb464b1 100644
--- a/app/test-mldev/test_inference_common.h
+++ b/app/test-mldev/test_inference_common.h
@@ -22,6 +22,7 @@ struct ml_core_args {
uint64_t nb_reqs;
uint16_t start_fid;
uint16_t end_fid;
+ uint32_t qp_id;
struct rte_ml_op **enq_ops;
struct rte_ml_op **deq_ops;
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v5 09/12] app/mldev: enable support for inference batches
2023-03-10 8:09 ` [PATCH v5 00/12] Implementation of mldev test application Srikanth Yalavarthi
` (7 preceding siblings ...)
2023-03-10 8:09 ` [PATCH v5 08/12] app/mldev: enable support for queue pairs and size Srikanth Yalavarthi
@ 2023-03-10 8:09 ` Srikanth Yalavarthi
2023-03-10 8:09 ` [PATCH v5 10/12] app/mldev: enable support for inference validation Srikanth Yalavarthi
` (2 subsequent siblings)
11 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-10 8:09 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Enabled support to execute multiple batches of inferences
per each enqueue request. Input and reference for the test
should be appropriately provided for multi-batch run. Number
of batches can be specified through "--batches" option.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/ml_options.c | 15 ++++++++++++---
app/test-mldev/ml_options.h | 2 ++
app/test-mldev/test_inference_common.c | 23 ++++++++++++++---------
app/test-mldev/test_model_common.c | 6 ++++++
app/test-mldev/test_model_common.h | 1 +
5 files changed, 35 insertions(+), 12 deletions(-)
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index c81dec6e30..499bfde899 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -27,6 +27,7 @@ ml_options_default(struct ml_options *opt)
opt->burst_size = 1;
opt->queue_pairs = 1;
opt->queue_size = 1;
+ opt->batches = 0;
opt->debug = false;
}
@@ -170,6 +171,12 @@ ml_parse_queue_size(struct ml_options *opt, const char *arg)
return parser_read_uint16(&opt->queue_size, arg);
}
+static int
+ml_parse_batches(struct ml_options *opt, const char *arg)
+{
+ return parser_read_uint16(&opt->batches, arg);
+}
+
static void
ml_dump_test_options(const char *testname)
{
@@ -190,7 +197,8 @@ ml_dump_test_options(const char *testname)
"\t\t--repetitions : number of inference repetitions\n"
"\t\t--burst_size : inference burst size\n"
"\t\t--queue_pairs : number of queue pairs to create\n"
- "\t\t--queue_size : size fo queue-pair\n");
+ "\t\t--queue_size : size fo queue-pair\n"
+ "\t\t--batches : number of batches of input\n");
printf("\n");
}
}
@@ -214,7 +222,8 @@ static struct option lgopts[] = {
{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
{ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
{ML_BURST_SIZE, 1, 0, 0}, {ML_QUEUE_PAIRS, 1, 0, 0}, {ML_QUEUE_SIZE, 1, 0, 0},
- {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
+ {ML_BATCHES, 1, 0, 0}, {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0},
+ {NULL, 0, 0, 0}};
static int
ml_opts_parse_long(int opt_idx, struct ml_options *opt)
@@ -226,7 +235,7 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
{ML_SOCKET_ID, ml_parse_socket_id}, {ML_MODELS, ml_parse_models},
{ML_FILELIST, ml_parse_filelist}, {ML_REPETITIONS, ml_parse_repetitions},
{ML_BURST_SIZE, ml_parse_burst_size}, {ML_QUEUE_PAIRS, ml_parse_queue_pairs},
- {ML_QUEUE_SIZE, ml_parse_queue_size},
+ {ML_QUEUE_SIZE, ml_parse_queue_size}, {ML_BATCHES, ml_parse_batches},
};
for (i = 0; i < RTE_DIM(parsermap); i++) {
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index c4018ee9d1..48fe064150 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -21,6 +21,7 @@
#define ML_BURST_SIZE ("burst_size")
#define ML_QUEUE_PAIRS ("queue_pairs")
#define ML_QUEUE_SIZE ("queue_size")
+#define ML_BATCHES ("batches")
#define ML_DEBUG ("debug")
#define ML_HELP ("help")
@@ -40,6 +41,7 @@ struct ml_options {
uint16_t burst_size;
uint16_t queue_pairs;
uint16_t queue_size;
+ uint16_t batches;
bool debug;
};
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
index 7637d0833e..3c8c3694e8 100644
--- a/app/test-mldev/test_inference_common.c
+++ b/app/test-mldev/test_inference_common.c
@@ -50,7 +50,7 @@ ml_enqueue_single(void *arg)
goto retry;
op->model_id = t->model[fid].id;
- op->nb_batches = t->model[fid].info.batch_size;
+ op->nb_batches = t->model[fid].nb_batches;
op->mempool = t->op_pool;
op->input.addr = req->input;
@@ -162,7 +162,7 @@ ml_enqueue_burst(void *arg)
for (i = 0; i < ops_count; i++) {
args->enq_ops[i]->model_id = t->model[fid].id;
- args->enq_ops[i]->nb_batches = t->model[fid].info.batch_size;
+ args->enq_ops[i]->nb_batches = t->model[fid].nb_batches;
args->enq_ops[i]->mempool = t->op_pool;
args->enq_ops[i]->input.addr = args->reqs[i]->input;
@@ -357,6 +357,11 @@ test_inference_opt_dump(struct ml_options *opt)
ml_dump("queue_pairs", "%u", opt->queue_pairs);
ml_dump("queue_size", "%u", opt->queue_size);
+ if (opt->batches == 0)
+ ml_dump("batches", "%u (default)", opt->batches);
+ else
+ ml_dump("batches", "%u", opt->batches);
+
ml_dump_begin("filelist");
for (i = 0; i < opt->nb_filelist; i++) {
ml_dump_list("model", i, opt->filelist[i].model);
@@ -525,8 +530,8 @@ ml_request_initialize(struct rte_mempool *mp, void *opaque, void *obj, unsigned
req->niters = 0;
/* quantize data */
- rte_ml_io_quantize(t->cmn.opt->dev_id, t->model[t->fid].id,
- t->model[t->fid].info.batch_size, t->model[t->fid].input, req->input);
+ rte_ml_io_quantize(t->cmn.opt->dev_id, t->model[t->fid].id, t->model[t->fid].nb_batches,
+ t->model[t->fid].input, req->input);
}
int
@@ -544,7 +549,7 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, uint16_t
int ret;
/* get input buffer size */
- ret = rte_ml_io_input_size_get(opt->dev_id, t->model[fid].id, t->model[fid].info.batch_size,
+ ret = rte_ml_io_input_size_get(opt->dev_id, t->model[fid].id, t->model[fid].nb_batches,
(uint64_t *)&t->model[fid].inp_qsize,
(uint64_t *)&t->model[fid].inp_dsize);
if (ret != 0) {
@@ -553,9 +558,9 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, uint16_t
}
/* get output buffer size */
- ret = rte_ml_io_output_size_get(
- opt->dev_id, t->model[fid].id, t->model[fid].info.batch_size,
- (uint64_t *)&t->model[fid].out_qsize, (uint64_t *)&t->model[fid].out_dsize);
+ ret = rte_ml_io_output_size_get(opt->dev_id, t->model[fid].id, t->model[fid].nb_batches,
+ (uint64_t *)&t->model[fid].out_qsize,
+ (uint64_t *)&t->model[fid].out_dsize);
if (ret != 0) {
ml_err("Failed to get input size, model : %s\n", opt->filelist[fid].model);
return ret;
@@ -700,7 +705,7 @@ ml_request_finish(struct rte_mempool *mp, void *opaque, void *obj, unsigned int
return;
t->nb_used++;
- rte_ml_io_dequantize(t->cmn.opt->dev_id, model->id, t->model[req->fid].info.batch_size,
+ rte_ml_io_dequantize(t->cmn.opt->dev_id, model->id, t->model[req->fid].nb_batches,
req->output, model->output);
}
diff --git a/app/test-mldev/test_model_common.c b/app/test-mldev/test_model_common.c
index b94d46154d..c28e452f29 100644
--- a/app/test-mldev/test_model_common.c
+++ b/app/test-mldev/test_model_common.c
@@ -71,6 +71,12 @@ ml_model_load(struct ml_test *test, struct ml_options *opt, struct ml_model *mod
return ret;
}
+ /* Update number of batches */
+ if (opt->batches == 0)
+ model->nb_batches = model->info.batch_size;
+ else
+ model->nb_batches = opt->batches;
+
model->state = MODEL_LOADED;
return 0;
diff --git a/app/test-mldev/test_model_common.h b/app/test-mldev/test_model_common.h
index 1c89ef83aa..4ee5e26b1e 100644
--- a/app/test-mldev/test_model_common.h
+++ b/app/test-mldev/test_model_common.h
@@ -30,6 +30,7 @@ struct ml_model {
uint8_t *output;
struct rte_mempool *io_pool;
+ uint32_t nb_batches;
};
int ml_model_load(struct ml_test *test, struct ml_options *opt, struct ml_model *model,
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v5 10/12] app/mldev: enable support for inference validation
2023-03-10 8:09 ` [PATCH v5 00/12] Implementation of mldev test application Srikanth Yalavarthi
` (8 preceding siblings ...)
2023-03-10 8:09 ` [PATCH v5 09/12] app/mldev: enable support for inference batches Srikanth Yalavarthi
@ 2023-03-10 8:09 ` Srikanth Yalavarthi
2023-03-10 8:09 ` [PATCH v5 11/12] app/mldev: enable reporting stats in mldev app Srikanth Yalavarthi
2023-03-10 8:09 ` [PATCH v5 12/12] app/mldev: add documentation for mldev test cases Srikanth Yalavarthi
11 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-10 8:09 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Enabled support to validate inference output with reference
output provided by the user. Validation would be successful
only when the inference outputs are within the 'tolerance'
specified through command line option "--tolerance".
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/meson.build | 2 +-
app/test-mldev/ml_options.c | 36 +++-
app/test-mldev/ml_options.h | 3 +
app/test-mldev/test_inference_common.c | 217 ++++++++++++++++++++++++-
app/test-mldev/test_inference_common.h | 1 +
app/test-mldev/test_model_common.h | 1 +
6 files changed, 249 insertions(+), 11 deletions(-)
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
index 41d22fb22c..15db534dc2 100644
--- a/app/test-mldev/meson.build
+++ b/app/test-mldev/meson.build
@@ -21,4 +21,4 @@ sources = files(
'test_inference_interleave.c',
)
-deps += ['mldev']
+deps += ['mldev', 'hash']
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 499bfde899..9dfa5ec0d8 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -4,6 +4,7 @@
#include <errno.h>
#include <getopt.h>
+#include <math.h>
#include <rte_memory.h>
#include <rte_mldev.h>
@@ -28,6 +29,7 @@ ml_options_default(struct ml_options *opt)
opt->queue_pairs = 1;
opt->queue_size = 1;
opt->batches = 0;
+ opt->tolerance = 0.0;
opt->debug = false;
}
@@ -133,6 +135,13 @@ ml_parse_filelist(struct ml_options *opt, const char *arg)
}
strlcpy(opt->filelist[opt->nb_filelist].output, token, PATH_MAX);
+ /* reference - optional */
+ token = strtok(NULL, delim);
+ if (token != NULL)
+ strlcpy(opt->filelist[opt->nb_filelist].reference, token, PATH_MAX);
+ else
+ memset(opt->filelist[opt->nb_filelist].reference, 0, PATH_MAX);
+
opt->nb_filelist++;
if (opt->nb_filelist == 0) {
@@ -177,6 +186,14 @@ ml_parse_batches(struct ml_options *opt, const char *arg)
return parser_read_uint16(&opt->batches, arg);
}
+static int
+ml_parse_tolerance(struct ml_options *opt, const char *arg)
+{
+ opt->tolerance = fabs(atof(arg));
+
+ return 0;
+}
+
static void
ml_dump_test_options(const char *testname)
{
@@ -193,12 +210,13 @@ ml_dump_test_options(const char *testname)
if ((strcmp(testname, "inference_ordered") == 0) ||
(strcmp(testname, "inference_interleave") == 0)) {
- printf("\t\t--filelist : comma separated list of model, input and output\n"
+ printf("\t\t--filelist : comma separated list of model, input, output and reference\n"
"\t\t--repetitions : number of inference repetitions\n"
"\t\t--burst_size : inference burst size\n"
"\t\t--queue_pairs : number of queue pairs to create\n"
"\t\t--queue_size : size fo queue-pair\n"
- "\t\t--batches : number of batches of input\n");
+ "\t\t--batches : number of batches of input\n"
+ "\t\t--tolerance : maximum tolerance (%%) for output validation\n");
printf("\n");
}
}
@@ -218,12 +236,13 @@ print_usage(char *program)
ml_test_dump_names(ml_dump_test_options);
}
-static struct option lgopts[] = {
- {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
- {ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
- {ML_BURST_SIZE, 1, 0, 0}, {ML_QUEUE_PAIRS, 1, 0, 0}, {ML_QUEUE_SIZE, 1, 0, 0},
- {ML_BATCHES, 1, 0, 0}, {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0},
- {NULL, 0, 0, 0}};
+static struct option lgopts[] = {{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0},
+ {ML_SOCKET_ID, 1, 0, 0}, {ML_MODELS, 1, 0, 0},
+ {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
+ {ML_BURST_SIZE, 1, 0, 0}, {ML_QUEUE_PAIRS, 1, 0, 0},
+ {ML_QUEUE_SIZE, 1, 0, 0}, {ML_BATCHES, 1, 0, 0},
+ {ML_TOLERANCE, 1, 0, 0}, {ML_DEBUG, 0, 0, 0},
+ {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
static int
ml_opts_parse_long(int opt_idx, struct ml_options *opt)
@@ -236,6 +255,7 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
{ML_FILELIST, ml_parse_filelist}, {ML_REPETITIONS, ml_parse_repetitions},
{ML_BURST_SIZE, ml_parse_burst_size}, {ML_QUEUE_PAIRS, ml_parse_queue_pairs},
{ML_QUEUE_SIZE, ml_parse_queue_size}, {ML_BATCHES, ml_parse_batches},
+ {ML_TOLERANCE, ml_parse_tolerance},
};
for (i = 0; i < RTE_DIM(parsermap); i++) {
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index 48fe064150..7f3db29656 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -22,6 +22,7 @@
#define ML_QUEUE_PAIRS ("queue_pairs")
#define ML_QUEUE_SIZE ("queue_size")
#define ML_BATCHES ("batches")
+#define ML_TOLERANCE ("tolerance")
#define ML_DEBUG ("debug")
#define ML_HELP ("help")
@@ -29,6 +30,7 @@ struct ml_filelist {
char model[PATH_MAX];
char input[PATH_MAX];
char output[PATH_MAX];
+ char reference[PATH_MAX];
};
struct ml_options {
@@ -42,6 +44,7 @@ struct ml_options {
uint16_t queue_pairs;
uint16_t queue_size;
uint16_t batches;
+ float tolerance;
bool debug;
};
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
index 3c8c3694e8..87e9efbb37 100644
--- a/app/test-mldev/test_inference_common.c
+++ b/app/test-mldev/test_inference_common.c
@@ -6,6 +6,7 @@
#include <unistd.h>
#include <rte_common.h>
+#include <rte_hash_crc.h>
#include <rte_launch.h>
#include <rte_lcore.h>
#include <rte_malloc.h>
@@ -15,6 +16,27 @@
#include "ml_common.h"
#include "test_inference_common.h"
+#define ML_TEST_READ_TYPE(buffer, type) (*((type *)buffer))
+
+#define ML_TEST_CHECK_OUTPUT(output, reference, tolerance) \
+ (((float)output - (float)reference) <= (((float)reference * tolerance) / 100.0))
+
+#define ML_OPEN_WRITE_GET_ERR(name, buffer, size, err) \
+ do { \
+ FILE *fp = fopen(name, "w+"); \
+ if (fp == NULL) { \
+ ml_err("Unable to create file: %s, error: %s", name, strerror(errno)); \
+ err = true; \
+ } else { \
+ if (fwrite(buffer, 1, size, fp) != size) { \
+ ml_err("Error writing output, file: %s, error: %s", name, \
+ strerror(errno)); \
+ err = true; \
+ } \
+ fclose(fp); \
+ } \
+ } while (0)
+
/* Enqueue inference requests with burst size equal to 1 */
static int
ml_enqueue_single(void *arg)
@@ -356,6 +378,7 @@ test_inference_opt_dump(struct ml_options *opt)
ml_dump("burst_size", "%u", opt->burst_size);
ml_dump("queue_pairs", "%u", opt->queue_pairs);
ml_dump("queue_size", "%u", opt->queue_size);
+ ml_dump("tolerance", "%-7.3f", opt->tolerance);
if (opt->batches == 0)
ml_dump("batches", "%u (default)", opt->batches);
@@ -367,6 +390,8 @@ test_inference_opt_dump(struct ml_options *opt)
ml_dump_list("model", i, opt->filelist[i].model);
ml_dump_list("input", i, opt->filelist[i].input);
ml_dump_list("output", i, opt->filelist[i].output);
+ if (strcmp(opt->filelist[i].reference, "\0") != 0)
+ ml_dump_list("reference", i, opt->filelist[i].reference);
}
ml_dump_end;
}
@@ -391,6 +416,7 @@ test_inference_setup(struct ml_test *test, struct ml_options *opt)
t = ml_test_priv(test);
t->nb_used = 0;
+ t->nb_valid = 0;
t->cmn.result = ML_TEST_FAILED;
t->cmn.opt = opt;
@@ -568,6 +594,9 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, uint16_t
/* allocate buffer for user data */
mz_size = t->model[fid].inp_dsize + t->model[fid].out_dsize;
+ if (strcmp(opt->filelist[fid].reference, "\0") != 0)
+ mz_size += t->model[fid].out_dsize;
+
sprintf(mz_name, "ml_user_data_%d", fid);
mz = rte_memzone_reserve(mz_name, mz_size, opt->socket_id, 0);
if (mz == NULL) {
@@ -578,6 +607,11 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, uint16_t
t->model[fid].input = mz->addr;
t->model[fid].output = RTE_PTR_ADD(t->model[fid].input, t->model[fid].inp_dsize);
+ if (strcmp(opt->filelist[fid].reference, "\0") != 0)
+ t->model[fid].reference =
+ RTE_PTR_ADD(t->model[fid].output, t->model[fid].out_dsize);
+ else
+ t->model[fid].reference = NULL;
/* load input file */
fp = fopen(opt->filelist[fid].input, "r");
@@ -606,6 +640,27 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, uint16_t
}
fclose(fp);
+ /* load reference file */
+ if (t->model[fid].reference != NULL) {
+ fp = fopen(opt->filelist[fid].reference, "r");
+ if (fp == NULL) {
+ ml_err("Failed to open reference file : %s\n",
+ opt->filelist[fid].reference);
+ ret = -errno;
+ goto error;
+ }
+
+ if (fread(t->model[fid].reference, 1, t->model[fid].out_dsize, fp) !=
+ t->model[fid].out_dsize) {
+ ml_err("Failed to read reference file : %s\n",
+ opt->filelist[fid].reference);
+ ret = -errno;
+ fclose(fp);
+ goto error;
+ }
+ fclose(fp);
+ }
+
/* create mempool for quantized input and output buffers. ml_request_initialize is
* used as a callback for object creation.
*/
@@ -690,6 +745,121 @@ ml_inference_mem_destroy(struct ml_test *test, struct ml_options *opt)
rte_mempool_free(t->op_pool);
}
+static bool
+ml_inference_validation(struct ml_test *test, struct ml_request *req)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)test);
+ struct ml_model *model;
+ uint32_t nb_elements;
+ uint8_t *reference;
+ uint8_t *output;
+ bool match;
+ uint32_t i;
+ uint32_t j;
+
+ model = &t->model[req->fid];
+
+ /* compare crc when tolerance is 0 */
+ if (t->cmn.opt->tolerance == 0.0) {
+ match = (rte_hash_crc(model->output, model->out_dsize, 0) ==
+ rte_hash_crc(model->reference, model->out_dsize, 0));
+ } else {
+ output = model->output;
+ reference = model->reference;
+
+ i = 0;
+next_output:
+ nb_elements =
+ model->info.output_info[i].shape.w * model->info.output_info[i].shape.x *
+ model->info.output_info[i].shape.y * model->info.output_info[i].shape.z;
+ j = 0;
+next_element:
+ match = false;
+ switch (model->info.output_info[i].dtype) {
+ case RTE_ML_IO_TYPE_INT8:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, int8_t),
+ ML_TEST_READ_TYPE(reference, int8_t),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(int8_t);
+ reference += sizeof(int8_t);
+ break;
+ case RTE_ML_IO_TYPE_UINT8:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, uint8_t),
+ ML_TEST_READ_TYPE(reference, uint8_t),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(float);
+ reference += sizeof(float);
+ break;
+ case RTE_ML_IO_TYPE_INT16:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, int16_t),
+ ML_TEST_READ_TYPE(reference, int16_t),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(int16_t);
+ reference += sizeof(int16_t);
+ break;
+ case RTE_ML_IO_TYPE_UINT16:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, uint16_t),
+ ML_TEST_READ_TYPE(reference, uint16_t),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(uint16_t);
+ reference += sizeof(uint16_t);
+ break;
+ case RTE_ML_IO_TYPE_INT32:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, int32_t),
+ ML_TEST_READ_TYPE(reference, int32_t),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(int32_t);
+ reference += sizeof(int32_t);
+ break;
+ case RTE_ML_IO_TYPE_UINT32:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, uint32_t),
+ ML_TEST_READ_TYPE(reference, uint32_t),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(uint32_t);
+ reference += sizeof(uint32_t);
+ break;
+ case RTE_ML_IO_TYPE_FP32:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, float),
+ ML_TEST_READ_TYPE(reference, float),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(float);
+ reference += sizeof(float);
+ break;
+ default: /* other types, fp8, fp16, bfloat16 */
+ match = true;
+ }
+
+ if (!match)
+ goto done;
+ j++;
+ if (j < nb_elements)
+ goto next_element;
+
+ i++;
+ if (i < model->info.nb_outputs)
+ goto next_output;
+ }
+done:
+ if (match)
+ t->nb_valid++;
+
+ return match;
+}
+
/* Callback for mempool object iteration. This call would dequantize output data. */
static void
ml_request_finish(struct rte_mempool *mp, void *opaque, void *obj, unsigned int obj_idx)
@@ -697,9 +867,10 @@ ml_request_finish(struct rte_mempool *mp, void *opaque, void *obj, unsigned int
struct test_inference *t = ml_test_priv((struct ml_test *)opaque);
struct ml_request *req = (struct ml_request *)obj;
struct ml_model *model = &t->model[req->fid];
+ char str[PATH_MAX];
+ bool error = false;
RTE_SET_USED(mp);
- RTE_SET_USED(obj_idx);
if (req->niters == 0)
return;
@@ -707,6 +878,48 @@ ml_request_finish(struct rte_mempool *mp, void *opaque, void *obj, unsigned int
t->nb_used++;
rte_ml_io_dequantize(t->cmn.opt->dev_id, model->id, t->model[req->fid].nb_batches,
req->output, model->output);
+
+ if (model->reference == NULL) {
+ t->nb_valid++;
+ goto dump_output_pass;
+ }
+
+ if (!ml_inference_validation(opaque, req))
+ goto dump_output_fail;
+ else
+ goto dump_output_pass;
+
+dump_output_pass:
+ if (obj_idx == 0) {
+ /* write quantized output */
+ snprintf(str, PATH_MAX, "%s.q", t->cmn.opt->filelist[req->fid].output);
+ ML_OPEN_WRITE_GET_ERR(str, req->output, model->out_qsize, error);
+ if (error)
+ return;
+
+ /* write dequantized output */
+ snprintf(str, PATH_MAX, "%s", t->cmn.opt->filelist[req->fid].output);
+ ML_OPEN_WRITE_GET_ERR(str, model->output, model->out_dsize, error);
+ if (error)
+ return;
+ }
+
+ return;
+
+dump_output_fail:
+ if (t->cmn.opt->debug) {
+ /* dump quantized output buffer */
+ snprintf(str, PATH_MAX, "%s.q.%d", t->cmn.opt->filelist[req->fid].output, obj_idx);
+ ML_OPEN_WRITE_GET_ERR(str, req->output, model->out_qsize, error);
+ if (error)
+ return;
+
+ /* dump dequantized output buffer */
+ snprintf(str, PATH_MAX, "%s.%d", t->cmn.opt->filelist[req->fid].output, obj_idx);
+ ML_OPEN_WRITE_GET_ERR(str, model->output, model->out_dsize, error);
+ if (error)
+ return;
+ }
}
int
@@ -718,7 +931,7 @@ ml_inference_result(struct ml_test *test, struct ml_options *opt, uint16_t fid)
rte_mempool_obj_iter(t->model[fid].io_pool, ml_request_finish, test);
- if (t->nb_used > 0)
+ if (t->nb_used == t->nb_valid)
t->cmn.result = ML_TEST_SUCCESS;
else
t->cmn.result = ML_TEST_FAILED;
diff --git a/app/test-mldev/test_inference_common.h b/app/test-mldev/test_inference_common.h
index b85fb464b1..3f7373adf5 100644
--- a/app/test-mldev/test_inference_common.h
+++ b/app/test-mldev/test_inference_common.h
@@ -38,6 +38,7 @@ struct test_inference {
struct rte_mempool *op_pool;
uint64_t nb_used;
+ uint64_t nb_valid;
uint16_t fid;
int (*enqueue)(void *arg);
diff --git a/app/test-mldev/test_model_common.h b/app/test-mldev/test_model_common.h
index 4ee5e26b1e..8e4d8ceee1 100644
--- a/app/test-mldev/test_model_common.h
+++ b/app/test-mldev/test_model_common.h
@@ -28,6 +28,7 @@ struct ml_model {
uint8_t *input;
uint8_t *output;
+ uint8_t *reference;
struct rte_mempool *io_pool;
uint32_t nb_batches;
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v5 11/12] app/mldev: enable reporting stats in mldev app
2023-03-10 8:09 ` [PATCH v5 00/12] Implementation of mldev test application Srikanth Yalavarthi
` (9 preceding siblings ...)
2023-03-10 8:09 ` [PATCH v5 10/12] app/mldev: enable support for inference validation Srikanth Yalavarthi
@ 2023-03-10 8:09 ` Srikanth Yalavarthi
2023-03-10 8:09 ` [PATCH v5 12/12] app/mldev: add documentation for mldev test cases Srikanth Yalavarthi
11 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-10 8:09 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Enable reporting driver xstats and inference end-to-end
latency and throughput in mldev inference tests. Reporting
of stats can be enabled using "--stats" option.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/ml_options.c | 22 ++--
app/test-mldev/ml_options.h | 2 +
app/test-mldev/test_inference_common.c | 140 +++++++++++++++++++++
app/test-mldev/test_inference_common.h | 8 ++
app/test-mldev/test_inference_interleave.c | 4 +
app/test-mldev/test_inference_ordered.c | 1 +
6 files changed, 169 insertions(+), 8 deletions(-)
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 9dfa5ec0d8..769c3bffd1 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -30,6 +30,7 @@ ml_options_default(struct ml_options *opt)
opt->queue_size = 1;
opt->batches = 0;
opt->tolerance = 0.0;
+ opt->stats = false;
opt->debug = false;
}
@@ -216,7 +217,8 @@ ml_dump_test_options(const char *testname)
"\t\t--queue_pairs : number of queue pairs to create\n"
"\t\t--queue_size : size fo queue-pair\n"
"\t\t--batches : number of batches of input\n"
- "\t\t--tolerance : maximum tolerance (%%) for output validation\n");
+ "\t\t--tolerance : maximum tolerance (%%) for output validation\n"
+ "\t\t--stats : enable reporting performance statistics\n");
printf("\n");
}
}
@@ -236,13 +238,12 @@ print_usage(char *program)
ml_test_dump_names(ml_dump_test_options);
}
-static struct option lgopts[] = {{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0},
- {ML_SOCKET_ID, 1, 0, 0}, {ML_MODELS, 1, 0, 0},
- {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
- {ML_BURST_SIZE, 1, 0, 0}, {ML_QUEUE_PAIRS, 1, 0, 0},
- {ML_QUEUE_SIZE, 1, 0, 0}, {ML_BATCHES, 1, 0, 0},
- {ML_TOLERANCE, 1, 0, 0}, {ML_DEBUG, 0, 0, 0},
- {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
+static struct option lgopts[] = {
+ {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
+ {ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
+ {ML_BURST_SIZE, 1, 0, 0}, {ML_QUEUE_PAIRS, 1, 0, 0}, {ML_QUEUE_SIZE, 1, 0, 0},
+ {ML_BATCHES, 1, 0, 0}, {ML_TOLERANCE, 1, 0, 0}, {ML_STATS, 0, 0, 0},
+ {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
static int
ml_opts_parse_long(int opt_idx, struct ml_options *opt)
@@ -277,6 +278,11 @@ ml_options_parse(struct ml_options *opt, int argc, char **argv)
while ((opts = getopt_long(argc, argv, "", lgopts, &opt_idx)) != EOF) {
switch (opts) {
case 0: /* parse long options */
+ if (!strcmp(lgopts[opt_idx].name, "stats")) {
+ opt->stats = true;
+ break;
+ }
+
if (!strcmp(lgopts[opt_idx].name, "debug")) {
opt->debug = true;
break;
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index 7f3db29656..beb0fe69c6 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -23,6 +23,7 @@
#define ML_QUEUE_SIZE ("queue_size")
#define ML_BATCHES ("batches")
#define ML_TOLERANCE ("tolerance")
+#define ML_STATS ("stats")
#define ML_DEBUG ("debug")
#define ML_HELP ("help")
@@ -45,6 +46,7 @@ struct ml_options {
uint16_t queue_size;
uint16_t batches;
float tolerance;
+ bool stats;
bool debug;
};
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
index 87e9efbb37..46bd20f1a5 100644
--- a/app/test-mldev/test_inference_common.c
+++ b/app/test-mldev/test_inference_common.c
@@ -6,6 +6,7 @@
#include <unistd.h>
#include <rte_common.h>
+#include <rte_cycles.h>
#include <rte_hash_crc.h>
#include <rte_launch.h>
#include <rte_lcore.h>
@@ -37,6 +38,17 @@
} \
} while (0)
+static void
+print_line(uint16_t len)
+{
+ uint16_t i;
+
+ for (i = 0; i < len; i++)
+ printf("-");
+
+ printf("\n");
+}
+
/* Enqueue inference requests with burst size equal to 1 */
static int
ml_enqueue_single(void *arg)
@@ -46,6 +58,7 @@ ml_enqueue_single(void *arg)
struct rte_ml_op *op = NULL;
struct ml_core_args *args;
uint64_t model_enq = 0;
+ uint64_t start_cycle;
uint32_t burst_enq;
uint32_t lcore_id;
uint16_t fid;
@@ -53,6 +66,7 @@ ml_enqueue_single(void *arg)
lcore_id = rte_lcore_id();
args = &t->args[lcore_id];
+ args->start_cycles = 0;
model_enq = 0;
if (args->nb_reqs == 0)
@@ -88,10 +102,12 @@ ml_enqueue_single(void *arg)
req->fid = fid;
enqueue_req:
+ start_cycle = rte_get_tsc_cycles();
burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, args->qp_id, &op, 1);
if (burst_enq == 0)
goto enqueue_req;
+ args->start_cycles += start_cycle;
fid++;
if (likely(fid <= args->end_fid))
goto next_model;
@@ -115,10 +131,12 @@ ml_dequeue_single(void *arg)
uint64_t total_deq = 0;
uint8_t nb_filelist;
uint32_t burst_deq;
+ uint64_t end_cycle;
uint32_t lcore_id;
lcore_id = rte_lcore_id();
args = &t->args[lcore_id];
+ args->end_cycles = 0;
nb_filelist = args->end_fid - args->start_fid + 1;
if (args->nb_reqs == 0)
@@ -126,9 +144,11 @@ ml_dequeue_single(void *arg)
dequeue_req:
burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, args->qp_id, &op, 1);
+ end_cycle = rte_get_tsc_cycles();
if (likely(burst_deq == 1)) {
total_deq += burst_deq;
+ args->end_cycles += end_cycle;
if (unlikely(op->status == RTE_ML_OP_STATUS_ERROR)) {
rte_ml_op_error_get(t->cmn.opt->dev_id, op, &error);
ml_err("error_code = 0x%" PRIx64 ", error_message = %s\n", error.errcode,
@@ -151,6 +171,7 @@ ml_enqueue_burst(void *arg)
{
struct test_inference *t = ml_test_priv((struct ml_test *)arg);
struct ml_core_args *args;
+ uint64_t start_cycle;
uint16_t ops_count;
uint64_t model_enq;
uint16_t burst_enq;
@@ -163,6 +184,7 @@ ml_enqueue_burst(void *arg)
lcore_id = rte_lcore_id();
args = &t->args[lcore_id];
+ args->start_cycles = 0;
model_enq = 0;
if (args->nb_reqs == 0)
@@ -204,8 +226,10 @@ ml_enqueue_burst(void *arg)
pending = ops_count;
enqueue_reqs:
+ start_cycle = rte_get_tsc_cycles();
burst_enq =
rte_ml_enqueue_burst(t->cmn.opt->dev_id, args->qp_id, &args->enq_ops[idx], pending);
+ args->start_cycles += burst_enq * start_cycle;
pending = pending - burst_enq;
if (pending > 0) {
@@ -235,11 +259,13 @@ ml_dequeue_burst(void *arg)
uint64_t total_deq = 0;
uint16_t burst_deq = 0;
uint8_t nb_filelist;
+ uint64_t end_cycle;
uint32_t lcore_id;
uint32_t i;
lcore_id = rte_lcore_id();
args = &t->args[lcore_id];
+ args->end_cycles = 0;
nb_filelist = args->end_fid - args->start_fid + 1;
if (args->nb_reqs == 0)
@@ -248,9 +274,11 @@ ml_dequeue_burst(void *arg)
dequeue_burst:
burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, args->qp_id, args->deq_ops,
t->cmn.opt->burst_size);
+ end_cycle = rte_get_tsc_cycles();
if (likely(burst_deq > 0)) {
total_deq += burst_deq;
+ args->end_cycles += burst_deq * end_cycle;
for (i = 0; i < burst_deq; i++) {
if (unlikely(args->deq_ops[i]->status == RTE_ML_OP_STATUS_ERROR)) {
@@ -379,6 +407,7 @@ test_inference_opt_dump(struct ml_options *opt)
ml_dump("queue_pairs", "%u", opt->queue_pairs);
ml_dump("queue_size", "%u", opt->queue_size);
ml_dump("tolerance", "%-7.3f", opt->tolerance);
+ ml_dump("stats", "%s", (opt->stats ? "true" : "false"));
if (opt->batches == 0)
ml_dump("batches", "%u (default)", opt->batches);
@@ -451,6 +480,11 @@ test_inference_setup(struct ml_test *test, struct ml_options *opt)
RTE_CACHE_LINE_SIZE, opt->socket_id);
}
+ for (i = 0; i < RTE_MAX_LCORE; i++) {
+ t->args[i].start_cycles = 0;
+ t->args[i].end_cycles = 0;
+ }
+
return 0;
error:
@@ -980,3 +1014,109 @@ ml_inference_launch_cores(struct ml_test *test, struct ml_options *opt, uint16_t
return 0;
}
+
+int
+ml_inference_stats_get(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t = ml_test_priv(test);
+ uint64_t total_cycles = 0;
+ uint32_t nb_filelist;
+ uint64_t throughput;
+ uint64_t avg_e2e;
+ uint32_t qp_id;
+ uint64_t freq;
+ int ret;
+ int i;
+
+ if (!opt->stats)
+ return 0;
+
+ /* get xstats size */
+ t->xstats_size = rte_ml_dev_xstats_names_get(opt->dev_id, NULL, 0);
+ if (t->xstats_size >= 0) {
+ /* allocate for xstats_map and values */
+ t->xstats_map = rte_malloc(
+ "ml_xstats_map", t->xstats_size * sizeof(struct rte_ml_dev_xstats_map), 0);
+ if (t->xstats_map == NULL) {
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ t->xstats_values =
+ rte_malloc("ml_xstats_values", t->xstats_size * sizeof(uint64_t), 0);
+ if (t->xstats_values == NULL) {
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ ret = rte_ml_dev_xstats_names_get(opt->dev_id, t->xstats_map, t->xstats_size);
+ if (ret != t->xstats_size) {
+ printf("Unable to get xstats names, ret = %d\n", ret);
+ ret = -1;
+ goto error;
+ }
+
+ for (i = 0; i < t->xstats_size; i++)
+ rte_ml_dev_xstats_get(opt->dev_id, &t->xstats_map[i].id,
+ &t->xstats_values[i], 1);
+ }
+
+ /* print xstats*/
+ printf("\n");
+ print_line(80);
+ printf(" ML Device Extended Statistics\n");
+ print_line(80);
+ for (i = 0; i < t->xstats_size; i++)
+ printf(" %-64s = %" PRIu64 "\n", t->xstats_map[i].name, t->xstats_values[i]);
+ print_line(80);
+
+ /* release buffers */
+ if (t->xstats_map)
+ rte_free(t->xstats_map);
+
+ if (t->xstats_values)
+ rte_free(t->xstats_values);
+
+ /* print end-to-end stats */
+ freq = rte_get_tsc_hz();
+ for (qp_id = 0; qp_id < RTE_MAX_LCORE; qp_id++)
+ total_cycles += t->args[qp_id].end_cycles - t->args[qp_id].start_cycles;
+ avg_e2e = total_cycles / opt->repetitions;
+
+ if (freq == 0) {
+ avg_e2e = total_cycles / opt->repetitions;
+ printf(" %-64s = %" PRIu64 "\n", "Average End-to-End Latency (cycles)", avg_e2e);
+ } else {
+ avg_e2e = (total_cycles * NS_PER_S) / (opt->repetitions * freq);
+ printf(" %-64s = %" PRIu64 "\n", "Average End-to-End Latency (ns)", avg_e2e);
+ }
+
+ /* print inference throughput */
+ if (strcmp(opt->test_name, "inference_ordered") == 0)
+ nb_filelist = 1;
+ else
+ nb_filelist = opt->nb_filelist;
+
+ if (freq == 0) {
+ throughput = (nb_filelist * t->cmn.opt->repetitions * 1000000) / total_cycles;
+ printf(" %-64s = %" PRIu64 "\n", "Average Throughput (inferences / million cycles)",
+ throughput);
+ } else {
+ throughput = (nb_filelist * t->cmn.opt->repetitions * freq) / total_cycles;
+ printf(" %-64s = %" PRIu64 "\n", "Average Throughput (inferences / second)",
+ throughput);
+ }
+
+ print_line(80);
+
+ return 0;
+
+error:
+ if (t->xstats_map)
+ rte_free(t->xstats_map);
+
+ if (t->xstats_values)
+ rte_free(t->xstats_values);
+
+ return ret;
+}
diff --git a/app/test-mldev/test_inference_common.h b/app/test-mldev/test_inference_common.h
index 3f7373adf5..6f67e94ef8 100644
--- a/app/test-mldev/test_inference_common.h
+++ b/app/test-mldev/test_inference_common.h
@@ -27,6 +27,9 @@ struct ml_core_args {
struct rte_ml_op **enq_ops;
struct rte_ml_op **deq_ops;
struct ml_request **reqs;
+
+ uint64_t start_cycles;
+ uint64_t end_cycles;
};
struct test_inference {
@@ -45,6 +48,10 @@ struct test_inference {
int (*dequeue)(void *arg);
struct ml_core_args args[RTE_MAX_LCORE];
+
+ struct rte_ml_dev_xstats_map *xstats_map;
+ uint64_t *xstats_values;
+ int xstats_size;
} __rte_cache_aligned;
bool test_inference_cap_check(struct ml_options *opt);
@@ -62,5 +69,6 @@ void ml_inference_mem_destroy(struct ml_test *test, struct ml_options *opt);
int ml_inference_result(struct ml_test *test, struct ml_options *opt, uint16_t fid);
int ml_inference_launch_cores(struct ml_test *test, struct ml_options *opt, uint16_t start_fid,
uint16_t end_fid);
+int ml_inference_stats_get(struct ml_test *test, struct ml_options *opt);
#endif /* _ML_TEST_INFERENCE_COMMON_ */
diff --git a/app/test-mldev/test_inference_interleave.c b/app/test-mldev/test_inference_interleave.c
index 9cf4cfa197..bd2c286737 100644
--- a/app/test-mldev/test_inference_interleave.c
+++ b/app/test-mldev/test_inference_interleave.c
@@ -56,7 +56,11 @@ test_inference_interleave_driver(struct ml_test *test, struct ml_options *opt)
goto error;
ml_inference_iomem_destroy(test, opt, fid);
+ }
+
+ ml_inference_stats_get(test, opt);
+ for (fid = 0; fid < opt->nb_filelist; fid++) {
ret = ml_model_stop(test, opt, &t->model[fid], fid);
if (ret != 0)
goto error;
diff --git a/app/test-mldev/test_inference_ordered.c b/app/test-mldev/test_inference_ordered.c
index 1cd91dc3d3..8992358936 100644
--- a/app/test-mldev/test_inference_ordered.c
+++ b/app/test-mldev/test_inference_ordered.c
@@ -54,6 +54,7 @@ test_inference_ordered_driver(struct ml_test *test, struct ml_options *opt)
goto error;
ml_inference_iomem_destroy(test, opt, fid);
+ ml_inference_stats_get(test, opt);
/* stop model */
ret = ml_model_stop(test, opt, &t->model[fid], fid);
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v5 12/12] app/mldev: add documentation for mldev test cases
2023-03-10 8:09 ` [PATCH v5 00/12] Implementation of mldev test application Srikanth Yalavarthi
` (10 preceding siblings ...)
2023-03-10 8:09 ` [PATCH v5 11/12] app/mldev: enable reporting stats in mldev app Srikanth Yalavarthi
@ 2023-03-10 8:09 ` Srikanth Yalavarthi
11 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-10 8:09 UTC (permalink / raw)
To: Thomas Monjalon, Srikanth Yalavarthi
Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Added documentation specific to mldev test cases. Added details
about all test cases and option supported by individual tests.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
MAINTAINERS | 1 +
.../tools/img/mldev_inference_interleave.svg | 669 ++++++++++++++++++
.../tools/img/mldev_inference_ordered.svg | 528 ++++++++++++++
.../tools/img/mldev_model_ops_subtest_a.svg | 420 +++++++++++
.../tools/img/mldev_model_ops_subtest_b.svg | 423 +++++++++++
.../tools/img/mldev_model_ops_subtest_c.svg | 366 ++++++++++
.../tools/img/mldev_model_ops_subtest_d.svg | 424 +++++++++++
doc/guides/tools/index.rst | 1 +
doc/guides/tools/testmldev.rst | 441 ++++++++++++
9 files changed, 3273 insertions(+)
create mode 100644 doc/guides/tools/img/mldev_inference_interleave.svg
create mode 100644 doc/guides/tools/img/mldev_inference_ordered.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_a.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_b.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_c.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_d.svg
create mode 100644 doc/guides/tools/testmldev.rst
diff --git a/MAINTAINERS b/MAINTAINERS
index 1914c4d614..320842e13f 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -481,6 +481,7 @@ M: Srikanth Yalavarthi <syalavarthi@marvell.com>
F: lib/mldev/
F: doc/guides/prog_guide/mldev.rst
F: app/test-mldev
+F: doc/guides/tools/testmldev.rst
DMA device API - EXPERIMENTAL
M: Chengwen Feng <fengchengwen@huawei.com>
diff --git a/doc/guides/tools/img/mldev_inference_interleave.svg b/doc/guides/tools/img/mldev_inference_interleave.svg
new file mode 100644
index 0000000000..3a741ea627
--- /dev/null
+++ b/doc/guides/tools/img/mldev_inference_interleave.svg
@@ -0,0 +1,669 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- SPDX-License-Identifier: BSD-3-Clause -->
+<!-- Copyright (c) 2022 Marvell. -->
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="224mm"
+ height="150mm"
+ viewBox="0 0 224 150"
+ version="1.1"
+ id="svg5369"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="inference_interleave.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns:xlink="http://www.w3.org/1999/xlink"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg">
+ <sodipodi:namedview
+ id="namedview5371"
+ pagecolor="#ffffff"
+ bordercolor="#000000"
+ borderopacity="0.25"
+ inkscape:showpageshadow="2"
+ inkscape:pageopacity="0.0"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1"
+ inkscape:document-units="mm"
+ showgrid="false"
+ inkscape:zoom="0.74564394"
+ inkscape:cx="415.07747"
+ inkscape:cy="348.6919"
+ inkscape:window-width="1920"
+ inkscape:window-height="1017"
+ inkscape:window-x="1912"
+ inkscape:window-y="-8"
+ inkscape:window-maximized="1"
+ inkscape:current-layer="layer1" />
+ <defs
+ id="defs5366">
+ <linearGradient
+ inkscape:collect="always"
+ id="linearGradient19189">
+ <stop
+ style="stop-color:#ffeeaa;stop-opacity:1;"
+ offset="0"
+ id="stop19185" />
+ <stop
+ style="stop-color:#ffeeaa;stop-opacity:0;"
+ offset="1"
+ id="stop19187" />
+ </linearGradient>
+ <marker
+ style="overflow:visible"
+ id="TriangleStart"
+ refX="4"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="TriangleStart"
+ markerWidth="5.3244081"
+ markerHeight="6.155385"
+ viewBox="0 0 5.3244081 6.1553851"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.5)"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:context-stroke;stroke-width:1pt"
+ d="M 5.77,0 -2.88,5 V -5 Z"
+ id="path135" />
+ </marker>
+ <marker
+ style="overflow:visible"
+ id="RoundedArrow"
+ refX="5"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="RoundedArrow"
+ markerWidth="6.1347523"
+ markerHeight="5.9304948"
+ viewBox="0 0 6.1347524 5.9304951"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.7)"
+ d="m -0.21114562,-4.1055728 6.42229122,3.21114561 a 1,1 90 0 1 0,1.78885438 L -0.21114562,4.1055728 A 1.236068,1.236068 31.717474 0 1 -2,3 v -6 a 1.236068,1.236068 148.28253 0 1 1.78885438,-1.1055728 z"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:none"
+ id="path1367" />
+ </marker>
+ <linearGradient
+ inkscape:collect="always"
+ xlink:href="#linearGradient19189"
+ id="linearGradient19191"
+ x1="12.169352"
+ y1="105"
+ x2="284.83066"
+ y2="105"
+ gradientUnits="userSpaceOnUse"
+ gradientTransform="matrix(0.79055445,0,0,0.74078976,29.505892,28.991272)" />
+ </defs>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1">
+ <g
+ id="g1477"
+ transform="translate(-34.903236,-31.774189)">
+ <rect
+ style="fill:url(#linearGradient19191);fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.396267;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect16635"
+ width="215.18147"
+ height="139.60078"
+ x="39.3125"
+ y="36.973797"
+ ry="2.2354064" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-169.3954"
+ y="214.97237"
+ id="text5181"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan5179"
+ style="font-size:5.64444px;stroke-width:0.75"
+ x="-169.3954"
+ y="214.97237">test: inference_interleave</tspan></text>
+ <path
+ style="display:inline;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 138.05133,87.886263 17.45982,-10e-7"
+ id="path1912"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1724-0"
+ inkscape:connection-end="#rect1679" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 191.51115,87.886262 17.45981,10e-7"
+ id="path1914"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1679"
+ inkscape:connection-end="#rect1724" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 138.05133,109.78102 17.45982,-1e-5"
+ id="path1916"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1724-4-8"
+ inkscape:connection-end="#rect1679-4" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 191.51115,109.78101 17.45981,1e-5"
+ id="path1918"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1679-4"
+ inkscape:connection-end="#rect1724-4" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 138.05133,131.67576 17.45982,-1e-5"
+ id="path1920"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1724-6-7"
+ inkscape:connection-end="#rect1679-8" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 191.51115,131.67575 17.45981,1e-5"
+ id="path1922"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1679-8"
+ inkscape:connection-end="#rect1724-6" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.75;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;marker-start:url(#RoundedArrow);marker-end:url(#RoundedArrow)"
+ d="m 173.51116,60.08164 0,12.907336"
+ id="path1933"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1811"
+ inkscape:connection-end="#rect1924" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.368668;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.737336, 0.737336;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924"
+ width="46.97015"
+ height="73.58287"
+ x="150.02565"
+ y="72.988976"
+ ry="2.4685853" />
+ <text
+ xml:space="preserve"
+ style="font-size:6.35px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-84.44075"
+ y="173.5065"
+ id="text4097"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4095"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-84.44075"
+ y="173.5065">Queue</tspan><tspan
+ sodipodi:role="line"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-92.37825"
+ y="173.5065"
+ id="tspan4099">Pair 0</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:6.35px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-128.2318"
+ y="173.5065"
+ id="text4097-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4095-6"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-128.2318"
+ y="173.5065">Queue</tspan><tspan
+ sodipodi:role="line"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-136.1693"
+ y="173.5065"
+ id="tspan4099-4">Pair 2</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1679"
+ width="36"
+ height="18"
+ x="155.51115"
+ y="78.886261"
+ ry="3" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1679-8"
+ width="36"
+ height="18"
+ x="155.51115"
+ y="122.67575"
+ ry="3" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1679-4"
+ width="36"
+ height="18"
+ x="155.51115"
+ y="100.78101"
+ ry="3" />
+ <text
+ xml:space="preserve"
+ style="font-size:6.35px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-106.33705"
+ y="173.5065"
+ id="text4097-8"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4095-4"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-106.33705"
+ y="173.5065">Queue</tspan><tspan
+ sodipodi:role="line"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-114.27455"
+ y="173.5065"
+ id="tspan4099-5">Pair 1</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.388863;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1811"
+ width="58.909527"
+ height="18.812746"
+ x="144.0564"
+ y="41.268894"
+ ry="2.2255962" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-47.017281"
+ y="173.49187"
+ id="text4156"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4154"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';stroke-width:0.75"
+ x="-47.017281"
+ y="173.49187">Machine Learning</tspan><tspan
+ sodipodi:role="line"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';stroke-width:0.75"
+ x="-54.07283"
+ y="173.49187"
+ id="tspan4158">Hardware Engine</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.368668;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.737336, 0.737336;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924-0"
+ width="46.97015"
+ height="73.58287"
+ x="98.42067"
+ y="72.988976"
+ ry="2.4685853" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-88.32518"
+ y="122.24379"
+ id="text3708"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-88.32518"
+ y="122.24379">lcore 1</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-132.10504"
+ y="121.83865"
+ id="text3708-8"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-7"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-132.10504"
+ y="121.83865">lcore 5</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-0"
+ width="32.290321"
+ height="11.709678"
+ x="105.76101"
+ y="82.031425"
+ ry="3.0161259" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-6-7"
+ width="32.290321"
+ height="11.709678"
+ x="105.76101"
+ y="125.82092"
+ ry="3.0161259"
+ inkscape:connector-avoid="true" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-4-8"
+ width="32.290321"
+ height="11.709678"
+ x="105.76101"
+ y="103.92618"
+ ry="3.0161259"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-110.21718"
+ y="121.85381"
+ id="text3708-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-87"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-110.21718"
+ y="121.85381">lcore 3</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-150.82878"
+ y="121.79179"
+ id="text4542"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4540"
+ style="font-size:5.64444px;stroke-width:0.75"
+ x="-150.82878"
+ y="121.79179">Enqueue Workers</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.368668;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.737336, 0.737336;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924-7"
+ width="46.97015"
+ height="73.58287"
+ x="201.63062"
+ y="72.988976"
+ ry="2.4685853" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-88.32518"
+ y="225.08443"
+ id="text3708-9"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-9"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-88.32518"
+ y="225.08443">lcore 2</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-110.21167"
+ y="225.07202"
+ id="text3708-7"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-8"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-110.21167"
+ y="225.07202">lcore 4</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724"
+ width="32.290321"
+ height="11.709678"
+ x="208.97096"
+ y="82.031425"
+ ry="3.0161259" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-4"
+ width="32.290321"
+ height="11.709678"
+ x="208.97096"
+ y="103.92618"
+ ry="3.0161259" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-6"
+ width="32.290321"
+ height="11.709678"
+ x="208.97096"
+ y="125.82092"
+ ry="3.0161259" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-132.1133"
+ y="225.06514"
+ id="text3708-78"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-0"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-132.1133"
+ y="225.06514">lcore 6</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-150.82878"
+ y="225.00725"
+ id="text4542-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4540-7"
+ style="font-size:5.64444px;stroke-width:0.75"
+ x="-150.82878"
+ y="225.00725">Dequeue Workers</tspan></text>
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 79.368074,78.962117 26.440446,5.546991"
+ id="path6677"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-0"
+ inkscape:connection-start="#rect6252" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="M 78.978262,84.050781 112.13805,103.92618"
+ id="path6679"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-4-8"
+ inkscape:connection-start="#rect6252" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 73.959073,84.25738 42.026977,41.56354"
+ id="path6681"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-6-7"
+ inkscape:connection-start="#rect6252" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="M 79.368074,95.959838 105.76913,90.949016"
+ id="path6683"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-0"
+ inkscape:connection-start="#rect6252-8" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 79.368074,101.28215 26.416716,5.27791"
+ id="path7830"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-4-8"
+ inkscape:connection-start="#rect6252-8" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 79.069199,106.4283 32.903161,19.39262"
+ id="path7862"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-6-7"
+ inkscape:connection-start="#rect6252-8" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="M 79.069201,113.13371 111.97235,93.741103"
+ id="path7900"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-0"
+ inkscape:connection-start="#rect6252-2" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 79.368074,118.27987 26.416716,-5.2779"
+ id="path7932"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-4-8"
+ inkscape:connection-start="#rect6252-2" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 79.368074,123.60218 26.401056,5.01083"
+ id="path7998"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-6-7"
+ inkscape:connection-start="#rect6252-2" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="M 73.959074,135.30464 115.98605,93.741103"
+ id="path8000"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-0"
+ inkscape:connection-start="#rect6252-6" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="M 78.978264,135.51124 112.13805,115.63586"
+ id="path8002"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-4-8"
+ inkscape:connection-start="#rect6252-6" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 79.368074,140.59991 26.440446,-5.54699"
+ id="path8004"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-6-7"
+ inkscape:connection-start="#rect6252-6" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-76.53363"
+ y="65.63237"
+ id="text3708-84"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-4"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-76.53363"
+ y="65.63237">Model 0</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-98.991623"
+ y="66.015465"
+ id="text3708-84-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-4-6"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-98.991623"
+ y="66.015465">Model 1</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-121.44823"
+ y="65.646149"
+ id="text3708-84-9"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-4-1"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-121.44823"
+ y="65.646149">Model 2</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-143.9021"
+ y="65.625481"
+ id="text3708-84-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-4-8"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-143.9021"
+ y="65.625481">Model 3</tspan></text>
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect6252"
+ width="27.32258"
+ height="16.32258"
+ x="52.045494"
+ y="67.934799"
+ ry="1" />
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect6252-8"
+ width="27.32258"
+ height="16.32258"
+ x="52.045494"
+ y="90.391411"
+ ry="1" />
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect6252-2"
+ width="27.32258"
+ height="16.32258"
+ x="52.045494"
+ y="112.84802"
+ ry="1" />
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect6252-6"
+ width="27.32258"
+ height="16.32258"
+ x="52.045494"
+ y="135.30464"
+ ry="1" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.399886;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.799773, 0.799773;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924-1-3"
+ width="43.035683"
+ height="94.487152"
+ x="44.188942"
+ y="62.536991"
+ ry="3.1694498" />
+ <g
+ id="g1010"
+ transform="translate(21.464467,-15.875004)">
+ <text
+ xml:space="preserve"
+ style="font-weight:bold;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:'Arial Bold';text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-185.88483"
+ y="86.542366"
+ id="text15571"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan15569"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.5;stroke-opacity:1"
+ x="-185.88483"
+ y="86.542366">nb_worker_threads = 2 * MIN(nb_queue_pairs, (lcore_count - 1) / 2)</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-weight:bold;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:'Arial Bold';text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-178.43243"
+ y="90.903854"
+ id="text15571-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan15569-9"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.5;stroke-opacity:1"
+ x="-178.43243"
+ y="90.903854">inferences_per_queue_pair = nb_models * (repetitions / nb_queue_pairs)</tspan></text>
+ </g>
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/tools/img/mldev_inference_ordered.svg b/doc/guides/tools/img/mldev_inference_ordered.svg
new file mode 100644
index 0000000000..12fa6acaec
--- /dev/null
+++ b/doc/guides/tools/img/mldev_inference_ordered.svg
@@ -0,0 +1,528 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- SPDX-License-Identifier: BSD-3-Clause -->
+<!-- Copyright (c) 2022 Marvell. -->
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="243mm"
+ height="144mm"
+ viewBox="0 0 243 144"
+ version="1.1"
+ id="svg5"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="inference_ordered.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns:xlink="http://www.w3.org/1999/xlink"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg">
+ <sodipodi:namedview
+ id="namedview7"
+ pagecolor="#ffffff"
+ bordercolor="#000000"
+ borderopacity="0.25"
+ inkscape:showpageshadow="2"
+ inkscape:pageopacity="0.0"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1"
+ inkscape:document-units="mm"
+ showgrid="false"
+ inkscape:zoom="0.74564394"
+ inkscape:cx="488.83922"
+ inkscape:cy="234.69647"
+ inkscape:window-width="1920"
+ inkscape:window-height="1017"
+ inkscape:window-x="1912"
+ inkscape:window-y="-8"
+ inkscape:window-maximized="1"
+ inkscape:current-layer="layer1" />
+ <defs
+ id="defs2">
+ <marker
+ style="overflow:visible"
+ id="RoundedArrow"
+ refX="5"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="RoundedArrow"
+ markerWidth="6.1347523"
+ markerHeight="5.9304948"
+ viewBox="0 0 6.1347524 5.9304951"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.7)"
+ d="m -0.21114562,-4.1055728 6.42229122,3.21114561 a 1,1 90 0 1 0,1.78885438 L -0.21114562,4.1055728 A 1.236068,1.236068 31.717474 0 1 -2,3 v -6 a 1.236068,1.236068 148.28253 0 1 1.78885438,-1.1055728 z"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:none"
+ id="path1367" />
+ </marker>
+ <linearGradient
+ inkscape:collect="always"
+ id="linearGradient31002">
+ <stop
+ style="stop-color:#fff6d5;stop-opacity:1;"
+ offset="0"
+ id="stop30998" />
+ <stop
+ style="stop-color:#fff6d5;stop-opacity:0;"
+ offset="1"
+ id="stop31000" />
+ </linearGradient>
+ <marker
+ style="overflow:visible"
+ id="TriangleStart"
+ refX="4"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="TriangleStart"
+ markerWidth="5.3244081"
+ markerHeight="6.155385"
+ viewBox="0 0 5.3244081 6.1553851"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.5)"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:context-stroke;stroke-width:1pt"
+ d="M 5.77,0 -2.88,5 V -5 Z"
+ id="path135" />
+ </marker>
+ <linearGradient
+ inkscape:collect="always"
+ xlink:href="#linearGradient31002"
+ id="linearGradient31004"
+ x1="19.620968"
+ y1="102.90323"
+ x2="279.1532"
+ y2="102.90323"
+ gradientUnits="userSpaceOnUse"
+ gradientTransform="matrix(0.89215122,0,0,0.73190287,13.449912,42.668706)" />
+ </defs>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1">
+ <g
+ id="g1340"
+ transform="translate(-25.225796,-45.983871)">
+ <rect
+ style="fill:url(#linearGradient31004);fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.404032;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect27876"
+ width="231.09595"
+ height="132.45081"
+ x="31.177822"
+ y="51.758469"
+ ry="3.5071263" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1813"
+ width="38.408459"
+ height="45.86002"
+ x="34.901794"
+ y="99.14959"
+ ry="5.2246051"
+ inkscape:connector-avoid="true" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 73.310253,115.94935 36.498807,-11.6509"
+ id="path1906"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1813"
+ inkscape:connection-end="#rect1724-0" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 73.310253,122.0796 36.117817,1e-5"
+ id="path1908"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1813"
+ inkscape:connection-end="#rect1724-4-8" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="M 73.310253,128.20983 109.80905,139.8607"
+ id="path1910"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1813"
+ inkscape:connection-end="#rect1724-6-7" />
+ <path
+ style="display:inline;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 141.71839,99.266314 19.42262,-10e-7"
+ id="path1912"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1724-0"
+ inkscape:connection-end="#rect1679" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 197.14101,99.266313 19.42259,10e-7"
+ id="path1914"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1679"
+ inkscape:connection-end="#rect1724" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 141.71839,122.07961 19.42262,-1e-5"
+ id="path1916"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1724-4-8"
+ inkscape:connection-end="#rect1679-4" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 197.14101,122.0796 19.42259,1e-5"
+ id="path1918"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1679-4"
+ inkscape:connection-end="#rect1724-4" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 141.71839,144.89282 19.42262,0"
+ id="path1920"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1724-6-7"
+ inkscape:connection-end="#rect1679-8" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 197.14101,144.89282 19.42259,0"
+ id="path1922"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1679-8"
+ inkscape:connection-end="#rect1724-6" />
+ <text
+ xml:space="preserve"
+ style="font-size:8.46667px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-121.09793"
+ y="54.031597"
+ id="text4093"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4091"
+ style="font-size:8.46667px;stroke-width:0.75"
+ x="-121.09793"
+ y="54.031597">Model X</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:6.35px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-118.63563"
+ y="179.13635"
+ id="text4097"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4095"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-118.63563"
+ y="179.13635">Queue</tspan><tspan
+ sodipodi:role="line"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-126.57313"
+ y="179.13635"
+ id="tspan4099">Pair 1</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:6.35px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-141.44887"
+ y="179.13635"
+ id="text4097-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4095-6"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-141.44887"
+ y="179.13635">Queue</tspan><tspan
+ sodipodi:role="line"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-149.38637"
+ y="179.13635"
+ id="tspan4099-4">Pair 2</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.354849;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.709699, 0.709699;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924"
+ width="44.145252"
+ height="72.532341"
+ x="157.06865"
+ y="85.813438"
+ ry="4.31247" />
+ <g
+ id="g1224">
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1679"
+ width="36"
+ height="18"
+ x="161.14101"
+ y="90.266312"
+ ry="3" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1679-8"
+ width="36"
+ height="18"
+ x="161.14101"
+ y="135.89282"
+ ry="3" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1679-4"
+ width="36"
+ height="18"
+ x="161.14101"
+ y="113.07959"
+ ry="3" />
+ </g>
+ <text
+ xml:space="preserve"
+ style="font-size:6.35px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-95.820801"
+ y="179.13635"
+ id="text4097-8"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4095-4"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-95.820801"
+ y="179.13635">Queue</tspan><tspan
+ sodipodi:role="line"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-103.7583"
+ y="179.13635"
+ id="tspan4099-5">Pair 0</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.317648;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1811"
+ width="44.196934"
+ height="16.731901"
+ x="157.04254"
+ y="56.49292"
+ ry="2.761292" />
+ <text
+ xml:space="preserve"
+ style="font-size:3.5859px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.317649;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.952945, 0.317649;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-60.009941"
+ y="186.38451"
+ id="text4156"
+ transform="matrix(0,-1.040508,0.96106903,0,0,0)"><tspan
+ sodipodi:role="line"
+ id="tspan4154"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-family:Arial;-inkscape-font-specification:'Arial Bold';stroke-width:0.317648"
+ x="-60.009941"
+ y="186.38451">Machine Learning</tspan><tspan
+ sodipodi:role="line"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-family:Arial;-inkscape-font-specification:'Arial Bold';stroke-width:0.317648"
+ x="-64.492317"
+ y="186.38451"
+ id="tspan4158">Hardware Engine</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-99.705231"
+ y="125.91087"
+ id="text3708"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-99.705231"
+ y="125.91087">lcore 1</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-145.3221"
+ y="125.50572"
+ id="text3708-8"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-7"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-145.3221"
+ y="125.50572">lcore 5</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-122.51577"
+ y="125.52089"
+ id="text3708-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-87"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-122.51577"
+ y="125.52089">lcore 3</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-162.06549"
+ y="125.4589"
+ id="text4542"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4540"
+ style="font-size:5.64444px;stroke-width:0.75"
+ x="-162.06549"
+ y="125.4589">Enqueue Workers</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-99.705231"
+ y="232.67706"
+ id="text3708-9"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-9"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-99.705231"
+ y="232.67706">lcore 2</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-122.51025"
+ y="232.66466"
+ id="text3708-7"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-8"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-122.51025"
+ y="232.66466">lcore 4</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-145.33035"
+ y="232.65778"
+ id="text3708-78"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-0"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-145.33035"
+ y="232.65778">lcore 6</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-162.06549"
+ y="232.59988"
+ id="text4542-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4540-7"
+ style="font-size:5.64444px;stroke-width:0.75"
+ x="-162.06549"
+ y="232.59988">Dequeue Workers</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:6.35px;font-family:Arial;-inkscape-font-specification:'Arial Bold';text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-177.01665"
+ y="220.07283"
+ id="text5181"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan5179"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-177.01665"
+ y="220.07283">test: inference_ordered</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-weight:bold;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:'Arial Bold';text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-178.63324"
+ y="98.67057"
+ id="text15571"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan15569"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.5;stroke-opacity:1"
+ x="-178.63324"
+ y="98.67057">nb_worker_threads = 2 * MIN(nb_queue_pairs, (lcore_count - 1) / 2)</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-weight:bold;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:'Arial Bold';text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-171.18085"
+ y="89.26754"
+ id="text15571-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan15569-9"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.5;stroke-opacity:1"
+ x="-171.18085"
+ y="89.26754">inferences_per_queue_pair = repetitions / nb_queue_pairs</tspan></text>
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.75;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#RoundedArrow);marker-end:url(#RoundedArrow)"
+ d="m 179.14101,85.813438 0,-12.588618"
+ id="path31090"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1924" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724"
+ width="32.290321"
+ height="11.709678"
+ x="216.5636"
+ y="93.411476"
+ ry="2" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-4"
+ width="32.290321"
+ height="11.709678"
+ x="216.5636"
+ y="116.22478"
+ ry="2" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-6"
+ width="32.290321"
+ height="11.709678"
+ x="216.5636"
+ y="139.03798"
+ ry="2" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.354849;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.709699, 0.709699;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924-6"
+ width="44.145252"
+ height="72.532341"
+ x="210.6364"
+ y="85.813477"
+ ry="4.31247" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-0"
+ width="32.290321"
+ height="11.709678"
+ x="109.42807"
+ y="93.411476"
+ ry="2" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-6-7"
+ width="32.290321"
+ height="11.709678"
+ x="109.42807"
+ y="139.03798"
+ ry="2"
+ inkscape:connector-avoid="true" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-4-8"
+ width="32.290321"
+ height="11.709678"
+ x="109.42807"
+ y="116.22478"
+ ry="2"
+ inkscape:connector-avoid="true" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.354849;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.709699, 0.709699;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924-6-3"
+ width="44.145252"
+ height="72.532341"
+ x="103.50092"
+ y="85.813477"
+ ry="4.31247" />
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/tools/img/mldev_model_ops_subtest_a.svg b/doc/guides/tools/img/mldev_model_ops_subtest_a.svg
new file mode 100644
index 0000000000..ed12cc5a05
--- /dev/null
+++ b/doc/guides/tools/img/mldev_model_ops_subtest_a.svg
@@ -0,0 +1,420 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- SPDX-License-Identifier: BSD-3-Clause -->
+<!-- Copyright (c) 2022 Marvell. -->
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="275mm"
+ height="135mm"
+ viewBox="0 0 275 135"
+ version="1.1"
+ id="svg5"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="mldev_model_ops_subtest_d.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg">
+ <sodipodi:namedview
+ id="namedview7"
+ pagecolor="#ffffff"
+ bordercolor="#000000"
+ borderopacity="0.25"
+ inkscape:showpageshadow="2"
+ inkscape:pageopacity="0.0"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1"
+ inkscape:document-units="mm"
+ showgrid="false"
+ inkscape:zoom="0.72515432"
+ inkscape:cx="372.33454"
+ inkscape:cy="401.29389"
+ inkscape:window-width="1920"
+ inkscape:window-height="986"
+ inkscape:window-x="-11"
+ inkscape:window-y="-11"
+ inkscape:window-maximized="1"
+ inkscape:current-layer="layer1" />
+ <defs
+ id="defs2">
+ <marker
+ style="overflow:visible"
+ id="RoundedArrow"
+ refX="6"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="RoundedArrow"
+ markerWidth="6.1347523"
+ markerHeight="5.9304948"
+ viewBox="0 0 6.1347524 5.9304951"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.7)"
+ d="m -0.21114562,-4.1055728 6.42229122,3.21114561 a 1,1 90 0 1 0,1.78885438 L -0.21114562,4.1055728 A 1.236068,1.236068 31.717474 0 1 -2,3 v -6 a 1.236068,1.236068 148.28253 0 1 1.78885438,-1.1055728 z"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:none"
+ id="path1367" />
+ </marker>
+ </defs>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1">
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 62.203489,25.425124 H 80.823463"
+ id="path1916"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234"
+ inkscape:connection-end="#rect234-07" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 128.19001,25.425124 h 18.6197"
+ id="path1918"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-07"
+ inkscape:connection-end="#rect234-6" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 194.17681,25.425125 h 18.61942"
+ id="path1922"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-6"
+ inkscape:connection-end="#rect234-0-6-6" />
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.633453;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234"
+ width="47.366547"
+ height="17.366549"
+ x="14.836943"
+ y="16.74185"
+ ry="1.4985789"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-25.888845"
+ y="38.514706"
+ id="text290"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-25.888845"
+ y="38.514706">Model 0 / Load</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.633453;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-07"
+ width="47.366547"
+ height="17.366549"
+ x="80.823463"
+ y="16.74185"
+ ry="1.4985789"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-25.888845"
+ y="104.31795"
+ id="text290-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-11"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-25.888845"
+ y="104.31795">Model 0 / Start</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.6329;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-6"
+ width="47.3671"
+ height="17.33604"
+ x="146.80971"
+ y="16.757105"
+ ry="1.4959463" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-25.362436"
+ y="170.39679"
+ id="text290-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-15"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-25.362436"
+ y="170.39679">Model 0 / Stop</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6"
+ width="47.3671"
+ height="17.33604"
+ x="212.79623"
+ y="16.757105"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-25.888849"
+ y="236.47427"
+ id="text290-3-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-25.888849"
+ y="236.47427">Model 0 / Unload</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 236.47978,34.093145 V 44.957249 H 38.520216 v 10.84885"
+ id="path1924"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6"
+ inkscape:connection-end="#rect234-0-6-6-3" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 62.203766,64.474119 H 80.823189"
+ id="path1926"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-3"
+ inkscape:connection-end="#rect234-0-6-6-2" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 128.19029,64.474119 h 18.61942"
+ id="path1928"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-2"
+ inkscape:connection-end="#rect234-0-6-1" />
+ <rect
+ style="fill:#fff6d5;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-3"
+ width="47.3671"
+ height="17.33604"
+ x="14.836666"
+ y="55.806099"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-64.937843"
+ y="38.541786"
+ id="text290-3-2-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-64.937843"
+ y="38.541786">Model 1 / Load</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-2"
+ width="47.3671"
+ height="17.33604"
+ x="80.823189"
+ y="55.806099"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-64.937843"
+ y="104.31795"
+ id="text290-3-2-2-28"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-2"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-64.937843"
+ y="104.31795">Model 1 / Start</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6"
+ width="47.3671"
+ height="17.33604"
+ x="212.79623"
+ y="55.806099"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-64.937843"
+ y="236.47427"
+ id="text290-3-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-64.937843"
+ y="236.47427">Model 1 / Unload</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-1"
+ width="47.3671"
+ height="17.33604"
+ x="146.80971"
+ y="55.806099"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-64.41143"
+ y="170.39679"
+ id="text290-3-2-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-8"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-64.41143"
+ y="170.39679">Model 1 / Stop</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 194.17681,64.474119 h 18.61942"
+ id="path1930"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6"
+ inkscape:connection-start="#rect234-0-6-1" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 236.47978,73.142139 V 83.990988 H 38.520216 V 94.85511"
+ id="path1932"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6-7"
+ inkscape:connection-start="#rect234-0-6" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 62.203766,103.52313 h 9.309711 v 1.3e-4 h 9.309712"
+ id="path1934"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-7" />
+ <rect
+ style="fill:#fff6d5;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-7"
+ width="47.3671"
+ height="17.33604"
+ x="14.836666"
+ y="94.85511"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.98687"
+ y="38.541786"
+ id="text290-3-2-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.98687"
+ y="38.541786">Model N / Load</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0"
+ width="47.3671"
+ height="17.33604"
+ x="80.823189"
+ y="94.855164"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.9869"
+ y="104.31795"
+ id="text290-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.9869"
+ y="104.31795">Model N / Start</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-9"
+ width="47.3671"
+ height="17.33604"
+ x="146.80971"
+ y="94.855164"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.4605"
+ y="170.39679"
+ id="text290-3-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.4605"
+ y="170.39679">Model N / Stop</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.633452;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-5"
+ width="47.366547"
+ height="17.366549"
+ x="212.79651"
+ y="94.839836"
+ ry="1.4985789"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.98682"
+ y="236.47427"
+ id="text290-3-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-3"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.98682"
+ y="236.47427">Model N / Unload</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 128.19029,103.52326 h 18.61942"
+ id="path1936"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 194.17681,103.52326 h 9.30985 v -1.5e-4 h 9.30985"
+ id="path1938"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-5" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#fff6d5;fill-rule:evenodd;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-120.71075"
+ y="220.77164"
+ id="text392"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan390"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';fill:#000000;stroke-width:0.5"
+ x="-120.71075"
+ y="220.77164">mldev: model_ops / subtest D</tspan></text>
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.448724;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect2164"
+ width="259.55127"
+ height="119.55128"
+ x="7.7243652"
+ y="7.7243614"
+ ry="1.17494" />
+ </g>
+</svg>
diff --git a/doc/guides/tools/img/mldev_model_ops_subtest_b.svg b/doc/guides/tools/img/mldev_model_ops_subtest_b.svg
new file mode 100644
index 0000000000..173a2c6c05
--- /dev/null
+++ b/doc/guides/tools/img/mldev_model_ops_subtest_b.svg
@@ -0,0 +1,423 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- SPDX-License-Identifier: BSD-3-Clause -->
+<!-- Copyright (c) 2022 Marvell. -->
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="275mm"
+ height="135mm"
+ viewBox="0 0 275 135"
+ version="1.1"
+ id="svg5"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="mldev_model_ops_subtest_a.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg">
+ <sodipodi:namedview
+ id="namedview7"
+ pagecolor="#ffffff"
+ bordercolor="#000000"
+ borderopacity="0.25"
+ inkscape:showpageshadow="2"
+ inkscape:pageopacity="0.0"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1"
+ inkscape:document-units="mm"
+ showgrid="false"
+ inkscape:zoom="0.72515432"
+ inkscape:cx="277.87189"
+ inkscape:cy="401.29389"
+ inkscape:window-width="1920"
+ inkscape:window-height="986"
+ inkscape:window-x="-11"
+ inkscape:window-y="-11"
+ inkscape:window-maximized="1"
+ inkscape:current-layer="layer1" />
+ <defs
+ id="defs2">
+ <marker
+ style="overflow:visible"
+ id="RoundedArrow"
+ refX="6"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="RoundedArrow"
+ markerWidth="6.1347523"
+ markerHeight="5.9304948"
+ viewBox="0 0 6.1347524 5.9304951"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.7)"
+ d="m -0.21114562,-4.1055728 6.42229122,3.21114561 a 1,1 90 0 1 0,1.78885438 L -0.21114562,4.1055728 A 1.236068,1.236068 31.717474 0 1 -2,3 v -6 a 1.236068,1.236068 148.28253 0 1 1.78885438,-1.1055728 z"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:none"
+ id="path1367" />
+ </marker>
+ </defs>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1">
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.671403;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234"
+ width="53.328598"
+ height="17.328598"
+ x="15.144302"
+ y="12.878438"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-22.00646"
+ y="41.803089"
+ id="text290"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-22.00646"
+ y="41.803089">Model 0 / Load</tspan></text>
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.671403;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-07"
+ width="53.328598"
+ height="17.328598"
+ x="87.552399"
+ y="12.878438"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-22.00646"
+ y="114.21132"
+ id="text290-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-11"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-22.00646"
+ y="114.21132">Model 1 / Load</tspan></text>
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.671403;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-6"
+ width="53.328598"
+ height="17.328598"
+ x="206.5271"
+ y="12.878438"
+ ry="1.4953041" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-22.00646"
+ y="233.18588"
+ id="text290-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-15"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-22.00646"
+ y="233.18588">Model N / Load</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 68.4729,21.542737 H 87.552399"
+ id="path1916"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234"
+ inkscape:connection-end="#rect234-07" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 140.881,21.542737 h 65.6461"
+ id="path1918"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-07"
+ inkscape:connection-end="#rect234-6" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 233.1914,30.207036 0,4.918771 H 41.808601 v 4.918772"
+ id="path1922"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-6"
+ inkscape:connection-end="#rect234-0-6-6" />
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6"
+ width="53.328598"
+ height="17.328598"
+ x="15.144302"
+ y="40.044579"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-49.1726"
+ y="41.619808"
+ id="text290-3-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-49.1726"
+ y="41.619808">Model 0 / Start</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-3"
+ width="53.328598"
+ height="17.328598"
+ x="87.552399"
+ y="40.044579"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-49.1726"
+ y="114.02805"
+ id="text290-3-2-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-49.1726"
+ y="114.02805">Model 1 / Start</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-2"
+ width="53.328598"
+ height="17.328598"
+ x="206.5271"
+ y="40.044579"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-49.1726"
+ y="233.00261"
+ id="text290-3-2-2-28"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-2"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-49.1726"
+ y="233.00261">Model N / Start</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 68.4729,48.708878 H 87.552399"
+ id="path1924"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6"
+ inkscape:connection-end="#rect234-0-6-6-3" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 140.881,48.708878 h 65.6461"
+ id="path1926"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-3"
+ inkscape:connection-end="#rect234-0-6-6-2" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 233.1914,57.373177 V 62.29195 H 41.808599 v 4.918774"
+ id="path1928"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-2"
+ inkscape:connection-end="#rect234-0-6-1" />
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6"
+ width="53.328598"
+ height="17.328598"
+ x="87.552399"
+ y="67.210724"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-75.812294"
+ y="114.12037"
+ id="text290-3-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-75.812294"
+ y="114.12037">Model 1 / Stop</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-1"
+ width="53.328598"
+ height="17.328598"
+ x="15.1443"
+ y="67.210724"
+ ry="1.4953041" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-75.812294"
+ y="41.712139"
+ id="text290-3-2-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-8"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-75.812294"
+ y="41.712139">Model 0 / Stop</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-7"
+ width="53.328598"
+ height="17.328598"
+ x="206.5271"
+ y="67.210724"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-75.812294"
+ y="233.09494"
+ id="text290-3-2-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-75.812294"
+ y="233.09494">Model N / Stop</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 68.472898,75.875023 H 87.552399"
+ id="path1930"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6"
+ inkscape:connection-start="#rect234-0-6-1" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 140.881,75.875023 h 65.6461"
+ id="path1932"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6-7"
+ inkscape:connection-start="#rect234-0-6" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 233.1914,84.539322 0,4.91877 H 41.808602 v 4.91877"
+ id="path1934"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0"
+ inkscape:connection-start="#rect234-0-6-7" />
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0"
+ width="53.328598"
+ height="17.328598"
+ x="15.144303"
+ y="94.376862"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.50489"
+ y="41.803085"
+ id="text290-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.50489"
+ y="41.803085">Model 0 / Unload</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-9"
+ width="53.328598"
+ height="17.328598"
+ x="87.552399"
+ y="94.376862"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.50489"
+ y="114.2113"
+ id="text290-3-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.50489"
+ y="114.2113">Model 1 / Unload</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-5"
+ width="53.328598"
+ height="17.328598"
+ x="206.5271"
+ y="94.376862"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.50489"
+ y="233.18588"
+ id="text290-3-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-3"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.50489"
+ y="233.18588">Model N / Unload</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 68.472901,103.04116 H 87.552399"
+ id="path1936"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-9"
+ inkscape:connection-start="#rect234-0" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 140.881,103.04116 h 65.6461"
+ id="path1938"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-5"
+ inkscape:connection-start="#rect234-0-9" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#fff6d5;fill-rule:evenodd;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-119.78458"
+ y="220.32892"
+ id="text392"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan390"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';fill:#000000;stroke-width:0.5"
+ x="-119.78458"
+ y="220.32892">mldev: model_ops / subtest A</tspan></text>
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.442854;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect3967"
+ width="259.55716"
+ height="119.55714"
+ x="7.7214203"
+ y="7.7214317"
+ ry="1.1806604" />
+ </g>
+</svg>
diff --git a/doc/guides/tools/img/mldev_model_ops_subtest_c.svg b/doc/guides/tools/img/mldev_model_ops_subtest_c.svg
new file mode 100644
index 0000000000..f66f146d05
--- /dev/null
+++ b/doc/guides/tools/img/mldev_model_ops_subtest_c.svg
@@ -0,0 +1,366 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- SPDX-License-Identifier: BSD-3-Clause -->
+<!-- Copyright (c) 2022 Marvell. -->
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="275mm"
+ height="135mm"
+ viewBox="0 0 275 135"
+ version="1.1"
+ id="svg5"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="mldev_model_ops_subtest_c.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg">
+ <sodipodi:namedview
+ id="namedview7"
+ pagecolor="#ffffff"
+ bordercolor="#000000"
+ borderopacity="0.25"
+ inkscape:showpageshadow="2"
+ inkscape:pageopacity="0.0"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1"
+ inkscape:document-units="mm"
+ showgrid="false"
+ inkscape:zoom="0.72515432"
+ inkscape:cx="479.89785"
+ inkscape:cy="401.29389"
+ inkscape:window-width="1920"
+ inkscape:window-height="986"
+ inkscape:window-x="-11"
+ inkscape:window-y="-11"
+ inkscape:window-maximized="1"
+ inkscape:current-layer="layer1" />
+ <defs
+ id="defs2">
+ <marker
+ style="overflow:visible"
+ id="RoundedArrow"
+ refX="6"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="RoundedArrow"
+ markerWidth="6.1347523"
+ markerHeight="5.9304948"
+ viewBox="0 0 6.1347524 5.9304951"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.7)"
+ d="m -0.21114562,-4.1055728 6.42229122,3.21114561 a 1,1 90 0 1 0,1.78885438 L -0.21114562,4.1055728 A 1.236068,1.236068 31.717474 0 1 -2,3 v -6 a 1.236068,1.236068 148.28253 0 1 1.78885438,-1.1055728 z"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:none"
+ id="path1367" />
+ </marker>
+ </defs>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1">
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.671403;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234"
+ width="53.328598"
+ height="17.328598"
+ x="12.6443"
+ y="13.208546"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-22.336561"
+ y="39.303089"
+ id="text290"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-22.336561"
+ y="39.303089">Model 0 / Load</tspan></text>
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.671403;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-07"
+ width="53.328598"
+ height="17.328598"
+ x="85.052399"
+ y="13.208546"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-22.336561"
+ y="111.71132"
+ id="text290-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-11"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-22.336561"
+ y="111.71132">Model 1 / Load</tspan></text>
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.671403;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-6"
+ width="53.328598"
+ height="17.328598"
+ x="204.0271"
+ y="13.208546"
+ ry="1.4953041" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-22.336561"
+ y="230.68588"
+ id="text290-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-15"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-22.336561"
+ y="230.68588">Model N / Load</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 65.972898,21.872845 H 85.052399"
+ id="path1916"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234"
+ inkscape:connection-end="#rect234-07" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 138.381,21.872845 h 65.6461"
+ id="path1918"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-07"
+ inkscape:connection-end="#rect234-6" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 230.6914,30.537144 V 36.33787 H 39.308599 v 5.800726"
+ id="path1922"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-6"
+ inkscape:connection-end="#rect234-0-6-6" />
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6"
+ width="53.328598"
+ height="17.328598"
+ x="12.6443"
+ y="42.138596"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-51.266617"
+ y="39.119808"
+ id="text290-3-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-51.266617"
+ y="39.119808">Model 0 / Start</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-3"
+ width="53.328598"
+ height="17.328598"
+ x="85.052399"
+ y="42.138596"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-51.266617"
+ y="111.52805"
+ id="text290-3-2-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-51.266617"
+ y="111.52805">Model 0 / Stop</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 65.972898,50.802895 H 85.052399"
+ id="path1924"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6"
+ inkscape:connection-end="#rect234-0-6-6-3" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 184.94759,79.732941 H 204.0271"
+ id="path1930"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6"
+ inkscape:connection-start="#rect234-0-6-1" />
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0"
+ width="53.328598"
+ height="17.328598"
+ x="12.644301"
+ y="99.998688"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-109.12671"
+ y="39.303085"
+ id="text290-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-109.12671"
+ y="39.303085">Model 0 / Unload</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-9"
+ width="53.328598"
+ height="17.328598"
+ x="85.052399"
+ y="99.998688"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-109.12671"
+ y="111.7113"
+ id="text290-3-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-109.12671"
+ y="111.7113">Model 1 / Unload</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-5"
+ width="53.328598"
+ height="17.328598"
+ x="204.0271"
+ y="99.998688"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-109.12671"
+ y="230.68588"
+ id="text290-3-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-3"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-109.12671"
+ y="230.68588">Model N / Unload</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 65.972899,108.66299 h 19.0795"
+ id="path1936"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-9"
+ inkscape:connection-start="#rect234-0" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 138.381,108.66299 h 65.6461"
+ id="path1938"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-5"
+ inkscape:connection-start="#rect234-0-9" />
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6"
+ width="53.328598"
+ height="17.328598"
+ x="204.0271"
+ y="71.068642"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-79.670258"
+ y="230.59494"
+ id="text290-3-2-2-28"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-2"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-79.670258"
+ y="230.59494">Model N / Stop</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-1"
+ width="53.328598"
+ height="17.328598"
+ x="131.61899"
+ y="71.068642"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-80.196663"
+ y="158.0945"
+ id="text290-3-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-80.196663"
+ y="158.0945">Model N / Start</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 138.381,50.802895 h 19.90229 v 20.265747"
+ id="path1158"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-3"
+ inkscape:connection-end="#rect234-0-6-1" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 230.6914,88.39724 v 5.800724 H 39.3086 v 5.800724"
+ id="path1160"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6"
+ inkscape:connection-end="#rect234-0" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#fff6d5;fill-rule:evenodd;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-123.28999"
+ y="217.99152"
+ id="text392"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan390"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';fill:#000000;stroke-width:0.5"
+ x="-123.28999"
+ y="217.99152">mldev: model_ops / subtest C</tspan></text>
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.441855;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect3015"
+ width="259.55814"
+ height="119.55814"
+ x="7.720932"
+ y="7.7209282"
+ ry="1.1638433" />
+ </g>
+</svg>
diff --git a/doc/guides/tools/img/mldev_model_ops_subtest_d.svg b/doc/guides/tools/img/mldev_model_ops_subtest_d.svg
new file mode 100644
index 0000000000..3e2b89ad25
--- /dev/null
+++ b/doc/guides/tools/img/mldev_model_ops_subtest_d.svg
@@ -0,0 +1,424 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- SPDX-License-Identifier: BSD-3-Clause -->
+<!-- Copyright (c) 2022 Marvell. -->
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="275mm"
+ height="135mm"
+ viewBox="0 0 275 135"
+ version="1.1"
+ id="svg5"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="mldev_model_ops_subtest_b.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg">
+ <sodipodi:namedview
+ id="namedview7"
+ pagecolor="#ffffff"
+ bordercolor="#000000"
+ borderopacity="0.25"
+ inkscape:showpageshadow="2"
+ inkscape:pageopacity="0.0"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1"
+ inkscape:document-units="mm"
+ showgrid="false"
+ inkscape:zoom="0.72515432"
+ inkscape:cx="324.06895"
+ inkscape:cy="295.1096"
+ inkscape:window-width="1920"
+ inkscape:window-height="986"
+ inkscape:window-x="-11"
+ inkscape:window-y="-11"
+ inkscape:window-maximized="1"
+ inkscape:current-layer="layer1" />
+ <defs
+ id="defs2">
+ <marker
+ style="overflow:visible"
+ id="RoundedArrow"
+ refX="6"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="RoundedArrow"
+ markerWidth="6.1347523"
+ markerHeight="5.9304948"
+ viewBox="0 0 6.1347524 5.9304951"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.7)"
+ d="m -0.21114562,-4.1055728 6.42229122,3.21114561 a 1,1 90 0 1 0,1.78885438 L -0.21114562,4.1055728 A 1.236068,1.236068 31.717474 0 1 -2,3 v -6 a 1.236068,1.236068 148.28253 0 1 1.78885438,-1.1055728 z"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:none"
+ id="path1367" />
+ </marker>
+ </defs>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1">
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.633453;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234"
+ width="47.366547"
+ height="17.366549"
+ x="14.864025"
+ y="14.24185"
+ ry="1.4985789"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-23.388845"
+ y="38.541786"
+ id="text290"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-23.388845"
+ y="38.541786">Model 0 / Load</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.633453;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-07"
+ width="47.366547"
+ height="17.366549"
+ x="81.026939"
+ y="14.24185"
+ ry="1.4985789"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-23.388845"
+ y="104.52142"
+ id="text290-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-11"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-23.388845"
+ y="104.52142">Model 0 / Start</tspan></text>
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.6329;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-6"
+ width="47.3671"
+ height="17.33604"
+ x="147.18958"
+ y="14.257105"
+ ry="1.4959463" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-23.388849"
+ y="170.86761"
+ id="text290-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-15"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-23.388849"
+ y="170.86761">Model 1 / Load</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 62.230572,22.925124 H 81.026939"
+ id="path1916"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234"
+ inkscape:connection-end="#rect234-07" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 128.39349,22.925124 h 18.79609"
+ id="path1918"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-07"
+ inkscape:connection-end="#rect234-6" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 194.55667,22.925125 18.79581,0"
+ id="path1922"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-6"
+ inkscape:connection-end="#rect234-0-6-6" />
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6"
+ width="47.3671"
+ height="17.33604"
+ x="213.35248"
+ y="14.257105"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-23.388849"
+ y="236.84723"
+ id="text290-3-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-23.388849"
+ y="236.84723">Model 1 / Start</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 237.03603,31.593145 0,5.328675 H 38.547297 v 5.313421"
+ id="path1924"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6"
+ inkscape:connection-end="#rect234-0-6-6-3" />
+ <rect
+ style="fill:#fff6d5;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-3"
+ width="47.3671"
+ height="17.33604"
+ x="14.863747"
+ y="42.235241"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-51.366989"
+ y="38.541786"
+ id="text290-3-2-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-51.366989"
+ y="38.541786">Model N / Load</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-2"
+ width="47.3671"
+ height="17.33604"
+ x="81.026657"
+ y="42.235241"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-51.366989"
+ y="104.52142"
+ id="text290-3-2-2-28"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-2"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-51.366989"
+ y="104.52142">Model N / Start</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 62.230846,50.903261 H 81.026657"
+ id="path1926"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-3"
+ inkscape:connection-end="#rect234-0-6-6-2" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 128.39376,50.903261 h 42.47937 v 19.294584"
+ id="path1928"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-2"
+ inkscape:connection-end="#rect234-0-6-1" />
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6"
+ width="47.3671"
+ height="17.33604"
+ x="213.35248"
+ y="70.197845"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-79.32959"
+ y="237.03052"
+ id="text290-3-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-79.32959"
+ y="237.03052">Model N / Unload</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-1"
+ width="47.3671"
+ height="17.33604"
+ x="147.18958"
+ y="70.197845"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-78.803177"
+ y="170.77666"
+ id="text290-3-2-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-8"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-78.803177"
+ y="170.77666">Model N / Stop</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 194.55667,78.865866 h 18.79581"
+ id="path1930"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6"
+ inkscape:connection-start="#rect234-0-6-1" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 237.03603,87.533886 v 5.313417 H 38.547297 v 5.328677"
+ id="path1932"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6-7"
+ inkscape:connection-start="#rect234-0-6" />
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-7"
+ width="47.3671"
+ height="17.33604"
+ x="14.863747"
+ y="98.17598"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-106.78132"
+ y="38.450832"
+ id="text290-3-2-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-106.78132"
+ y="38.450832">Model 1 / Stop</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 62.230846,106.844 H 81.026657"
+ id="path1934"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0"
+ inkscape:connection-start="#rect234-0-6-7" />
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0"
+ width="47.3671"
+ height="17.33604"
+ x="81.026657"
+ y="98.17598"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-107.30773"
+ y="104.7047"
+ id="text290-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-107.30773"
+ y="104.7047">Model 1 / Unload</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-9"
+ width="47.3671"
+ height="17.33604"
+ x="147.18958"
+ y="98.17598"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-106.78133"
+ y="170.77666"
+ id="text290-3-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-106.78133"
+ y="170.77666">Model 0 / Stop</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.633452;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-5"
+ width="47.366547"
+ height="17.366549"
+ x="213.35277"
+ y="98.160721"
+ ry="1.4985789"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-107.30773"
+ y="237.03053"
+ id="text290-3-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-3"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-107.30773"
+ y="237.03053">Model 0 / Unload</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 128.39376,106.844 h 18.79582"
+ id="path1936"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-9"
+ inkscape:connection-start="#rect234-0" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 194.55667,106.844 9.39805,0 v 0 h 9.39805"
+ id="path1938"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-5"
+ inkscape:connection-start="#rect234-0-9" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#fff6d5;fill-rule:evenodd;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-122.00021"
+ y="222.26495"
+ id="text392"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan390"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';fill:#000000;stroke-width:0.5"
+ x="-122.00021"
+ y="222.26495">mldev: model_ops / subest B</tspan></text>
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.462341;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect3475"
+ width="259.53769"
+ height="119.53766"
+ x="7.7311554"
+ y="7.7311707"
+ ry="1.2186421" />
+ </g>
+</svg>
diff --git a/doc/guides/tools/index.rst b/doc/guides/tools/index.rst
index f1f5b94c8c..6f84fc31ff 100644
--- a/doc/guides/tools/index.rst
+++ b/doc/guides/tools/index.rst
@@ -21,4 +21,5 @@ DPDK Tools User Guides
comp_perf
testeventdev
testregex
+ testmldev
dts
diff --git a/doc/guides/tools/testmldev.rst b/doc/guides/tools/testmldev.rst
new file mode 100644
index 0000000000..845c2d9381
--- /dev/null
+++ b/doc/guides/tools/testmldev.rst
@@ -0,0 +1,441 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright (c) 2022 Marvell.
+
+dpdk-test-mldev Application
+===========================
+
+The ``dpdk-test-mldev`` tool is a Data Plane Development Kit (DPDK) application that allows testing
+various mldev use cases. This application has a generic framework to add new mldev based test cases
+to verify functionality and measure the performance of inference execution on DPDK ML devices.
+
+
+Application and Options
+-----------------------
+
+The application has a number of command line options:
+
+.. code-block:: console
+
+ dpdk-test-mldev [EAL Options] -- [application options]
+
+EAL Options
+~~~~~~~~~~~
+
+The following are the EAL command-line options that can be used with the ``dpdk-test-mldev``
+application. See the DPDK Getting Started Guides for more information on these options.
+
+* ``-c <COREMASK>`` or ``-l <CORELIST>``
+
+ Set the hexadecimal bitmask of the cores to run on. The corelist is a list of cores to use.
+
+* ``-a <PCI_ID>``
+
+ Attach a PCI based ML device. Specific to drivers using a PCI based ML devices.
+
+* ``--vdev <driver>``
+
+ Add a virtual mldev device. Specific to drivers using a ML virtual device.
+
+
+Application Options
+~~~~~~~~~~~~~~~~~~~
+
+The following are the command-line options supported by the test application.
+
+* ``--test <name>``
+
+ ML tests are divided into two groups, Model and Device tests and Inference tests. Test
+ name one of the following supported tests.
+
+ **ML Device Tests** ::
+
+ device_ops
+
+ **ML Model Tests** ::
+
+ model_ops
+
+ **ML Inference Tests** ::
+
+ inference_ordered
+ inference_interleave
+
+* ``--dev_id <n>``
+
+ Set the device id of the ML device to be used for the test. Default value is `0`.
+
+* ``--socket_id <n>``
+
+ Set the socket id of the application resources. Default value is `SOCKET_ID_ANY`.
+
+* ``--debug``
+
+ Enable the tests to run in debug mode.
+
+* ``--models <model_list>``
+
+ Set the list of model files to be used for the tests. Application expects the
+ ``model_list`` in comma separated form (i.e. ``--models model_A.bin,model_B.bin``).
+ Maximum number of models supported by the test is ``8``.
+
+* ``--filelist <file_list>``
+
+ Set the list of model, input, output and reference files to be used for the tests.
+ Application expects the ``file_list`` to be in comma separated form
+ (i.e. ``--filelist <model,input,output>[,reference]``).
+
+ Multiple filelist entries can be specified when running the tests with multiple models.
+ Both quantized and dequantized outputs are written to the disk. Dequantized output file
+ would have the name specified by the user through ``--filelist`` option. A suffix ``.q``
+ is appended to quantized output filename. Maximum number of filelist entries supported
+ by the test is ``8``.
+
+* ``--repetitions <n>``
+
+ Set the number of inference repetitions to be executed in the test per each model. Default
+ value is `1`.
+
+* ``--burst_size <n>``
+
+ Set the burst size to be used when enqueuing / dequeuing inferences. Default value is `1`.
+
+* ``--queue_pairs <n>``
+
+ Set the number of queue-pairs to be used for inference enqueue and dequeue operations.
+ Default value is `1`.
+
+* ``--queue_size <n>``
+
+ Set the size of queue-pair to be created for inference enqueue / dequeue operations.
+ Queue size would translate into `rte_ml_dev_qp_conf::nb_desc` field during queue-pair
+ creation. Default value is `1`.
+
+* ``--batches <n>``
+
+ Set the number batches in the input file provided for inference run. When not specified
+ the test would assume the number of batches is equal to the batch size of the model.
+
+* ``--tolerance <n>``
+
+ Set the tolerance value in percentage to be used for output validation. Default value
+ is `0`.
+
+* ``--stats``
+
+ Enable reporting device extended stats.
+
+
+ML Device Tests
+-------------------------
+
+ML device tests are functional tests to validate ML device APIs. Device tests validate the ML device
+handling APIs configure, close, start and stop APIs.
+
+
+Application Options
+~~~~~~~~~~~~~~~~~~~
+
+Supported command line options for the `model_ops` test are following::
+
+ --debug
+ --test
+ --dev_id
+ --socket_id
+ --queue_pairs
+ --queue_size
+
+
+DEVICE_OPS Test
+~~~~~~~~~~~~~~~
+
+Device ops test validates the device configuration and reconfiguration support. The test configures
+ML device based on the option ``--queue_pairs`` and ``--queue_size`` specified by the user, and
+later reconfigures the ML device with the number of queue pairs and queue size based the maximum
+specified through the device info.
+
+
+Example
+^^^^^^^
+
+Command to run device_ops test:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=device_ops
+
+
+Command to run device_ops test with user options:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=device_ops --queue_pairs <M> --queue_size <N>
+
+
+ML Model Tests
+-------------------------
+
+Model tests are functional tests to validate ML model APIs. Model tests validate the functioning
+of APIs to load, start, stop and unload ML models.
+
+
+Application Options
+~~~~~~~~~~~~~~~~~~~
+
+Supported command line options for the `model_ops` test are following::
+
+ --debug
+ --test
+ --dev_id
+ --socket_id
+ --models
+
+
+List of model files to be used for the `model_ops` test can be specified through the option
+``--models <model_list>`` as a comma separated list. Maximum number of models supported in
+the test is `8`.
+
+.. Note::
+
+ * The ``--models <model_list>`` is a mandatory option for running this test.
+ * Options not supported by the test are ignored if specified.
+
+
+MODEL_OPS Test
+~~~~~~~~~~~~~~
+
+The test is a collection of multiple sub-tests, each with a different order of slow-path
+operations when handling with `N` number of models.
+
+
+**Sub-test A:** executes the sequence of load / start / stop / unload for a model in order,
+followed by next model.
+.. _figure_mldev_model_ops_subtest_a:
+
+.. figure:: img/mldev_model_ops_subtest_a.*
+
+ Execution sequence of model_ops subtest A.
+
+
+**Sub-test B:** executes load for all models, followed by a start for all models. Upon successful
+start of all models, stop is invoked for all models followed by unload.
+.. _figure_mldev_model_ops_subtest_b:
+
+.. figure:: img/mldev_model_ops_subtest_b.*
+
+ Execution sequence of model_ops subtest B.
+
+
+**Sub-test C:** loads all models, followed by a start and stop of all models in order. Upon
+completion of stop, unload is invoked for all models.
+.. _figure_mldev_model_ops_subtest_c:
+
+.. figure:: img/mldev_model_ops_subtest_c.*
+
+ Execution sequence of model_ops subtest C.
+
+
+**Sub-test D:** executes load and start for all models available. Upon successful start of all
+models, stop and stop is executed for the models.
+.. _figure_mldev_model_ops_subtest_d:
+
+.. figure:: img/mldev_model_ops_subtest_d.*
+
+ Execution sequence of model_ops subtest D.
+
+
+Example
+^^^^^^^
+
+Command to run model_ops test:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=model_ops --models model_1.bin,model_2.bin,model_3.bin, model_4.bin
+
+
+ML Inference Tests
+------------------
+
+Inference tests are a set of tests to validate end-to-end inference execution on ML device.
+These tests executes the full sequence of operations required to run inferences with one or
+multiple models.
+
+Application Options
+~~~~~~~~~~~~~~~~~~~
+
+Supported command line options for inference tests are following::
+
+ --debug
+ --test
+ --dev_id
+ --socket_id
+ --filelist
+ --repetitions
+ --burst_size
+ --queue_pairs
+ --queue_size
+ --batches
+ --tolerance
+ --stats
+
+
+List of files to be used for the inference tests can be specified through the option
+``--filelist <file_list>`` as a comma separated list. A filelist entry would be of the format
+``--filelist <model_file,input_file,output_file>[,reference_file]`` and is used to specify the
+list of files required to test with a single model. Multiple filelist entries are supported by
+the test, one entry per model. Maximum number of file entries supported by the test is `8`.
+
+When ``--burst_size <num>`` option is specified for the test, enqueue and dequeue burst would
+try to enqueue or dequeue ``num`` number of inferences per each call respectively.
+
+In the inference test, a pair of lcores are mapped to each queue pair. Minimum number of lcores
+required for the tests is equal to ``(queue_pairs * 2 + 1)``.
+
+Output validation of inference would be enabled only when a reference file is specified through
+the ``--filelist`` option. Application would additionally consider the tolerance value provided
+through ``--tolerance`` option during validation. When the tolerance values is 0, CRC32 hash of
+inference output and reference output are compared. When the tolerance is non-zero, element wise
+comparison of output is performed. Validation is considered as successful only when all the
+elements of the output tensor are with in the tolerance range specified.
+
+When ``--debug`` option is specified, tests are run in debug mode.
+
+Enabling ``--stats`` would print the extended stats supported by the driver.
+
+.. Note::
+
+ * The ``--filelist <file_list>`` is a mandatory option for running inference tests.
+ * Options not supported by the tests are ignored if specified.
+ * Element wise comparison is not supported when the output dtype is either fp8, fp16
+ or bfloat16. This is applicable only when the tolerance is greater than zero and for
+ pre-quantized models only.
+
+
+INFERENCE_ORDERED Test
+~~~~~~~~~~~~~~~~~~~~~~
+
+This is a functional test for validating the end-to-end inference execution on ML device. This
+test configures ML device and queue pairs as per the queue-pair related options (queue_pairs and
+queue_size) specified by the user. Upon successful configuration of the device and queue pairs,
+the first model specified through the filelist is loaded to the device and inferences are enqueued
+by a pool of worker threads to the ML device. Total number of inferences enqueued for the model
+are equal to the repetitions specified. A dedicated pool of worker threads would dequeue the
+inferences from the device. The model is unloaded upon completion of all inferences for the model.
+The test would continue loading and executing inference requests for all models specified
+through ``filelist`` option in an ordered manner.
+
+.. _figure_mldev_inference_ordered:
+
+.. figure:: img/mldev_inference_ordered.*
+
+ Execution of inference_ordered on single model.
+
+
+Example
+^^^^^^^
+
+Example command to run inference_ordered test:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_ordered --filelist model.bin,input.bin,output.bin
+
+Example command to run inference_ordered with output validation using tolerance of `1%``:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_ordered --filelist model.bin,input.bin,output.bin,reference.bin \
+ --tolerance 1.0
+
+Example command to run inference_ordered test with multiple queue-pairs and queue size:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_ordered --filelist model.bin,input.bin,output.bin \
+ --queue_pairs 4 --queue_size 16
+
+Example command to run inference_ordered test with a specific burst size:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_ordered --filelist model.bin,input.bin,output.bin \
+ --burst_size 12
+
+
+INFERENCE_INTERLEAVE Test
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This is a stress test for validating the end-to-end inference execution on ML device. The test
+configures the ML device and queue pairs as per the queue-pair related options (queue_pairs
+and queue_size) specified by the user. Upon successful configuration of the device and queue
+pairs, all models specified through the filelist are loaded to the device. Inferences for multiple
+models are enqueued by a pool of worker threads in parallel. Inference execution by the device is
+interleaved between multiple models. Total number of inferences enqueued for a model are equal to
+the repetitions specified. An additional pool of threads would dequeue the inferences from the
+device. Models would be unloaded upon completion of inferences for all models loaded.
+
+
+.. _figure_mldev_inference_interleave:
+
+.. figure:: img/mldev_inference_interleave.*
+
+ Execution of inference_interleave on single model.
+
+
+Example
+^^^^^^^
+
+Example command to run inference_interleave test:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_interleave --filelist model.bin,input.bin,output.bin
+
+
+Example command to run inference_interleave test with multiple models:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_interleave --filelist model_A.bin,input_A.bin,output_A.bin \
+ --filelist model_B.bin,input_B.bin,output_B.bin
+
+
+Example command to run inference_interleave test with multiple models ad output validation
+using tolerance of `2.0%``:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_interleave \
+ --filelist model_A.bin,input_A.bin,output_A.bin,reference_A.bin \
+ --filelist model_B.bin,input_B.bin,output_B.bin,reference_B.bin \
+ --tolerance 2.0
+
+Example command to run inference_interleave test with multiple queue-pairs and queue size
+and burst size:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_interleave --filelist model.bin,input.bin,output.bin \
+ --queue_pairs 8 --queue_size 12 --burst_size 16
+
+
+Debug mode
+----------
+
+ML tests can be executed in debug mode by enabling the option ``--debug``. Execution of tests in
+debug mode would enable additional prints.
+
+When a validation failure is observed, output from that buffer is written to the disk, with the
+filenames having similar convention when the test has passed. Additionally index of the buffer
+would be appended to the filenames.
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread
* [PATCH v6 00/12] Implementation of mldev test application
2022-11-29 7:07 [PATCH v1 00/12] implement mldev test application Srikanth Yalavarthi
` (13 preceding siblings ...)
2023-03-10 8:09 ` [PATCH v5 00/12] Implementation of mldev test application Srikanth Yalavarthi
@ 2023-03-11 15:08 ` Srikanth Yalavarthi
2023-03-11 15:08 ` [PATCH v6 01/12] app/mldev: implement test framework for mldev Srikanth Yalavarthi
` (11 more replies)
2023-03-16 21:14 ` [PATCH v7 00/11] Implementation of mldev test application Srikanth Yalavarthi
15 siblings, 12 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-11 15:08 UTC (permalink / raw)
Cc: dev, syalavarthi, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Machine learning device APIs test application
=============================================
This series of patches introduces a test application for machine
learning device APIs. A test framework is implemented with multiple
test enabled, to validate the device, model and fast-path functions.
New tests can be added using the test framework.
List of tests supported
-----------------------
1) device_ops: Test case to validate device re-configuration
2) model_ops: Collection of 4 sub-tests to validate model slow APIs.
Each sub-test would invoke the slow path model APIs (load / start /
stop / unload) in different order.
3) inference_ordered: Test case to validate execution of end-to-end
inferences on ML device with one active model at a time. This test
can execute inference requests for multiple models, with inferences
for a model executed after completion of inferences for a previously
loaded model.
4) inference_interleave: Test case to validate end-to-end inferences
with multiple active models concurrently. This case would work as a
stress test to validate ML device.
Options supported the by tests include burst size for enqueuing and
dequeuing inference requests, enabling multiple queue pairs with a
user specified value for queue size. Support is also enabled for
batch inferencing, output validation and statistics.
v6:
* Enable error count
* Fix segfault due to incorrect i/o size types
* Revert i/o size variables to uint64_t
v5:
* Cleanup of header includes
* Addressed issues with unclean patches
* Changed input/output size variables to uint32_t
* Rebase over main branch
v4:
* Updated model_id as uint16_t
* Updated license info in SVG files
* Updated release notes
v3:
* Code rebase
Srikanth Yalavarthi (12):
app/mldev: implement test framework for mldev
app/mldev: add common test functions
app/mldev: add test case to validate device ops
app/mldev: add test case to validate model ops
app/mldev: add ordered inference test case
app/mldev: add test case to interleave inferences
app/mldev: enable support for burst inferences
app/mldev: enable support for queue pairs and size
app/mldev: enable support for inference batches
app/mldev: enable support for inference validation
app/mldev: enable reporting stats in mldev app
app/mldev: add documentation for mldev test cases
.mailmap | 1 +
MAINTAINERS | 2 +
app/meson.build | 1 +
app/test-mldev/meson.build | 24 +
app/test-mldev/ml_common.h | 29 +
app/test-mldev/ml_main.c | 113 ++
app/test-mldev/ml_options.c | 325 +++++
app/test-mldev/ml_options.h | 57 +
app/test-mldev/ml_test.c | 41 +
app/test-mldev/ml_test.h | 76 ++
app/test-mldev/parser.c | 380 ++++++
app/test-mldev/parser.h | 55 +
app/test-mldev/test_common.c | 136 ++
app/test-mldev/test_common.h | 27 +
app/test-mldev/test_device_ops.c | 228 ++++
app/test-mldev/test_device_ops.h | 17 +
app/test-mldev/test_inference_common.c | 1128 +++++++++++++++++
app/test-mldev/test_inference_common.h | 75 ++
app/test-mldev/test_inference_interleave.c | 118 ++
app/test-mldev/test_inference_ordered.c | 116 ++
app/test-mldev/test_model_common.c | 164 +++
app/test-mldev/test_model_common.h | 46 +
app/test-mldev/test_model_ops.c | 428 +++++++
app/test-mldev/test_model_ops.h | 20 +
doc/guides/rel_notes/release_23_03.rst | 8 +
.../tools/img/mldev_inference_interleave.svg | 669 ++++++++++
.../tools/img/mldev_inference_ordered.svg | 528 ++++++++
.../tools/img/mldev_model_ops_subtest_a.svg | 420 ++++++
.../tools/img/mldev_model_ops_subtest_b.svg | 423 +++++++
.../tools/img/mldev_model_ops_subtest_c.svg | 366 ++++++
.../tools/img/mldev_model_ops_subtest_d.svg | 424 +++++++
doc/guides/tools/index.rst | 1 +
doc/guides/tools/testmldev.rst | 441 +++++++
33 files changed, 6887 insertions(+)
create mode 100644 app/test-mldev/meson.build
create mode 100644 app/test-mldev/ml_common.h
create mode 100644 app/test-mldev/ml_main.c
create mode 100644 app/test-mldev/ml_options.c
create mode 100644 app/test-mldev/ml_options.h
create mode 100644 app/test-mldev/ml_test.c
create mode 100644 app/test-mldev/ml_test.h
create mode 100644 app/test-mldev/parser.c
create mode 100644 app/test-mldev/parser.h
create mode 100644 app/test-mldev/test_common.c
create mode 100644 app/test-mldev/test_common.h
create mode 100644 app/test-mldev/test_device_ops.c
create mode 100644 app/test-mldev/test_device_ops.h
create mode 100644 app/test-mldev/test_inference_common.c
create mode 100644 app/test-mldev/test_inference_common.h
create mode 100644 app/test-mldev/test_inference_interleave.c
create mode 100644 app/test-mldev/test_inference_ordered.c
create mode 100644 app/test-mldev/test_model_common.c
create mode 100644 app/test-mldev/test_model_common.h
create mode 100644 app/test-mldev/test_model_ops.c
create mode 100644 app/test-mldev/test_model_ops.h
create mode 100644 doc/guides/tools/img/mldev_inference_interleave.svg
create mode 100644 doc/guides/tools/img/mldev_inference_ordered.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_a.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_b.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_c.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_d.svg
create mode 100644 doc/guides/tools/testmldev.rst
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v6 01/12] app/mldev: implement test framework for mldev
2023-03-11 15:08 ` [PATCH v6 00/12] Implementation of mldev test application Srikanth Yalavarthi
@ 2023-03-11 15:08 ` Srikanth Yalavarthi
2023-03-11 15:08 ` [PATCH v6 02/12] app/mldev: add common test functions Srikanth Yalavarthi
` (10 subsequent siblings)
11 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-11 15:08 UTC (permalink / raw)
To: Thomas Monjalon, Srikanth Yalavarthi
Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Implemented framework for mldev test application. New test cases
can be added using the framework. Support is also enabled to add
options specific to the test cases. User can launch the tests by
specifying the name of test as part of launch arguments.
Code to parse command line arguments is imported from
test-eventdev, with support to parse additional data types.
Common arguments supported include:
test : name of the test application to run
dev_id : device id of the ML device
socket_id : socket_id of application resources
debug : enable debugging
help : print help
Sample launch command:
./dpdk-test-mldev -- --test <testname> --dev_id <dev_id> \
--socket_id <socket_id>
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
Acked-by: Shivah Shankar S <sshankarnara@marvell.com>
Acked-by: Anup Prabhu <aprabhu@marvell.com>
---
.mailmap | 1 +
MAINTAINERS | 1 +
app/meson.build | 1 +
app/test-mldev/meson.build | 17 ++
app/test-mldev/ml_common.h | 29 ++
app/test-mldev/ml_main.c | 113 ++++++++
app/test-mldev/ml_options.c | 155 ++++++++++
app/test-mldev/ml_options.h | 31 ++
app/test-mldev/ml_test.c | 41 +++
app/test-mldev/ml_test.h | 76 +++++
app/test-mldev/parser.c | 380 +++++++++++++++++++++++++
app/test-mldev/parser.h | 55 ++++
doc/guides/rel_notes/release_23_03.rst | 8 +
13 files changed, 908 insertions(+)
create mode 100644 app/test-mldev/meson.build
create mode 100644 app/test-mldev/ml_common.h
create mode 100644 app/test-mldev/ml_main.c
create mode 100644 app/test-mldev/ml_options.c
create mode 100644 app/test-mldev/ml_options.h
create mode 100644 app/test-mldev/ml_test.c
create mode 100644 app/test-mldev/ml_test.h
create mode 100644 app/test-mldev/parser.c
create mode 100644 app/test-mldev/parser.h
diff --git a/.mailmap b/.mailmap
index f3303ca6ec..61d0d94d16 100644
--- a/.mailmap
+++ b/.mailmap
@@ -112,6 +112,7 @@ Anoob Joseph <anoobj@marvell.com> <anoob.joseph@caviumnetworks.com>
Antara Ganesh Kolar <antara.ganesh.kolar@intel.com>
Anthony Fee <anthonyx.fee@intel.com>
Antonio Fischetti <antonio.fischetti@intel.com>
+Anup Prabhu <aprabhu@marvell.com>
Anupam Kapoor <anupam.kapoor@gmail.com>
Apeksha Gupta <apeksha.gupta@nxp.com>
Archana Muniganti <marchana@marvell.com> <muniganti.archana@caviumnetworks.com>
diff --git a/MAINTAINERS b/MAINTAINERS
index d99eec0298..b0fd46ef02 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -480,6 +480,7 @@ Machine Learning device API - EXPERIMENTAL
M: Srikanth Yalavarthi <syalavarthi@marvell.com>
F: lib/mldev/
F: doc/guides/prog_guide/mldev.rst
+F: app/test-mldev
DMA device API - EXPERIMENTAL
M: Chengwen Feng <fengchengwen@huawei.com>
diff --git a/app/meson.build b/app/meson.build
index e32ea4bd5c..74d2420f67 100644
--- a/app/meson.build
+++ b/app/meson.build
@@ -23,6 +23,7 @@ apps = [
'test-fib',
'test-flow-perf',
'test-gpudev',
+ 'test-mldev',
'test-pipeline',
'test-pmd',
'test-regex',
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
new file mode 100644
index 0000000000..8ca2e1a1c1
--- /dev/null
+++ b/app/test-mldev/meson.build
@@ -0,0 +1,17 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright (c) 2022 Marvell.
+
+if is_windows
+ build = false
+ reason = 'not supported on Windows'
+ subdir_done()
+endif
+
+sources = files(
+ 'ml_main.c',
+ 'ml_options.c',
+ 'ml_test.c',
+ 'parser.c',
+)
+
+deps += ['mldev']
diff --git a/app/test-mldev/ml_common.h b/app/test-mldev/ml_common.h
new file mode 100644
index 0000000000..065180b619
--- /dev/null
+++ b/app/test-mldev/ml_common.h
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_COMMON_
+#define _ML_COMMON_
+
+#include <stdio.h>
+
+#define CLNRM "\x1b[0m"
+#define CLRED "\x1b[31m"
+#define CLGRN "\x1b[32m"
+#define CLYEL "\x1b[33m"
+
+#define ML_STR_FMT 20
+
+#define ml_err(fmt, args...) fprintf(stderr, CLRED "error: %s() " fmt CLNRM "\n", __func__, ##args)
+
+#define ml_info(fmt, args...) fprintf(stdout, CLYEL "" fmt CLNRM "\n", ##args)
+
+#define ml_dump(str, fmt, val...) printf("\t%-*s : " fmt "\n", ML_STR_FMT, str, ##val)
+
+#define ml_dump_begin(str) printf("\t%-*s :\n\t{\n", ML_STR_FMT, str)
+
+#define ml_dump_list(str, id, val) printf("\t%*s[%2u] : %s\n", ML_STR_FMT - 4, str, id, val)
+
+#define ml_dump_end printf("\b\t}\n\n")
+
+#endif /* _ML_COMMON_*/
diff --git a/app/test-mldev/ml_main.c b/app/test-mldev/ml_main.c
new file mode 100644
index 0000000000..940e609c9a
--- /dev/null
+++ b/app/test-mldev/ml_main.c
@@ -0,0 +1,113 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <rte_debug.h>
+#include <rte_eal.h>
+#include <rte_mldev.h>
+
+#include "ml_common.h"
+#include "ml_test.h"
+
+struct ml_options opt;
+struct ml_test *test;
+
+int
+main(int argc, char **argv)
+{
+ uint16_t mldevs;
+ int ret;
+
+ ret = rte_eal_init(argc, argv);
+ if (ret < 0)
+ rte_panic("invalid EAL arguments\n");
+ argc -= ret;
+ argv += ret;
+
+ mldevs = rte_ml_dev_count();
+ if (!mldevs)
+ rte_panic("no mldev devices found\n");
+
+ /* set default values for options */
+ ml_options_default(&opt);
+
+ /* parse the command line arguments */
+ ret = ml_options_parse(&opt, argc, argv);
+ if (ret) {
+ ml_err("parsing one or more user options failed");
+ goto error;
+ }
+
+ /* get test struct from name */
+ test = ml_test_get(opt.test_name);
+ if (test == NULL) {
+ ml_err("failed to find requested test: %s", opt.test_name);
+ goto error;
+ }
+
+ if (test->ops.test_result == NULL) {
+ ml_err("%s: ops.test_result not found", opt.test_name);
+ goto error;
+ }
+
+ /* check test options */
+ if (test->ops.opt_check) {
+ if (test->ops.opt_check(&opt)) {
+ ml_err("invalid command line argument");
+ goto error;
+ }
+ }
+
+ /* check the device capability */
+ if (test->ops.cap_check) {
+ if (test->ops.cap_check(&opt) == false) {
+ ml_info("unsupported test: %s", opt.test_name);
+ ret = ML_TEST_UNSUPPORTED;
+ goto no_cap;
+ }
+ }
+
+ /* dump options */
+ if (opt.debug) {
+ if (test->ops.opt_dump)
+ test->ops.opt_dump(&opt);
+ }
+
+ /* test specific setup */
+ if (test->ops.test_setup) {
+ if (test->ops.test_setup(test, &opt)) {
+ ml_err("failed to setup test: %s", opt.test_name);
+ goto error;
+ }
+ }
+
+ /* test driver */
+ if (test->ops.test_driver)
+ test->ops.test_driver(test, &opt);
+
+ /* get result */
+ if (test->ops.test_result)
+ ret = test->ops.test_result(test, &opt);
+
+ if (test->ops.test_destroy)
+ test->ops.test_destroy(test, &opt);
+
+no_cap:
+ if (ret == ML_TEST_SUCCESS) {
+ printf("Result: " CLGRN "%s" CLNRM "\n", "Success");
+ } else if (ret == ML_TEST_FAILED) {
+ printf("Result: " CLRED "%s" CLNRM "\n", "Failed");
+ return EXIT_FAILURE;
+ } else if (ret == ML_TEST_UNSUPPORTED) {
+ printf("Result: " CLYEL "%s" CLNRM "\n", "Unsupported");
+ }
+
+ rte_eal_cleanup();
+
+ return 0;
+
+error:
+ rte_eal_cleanup();
+
+ return EXIT_FAILURE;
+}
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
new file mode 100644
index 0000000000..521e9a2f97
--- /dev/null
+++ b/app/test-mldev/ml_options.c
@@ -0,0 +1,155 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <errno.h>
+#include <getopt.h>
+
+#include <rte_memory.h>
+#include <rte_mldev.h>
+#include <rte_string_fns.h>
+
+#include "ml_common.h"
+#include "ml_test.h"
+#include "parser.h"
+
+typedef int (*option_parser_t)(struct ml_options *opt, const char *arg);
+
+void
+ml_options_default(struct ml_options *opt)
+{
+ memset(opt, 0, sizeof(*opt));
+ strlcpy(opt->test_name, "ml_test", ML_TEST_NAME_MAX_LEN);
+ opt->dev_id = 0;
+ opt->socket_id = SOCKET_ID_ANY;
+ opt->debug = false;
+}
+
+struct long_opt_parser {
+ const char *lgopt_name;
+ option_parser_t parser_fn;
+};
+
+static int
+ml_parse_test_name(struct ml_options *opt, const char *arg)
+{
+ strlcpy(opt->test_name, arg, ML_TEST_NAME_MAX_LEN);
+ return 0;
+}
+
+static int
+ml_parse_dev_id(struct ml_options *opt, const char *arg)
+{
+ int ret;
+
+ ret = parser_read_int16(&opt->dev_id, arg);
+
+ if (ret < 0)
+ return -EINVAL;
+
+ return ret;
+}
+
+static int
+ml_parse_socket_id(struct ml_options *opt, const char *arg)
+{
+ opt->socket_id = atoi(arg);
+
+ return 0;
+}
+
+static void
+ml_dump_test_options(const char *testname)
+{
+ RTE_SET_USED(testname);
+}
+
+static void
+print_usage(char *program)
+{
+ printf("\nusage : %s [EAL options] -- [application options]\n", program);
+ printf("application options:\n");
+ printf("\t--test : name of the test application to run\n"
+ "\t--dev_id : device id of the ML device\n"
+ "\t--socket_id : socket_id of application resources\n"
+ "\t--debug : enable debug mode\n"
+ "\t--help : print help\n");
+ printf("\n");
+ printf("available tests and test specific application options:\n");
+ ml_test_dump_names(ml_dump_test_options);
+}
+
+static struct option lgopts[] = {{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0},
+ {ML_SOCKET_ID, 1, 0, 0}, {ML_DEBUG, 0, 0, 0},
+ {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
+
+static int
+ml_opts_parse_long(int opt_idx, struct ml_options *opt)
+{
+ unsigned int i;
+
+ struct long_opt_parser parsermap[] = {
+ {ML_TEST, ml_parse_test_name},
+ {ML_DEVICE_ID, ml_parse_dev_id},
+ {ML_SOCKET_ID, ml_parse_socket_id},
+ };
+
+ for (i = 0; i < RTE_DIM(parsermap); i++) {
+ if (strncmp(lgopts[opt_idx].name, parsermap[i].lgopt_name,
+ strlen(lgopts[opt_idx].name)) == 0)
+ return parsermap[i].parser_fn(opt, optarg);
+ }
+
+ return -EINVAL;
+}
+
+int
+ml_options_parse(struct ml_options *opt, int argc, char **argv)
+{
+ int opt_idx;
+ int retval;
+ int opts;
+
+ while ((opts = getopt_long(argc, argv, "", lgopts, &opt_idx)) != EOF) {
+ switch (opts) {
+ case 0: /* parse long options */
+ if (!strcmp(lgopts[opt_idx].name, "debug")) {
+ opt->debug = true;
+ break;
+ }
+
+ if (!strcmp(lgopts[opt_idx].name, "help")) {
+ print_usage(argv[0]);
+ exit(EXIT_SUCCESS);
+ }
+
+ retval = ml_opts_parse_long(opt_idx, opt);
+ if (retval != 0)
+ return retval;
+ break;
+ default:
+ return -EINVAL;
+ }
+ }
+
+ return 0;
+}
+
+void
+ml_options_dump(struct ml_options *opt)
+{
+ struct rte_ml_dev_info dev_info;
+
+ rte_ml_dev_info_get(opt->dev_id, &dev_info);
+
+ ml_dump("driver", "%s", dev_info.driver_name);
+ ml_dump("test", "%s", opt->test_name);
+ ml_dump("dev_id", "%d", opt->dev_id);
+
+ if (opt->socket_id == SOCKET_ID_ANY)
+ ml_dump("socket_id", "%d (SOCKET_ID_ANY)", opt->socket_id);
+ else
+ ml_dump("socket_id", "%d", opt->socket_id);
+
+ ml_dump("debug", "%s", (opt->debug ? "true" : "false"));
+}
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
new file mode 100644
index 0000000000..05311a9a47
--- /dev/null
+++ b/app/test-mldev/ml_options.h
@@ -0,0 +1,31 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_OPTIONS_
+#define _ML_OPTIONS_
+
+#include <stdbool.h>
+#include <stdint.h>
+
+#define ML_TEST_NAME_MAX_LEN 32
+
+/* Options names */
+#define ML_TEST ("test")
+#define ML_DEVICE_ID ("dev_id")
+#define ML_SOCKET_ID ("socket_id")
+#define ML_DEBUG ("debug")
+#define ML_HELP ("help")
+
+struct ml_options {
+ char test_name[ML_TEST_NAME_MAX_LEN];
+ int16_t dev_id;
+ int socket_id;
+ bool debug;
+};
+
+void ml_options_default(struct ml_options *opt);
+int ml_options_parse(struct ml_options *opt, int argc, char **argv);
+void ml_options_dump(struct ml_options *opt);
+
+#endif /* _ML_OPTIONS_ */
diff --git a/app/test-mldev/ml_test.c b/app/test-mldev/ml_test.c
new file mode 100644
index 0000000000..dcdeabe0bd
--- /dev/null
+++ b/app/test-mldev/ml_test.c
@@ -0,0 +1,41 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include "ml_test.h"
+
+static STAILQ_HEAD(, ml_test_entry) head = STAILQ_HEAD_INITIALIZER(head);
+
+void
+ml_test_register(struct ml_test_entry *entry)
+{
+ STAILQ_INSERT_TAIL(&head, entry, next);
+}
+
+struct ml_test *
+ml_test_get(const char *name)
+{
+ struct ml_test_entry *entry;
+
+ if (!name)
+ return NULL;
+
+ STAILQ_FOREACH(entry, &head, next)
+ if (!strncmp(entry->test.name, name, strlen(name)))
+ return &entry->test;
+
+ return NULL;
+}
+
+void
+ml_test_dump_names(void (*f)(const char *name))
+{
+ struct ml_test_entry *entry;
+
+ STAILQ_FOREACH(entry, &head, next)
+ {
+ if (entry->test.name)
+ printf("\t %s\n", entry->test.name);
+ f(entry->test.name);
+ }
+}
diff --git a/app/test-mldev/ml_test.h b/app/test-mldev/ml_test.h
new file mode 100644
index 0000000000..f7e6bf2f44
--- /dev/null
+++ b/app/test-mldev/ml_test.h
@@ -0,0 +1,76 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_TEST_
+#define _ML_TEST_
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/queue.h>
+
+#include <rte_common.h>
+
+#include "ml_options.h"
+
+#define ML_TEST_MAX_POOL_SIZE 256
+
+enum ml_test_result {
+ ML_TEST_SUCCESS,
+ ML_TEST_FAILED,
+ ML_TEST_UNSUPPORTED,
+};
+
+struct ml_test;
+
+typedef bool (*ml_test_capability_check_t)(struct ml_options *opt);
+typedef int (*ml_test_options_check_t)(struct ml_options *opt);
+typedef void (*ml_test_options_dump_t)(struct ml_options *opt);
+typedef int (*ml_test_setup_t)(struct ml_test *test, struct ml_options *opt);
+typedef void (*ml_test_destroy_t)(struct ml_test *test, struct ml_options *opt);
+typedef int (*ml_test_driver_t)(struct ml_test *test, struct ml_options *opt);
+typedef int (*ml_test_result_t)(struct ml_test *test, struct ml_options *opt);
+
+struct ml_test_ops {
+ ml_test_capability_check_t cap_check;
+ ml_test_options_check_t opt_check;
+ ml_test_options_dump_t opt_dump;
+ ml_test_setup_t test_setup;
+ ml_test_destroy_t test_destroy;
+ ml_test_driver_t test_driver;
+ ml_test_result_t test_result;
+};
+
+struct ml_test {
+ const char *name;
+ void *test_priv;
+ struct ml_test_ops ops;
+};
+
+struct ml_test_entry {
+ struct ml_test test;
+
+ STAILQ_ENTRY(ml_test_entry) next;
+};
+
+static inline void *
+ml_test_priv(struct ml_test *test)
+{
+ return test->test_priv;
+}
+
+struct ml_test *ml_test_get(const char *name);
+void ml_test_register(struct ml_test_entry *test);
+void ml_test_dump_names(void (*f)(const char *));
+
+#define ML_TEST_REGISTER(nm) \
+ static struct ml_test_entry _ml_test_entry_##nm; \
+ RTE_INIT(ml_test_##nm) \
+ { \
+ _ml_test_entry_##nm.test.name = RTE_STR(nm); \
+ memcpy(&_ml_test_entry_##nm.test.ops, &nm, sizeof(struct ml_test_ops)); \
+ ml_test_register(&_ml_test_entry_##nm); \
+ }
+
+#endif /* _ML_TEST_ */
diff --git a/app/test-mldev/parser.c b/app/test-mldev/parser.c
new file mode 100644
index 0000000000..0b7fb63fe5
--- /dev/null
+++ b/app/test-mldev/parser.c
@@ -0,0 +1,380 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2016 Intel Corporation.
+ * Copyright (c) 2017 Cavium, Inc.
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <ctype.h>
+#include <errno.h>
+#include <stdbool.h>
+#include <stdint.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include <rte_common.h>
+
+#include "parser.h"
+
+static uint32_t
+get_hex_val(char c)
+{
+ switch (c) {
+ case '0':
+ case '1':
+ case '2':
+ case '3':
+ case '4':
+ case '5':
+ case '6':
+ case '7':
+ case '8':
+ case '9':
+ return c - '0';
+ case 'A':
+ case 'B':
+ case 'C':
+ case 'D':
+ case 'E':
+ case 'F':
+ return c - 'A' + 10;
+ case 'a':
+ case 'b':
+ case 'c':
+ case 'd':
+ case 'e':
+ case 'f':
+ return c - 'a' + 10;
+ default:
+ return 0;
+ }
+}
+
+int
+parser_read_arg_bool(const char *p)
+{
+ p = skip_white_spaces(p);
+ int result = -EINVAL;
+
+ if (((p[0] == 'y') && (p[1] == 'e') && (p[2] == 's')) ||
+ ((p[0] == 'Y') && (p[1] == 'E') && (p[2] == 'S'))) {
+ p += 3;
+ result = 1;
+ }
+
+ if (((p[0] == 'o') && (p[1] == 'n')) || ((p[0] == 'O') && (p[1] == 'N'))) {
+ p += 2;
+ result = 1;
+ }
+
+ if (((p[0] == 'n') && (p[1] == 'o')) || ((p[0] == 'N') && (p[1] == 'O'))) {
+ p += 2;
+ result = 0;
+ }
+
+ if (((p[0] == 'o') && (p[1] == 'f') && (p[2] == 'f')) ||
+ ((p[0] == 'O') && (p[1] == 'F') && (p[2] == 'F'))) {
+ p += 3;
+ result = 0;
+ }
+
+ p = skip_white_spaces(p);
+
+ if (p[0] != '\0')
+ return -EINVAL;
+
+ return result;
+}
+
+int
+parser_read_uint64(uint64_t *value, const char *p)
+{
+ char *next;
+ uint64_t val;
+
+ p = skip_white_spaces(p);
+ if (!isdigit(*p))
+ return -EINVAL;
+
+ val = strtoul(p, &next, 10);
+ if (p == next)
+ return -EINVAL;
+
+ p = next;
+ switch (*p) {
+ case 'T':
+ val *= 1024ULL;
+ /* fall through */
+ case 'G':
+ val *= 1024ULL;
+ /* fall through */
+ case 'M':
+ val *= 1024ULL;
+ /* fall through */
+ case 'k':
+ case 'K':
+ val *= 1024ULL;
+ p++;
+ break;
+ }
+
+ p = skip_white_spaces(p);
+ if (*p != '\0')
+ return -EINVAL;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_int32(int32_t *value, const char *p)
+{
+ char *next;
+ int32_t val;
+
+ p = skip_white_spaces(p);
+ if (!isdigit(*p))
+ return -EINVAL;
+
+ val = strtol(p, &next, 10);
+ if (p == next)
+ return -EINVAL;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_int16(int16_t *value, const char *p)
+{
+ char *next;
+ int16_t val;
+
+ p = skip_white_spaces(p);
+ if (!isdigit(*p))
+ return -EINVAL;
+
+ val = strtol(p, &next, 10);
+ if (p == next)
+ return -EINVAL;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint64_hex(uint64_t *value, const char *p)
+{
+ char *next;
+ uint64_t val;
+
+ p = skip_white_spaces(p);
+
+ val = strtoul(p, &next, 16);
+ if (p == next)
+ return -EINVAL;
+
+ p = skip_white_spaces(next);
+ if (*p != '\0')
+ return -EINVAL;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint32(uint32_t *value, const char *p)
+{
+ uint64_t val = 0;
+ int ret = parser_read_uint64(&val, p);
+
+ if (ret < 0)
+ return ret;
+
+ if (val > UINT32_MAX)
+ return -ERANGE;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint32_hex(uint32_t *value, const char *p)
+{
+ uint64_t val = 0;
+ int ret = parser_read_uint64_hex(&val, p);
+
+ if (ret < 0)
+ return ret;
+
+ if (val > UINT32_MAX)
+ return -ERANGE;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint16(uint16_t *value, const char *p)
+{
+ uint64_t val = 0;
+ int ret = parser_read_uint64(&val, p);
+
+ if (ret < 0)
+ return ret;
+
+ if (val > UINT16_MAX)
+ return -ERANGE;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint16_hex(uint16_t *value, const char *p)
+{
+ uint64_t val = 0;
+ int ret = parser_read_uint64_hex(&val, p);
+
+ if (ret < 0)
+ return ret;
+
+ if (val > UINT16_MAX)
+ return -ERANGE;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint8(uint8_t *value, const char *p)
+{
+ uint64_t val = 0;
+ int ret = parser_read_uint64(&val, p);
+
+ if (ret < 0)
+ return ret;
+
+ if (val > UINT8_MAX)
+ return -ERANGE;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint8_hex(uint8_t *value, const char *p)
+{
+ uint64_t val = 0;
+ int ret = parser_read_uint64_hex(&val, p);
+
+ if (ret < 0)
+ return ret;
+
+ if (val > UINT8_MAX)
+ return -ERANGE;
+
+ *value = val;
+ return 0;
+}
+
+int
+parse_tokenize_string(char *string, char *tokens[], uint32_t *n_tokens)
+{
+ uint32_t i;
+
+ if ((string == NULL) || (tokens == NULL) || (*n_tokens < 1))
+ return -EINVAL;
+
+ for (i = 0; i < *n_tokens; i++) {
+ tokens[i] = strtok_r(string, PARSE_DELIMITER, &string);
+ if (tokens[i] == NULL)
+ break;
+ }
+
+ if ((i == *n_tokens) && (strtok_r(string, PARSE_DELIMITER, &string) != NULL))
+ return -E2BIG;
+
+ *n_tokens = i;
+ return 0;
+}
+
+int
+parse_hex_string(char *src, uint8_t *dst, uint32_t *size)
+{
+ char *c;
+ uint32_t len, i;
+
+ /* Check input parameters */
+ if ((src == NULL) || (dst == NULL) || (size == NULL) || (*size == 0))
+ return -1;
+
+ len = strlen(src);
+ if (((len & 3) != 0) || (len > (*size) * 2))
+ return -1;
+ *size = len / 2;
+
+ for (c = src; *c != 0; c++) {
+ if ((((*c) >= '0') && ((*c) <= '9')) || (((*c) >= 'A') && ((*c) <= 'F')) ||
+ (((*c) >= 'a') && ((*c) <= 'f')))
+ continue;
+
+ return -1;
+ }
+
+ /* Convert chars to bytes */
+ for (i = 0; i < *size; i++)
+ dst[i] = get_hex_val(src[2 * i]) * 16 + get_hex_val(src[2 * i + 1]);
+
+ return 0;
+}
+
+int
+parse_lcores_list(bool lcores[], int lcores_num, const char *corelist)
+{
+ int i, idx = 0;
+ int min, max;
+ char *end = NULL;
+
+ if (corelist == NULL)
+ return -1;
+ while (isblank(*corelist))
+ corelist++;
+ i = strlen(corelist);
+ while ((i > 0) && isblank(corelist[i - 1]))
+ i--;
+
+ /* Get list of lcores */
+ min = RTE_MAX_LCORE;
+ do {
+ while (isblank(*corelist))
+ corelist++;
+ if (*corelist == '\0')
+ return -1;
+ idx = strtoul(corelist, &end, 10);
+ if (idx < 0 || idx > lcores_num)
+ return -1;
+
+ if (end == NULL)
+ return -1;
+ while (isblank(*end))
+ end++;
+ if (*end == '-') {
+ min = idx;
+ } else if ((*end == ',') || (*end == '\0')) {
+ max = idx;
+ if (min == RTE_MAX_LCORE)
+ min = idx;
+ for (idx = min; idx <= max; idx++) {
+ if (lcores[idx] == 1)
+ return -E2BIG;
+ lcores[idx] = 1;
+ }
+
+ min = RTE_MAX_LCORE;
+ } else
+ return -1;
+ corelist = end + 1;
+ } while (*end != '\0');
+
+ return 0;
+}
diff --git a/app/test-mldev/parser.h b/app/test-mldev/parser.h
new file mode 100644
index 0000000000..f0d5e79e4b
--- /dev/null
+++ b/app/test-mldev/parser.h
@@ -0,0 +1,55 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2010-2016 Intel Corporation.
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef __INCLUDE_PARSER_H__
+#define __INCLUDE_PARSER_H__
+
+#include <ctype.h>
+#include <stdint.h>
+#include <stdlib.h>
+
+#define PARSE_DELIMITER " \f\n\r\t\v"
+
+#define skip_white_spaces(pos) \
+ ({ \
+ __typeof__(pos) _p = (pos); \
+ for (; isspace(*_p); _p++) \
+ ; \
+ _p; \
+ })
+
+static inline size_t
+skip_digits(const char *src)
+{
+ size_t i;
+
+ for (i = 0; isdigit(src[i]); i++)
+ ;
+
+ return i;
+}
+
+int parser_read_arg_bool(const char *p);
+
+int parser_read_uint64(uint64_t *value, const char *p);
+int parser_read_uint32(uint32_t *value, const char *p);
+int parser_read_uint16(uint16_t *value, const char *p);
+int parser_read_uint8(uint8_t *value, const char *p);
+
+int parser_read_uint64_hex(uint64_t *value, const char *p);
+int parser_read_uint32_hex(uint32_t *value, const char *p);
+int parser_read_uint16_hex(uint16_t *value, const char *p);
+int parser_read_uint8_hex(uint8_t *value, const char *p);
+
+int parser_read_int32(int32_t *value, const char *p);
+int parser_read_int16(int16_t *value, const char *p);
+
+int parse_hex_string(char *src, uint8_t *dst, uint32_t *size);
+
+int parse_tokenize_string(char *string, char *tokens[], uint32_t *n_tokens);
+
+int parse_lcores_list(bool lcores[], int lcores_num, const char *corelist);
+
+#endif /* __INCLUDE_PARSER_H__ */
diff --git a/doc/guides/rel_notes/release_23_03.rst b/doc/guides/rel_notes/release_23_03.rst
index 17ffabbb8c..fec97a5dfb 100644
--- a/doc/guides/rel_notes/release_23_03.rst
+++ b/doc/guides/rel_notes/release_23_03.rst
@@ -226,6 +226,14 @@ New Features
* Added support to capture packets at each graph node with packet metadata and
node name.
+* **Added test application for machine learning inference device library.**
+
+ * Added test application for mldev library with support for multiple test cases.
+ * Test case for device operations.
+ * Test case for model operations.
+ * Inference test case using multiple models in ordered mode.
+ * Inference test case using multiple models in interleave mode.
+
Removed Items
-------------
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v6 02/12] app/mldev: add common test functions
2023-03-11 15:08 ` [PATCH v6 00/12] Implementation of mldev test application Srikanth Yalavarthi
2023-03-11 15:08 ` [PATCH v6 01/12] app/mldev: implement test framework for mldev Srikanth Yalavarthi
@ 2023-03-11 15:08 ` Srikanth Yalavarthi
2023-03-11 15:08 ` [PATCH v6 03/12] app/mldev: add test case to validate device ops Srikanth Yalavarthi
` (9 subsequent siblings)
11 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-11 15:08 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Added common functions used by all tests. Common code
includes functions to check capabilities, options, and
handle ML devices.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
Acked-by: Anup Prabhu <aprabhu@marvell.com>
---
app/test-mldev/meson.build | 1 +
app/test-mldev/test_common.c | 136 +++++++++++++++++++++++++++++++++++
app/test-mldev/test_common.h | 27 +++++++
3 files changed, 164 insertions(+)
create mode 100644 app/test-mldev/test_common.c
create mode 100644 app/test-mldev/test_common.h
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
index 8ca2e1a1c1..964bb9ddc4 100644
--- a/app/test-mldev/meson.build
+++ b/app/test-mldev/meson.build
@@ -12,6 +12,7 @@ sources = files(
'ml_options.c',
'ml_test.c',
'parser.c',
+ 'test_common.c',
)
deps += ['mldev']
diff --git a/app/test-mldev/test_common.c b/app/test-mldev/test_common.c
new file mode 100644
index 0000000000..8c4da4609a
--- /dev/null
+++ b/app/test-mldev/test_common.c
@@ -0,0 +1,136 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <errno.h>
+
+#include <rte_common.h>
+#include <rte_memory.h>
+#include <rte_mldev.h>
+
+#include "ml_common.h"
+#include "test_common.h"
+
+bool
+ml_test_cap_check(struct ml_options *opt)
+{
+ struct rte_ml_dev_info dev_info;
+
+ rte_ml_dev_info_get(opt->dev_id, &dev_info);
+ if (dev_info.max_models == 0) {
+ ml_err("Not enough mldev models supported = %u", dev_info.max_models);
+ return false;
+ }
+
+ return true;
+}
+
+int
+ml_test_opt_check(struct ml_options *opt)
+{
+ uint16_t dev_count;
+ int socket_id;
+
+ RTE_SET_USED(opt);
+
+ dev_count = rte_ml_dev_count();
+ if (dev_count == 0) {
+ ml_err("No ML devices found");
+ return -ENODEV;
+ }
+
+ if (opt->dev_id >= dev_count) {
+ ml_err("Invalid option dev_id = %d", opt->dev_id);
+ return -EINVAL;
+ }
+
+ socket_id = rte_ml_dev_socket_id(opt->dev_id);
+ if (!((opt->socket_id != SOCKET_ID_ANY) || (opt->socket_id != socket_id))) {
+ ml_err("Invalid option, socket_id = %d\n", opt->socket_id);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+void
+ml_test_opt_dump(struct ml_options *opt)
+{
+ ml_options_dump(opt);
+}
+
+int
+ml_test_device_configure(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_common *t = ml_test_priv(test);
+ struct rte_ml_dev_config dev_config;
+ int ret;
+
+ ret = rte_ml_dev_info_get(opt->dev_id, &t->dev_info);
+ if (ret != 0) {
+ ml_err("Failed to get mldev info, dev_id = %d\n", opt->dev_id);
+ return ret;
+ }
+
+ /* configure device */
+ dev_config.socket_id = opt->socket_id;
+ dev_config.nb_models = t->dev_info.max_models;
+ dev_config.nb_queue_pairs = t->dev_info.max_queue_pairs;
+ ret = rte_ml_dev_configure(opt->dev_id, &dev_config);
+ if (ret != 0) {
+ ml_err("Failed to configure ml device, dev_id = %d\n", opt->dev_id);
+ return ret;
+ }
+
+ return 0;
+}
+
+int
+ml_test_device_close(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_common *t = ml_test_priv(test);
+ int ret = 0;
+
+ RTE_SET_USED(t);
+
+ /* close device */
+ ret = rte_ml_dev_close(opt->dev_id);
+ if (ret != 0)
+ ml_err("Failed to close ML device, dev_id = %d\n", opt->dev_id);
+
+ return ret;
+}
+
+int
+ml_test_device_start(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_common *t = ml_test_priv(test);
+ int ret;
+
+ RTE_SET_USED(t);
+
+ /* start device */
+ ret = rte_ml_dev_start(opt->dev_id);
+ if (ret != 0) {
+ ml_err("Failed to start ml device, dev_id = %d\n", opt->dev_id);
+ return ret;
+ }
+
+ return 0;
+}
+
+int
+ml_test_device_stop(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_common *t = ml_test_priv(test);
+ int ret = 0;
+
+ RTE_SET_USED(t);
+
+ /* stop device */
+ ret = rte_ml_dev_stop(opt->dev_id);
+ if (ret != 0)
+ ml_err("Failed to stop ML device, dev_id = %d\n", opt->dev_id);
+
+ return ret;
+}
diff --git a/app/test-mldev/test_common.h b/app/test-mldev/test_common.h
new file mode 100644
index 0000000000..cb286adcd6
--- /dev/null
+++ b/app/test-mldev/test_common.h
@@ -0,0 +1,27 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_TEST_COMMON_
+#define _ML_TEST_COMMON_
+
+#include <rte_mldev.h>
+
+#include "ml_common.h"
+#include "ml_test.h"
+
+struct test_common {
+ struct ml_options *opt;
+ enum ml_test_result result;
+ struct rte_ml_dev_info dev_info;
+};
+
+bool ml_test_cap_check(struct ml_options *opt);
+int ml_test_opt_check(struct ml_options *opt);
+void ml_test_opt_dump(struct ml_options *opt);
+int ml_test_device_configure(struct ml_test *test, struct ml_options *opt);
+int ml_test_device_close(struct ml_test *test, struct ml_options *opt);
+int ml_test_device_start(struct ml_test *test, struct ml_options *opt);
+int ml_test_device_stop(struct ml_test *test, struct ml_options *opt);
+
+#endif /* _ML_TEST_COMMON_ */
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v6 03/12] app/mldev: add test case to validate device ops
2023-03-11 15:08 ` [PATCH v6 00/12] Implementation of mldev test application Srikanth Yalavarthi
2023-03-11 15:08 ` [PATCH v6 01/12] app/mldev: implement test framework for mldev Srikanth Yalavarthi
2023-03-11 15:08 ` [PATCH v6 02/12] app/mldev: add common test functions Srikanth Yalavarthi
@ 2023-03-11 15:08 ` Srikanth Yalavarthi
2023-03-11 15:08 ` [PATCH v6 04/12] app/mldev: add test case to validate model ops Srikanth Yalavarthi
` (8 subsequent siblings)
11 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-11 15:08 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Added test case to validate device handling operations. Device ops
test is a collection of multiple sub-tests. Enabled sub-test to
validate device reconfiguration. Set device_ops as the default test.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
Acked-by: Anup Prabhu <aprabhu@marvell.com>
---
app/test-mldev/meson.build | 1 +
app/test-mldev/ml_options.c | 5 +-
app/test-mldev/ml_options.h | 2 +
app/test-mldev/test_device_ops.c | 228 +++++++++++++++++++++++++++++++
app/test-mldev/test_device_ops.h | 17 +++
5 files changed, 251 insertions(+), 2 deletions(-)
create mode 100644 app/test-mldev/test_device_ops.c
create mode 100644 app/test-mldev/test_device_ops.h
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
index 964bb9ddc4..60ea23d142 100644
--- a/app/test-mldev/meson.build
+++ b/app/test-mldev/meson.build
@@ -13,6 +13,7 @@ sources = files(
'ml_test.c',
'parser.c',
'test_common.c',
+ 'test_device_ops.c',
)
deps += ['mldev']
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 521e9a2f97..b0837dde14 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -19,7 +19,7 @@ void
ml_options_default(struct ml_options *opt)
{
memset(opt, 0, sizeof(*opt));
- strlcpy(opt->test_name, "ml_test", ML_TEST_NAME_MAX_LEN);
+ strlcpy(opt->test_name, "device_ops", ML_TEST_NAME_MAX_LEN);
opt->dev_id = 0;
opt->socket_id = SOCKET_ID_ANY;
opt->debug = false;
@@ -61,7 +61,8 @@ ml_parse_socket_id(struct ml_options *opt, const char *arg)
static void
ml_dump_test_options(const char *testname)
{
- RTE_SET_USED(testname);
+ if (strcmp(testname, "device_ops") == 0)
+ printf("\n");
}
static void
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index 05311a9a47..6899cc2f88 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -9,6 +9,7 @@
#include <stdint.h>
#define ML_TEST_NAME_MAX_LEN 32
+#define ML_TEST_MAX_MODELS 8
/* Options names */
#define ML_TEST ("test")
@@ -21,6 +22,7 @@ struct ml_options {
char test_name[ML_TEST_NAME_MAX_LEN];
int16_t dev_id;
int socket_id;
+ uint8_t nb_filelist;
bool debug;
};
diff --git a/app/test-mldev/test_device_ops.c b/app/test-mldev/test_device_ops.c
new file mode 100644
index 0000000000..be0c5f0780
--- /dev/null
+++ b/app/test-mldev/test_device_ops.c
@@ -0,0 +1,228 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <errno.h>
+
+#include <rte_common.h>
+#include <rte_malloc.h>
+#include <rte_mldev.h>
+
+#include "test_device_ops.h"
+
+static bool
+test_device_cap_check(struct ml_options *opt)
+{
+ if (!ml_test_cap_check(opt))
+ return false;
+
+ return true;
+}
+
+static int
+test_device_opt_check(struct ml_options *opt)
+{
+ int ret;
+
+ /* check common opts */
+ ret = ml_test_opt_check(opt);
+ if (ret != 0)
+ return ret;
+
+ return 0;
+}
+
+static void
+test_device_opt_dump(struct ml_options *opt)
+{
+ /* dump common opts */
+ ml_test_opt_dump(opt);
+}
+
+static int
+test_device_setup(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_device *t;
+ void *test_device;
+ int ret = 0;
+
+ /* allocate for test structure */
+ test_device = rte_zmalloc_socket(test->name, sizeof(struct test_device),
+ RTE_CACHE_LINE_SIZE, opt->socket_id);
+ if (test_device == NULL) {
+ ml_err("failed to allocate memory for test_model");
+ ret = -ENOMEM;
+ goto error;
+ }
+ test->test_priv = test_device;
+ t = ml_test_priv(test);
+
+ t->cmn.result = ML_TEST_FAILED;
+ t->cmn.opt = opt;
+
+ /* get device info */
+ ret = rte_ml_dev_info_get(opt->dev_id, &t->cmn.dev_info);
+ if (ret < 0) {
+ ml_err("failed to get device info");
+ goto error;
+ }
+
+ return 0;
+
+error:
+ if (test_device != NULL)
+ rte_free(test_device);
+
+ return ret;
+}
+
+static void
+test_device_destroy(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_device *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+ if (t != NULL)
+ rte_free(t);
+}
+
+static int
+test_device_reconfigure(struct ml_test *test, struct ml_options *opt)
+{
+ struct rte_ml_dev_config dev_config;
+ struct rte_ml_dev_qp_conf qp_conf;
+ struct test_device *t;
+ uint16_t qp_id = 0;
+ int ret = 0;
+
+ t = ml_test_priv(test);
+
+ /* configure with default options */
+ ret = ml_test_device_configure(test, opt);
+ if (ret != 0)
+ return ret;
+
+ /* setup one queue pair with nb_desc = 1 */
+ qp_conf.nb_desc = 1;
+ qp_conf.cb = NULL;
+
+ ret = rte_ml_dev_queue_pair_setup(opt->dev_id, qp_id, &qp_conf, opt->socket_id);
+ if (ret != 0) {
+ ml_err("Failed to setup ML device queue-pair, dev_id = %d, qp_id = %u\n",
+ opt->dev_id, qp_id);
+ goto error;
+ }
+
+ /* start device */
+ ret = ml_test_device_start(test, opt);
+ if (ret != 0)
+ goto error;
+
+ /* stop device */
+ ret = ml_test_device_stop(test, opt);
+ if (ret != 0) {
+ ml_err("Failed to stop device");
+ goto error;
+ }
+
+ /* reconfigure device based on dev_info */
+ dev_config.socket_id = opt->socket_id;
+ dev_config.nb_models = t->cmn.dev_info.max_models;
+ dev_config.nb_queue_pairs = t->cmn.dev_info.max_queue_pairs;
+ ret = rte_ml_dev_configure(opt->dev_id, &dev_config);
+ if (ret != 0) {
+ ml_err("Failed to reconfigure ML device, dev_id = %d\n", opt->dev_id);
+ return ret;
+ }
+
+ /* setup queue pairs */
+ for (qp_id = 0; qp_id < t->cmn.dev_info.max_queue_pairs; qp_id++) {
+ qp_conf.nb_desc = t->cmn.dev_info.max_desc;
+ qp_conf.cb = NULL;
+
+ ret = rte_ml_dev_queue_pair_setup(opt->dev_id, qp_id, &qp_conf, opt->socket_id);
+ if (ret != 0) {
+ ml_err("Failed to setup ML device queue-pair, dev_id = %d, qp_id = %u\n",
+ opt->dev_id, qp_id);
+ goto error;
+ }
+ }
+
+ /* start device */
+ ret = ml_test_device_start(test, opt);
+ if (ret != 0)
+ goto error;
+
+ /* stop device */
+ ret = ml_test_device_stop(test, opt);
+ if (ret != 0)
+ goto error;
+
+ /* close device */
+ ret = ml_test_device_close(test, opt);
+ if (ret != 0)
+ return ret;
+
+ return 0;
+
+error:
+ ml_test_device_close(test, opt);
+
+ return ret;
+}
+
+static int
+test_device_driver(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_device *t;
+ int ret = 0;
+
+ t = ml_test_priv(test);
+
+ /* sub-test: device reconfigure */
+ ret = test_device_reconfigure(test, opt);
+ if (ret != 0) {
+ printf("\n");
+ printf("Model Device Reconfigure Test: " CLRED "%s" CLNRM "\n", "Failed");
+ goto error;
+ } else {
+ printf("\n");
+ printf("Model Device Reconfigure Test: " CLYEL "%s" CLNRM "\n", "Passed");
+ }
+
+ printf("\n");
+
+ t->cmn.result = ML_TEST_SUCCESS;
+
+ return 0;
+
+error:
+ t->cmn.result = ML_TEST_FAILED;
+ return -1;
+}
+
+static int
+test_device_result(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_device *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+
+ return t->cmn.result;
+}
+
+static const struct ml_test_ops device_ops = {
+ .cap_check = test_device_cap_check,
+ .opt_check = test_device_opt_check,
+ .opt_dump = test_device_opt_dump,
+ .test_setup = test_device_setup,
+ .test_destroy = test_device_destroy,
+ .test_driver = test_device_driver,
+ .test_result = test_device_result,
+};
+
+ML_TEST_REGISTER(device_ops);
diff --git a/app/test-mldev/test_device_ops.h b/app/test-mldev/test_device_ops.h
new file mode 100644
index 0000000000..115b1072a2
--- /dev/null
+++ b/app/test-mldev/test_device_ops.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_TEST_DEVICE_OPS_
+#define _ML_TEST_DEVICE_OPS_
+
+#include <rte_common.h>
+
+#include "test_common.h"
+
+struct test_device {
+ /* common data */
+ struct test_common cmn;
+} __rte_cache_aligned;
+
+#endif /* _ML_TEST_DEVICE_OPS_ */
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v6 04/12] app/mldev: add test case to validate model ops
2023-03-11 15:08 ` [PATCH v6 00/12] Implementation of mldev test application Srikanth Yalavarthi
` (2 preceding siblings ...)
2023-03-11 15:08 ` [PATCH v6 03/12] app/mldev: add test case to validate device ops Srikanth Yalavarthi
@ 2023-03-11 15:08 ` Srikanth Yalavarthi
2023-03-11 15:08 ` [PATCH v6 05/12] app/mldev: add ordered inference test case Srikanth Yalavarthi
` (7 subsequent siblings)
11 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-11 15:08 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Added test case to validate model operations. Model ops test
is a collection of sub-tests. Each sub-test invokes the model
operations in a specific order.
Sub-test A: (load -> start -> stop -> unload) x n
Sub-test B: load x n -> start x n -> stop x n -> unload x n
Sub-test C: load x n + (start + stop) x n + unload x n
Sub-test D: (load + start) x n -> (stop + unload) x n
Added internal functions to handle model load, start, stop and
unload. List of models to be used for testing can be specified
through application argument "--models"
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
Acked-by: Anup Prabhu <aprabhu@marvell.com>
---
app/test-mldev/meson.build | 2 +
app/test-mldev/ml_options.c | 44 ++-
app/test-mldev/ml_options.h | 6 +
app/test-mldev/test_model_common.c | 158 +++++++++++
app/test-mldev/test_model_common.h | 34 +++
app/test-mldev/test_model_ops.c | 428 +++++++++++++++++++++++++++++
app/test-mldev/test_model_ops.h | 20 ++
7 files changed, 689 insertions(+), 3 deletions(-)
create mode 100644 app/test-mldev/test_model_common.c
create mode 100644 app/test-mldev/test_model_common.h
create mode 100644 app/test-mldev/test_model_ops.c
create mode 100644 app/test-mldev/test_model_ops.h
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
index 60ea23d142..b09e1ccc8a 100644
--- a/app/test-mldev/meson.build
+++ b/app/test-mldev/meson.build
@@ -14,6 +14,8 @@ sources = files(
'parser.c',
'test_common.c',
'test_device_ops.c',
+ 'test_model_common.c',
+ 'test_model_ops.c',
)
deps += ['mldev']
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index b0837dde14..91259e1427 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -22,6 +22,7 @@ ml_options_default(struct ml_options *opt)
strlcpy(opt->test_name, "device_ops", ML_TEST_NAME_MAX_LEN);
opt->dev_id = 0;
opt->socket_id = SOCKET_ID_ANY;
+ opt->nb_filelist = 0;
opt->debug = false;
}
@@ -58,11 +59,47 @@ ml_parse_socket_id(struct ml_options *opt, const char *arg)
return 0;
}
+static int
+ml_parse_models(struct ml_options *opt, const char *arg)
+{
+ const char *delim = ",";
+ char models[PATH_MAX];
+ char *token;
+ int ret = 0;
+
+ strlcpy(models, arg, PATH_MAX);
+
+ token = strtok(models, delim);
+ while (token != NULL) {
+ strlcpy(opt->filelist[opt->nb_filelist].model, token, PATH_MAX);
+ opt->nb_filelist++;
+
+ if (opt->nb_filelist >= ML_TEST_MAX_MODELS) {
+ ml_err("Exceeded model count, max = %d\n", ML_TEST_MAX_MODELS);
+ ret = -EINVAL;
+ break;
+ }
+ token = strtok(NULL, delim);
+ }
+
+ if (opt->nb_filelist == 0) {
+ ml_err("Models list is empty. Need at least one model for the test");
+ ret = -EINVAL;
+ }
+
+ return ret;
+}
+
static void
ml_dump_test_options(const char *testname)
{
if (strcmp(testname, "device_ops") == 0)
printf("\n");
+
+ if (strcmp(testname, "model_ops") == 0) {
+ printf("\t\t--models : comma separated list of models\n");
+ printf("\n");
+ }
}
static void
@@ -80,9 +117,9 @@ print_usage(char *program)
ml_test_dump_names(ml_dump_test_options);
}
-static struct option lgopts[] = {{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0},
- {ML_SOCKET_ID, 1, 0, 0}, {ML_DEBUG, 0, 0, 0},
- {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
+static struct option lgopts[] = {
+ {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0}, {ML_MODELS, 1, 0, 0},
+ {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
static int
ml_opts_parse_long(int opt_idx, struct ml_options *opt)
@@ -93,6 +130,7 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
{ML_TEST, ml_parse_test_name},
{ML_DEVICE_ID, ml_parse_dev_id},
{ML_SOCKET_ID, ml_parse_socket_id},
+ {ML_MODELS, ml_parse_models},
};
for (i = 0; i < RTE_DIM(parsermap); i++) {
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index 6899cc2f88..61e938d2e2 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -15,13 +15,19 @@
#define ML_TEST ("test")
#define ML_DEVICE_ID ("dev_id")
#define ML_SOCKET_ID ("socket_id")
+#define ML_MODELS ("models")
#define ML_DEBUG ("debug")
#define ML_HELP ("help")
+struct ml_filelist {
+ char model[PATH_MAX];
+};
+
struct ml_options {
char test_name[ML_TEST_NAME_MAX_LEN];
int16_t dev_id;
int socket_id;
+ struct ml_filelist filelist[ML_TEST_MAX_MODELS];
uint8_t nb_filelist;
bool debug;
};
diff --git a/app/test-mldev/test_model_common.c b/app/test-mldev/test_model_common.c
new file mode 100644
index 0000000000..b94d46154d
--- /dev/null
+++ b/app/test-mldev/test_model_common.c
@@ -0,0 +1,158 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <errno.h>
+
+#include <rte_common.h>
+#include <rte_malloc.h>
+#include <rte_mldev.h>
+
+#include "ml_common.h"
+#include "test_model_common.h"
+
+int
+ml_model_load(struct ml_test *test, struct ml_options *opt, struct ml_model *model, uint16_t fid)
+{
+ struct test_common *t = ml_test_priv(test);
+ struct rte_ml_model_params model_params;
+ FILE *fp;
+ int ret;
+
+ if (model->state == MODEL_LOADED)
+ return 0;
+
+ if (model->state != MODEL_INITIAL)
+ return -EINVAL;
+
+ /* read model binary */
+ fp = fopen(opt->filelist[fid].model, "r");
+ if (fp == NULL) {
+ ml_err("Failed to open model file : %s\n", opt->filelist[fid].model);
+ return -1;
+ }
+
+ fseek(fp, 0, SEEK_END);
+ model_params.size = ftell(fp);
+ fseek(fp, 0, SEEK_SET);
+
+ model_params.addr = rte_malloc_socket("ml_model", model_params.size,
+ t->dev_info.min_align_size, opt->socket_id);
+ if (model_params.addr == NULL) {
+ ml_err("Failed to allocate memory for model: %s\n", opt->filelist[fid].model);
+ fclose(fp);
+ return -ENOMEM;
+ }
+
+ if (fread(model_params.addr, 1, model_params.size, fp) != model_params.size) {
+ ml_err("Failed to read model file : %s\n", opt->filelist[fid].model);
+ rte_free(model_params.addr);
+ fclose(fp);
+ return -1;
+ }
+ fclose(fp);
+
+ /* load model to device */
+ ret = rte_ml_model_load(opt->dev_id, &model_params, &model->id);
+ if (ret != 0) {
+ ml_err("Failed to load model : %s\n", opt->filelist[fid].model);
+ model->state = MODEL_ERROR;
+ rte_free(model_params.addr);
+ return ret;
+ }
+
+ /* release mz */
+ rte_free(model_params.addr);
+
+ /* get model info */
+ ret = rte_ml_model_info_get(opt->dev_id, model->id, &model->info);
+ if (ret != 0) {
+ ml_err("Failed to get model info : %s\n", opt->filelist[fid].model);
+ return ret;
+ }
+
+ model->state = MODEL_LOADED;
+
+ return 0;
+}
+
+int
+ml_model_unload(struct ml_test *test, struct ml_options *opt, struct ml_model *model, uint16_t fid)
+{
+ struct test_common *t = ml_test_priv(test);
+ int ret;
+
+ RTE_SET_USED(t);
+
+ if (model->state == MODEL_INITIAL)
+ return 0;
+
+ if (model->state != MODEL_LOADED)
+ return -EINVAL;
+
+ /* unload model */
+ ret = rte_ml_model_unload(opt->dev_id, model->id);
+ if (ret != 0) {
+ ml_err("Failed to unload model: %s\n", opt->filelist[fid].model);
+ model->state = MODEL_ERROR;
+ return ret;
+ }
+
+ model->state = MODEL_INITIAL;
+
+ return 0;
+}
+
+int
+ml_model_start(struct ml_test *test, struct ml_options *opt, struct ml_model *model, uint16_t fid)
+{
+ struct test_common *t = ml_test_priv(test);
+ int ret;
+
+ RTE_SET_USED(t);
+
+ if (model->state == MODEL_STARTED)
+ return 0;
+
+ if (model->state != MODEL_LOADED)
+ return -EINVAL;
+
+ /* start model */
+ ret = rte_ml_model_start(opt->dev_id, model->id);
+ if (ret != 0) {
+ ml_err("Failed to start model : %s\n", opt->filelist[fid].model);
+ model->state = MODEL_ERROR;
+ return ret;
+ }
+
+ model->state = MODEL_STARTED;
+
+ return 0;
+}
+
+int
+ml_model_stop(struct ml_test *test, struct ml_options *opt, struct ml_model *model, uint16_t fid)
+{
+ struct test_common *t = ml_test_priv(test);
+ int ret;
+
+ RTE_SET_USED(t);
+
+ if (model->state == MODEL_LOADED)
+ return 0;
+
+ if (model->state != MODEL_STARTED)
+ return -EINVAL;
+
+ /* stop model */
+ ret = rte_ml_model_stop(opt->dev_id, model->id);
+ if (ret != 0) {
+ ml_err("Failed to stop model: %s\n", opt->filelist[fid].model);
+ model->state = MODEL_ERROR;
+ return ret;
+ }
+
+ model->state = MODEL_LOADED;
+
+ return 0;
+}
diff --git a/app/test-mldev/test_model_common.h b/app/test-mldev/test_model_common.h
new file mode 100644
index 0000000000..74aec0a797
--- /dev/null
+++ b/app/test-mldev/test_model_common.h
@@ -0,0 +1,34 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_TEST_MODEL_COMMON_
+#define _ML_TEST_MODEL_COMMON_
+
+#include <rte_mldev.h>
+
+#include "test_common.h"
+
+enum model_state {
+ MODEL_INITIAL,
+ MODEL_LOADED,
+ MODEL_STARTED,
+ MODEL_ERROR,
+};
+
+struct ml_model {
+ uint16_t id;
+ struct rte_ml_model_info info;
+ enum model_state state;
+};
+
+int ml_model_load(struct ml_test *test, struct ml_options *opt, struct ml_model *model,
+ uint16_t fid);
+int ml_model_unload(struct ml_test *test, struct ml_options *opt, struct ml_model *model,
+ uint16_t fid);
+int ml_model_start(struct ml_test *test, struct ml_options *opt, struct ml_model *model,
+ uint16_t fid);
+int ml_model_stop(struct ml_test *test, struct ml_options *opt, struct ml_model *model,
+ uint16_t fid);
+
+#endif /* _ML_TEST_MODEL_COMMON_ */
diff --git a/app/test-mldev/test_model_ops.c b/app/test-mldev/test_model_ops.c
new file mode 100644
index 0000000000..0202b31190
--- /dev/null
+++ b/app/test-mldev/test_model_ops.c
@@ -0,0 +1,428 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <unistd.h>
+
+#include <rte_common.h>
+#include <rte_malloc.h>
+#include <rte_mldev.h>
+
+#include "test_model_ops.h"
+
+static bool
+test_model_ops_cap_check(struct ml_options *opt)
+{
+ if (!ml_test_cap_check(opt))
+ return false;
+
+ return true;
+}
+
+static int
+test_model_ops_opt_check(struct ml_options *opt)
+{
+ uint32_t i;
+ int ret;
+
+ /* check common opts */
+ ret = ml_test_opt_check(opt);
+ if (ret != 0)
+ return ret;
+
+ /* check model file availability */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ if (access(opt->filelist[i].model, F_OK) == -1) {
+ ml_err("Model file not available: id = %u, file = %s", i,
+ opt->filelist[i].model);
+ return -ENOENT;
+ }
+ }
+
+ return 0;
+}
+
+static void
+test_model_ops_opt_dump(struct ml_options *opt)
+{
+ uint32_t i;
+
+ /* dump common opts */
+ ml_test_opt_dump(opt);
+
+ /* dump test specific opts */
+ ml_dump_begin("models");
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_dump_list("model", i, opt->filelist[i].model);
+ ml_dump_end;
+}
+
+static int
+test_model_ops_setup(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+ void *test_model_ops;
+ int ret = 0;
+ uint32_t i;
+
+ /* allocate model ops test structure */
+ test_model_ops = rte_zmalloc_socket(test->name, sizeof(struct test_model_ops),
+ RTE_CACHE_LINE_SIZE, opt->socket_id);
+ if (test_model_ops == NULL) {
+ ml_err("Failed to allocate memory for test_model");
+ ret = -ENOMEM;
+ goto error;
+ }
+ test->test_priv = test_model_ops;
+ t = ml_test_priv(test);
+
+ t->cmn.result = ML_TEST_FAILED;
+ t->cmn.opt = opt;
+
+ /* get device info */
+ ret = rte_ml_dev_info_get(opt->dev_id, &t->cmn.dev_info);
+ if (ret < 0) {
+ ml_err("Failed to get device info");
+ goto error;
+ }
+
+ /* set model initial state */
+ for (i = 0; i < opt->nb_filelist; i++)
+ t->model[i].state = MODEL_INITIAL;
+
+ return 0;
+
+error:
+ if (test_model_ops != NULL)
+ rte_free(test_model_ops);
+
+ return ret;
+}
+
+static void
+test_model_ops_destroy(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+ if (t != NULL)
+ rte_free(t);
+}
+
+static int
+test_model_ops_mldev_setup(struct ml_test *test, struct ml_options *opt)
+{
+ int ret;
+
+ ret = ml_test_device_configure(test, opt);
+ if (ret != 0)
+ return ret;
+
+ ret = ml_test_device_start(test, opt);
+ if (ret != 0)
+ goto error;
+
+ return 0;
+
+error:
+ ml_test_device_close(test, opt);
+
+ return ret;
+}
+
+static int
+test_model_ops_mldev_destroy(struct ml_test *test, struct ml_options *opt)
+{
+ int ret;
+
+ ret = ml_test_device_stop(test, opt);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_test_device_close(test, opt);
+ if (ret != 0)
+ return ret;
+
+ return 0;
+
+error:
+ ml_test_device_close(test, opt);
+
+ return ret;
+}
+
+/* Sub-test A: (load -> start -> stop -> unload) x n */
+static int
+test_model_ops_subtest_a(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+ int ret = 0;
+ uint32_t i;
+
+ t = ml_test_priv(test);
+
+ /* load + start + stop + unload */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_load(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_start(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_stop(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_unload(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+error:
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_stop(test, opt, &t->model[i], i);
+
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_unload(test, opt, &t->model[i], i);
+
+ return ret;
+}
+
+/* Sub-test B: load x n -> start x n -> stop x n -> unload x n */
+static int
+test_model_ops_subtest_b(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+ int ret = 0;
+ uint32_t i;
+
+ t = ml_test_priv(test);
+
+ /* load */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_load(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* start */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_start(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* stop */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_stop(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* unload */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_unload(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ return 0;
+
+error:
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_stop(test, opt, &t->model[i], i);
+
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_unload(test, opt, &t->model[i], i);
+
+ return ret;
+}
+
+/* Sub-test C: load x n + (start + stop) x n + unload x n */
+static int
+test_model_ops_subtest_c(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+ int ret = 0;
+ uint32_t i;
+
+ t = ml_test_priv(test);
+
+ /* load */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_load(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* start + stop */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_start(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_stop(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* unload */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_unload(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ return 0;
+
+error:
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_stop(test, opt, &t->model[i], i);
+
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_unload(test, opt, &t->model[i], i);
+
+ return ret;
+}
+
+/* Sub-test D: (load + start) x n -> (stop + unload) x n */
+static int
+test_model_ops_subtest_d(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+ int ret = 0;
+ uint32_t i;
+
+ t = ml_test_priv(test);
+
+ /* load + start */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_load(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_start(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* stop + unload */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_stop(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_unload(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ return 0;
+
+error:
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_stop(test, opt, &t->model[i], i);
+
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_unload(test, opt, &t->model[i], i);
+
+ return ret;
+}
+
+static int
+test_model_ops_driver(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+ int ret = 0;
+
+ t = ml_test_priv(test);
+
+ /* device setup */
+ ret = test_model_ops_mldev_setup(test, opt);
+ if (ret != 0)
+ return ret;
+
+ printf("\n");
+
+ /* sub-test A */
+ ret = test_model_ops_subtest_a(test, opt);
+ if (ret != 0) {
+ printf("Model Ops Sub-test A: " CLRED "%s" CLNRM "\n", "Failed");
+ goto error;
+ } else {
+ printf("Model Ops Sub-test A: " CLYEL "%s" CLNRM "\n", "Passed");
+ }
+
+ /* sub-test B */
+ ret = test_model_ops_subtest_b(test, opt);
+ if (ret != 0) {
+ printf("Model Ops Sub-test B: " CLRED "%s" CLNRM "\n", "Failed");
+ goto error;
+ } else {
+ printf("Model Ops Sub-test B: " CLYEL "%s" CLNRM "\n", "Passed");
+ }
+
+ /* sub-test C */
+ ret = test_model_ops_subtest_c(test, opt);
+ if (ret != 0) {
+ printf("Model Ops Sub-test C: " CLRED "%s" CLNRM "\n", "Failed");
+ goto error;
+ } else {
+ printf("Model Ops Sub-test C: " CLYEL "%s" CLNRM "\n", "Passed");
+ }
+
+ /* sub-test D */
+ ret = test_model_ops_subtest_d(test, opt);
+ if (ret != 0) {
+ printf("Model Ops Sub-test D: " CLRED "%s" CLNRM "\n", "Failed");
+ goto error;
+ } else {
+ printf("Model Ops Sub-test D: " CLYEL "%s" CLNRM "\n", "Passed");
+ }
+
+ printf("\n");
+
+ /* device destroy */
+ ret = test_model_ops_mldev_destroy(test, opt);
+ if (ret != 0)
+ return ret;
+
+ t->cmn.result = ML_TEST_SUCCESS;
+
+ return 0;
+
+error:
+ test_model_ops_mldev_destroy(test, opt);
+
+ t->cmn.result = ML_TEST_FAILED;
+
+ return ret;
+}
+
+static int
+test_model_ops_result(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+
+ return t->cmn.result;
+}
+
+static const struct ml_test_ops model_ops = {
+ .cap_check = test_model_ops_cap_check,
+ .opt_check = test_model_ops_opt_check,
+ .opt_dump = test_model_ops_opt_dump,
+ .test_setup = test_model_ops_setup,
+ .test_destroy = test_model_ops_destroy,
+ .test_driver = test_model_ops_driver,
+ .test_result = test_model_ops_result,
+};
+
+ML_TEST_REGISTER(model_ops);
diff --git a/app/test-mldev/test_model_ops.h b/app/test-mldev/test_model_ops.h
new file mode 100644
index 0000000000..c21f12b788
--- /dev/null
+++ b/app/test-mldev/test_model_ops.h
@@ -0,0 +1,20 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_TEST_MODEL_OPS_
+#define _ML_TEST_MODEL_OPS_
+
+#include <rte_common.h>
+
+#include "test_model_common.h"
+
+struct test_model_ops {
+ /* common data */
+ struct test_common cmn;
+
+ /* test specific data */
+ struct ml_model model[ML_TEST_MAX_MODELS];
+} __rte_cache_aligned;
+
+#endif /* _ML_TEST_MODEL_OPS_ */
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v6 05/12] app/mldev: add ordered inference test case
2023-03-11 15:08 ` [PATCH v6 00/12] Implementation of mldev test application Srikanth Yalavarthi
` (3 preceding siblings ...)
2023-03-11 15:08 ` [PATCH v6 04/12] app/mldev: add test case to validate model ops Srikanth Yalavarthi
@ 2023-03-11 15:08 ` Srikanth Yalavarthi
2023-03-16 17:45 ` Thomas Monjalon
2023-03-11 15:08 ` [PATCH v6 06/12] app/mldev: add test case to interleave inferences Srikanth Yalavarthi
` (6 subsequent siblings)
11 siblings, 1 reply; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-11 15:08 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Added an ordered test case to execute inferences with single
or multiple models. In this test case inference requests for
a model are enqueued after completion of all requests for
the previous model. Test supports inference repetitions.
Operations sequence when testing with N models and R reps,
(load -> start -> (enqueue + dequeue) x R -> stop -> unload) x N
Test case can be executed by selecting "inference_ordered" test
and repetitions can be specified through "--repetitions" argument.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
Acked-by: Anup Prabhu <aprabhu@marvell.com>
---
app/test-mldev/meson.build | 2 +
app/test-mldev/ml_options.c | 73 ++-
app/test-mldev/ml_options.h | 17 +-
app/test-mldev/test_inference_common.c | 567 ++++++++++++++++++++++++
app/test-mldev/test_inference_common.h | 61 +++
app/test-mldev/test_inference_ordered.c | 115 +++++
app/test-mldev/test_model_common.h | 10 +
7 files changed, 833 insertions(+), 12 deletions(-)
create mode 100644 app/test-mldev/test_inference_common.c
create mode 100644 app/test-mldev/test_inference_common.h
create mode 100644 app/test-mldev/test_inference_ordered.c
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
index b09e1ccc8a..475d76d126 100644
--- a/app/test-mldev/meson.build
+++ b/app/test-mldev/meson.build
@@ -16,6 +16,8 @@ sources = files(
'test_device_ops.c',
'test_model_common.c',
'test_model_ops.c',
+ 'test_inference_common.c',
+ 'test_inference_ordered.c',
)
deps += ['mldev']
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 91259e1427..b7215a7e88 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -23,6 +23,7 @@ ml_options_default(struct ml_options *opt)
opt->dev_id = 0;
opt->socket_id = SOCKET_ID_ANY;
opt->nb_filelist = 0;
+ opt->repetitions = 1;
opt->debug = false;
}
@@ -90,6 +91,60 @@ ml_parse_models(struct ml_options *opt, const char *arg)
return ret;
}
+static int
+ml_parse_filelist(struct ml_options *opt, const char *arg)
+{
+ const char *delim = ",";
+ char filelist[PATH_MAX];
+ char *token;
+
+ if (opt->nb_filelist >= ML_TEST_MAX_MODELS) {
+ ml_err("Exceeded filelist count, max = %d\n", ML_TEST_MAX_MODELS);
+ return -1;
+ }
+
+ strlcpy(filelist, arg, PATH_MAX);
+
+ /* model */
+ token = strtok(filelist, delim);
+ if (token == NULL) {
+ ml_err("Invalid filelist, model not specified = %s\n", arg);
+ return -EINVAL;
+ }
+ strlcpy(opt->filelist[opt->nb_filelist].model, token, PATH_MAX);
+
+ /* input */
+ token = strtok(NULL, delim);
+ if (token == NULL) {
+ ml_err("Invalid filelist, input not specified = %s\n", arg);
+ return -EINVAL;
+ }
+ strlcpy(opt->filelist[opt->nb_filelist].input, token, PATH_MAX);
+
+ /* output */
+ token = strtok(NULL, delim);
+ if (token == NULL) {
+ ml_err("Invalid filelist, output not specified = %s\n", arg);
+ return -EINVAL;
+ }
+ strlcpy(opt->filelist[opt->nb_filelist].output, token, PATH_MAX);
+
+ opt->nb_filelist++;
+
+ if (opt->nb_filelist == 0) {
+ ml_err("Empty filelist. Need at least one filelist entry for the test.");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int
+ml_parse_repetitions(struct ml_options *opt, const char *arg)
+{
+ return parser_read_uint64(&opt->repetitions, arg);
+}
+
static void
ml_dump_test_options(const char *testname)
{
@@ -100,6 +155,12 @@ ml_dump_test_options(const char *testname)
printf("\t\t--models : comma separated list of models\n");
printf("\n");
}
+
+ if (strcmp(testname, "inference_ordered") == 0) {
+ printf("\t\t--filelist : comma separated list of model, input and output\n"
+ "\t\t--repetitions : number of inference repetitions\n");
+ printf("\n");
+ }
}
static void
@@ -118,8 +179,9 @@ print_usage(char *program)
}
static struct option lgopts[] = {
- {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0}, {ML_MODELS, 1, 0, 0},
- {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
+ {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
+ {ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
+ {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
static int
ml_opts_parse_long(int opt_idx, struct ml_options *opt)
@@ -127,10 +189,9 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
unsigned int i;
struct long_opt_parser parsermap[] = {
- {ML_TEST, ml_parse_test_name},
- {ML_DEVICE_ID, ml_parse_dev_id},
- {ML_SOCKET_ID, ml_parse_socket_id},
- {ML_MODELS, ml_parse_models},
+ {ML_TEST, ml_parse_test_name}, {ML_DEVICE_ID, ml_parse_dev_id},
+ {ML_SOCKET_ID, ml_parse_socket_id}, {ML_MODELS, ml_parse_models},
+ {ML_FILELIST, ml_parse_filelist}, {ML_REPETITIONS, ml_parse_repetitions},
};
for (i = 0; i < RTE_DIM(parsermap); i++) {
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index 61e938d2e2..6a13f97a30 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -12,15 +12,19 @@
#define ML_TEST_MAX_MODELS 8
/* Options names */
-#define ML_TEST ("test")
-#define ML_DEVICE_ID ("dev_id")
-#define ML_SOCKET_ID ("socket_id")
-#define ML_MODELS ("models")
-#define ML_DEBUG ("debug")
-#define ML_HELP ("help")
+#define ML_TEST ("test")
+#define ML_DEVICE_ID ("dev_id")
+#define ML_SOCKET_ID ("socket_id")
+#define ML_MODELS ("models")
+#define ML_FILELIST ("filelist")
+#define ML_REPETITIONS ("repetitions")
+#define ML_DEBUG ("debug")
+#define ML_HELP ("help")
struct ml_filelist {
char model[PATH_MAX];
+ char input[PATH_MAX];
+ char output[PATH_MAX];
};
struct ml_options {
@@ -29,6 +33,7 @@ struct ml_options {
int socket_id;
struct ml_filelist filelist[ML_TEST_MAX_MODELS];
uint8_t nb_filelist;
+ uint64_t repetitions;
bool debug;
};
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
new file mode 100644
index 0000000000..6a6999d524
--- /dev/null
+++ b/app/test-mldev/test_inference_common.c
@@ -0,0 +1,567 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <errno.h>
+#include <unistd.h>
+
+#include <rte_common.h>
+#include <rte_launch.h>
+#include <rte_lcore.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_mldev.h>
+
+#include "ml_common.h"
+#include "test_inference_common.h"
+
+/* Enqueue inference requests with burst size equal to 1 */
+static int
+ml_enqueue_single(void *arg)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)arg);
+ struct ml_request *req = NULL;
+ struct rte_ml_op *op = NULL;
+ struct ml_core_args *args;
+ uint64_t model_enq = 0;
+ uint32_t burst_enq;
+ uint32_t lcore_id;
+ uint16_t fid;
+ int ret;
+
+ lcore_id = rte_lcore_id();
+ args = &t->args[lcore_id];
+ model_enq = 0;
+
+ if (args->nb_reqs == 0)
+ return 0;
+
+next_rep:
+ fid = args->start_fid;
+
+next_model:
+ ret = rte_mempool_get(t->op_pool, (void **)&op);
+ if (ret != 0)
+ goto next_model;
+
+retry:
+ ret = rte_mempool_get(t->model[fid].io_pool, (void **)&req);
+ if (ret != 0)
+ goto retry;
+
+ op->model_id = t->model[fid].id;
+ op->nb_batches = t->model[fid].info.batch_size;
+ op->mempool = t->op_pool;
+
+ op->input.addr = req->input;
+ op->input.length = t->model[fid].inp_qsize;
+ op->input.next = NULL;
+
+ op->output.addr = req->output;
+ op->output.length = t->model[fid].out_qsize;
+ op->output.next = NULL;
+
+ op->user_ptr = req;
+ req->niters++;
+ req->fid = fid;
+
+enqueue_req:
+ burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, 0, &op, 1);
+ if (burst_enq == 0)
+ goto enqueue_req;
+
+ fid++;
+ if (likely(fid <= args->end_fid))
+ goto next_model;
+
+ model_enq++;
+ if (likely(model_enq < args->nb_reqs))
+ goto next_rep;
+
+ return 0;
+}
+
+/* Dequeue inference requests with burst size equal to 1 */
+static int
+ml_dequeue_single(void *arg)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)arg);
+ struct rte_ml_op_error error;
+ struct rte_ml_op *op = NULL;
+ struct ml_core_args *args;
+ struct ml_request *req;
+ uint64_t total_deq = 0;
+ uint8_t nb_filelist;
+ uint32_t burst_deq;
+ uint32_t lcore_id;
+
+ lcore_id = rte_lcore_id();
+ args = &t->args[lcore_id];
+ nb_filelist = args->end_fid - args->start_fid + 1;
+
+ if (args->nb_reqs == 0)
+ return 0;
+
+dequeue_req:
+ burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, 0, &op, 1);
+
+ if (likely(burst_deq == 1)) {
+ total_deq += burst_deq;
+ if (unlikely(op->status == RTE_ML_OP_STATUS_ERROR)) {
+ rte_ml_op_error_get(t->cmn.opt->dev_id, op, &error);
+ ml_err("error_code = 0x%" PRIx64 ", error_message = %s\n", error.errcode,
+ error.message);
+ t->error_count[lcore_id]++;
+ }
+ req = (struct ml_request *)op->user_ptr;
+ rte_mempool_put(t->model[req->fid].io_pool, req);
+ rte_mempool_put(t->op_pool, op);
+ }
+
+ if (likely(total_deq < args->nb_reqs * nb_filelist))
+ goto dequeue_req;
+
+ return 0;
+}
+
+bool
+test_inference_cap_check(struct ml_options *opt)
+{
+ struct rte_ml_dev_info dev_info;
+
+ if (!ml_test_cap_check(opt))
+ return false;
+
+ rte_ml_dev_info_get(opt->dev_id, &dev_info);
+ if (opt->nb_filelist > dev_info.max_models) {
+ ml_err("Insufficient capabilities: Filelist count exceeded device limit, count = %u (max limit = %u)",
+ opt->nb_filelist, dev_info.max_models);
+ return false;
+ }
+
+ return true;
+}
+
+int
+test_inference_opt_check(struct ml_options *opt)
+{
+ uint32_t i;
+ int ret;
+
+ /* check common opts */
+ ret = ml_test_opt_check(opt);
+ if (ret != 0)
+ return ret;
+
+ /* check file availability */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ if (access(opt->filelist[i].model, F_OK) == -1) {
+ ml_err("Model file not accessible: id = %u, file = %s", i,
+ opt->filelist[i].model);
+ return -ENOENT;
+ }
+
+ if (access(opt->filelist[i].input, F_OK) == -1) {
+ ml_err("Input file not accessible: id = %u, file = %s", i,
+ opt->filelist[i].input);
+ return -ENOENT;
+ }
+ }
+
+ if (opt->repetitions == 0) {
+ ml_err("Invalid option, repetitions = %" PRIu64 "\n", opt->repetitions);
+ return -EINVAL;
+ }
+
+ /* check number of available lcores. */
+ if (rte_lcore_count() < 3) {
+ ml_err("Insufficient lcores = %u\n", rte_lcore_count());
+ ml_err("Minimum lcores required to create %u queue-pairs = %u\n", 1, 3);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+void
+test_inference_opt_dump(struct ml_options *opt)
+{
+ uint32_t i;
+
+ /* dump common opts */
+ ml_test_opt_dump(opt);
+
+ /* dump test opts */
+ ml_dump("repetitions", "%" PRIu64, opt->repetitions);
+
+ ml_dump_begin("filelist");
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ml_dump_list("model", i, opt->filelist[i].model);
+ ml_dump_list("input", i, opt->filelist[i].input);
+ ml_dump_list("output", i, opt->filelist[i].output);
+ }
+ ml_dump_end;
+}
+
+int
+test_inference_setup(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t;
+ void *test_inference;
+ int ret = 0;
+ uint32_t i;
+
+ test_inference = rte_zmalloc_socket(test->name, sizeof(struct test_inference),
+ RTE_CACHE_LINE_SIZE, opt->socket_id);
+ if (test_inference == NULL) {
+ ml_err("failed to allocate memory for test_model");
+ ret = -ENOMEM;
+ goto error;
+ }
+ test->test_priv = test_inference;
+ t = ml_test_priv(test);
+
+ t->nb_used = 0;
+ t->cmn.result = ML_TEST_FAILED;
+ t->cmn.opt = opt;
+ memset(t->error_count, 0, RTE_MAX_LCORE * sizeof(uint64_t));
+
+ /* get device info */
+ ret = rte_ml_dev_info_get(opt->dev_id, &t->cmn.dev_info);
+ if (ret < 0) {
+ ml_err("failed to get device info");
+ goto error;
+ }
+
+ t->enqueue = ml_enqueue_single;
+ t->dequeue = ml_dequeue_single;
+
+ /* set model initial state */
+ for (i = 0; i < opt->nb_filelist; i++)
+ t->model[i].state = MODEL_INITIAL;
+
+ return 0;
+
+error:
+ if (test_inference != NULL)
+ rte_free(test_inference);
+
+ return ret;
+}
+
+void
+test_inference_destroy(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+ if (t != NULL)
+ rte_free(t);
+}
+
+int
+ml_inference_mldev_setup(struct ml_test *test, struct ml_options *opt)
+{
+ struct rte_ml_dev_qp_conf qp_conf;
+ struct test_inference *t;
+ int ret;
+
+ t = ml_test_priv(test);
+
+ ret = ml_test_device_configure(test, opt);
+ if (ret != 0)
+ return ret;
+
+ /* setup queue pairs */
+ qp_conf.nb_desc = t->cmn.dev_info.max_desc;
+ qp_conf.cb = NULL;
+
+ ret = rte_ml_dev_queue_pair_setup(opt->dev_id, 0, &qp_conf, opt->socket_id);
+ if (ret != 0) {
+ ml_err("Failed to setup ml device queue-pair, dev_id = %d, qp_id = %u\n",
+ opt->dev_id, 0);
+ goto error;
+ }
+
+ ret = ml_test_device_start(test, opt);
+ if (ret != 0)
+ goto error;
+
+ return 0;
+
+error:
+ ml_test_device_close(test, opt);
+
+ return ret;
+}
+
+int
+ml_inference_mldev_destroy(struct ml_test *test, struct ml_options *opt)
+{
+ int ret;
+
+ ret = ml_test_device_stop(test, opt);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_test_device_close(test, opt);
+ if (ret != 0)
+ return ret;
+
+ return 0;
+
+error:
+ ml_test_device_close(test, opt);
+
+ return ret;
+}
+
+/* Callback for IO pool create. This function would compute the fields of ml_request
+ * structure and prepare the quantized input data.
+ */
+static void
+ml_request_initialize(struct rte_mempool *mp, void *opaque, void *obj, unsigned int obj_idx)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)opaque);
+ struct ml_request *req = (struct ml_request *)obj;
+
+ RTE_SET_USED(mp);
+ RTE_SET_USED(obj_idx);
+
+ req->input = (uint8_t *)obj +
+ RTE_ALIGN_CEIL(sizeof(struct ml_request), t->cmn.dev_info.min_align_size);
+ req->output = req->input +
+ RTE_ALIGN_CEIL(t->model[t->fid].inp_qsize, t->cmn.dev_info.min_align_size);
+ req->niters = 0;
+
+ /* quantize data */
+ rte_ml_io_quantize(t->cmn.opt->dev_id, t->model[t->fid].id,
+ t->model[t->fid].info.batch_size, t->model[t->fid].input, req->input);
+}
+
+int
+ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, uint16_t fid)
+{
+ struct test_inference *t = ml_test_priv(test);
+ char mz_name[RTE_MEMZONE_NAMESIZE];
+ char mp_name[RTE_MEMPOOL_NAMESIZE];
+ const struct rte_memzone *mz;
+ uint64_t nb_buffers;
+ uint32_t buff_size;
+ uint32_t mz_size;
+ uint32_t fsize;
+ FILE *fp;
+ int ret;
+
+ /* get input buffer size */
+ ret = rte_ml_io_input_size_get(opt->dev_id, t->model[fid].id, t->model[fid].info.batch_size,
+ &t->model[fid].inp_qsize, &t->model[fid].inp_dsize);
+ if (ret != 0) {
+ ml_err("Failed to get input size, model : %s\n", opt->filelist[fid].model);
+ return ret;
+ }
+
+ /* get output buffer size */
+ ret = rte_ml_io_output_size_get(opt->dev_id, t->model[fid].id,
+ t->model[fid].info.batch_size, &t->model[fid].out_qsize,
+ &t->model[fid].out_dsize);
+ if (ret != 0) {
+ ml_err("Failed to get input size, model : %s\n", opt->filelist[fid].model);
+ return ret;
+ }
+
+ /* allocate buffer for user data */
+ mz_size = t->model[fid].inp_dsize + t->model[fid].out_dsize;
+ sprintf(mz_name, "ml_user_data_%d", fid);
+ mz = rte_memzone_reserve(mz_name, mz_size, opt->socket_id, 0);
+ if (mz == NULL) {
+ ml_err("Memzone allocation failed for ml_user_data\n");
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ t->model[fid].input = mz->addr;
+ t->model[fid].output = t->model[fid].input + t->model[fid].inp_dsize;
+
+ /* load input file */
+ fp = fopen(opt->filelist[fid].input, "r");
+ if (fp == NULL) {
+ ml_err("Failed to open input file : %s\n", opt->filelist[fid].input);
+ ret = -errno;
+ goto error;
+ }
+
+ fseek(fp, 0, SEEK_END);
+ fsize = ftell(fp);
+ fseek(fp, 0, SEEK_SET);
+ if (fsize != t->model[fid].inp_dsize) {
+ ml_err("Invalid input file, size = %u (expected size = %" PRIu64 ")\n", fsize,
+ t->model[fid].inp_dsize);
+ ret = -EINVAL;
+ fclose(fp);
+ goto error;
+ }
+
+ if (fread(t->model[fid].input, 1, t->model[fid].inp_dsize, fp) != t->model[fid].inp_dsize) {
+ ml_err("Failed to read input file : %s\n", opt->filelist[fid].input);
+ ret = -errno;
+ fclose(fp);
+ goto error;
+ }
+ fclose(fp);
+
+ /* create mempool for quantized input and output buffers. ml_request_initialize is
+ * used as a callback for object creation.
+ */
+ buff_size = RTE_ALIGN_CEIL(sizeof(struct ml_request), t->cmn.dev_info.min_align_size) +
+ RTE_ALIGN_CEIL(t->model[fid].inp_qsize, t->cmn.dev_info.min_align_size) +
+ RTE_ALIGN_CEIL(t->model[fid].out_qsize, t->cmn.dev_info.min_align_size);
+ nb_buffers = RTE_MIN((uint64_t)ML_TEST_MAX_POOL_SIZE, opt->repetitions);
+
+ t->fid = fid;
+ sprintf(mp_name, "ml_io_pool_%d", fid);
+ t->model[fid].io_pool = rte_mempool_create(mp_name, nb_buffers, buff_size, 0, 0, NULL, NULL,
+ ml_request_initialize, test, opt->socket_id, 0);
+ if (t->model[fid].io_pool == NULL) {
+ ml_err("Failed to create io pool : %s\n", "ml_io_pool");
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ return 0;
+
+error:
+ if (mz != NULL)
+ rte_memzone_free(mz);
+
+ if (t->model[fid].io_pool != NULL) {
+ rte_mempool_free(t->model[fid].io_pool);
+ t->model[fid].io_pool = NULL;
+ }
+
+ return ret;
+}
+
+void
+ml_inference_iomem_destroy(struct ml_test *test, struct ml_options *opt, uint16_t fid)
+{
+ char mz_name[RTE_MEMZONE_NAMESIZE];
+ char mp_name[RTE_MEMPOOL_NAMESIZE];
+ const struct rte_memzone *mz;
+ struct rte_mempool *mp;
+
+ RTE_SET_USED(test);
+ RTE_SET_USED(opt);
+
+ /* release user data memzone */
+ sprintf(mz_name, "ml_user_data_%d", fid);
+ mz = rte_memzone_lookup(mz_name);
+ if (mz != NULL)
+ rte_memzone_free(mz);
+
+ /* destroy io pool */
+ sprintf(mp_name, "ml_io_pool_%d", fid);
+ mp = rte_mempool_lookup(mp_name);
+ if (mp != NULL)
+ rte_mempool_free(mp);
+}
+
+int
+ml_inference_mem_setup(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t = ml_test_priv(test);
+
+ /* create op pool */
+ t->op_pool = rte_ml_op_pool_create("ml_test_op_pool", ML_TEST_MAX_POOL_SIZE, 0, 0,
+ opt->socket_id);
+ if (t->op_pool == NULL) {
+ ml_err("Failed to create op pool : %s\n", "ml_op_pool");
+ return -ENOMEM;
+ }
+
+ return 0;
+}
+
+void
+ml_inference_mem_destroy(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t = ml_test_priv(test);
+
+ RTE_SET_USED(opt);
+
+ /* release op pool */
+ if (t->op_pool != NULL)
+ rte_mempool_free(t->op_pool);
+}
+
+/* Callback for mempool object iteration. This call would dequantize output data. */
+static void
+ml_request_finish(struct rte_mempool *mp, void *opaque, void *obj, unsigned int obj_idx)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)opaque);
+ struct ml_request *req = (struct ml_request *)obj;
+ struct ml_model *model = &t->model[req->fid];
+
+ RTE_SET_USED(mp);
+ RTE_SET_USED(obj_idx);
+
+ if (req->niters == 0)
+ return;
+
+ t->nb_used++;
+ rte_ml_io_dequantize(t->cmn.opt->dev_id, model->id, t->model[req->fid].info.batch_size,
+ req->output, model->output);
+}
+
+int
+ml_inference_result(struct ml_test *test, struct ml_options *opt, uint16_t fid)
+{
+ struct test_inference *t = ml_test_priv(test);
+ uint64_t error_count = 0;
+ uint32_t i;
+
+ RTE_SET_USED(opt);
+
+ /* check for errors */
+ for (i = 0; i < RTE_MAX_LCORE; i++)
+ error_count += t->error_count[i];
+
+ rte_mempool_obj_iter(t->model[fid].io_pool, ml_request_finish, test);
+
+ if ((t->nb_used > 0) && (error_count == 0))
+ t->cmn.result = ML_TEST_SUCCESS;
+ else
+ t->cmn.result = ML_TEST_FAILED;
+
+ return t->cmn.result;
+}
+
+int
+ml_inference_launch_cores(struct ml_test *test, struct ml_options *opt, uint16_t start_fid,
+ uint16_t end_fid)
+{
+ struct test_inference *t = ml_test_priv(test);
+ uint32_t lcore_id;
+ uint32_t id = 0;
+
+ RTE_LCORE_FOREACH_WORKER(lcore_id)
+ {
+ if (id == 2)
+ break;
+
+ t->args[lcore_id].nb_reqs = opt->repetitions;
+ t->args[lcore_id].start_fid = start_fid;
+ t->args[lcore_id].end_fid = end_fid;
+
+ if (id % 2 == 0)
+ rte_eal_remote_launch(t->enqueue, test, lcore_id);
+ else
+ rte_eal_remote_launch(t->dequeue, test, lcore_id);
+
+ id++;
+ }
+
+ return 0;
+}
diff --git a/app/test-mldev/test_inference_common.h b/app/test-mldev/test_inference_common.h
new file mode 100644
index 0000000000..abb20fc9fb
--- /dev/null
+++ b/app/test-mldev/test_inference_common.h
@@ -0,0 +1,61 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_TEST_INFERENCE_COMMON_
+#define _ML_TEST_INFERENCE_COMMON_
+
+#include <rte_common.h>
+#include <rte_mempool.h>
+#include <rte_mldev.h>
+
+#include "test_model_common.h"
+
+struct ml_request {
+ uint8_t *input;
+ uint8_t *output;
+ uint16_t fid;
+ uint64_t niters;
+};
+
+struct ml_core_args {
+ uint64_t nb_reqs;
+ uint16_t start_fid;
+ uint16_t end_fid;
+};
+
+struct test_inference {
+ /* common data */
+ struct test_common cmn;
+
+ /* test specific data */
+ struct ml_model model[ML_TEST_MAX_MODELS];
+ struct rte_mempool *op_pool;
+
+ uint64_t nb_used;
+ uint16_t fid;
+
+ int (*enqueue)(void *arg);
+ int (*dequeue)(void *arg);
+
+ struct ml_core_args args[RTE_MAX_LCORE];
+ uint64_t error_count[RTE_MAX_LCORE];
+} __rte_cache_aligned;
+
+bool test_inference_cap_check(struct ml_options *opt);
+int test_inference_opt_check(struct ml_options *opt);
+void test_inference_opt_dump(struct ml_options *opt);
+int test_inference_setup(struct ml_test *test, struct ml_options *opt);
+void test_inference_destroy(struct ml_test *test, struct ml_options *opt);
+
+int ml_inference_mldev_setup(struct ml_test *test, struct ml_options *opt);
+int ml_inference_mldev_destroy(struct ml_test *test, struct ml_options *opt);
+int ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, uint16_t fid);
+void ml_inference_iomem_destroy(struct ml_test *test, struct ml_options *opt, uint16_t fid);
+int ml_inference_mem_setup(struct ml_test *test, struct ml_options *opt);
+void ml_inference_mem_destroy(struct ml_test *test, struct ml_options *opt);
+int ml_inference_result(struct ml_test *test, struct ml_options *opt, uint16_t fid);
+int ml_inference_launch_cores(struct ml_test *test, struct ml_options *opt, uint16_t start_fid,
+ uint16_t end_fid);
+
+#endif /* _ML_TEST_INFERENCE_COMMON_ */
diff --git a/app/test-mldev/test_inference_ordered.c b/app/test-mldev/test_inference_ordered.c
new file mode 100644
index 0000000000..1cd91dc3d3
--- /dev/null
+++ b/app/test-mldev/test_inference_ordered.c
@@ -0,0 +1,115 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <rte_common.h>
+#include <rte_launch.h>
+
+#include "ml_common.h"
+#include "test_inference_common.h"
+
+static int
+test_inference_ordered_driver(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t;
+ uint16_t fid = 0;
+ int ret = 0;
+
+ t = ml_test_priv(test);
+
+ ret = ml_inference_mldev_setup(test, opt);
+ if (ret != 0)
+ return ret;
+
+ ret = ml_inference_mem_setup(test, opt);
+ if (ret != 0)
+ return ret;
+
+next_model:
+ /* load model */
+ ret = ml_model_load(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ /* start model */
+ ret = ml_model_start(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_inference_iomem_setup(test, opt, fid);
+ if (ret != 0)
+ goto error;
+
+ /* launch inferences for one model using available queue pairs */
+ ret = ml_inference_launch_cores(test, opt, fid, fid);
+ if (ret != 0) {
+ ml_err("failed to launch cores");
+ goto error;
+ }
+
+ rte_eal_mp_wait_lcore();
+
+ ret = ml_inference_result(test, opt, fid);
+ if (ret != ML_TEST_SUCCESS)
+ goto error;
+
+ ml_inference_iomem_destroy(test, opt, fid);
+
+ /* stop model */
+ ret = ml_model_stop(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ /* unload model */
+ ret = ml_model_unload(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ fid++;
+ if (fid < opt->nb_filelist)
+ goto next_model;
+
+ ml_inference_mem_destroy(test, opt);
+
+ ret = ml_inference_mldev_destroy(test, opt);
+ if (ret != 0)
+ return ret;
+
+ t->cmn.result = ML_TEST_SUCCESS;
+
+ return 0;
+
+error:
+ ml_inference_iomem_destroy(test, opt, fid);
+ ml_inference_mem_destroy(test, opt);
+ ml_model_stop(test, opt, &t->model[fid], fid);
+ ml_model_unload(test, opt, &t->model[fid], fid);
+
+ t->cmn.result = ML_TEST_FAILED;
+
+ return ret;
+}
+
+static int
+test_inference_ordered_result(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+
+ return t->cmn.result;
+}
+
+static const struct ml_test_ops inference_ordered = {
+ .cap_check = test_inference_cap_check,
+ .opt_check = test_inference_opt_check,
+ .opt_dump = test_inference_opt_dump,
+ .test_setup = test_inference_setup,
+ .test_destroy = test_inference_destroy,
+ .test_driver = test_inference_ordered_driver,
+ .test_result = test_inference_ordered_result,
+};
+
+ML_TEST_REGISTER(inference_ordered);
diff --git a/app/test-mldev/test_model_common.h b/app/test-mldev/test_model_common.h
index 74aec0a797..5ee975109d 100644
--- a/app/test-mldev/test_model_common.h
+++ b/app/test-mldev/test_model_common.h
@@ -20,6 +20,16 @@ struct ml_model {
uint16_t id;
struct rte_ml_model_info info;
enum model_state state;
+
+ uint64_t inp_dsize;
+ uint64_t inp_qsize;
+ uint64_t out_dsize;
+ uint64_t out_qsize;
+
+ uint8_t *input;
+ uint8_t *output;
+
+ struct rte_mempool *io_pool;
};
int ml_model_load(struct ml_test *test, struct ml_options *opt, struct ml_model *model,
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [PATCH v6 05/12] app/mldev: add ordered inference test case
2023-03-11 15:08 ` [PATCH v6 05/12] app/mldev: add ordered inference test case Srikanth Yalavarthi
@ 2023-03-16 17:45 ` Thomas Monjalon
2023-03-16 17:47 ` [EXT] " Srikanth Yalavarthi
0 siblings, 1 reply; 122+ messages in thread
From: Thomas Monjalon @ 2023-03-16 17:45 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
11/03/2023 16:08, Srikanth Yalavarthi:
> static struct option lgopts[] = {
> - {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0}, {ML_MODELS, 1, 0, 0},
> - {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
> + {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
> + {ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
> + {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
I don't understand the need to have multiple options per line.
When doing a list, it is simpler to read on item per line.
^ permalink raw reply [flat|nested] 122+ messages in thread* RE: [EXT] Re: [PATCH v6 05/12] app/mldev: add ordered inference test case
2023-03-16 17:45 ` Thomas Monjalon
@ 2023-03-16 17:47 ` Srikanth Yalavarthi
2023-03-16 18:01 ` Thomas Monjalon
0 siblings, 1 reply; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-16 17:47 UTC (permalink / raw)
To: Thomas Monjalon
Cc: dev, Shivah Shankar Shankar Narayan Rao,
Jerin Jacob Kollanukkaran, Anup Prabhu, Prince Takkar,
Parijat Shukla, Srikanth Yalavarthi
> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: 16 March 2023 23:15
> To: Srikanth Yalavarthi <syalavarthi@marvell.com>
> Cc: dev@dpdk.org; Shivah Shankar Shankar Narayan Rao
> <sshankarnara@marvell.com>; Jerin Jacob Kollanukkaran
> <jerinj@marvell.com>; Anup Prabhu <aprabhu@marvell.com>; Prince Takkar
> <ptakkar@marvell.com>; Parijat Shukla <pshukla@marvell.com>
> Subject: [EXT] Re: [PATCH v6 05/12] app/mldev: add ordered inference test
> case
>
> External Email
>
> ----------------------------------------------------------------------
> 11/03/2023 16:08, Srikanth Yalavarthi:
> > static struct option lgopts[] = {
> > - {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
> {ML_MODELS, 1, 0, 0},
> > - {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
> > + {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0,
> 0},
> > + {ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0,
> 0},
> > + {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
>
> I don't understand the need to have multiple options per line.
> When doing a list, it is simpler to read on item per line.
This is a result of running checkformat / clang-format tool. The tool is automatically arranging multiple options per line in the initial patches. The final patch would have one option per line.
>
>
>
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [EXT] Re: [PATCH v6 05/12] app/mldev: add ordered inference test case
2023-03-16 17:47 ` [EXT] " Srikanth Yalavarthi
@ 2023-03-16 18:01 ` Thomas Monjalon
2023-03-16 21:31 ` Srikanth Yalavarthi
0 siblings, 1 reply; 122+ messages in thread
From: Thomas Monjalon @ 2023-03-16 18:01 UTC (permalink / raw)
To: Srikanth Yalavarthi
Cc: dev, Shivah Shankar Shankar Narayan Rao,
Jerin Jacob Kollanukkaran, Anup Prabhu, Prince Takkar,
Parijat Shukla
16/03/2023 18:47, Srikanth Yalavarthi:
> From: Thomas Monjalon <thomas@monjalon.net>
> > 11/03/2023 16:08, Srikanth Yalavarthi:
> > > static struct option lgopts[] = {
> > > - {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
> > {ML_MODELS, 1, 0, 0},
> > > - {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
> > > + {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0,
> > 0},
> > > + {ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0,
> > 0},
> > > + {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
> >
> > I don't understand the need to have multiple options per line.
> > When doing a list, it is simpler to read on item per line.
>
> This is a result of running checkformat / clang-format tool. The tool is automatically arranging multiple options per line in the initial patches. The final patch would have one option per line.
Please do one option per line in each patch.
^ permalink raw reply [flat|nested] 122+ messages in thread* RE: [EXT] Re: [PATCH v6 05/12] app/mldev: add ordered inference test case
2023-03-16 18:01 ` Thomas Monjalon
@ 2023-03-16 21:31 ` Srikanth Yalavarthi
0 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-16 21:31 UTC (permalink / raw)
To: Thomas Monjalon
Cc: dev, Shivah Shankar Shankar Narayan Rao,
Jerin Jacob Kollanukkaran, Anup Prabhu, Prince Takkar,
Parijat Shukla, Srikanth Yalavarthi
> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: 16 March 2023 23:32
> To: Srikanth Yalavarthi <syalavarthi@marvell.com>
> Cc: dev@dpdk.org; Shivah Shankar Shankar Narayan Rao
> <sshankarnara@marvell.com>; Jerin Jacob Kollanukkaran
> <jerinj@marvell.com>; Anup Prabhu <aprabhu@marvell.com>; Prince Takkar
> <ptakkar@marvell.com>; Parijat Shukla <pshukla@marvell.com>; Srikanth
> Yalavarthi <syalavarthi@marvell.com>
> Subject: Re: [EXT] Re: [PATCH v6 05/12] app/mldev: add ordered inference
> test case
>
> 16/03/2023 18:47, Srikanth Yalavarthi:
> > From: Thomas Monjalon <thomas@monjalon.net>
> > > 11/03/2023 16:08, Srikanth Yalavarthi:
> > > > static struct option lgopts[] = {
> > > > - {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
> > > {ML_MODELS, 1, 0, 0},
> > > > - {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
> > > > + {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0,
> > > 0},
> > > > + {ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0,
> > > 0},
> > > > + {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
> > >
> > > I don't understand the need to have multiple options per line.
> > > When doing a list, it is simpler to read on item per line.
> >
> > This is a result of running checkformat / clang-format tool. The tool is
> automatically arranging multiple options per line in the initial patches. The
> final patch would have one option per line.
>
> Please do one option per line in each patch.
Done, changes are part of v7.
>
^ permalink raw reply [flat|nested] 122+ messages in thread
* [PATCH v6 06/12] app/mldev: add test case to interleave inferences
2023-03-11 15:08 ` [PATCH v6 00/12] Implementation of mldev test application Srikanth Yalavarthi
` (4 preceding siblings ...)
2023-03-11 15:08 ` [PATCH v6 05/12] app/mldev: add ordered inference test case Srikanth Yalavarthi
@ 2023-03-11 15:08 ` Srikanth Yalavarthi
2023-03-11 15:09 ` [PATCH v6 07/12] app/mldev: enable support for burst inferences Srikanth Yalavarthi
` (5 subsequent siblings)
11 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-11 15:08 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Added test case to interleave inference requests from multiple
models. Interleaving would load and start all models and launch
inference requests for the models using available queue-pairs
Operations sequence when testing with N models and R reps,
(load + start) x N -> (enqueue + dequeue) x N x R ...
-> (stop + unload) x N
Test can be executed by selecting "inference_interleave" test.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
Acked-by: Anup Prabhu <aprabhu@marvell.com>
---
app/test-mldev/meson.build | 1 +
app/test-mldev/ml_options.c | 3 +-
app/test-mldev/test_inference_interleave.c | 114 +++++++++++++++++++++
3 files changed, 117 insertions(+), 1 deletion(-)
create mode 100644 app/test-mldev/test_inference_interleave.c
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
index 475d76d126..41d22fb22c 100644
--- a/app/test-mldev/meson.build
+++ b/app/test-mldev/meson.build
@@ -18,6 +18,7 @@ sources = files(
'test_model_ops.c',
'test_inference_common.c',
'test_inference_ordered.c',
+ 'test_inference_interleave.c',
)
deps += ['mldev']
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index b7215a7e88..f9e3ce8e6f 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -156,7 +156,8 @@ ml_dump_test_options(const char *testname)
printf("\n");
}
- if (strcmp(testname, "inference_ordered") == 0) {
+ if ((strcmp(testname, "inference_ordered") == 0) ||
+ (strcmp(testname, "inference_interleave") == 0)) {
printf("\t\t--filelist : comma separated list of model, input and output\n"
"\t\t--repetitions : number of inference repetitions\n");
printf("\n");
diff --git a/app/test-mldev/test_inference_interleave.c b/app/test-mldev/test_inference_interleave.c
new file mode 100644
index 0000000000..9cf4cfa197
--- /dev/null
+++ b/app/test-mldev/test_inference_interleave.c
@@ -0,0 +1,114 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <rte_common.h>
+#include <rte_launch.h>
+
+#include "ml_common.h"
+#include "test_inference_common.h"
+
+static int
+test_inference_interleave_driver(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t;
+ uint16_t fid = 0;
+ int ret = 0;
+
+ t = ml_test_priv(test);
+
+ ret = ml_inference_mldev_setup(test, opt);
+ if (ret != 0)
+ return ret;
+
+ ret = ml_inference_mem_setup(test, opt);
+ if (ret != 0)
+ return ret;
+
+ /* load and start all models */
+ for (fid = 0; fid < opt->nb_filelist; fid++) {
+ ret = ml_model_load(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_start(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_inference_iomem_setup(test, opt, fid);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* launch inference requests */
+ ret = ml_inference_launch_cores(test, opt, 0, opt->nb_filelist - 1);
+ if (ret != 0) {
+ ml_err("failed to launch cores");
+ goto error;
+ }
+
+ rte_eal_mp_wait_lcore();
+
+ /* stop and unload all models */
+ for (fid = 0; fid < opt->nb_filelist; fid++) {
+ ret = ml_inference_result(test, opt, fid);
+ if (ret != ML_TEST_SUCCESS)
+ goto error;
+
+ ml_inference_iomem_destroy(test, opt, fid);
+
+ ret = ml_model_stop(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_unload(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+ }
+
+ ml_inference_mem_destroy(test, opt);
+
+ ret = ml_inference_mldev_destroy(test, opt);
+ if (ret != 0)
+ return ret;
+
+ t->cmn.result = ML_TEST_SUCCESS;
+
+ return 0;
+
+error:
+ ml_inference_mem_destroy(test, opt);
+ for (fid = 0; fid < opt->nb_filelist; fid++) {
+ ml_inference_iomem_destroy(test, opt, fid);
+ ml_model_stop(test, opt, &t->model[fid], fid);
+ ml_model_unload(test, opt, &t->model[fid], fid);
+ }
+
+ t->cmn.result = ML_TEST_FAILED;
+
+ return ret;
+}
+
+static int
+test_inference_interleave_result(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+
+ return t->cmn.result;
+}
+
+static const struct ml_test_ops inference_interleave = {
+ .cap_check = test_inference_cap_check,
+ .opt_check = test_inference_opt_check,
+ .opt_dump = test_inference_opt_dump,
+ .test_setup = test_inference_setup,
+ .test_destroy = test_inference_destroy,
+ .test_driver = test_inference_interleave_driver,
+ .test_result = test_inference_interleave_result,
+};
+
+ML_TEST_REGISTER(inference_interleave);
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v6 07/12] app/mldev: enable support for burst inferences
2023-03-11 15:08 ` [PATCH v6 00/12] Implementation of mldev test application Srikanth Yalavarthi
` (5 preceding siblings ...)
2023-03-11 15:08 ` [PATCH v6 06/12] app/mldev: add test case to interleave inferences Srikanth Yalavarthi
@ 2023-03-11 15:09 ` Srikanth Yalavarthi
2023-03-11 15:09 ` [PATCH v6 08/12] app/mldev: enable support for queue pairs and size Srikanth Yalavarthi
` (4 subsequent siblings)
11 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-11 15:09 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Added 'burst_size' support for inference tests. Burst size
controls the number of inference requests handled during
the burst enqueue and dequeue operations of the test case.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
Acked-by: Anup Prabhu <aprabhu@marvell.com>
---
app/test-mldev/ml_options.c | 26 ++--
app/test-mldev/ml_options.h | 2 +
app/test-mldev/test_inference_common.c | 160 ++++++++++++++++++++++++-
app/test-mldev/test_inference_common.h | 4 +
4 files changed, 182 insertions(+), 10 deletions(-)
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index f9e3ce8e6f..7fbf9c5678 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -24,6 +24,7 @@ ml_options_default(struct ml_options *opt)
opt->socket_id = SOCKET_ID_ANY;
opt->nb_filelist = 0;
opt->repetitions = 1;
+ opt->burst_size = 1;
opt->debug = false;
}
@@ -145,6 +146,12 @@ ml_parse_repetitions(struct ml_options *opt, const char *arg)
return parser_read_uint64(&opt->repetitions, arg);
}
+static int
+ml_parse_burst_size(struct ml_options *opt, const char *arg)
+{
+ return parser_read_uint16(&opt->burst_size, arg);
+}
+
static void
ml_dump_test_options(const char *testname)
{
@@ -159,7 +166,8 @@ ml_dump_test_options(const char *testname)
if ((strcmp(testname, "inference_ordered") == 0) ||
(strcmp(testname, "inference_interleave") == 0)) {
printf("\t\t--filelist : comma separated list of model, input and output\n"
- "\t\t--repetitions : number of inference repetitions\n");
+ "\t\t--repetitions : number of inference repetitions\n"
+ "\t\t--burst_size : inference burst size\n");
printf("\n");
}
}
@@ -179,10 +187,11 @@ print_usage(char *program)
ml_test_dump_names(ml_dump_test_options);
}
-static struct option lgopts[] = {
- {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
- {ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
- {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
+static struct option lgopts[] = {{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0},
+ {ML_SOCKET_ID, 1, 0, 0}, {ML_MODELS, 1, 0, 0},
+ {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
+ {ML_BURST_SIZE, 1, 0, 0}, {ML_DEBUG, 0, 0, 0},
+ {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
static int
ml_opts_parse_long(int opt_idx, struct ml_options *opt)
@@ -190,9 +199,10 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
unsigned int i;
struct long_opt_parser parsermap[] = {
- {ML_TEST, ml_parse_test_name}, {ML_DEVICE_ID, ml_parse_dev_id},
- {ML_SOCKET_ID, ml_parse_socket_id}, {ML_MODELS, ml_parse_models},
- {ML_FILELIST, ml_parse_filelist}, {ML_REPETITIONS, ml_parse_repetitions},
+ {ML_TEST, ml_parse_test_name}, {ML_DEVICE_ID, ml_parse_dev_id},
+ {ML_SOCKET_ID, ml_parse_socket_id}, {ML_MODELS, ml_parse_models},
+ {ML_FILELIST, ml_parse_filelist}, {ML_REPETITIONS, ml_parse_repetitions},
+ {ML_BURST_SIZE, ml_parse_burst_size},
};
for (i = 0; i < RTE_DIM(parsermap); i++) {
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index 6a13f97a30..00342d8a0c 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -18,6 +18,7 @@
#define ML_MODELS ("models")
#define ML_FILELIST ("filelist")
#define ML_REPETITIONS ("repetitions")
+#define ML_BURST_SIZE ("burst_size")
#define ML_DEBUG ("debug")
#define ML_HELP ("help")
@@ -34,6 +35,7 @@ struct ml_options {
struct ml_filelist filelist[ML_TEST_MAX_MODELS];
uint8_t nb_filelist;
uint64_t repetitions;
+ uint16_t burst_size;
bool debug;
};
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
index 6a6999d524..35323306de 100644
--- a/app/test-mldev/test_inference_common.c
+++ b/app/test-mldev/test_inference_common.c
@@ -124,6 +124,132 @@ ml_dequeue_single(void *arg)
return 0;
}
+/* Enqueue inference requests with burst size greater than 1 */
+static int
+ml_enqueue_burst(void *arg)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)arg);
+ struct ml_core_args *args;
+ uint16_t ops_count;
+ uint64_t model_enq;
+ uint16_t burst_enq;
+ uint32_t lcore_id;
+ uint16_t pending;
+ uint16_t idx;
+ uint16_t fid;
+ uint16_t i;
+ int ret;
+
+ lcore_id = rte_lcore_id();
+ args = &t->args[lcore_id];
+ model_enq = 0;
+
+ if (args->nb_reqs == 0)
+ return 0;
+
+next_rep:
+ fid = args->start_fid;
+
+next_model:
+ ops_count = RTE_MIN(t->cmn.opt->burst_size, args->nb_reqs - model_enq);
+ ret = rte_mempool_get_bulk(t->op_pool, (void **)args->enq_ops, ops_count);
+ if (ret != 0)
+ goto next_model;
+
+retry:
+ ret = rte_mempool_get_bulk(t->model[fid].io_pool, (void **)args->reqs, ops_count);
+ if (ret != 0)
+ goto retry;
+
+ for (i = 0; i < ops_count; i++) {
+ args->enq_ops[i]->model_id = t->model[fid].id;
+ args->enq_ops[i]->nb_batches = t->model[fid].info.batch_size;
+ args->enq_ops[i]->mempool = t->op_pool;
+
+ args->enq_ops[i]->input.addr = args->reqs[i]->input;
+ args->enq_ops[i]->input.length = t->model[fid].inp_qsize;
+ args->enq_ops[i]->input.next = NULL;
+
+ args->enq_ops[i]->output.addr = args->reqs[i]->output;
+ args->enq_ops[i]->output.length = t->model[fid].out_qsize;
+ args->enq_ops[i]->output.next = NULL;
+
+ args->enq_ops[i]->user_ptr = args->reqs[i];
+ args->reqs[i]->niters++;
+ args->reqs[i]->fid = fid;
+ }
+
+ idx = 0;
+ pending = ops_count;
+
+enqueue_reqs:
+ burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, 0, &args->enq_ops[idx], pending);
+ pending = pending - burst_enq;
+
+ if (pending > 0) {
+ idx = idx + burst_enq;
+ goto enqueue_reqs;
+ }
+
+ fid++;
+ if (fid <= args->end_fid)
+ goto next_model;
+
+ model_enq = model_enq + ops_count;
+ if (model_enq < args->nb_reqs)
+ goto next_rep;
+
+ return 0;
+}
+
+/* Dequeue inference requests with burst size greater than 1 */
+static int
+ml_dequeue_burst(void *arg)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)arg);
+ struct rte_ml_op_error error;
+ struct ml_core_args *args;
+ struct ml_request *req;
+ uint64_t total_deq = 0;
+ uint16_t burst_deq = 0;
+ uint8_t nb_filelist;
+ uint32_t lcore_id;
+ uint32_t i;
+
+ lcore_id = rte_lcore_id();
+ args = &t->args[lcore_id];
+ nb_filelist = args->end_fid - args->start_fid + 1;
+
+ if (args->nb_reqs == 0)
+ return 0;
+
+dequeue_burst:
+ burst_deq =
+ rte_ml_dequeue_burst(t->cmn.opt->dev_id, 0, args->deq_ops, t->cmn.opt->burst_size);
+
+ if (likely(burst_deq > 0)) {
+ total_deq += burst_deq;
+
+ for (i = 0; i < burst_deq; i++) {
+ if (unlikely(args->deq_ops[i]->status == RTE_ML_OP_STATUS_ERROR)) {
+ rte_ml_op_error_get(t->cmn.opt->dev_id, args->deq_ops[i], &error);
+ ml_err("error_code = 0x%" PRIx64 ", error_message = %s\n",
+ error.errcode, error.message);
+ t->error_count[lcore_id]++;
+ }
+ req = (struct ml_request *)args->deq_ops[i]->user_ptr;
+ if (req != NULL)
+ rte_mempool_put(t->model[req->fid].io_pool, req);
+ }
+ rte_mempool_put_bulk(t->op_pool, (void *)args->deq_ops, burst_deq);
+ }
+
+ if (total_deq < args->nb_reqs * nb_filelist)
+ goto dequeue_burst;
+
+ return 0;
+}
+
bool
test_inference_cap_check(struct ml_options *opt)
{
@@ -173,6 +299,17 @@ test_inference_opt_check(struct ml_options *opt)
return -EINVAL;
}
+ if (opt->burst_size == 0) {
+ ml_err("Invalid option, burst_size = %u\n", opt->burst_size);
+ return -EINVAL;
+ }
+
+ if (opt->burst_size > ML_TEST_MAX_POOL_SIZE) {
+ ml_err("Invalid option, burst_size = %u (> max supported = %d)\n", opt->burst_size,
+ ML_TEST_MAX_POOL_SIZE);
+ return -EINVAL;
+ }
+
/* check number of available lcores. */
if (rte_lcore_count() < 3) {
ml_err("Insufficient lcores = %u\n", rte_lcore_count());
@@ -193,6 +330,7 @@ test_inference_opt_dump(struct ml_options *opt)
/* dump test opts */
ml_dump("repetitions", "%" PRIu64, opt->repetitions);
+ ml_dump("burst_size", "%u", opt->burst_size);
ml_dump_begin("filelist");
for (i = 0; i < opt->nb_filelist; i++) {
@@ -208,6 +346,7 @@ test_inference_setup(struct ml_test *test, struct ml_options *opt)
{
struct test_inference *t;
void *test_inference;
+ uint32_t lcore_id;
int ret = 0;
uint32_t i;
@@ -233,13 +372,30 @@ test_inference_setup(struct ml_test *test, struct ml_options *opt)
goto error;
}
- t->enqueue = ml_enqueue_single;
- t->dequeue = ml_dequeue_single;
+ if (opt->burst_size == 1) {
+ t->enqueue = ml_enqueue_single;
+ t->dequeue = ml_dequeue_single;
+ } else {
+ t->enqueue = ml_enqueue_burst;
+ t->dequeue = ml_dequeue_burst;
+ }
/* set model initial state */
for (i = 0; i < opt->nb_filelist; i++)
t->model[i].state = MODEL_INITIAL;
+ for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) {
+ t->args[lcore_id].enq_ops = rte_zmalloc_socket(
+ "ml_test_enq_ops", opt->burst_size * sizeof(struct rte_ml_op *),
+ RTE_CACHE_LINE_SIZE, opt->socket_id);
+ t->args[lcore_id].deq_ops = rte_zmalloc_socket(
+ "ml_test_deq_ops", opt->burst_size * sizeof(struct rte_ml_op *),
+ RTE_CACHE_LINE_SIZE, opt->socket_id);
+ t->args[lcore_id].reqs = rte_zmalloc_socket(
+ "ml_test_requests", opt->burst_size * sizeof(struct ml_request *),
+ RTE_CACHE_LINE_SIZE, opt->socket_id);
+ }
+
return 0;
error:
diff --git a/app/test-mldev/test_inference_common.h b/app/test-mldev/test_inference_common.h
index abb20fc9fb..da800f2bd4 100644
--- a/app/test-mldev/test_inference_common.h
+++ b/app/test-mldev/test_inference_common.h
@@ -22,6 +22,10 @@ struct ml_core_args {
uint64_t nb_reqs;
uint16_t start_fid;
uint16_t end_fid;
+
+ struct rte_ml_op **enq_ops;
+ struct rte_ml_op **deq_ops;
+ struct ml_request **reqs;
};
struct test_inference {
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v6 08/12] app/mldev: enable support for queue pairs and size
2023-03-11 15:08 ` [PATCH v6 00/12] Implementation of mldev test application Srikanth Yalavarthi
` (6 preceding siblings ...)
2023-03-11 15:09 ` [PATCH v6 07/12] app/mldev: enable support for burst inferences Srikanth Yalavarthi
@ 2023-03-11 15:09 ` Srikanth Yalavarthi
2023-03-11 15:09 ` [PATCH v6 09/12] app/mldev: enable support for inference batches Srikanth Yalavarthi
` (3 subsequent siblings)
11 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-11 15:09 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Added support to create multiple queue-pairs per device to
enqueue and dequeue inference requests. Number of queue pairs
to be created can be specified through "--queue_pairs" option.
Support is also enabled to control the number of descriptors
per each queue pair through "--queue_size" option. Inference
requests for a model are distributed across all available
queue-pairs.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
Acked-by: Anup Prabhu <aprabhu@marvell.com>
---
app/test-mldev/ml_options.c | 40 ++++++++++---
app/test-mldev/ml_options.h | 4 ++
app/test-mldev/test_common.c | 2 +-
app/test-mldev/test_inference_common.c | 79 +++++++++++++++++++++-----
app/test-mldev/test_inference_common.h | 1 +
5 files changed, 102 insertions(+), 24 deletions(-)
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 7fbf9c5678..c81dec6e30 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -25,6 +25,8 @@ ml_options_default(struct ml_options *opt)
opt->nb_filelist = 0;
opt->repetitions = 1;
opt->burst_size = 1;
+ opt->queue_pairs = 1;
+ opt->queue_size = 1;
opt->debug = false;
}
@@ -152,11 +154,30 @@ ml_parse_burst_size(struct ml_options *opt, const char *arg)
return parser_read_uint16(&opt->burst_size, arg);
}
+static int
+ml_parse_queue_pairs(struct ml_options *opt, const char *arg)
+{
+ int ret;
+
+ ret = parser_read_uint16(&opt->queue_pairs, arg);
+
+ return ret;
+}
+
+static int
+ml_parse_queue_size(struct ml_options *opt, const char *arg)
+{
+ return parser_read_uint16(&opt->queue_size, arg);
+}
+
static void
ml_dump_test_options(const char *testname)
{
- if (strcmp(testname, "device_ops") == 0)
+ if (strcmp(testname, "device_ops") == 0) {
+ printf("\t\t--queue_pairs : number of queue pairs to create\n"
+ "\t\t--queue_size : size fo queue-pair\n");
printf("\n");
+ }
if (strcmp(testname, "model_ops") == 0) {
printf("\t\t--models : comma separated list of models\n");
@@ -167,7 +188,9 @@ ml_dump_test_options(const char *testname)
(strcmp(testname, "inference_interleave") == 0)) {
printf("\t\t--filelist : comma separated list of model, input and output\n"
"\t\t--repetitions : number of inference repetitions\n"
- "\t\t--burst_size : inference burst size\n");
+ "\t\t--burst_size : inference burst size\n"
+ "\t\t--queue_pairs : number of queue pairs to create\n"
+ "\t\t--queue_size : size fo queue-pair\n");
printf("\n");
}
}
@@ -187,11 +210,11 @@ print_usage(char *program)
ml_test_dump_names(ml_dump_test_options);
}
-static struct option lgopts[] = {{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0},
- {ML_SOCKET_ID, 1, 0, 0}, {ML_MODELS, 1, 0, 0},
- {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
- {ML_BURST_SIZE, 1, 0, 0}, {ML_DEBUG, 0, 0, 0},
- {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
+static struct option lgopts[] = {
+ {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
+ {ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
+ {ML_BURST_SIZE, 1, 0, 0}, {ML_QUEUE_PAIRS, 1, 0, 0}, {ML_QUEUE_SIZE, 1, 0, 0},
+ {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
static int
ml_opts_parse_long(int opt_idx, struct ml_options *opt)
@@ -202,7 +225,8 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
{ML_TEST, ml_parse_test_name}, {ML_DEVICE_ID, ml_parse_dev_id},
{ML_SOCKET_ID, ml_parse_socket_id}, {ML_MODELS, ml_parse_models},
{ML_FILELIST, ml_parse_filelist}, {ML_REPETITIONS, ml_parse_repetitions},
- {ML_BURST_SIZE, ml_parse_burst_size},
+ {ML_BURST_SIZE, ml_parse_burst_size}, {ML_QUEUE_PAIRS, ml_parse_queue_pairs},
+ {ML_QUEUE_SIZE, ml_parse_queue_size},
};
for (i = 0; i < RTE_DIM(parsermap); i++) {
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index 00342d8a0c..c4018ee9d1 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -19,6 +19,8 @@
#define ML_FILELIST ("filelist")
#define ML_REPETITIONS ("repetitions")
#define ML_BURST_SIZE ("burst_size")
+#define ML_QUEUE_PAIRS ("queue_pairs")
+#define ML_QUEUE_SIZE ("queue_size")
#define ML_DEBUG ("debug")
#define ML_HELP ("help")
@@ -36,6 +38,8 @@ struct ml_options {
uint8_t nb_filelist;
uint64_t repetitions;
uint16_t burst_size;
+ uint16_t queue_pairs;
+ uint16_t queue_size;
bool debug;
};
diff --git a/app/test-mldev/test_common.c b/app/test-mldev/test_common.c
index 8c4da4609a..016b31c6ba 100644
--- a/app/test-mldev/test_common.c
+++ b/app/test-mldev/test_common.c
@@ -75,7 +75,7 @@ ml_test_device_configure(struct ml_test *test, struct ml_options *opt)
/* configure device */
dev_config.socket_id = opt->socket_id;
dev_config.nb_models = t->dev_info.max_models;
- dev_config.nb_queue_pairs = t->dev_info.max_queue_pairs;
+ dev_config.nb_queue_pairs = opt->queue_pairs;
ret = rte_ml_dev_configure(opt->dev_id, &dev_config);
if (ret != 0) {
ml_err("Failed to configure ml device, dev_id = %d\n", opt->dev_id);
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
index 35323306de..b4ad3c4b72 100644
--- a/app/test-mldev/test_inference_common.c
+++ b/app/test-mldev/test_inference_common.c
@@ -66,7 +66,7 @@ ml_enqueue_single(void *arg)
req->fid = fid;
enqueue_req:
- burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, 0, &op, 1);
+ burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, args->qp_id, &op, 1);
if (burst_enq == 0)
goto enqueue_req;
@@ -103,7 +103,7 @@ ml_dequeue_single(void *arg)
return 0;
dequeue_req:
- burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, 0, &op, 1);
+ burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, args->qp_id, &op, 1);
if (likely(burst_deq == 1)) {
total_deq += burst_deq;
@@ -183,7 +183,8 @@ ml_enqueue_burst(void *arg)
pending = ops_count;
enqueue_reqs:
- burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, 0, &args->enq_ops[idx], pending);
+ burst_enq =
+ rte_ml_enqueue_burst(t->cmn.opt->dev_id, args->qp_id, &args->enq_ops[idx], pending);
pending = pending - burst_enq;
if (pending > 0) {
@@ -224,8 +225,8 @@ ml_dequeue_burst(void *arg)
return 0;
dequeue_burst:
- burst_deq =
- rte_ml_dequeue_burst(t->cmn.opt->dev_id, 0, args->deq_ops, t->cmn.opt->burst_size);
+ burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, args->qp_id, args->deq_ops,
+ t->cmn.opt->burst_size);
if (likely(burst_deq > 0)) {
total_deq += burst_deq;
@@ -259,6 +260,19 @@ test_inference_cap_check(struct ml_options *opt)
return false;
rte_ml_dev_info_get(opt->dev_id, &dev_info);
+
+ if (opt->queue_pairs > dev_info.max_queue_pairs) {
+ ml_err("Insufficient capabilities: queue_pairs = %u, max_queue_pairs = %u",
+ opt->queue_pairs, dev_info.max_queue_pairs);
+ return false;
+ }
+
+ if (opt->queue_size > dev_info.max_desc) {
+ ml_err("Insufficient capabilities: queue_size = %u, max_desc = %u", opt->queue_size,
+ dev_info.max_desc);
+ return false;
+ }
+
if (opt->nb_filelist > dev_info.max_models) {
ml_err("Insufficient capabilities: Filelist count exceeded device limit, count = %u (max limit = %u)",
opt->nb_filelist, dev_info.max_models);
@@ -310,10 +324,21 @@ test_inference_opt_check(struct ml_options *opt)
return -EINVAL;
}
+ if (opt->queue_pairs == 0) {
+ ml_err("Invalid option, queue_pairs = %u\n", opt->queue_pairs);
+ return -EINVAL;
+ }
+
+ if (opt->queue_size == 0) {
+ ml_err("Invalid option, queue_size = %u\n", opt->queue_size);
+ return -EINVAL;
+ }
+
/* check number of available lcores. */
- if (rte_lcore_count() < 3) {
+ if (rte_lcore_count() < (uint32_t)(opt->queue_pairs * 2 + 1)) {
ml_err("Insufficient lcores = %u\n", rte_lcore_count());
- ml_err("Minimum lcores required to create %u queue-pairs = %u\n", 1, 3);
+ ml_err("Minimum lcores required to create %u queue-pairs = %u\n", opt->queue_pairs,
+ (opt->queue_pairs * 2 + 1));
return -EINVAL;
}
@@ -331,6 +356,8 @@ test_inference_opt_dump(struct ml_options *opt)
/* dump test opts */
ml_dump("repetitions", "%" PRIu64, opt->repetitions);
ml_dump("burst_size", "%u", opt->burst_size);
+ ml_dump("queue_pairs", "%u", opt->queue_pairs);
+ ml_dump("queue_size", "%u", opt->queue_size);
ml_dump_begin("filelist");
for (i = 0; i < opt->nb_filelist; i++) {
@@ -422,23 +449,31 @@ ml_inference_mldev_setup(struct ml_test *test, struct ml_options *opt)
{
struct rte_ml_dev_qp_conf qp_conf;
struct test_inference *t;
+ uint16_t qp_id;
int ret;
t = ml_test_priv(test);
+ RTE_SET_USED(t);
+
ret = ml_test_device_configure(test, opt);
if (ret != 0)
return ret;
/* setup queue pairs */
- qp_conf.nb_desc = t->cmn.dev_info.max_desc;
+ qp_conf.nb_desc = opt->queue_size;
qp_conf.cb = NULL;
- ret = rte_ml_dev_queue_pair_setup(opt->dev_id, 0, &qp_conf, opt->socket_id);
- if (ret != 0) {
- ml_err("Failed to setup ml device queue-pair, dev_id = %d, qp_id = %u\n",
- opt->dev_id, 0);
- goto error;
+ for (qp_id = 0; qp_id < opt->queue_pairs; qp_id++) {
+ qp_conf.nb_desc = opt->queue_size;
+ qp_conf.cb = NULL;
+
+ ret = rte_ml_dev_queue_pair_setup(opt->dev_id, qp_id, &qp_conf, opt->socket_id);
+ if (ret != 0) {
+ ml_err("Failed to setup ml device queue-pair, dev_id = %d, qp_id = %u\n",
+ opt->dev_id, qp_id);
+ return ret;
+ }
}
ret = ml_test_device_start(test, opt);
@@ -700,14 +735,28 @@ ml_inference_launch_cores(struct ml_test *test, struct ml_options *opt, uint16_t
{
struct test_inference *t = ml_test_priv(test);
uint32_t lcore_id;
+ uint32_t nb_reqs;
uint32_t id = 0;
+ uint32_t qp_id;
+
+ nb_reqs = opt->repetitions / opt->queue_pairs;
RTE_LCORE_FOREACH_WORKER(lcore_id)
{
- if (id == 2)
+ if (id >= opt->queue_pairs * 2)
break;
- t->args[lcore_id].nb_reqs = opt->repetitions;
+ qp_id = id / 2;
+ t->args[lcore_id].qp_id = qp_id;
+ t->args[lcore_id].nb_reqs = nb_reqs;
+ if (qp_id == 0)
+ t->args[lcore_id].nb_reqs += opt->repetitions - nb_reqs * opt->queue_pairs;
+
+ if (t->args[lcore_id].nb_reqs == 0) {
+ id++;
+ break;
+ }
+
t->args[lcore_id].start_fid = start_fid;
t->args[lcore_id].end_fid = end_fid;
diff --git a/app/test-mldev/test_inference_common.h b/app/test-mldev/test_inference_common.h
index da800f2bd4..81d9b07d41 100644
--- a/app/test-mldev/test_inference_common.h
+++ b/app/test-mldev/test_inference_common.h
@@ -22,6 +22,7 @@ struct ml_core_args {
uint64_t nb_reqs;
uint16_t start_fid;
uint16_t end_fid;
+ uint32_t qp_id;
struct rte_ml_op **enq_ops;
struct rte_ml_op **deq_ops;
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v6 09/12] app/mldev: enable support for inference batches
2023-03-11 15:08 ` [PATCH v6 00/12] Implementation of mldev test application Srikanth Yalavarthi
` (7 preceding siblings ...)
2023-03-11 15:09 ` [PATCH v6 08/12] app/mldev: enable support for queue pairs and size Srikanth Yalavarthi
@ 2023-03-11 15:09 ` Srikanth Yalavarthi
2023-03-16 17:47 ` Thomas Monjalon
2023-03-11 15:09 ` [PATCH v6 10/12] app/mldev: enable support for inference validation Srikanth Yalavarthi
` (2 subsequent siblings)
11 siblings, 1 reply; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-11 15:09 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Enabled support to execute multiple batches of inferences
per each enqueue request. Input and reference for the test
should be appropriately provided for multi-batch run. Number
of batches can be specified through "--batches" option.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
Acked-by: Anup Prabhu <aprabhu@marvell.com>
---
app/test-mldev/ml_options.c | 15 ++++++++++++---
app/test-mldev/ml_options.h | 2 ++
app/test-mldev/test_inference_common.c | 22 +++++++++++++---------
app/test-mldev/test_model_common.c | 6 ++++++
app/test-mldev/test_model_common.h | 1 +
5 files changed, 34 insertions(+), 12 deletions(-)
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index c81dec6e30..499bfde899 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -27,6 +27,7 @@ ml_options_default(struct ml_options *opt)
opt->burst_size = 1;
opt->queue_pairs = 1;
opt->queue_size = 1;
+ opt->batches = 0;
opt->debug = false;
}
@@ -170,6 +171,12 @@ ml_parse_queue_size(struct ml_options *opt, const char *arg)
return parser_read_uint16(&opt->queue_size, arg);
}
+static int
+ml_parse_batches(struct ml_options *opt, const char *arg)
+{
+ return parser_read_uint16(&opt->batches, arg);
+}
+
static void
ml_dump_test_options(const char *testname)
{
@@ -190,7 +197,8 @@ ml_dump_test_options(const char *testname)
"\t\t--repetitions : number of inference repetitions\n"
"\t\t--burst_size : inference burst size\n"
"\t\t--queue_pairs : number of queue pairs to create\n"
- "\t\t--queue_size : size fo queue-pair\n");
+ "\t\t--queue_size : size fo queue-pair\n"
+ "\t\t--batches : number of batches of input\n");
printf("\n");
}
}
@@ -214,7 +222,8 @@ static struct option lgopts[] = {
{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
{ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
{ML_BURST_SIZE, 1, 0, 0}, {ML_QUEUE_PAIRS, 1, 0, 0}, {ML_QUEUE_SIZE, 1, 0, 0},
- {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
+ {ML_BATCHES, 1, 0, 0}, {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0},
+ {NULL, 0, 0, 0}};
static int
ml_opts_parse_long(int opt_idx, struct ml_options *opt)
@@ -226,7 +235,7 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
{ML_SOCKET_ID, ml_parse_socket_id}, {ML_MODELS, ml_parse_models},
{ML_FILELIST, ml_parse_filelist}, {ML_REPETITIONS, ml_parse_repetitions},
{ML_BURST_SIZE, ml_parse_burst_size}, {ML_QUEUE_PAIRS, ml_parse_queue_pairs},
- {ML_QUEUE_SIZE, ml_parse_queue_size},
+ {ML_QUEUE_SIZE, ml_parse_queue_size}, {ML_BATCHES, ml_parse_batches},
};
for (i = 0; i < RTE_DIM(parsermap); i++) {
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index c4018ee9d1..48fe064150 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -21,6 +21,7 @@
#define ML_BURST_SIZE ("burst_size")
#define ML_QUEUE_PAIRS ("queue_pairs")
#define ML_QUEUE_SIZE ("queue_size")
+#define ML_BATCHES ("batches")
#define ML_DEBUG ("debug")
#define ML_HELP ("help")
@@ -40,6 +41,7 @@ struct ml_options {
uint16_t burst_size;
uint16_t queue_pairs;
uint16_t queue_size;
+ uint16_t batches;
bool debug;
};
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
index b4ad3c4b72..0f281aed6c 100644
--- a/app/test-mldev/test_inference_common.c
+++ b/app/test-mldev/test_inference_common.c
@@ -50,7 +50,7 @@ ml_enqueue_single(void *arg)
goto retry;
op->model_id = t->model[fid].id;
- op->nb_batches = t->model[fid].info.batch_size;
+ op->nb_batches = t->model[fid].nb_batches;
op->mempool = t->op_pool;
op->input.addr = req->input;
@@ -163,7 +163,7 @@ ml_enqueue_burst(void *arg)
for (i = 0; i < ops_count; i++) {
args->enq_ops[i]->model_id = t->model[fid].id;
- args->enq_ops[i]->nb_batches = t->model[fid].info.batch_size;
+ args->enq_ops[i]->nb_batches = t->model[fid].nb_batches;
args->enq_ops[i]->mempool = t->op_pool;
args->enq_ops[i]->input.addr = args->reqs[i]->input;
@@ -359,6 +359,11 @@ test_inference_opt_dump(struct ml_options *opt)
ml_dump("queue_pairs", "%u", opt->queue_pairs);
ml_dump("queue_size", "%u", opt->queue_size);
+ if (opt->batches == 0)
+ ml_dump("batches", "%u (default)", opt->batches);
+ else
+ ml_dump("batches", "%u", opt->batches);
+
ml_dump_begin("filelist");
for (i = 0; i < opt->nb_filelist; i++) {
ml_dump_list("model", i, opt->filelist[i].model);
@@ -528,8 +533,8 @@ ml_request_initialize(struct rte_mempool *mp, void *opaque, void *obj, unsigned
req->niters = 0;
/* quantize data */
- rte_ml_io_quantize(t->cmn.opt->dev_id, t->model[t->fid].id,
- t->model[t->fid].info.batch_size, t->model[t->fid].input, req->input);
+ rte_ml_io_quantize(t->cmn.opt->dev_id, t->model[t->fid].id, t->model[t->fid].nb_batches,
+ t->model[t->fid].input, req->input);
}
int
@@ -547,7 +552,7 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, uint16_t
int ret;
/* get input buffer size */
- ret = rte_ml_io_input_size_get(opt->dev_id, t->model[fid].id, t->model[fid].info.batch_size,
+ ret = rte_ml_io_input_size_get(opt->dev_id, t->model[fid].id, t->model[fid].nb_batches,
&t->model[fid].inp_qsize, &t->model[fid].inp_dsize);
if (ret != 0) {
ml_err("Failed to get input size, model : %s\n", opt->filelist[fid].model);
@@ -555,9 +560,8 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, uint16_t
}
/* get output buffer size */
- ret = rte_ml_io_output_size_get(opt->dev_id, t->model[fid].id,
- t->model[fid].info.batch_size, &t->model[fid].out_qsize,
- &t->model[fid].out_dsize);
+ ret = rte_ml_io_output_size_get(opt->dev_id, t->model[fid].id, t->model[fid].nb_batches,
+ &t->model[fid].out_qsize, &t->model[fid].out_dsize);
if (ret != 0) {
ml_err("Failed to get input size, model : %s\n", opt->filelist[fid].model);
return ret;
@@ -702,7 +706,7 @@ ml_request_finish(struct rte_mempool *mp, void *opaque, void *obj, unsigned int
return;
t->nb_used++;
- rte_ml_io_dequantize(t->cmn.opt->dev_id, model->id, t->model[req->fid].info.batch_size,
+ rte_ml_io_dequantize(t->cmn.opt->dev_id, model->id, t->model[req->fid].nb_batches,
req->output, model->output);
}
diff --git a/app/test-mldev/test_model_common.c b/app/test-mldev/test_model_common.c
index b94d46154d..c28e452f29 100644
--- a/app/test-mldev/test_model_common.c
+++ b/app/test-mldev/test_model_common.c
@@ -71,6 +71,12 @@ ml_model_load(struct ml_test *test, struct ml_options *opt, struct ml_model *mod
return ret;
}
+ /* Update number of batches */
+ if (opt->batches == 0)
+ model->nb_batches = model->info.batch_size;
+ else
+ model->nb_batches = opt->batches;
+
model->state = MODEL_LOADED;
return 0;
diff --git a/app/test-mldev/test_model_common.h b/app/test-mldev/test_model_common.h
index 5ee975109d..19429ce142 100644
--- a/app/test-mldev/test_model_common.h
+++ b/app/test-mldev/test_model_common.h
@@ -30,6 +30,7 @@ struct ml_model {
uint8_t *output;
struct rte_mempool *io_pool;
+ uint32_t nb_batches;
};
int ml_model_load(struct ml_test *test, struct ml_options *opt, struct ml_model *model,
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [PATCH v6 09/12] app/mldev: enable support for inference batches
2023-03-11 15:09 ` [PATCH v6 09/12] app/mldev: enable support for inference batches Srikanth Yalavarthi
@ 2023-03-16 17:47 ` Thomas Monjalon
2023-03-16 17:52 ` [EXT] " Srikanth Yalavarthi
0 siblings, 1 reply; 122+ messages in thread
From: Thomas Monjalon @ 2023-03-16 17:47 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
11/03/2023 16:09, Srikanth Yalavarthi:
> @@ -528,8 +533,8 @@ ml_request_initialize(struct rte_mempool *mp, void *opaque, void *obj, unsigned
> req->niters = 0;
>
> /* quantize data */
> - rte_ml_io_quantize(t->cmn.opt->dev_id, t->model[t->fid].id,
> - t->model[t->fid].info.batch_size, t->model[t->fid].input, req->input);
> + rte_ml_io_quantize(t->cmn.opt->dev_id, t->model[t->fid].id, t->model[t->fid].nb_batches,
> + t->model[t->fid].input, req->input);
> }
>
> int
> @@ -547,7 +552,7 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, uint16_t
> int ret;
>
> /* get input buffer size */
> - ret = rte_ml_io_input_size_get(opt->dev_id, t->model[fid].id, t->model[fid].info.batch_size,
> + ret = rte_ml_io_input_size_get(opt->dev_id, t->model[fid].id, t->model[fid].nb_batches,
> &t->model[fid].inp_qsize, &t->model[fid].inp_dsize);
> if (ret != 0) {
> ml_err("Failed to get input size, model : %s\n", opt->filelist[fid].model);
> @@ -555,9 +560,8 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, uint16_t
> }
>
> /* get output buffer size */
> - ret = rte_ml_io_output_size_get(opt->dev_id, t->model[fid].id,
> - t->model[fid].info.batch_size, &t->model[fid].out_qsize,
> - &t->model[fid].out_dsize);
> + ret = rte_ml_io_output_size_get(opt->dev_id, t->model[fid].id, t->model[fid].nb_batches,
> + &t->model[fid].out_qsize, &t->model[fid].out_dsize);
> if (ret != 0) {
> ml_err("Failed to get input size, model : %s\n", opt->filelist[fid].model);
> return ret;
> @@ -702,7 +706,7 @@ ml_request_finish(struct rte_mempool *mp, void *opaque, void *obj, unsigned int
> return;
>
> t->nb_used++;
> - rte_ml_io_dequantize(t->cmn.opt->dev_id, model->id, t->model[req->fid].info.batch_size,
> + rte_ml_io_dequantize(t->cmn.opt->dev_id, model->id, t->model[req->fid].nb_batches,
> req->output, model->output);
These changes look unrelated with the topic of the patch.
You should probably fix it when adding those lines at first.
^ permalink raw reply [flat|nested] 122+ messages in thread* RE: [EXT] Re: [PATCH v6 09/12] app/mldev: enable support for inference batches
2023-03-16 17:47 ` Thomas Monjalon
@ 2023-03-16 17:52 ` Srikanth Yalavarthi
0 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-16 17:52 UTC (permalink / raw)
To: Thomas Monjalon
Cc: dev, Shivah Shankar Shankar Narayan Rao,
Jerin Jacob Kollanukkaran, Anup Prabhu, Prince Takkar,
Parijat Shukla, Srikanth Yalavarthi
> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: 16 March 2023 23:18
> To: Srikanth Yalavarthi <syalavarthi@marvell.com>
> Cc: dev@dpdk.org; Shivah Shankar Shankar Narayan Rao
> <sshankarnara@marvell.com>; Jerin Jacob Kollanukkaran
> <jerinj@marvell.com>; Anup Prabhu <aprabhu@marvell.com>; Prince Takkar
> <ptakkar@marvell.com>; Parijat Shukla <pshukla@marvell.com>
> Subject: [EXT] Re: [PATCH v6 09/12] app/mldev: enable support for inference
> batches
>
> External Email
>
> ----------------------------------------------------------------------
> 11/03/2023 16:09, Srikanth Yalavarthi:
> > @@ -528,8 +533,8 @@ ml_request_initialize(struct rte_mempool *mp,
> void *opaque, void *obj, unsigned
> > req->niters = 0;
> >
> > /* quantize data */
> > - rte_ml_io_quantize(t->cmn.opt->dev_id, t->model[t->fid].id,
> > - t->model[t->fid].info.batch_size, t->model[t-
> >fid].input, req->input);
> > + rte_ml_io_quantize(t->cmn.opt->dev_id, t->model[t->fid].id, t-
> >model[t->fid].nb_batches,
> > + t->model[t->fid].input, req->input);
> > }
> >
> > int
> > @@ -547,7 +552,7 @@ ml_inference_iomem_setup(struct ml_test *test,
> struct ml_options *opt, uint16_t
> > int ret;
> >
> > /* get input buffer size */
> > - ret = rte_ml_io_input_size_get(opt->dev_id, t->model[fid].id, t-
> >model[fid].info.batch_size,
> > + ret = rte_ml_io_input_size_get(opt->dev_id, t->model[fid].id,
> > +t->model[fid].nb_batches,
> > &t->model[fid].inp_qsize, &t-
> >model[fid].inp_dsize);
> > if (ret != 0) {
> > ml_err("Failed to get input size, model : %s\n",
> > opt->filelist[fid].model); @@ -555,9 +560,8 @@
> ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt,
> uint16_t
> > }
> >
> > /* get output buffer size */
> > - ret = rte_ml_io_output_size_get(opt->dev_id, t->model[fid].id,
> > - t->model[fid].info.batch_size, &t-
> >model[fid].out_qsize,
> > - &t->model[fid].out_dsize);
> > + ret = rte_ml_io_output_size_get(opt->dev_id, t->model[fid].id, t-
> >model[fid].nb_batches,
> > + &t->model[fid].out_qsize, &t-
> >model[fid].out_dsize);
> > if (ret != 0) {
> > ml_err("Failed to get input size, model : %s\n", opt-
> >filelist[fid].model);
> > return ret;
> > @@ -702,7 +706,7 @@ ml_request_finish(struct rte_mempool *mp, void
> *opaque, void *obj, unsigned int
> > return;
> >
> > t->nb_used++;
> > - rte_ml_io_dequantize(t->cmn.opt->dev_id, model->id, t->model[req-
> >fid].info.batch_size,
> > + rte_ml_io_dequantize(t->cmn.opt->dev_id, model->id,
> > +t->model[req->fid].nb_batches,
> > req->output, model->output);
>
> These changes look unrelated with the topic of the patch.
> You should probably fix it when adding those lines at first.
The changes are related to the patch. Initially the number of batches run per inference is set to the default batch_size value of the mode, which is reported to the user through rte_ml_model_info_get.
This patch adds support to specify the number of batches to be run per inference. Hence, the default batch_size is replace with nb_batches values specified by the user.
>
^ permalink raw reply [flat|nested] 122+ messages in thread
* [PATCH v6 10/12] app/mldev: enable support for inference validation
2023-03-11 15:08 ` [PATCH v6 00/12] Implementation of mldev test application Srikanth Yalavarthi
` (8 preceding siblings ...)
2023-03-11 15:09 ` [PATCH v6 09/12] app/mldev: enable support for inference batches Srikanth Yalavarthi
@ 2023-03-11 15:09 ` Srikanth Yalavarthi
2023-03-11 15:09 ` [PATCH v6 11/12] app/mldev: enable reporting stats in mldev app Srikanth Yalavarthi
2023-03-11 15:09 ` [PATCH v6 12/12] app/mldev: add documentation for mldev test cases Srikanth Yalavarthi
11 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-11 15:09 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Enabled support to validate inference output with reference
output provided by the user. Validation would be successful
only when the inference outputs are within the 'tolerance'
specified through command line option "--tolerance".
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
Acked-by: Anup Prabhu <aprabhu@marvell.com>
---
app/test-mldev/meson.build | 2 +-
app/test-mldev/ml_options.c | 36 ++++-
app/test-mldev/ml_options.h | 3 +
app/test-mldev/test_inference_common.c | 216 ++++++++++++++++++++++++-
app/test-mldev/test_inference_common.h | 1 +
app/test-mldev/test_model_common.h | 1 +
6 files changed, 248 insertions(+), 11 deletions(-)
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
index 41d22fb22c..15db534dc2 100644
--- a/app/test-mldev/meson.build
+++ b/app/test-mldev/meson.build
@@ -21,4 +21,4 @@ sources = files(
'test_inference_interleave.c',
)
-deps += ['mldev']
+deps += ['mldev', 'hash']
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 499bfde899..9dfa5ec0d8 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -4,6 +4,7 @@
#include <errno.h>
#include <getopt.h>
+#include <math.h>
#include <rte_memory.h>
#include <rte_mldev.h>
@@ -28,6 +29,7 @@ ml_options_default(struct ml_options *opt)
opt->queue_pairs = 1;
opt->queue_size = 1;
opt->batches = 0;
+ opt->tolerance = 0.0;
opt->debug = false;
}
@@ -133,6 +135,13 @@ ml_parse_filelist(struct ml_options *opt, const char *arg)
}
strlcpy(opt->filelist[opt->nb_filelist].output, token, PATH_MAX);
+ /* reference - optional */
+ token = strtok(NULL, delim);
+ if (token != NULL)
+ strlcpy(opt->filelist[opt->nb_filelist].reference, token, PATH_MAX);
+ else
+ memset(opt->filelist[opt->nb_filelist].reference, 0, PATH_MAX);
+
opt->nb_filelist++;
if (opt->nb_filelist == 0) {
@@ -177,6 +186,14 @@ ml_parse_batches(struct ml_options *opt, const char *arg)
return parser_read_uint16(&opt->batches, arg);
}
+static int
+ml_parse_tolerance(struct ml_options *opt, const char *arg)
+{
+ opt->tolerance = fabs(atof(arg));
+
+ return 0;
+}
+
static void
ml_dump_test_options(const char *testname)
{
@@ -193,12 +210,13 @@ ml_dump_test_options(const char *testname)
if ((strcmp(testname, "inference_ordered") == 0) ||
(strcmp(testname, "inference_interleave") == 0)) {
- printf("\t\t--filelist : comma separated list of model, input and output\n"
+ printf("\t\t--filelist : comma separated list of model, input, output and reference\n"
"\t\t--repetitions : number of inference repetitions\n"
"\t\t--burst_size : inference burst size\n"
"\t\t--queue_pairs : number of queue pairs to create\n"
"\t\t--queue_size : size fo queue-pair\n"
- "\t\t--batches : number of batches of input\n");
+ "\t\t--batches : number of batches of input\n"
+ "\t\t--tolerance : maximum tolerance (%%) for output validation\n");
printf("\n");
}
}
@@ -218,12 +236,13 @@ print_usage(char *program)
ml_test_dump_names(ml_dump_test_options);
}
-static struct option lgopts[] = {
- {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
- {ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
- {ML_BURST_SIZE, 1, 0, 0}, {ML_QUEUE_PAIRS, 1, 0, 0}, {ML_QUEUE_SIZE, 1, 0, 0},
- {ML_BATCHES, 1, 0, 0}, {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0},
- {NULL, 0, 0, 0}};
+static struct option lgopts[] = {{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0},
+ {ML_SOCKET_ID, 1, 0, 0}, {ML_MODELS, 1, 0, 0},
+ {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
+ {ML_BURST_SIZE, 1, 0, 0}, {ML_QUEUE_PAIRS, 1, 0, 0},
+ {ML_QUEUE_SIZE, 1, 0, 0}, {ML_BATCHES, 1, 0, 0},
+ {ML_TOLERANCE, 1, 0, 0}, {ML_DEBUG, 0, 0, 0},
+ {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
static int
ml_opts_parse_long(int opt_idx, struct ml_options *opt)
@@ -236,6 +255,7 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
{ML_FILELIST, ml_parse_filelist}, {ML_REPETITIONS, ml_parse_repetitions},
{ML_BURST_SIZE, ml_parse_burst_size}, {ML_QUEUE_PAIRS, ml_parse_queue_pairs},
{ML_QUEUE_SIZE, ml_parse_queue_size}, {ML_BATCHES, ml_parse_batches},
+ {ML_TOLERANCE, ml_parse_tolerance},
};
for (i = 0; i < RTE_DIM(parsermap); i++) {
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index 48fe064150..7f3db29656 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -22,6 +22,7 @@
#define ML_QUEUE_PAIRS ("queue_pairs")
#define ML_QUEUE_SIZE ("queue_size")
#define ML_BATCHES ("batches")
+#define ML_TOLERANCE ("tolerance")
#define ML_DEBUG ("debug")
#define ML_HELP ("help")
@@ -29,6 +30,7 @@ struct ml_filelist {
char model[PATH_MAX];
char input[PATH_MAX];
char output[PATH_MAX];
+ char reference[PATH_MAX];
};
struct ml_options {
@@ -42,6 +44,7 @@ struct ml_options {
uint16_t queue_pairs;
uint16_t queue_size;
uint16_t batches;
+ float tolerance;
bool debug;
};
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
index 0f281aed6c..b605c1f5d3 100644
--- a/app/test-mldev/test_inference_common.c
+++ b/app/test-mldev/test_inference_common.c
@@ -6,6 +6,7 @@
#include <unistd.h>
#include <rte_common.h>
+#include <rte_hash_crc.h>
#include <rte_launch.h>
#include <rte_lcore.h>
#include <rte_malloc.h>
@@ -15,6 +16,27 @@
#include "ml_common.h"
#include "test_inference_common.h"
+#define ML_TEST_READ_TYPE(buffer, type) (*((type *)buffer))
+
+#define ML_TEST_CHECK_OUTPUT(output, reference, tolerance) \
+ (((float)output - (float)reference) <= (((float)reference * tolerance) / 100.0))
+
+#define ML_OPEN_WRITE_GET_ERR(name, buffer, size, err) \
+ do { \
+ FILE *fp = fopen(name, "w+"); \
+ if (fp == NULL) { \
+ ml_err("Unable to create file: %s, error: %s", name, strerror(errno)); \
+ err = true; \
+ } else { \
+ if (fwrite(buffer, 1, size, fp) != size) { \
+ ml_err("Error writing output, file: %s, error: %s", name, \
+ strerror(errno)); \
+ err = true; \
+ } \
+ fclose(fp); \
+ } \
+ } while (0)
+
/* Enqueue inference requests with burst size equal to 1 */
static int
ml_enqueue_single(void *arg)
@@ -358,6 +380,7 @@ test_inference_opt_dump(struct ml_options *opt)
ml_dump("burst_size", "%u", opt->burst_size);
ml_dump("queue_pairs", "%u", opt->queue_pairs);
ml_dump("queue_size", "%u", opt->queue_size);
+ ml_dump("tolerance", "%-7.3f", opt->tolerance);
if (opt->batches == 0)
ml_dump("batches", "%u (default)", opt->batches);
@@ -369,6 +392,8 @@ test_inference_opt_dump(struct ml_options *opt)
ml_dump_list("model", i, opt->filelist[i].model);
ml_dump_list("input", i, opt->filelist[i].input);
ml_dump_list("output", i, opt->filelist[i].output);
+ if (strcmp(opt->filelist[i].reference, "\0") != 0)
+ ml_dump_list("reference", i, opt->filelist[i].reference);
}
ml_dump_end;
}
@@ -393,6 +418,7 @@ test_inference_setup(struct ml_test *test, struct ml_options *opt)
t = ml_test_priv(test);
t->nb_used = 0;
+ t->nb_valid = 0;
t->cmn.result = ML_TEST_FAILED;
t->cmn.opt = opt;
memset(t->error_count, 0, RTE_MAX_LCORE * sizeof(uint64_t));
@@ -569,6 +595,9 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, uint16_t
/* allocate buffer for user data */
mz_size = t->model[fid].inp_dsize + t->model[fid].out_dsize;
+ if (strcmp(opt->filelist[fid].reference, "\0") != 0)
+ mz_size += t->model[fid].out_dsize;
+
sprintf(mz_name, "ml_user_data_%d", fid);
mz = rte_memzone_reserve(mz_name, mz_size, opt->socket_id, 0);
if (mz == NULL) {
@@ -579,6 +608,10 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, uint16_t
t->model[fid].input = mz->addr;
t->model[fid].output = t->model[fid].input + t->model[fid].inp_dsize;
+ if (strcmp(opt->filelist[fid].reference, "\0") != 0)
+ t->model[fid].reference = t->model[fid].output + t->model[fid].out_dsize;
+ else
+ t->model[fid].reference = NULL;
/* load input file */
fp = fopen(opt->filelist[fid].input, "r");
@@ -607,6 +640,27 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, uint16_t
}
fclose(fp);
+ /* load reference file */
+ if (t->model[fid].reference != NULL) {
+ fp = fopen(opt->filelist[fid].reference, "r");
+ if (fp == NULL) {
+ ml_err("Failed to open reference file : %s\n",
+ opt->filelist[fid].reference);
+ ret = -errno;
+ goto error;
+ }
+
+ if (fread(t->model[fid].reference, 1, t->model[fid].out_dsize, fp) !=
+ t->model[fid].out_dsize) {
+ ml_err("Failed to read reference file : %s\n",
+ opt->filelist[fid].reference);
+ ret = -errno;
+ fclose(fp);
+ goto error;
+ }
+ fclose(fp);
+ }
+
/* create mempool for quantized input and output buffers. ml_request_initialize is
* used as a callback for object creation.
*/
@@ -691,6 +745,121 @@ ml_inference_mem_destroy(struct ml_test *test, struct ml_options *opt)
rte_mempool_free(t->op_pool);
}
+static bool
+ml_inference_validation(struct ml_test *test, struct ml_request *req)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)test);
+ struct ml_model *model;
+ uint32_t nb_elements;
+ uint8_t *reference;
+ uint8_t *output;
+ bool match;
+ uint32_t i;
+ uint32_t j;
+
+ model = &t->model[req->fid];
+
+ /* compare crc when tolerance is 0 */
+ if (t->cmn.opt->tolerance == 0.0) {
+ match = (rte_hash_crc(model->output, model->out_dsize, 0) ==
+ rte_hash_crc(model->reference, model->out_dsize, 0));
+ } else {
+ output = model->output;
+ reference = model->reference;
+
+ i = 0;
+next_output:
+ nb_elements =
+ model->info.output_info[i].shape.w * model->info.output_info[i].shape.x *
+ model->info.output_info[i].shape.y * model->info.output_info[i].shape.z;
+ j = 0;
+next_element:
+ match = false;
+ switch (model->info.output_info[i].dtype) {
+ case RTE_ML_IO_TYPE_INT8:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, int8_t),
+ ML_TEST_READ_TYPE(reference, int8_t),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(int8_t);
+ reference += sizeof(int8_t);
+ break;
+ case RTE_ML_IO_TYPE_UINT8:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, uint8_t),
+ ML_TEST_READ_TYPE(reference, uint8_t),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(float);
+ reference += sizeof(float);
+ break;
+ case RTE_ML_IO_TYPE_INT16:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, int16_t),
+ ML_TEST_READ_TYPE(reference, int16_t),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(int16_t);
+ reference += sizeof(int16_t);
+ break;
+ case RTE_ML_IO_TYPE_UINT16:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, uint16_t),
+ ML_TEST_READ_TYPE(reference, uint16_t),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(uint16_t);
+ reference += sizeof(uint16_t);
+ break;
+ case RTE_ML_IO_TYPE_INT32:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, int32_t),
+ ML_TEST_READ_TYPE(reference, int32_t),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(int32_t);
+ reference += sizeof(int32_t);
+ break;
+ case RTE_ML_IO_TYPE_UINT32:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, uint32_t),
+ ML_TEST_READ_TYPE(reference, uint32_t),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(uint32_t);
+ reference += sizeof(uint32_t);
+ break;
+ case RTE_ML_IO_TYPE_FP32:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, float),
+ ML_TEST_READ_TYPE(reference, float),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(float);
+ reference += sizeof(float);
+ break;
+ default: /* other types, fp8, fp16, bfloat16 */
+ match = true;
+ }
+
+ if (!match)
+ goto done;
+ j++;
+ if (j < nb_elements)
+ goto next_element;
+
+ i++;
+ if (i < model->info.nb_outputs)
+ goto next_output;
+ }
+done:
+ if (match)
+ t->nb_valid++;
+
+ return match;
+}
+
/* Callback for mempool object iteration. This call would dequantize output data. */
static void
ml_request_finish(struct rte_mempool *mp, void *opaque, void *obj, unsigned int obj_idx)
@@ -698,9 +867,10 @@ ml_request_finish(struct rte_mempool *mp, void *opaque, void *obj, unsigned int
struct test_inference *t = ml_test_priv((struct ml_test *)opaque);
struct ml_request *req = (struct ml_request *)obj;
struct ml_model *model = &t->model[req->fid];
+ char str[PATH_MAX];
+ bool error = false;
RTE_SET_USED(mp);
- RTE_SET_USED(obj_idx);
if (req->niters == 0)
return;
@@ -708,6 +878,48 @@ ml_request_finish(struct rte_mempool *mp, void *opaque, void *obj, unsigned int
t->nb_used++;
rte_ml_io_dequantize(t->cmn.opt->dev_id, model->id, t->model[req->fid].nb_batches,
req->output, model->output);
+
+ if (model->reference == NULL) {
+ t->nb_valid++;
+ goto dump_output_pass;
+ }
+
+ if (!ml_inference_validation(opaque, req))
+ goto dump_output_fail;
+ else
+ goto dump_output_pass;
+
+dump_output_pass:
+ if (obj_idx == 0) {
+ /* write quantized output */
+ snprintf(str, PATH_MAX, "%s.q", t->cmn.opt->filelist[req->fid].output);
+ ML_OPEN_WRITE_GET_ERR(str, req->output, model->out_qsize, error);
+ if (error)
+ return;
+
+ /* write dequantized output */
+ snprintf(str, PATH_MAX, "%s", t->cmn.opt->filelist[req->fid].output);
+ ML_OPEN_WRITE_GET_ERR(str, model->output, model->out_dsize, error);
+ if (error)
+ return;
+ }
+
+ return;
+
+dump_output_fail:
+ if (t->cmn.opt->debug) {
+ /* dump quantized output buffer */
+ snprintf(str, PATH_MAX, "%s.q.%d", t->cmn.opt->filelist[req->fid].output, obj_idx);
+ ML_OPEN_WRITE_GET_ERR(str, req->output, model->out_qsize, error);
+ if (error)
+ return;
+
+ /* dump dequantized output buffer */
+ snprintf(str, PATH_MAX, "%s.%d", t->cmn.opt->filelist[req->fid].output, obj_idx);
+ ML_OPEN_WRITE_GET_ERR(str, model->output, model->out_dsize, error);
+ if (error)
+ return;
+ }
}
int
@@ -725,7 +937,7 @@ ml_inference_result(struct ml_test *test, struct ml_options *opt, uint16_t fid)
rte_mempool_obj_iter(t->model[fid].io_pool, ml_request_finish, test);
- if ((t->nb_used > 0) && (error_count == 0))
+ if ((t->nb_used == t->nb_valid) && (error_count == 0))
t->cmn.result = ML_TEST_SUCCESS;
else
t->cmn.result = ML_TEST_FAILED;
diff --git a/app/test-mldev/test_inference_common.h b/app/test-mldev/test_inference_common.h
index 81d9b07d41..2e4889e1f7 100644
--- a/app/test-mldev/test_inference_common.h
+++ b/app/test-mldev/test_inference_common.h
@@ -38,6 +38,7 @@ struct test_inference {
struct rte_mempool *op_pool;
uint64_t nb_used;
+ uint64_t nb_valid;
uint16_t fid;
int (*enqueue)(void *arg);
diff --git a/app/test-mldev/test_model_common.h b/app/test-mldev/test_model_common.h
index 19429ce142..b2a75a3261 100644
--- a/app/test-mldev/test_model_common.h
+++ b/app/test-mldev/test_model_common.h
@@ -28,6 +28,7 @@ struct ml_model {
uint8_t *input;
uint8_t *output;
+ uint8_t *reference;
struct rte_mempool *io_pool;
uint32_t nb_batches;
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v6 11/12] app/mldev: enable reporting stats in mldev app
2023-03-11 15:08 ` [PATCH v6 00/12] Implementation of mldev test application Srikanth Yalavarthi
` (9 preceding siblings ...)
2023-03-11 15:09 ` [PATCH v6 10/12] app/mldev: enable support for inference validation Srikanth Yalavarthi
@ 2023-03-11 15:09 ` Srikanth Yalavarthi
2023-03-11 15:09 ` [PATCH v6 12/12] app/mldev: add documentation for mldev test cases Srikanth Yalavarthi
11 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-11 15:09 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Enable reporting driver xstats and inference end-to-end
latency and throughput in mldev inference tests. Reporting
of stats can be enabled using "--stats" option.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
Acked-by: Anup Prabhu <aprabhu@marvell.com>
---
app/test-mldev/ml_options.c | 22 ++--
app/test-mldev/ml_options.h | 2 +
app/test-mldev/test_inference_common.c | 140 +++++++++++++++++++++
app/test-mldev/test_inference_common.h | 8 ++
app/test-mldev/test_inference_interleave.c | 4 +
app/test-mldev/test_inference_ordered.c | 1 +
6 files changed, 169 insertions(+), 8 deletions(-)
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 9dfa5ec0d8..769c3bffd1 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -30,6 +30,7 @@ ml_options_default(struct ml_options *opt)
opt->queue_size = 1;
opt->batches = 0;
opt->tolerance = 0.0;
+ opt->stats = false;
opt->debug = false;
}
@@ -216,7 +217,8 @@ ml_dump_test_options(const char *testname)
"\t\t--queue_pairs : number of queue pairs to create\n"
"\t\t--queue_size : size fo queue-pair\n"
"\t\t--batches : number of batches of input\n"
- "\t\t--tolerance : maximum tolerance (%%) for output validation\n");
+ "\t\t--tolerance : maximum tolerance (%%) for output validation\n"
+ "\t\t--stats : enable reporting performance statistics\n");
printf("\n");
}
}
@@ -236,13 +238,12 @@ print_usage(char *program)
ml_test_dump_names(ml_dump_test_options);
}
-static struct option lgopts[] = {{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0},
- {ML_SOCKET_ID, 1, 0, 0}, {ML_MODELS, 1, 0, 0},
- {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
- {ML_BURST_SIZE, 1, 0, 0}, {ML_QUEUE_PAIRS, 1, 0, 0},
- {ML_QUEUE_SIZE, 1, 0, 0}, {ML_BATCHES, 1, 0, 0},
- {ML_TOLERANCE, 1, 0, 0}, {ML_DEBUG, 0, 0, 0},
- {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
+static struct option lgopts[] = {
+ {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
+ {ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
+ {ML_BURST_SIZE, 1, 0, 0}, {ML_QUEUE_PAIRS, 1, 0, 0}, {ML_QUEUE_SIZE, 1, 0, 0},
+ {ML_BATCHES, 1, 0, 0}, {ML_TOLERANCE, 1, 0, 0}, {ML_STATS, 0, 0, 0},
+ {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
static int
ml_opts_parse_long(int opt_idx, struct ml_options *opt)
@@ -277,6 +278,11 @@ ml_options_parse(struct ml_options *opt, int argc, char **argv)
while ((opts = getopt_long(argc, argv, "", lgopts, &opt_idx)) != EOF) {
switch (opts) {
case 0: /* parse long options */
+ if (!strcmp(lgopts[opt_idx].name, "stats")) {
+ opt->stats = true;
+ break;
+ }
+
if (!strcmp(lgopts[opt_idx].name, "debug")) {
opt->debug = true;
break;
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index 7f3db29656..beb0fe69c6 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -23,6 +23,7 @@
#define ML_QUEUE_SIZE ("queue_size")
#define ML_BATCHES ("batches")
#define ML_TOLERANCE ("tolerance")
+#define ML_STATS ("stats")
#define ML_DEBUG ("debug")
#define ML_HELP ("help")
@@ -45,6 +46,7 @@ struct ml_options {
uint16_t queue_size;
uint16_t batches;
float tolerance;
+ bool stats;
bool debug;
};
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
index b605c1f5d3..e85f32be60 100644
--- a/app/test-mldev/test_inference_common.c
+++ b/app/test-mldev/test_inference_common.c
@@ -6,6 +6,7 @@
#include <unistd.h>
#include <rte_common.h>
+#include <rte_cycles.h>
#include <rte_hash_crc.h>
#include <rte_launch.h>
#include <rte_lcore.h>
@@ -37,6 +38,17 @@
} \
} while (0)
+static void
+print_line(uint16_t len)
+{
+ uint16_t i;
+
+ for (i = 0; i < len; i++)
+ printf("-");
+
+ printf("\n");
+}
+
/* Enqueue inference requests with burst size equal to 1 */
static int
ml_enqueue_single(void *arg)
@@ -46,6 +58,7 @@ ml_enqueue_single(void *arg)
struct rte_ml_op *op = NULL;
struct ml_core_args *args;
uint64_t model_enq = 0;
+ uint64_t start_cycle;
uint32_t burst_enq;
uint32_t lcore_id;
uint16_t fid;
@@ -53,6 +66,7 @@ ml_enqueue_single(void *arg)
lcore_id = rte_lcore_id();
args = &t->args[lcore_id];
+ args->start_cycles = 0;
model_enq = 0;
if (args->nb_reqs == 0)
@@ -88,10 +102,12 @@ ml_enqueue_single(void *arg)
req->fid = fid;
enqueue_req:
+ start_cycle = rte_get_tsc_cycles();
burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, args->qp_id, &op, 1);
if (burst_enq == 0)
goto enqueue_req;
+ args->start_cycles += start_cycle;
fid++;
if (likely(fid <= args->end_fid))
goto next_model;
@@ -115,10 +131,12 @@ ml_dequeue_single(void *arg)
uint64_t total_deq = 0;
uint8_t nb_filelist;
uint32_t burst_deq;
+ uint64_t end_cycle;
uint32_t lcore_id;
lcore_id = rte_lcore_id();
args = &t->args[lcore_id];
+ args->end_cycles = 0;
nb_filelist = args->end_fid - args->start_fid + 1;
if (args->nb_reqs == 0)
@@ -126,9 +144,11 @@ ml_dequeue_single(void *arg)
dequeue_req:
burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, args->qp_id, &op, 1);
+ end_cycle = rte_get_tsc_cycles();
if (likely(burst_deq == 1)) {
total_deq += burst_deq;
+ args->end_cycles += end_cycle;
if (unlikely(op->status == RTE_ML_OP_STATUS_ERROR)) {
rte_ml_op_error_get(t->cmn.opt->dev_id, op, &error);
ml_err("error_code = 0x%" PRIx64 ", error_message = %s\n", error.errcode,
@@ -152,6 +172,7 @@ ml_enqueue_burst(void *arg)
{
struct test_inference *t = ml_test_priv((struct ml_test *)arg);
struct ml_core_args *args;
+ uint64_t start_cycle;
uint16_t ops_count;
uint64_t model_enq;
uint16_t burst_enq;
@@ -164,6 +185,7 @@ ml_enqueue_burst(void *arg)
lcore_id = rte_lcore_id();
args = &t->args[lcore_id];
+ args->start_cycles = 0;
model_enq = 0;
if (args->nb_reqs == 0)
@@ -205,8 +227,10 @@ ml_enqueue_burst(void *arg)
pending = ops_count;
enqueue_reqs:
+ start_cycle = rte_get_tsc_cycles();
burst_enq =
rte_ml_enqueue_burst(t->cmn.opt->dev_id, args->qp_id, &args->enq_ops[idx], pending);
+ args->start_cycles += burst_enq * start_cycle;
pending = pending - burst_enq;
if (pending > 0) {
@@ -236,11 +260,13 @@ ml_dequeue_burst(void *arg)
uint64_t total_deq = 0;
uint16_t burst_deq = 0;
uint8_t nb_filelist;
+ uint64_t end_cycle;
uint32_t lcore_id;
uint32_t i;
lcore_id = rte_lcore_id();
args = &t->args[lcore_id];
+ args->end_cycles = 0;
nb_filelist = args->end_fid - args->start_fid + 1;
if (args->nb_reqs == 0)
@@ -249,9 +275,11 @@ ml_dequeue_burst(void *arg)
dequeue_burst:
burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, args->qp_id, args->deq_ops,
t->cmn.opt->burst_size);
+ end_cycle = rte_get_tsc_cycles();
if (likely(burst_deq > 0)) {
total_deq += burst_deq;
+ args->end_cycles += burst_deq * end_cycle;
for (i = 0; i < burst_deq; i++) {
if (unlikely(args->deq_ops[i]->status == RTE_ML_OP_STATUS_ERROR)) {
@@ -381,6 +409,7 @@ test_inference_opt_dump(struct ml_options *opt)
ml_dump("queue_pairs", "%u", opt->queue_pairs);
ml_dump("queue_size", "%u", opt->queue_size);
ml_dump("tolerance", "%-7.3f", opt->tolerance);
+ ml_dump("stats", "%s", (opt->stats ? "true" : "false"));
if (opt->batches == 0)
ml_dump("batches", "%u (default)", opt->batches);
@@ -454,6 +483,11 @@ test_inference_setup(struct ml_test *test, struct ml_options *opt)
RTE_CACHE_LINE_SIZE, opt->socket_id);
}
+ for (i = 0; i < RTE_MAX_LCORE; i++) {
+ t->args[i].start_cycles = 0;
+ t->args[i].end_cycles = 0;
+ }
+
return 0;
error:
@@ -986,3 +1020,109 @@ ml_inference_launch_cores(struct ml_test *test, struct ml_options *opt, uint16_t
return 0;
}
+
+int
+ml_inference_stats_get(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t = ml_test_priv(test);
+ uint64_t total_cycles = 0;
+ uint32_t nb_filelist;
+ uint64_t throughput;
+ uint64_t avg_e2e;
+ uint32_t qp_id;
+ uint64_t freq;
+ int ret;
+ int i;
+
+ if (!opt->stats)
+ return 0;
+
+ /* get xstats size */
+ t->xstats_size = rte_ml_dev_xstats_names_get(opt->dev_id, NULL, 0);
+ if (t->xstats_size >= 0) {
+ /* allocate for xstats_map and values */
+ t->xstats_map = rte_malloc(
+ "ml_xstats_map", t->xstats_size * sizeof(struct rte_ml_dev_xstats_map), 0);
+ if (t->xstats_map == NULL) {
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ t->xstats_values =
+ rte_malloc("ml_xstats_values", t->xstats_size * sizeof(uint64_t), 0);
+ if (t->xstats_values == NULL) {
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ ret = rte_ml_dev_xstats_names_get(opt->dev_id, t->xstats_map, t->xstats_size);
+ if (ret != t->xstats_size) {
+ printf("Unable to get xstats names, ret = %d\n", ret);
+ ret = -1;
+ goto error;
+ }
+
+ for (i = 0; i < t->xstats_size; i++)
+ rte_ml_dev_xstats_get(opt->dev_id, &t->xstats_map[i].id,
+ &t->xstats_values[i], 1);
+ }
+
+ /* print xstats*/
+ printf("\n");
+ print_line(80);
+ printf(" ML Device Extended Statistics\n");
+ print_line(80);
+ for (i = 0; i < t->xstats_size; i++)
+ printf(" %-64s = %" PRIu64 "\n", t->xstats_map[i].name, t->xstats_values[i]);
+ print_line(80);
+
+ /* release buffers */
+ if (t->xstats_map)
+ rte_free(t->xstats_map);
+
+ if (t->xstats_values)
+ rte_free(t->xstats_values);
+
+ /* print end-to-end stats */
+ freq = rte_get_tsc_hz();
+ for (qp_id = 0; qp_id < RTE_MAX_LCORE; qp_id++)
+ total_cycles += t->args[qp_id].end_cycles - t->args[qp_id].start_cycles;
+ avg_e2e = total_cycles / opt->repetitions;
+
+ if (freq == 0) {
+ avg_e2e = total_cycles / opt->repetitions;
+ printf(" %-64s = %" PRIu64 "\n", "Average End-to-End Latency (cycles)", avg_e2e);
+ } else {
+ avg_e2e = (total_cycles * NS_PER_S) / (opt->repetitions * freq);
+ printf(" %-64s = %" PRIu64 "\n", "Average End-to-End Latency (ns)", avg_e2e);
+ }
+
+ /* print inference throughput */
+ if (strcmp(opt->test_name, "inference_ordered") == 0)
+ nb_filelist = 1;
+ else
+ nb_filelist = opt->nb_filelist;
+
+ if (freq == 0) {
+ throughput = (nb_filelist * t->cmn.opt->repetitions * 1000000) / total_cycles;
+ printf(" %-64s = %" PRIu64 "\n", "Average Throughput (inferences / million cycles)",
+ throughput);
+ } else {
+ throughput = (nb_filelist * t->cmn.opt->repetitions * freq) / total_cycles;
+ printf(" %-64s = %" PRIu64 "\n", "Average Throughput (inferences / second)",
+ throughput);
+ }
+
+ print_line(80);
+
+ return 0;
+
+error:
+ if (t->xstats_map)
+ rte_free(t->xstats_map);
+
+ if (t->xstats_values)
+ rte_free(t->xstats_values);
+
+ return ret;
+}
diff --git a/app/test-mldev/test_inference_common.h b/app/test-mldev/test_inference_common.h
index 2e4889e1f7..0b4fba78e0 100644
--- a/app/test-mldev/test_inference_common.h
+++ b/app/test-mldev/test_inference_common.h
@@ -27,6 +27,9 @@ struct ml_core_args {
struct rte_ml_op **enq_ops;
struct rte_ml_op **deq_ops;
struct ml_request **reqs;
+
+ uint64_t start_cycles;
+ uint64_t end_cycles;
};
struct test_inference {
@@ -46,6 +49,10 @@ struct test_inference {
struct ml_core_args args[RTE_MAX_LCORE];
uint64_t error_count[RTE_MAX_LCORE];
+
+ struct rte_ml_dev_xstats_map *xstats_map;
+ uint64_t *xstats_values;
+ int xstats_size;
} __rte_cache_aligned;
bool test_inference_cap_check(struct ml_options *opt);
@@ -63,5 +70,6 @@ void ml_inference_mem_destroy(struct ml_test *test, struct ml_options *opt);
int ml_inference_result(struct ml_test *test, struct ml_options *opt, uint16_t fid);
int ml_inference_launch_cores(struct ml_test *test, struct ml_options *opt, uint16_t start_fid,
uint16_t end_fid);
+int ml_inference_stats_get(struct ml_test *test, struct ml_options *opt);
#endif /* _ML_TEST_INFERENCE_COMMON_ */
diff --git a/app/test-mldev/test_inference_interleave.c b/app/test-mldev/test_inference_interleave.c
index 9cf4cfa197..bd2c286737 100644
--- a/app/test-mldev/test_inference_interleave.c
+++ b/app/test-mldev/test_inference_interleave.c
@@ -56,7 +56,11 @@ test_inference_interleave_driver(struct ml_test *test, struct ml_options *opt)
goto error;
ml_inference_iomem_destroy(test, opt, fid);
+ }
+
+ ml_inference_stats_get(test, opt);
+ for (fid = 0; fid < opt->nb_filelist; fid++) {
ret = ml_model_stop(test, opt, &t->model[fid], fid);
if (ret != 0)
goto error;
diff --git a/app/test-mldev/test_inference_ordered.c b/app/test-mldev/test_inference_ordered.c
index 1cd91dc3d3..8992358936 100644
--- a/app/test-mldev/test_inference_ordered.c
+++ b/app/test-mldev/test_inference_ordered.c
@@ -54,6 +54,7 @@ test_inference_ordered_driver(struct ml_test *test, struct ml_options *opt)
goto error;
ml_inference_iomem_destroy(test, opt, fid);
+ ml_inference_stats_get(test, opt);
/* stop model */
ret = ml_model_stop(test, opt, &t->model[fid], fid);
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v6 12/12] app/mldev: add documentation for mldev test cases
2023-03-11 15:08 ` [PATCH v6 00/12] Implementation of mldev test application Srikanth Yalavarthi
` (10 preceding siblings ...)
2023-03-11 15:09 ` [PATCH v6 11/12] app/mldev: enable reporting stats in mldev app Srikanth Yalavarthi
@ 2023-03-11 15:09 ` Srikanth Yalavarthi
2023-03-16 17:50 ` Thomas Monjalon
11 siblings, 1 reply; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-11 15:09 UTC (permalink / raw)
To: Thomas Monjalon, Srikanth Yalavarthi
Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Added documentation specific to mldev test cases. Added details
about all test cases and option supported by individual tests.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
Acked-by: Shivah Shankar S <sshankarnara@marvell.com>
Acked-by: Anup Prabhu <aprabhu@marvell.com>
---
MAINTAINERS | 1 +
.../tools/img/mldev_inference_interleave.svg | 669 ++++++++++++++++++
.../tools/img/mldev_inference_ordered.svg | 528 ++++++++++++++
.../tools/img/mldev_model_ops_subtest_a.svg | 420 +++++++++++
.../tools/img/mldev_model_ops_subtest_b.svg | 423 +++++++++++
.../tools/img/mldev_model_ops_subtest_c.svg | 366 ++++++++++
.../tools/img/mldev_model_ops_subtest_d.svg | 424 +++++++++++
doc/guides/tools/index.rst | 1 +
doc/guides/tools/testmldev.rst | 441 ++++++++++++
9 files changed, 3273 insertions(+)
create mode 100644 doc/guides/tools/img/mldev_inference_interleave.svg
create mode 100644 doc/guides/tools/img/mldev_inference_ordered.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_a.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_b.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_c.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_d.svg
create mode 100644 doc/guides/tools/testmldev.rst
diff --git a/MAINTAINERS b/MAINTAINERS
index b0fd46ef02..50ce63b838 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -481,6 +481,7 @@ M: Srikanth Yalavarthi <syalavarthi@marvell.com>
F: lib/mldev/
F: doc/guides/prog_guide/mldev.rst
F: app/test-mldev
+F: doc/guides/tools/testmldev.rst
DMA device API - EXPERIMENTAL
M: Chengwen Feng <fengchengwen@huawei.com>
diff --git a/doc/guides/tools/img/mldev_inference_interleave.svg b/doc/guides/tools/img/mldev_inference_interleave.svg
new file mode 100644
index 0000000000..3a741ea627
--- /dev/null
+++ b/doc/guides/tools/img/mldev_inference_interleave.svg
@@ -0,0 +1,669 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- SPDX-License-Identifier: BSD-3-Clause -->
+<!-- Copyright (c) 2022 Marvell. -->
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="224mm"
+ height="150mm"
+ viewBox="0 0 224 150"
+ version="1.1"
+ id="svg5369"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="inference_interleave.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns:xlink="http://www.w3.org/1999/xlink"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg">
+ <sodipodi:namedview
+ id="namedview5371"
+ pagecolor="#ffffff"
+ bordercolor="#000000"
+ borderopacity="0.25"
+ inkscape:showpageshadow="2"
+ inkscape:pageopacity="0.0"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1"
+ inkscape:document-units="mm"
+ showgrid="false"
+ inkscape:zoom="0.74564394"
+ inkscape:cx="415.07747"
+ inkscape:cy="348.6919"
+ inkscape:window-width="1920"
+ inkscape:window-height="1017"
+ inkscape:window-x="1912"
+ inkscape:window-y="-8"
+ inkscape:window-maximized="1"
+ inkscape:current-layer="layer1" />
+ <defs
+ id="defs5366">
+ <linearGradient
+ inkscape:collect="always"
+ id="linearGradient19189">
+ <stop
+ style="stop-color:#ffeeaa;stop-opacity:1;"
+ offset="0"
+ id="stop19185" />
+ <stop
+ style="stop-color:#ffeeaa;stop-opacity:0;"
+ offset="1"
+ id="stop19187" />
+ </linearGradient>
+ <marker
+ style="overflow:visible"
+ id="TriangleStart"
+ refX="4"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="TriangleStart"
+ markerWidth="5.3244081"
+ markerHeight="6.155385"
+ viewBox="0 0 5.3244081 6.1553851"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.5)"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:context-stroke;stroke-width:1pt"
+ d="M 5.77,0 -2.88,5 V -5 Z"
+ id="path135" />
+ </marker>
+ <marker
+ style="overflow:visible"
+ id="RoundedArrow"
+ refX="5"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="RoundedArrow"
+ markerWidth="6.1347523"
+ markerHeight="5.9304948"
+ viewBox="0 0 6.1347524 5.9304951"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.7)"
+ d="m -0.21114562,-4.1055728 6.42229122,3.21114561 a 1,1 90 0 1 0,1.78885438 L -0.21114562,4.1055728 A 1.236068,1.236068 31.717474 0 1 -2,3 v -6 a 1.236068,1.236068 148.28253 0 1 1.78885438,-1.1055728 z"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:none"
+ id="path1367" />
+ </marker>
+ <linearGradient
+ inkscape:collect="always"
+ xlink:href="#linearGradient19189"
+ id="linearGradient19191"
+ x1="12.169352"
+ y1="105"
+ x2="284.83066"
+ y2="105"
+ gradientUnits="userSpaceOnUse"
+ gradientTransform="matrix(0.79055445,0,0,0.74078976,29.505892,28.991272)" />
+ </defs>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1">
+ <g
+ id="g1477"
+ transform="translate(-34.903236,-31.774189)">
+ <rect
+ style="fill:url(#linearGradient19191);fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.396267;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect16635"
+ width="215.18147"
+ height="139.60078"
+ x="39.3125"
+ y="36.973797"
+ ry="2.2354064" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-169.3954"
+ y="214.97237"
+ id="text5181"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan5179"
+ style="font-size:5.64444px;stroke-width:0.75"
+ x="-169.3954"
+ y="214.97237">test: inference_interleave</tspan></text>
+ <path
+ style="display:inline;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 138.05133,87.886263 17.45982,-10e-7"
+ id="path1912"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1724-0"
+ inkscape:connection-end="#rect1679" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 191.51115,87.886262 17.45981,10e-7"
+ id="path1914"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1679"
+ inkscape:connection-end="#rect1724" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 138.05133,109.78102 17.45982,-1e-5"
+ id="path1916"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1724-4-8"
+ inkscape:connection-end="#rect1679-4" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 191.51115,109.78101 17.45981,1e-5"
+ id="path1918"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1679-4"
+ inkscape:connection-end="#rect1724-4" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 138.05133,131.67576 17.45982,-1e-5"
+ id="path1920"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1724-6-7"
+ inkscape:connection-end="#rect1679-8" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 191.51115,131.67575 17.45981,1e-5"
+ id="path1922"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1679-8"
+ inkscape:connection-end="#rect1724-6" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.75;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;marker-start:url(#RoundedArrow);marker-end:url(#RoundedArrow)"
+ d="m 173.51116,60.08164 0,12.907336"
+ id="path1933"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1811"
+ inkscape:connection-end="#rect1924" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.368668;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.737336, 0.737336;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924"
+ width="46.97015"
+ height="73.58287"
+ x="150.02565"
+ y="72.988976"
+ ry="2.4685853" />
+ <text
+ xml:space="preserve"
+ style="font-size:6.35px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-84.44075"
+ y="173.5065"
+ id="text4097"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4095"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-84.44075"
+ y="173.5065">Queue</tspan><tspan
+ sodipodi:role="line"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-92.37825"
+ y="173.5065"
+ id="tspan4099">Pair 0</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:6.35px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-128.2318"
+ y="173.5065"
+ id="text4097-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4095-6"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-128.2318"
+ y="173.5065">Queue</tspan><tspan
+ sodipodi:role="line"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-136.1693"
+ y="173.5065"
+ id="tspan4099-4">Pair 2</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1679"
+ width="36"
+ height="18"
+ x="155.51115"
+ y="78.886261"
+ ry="3" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1679-8"
+ width="36"
+ height="18"
+ x="155.51115"
+ y="122.67575"
+ ry="3" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1679-4"
+ width="36"
+ height="18"
+ x="155.51115"
+ y="100.78101"
+ ry="3" />
+ <text
+ xml:space="preserve"
+ style="font-size:6.35px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-106.33705"
+ y="173.5065"
+ id="text4097-8"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4095-4"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-106.33705"
+ y="173.5065">Queue</tspan><tspan
+ sodipodi:role="line"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-114.27455"
+ y="173.5065"
+ id="tspan4099-5">Pair 1</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.388863;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1811"
+ width="58.909527"
+ height="18.812746"
+ x="144.0564"
+ y="41.268894"
+ ry="2.2255962" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-47.017281"
+ y="173.49187"
+ id="text4156"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4154"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';stroke-width:0.75"
+ x="-47.017281"
+ y="173.49187">Machine Learning</tspan><tspan
+ sodipodi:role="line"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';stroke-width:0.75"
+ x="-54.07283"
+ y="173.49187"
+ id="tspan4158">Hardware Engine</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.368668;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.737336, 0.737336;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924-0"
+ width="46.97015"
+ height="73.58287"
+ x="98.42067"
+ y="72.988976"
+ ry="2.4685853" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-88.32518"
+ y="122.24379"
+ id="text3708"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-88.32518"
+ y="122.24379">lcore 1</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-132.10504"
+ y="121.83865"
+ id="text3708-8"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-7"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-132.10504"
+ y="121.83865">lcore 5</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-0"
+ width="32.290321"
+ height="11.709678"
+ x="105.76101"
+ y="82.031425"
+ ry="3.0161259" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-6-7"
+ width="32.290321"
+ height="11.709678"
+ x="105.76101"
+ y="125.82092"
+ ry="3.0161259"
+ inkscape:connector-avoid="true" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-4-8"
+ width="32.290321"
+ height="11.709678"
+ x="105.76101"
+ y="103.92618"
+ ry="3.0161259"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-110.21718"
+ y="121.85381"
+ id="text3708-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-87"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-110.21718"
+ y="121.85381">lcore 3</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-150.82878"
+ y="121.79179"
+ id="text4542"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4540"
+ style="font-size:5.64444px;stroke-width:0.75"
+ x="-150.82878"
+ y="121.79179">Enqueue Workers</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.368668;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.737336, 0.737336;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924-7"
+ width="46.97015"
+ height="73.58287"
+ x="201.63062"
+ y="72.988976"
+ ry="2.4685853" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-88.32518"
+ y="225.08443"
+ id="text3708-9"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-9"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-88.32518"
+ y="225.08443">lcore 2</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-110.21167"
+ y="225.07202"
+ id="text3708-7"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-8"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-110.21167"
+ y="225.07202">lcore 4</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724"
+ width="32.290321"
+ height="11.709678"
+ x="208.97096"
+ y="82.031425"
+ ry="3.0161259" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-4"
+ width="32.290321"
+ height="11.709678"
+ x="208.97096"
+ y="103.92618"
+ ry="3.0161259" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-6"
+ width="32.290321"
+ height="11.709678"
+ x="208.97096"
+ y="125.82092"
+ ry="3.0161259" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-132.1133"
+ y="225.06514"
+ id="text3708-78"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-0"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-132.1133"
+ y="225.06514">lcore 6</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-150.82878"
+ y="225.00725"
+ id="text4542-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4540-7"
+ style="font-size:5.64444px;stroke-width:0.75"
+ x="-150.82878"
+ y="225.00725">Dequeue Workers</tspan></text>
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 79.368074,78.962117 26.440446,5.546991"
+ id="path6677"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-0"
+ inkscape:connection-start="#rect6252" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="M 78.978262,84.050781 112.13805,103.92618"
+ id="path6679"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-4-8"
+ inkscape:connection-start="#rect6252" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 73.959073,84.25738 42.026977,41.56354"
+ id="path6681"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-6-7"
+ inkscape:connection-start="#rect6252" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="M 79.368074,95.959838 105.76913,90.949016"
+ id="path6683"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-0"
+ inkscape:connection-start="#rect6252-8" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 79.368074,101.28215 26.416716,5.27791"
+ id="path7830"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-4-8"
+ inkscape:connection-start="#rect6252-8" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 79.069199,106.4283 32.903161,19.39262"
+ id="path7862"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-6-7"
+ inkscape:connection-start="#rect6252-8" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="M 79.069201,113.13371 111.97235,93.741103"
+ id="path7900"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-0"
+ inkscape:connection-start="#rect6252-2" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 79.368074,118.27987 26.416716,-5.2779"
+ id="path7932"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-4-8"
+ inkscape:connection-start="#rect6252-2" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 79.368074,123.60218 26.401056,5.01083"
+ id="path7998"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-6-7"
+ inkscape:connection-start="#rect6252-2" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="M 73.959074,135.30464 115.98605,93.741103"
+ id="path8000"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-0"
+ inkscape:connection-start="#rect6252-6" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="M 78.978264,135.51124 112.13805,115.63586"
+ id="path8002"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-4-8"
+ inkscape:connection-start="#rect6252-6" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 79.368074,140.59991 26.440446,-5.54699"
+ id="path8004"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-6-7"
+ inkscape:connection-start="#rect6252-6" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-76.53363"
+ y="65.63237"
+ id="text3708-84"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-4"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-76.53363"
+ y="65.63237">Model 0</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-98.991623"
+ y="66.015465"
+ id="text3708-84-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-4-6"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-98.991623"
+ y="66.015465">Model 1</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-121.44823"
+ y="65.646149"
+ id="text3708-84-9"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-4-1"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-121.44823"
+ y="65.646149">Model 2</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-143.9021"
+ y="65.625481"
+ id="text3708-84-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-4-8"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-143.9021"
+ y="65.625481">Model 3</tspan></text>
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect6252"
+ width="27.32258"
+ height="16.32258"
+ x="52.045494"
+ y="67.934799"
+ ry="1" />
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect6252-8"
+ width="27.32258"
+ height="16.32258"
+ x="52.045494"
+ y="90.391411"
+ ry="1" />
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect6252-2"
+ width="27.32258"
+ height="16.32258"
+ x="52.045494"
+ y="112.84802"
+ ry="1" />
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect6252-6"
+ width="27.32258"
+ height="16.32258"
+ x="52.045494"
+ y="135.30464"
+ ry="1" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.399886;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.799773, 0.799773;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924-1-3"
+ width="43.035683"
+ height="94.487152"
+ x="44.188942"
+ y="62.536991"
+ ry="3.1694498" />
+ <g
+ id="g1010"
+ transform="translate(21.464467,-15.875004)">
+ <text
+ xml:space="preserve"
+ style="font-weight:bold;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:'Arial Bold';text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-185.88483"
+ y="86.542366"
+ id="text15571"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan15569"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.5;stroke-opacity:1"
+ x="-185.88483"
+ y="86.542366">nb_worker_threads = 2 * MIN(nb_queue_pairs, (lcore_count - 1) / 2)</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-weight:bold;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:'Arial Bold';text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-178.43243"
+ y="90.903854"
+ id="text15571-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan15569-9"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.5;stroke-opacity:1"
+ x="-178.43243"
+ y="90.903854">inferences_per_queue_pair = nb_models * (repetitions / nb_queue_pairs)</tspan></text>
+ </g>
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/tools/img/mldev_inference_ordered.svg b/doc/guides/tools/img/mldev_inference_ordered.svg
new file mode 100644
index 0000000000..12fa6acaec
--- /dev/null
+++ b/doc/guides/tools/img/mldev_inference_ordered.svg
@@ -0,0 +1,528 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- SPDX-License-Identifier: BSD-3-Clause -->
+<!-- Copyright (c) 2022 Marvell. -->
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="243mm"
+ height="144mm"
+ viewBox="0 0 243 144"
+ version="1.1"
+ id="svg5"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="inference_ordered.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns:xlink="http://www.w3.org/1999/xlink"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg">
+ <sodipodi:namedview
+ id="namedview7"
+ pagecolor="#ffffff"
+ bordercolor="#000000"
+ borderopacity="0.25"
+ inkscape:showpageshadow="2"
+ inkscape:pageopacity="0.0"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1"
+ inkscape:document-units="mm"
+ showgrid="false"
+ inkscape:zoom="0.74564394"
+ inkscape:cx="488.83922"
+ inkscape:cy="234.69647"
+ inkscape:window-width="1920"
+ inkscape:window-height="1017"
+ inkscape:window-x="1912"
+ inkscape:window-y="-8"
+ inkscape:window-maximized="1"
+ inkscape:current-layer="layer1" />
+ <defs
+ id="defs2">
+ <marker
+ style="overflow:visible"
+ id="RoundedArrow"
+ refX="5"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="RoundedArrow"
+ markerWidth="6.1347523"
+ markerHeight="5.9304948"
+ viewBox="0 0 6.1347524 5.9304951"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.7)"
+ d="m -0.21114562,-4.1055728 6.42229122,3.21114561 a 1,1 90 0 1 0,1.78885438 L -0.21114562,4.1055728 A 1.236068,1.236068 31.717474 0 1 -2,3 v -6 a 1.236068,1.236068 148.28253 0 1 1.78885438,-1.1055728 z"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:none"
+ id="path1367" />
+ </marker>
+ <linearGradient
+ inkscape:collect="always"
+ id="linearGradient31002">
+ <stop
+ style="stop-color:#fff6d5;stop-opacity:1;"
+ offset="0"
+ id="stop30998" />
+ <stop
+ style="stop-color:#fff6d5;stop-opacity:0;"
+ offset="1"
+ id="stop31000" />
+ </linearGradient>
+ <marker
+ style="overflow:visible"
+ id="TriangleStart"
+ refX="4"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="TriangleStart"
+ markerWidth="5.3244081"
+ markerHeight="6.155385"
+ viewBox="0 0 5.3244081 6.1553851"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.5)"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:context-stroke;stroke-width:1pt"
+ d="M 5.77,0 -2.88,5 V -5 Z"
+ id="path135" />
+ </marker>
+ <linearGradient
+ inkscape:collect="always"
+ xlink:href="#linearGradient31002"
+ id="linearGradient31004"
+ x1="19.620968"
+ y1="102.90323"
+ x2="279.1532"
+ y2="102.90323"
+ gradientUnits="userSpaceOnUse"
+ gradientTransform="matrix(0.89215122,0,0,0.73190287,13.449912,42.668706)" />
+ </defs>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1">
+ <g
+ id="g1340"
+ transform="translate(-25.225796,-45.983871)">
+ <rect
+ style="fill:url(#linearGradient31004);fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.404032;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect27876"
+ width="231.09595"
+ height="132.45081"
+ x="31.177822"
+ y="51.758469"
+ ry="3.5071263" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1813"
+ width="38.408459"
+ height="45.86002"
+ x="34.901794"
+ y="99.14959"
+ ry="5.2246051"
+ inkscape:connector-avoid="true" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 73.310253,115.94935 36.498807,-11.6509"
+ id="path1906"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1813"
+ inkscape:connection-end="#rect1724-0" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 73.310253,122.0796 36.117817,1e-5"
+ id="path1908"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1813"
+ inkscape:connection-end="#rect1724-4-8" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="M 73.310253,128.20983 109.80905,139.8607"
+ id="path1910"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1813"
+ inkscape:connection-end="#rect1724-6-7" />
+ <path
+ style="display:inline;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 141.71839,99.266314 19.42262,-10e-7"
+ id="path1912"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1724-0"
+ inkscape:connection-end="#rect1679" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 197.14101,99.266313 19.42259,10e-7"
+ id="path1914"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1679"
+ inkscape:connection-end="#rect1724" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 141.71839,122.07961 19.42262,-1e-5"
+ id="path1916"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1724-4-8"
+ inkscape:connection-end="#rect1679-4" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 197.14101,122.0796 19.42259,1e-5"
+ id="path1918"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1679-4"
+ inkscape:connection-end="#rect1724-4" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 141.71839,144.89282 19.42262,0"
+ id="path1920"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1724-6-7"
+ inkscape:connection-end="#rect1679-8" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 197.14101,144.89282 19.42259,0"
+ id="path1922"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1679-8"
+ inkscape:connection-end="#rect1724-6" />
+ <text
+ xml:space="preserve"
+ style="font-size:8.46667px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-121.09793"
+ y="54.031597"
+ id="text4093"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4091"
+ style="font-size:8.46667px;stroke-width:0.75"
+ x="-121.09793"
+ y="54.031597">Model X</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:6.35px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-118.63563"
+ y="179.13635"
+ id="text4097"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4095"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-118.63563"
+ y="179.13635">Queue</tspan><tspan
+ sodipodi:role="line"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-126.57313"
+ y="179.13635"
+ id="tspan4099">Pair 1</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:6.35px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-141.44887"
+ y="179.13635"
+ id="text4097-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4095-6"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-141.44887"
+ y="179.13635">Queue</tspan><tspan
+ sodipodi:role="line"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-149.38637"
+ y="179.13635"
+ id="tspan4099-4">Pair 2</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.354849;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.709699, 0.709699;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924"
+ width="44.145252"
+ height="72.532341"
+ x="157.06865"
+ y="85.813438"
+ ry="4.31247" />
+ <g
+ id="g1224">
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1679"
+ width="36"
+ height="18"
+ x="161.14101"
+ y="90.266312"
+ ry="3" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1679-8"
+ width="36"
+ height="18"
+ x="161.14101"
+ y="135.89282"
+ ry="3" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1679-4"
+ width="36"
+ height="18"
+ x="161.14101"
+ y="113.07959"
+ ry="3" />
+ </g>
+ <text
+ xml:space="preserve"
+ style="font-size:6.35px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-95.820801"
+ y="179.13635"
+ id="text4097-8"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4095-4"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-95.820801"
+ y="179.13635">Queue</tspan><tspan
+ sodipodi:role="line"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-103.7583"
+ y="179.13635"
+ id="tspan4099-5">Pair 0</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.317648;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1811"
+ width="44.196934"
+ height="16.731901"
+ x="157.04254"
+ y="56.49292"
+ ry="2.761292" />
+ <text
+ xml:space="preserve"
+ style="font-size:3.5859px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.317649;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.952945, 0.317649;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-60.009941"
+ y="186.38451"
+ id="text4156"
+ transform="matrix(0,-1.040508,0.96106903,0,0,0)"><tspan
+ sodipodi:role="line"
+ id="tspan4154"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-family:Arial;-inkscape-font-specification:'Arial Bold';stroke-width:0.317648"
+ x="-60.009941"
+ y="186.38451">Machine Learning</tspan><tspan
+ sodipodi:role="line"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-family:Arial;-inkscape-font-specification:'Arial Bold';stroke-width:0.317648"
+ x="-64.492317"
+ y="186.38451"
+ id="tspan4158">Hardware Engine</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-99.705231"
+ y="125.91087"
+ id="text3708"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-99.705231"
+ y="125.91087">lcore 1</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-145.3221"
+ y="125.50572"
+ id="text3708-8"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-7"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-145.3221"
+ y="125.50572">lcore 5</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-122.51577"
+ y="125.52089"
+ id="text3708-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-87"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-122.51577"
+ y="125.52089">lcore 3</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-162.06549"
+ y="125.4589"
+ id="text4542"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4540"
+ style="font-size:5.64444px;stroke-width:0.75"
+ x="-162.06549"
+ y="125.4589">Enqueue Workers</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-99.705231"
+ y="232.67706"
+ id="text3708-9"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-9"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-99.705231"
+ y="232.67706">lcore 2</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-122.51025"
+ y="232.66466"
+ id="text3708-7"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-8"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-122.51025"
+ y="232.66466">lcore 4</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-145.33035"
+ y="232.65778"
+ id="text3708-78"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-0"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-145.33035"
+ y="232.65778">lcore 6</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-162.06549"
+ y="232.59988"
+ id="text4542-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4540-7"
+ style="font-size:5.64444px;stroke-width:0.75"
+ x="-162.06549"
+ y="232.59988">Dequeue Workers</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:6.35px;font-family:Arial;-inkscape-font-specification:'Arial Bold';text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-177.01665"
+ y="220.07283"
+ id="text5181"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan5179"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-177.01665"
+ y="220.07283">test: inference_ordered</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-weight:bold;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:'Arial Bold';text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-178.63324"
+ y="98.67057"
+ id="text15571"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan15569"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.5;stroke-opacity:1"
+ x="-178.63324"
+ y="98.67057">nb_worker_threads = 2 * MIN(nb_queue_pairs, (lcore_count - 1) / 2)</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-weight:bold;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:'Arial Bold';text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-171.18085"
+ y="89.26754"
+ id="text15571-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan15569-9"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.5;stroke-opacity:1"
+ x="-171.18085"
+ y="89.26754">inferences_per_queue_pair = repetitions / nb_queue_pairs</tspan></text>
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.75;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#RoundedArrow);marker-end:url(#RoundedArrow)"
+ d="m 179.14101,85.813438 0,-12.588618"
+ id="path31090"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1924" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724"
+ width="32.290321"
+ height="11.709678"
+ x="216.5636"
+ y="93.411476"
+ ry="2" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-4"
+ width="32.290321"
+ height="11.709678"
+ x="216.5636"
+ y="116.22478"
+ ry="2" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-6"
+ width="32.290321"
+ height="11.709678"
+ x="216.5636"
+ y="139.03798"
+ ry="2" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.354849;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.709699, 0.709699;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924-6"
+ width="44.145252"
+ height="72.532341"
+ x="210.6364"
+ y="85.813477"
+ ry="4.31247" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-0"
+ width="32.290321"
+ height="11.709678"
+ x="109.42807"
+ y="93.411476"
+ ry="2" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-6-7"
+ width="32.290321"
+ height="11.709678"
+ x="109.42807"
+ y="139.03798"
+ ry="2"
+ inkscape:connector-avoid="true" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-4-8"
+ width="32.290321"
+ height="11.709678"
+ x="109.42807"
+ y="116.22478"
+ ry="2"
+ inkscape:connector-avoid="true" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.354849;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.709699, 0.709699;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924-6-3"
+ width="44.145252"
+ height="72.532341"
+ x="103.50092"
+ y="85.813477"
+ ry="4.31247" />
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/tools/img/mldev_model_ops_subtest_a.svg b/doc/guides/tools/img/mldev_model_ops_subtest_a.svg
new file mode 100644
index 0000000000..ed12cc5a05
--- /dev/null
+++ b/doc/guides/tools/img/mldev_model_ops_subtest_a.svg
@@ -0,0 +1,420 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- SPDX-License-Identifier: BSD-3-Clause -->
+<!-- Copyright (c) 2022 Marvell. -->
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="275mm"
+ height="135mm"
+ viewBox="0 0 275 135"
+ version="1.1"
+ id="svg5"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="mldev_model_ops_subtest_d.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg">
+ <sodipodi:namedview
+ id="namedview7"
+ pagecolor="#ffffff"
+ bordercolor="#000000"
+ borderopacity="0.25"
+ inkscape:showpageshadow="2"
+ inkscape:pageopacity="0.0"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1"
+ inkscape:document-units="mm"
+ showgrid="false"
+ inkscape:zoom="0.72515432"
+ inkscape:cx="372.33454"
+ inkscape:cy="401.29389"
+ inkscape:window-width="1920"
+ inkscape:window-height="986"
+ inkscape:window-x="-11"
+ inkscape:window-y="-11"
+ inkscape:window-maximized="1"
+ inkscape:current-layer="layer1" />
+ <defs
+ id="defs2">
+ <marker
+ style="overflow:visible"
+ id="RoundedArrow"
+ refX="6"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="RoundedArrow"
+ markerWidth="6.1347523"
+ markerHeight="5.9304948"
+ viewBox="0 0 6.1347524 5.9304951"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.7)"
+ d="m -0.21114562,-4.1055728 6.42229122,3.21114561 a 1,1 90 0 1 0,1.78885438 L -0.21114562,4.1055728 A 1.236068,1.236068 31.717474 0 1 -2,3 v -6 a 1.236068,1.236068 148.28253 0 1 1.78885438,-1.1055728 z"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:none"
+ id="path1367" />
+ </marker>
+ </defs>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1">
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 62.203489,25.425124 H 80.823463"
+ id="path1916"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234"
+ inkscape:connection-end="#rect234-07" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 128.19001,25.425124 h 18.6197"
+ id="path1918"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-07"
+ inkscape:connection-end="#rect234-6" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 194.17681,25.425125 h 18.61942"
+ id="path1922"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-6"
+ inkscape:connection-end="#rect234-0-6-6" />
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.633453;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234"
+ width="47.366547"
+ height="17.366549"
+ x="14.836943"
+ y="16.74185"
+ ry="1.4985789"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-25.888845"
+ y="38.514706"
+ id="text290"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-25.888845"
+ y="38.514706">Model 0 / Load</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.633453;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-07"
+ width="47.366547"
+ height="17.366549"
+ x="80.823463"
+ y="16.74185"
+ ry="1.4985789"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-25.888845"
+ y="104.31795"
+ id="text290-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-11"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-25.888845"
+ y="104.31795">Model 0 / Start</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.6329;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-6"
+ width="47.3671"
+ height="17.33604"
+ x="146.80971"
+ y="16.757105"
+ ry="1.4959463" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-25.362436"
+ y="170.39679"
+ id="text290-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-15"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-25.362436"
+ y="170.39679">Model 0 / Stop</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6"
+ width="47.3671"
+ height="17.33604"
+ x="212.79623"
+ y="16.757105"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-25.888849"
+ y="236.47427"
+ id="text290-3-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-25.888849"
+ y="236.47427">Model 0 / Unload</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 236.47978,34.093145 V 44.957249 H 38.520216 v 10.84885"
+ id="path1924"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6"
+ inkscape:connection-end="#rect234-0-6-6-3" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 62.203766,64.474119 H 80.823189"
+ id="path1926"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-3"
+ inkscape:connection-end="#rect234-0-6-6-2" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 128.19029,64.474119 h 18.61942"
+ id="path1928"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-2"
+ inkscape:connection-end="#rect234-0-6-1" />
+ <rect
+ style="fill:#fff6d5;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-3"
+ width="47.3671"
+ height="17.33604"
+ x="14.836666"
+ y="55.806099"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-64.937843"
+ y="38.541786"
+ id="text290-3-2-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-64.937843"
+ y="38.541786">Model 1 / Load</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-2"
+ width="47.3671"
+ height="17.33604"
+ x="80.823189"
+ y="55.806099"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-64.937843"
+ y="104.31795"
+ id="text290-3-2-2-28"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-2"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-64.937843"
+ y="104.31795">Model 1 / Start</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6"
+ width="47.3671"
+ height="17.33604"
+ x="212.79623"
+ y="55.806099"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-64.937843"
+ y="236.47427"
+ id="text290-3-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-64.937843"
+ y="236.47427">Model 1 / Unload</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-1"
+ width="47.3671"
+ height="17.33604"
+ x="146.80971"
+ y="55.806099"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-64.41143"
+ y="170.39679"
+ id="text290-3-2-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-8"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-64.41143"
+ y="170.39679">Model 1 / Stop</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 194.17681,64.474119 h 18.61942"
+ id="path1930"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6"
+ inkscape:connection-start="#rect234-0-6-1" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 236.47978,73.142139 V 83.990988 H 38.520216 V 94.85511"
+ id="path1932"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6-7"
+ inkscape:connection-start="#rect234-0-6" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 62.203766,103.52313 h 9.309711 v 1.3e-4 h 9.309712"
+ id="path1934"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-7" />
+ <rect
+ style="fill:#fff6d5;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-7"
+ width="47.3671"
+ height="17.33604"
+ x="14.836666"
+ y="94.85511"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.98687"
+ y="38.541786"
+ id="text290-3-2-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.98687"
+ y="38.541786">Model N / Load</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0"
+ width="47.3671"
+ height="17.33604"
+ x="80.823189"
+ y="94.855164"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.9869"
+ y="104.31795"
+ id="text290-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.9869"
+ y="104.31795">Model N / Start</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-9"
+ width="47.3671"
+ height="17.33604"
+ x="146.80971"
+ y="94.855164"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.4605"
+ y="170.39679"
+ id="text290-3-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.4605"
+ y="170.39679">Model N / Stop</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.633452;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-5"
+ width="47.366547"
+ height="17.366549"
+ x="212.79651"
+ y="94.839836"
+ ry="1.4985789"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.98682"
+ y="236.47427"
+ id="text290-3-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-3"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.98682"
+ y="236.47427">Model N / Unload</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 128.19029,103.52326 h 18.61942"
+ id="path1936"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 194.17681,103.52326 h 9.30985 v -1.5e-4 h 9.30985"
+ id="path1938"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-5" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#fff6d5;fill-rule:evenodd;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-120.71075"
+ y="220.77164"
+ id="text392"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan390"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';fill:#000000;stroke-width:0.5"
+ x="-120.71075"
+ y="220.77164">mldev: model_ops / subtest D</tspan></text>
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.448724;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect2164"
+ width="259.55127"
+ height="119.55128"
+ x="7.7243652"
+ y="7.7243614"
+ ry="1.17494" />
+ </g>
+</svg>
diff --git a/doc/guides/tools/img/mldev_model_ops_subtest_b.svg b/doc/guides/tools/img/mldev_model_ops_subtest_b.svg
new file mode 100644
index 0000000000..173a2c6c05
--- /dev/null
+++ b/doc/guides/tools/img/mldev_model_ops_subtest_b.svg
@@ -0,0 +1,423 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- SPDX-License-Identifier: BSD-3-Clause -->
+<!-- Copyright (c) 2022 Marvell. -->
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="275mm"
+ height="135mm"
+ viewBox="0 0 275 135"
+ version="1.1"
+ id="svg5"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="mldev_model_ops_subtest_a.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg">
+ <sodipodi:namedview
+ id="namedview7"
+ pagecolor="#ffffff"
+ bordercolor="#000000"
+ borderopacity="0.25"
+ inkscape:showpageshadow="2"
+ inkscape:pageopacity="0.0"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1"
+ inkscape:document-units="mm"
+ showgrid="false"
+ inkscape:zoom="0.72515432"
+ inkscape:cx="277.87189"
+ inkscape:cy="401.29389"
+ inkscape:window-width="1920"
+ inkscape:window-height="986"
+ inkscape:window-x="-11"
+ inkscape:window-y="-11"
+ inkscape:window-maximized="1"
+ inkscape:current-layer="layer1" />
+ <defs
+ id="defs2">
+ <marker
+ style="overflow:visible"
+ id="RoundedArrow"
+ refX="6"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="RoundedArrow"
+ markerWidth="6.1347523"
+ markerHeight="5.9304948"
+ viewBox="0 0 6.1347524 5.9304951"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.7)"
+ d="m -0.21114562,-4.1055728 6.42229122,3.21114561 a 1,1 90 0 1 0,1.78885438 L -0.21114562,4.1055728 A 1.236068,1.236068 31.717474 0 1 -2,3 v -6 a 1.236068,1.236068 148.28253 0 1 1.78885438,-1.1055728 z"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:none"
+ id="path1367" />
+ </marker>
+ </defs>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1">
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.671403;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234"
+ width="53.328598"
+ height="17.328598"
+ x="15.144302"
+ y="12.878438"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-22.00646"
+ y="41.803089"
+ id="text290"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-22.00646"
+ y="41.803089">Model 0 / Load</tspan></text>
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.671403;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-07"
+ width="53.328598"
+ height="17.328598"
+ x="87.552399"
+ y="12.878438"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-22.00646"
+ y="114.21132"
+ id="text290-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-11"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-22.00646"
+ y="114.21132">Model 1 / Load</tspan></text>
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.671403;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-6"
+ width="53.328598"
+ height="17.328598"
+ x="206.5271"
+ y="12.878438"
+ ry="1.4953041" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-22.00646"
+ y="233.18588"
+ id="text290-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-15"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-22.00646"
+ y="233.18588">Model N / Load</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 68.4729,21.542737 H 87.552399"
+ id="path1916"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234"
+ inkscape:connection-end="#rect234-07" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 140.881,21.542737 h 65.6461"
+ id="path1918"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-07"
+ inkscape:connection-end="#rect234-6" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 233.1914,30.207036 0,4.918771 H 41.808601 v 4.918772"
+ id="path1922"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-6"
+ inkscape:connection-end="#rect234-0-6-6" />
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6"
+ width="53.328598"
+ height="17.328598"
+ x="15.144302"
+ y="40.044579"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-49.1726"
+ y="41.619808"
+ id="text290-3-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-49.1726"
+ y="41.619808">Model 0 / Start</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-3"
+ width="53.328598"
+ height="17.328598"
+ x="87.552399"
+ y="40.044579"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-49.1726"
+ y="114.02805"
+ id="text290-3-2-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-49.1726"
+ y="114.02805">Model 1 / Start</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-2"
+ width="53.328598"
+ height="17.328598"
+ x="206.5271"
+ y="40.044579"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-49.1726"
+ y="233.00261"
+ id="text290-3-2-2-28"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-2"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-49.1726"
+ y="233.00261">Model N / Start</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 68.4729,48.708878 H 87.552399"
+ id="path1924"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6"
+ inkscape:connection-end="#rect234-0-6-6-3" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 140.881,48.708878 h 65.6461"
+ id="path1926"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-3"
+ inkscape:connection-end="#rect234-0-6-6-2" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 233.1914,57.373177 V 62.29195 H 41.808599 v 4.918774"
+ id="path1928"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-2"
+ inkscape:connection-end="#rect234-0-6-1" />
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6"
+ width="53.328598"
+ height="17.328598"
+ x="87.552399"
+ y="67.210724"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-75.812294"
+ y="114.12037"
+ id="text290-3-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-75.812294"
+ y="114.12037">Model 1 / Stop</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-1"
+ width="53.328598"
+ height="17.328598"
+ x="15.1443"
+ y="67.210724"
+ ry="1.4953041" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-75.812294"
+ y="41.712139"
+ id="text290-3-2-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-8"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-75.812294"
+ y="41.712139">Model 0 / Stop</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-7"
+ width="53.328598"
+ height="17.328598"
+ x="206.5271"
+ y="67.210724"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-75.812294"
+ y="233.09494"
+ id="text290-3-2-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-75.812294"
+ y="233.09494">Model N / Stop</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 68.472898,75.875023 H 87.552399"
+ id="path1930"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6"
+ inkscape:connection-start="#rect234-0-6-1" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 140.881,75.875023 h 65.6461"
+ id="path1932"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6-7"
+ inkscape:connection-start="#rect234-0-6" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 233.1914,84.539322 0,4.91877 H 41.808602 v 4.91877"
+ id="path1934"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0"
+ inkscape:connection-start="#rect234-0-6-7" />
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0"
+ width="53.328598"
+ height="17.328598"
+ x="15.144303"
+ y="94.376862"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.50489"
+ y="41.803085"
+ id="text290-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.50489"
+ y="41.803085">Model 0 / Unload</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-9"
+ width="53.328598"
+ height="17.328598"
+ x="87.552399"
+ y="94.376862"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.50489"
+ y="114.2113"
+ id="text290-3-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.50489"
+ y="114.2113">Model 1 / Unload</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-5"
+ width="53.328598"
+ height="17.328598"
+ x="206.5271"
+ y="94.376862"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.50489"
+ y="233.18588"
+ id="text290-3-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-3"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.50489"
+ y="233.18588">Model N / Unload</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 68.472901,103.04116 H 87.552399"
+ id="path1936"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-9"
+ inkscape:connection-start="#rect234-0" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 140.881,103.04116 h 65.6461"
+ id="path1938"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-5"
+ inkscape:connection-start="#rect234-0-9" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#fff6d5;fill-rule:evenodd;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-119.78458"
+ y="220.32892"
+ id="text392"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan390"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';fill:#000000;stroke-width:0.5"
+ x="-119.78458"
+ y="220.32892">mldev: model_ops / subtest A</tspan></text>
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.442854;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect3967"
+ width="259.55716"
+ height="119.55714"
+ x="7.7214203"
+ y="7.7214317"
+ ry="1.1806604" />
+ </g>
+</svg>
diff --git a/doc/guides/tools/img/mldev_model_ops_subtest_c.svg b/doc/guides/tools/img/mldev_model_ops_subtest_c.svg
new file mode 100644
index 0000000000..f66f146d05
--- /dev/null
+++ b/doc/guides/tools/img/mldev_model_ops_subtest_c.svg
@@ -0,0 +1,366 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- SPDX-License-Identifier: BSD-3-Clause -->
+<!-- Copyright (c) 2022 Marvell. -->
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="275mm"
+ height="135mm"
+ viewBox="0 0 275 135"
+ version="1.1"
+ id="svg5"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="mldev_model_ops_subtest_c.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg">
+ <sodipodi:namedview
+ id="namedview7"
+ pagecolor="#ffffff"
+ bordercolor="#000000"
+ borderopacity="0.25"
+ inkscape:showpageshadow="2"
+ inkscape:pageopacity="0.0"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1"
+ inkscape:document-units="mm"
+ showgrid="false"
+ inkscape:zoom="0.72515432"
+ inkscape:cx="479.89785"
+ inkscape:cy="401.29389"
+ inkscape:window-width="1920"
+ inkscape:window-height="986"
+ inkscape:window-x="-11"
+ inkscape:window-y="-11"
+ inkscape:window-maximized="1"
+ inkscape:current-layer="layer1" />
+ <defs
+ id="defs2">
+ <marker
+ style="overflow:visible"
+ id="RoundedArrow"
+ refX="6"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="RoundedArrow"
+ markerWidth="6.1347523"
+ markerHeight="5.9304948"
+ viewBox="0 0 6.1347524 5.9304951"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.7)"
+ d="m -0.21114562,-4.1055728 6.42229122,3.21114561 a 1,1 90 0 1 0,1.78885438 L -0.21114562,4.1055728 A 1.236068,1.236068 31.717474 0 1 -2,3 v -6 a 1.236068,1.236068 148.28253 0 1 1.78885438,-1.1055728 z"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:none"
+ id="path1367" />
+ </marker>
+ </defs>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1">
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.671403;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234"
+ width="53.328598"
+ height="17.328598"
+ x="12.6443"
+ y="13.208546"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-22.336561"
+ y="39.303089"
+ id="text290"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-22.336561"
+ y="39.303089">Model 0 / Load</tspan></text>
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.671403;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-07"
+ width="53.328598"
+ height="17.328598"
+ x="85.052399"
+ y="13.208546"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-22.336561"
+ y="111.71132"
+ id="text290-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-11"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-22.336561"
+ y="111.71132">Model 1 / Load</tspan></text>
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.671403;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-6"
+ width="53.328598"
+ height="17.328598"
+ x="204.0271"
+ y="13.208546"
+ ry="1.4953041" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-22.336561"
+ y="230.68588"
+ id="text290-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-15"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-22.336561"
+ y="230.68588">Model N / Load</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 65.972898,21.872845 H 85.052399"
+ id="path1916"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234"
+ inkscape:connection-end="#rect234-07" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 138.381,21.872845 h 65.6461"
+ id="path1918"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-07"
+ inkscape:connection-end="#rect234-6" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 230.6914,30.537144 V 36.33787 H 39.308599 v 5.800726"
+ id="path1922"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-6"
+ inkscape:connection-end="#rect234-0-6-6" />
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6"
+ width="53.328598"
+ height="17.328598"
+ x="12.6443"
+ y="42.138596"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-51.266617"
+ y="39.119808"
+ id="text290-3-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-51.266617"
+ y="39.119808">Model 0 / Start</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-3"
+ width="53.328598"
+ height="17.328598"
+ x="85.052399"
+ y="42.138596"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-51.266617"
+ y="111.52805"
+ id="text290-3-2-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-51.266617"
+ y="111.52805">Model 0 / Stop</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 65.972898,50.802895 H 85.052399"
+ id="path1924"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6"
+ inkscape:connection-end="#rect234-0-6-6-3" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 184.94759,79.732941 H 204.0271"
+ id="path1930"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6"
+ inkscape:connection-start="#rect234-0-6-1" />
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0"
+ width="53.328598"
+ height="17.328598"
+ x="12.644301"
+ y="99.998688"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-109.12671"
+ y="39.303085"
+ id="text290-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-109.12671"
+ y="39.303085">Model 0 / Unload</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-9"
+ width="53.328598"
+ height="17.328598"
+ x="85.052399"
+ y="99.998688"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-109.12671"
+ y="111.7113"
+ id="text290-3-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-109.12671"
+ y="111.7113">Model 1 / Unload</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-5"
+ width="53.328598"
+ height="17.328598"
+ x="204.0271"
+ y="99.998688"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-109.12671"
+ y="230.68588"
+ id="text290-3-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-3"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-109.12671"
+ y="230.68588">Model N / Unload</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 65.972899,108.66299 h 19.0795"
+ id="path1936"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-9"
+ inkscape:connection-start="#rect234-0" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 138.381,108.66299 h 65.6461"
+ id="path1938"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-5"
+ inkscape:connection-start="#rect234-0-9" />
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6"
+ width="53.328598"
+ height="17.328598"
+ x="204.0271"
+ y="71.068642"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-79.670258"
+ y="230.59494"
+ id="text290-3-2-2-28"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-2"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-79.670258"
+ y="230.59494">Model N / Stop</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-1"
+ width="53.328598"
+ height="17.328598"
+ x="131.61899"
+ y="71.068642"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-80.196663"
+ y="158.0945"
+ id="text290-3-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-80.196663"
+ y="158.0945">Model N / Start</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 138.381,50.802895 h 19.90229 v 20.265747"
+ id="path1158"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-3"
+ inkscape:connection-end="#rect234-0-6-1" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 230.6914,88.39724 v 5.800724 H 39.3086 v 5.800724"
+ id="path1160"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6"
+ inkscape:connection-end="#rect234-0" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#fff6d5;fill-rule:evenodd;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-123.28999"
+ y="217.99152"
+ id="text392"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan390"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';fill:#000000;stroke-width:0.5"
+ x="-123.28999"
+ y="217.99152">mldev: model_ops / subtest C</tspan></text>
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.441855;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect3015"
+ width="259.55814"
+ height="119.55814"
+ x="7.720932"
+ y="7.7209282"
+ ry="1.1638433" />
+ </g>
+</svg>
diff --git a/doc/guides/tools/img/mldev_model_ops_subtest_d.svg b/doc/guides/tools/img/mldev_model_ops_subtest_d.svg
new file mode 100644
index 0000000000..3e2b89ad25
--- /dev/null
+++ b/doc/guides/tools/img/mldev_model_ops_subtest_d.svg
@@ -0,0 +1,424 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- SPDX-License-Identifier: BSD-3-Clause -->
+<!-- Copyright (c) 2022 Marvell. -->
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="275mm"
+ height="135mm"
+ viewBox="0 0 275 135"
+ version="1.1"
+ id="svg5"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="mldev_model_ops_subtest_b.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg">
+ <sodipodi:namedview
+ id="namedview7"
+ pagecolor="#ffffff"
+ bordercolor="#000000"
+ borderopacity="0.25"
+ inkscape:showpageshadow="2"
+ inkscape:pageopacity="0.0"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1"
+ inkscape:document-units="mm"
+ showgrid="false"
+ inkscape:zoom="0.72515432"
+ inkscape:cx="324.06895"
+ inkscape:cy="295.1096"
+ inkscape:window-width="1920"
+ inkscape:window-height="986"
+ inkscape:window-x="-11"
+ inkscape:window-y="-11"
+ inkscape:window-maximized="1"
+ inkscape:current-layer="layer1" />
+ <defs
+ id="defs2">
+ <marker
+ style="overflow:visible"
+ id="RoundedArrow"
+ refX="6"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="RoundedArrow"
+ markerWidth="6.1347523"
+ markerHeight="5.9304948"
+ viewBox="0 0 6.1347524 5.9304951"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.7)"
+ d="m -0.21114562,-4.1055728 6.42229122,3.21114561 a 1,1 90 0 1 0,1.78885438 L -0.21114562,4.1055728 A 1.236068,1.236068 31.717474 0 1 -2,3 v -6 a 1.236068,1.236068 148.28253 0 1 1.78885438,-1.1055728 z"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:none"
+ id="path1367" />
+ </marker>
+ </defs>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1">
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.633453;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234"
+ width="47.366547"
+ height="17.366549"
+ x="14.864025"
+ y="14.24185"
+ ry="1.4985789"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-23.388845"
+ y="38.541786"
+ id="text290"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-23.388845"
+ y="38.541786">Model 0 / Load</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.633453;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-07"
+ width="47.366547"
+ height="17.366549"
+ x="81.026939"
+ y="14.24185"
+ ry="1.4985789"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-23.388845"
+ y="104.52142"
+ id="text290-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-11"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-23.388845"
+ y="104.52142">Model 0 / Start</tspan></text>
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.6329;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-6"
+ width="47.3671"
+ height="17.33604"
+ x="147.18958"
+ y="14.257105"
+ ry="1.4959463" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-23.388849"
+ y="170.86761"
+ id="text290-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-15"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-23.388849"
+ y="170.86761">Model 1 / Load</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 62.230572,22.925124 H 81.026939"
+ id="path1916"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234"
+ inkscape:connection-end="#rect234-07" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 128.39349,22.925124 h 18.79609"
+ id="path1918"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-07"
+ inkscape:connection-end="#rect234-6" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 194.55667,22.925125 18.79581,0"
+ id="path1922"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-6"
+ inkscape:connection-end="#rect234-0-6-6" />
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6"
+ width="47.3671"
+ height="17.33604"
+ x="213.35248"
+ y="14.257105"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-23.388849"
+ y="236.84723"
+ id="text290-3-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-23.388849"
+ y="236.84723">Model 1 / Start</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 237.03603,31.593145 0,5.328675 H 38.547297 v 5.313421"
+ id="path1924"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6"
+ inkscape:connection-end="#rect234-0-6-6-3" />
+ <rect
+ style="fill:#fff6d5;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-3"
+ width="47.3671"
+ height="17.33604"
+ x="14.863747"
+ y="42.235241"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-51.366989"
+ y="38.541786"
+ id="text290-3-2-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-51.366989"
+ y="38.541786">Model N / Load</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-2"
+ width="47.3671"
+ height="17.33604"
+ x="81.026657"
+ y="42.235241"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-51.366989"
+ y="104.52142"
+ id="text290-3-2-2-28"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-2"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-51.366989"
+ y="104.52142">Model N / Start</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 62.230846,50.903261 H 81.026657"
+ id="path1926"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-3"
+ inkscape:connection-end="#rect234-0-6-6-2" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 128.39376,50.903261 h 42.47937 v 19.294584"
+ id="path1928"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-2"
+ inkscape:connection-end="#rect234-0-6-1" />
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6"
+ width="47.3671"
+ height="17.33604"
+ x="213.35248"
+ y="70.197845"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-79.32959"
+ y="237.03052"
+ id="text290-3-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-79.32959"
+ y="237.03052">Model N / Unload</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-1"
+ width="47.3671"
+ height="17.33604"
+ x="147.18958"
+ y="70.197845"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-78.803177"
+ y="170.77666"
+ id="text290-3-2-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-8"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-78.803177"
+ y="170.77666">Model N / Stop</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 194.55667,78.865866 h 18.79581"
+ id="path1930"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6"
+ inkscape:connection-start="#rect234-0-6-1" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 237.03603,87.533886 v 5.313417 H 38.547297 v 5.328677"
+ id="path1932"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6-7"
+ inkscape:connection-start="#rect234-0-6" />
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-7"
+ width="47.3671"
+ height="17.33604"
+ x="14.863747"
+ y="98.17598"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-106.78132"
+ y="38.450832"
+ id="text290-3-2-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-106.78132"
+ y="38.450832">Model 1 / Stop</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 62.230846,106.844 H 81.026657"
+ id="path1934"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0"
+ inkscape:connection-start="#rect234-0-6-7" />
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0"
+ width="47.3671"
+ height="17.33604"
+ x="81.026657"
+ y="98.17598"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-107.30773"
+ y="104.7047"
+ id="text290-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-107.30773"
+ y="104.7047">Model 1 / Unload</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-9"
+ width="47.3671"
+ height="17.33604"
+ x="147.18958"
+ y="98.17598"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-106.78133"
+ y="170.77666"
+ id="text290-3-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-106.78133"
+ y="170.77666">Model 0 / Stop</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.633452;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-5"
+ width="47.366547"
+ height="17.366549"
+ x="213.35277"
+ y="98.160721"
+ ry="1.4985789"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-107.30773"
+ y="237.03053"
+ id="text290-3-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-3"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-107.30773"
+ y="237.03053">Model 0 / Unload</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 128.39376,106.844 h 18.79582"
+ id="path1936"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-9"
+ inkscape:connection-start="#rect234-0" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 194.55667,106.844 9.39805,0 v 0 h 9.39805"
+ id="path1938"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-5"
+ inkscape:connection-start="#rect234-0-9" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#fff6d5;fill-rule:evenodd;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-122.00021"
+ y="222.26495"
+ id="text392"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan390"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';fill:#000000;stroke-width:0.5"
+ x="-122.00021"
+ y="222.26495">mldev: model_ops / subest B</tspan></text>
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.462341;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect3475"
+ width="259.53769"
+ height="119.53766"
+ x="7.7311554"
+ y="7.7311707"
+ ry="1.2186421" />
+ </g>
+</svg>
diff --git a/doc/guides/tools/index.rst b/doc/guides/tools/index.rst
index f1f5b94c8c..6f84fc31ff 100644
--- a/doc/guides/tools/index.rst
+++ b/doc/guides/tools/index.rst
@@ -21,4 +21,5 @@ DPDK Tools User Guides
comp_perf
testeventdev
testregex
+ testmldev
dts
diff --git a/doc/guides/tools/testmldev.rst b/doc/guides/tools/testmldev.rst
new file mode 100644
index 0000000000..845c2d9381
--- /dev/null
+++ b/doc/guides/tools/testmldev.rst
@@ -0,0 +1,441 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright (c) 2022 Marvell.
+
+dpdk-test-mldev Application
+===========================
+
+The ``dpdk-test-mldev`` tool is a Data Plane Development Kit (DPDK) application that allows testing
+various mldev use cases. This application has a generic framework to add new mldev based test cases
+to verify functionality and measure the performance of inference execution on DPDK ML devices.
+
+
+Application and Options
+-----------------------
+
+The application has a number of command line options:
+
+.. code-block:: console
+
+ dpdk-test-mldev [EAL Options] -- [application options]
+
+EAL Options
+~~~~~~~~~~~
+
+The following are the EAL command-line options that can be used with the ``dpdk-test-mldev``
+application. See the DPDK Getting Started Guides for more information on these options.
+
+* ``-c <COREMASK>`` or ``-l <CORELIST>``
+
+ Set the hexadecimal bitmask of the cores to run on. The corelist is a list of cores to use.
+
+* ``-a <PCI_ID>``
+
+ Attach a PCI based ML device. Specific to drivers using a PCI based ML devices.
+
+* ``--vdev <driver>``
+
+ Add a virtual mldev device. Specific to drivers using a ML virtual device.
+
+
+Application Options
+~~~~~~~~~~~~~~~~~~~
+
+The following are the command-line options supported by the test application.
+
+* ``--test <name>``
+
+ ML tests are divided into two groups, Model and Device tests and Inference tests. Test
+ name one of the following supported tests.
+
+ **ML Device Tests** ::
+
+ device_ops
+
+ **ML Model Tests** ::
+
+ model_ops
+
+ **ML Inference Tests** ::
+
+ inference_ordered
+ inference_interleave
+
+* ``--dev_id <n>``
+
+ Set the device id of the ML device to be used for the test. Default value is `0`.
+
+* ``--socket_id <n>``
+
+ Set the socket id of the application resources. Default value is `SOCKET_ID_ANY`.
+
+* ``--debug``
+
+ Enable the tests to run in debug mode.
+
+* ``--models <model_list>``
+
+ Set the list of model files to be used for the tests. Application expects the
+ ``model_list`` in comma separated form (i.e. ``--models model_A.bin,model_B.bin``).
+ Maximum number of models supported by the test is ``8``.
+
+* ``--filelist <file_list>``
+
+ Set the list of model, input, output and reference files to be used for the tests.
+ Application expects the ``file_list`` to be in comma separated form
+ (i.e. ``--filelist <model,input,output>[,reference]``).
+
+ Multiple filelist entries can be specified when running the tests with multiple models.
+ Both quantized and dequantized outputs are written to the disk. Dequantized output file
+ would have the name specified by the user through ``--filelist`` option. A suffix ``.q``
+ is appended to quantized output filename. Maximum number of filelist entries supported
+ by the test is ``8``.
+
+* ``--repetitions <n>``
+
+ Set the number of inference repetitions to be executed in the test per each model. Default
+ value is `1`.
+
+* ``--burst_size <n>``
+
+ Set the burst size to be used when enqueuing / dequeuing inferences. Default value is `1`.
+
+* ``--queue_pairs <n>``
+
+ Set the number of queue-pairs to be used for inference enqueue and dequeue operations.
+ Default value is `1`.
+
+* ``--queue_size <n>``
+
+ Set the size of queue-pair to be created for inference enqueue / dequeue operations.
+ Queue size would translate into `rte_ml_dev_qp_conf::nb_desc` field during queue-pair
+ creation. Default value is `1`.
+
+* ``--batches <n>``
+
+ Set the number batches in the input file provided for inference run. When not specified
+ the test would assume the number of batches is equal to the batch size of the model.
+
+* ``--tolerance <n>``
+
+ Set the tolerance value in percentage to be used for output validation. Default value
+ is `0`.
+
+* ``--stats``
+
+ Enable reporting device extended stats.
+
+
+ML Device Tests
+-------------------------
+
+ML device tests are functional tests to validate ML device APIs. Device tests validate the ML device
+handling APIs configure, close, start and stop APIs.
+
+
+Application Options
+~~~~~~~~~~~~~~~~~~~
+
+Supported command line options for the `model_ops` test are following::
+
+ --debug
+ --test
+ --dev_id
+ --socket_id
+ --queue_pairs
+ --queue_size
+
+
+DEVICE_OPS Test
+~~~~~~~~~~~~~~~
+
+Device ops test validates the device configuration and reconfiguration support. The test configures
+ML device based on the option ``--queue_pairs`` and ``--queue_size`` specified by the user, and
+later reconfigures the ML device with the number of queue pairs and queue size based the maximum
+specified through the device info.
+
+
+Example
+^^^^^^^
+
+Command to run device_ops test:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=device_ops
+
+
+Command to run device_ops test with user options:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=device_ops --queue_pairs <M> --queue_size <N>
+
+
+ML Model Tests
+-------------------------
+
+Model tests are functional tests to validate ML model APIs. Model tests validate the functioning
+of APIs to load, start, stop and unload ML models.
+
+
+Application Options
+~~~~~~~~~~~~~~~~~~~
+
+Supported command line options for the `model_ops` test are following::
+
+ --debug
+ --test
+ --dev_id
+ --socket_id
+ --models
+
+
+List of model files to be used for the `model_ops` test can be specified through the option
+``--models <model_list>`` as a comma separated list. Maximum number of models supported in
+the test is `8`.
+
+.. Note::
+
+ * The ``--models <model_list>`` is a mandatory option for running this test.
+ * Options not supported by the test are ignored if specified.
+
+
+MODEL_OPS Test
+~~~~~~~~~~~~~~
+
+The test is a collection of multiple sub-tests, each with a different order of slow-path
+operations when handling with `N` number of models.
+
+
+**Sub-test A:** executes the sequence of load / start / stop / unload for a model in order,
+followed by next model.
+.. _figure_mldev_model_ops_subtest_a:
+
+.. figure:: img/mldev_model_ops_subtest_a.*
+
+ Execution sequence of model_ops subtest A.
+
+
+**Sub-test B:** executes load for all models, followed by a start for all models. Upon successful
+start of all models, stop is invoked for all models followed by unload.
+.. _figure_mldev_model_ops_subtest_b:
+
+.. figure:: img/mldev_model_ops_subtest_b.*
+
+ Execution sequence of model_ops subtest B.
+
+
+**Sub-test C:** loads all models, followed by a start and stop of all models in order. Upon
+completion of stop, unload is invoked for all models.
+.. _figure_mldev_model_ops_subtest_c:
+
+.. figure:: img/mldev_model_ops_subtest_c.*
+
+ Execution sequence of model_ops subtest C.
+
+
+**Sub-test D:** executes load and start for all models available. Upon successful start of all
+models, stop and stop is executed for the models.
+.. _figure_mldev_model_ops_subtest_d:
+
+.. figure:: img/mldev_model_ops_subtest_d.*
+
+ Execution sequence of model_ops subtest D.
+
+
+Example
+^^^^^^^
+
+Command to run model_ops test:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=model_ops --models model_1.bin,model_2.bin,model_3.bin, model_4.bin
+
+
+ML Inference Tests
+------------------
+
+Inference tests are a set of tests to validate end-to-end inference execution on ML device.
+These tests executes the full sequence of operations required to run inferences with one or
+multiple models.
+
+Application Options
+~~~~~~~~~~~~~~~~~~~
+
+Supported command line options for inference tests are following::
+
+ --debug
+ --test
+ --dev_id
+ --socket_id
+ --filelist
+ --repetitions
+ --burst_size
+ --queue_pairs
+ --queue_size
+ --batches
+ --tolerance
+ --stats
+
+
+List of files to be used for the inference tests can be specified through the option
+``--filelist <file_list>`` as a comma separated list. A filelist entry would be of the format
+``--filelist <model_file,input_file,output_file>[,reference_file]`` and is used to specify the
+list of files required to test with a single model. Multiple filelist entries are supported by
+the test, one entry per model. Maximum number of file entries supported by the test is `8`.
+
+When ``--burst_size <num>`` option is specified for the test, enqueue and dequeue burst would
+try to enqueue or dequeue ``num`` number of inferences per each call respectively.
+
+In the inference test, a pair of lcores are mapped to each queue pair. Minimum number of lcores
+required for the tests is equal to ``(queue_pairs * 2 + 1)``.
+
+Output validation of inference would be enabled only when a reference file is specified through
+the ``--filelist`` option. Application would additionally consider the tolerance value provided
+through ``--tolerance`` option during validation. When the tolerance values is 0, CRC32 hash of
+inference output and reference output are compared. When the tolerance is non-zero, element wise
+comparison of output is performed. Validation is considered as successful only when all the
+elements of the output tensor are with in the tolerance range specified.
+
+When ``--debug`` option is specified, tests are run in debug mode.
+
+Enabling ``--stats`` would print the extended stats supported by the driver.
+
+.. Note::
+
+ * The ``--filelist <file_list>`` is a mandatory option for running inference tests.
+ * Options not supported by the tests are ignored if specified.
+ * Element wise comparison is not supported when the output dtype is either fp8, fp16
+ or bfloat16. This is applicable only when the tolerance is greater than zero and for
+ pre-quantized models only.
+
+
+INFERENCE_ORDERED Test
+~~~~~~~~~~~~~~~~~~~~~~
+
+This is a functional test for validating the end-to-end inference execution on ML device. This
+test configures ML device and queue pairs as per the queue-pair related options (queue_pairs and
+queue_size) specified by the user. Upon successful configuration of the device and queue pairs,
+the first model specified through the filelist is loaded to the device and inferences are enqueued
+by a pool of worker threads to the ML device. Total number of inferences enqueued for the model
+are equal to the repetitions specified. A dedicated pool of worker threads would dequeue the
+inferences from the device. The model is unloaded upon completion of all inferences for the model.
+The test would continue loading and executing inference requests for all models specified
+through ``filelist`` option in an ordered manner.
+
+.. _figure_mldev_inference_ordered:
+
+.. figure:: img/mldev_inference_ordered.*
+
+ Execution of inference_ordered on single model.
+
+
+Example
+^^^^^^^
+
+Example command to run inference_ordered test:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_ordered --filelist model.bin,input.bin,output.bin
+
+Example command to run inference_ordered with output validation using tolerance of `1%``:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_ordered --filelist model.bin,input.bin,output.bin,reference.bin \
+ --tolerance 1.0
+
+Example command to run inference_ordered test with multiple queue-pairs and queue size:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_ordered --filelist model.bin,input.bin,output.bin \
+ --queue_pairs 4 --queue_size 16
+
+Example command to run inference_ordered test with a specific burst size:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_ordered --filelist model.bin,input.bin,output.bin \
+ --burst_size 12
+
+
+INFERENCE_INTERLEAVE Test
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This is a stress test for validating the end-to-end inference execution on ML device. The test
+configures the ML device and queue pairs as per the queue-pair related options (queue_pairs
+and queue_size) specified by the user. Upon successful configuration of the device and queue
+pairs, all models specified through the filelist are loaded to the device. Inferences for multiple
+models are enqueued by a pool of worker threads in parallel. Inference execution by the device is
+interleaved between multiple models. Total number of inferences enqueued for a model are equal to
+the repetitions specified. An additional pool of threads would dequeue the inferences from the
+device. Models would be unloaded upon completion of inferences for all models loaded.
+
+
+.. _figure_mldev_inference_interleave:
+
+.. figure:: img/mldev_inference_interleave.*
+
+ Execution of inference_interleave on single model.
+
+
+Example
+^^^^^^^
+
+Example command to run inference_interleave test:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_interleave --filelist model.bin,input.bin,output.bin
+
+
+Example command to run inference_interleave test with multiple models:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_interleave --filelist model_A.bin,input_A.bin,output_A.bin \
+ --filelist model_B.bin,input_B.bin,output_B.bin
+
+
+Example command to run inference_interleave test with multiple models ad output validation
+using tolerance of `2.0%``:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_interleave \
+ --filelist model_A.bin,input_A.bin,output_A.bin,reference_A.bin \
+ --filelist model_B.bin,input_B.bin,output_B.bin,reference_B.bin \
+ --tolerance 2.0
+
+Example command to run inference_interleave test with multiple queue-pairs and queue size
+and burst size:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_interleave --filelist model.bin,input.bin,output.bin \
+ --queue_pairs 8 --queue_size 12 --burst_size 16
+
+
+Debug mode
+----------
+
+ML tests can be executed in debug mode by enabling the option ``--debug``. Execution of tests in
+debug mode would enable additional prints.
+
+When a validation failure is observed, output from that buffer is written to the disk, with the
+filenames having similar convention when the test has passed. Additionally index of the buffer
+would be appended to the filenames.
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [PATCH v6 12/12] app/mldev: add documentation for mldev test cases
2023-03-11 15:09 ` [PATCH v6 12/12] app/mldev: add documentation for mldev test cases Srikanth Yalavarthi
@ 2023-03-16 17:50 ` Thomas Monjalon
2023-03-16 17:56 ` [EXT] " Srikanth Yalavarthi
0 siblings, 1 reply; 122+ messages in thread
From: Thomas Monjalon @ 2023-03-16 17:50 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
11/03/2023 16:09, Srikanth Yalavarthi:
> Added documentation specific to mldev test cases. Added details
> about all test cases and option supported by individual tests.
Would it be possible to add documentation of test cases while adding the code?
Keeping related code and doc changes in the same commit may help in future
when looking at the git history.
^ permalink raw reply [flat|nested] 122+ messages in thread
* RE: [EXT] Re: [PATCH v6 12/12] app/mldev: add documentation for mldev test cases
2023-03-16 17:50 ` Thomas Monjalon
@ 2023-03-16 17:56 ` Srikanth Yalavarthi
2023-03-16 18:03 ` Thomas Monjalon
0 siblings, 1 reply; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-16 17:56 UTC (permalink / raw)
To: Thomas Monjalon
Cc: dev, Shivah Shankar Shankar Narayan Rao,
Jerin Jacob Kollanukkaran, Anup Prabhu, Prince Takkar,
Parijat Shukla, Srikanth Yalavarthi
> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: 16 March 2023 23:21
> To: Srikanth Yalavarthi <syalavarthi@marvell.com>
> Cc: dev@dpdk.org; Shivah Shankar Shankar Narayan Rao
> <sshankarnara@marvell.com>; Jerin Jacob Kollanukkaran
> <jerinj@marvell.com>; Anup Prabhu <aprabhu@marvell.com>; Prince Takkar
> <ptakkar@marvell.com>; Parijat Shukla <pshukla@marvell.com>
> Subject: [EXT] Re: [PATCH v6 12/12] app/mldev: add documentation for
> mldev test cases
>
> External Email
>
> ----------------------------------------------------------------------
> 11/03/2023 16:09, Srikanth Yalavarthi:
> > Added documentation specific to mldev test cases. Added details about
> > all test cases and option supported by individual tests.
>
> Would it be possible to add documentation of test cases while adding the
> code?
> Keeping related code and doc changes in the same commit may help in
> future when looking at the git history.
>
Since, there was a lot of moving around multiple patches during the app development. I have kept the entire documentation as a single patch for ease of development.
It is possible to incrementally add documentation to the previous 11 patches in the series. But, that would need rework on all patches.
Pls, suggest?
>
^ permalink raw reply [flat|nested] 122+ messages in thread
* Re: [EXT] Re: [PATCH v6 12/12] app/mldev: add documentation for mldev test cases
2023-03-16 17:56 ` [EXT] " Srikanth Yalavarthi
@ 2023-03-16 18:03 ` Thomas Monjalon
2023-03-16 18:07 ` Srikanth Yalavarthi
0 siblings, 1 reply; 122+ messages in thread
From: Thomas Monjalon @ 2023-03-16 18:03 UTC (permalink / raw)
To: Srikanth Yalavarthi
Cc: dev, Shivah Shankar Shankar Narayan Rao,
Jerin Jacob Kollanukkaran, Anup Prabhu, Prince Takkar,
Parijat Shukla, Srikanth Yalavarthi
16/03/2023 18:56, Srikanth Yalavarthi:
> From: Thomas Monjalon <thomas@monjalon.net>
> > 11/03/2023 16:09, Srikanth Yalavarthi:
> > > Added documentation specific to mldev test cases. Added details about
> > > all test cases and option supported by individual tests.
> >
> > Would it be possible to add documentation of test cases while adding the
> > code?
> > Keeping related code and doc changes in the same commit may help in
> > future when looking at the git history.
> >
> Since, there was a lot of moving around multiple patches during the app development. I have kept the entire documentation as a single patch for ease of development.
>
> It is possible to incrementally add documentation to the previous 11 patches in the series. But, that would need rework on all patches.
>
> Pls, suggest?
That's exactly what I did for your driver series.
First I've split the doc in multiple patches (in the same order as code patches),
then I've squashed the doc patches with the code patches.
^ permalink raw reply [flat|nested] 122+ messages in thread
* RE: [EXT] Re: [PATCH v6 12/12] app/mldev: add documentation for mldev test cases
2023-03-16 18:03 ` Thomas Monjalon
@ 2023-03-16 18:07 ` Srikanth Yalavarthi
2023-03-16 21:32 ` Srikanth Yalavarthi
0 siblings, 1 reply; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-16 18:07 UTC (permalink / raw)
To: Thomas Monjalon
Cc: dev, Shivah Shankar Shankar Narayan Rao,
Jerin Jacob Kollanukkaran, Anup Prabhu, Prince Takkar,
Parijat Shukla, Srikanth Yalavarthi
> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: 16 March 2023 23:34
> To: Srikanth Yalavarthi <syalavarthi@marvell.com>
> Cc: dev@dpdk.org; Shivah Shankar Shankar Narayan Rao
> <sshankarnara@marvell.com>; Jerin Jacob Kollanukkaran
> <jerinj@marvell.com>; Anup Prabhu <aprabhu@marvell.com>; Prince Takkar
> <ptakkar@marvell.com>; Parijat Shukla <pshukla@marvell.com>; Srikanth
> Yalavarthi <syalavarthi@marvell.com>
> Subject: Re: [EXT] Re: [PATCH v6 12/12] app/mldev: add documentation for
> mldev test cases
>
> 16/03/2023 18:56, Srikanth Yalavarthi:
> > From: Thomas Monjalon <thomas@monjalon.net>
> > > 11/03/2023 16:09, Srikanth Yalavarthi:
> > > > Added documentation specific to mldev test cases. Added details
> > > > about all test cases and option supported by individual tests.
> > >
> > > Would it be possible to add documentation of test cases while adding
> > > the code?
> > > Keeping related code and doc changes in the same commit may help in
> > > future when looking at the git history.
> > >
> > Since, there was a lot of moving around multiple patches during the app
> development. I have kept the entire documentation as a single patch for ease
> of development.
> >
> > It is possible to incrementally add documentation to the previous 11
> patches in the series. But, that would need rework on all patches.
> >
> > Pls, suggest?
>
> That's exactly what I did for your driver series.
> First I've split the doc in multiple patches (in the same order as code
> patches), then I've squashed the doc patches with the code patches.
>
Ack, will submit v7 with changes.
>
^ permalink raw reply [flat|nested] 122+ messages in thread
* RE: [EXT] Re: [PATCH v6 12/12] app/mldev: add documentation for mldev test cases
2023-03-16 18:07 ` Srikanth Yalavarthi
@ 2023-03-16 21:32 ` Srikanth Yalavarthi
0 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-16 21:32 UTC (permalink / raw)
To: Thomas Monjalon
Cc: dev, Shivah Shankar Shankar Narayan Rao,
Jerin Jacob Kollanukkaran, Anup Prabhu, Prince Takkar,
Parijat Shukla, Srikanth Yalavarthi
> -----Original Message-----
> From: Srikanth Yalavarthi <syalavarthi@marvell.com>
> Sent: 16 March 2023 23:37
> To: Thomas Monjalon <thomas@monjalon.net>
> Cc: dev@dpdk.org; Shivah Shankar Shankar Narayan Rao
> <sshankarnara@marvell.com>; Jerin Jacob Kollanukkaran
> <jerinj@marvell.com>; Anup Prabhu <aprabhu@marvell.com>; Prince Takkar
> <ptakkar@marvell.com>; Parijat Shukla <pshukla@marvell.com>; Srikanth
> Yalavarthi <syalavarthi@marvell.com>; Srikanth Yalavarthi
> <syalavarthi@marvell.com>
> Subject: RE: [EXT] Re: [PATCH v6 12/12] app/mldev: add documentation for
> mldev test cases
>
> > -----Original Message-----
> > From: Thomas Monjalon <thomas@monjalon.net>
> > Sent: 16 March 2023 23:34
> > To: Srikanth Yalavarthi <syalavarthi@marvell.com>
> > Cc: dev@dpdk.org; Shivah Shankar Shankar Narayan Rao
> > <sshankarnara@marvell.com>; Jerin Jacob Kollanukkaran
> > <jerinj@marvell.com>; Anup Prabhu <aprabhu@marvell.com>; Prince
> Takkar
> > <ptakkar@marvell.com>; Parijat Shukla <pshukla@marvell.com>; Srikanth
> > Yalavarthi <syalavarthi@marvell.com>
> > Subject: Re: [EXT] Re: [PATCH v6 12/12] app/mldev: add documentation
> > for mldev test cases
> >
> > 16/03/2023 18:56, Srikanth Yalavarthi:
> > > From: Thomas Monjalon <thomas@monjalon.net>
> > > > 11/03/2023 16:09, Srikanth Yalavarthi:
> > > > > Added documentation specific to mldev test cases. Added details
> > > > > about all test cases and option supported by individual tests.
> > > >
> > > > Would it be possible to add documentation of test cases while
> > > > adding the code?
> > > > Keeping related code and doc changes in the same commit may help
> > > > in future when looking at the git history.
> > > >
> > > Since, there was a lot of moving around multiple patches during the
> > > app
> > development. I have kept the entire documentation as a single patch
> > for ease of development.
> > >
> > > It is possible to incrementally add documentation to the previous 11
> > patches in the series. But, that would need rework on all patches.
> > >
> > > Pls, suggest?
> >
> > That's exactly what I did for your driver series.
> > First I've split the doc in multiple patches (in the same order as
> > code patches), then I've squashed the doc patches with the code patches.
> >
> Ack, will submit v7 with changes.
Done, changes part of v7.
> >
^ permalink raw reply [flat|nested] 122+ messages in thread
* [PATCH v7 00/11] Implementation of mldev test application
2022-11-29 7:07 [PATCH v1 00/12] implement mldev test application Srikanth Yalavarthi
` (14 preceding siblings ...)
2023-03-11 15:08 ` [PATCH v6 00/12] Implementation of mldev test application Srikanth Yalavarthi
@ 2023-03-16 21:14 ` Srikanth Yalavarthi
2023-03-16 21:14 ` [PATCH v7 01/11] app/mldev: implement test framework for mldev Srikanth Yalavarthi
` (11 more replies)
15 siblings, 12 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-16 21:14 UTC (permalink / raw)
Cc: dev, syalavarthi, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Machine learning device APIs test application
=============================================
This series of patches introduces a test application for machine
learning device APIs. A test framework is implemented with multiple
test enabled, to validate the device, model and fast-path functions.
New tests can be added using the test framework.
List of tests supported
-----------------------
1) device_ops: Test case to validate device re-configuration
2) model_ops: Collection of 4 sub-tests to validate model slow APIs.
Each sub-test would invoke the slow path model APIs (load / start /
stop / unload) in different order.
3) inference_ordered: Test case to validate execution of end-to-end
inferences on ML device with one active model at a time. This test
can execute inference requests for multiple models, with inferences
for a model executed after completion of inferences for a previously
loaded model.
4) inference_interleave: Test case to validate end-to-end inferences
with multiple active models concurrently. This case would work as a
stress test to validate ML device.
Options supported the by tests include burst size for enqueuing and
dequeuing inference requests, enabling multiple queue pairs with a
user specified value for queue size. Support is also enabled for
batch inferencing, output validation and statistics.
v7:
* Add documentation to individual commits
* Documentation fixes
* Update array initialization formatting
v6:
* Enable error count
* Fix segfault due to incorrect i/o size types
* Revert i/o size variables to uint64_t
v5:
* Cleanup of header includes
* Addressed issues with unclean patches
* Changed input/output size variables to uint32_t
* Rebase over main branch
v4:
* Updated model_id as uint16_t
* Updated license info in SVG files
* Updated release notes
v3:
* Code rebase
Srikanth Yalavarthi (11):
app/mldev: implement test framework for mldev
app/mldev: add common test functions
app/mldev: add test case to validate device ops
app/mldev: add test case to validate model ops
app/mldev: add ordered inference test case
app/mldev: add test case to interleave inferences
app/mldev: enable support for burst inferences
app/mldev: enable support for queue pairs and size
app/mldev: enable support for inference batches
app/mldev: enable support for inference validation
app/mldev: enable reporting stats in mldev app
.mailmap | 1 +
MAINTAINERS | 2 +
app/meson.build | 1 +
app/test-mldev/meson.build | 24 +
app/test-mldev/ml_common.h | 29 +
app/test-mldev/ml_main.c | 113 ++
app/test-mldev/ml_options.c | 340 +++++
app/test-mldev/ml_options.h | 57 +
app/test-mldev/ml_test.c | 41 +
app/test-mldev/ml_test.h | 76 ++
app/test-mldev/parser.c | 380 ++++++
app/test-mldev/parser.h | 55 +
app/test-mldev/test_common.c | 136 ++
app/test-mldev/test_common.h | 27 +
app/test-mldev/test_device_ops.c | 228 ++++
app/test-mldev/test_device_ops.h | 17 +
app/test-mldev/test_inference_common.c | 1128 +++++++++++++++++
app/test-mldev/test_inference_common.h | 75 ++
app/test-mldev/test_inference_interleave.c | 118 ++
app/test-mldev/test_inference_ordered.c | 116 ++
app/test-mldev/test_model_common.c | 164 +++
app/test-mldev/test_model_common.h | 46 +
app/test-mldev/test_model_ops.c | 428 +++++++
app/test-mldev/test_model_ops.h | 20 +
doc/guides/rel_notes/release_23_03.rst | 8 +
.../tools/img/mldev_inference_interleave.svg | 669 ++++++++++
.../tools/img/mldev_inference_ordered.svg | 528 ++++++++
.../tools/img/mldev_model_ops_subtest_a.svg | 420 ++++++
.../tools/img/mldev_model_ops_subtest_b.svg | 423 +++++++
.../tools/img/mldev_model_ops_subtest_c.svg | 366 ++++++
.../tools/img/mldev_model_ops_subtest_d.svg | 424 +++++++
doc/guides/tools/index.rst | 1 +
doc/guides/tools/testmldev.rst | 442 +++++++
33 files changed, 6903 insertions(+)
create mode 100644 app/test-mldev/meson.build
create mode 100644 app/test-mldev/ml_common.h
create mode 100644 app/test-mldev/ml_main.c
create mode 100644 app/test-mldev/ml_options.c
create mode 100644 app/test-mldev/ml_options.h
create mode 100644 app/test-mldev/ml_test.c
create mode 100644 app/test-mldev/ml_test.h
create mode 100644 app/test-mldev/parser.c
create mode 100644 app/test-mldev/parser.h
create mode 100644 app/test-mldev/test_common.c
create mode 100644 app/test-mldev/test_common.h
create mode 100644 app/test-mldev/test_device_ops.c
create mode 100644 app/test-mldev/test_device_ops.h
create mode 100644 app/test-mldev/test_inference_common.c
create mode 100644 app/test-mldev/test_inference_common.h
create mode 100644 app/test-mldev/test_inference_interleave.c
create mode 100644 app/test-mldev/test_inference_ordered.c
create mode 100644 app/test-mldev/test_model_common.c
create mode 100644 app/test-mldev/test_model_common.h
create mode 100644 app/test-mldev/test_model_ops.c
create mode 100644 app/test-mldev/test_model_ops.h
create mode 100644 doc/guides/tools/img/mldev_inference_interleave.svg
create mode 100644 doc/guides/tools/img/mldev_inference_ordered.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_a.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_b.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_c.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_d.svg
create mode 100644 doc/guides/tools/testmldev.rst
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v7 01/11] app/mldev: implement test framework for mldev
2023-03-16 21:14 ` [PATCH v7 00/11] Implementation of mldev test application Srikanth Yalavarthi
@ 2023-03-16 21:14 ` Srikanth Yalavarthi
2023-03-16 21:14 ` [PATCH v7 02/11] app/mldev: add common test functions Srikanth Yalavarthi
` (10 subsequent siblings)
11 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-16 21:14 UTC (permalink / raw)
To: Thomas Monjalon, Srikanth Yalavarthi
Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Implemented framework for mldev test application. New test cases
can be added using the framework. Support is also enabled to add
options specific to the test cases. User can launch the tests by
specifying the name of test as part of launch arguments.
Code to parse command line arguments is imported from
test-eventdev, with support to parse additional data types.
Common arguments supported include:
test : name of the test application to run
dev_id : device id of the ML device
socket_id : socket_id of application resources
debug : enable debugging
help : print help
Sample launch command:
./dpdk-test-mldev -- --test <testname> --dev_id <dev_id> \
--socket_id <socket_id>
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
Acked-by: Shivah Shankar S <sshankarnara@marvell.com>
Acked-by: Anup Prabhu <aprabhu@marvell.com>
---
.mailmap | 1 +
MAINTAINERS | 2 +
app/meson.build | 1 +
app/test-mldev/meson.build | 17 ++
app/test-mldev/ml_common.h | 29 ++
app/test-mldev/ml_main.c | 113 ++++++++
app/test-mldev/ml_options.c | 159 +++++++++++
app/test-mldev/ml_options.h | 31 ++
app/test-mldev/ml_test.c | 41 +++
app/test-mldev/ml_test.h | 76 +++++
app/test-mldev/parser.c | 380 +++++++++++++++++++++++++
app/test-mldev/parser.h | 55 ++++
doc/guides/rel_notes/release_23_03.rst | 8 +
doc/guides/tools/index.rst | 1 +
doc/guides/tools/testmldev.rst | 70 +++++
15 files changed, 984 insertions(+)
create mode 100644 app/test-mldev/meson.build
create mode 100644 app/test-mldev/ml_common.h
create mode 100644 app/test-mldev/ml_main.c
create mode 100644 app/test-mldev/ml_options.c
create mode 100644 app/test-mldev/ml_options.h
create mode 100644 app/test-mldev/ml_test.c
create mode 100644 app/test-mldev/ml_test.h
create mode 100644 app/test-mldev/parser.c
create mode 100644 app/test-mldev/parser.h
create mode 100644 doc/guides/tools/testmldev.rst
diff --git a/.mailmap b/.mailmap
index 6a56239c3a..0712888c03 100644
--- a/.mailmap
+++ b/.mailmap
@@ -112,6 +112,7 @@ Anoob Joseph <anoobj@marvell.com> <anoob.joseph@caviumnetworks.com>
Antara Ganesh Kolar <antara.ganesh.kolar@intel.com>
Anthony Fee <anthonyx.fee@intel.com>
Antonio Fischetti <antonio.fischetti@intel.com>
+Anup Prabhu <aprabhu@marvell.com>
Anupam Kapoor <anupam.kapoor@gmail.com>
Apeksha Gupta <apeksha.gupta@nxp.com>
Archana Muniganti <marchana@marvell.com> <muniganti.archana@caviumnetworks.com>
diff --git a/MAINTAINERS b/MAINTAINERS
index f70df9f297..df056e2b04 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -480,6 +480,8 @@ Machine Learning device API - EXPERIMENTAL
M: Srikanth Yalavarthi <syalavarthi@marvell.com>
F: lib/mldev/
F: doc/guides/prog_guide/mldev.rst
+F: app/test-mldev
+F: doc/guides/tools/testmldev.rst
DMA device API - EXPERIMENTAL
M: Chengwen Feng <fengchengwen@huawei.com>
diff --git a/app/meson.build b/app/meson.build
index e32ea4bd5c..74d2420f67 100644
--- a/app/meson.build
+++ b/app/meson.build
@@ -23,6 +23,7 @@ apps = [
'test-fib',
'test-flow-perf',
'test-gpudev',
+ 'test-mldev',
'test-pipeline',
'test-pmd',
'test-regex',
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
new file mode 100644
index 0000000000..8ca2e1a1c1
--- /dev/null
+++ b/app/test-mldev/meson.build
@@ -0,0 +1,17 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright (c) 2022 Marvell.
+
+if is_windows
+ build = false
+ reason = 'not supported on Windows'
+ subdir_done()
+endif
+
+sources = files(
+ 'ml_main.c',
+ 'ml_options.c',
+ 'ml_test.c',
+ 'parser.c',
+)
+
+deps += ['mldev']
diff --git a/app/test-mldev/ml_common.h b/app/test-mldev/ml_common.h
new file mode 100644
index 0000000000..065180b619
--- /dev/null
+++ b/app/test-mldev/ml_common.h
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_COMMON_
+#define _ML_COMMON_
+
+#include <stdio.h>
+
+#define CLNRM "\x1b[0m"
+#define CLRED "\x1b[31m"
+#define CLGRN "\x1b[32m"
+#define CLYEL "\x1b[33m"
+
+#define ML_STR_FMT 20
+
+#define ml_err(fmt, args...) fprintf(stderr, CLRED "error: %s() " fmt CLNRM "\n", __func__, ##args)
+
+#define ml_info(fmt, args...) fprintf(stdout, CLYEL "" fmt CLNRM "\n", ##args)
+
+#define ml_dump(str, fmt, val...) printf("\t%-*s : " fmt "\n", ML_STR_FMT, str, ##val)
+
+#define ml_dump_begin(str) printf("\t%-*s :\n\t{\n", ML_STR_FMT, str)
+
+#define ml_dump_list(str, id, val) printf("\t%*s[%2u] : %s\n", ML_STR_FMT - 4, str, id, val)
+
+#define ml_dump_end printf("\b\t}\n\n")
+
+#endif /* _ML_COMMON_*/
diff --git a/app/test-mldev/ml_main.c b/app/test-mldev/ml_main.c
new file mode 100644
index 0000000000..940e609c9a
--- /dev/null
+++ b/app/test-mldev/ml_main.c
@@ -0,0 +1,113 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <rte_debug.h>
+#include <rte_eal.h>
+#include <rte_mldev.h>
+
+#include "ml_common.h"
+#include "ml_test.h"
+
+struct ml_options opt;
+struct ml_test *test;
+
+int
+main(int argc, char **argv)
+{
+ uint16_t mldevs;
+ int ret;
+
+ ret = rte_eal_init(argc, argv);
+ if (ret < 0)
+ rte_panic("invalid EAL arguments\n");
+ argc -= ret;
+ argv += ret;
+
+ mldevs = rte_ml_dev_count();
+ if (!mldevs)
+ rte_panic("no mldev devices found\n");
+
+ /* set default values for options */
+ ml_options_default(&opt);
+
+ /* parse the command line arguments */
+ ret = ml_options_parse(&opt, argc, argv);
+ if (ret) {
+ ml_err("parsing one or more user options failed");
+ goto error;
+ }
+
+ /* get test struct from name */
+ test = ml_test_get(opt.test_name);
+ if (test == NULL) {
+ ml_err("failed to find requested test: %s", opt.test_name);
+ goto error;
+ }
+
+ if (test->ops.test_result == NULL) {
+ ml_err("%s: ops.test_result not found", opt.test_name);
+ goto error;
+ }
+
+ /* check test options */
+ if (test->ops.opt_check) {
+ if (test->ops.opt_check(&opt)) {
+ ml_err("invalid command line argument");
+ goto error;
+ }
+ }
+
+ /* check the device capability */
+ if (test->ops.cap_check) {
+ if (test->ops.cap_check(&opt) == false) {
+ ml_info("unsupported test: %s", opt.test_name);
+ ret = ML_TEST_UNSUPPORTED;
+ goto no_cap;
+ }
+ }
+
+ /* dump options */
+ if (opt.debug) {
+ if (test->ops.opt_dump)
+ test->ops.opt_dump(&opt);
+ }
+
+ /* test specific setup */
+ if (test->ops.test_setup) {
+ if (test->ops.test_setup(test, &opt)) {
+ ml_err("failed to setup test: %s", opt.test_name);
+ goto error;
+ }
+ }
+
+ /* test driver */
+ if (test->ops.test_driver)
+ test->ops.test_driver(test, &opt);
+
+ /* get result */
+ if (test->ops.test_result)
+ ret = test->ops.test_result(test, &opt);
+
+ if (test->ops.test_destroy)
+ test->ops.test_destroy(test, &opt);
+
+no_cap:
+ if (ret == ML_TEST_SUCCESS) {
+ printf("Result: " CLGRN "%s" CLNRM "\n", "Success");
+ } else if (ret == ML_TEST_FAILED) {
+ printf("Result: " CLRED "%s" CLNRM "\n", "Failed");
+ return EXIT_FAILURE;
+ } else if (ret == ML_TEST_UNSUPPORTED) {
+ printf("Result: " CLYEL "%s" CLNRM "\n", "Unsupported");
+ }
+
+ rte_eal_cleanup();
+
+ return 0;
+
+error:
+ rte_eal_cleanup();
+
+ return EXIT_FAILURE;
+}
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
new file mode 100644
index 0000000000..37559614b1
--- /dev/null
+++ b/app/test-mldev/ml_options.c
@@ -0,0 +1,159 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <errno.h>
+#include <getopt.h>
+
+#include <rte_memory.h>
+#include <rte_mldev.h>
+#include <rte_string_fns.h>
+
+#include "ml_common.h"
+#include "ml_test.h"
+#include "parser.h"
+
+typedef int (*option_parser_t)(struct ml_options *opt, const char *arg);
+
+void
+ml_options_default(struct ml_options *opt)
+{
+ memset(opt, 0, sizeof(*opt));
+ strlcpy(opt->test_name, "ml_test", ML_TEST_NAME_MAX_LEN);
+ opt->dev_id = 0;
+ opt->socket_id = SOCKET_ID_ANY;
+ opt->debug = false;
+}
+
+struct long_opt_parser {
+ const char *lgopt_name;
+ option_parser_t parser_fn;
+};
+
+static int
+ml_parse_test_name(struct ml_options *opt, const char *arg)
+{
+ strlcpy(opt->test_name, arg, ML_TEST_NAME_MAX_LEN);
+ return 0;
+}
+
+static int
+ml_parse_dev_id(struct ml_options *opt, const char *arg)
+{
+ int ret;
+
+ ret = parser_read_int16(&opt->dev_id, arg);
+
+ if (ret < 0)
+ return -EINVAL;
+
+ return ret;
+}
+
+static int
+ml_parse_socket_id(struct ml_options *opt, const char *arg)
+{
+ opt->socket_id = atoi(arg);
+
+ return 0;
+}
+
+static void
+ml_dump_test_options(const char *testname)
+{
+ RTE_SET_USED(testname);
+}
+
+static void
+print_usage(char *program)
+{
+ printf("\nusage : %s [EAL options] -- [application options]\n", program);
+ printf("application options:\n");
+ printf("\t--test : name of the test application to run\n"
+ "\t--dev_id : device id of the ML device\n"
+ "\t--socket_id : socket_id of application resources\n"
+ "\t--debug : enable debug mode\n"
+ "\t--help : print help\n");
+ printf("\n");
+ printf("available tests and test specific application options:\n");
+ ml_test_dump_names(ml_dump_test_options);
+}
+
+static struct option lgopts[] = {
+ {ML_TEST, 1, 0, 0},
+ {ML_DEVICE_ID, 1, 0, 0},
+ {ML_SOCKET_ID, 1, 0, 0},
+ {ML_DEBUG, 0, 0, 0},
+ {ML_HELP, 0, 0, 0},
+ {NULL, 0, 0, 0}};
+
+static int
+ml_opts_parse_long(int opt_idx, struct ml_options *opt)
+{
+ unsigned int i;
+
+ struct long_opt_parser parsermap[] = {
+ {ML_TEST, ml_parse_test_name},
+ {ML_DEVICE_ID, ml_parse_dev_id},
+ {ML_SOCKET_ID, ml_parse_socket_id},
+ };
+
+ for (i = 0; i < RTE_DIM(parsermap); i++) {
+ if (strncmp(lgopts[opt_idx].name, parsermap[i].lgopt_name,
+ strlen(lgopts[opt_idx].name)) == 0)
+ return parsermap[i].parser_fn(opt, optarg);
+ }
+
+ return -EINVAL;
+}
+
+int
+ml_options_parse(struct ml_options *opt, int argc, char **argv)
+{
+ int opt_idx;
+ int retval;
+ int opts;
+
+ while ((opts = getopt_long(argc, argv, "", lgopts, &opt_idx)) != EOF) {
+ switch (opts) {
+ case 0: /* parse long options */
+ if (!strcmp(lgopts[opt_idx].name, "debug")) {
+ opt->debug = true;
+ break;
+ }
+
+ if (!strcmp(lgopts[opt_idx].name, "help")) {
+ print_usage(argv[0]);
+ exit(EXIT_SUCCESS);
+ }
+
+ retval = ml_opts_parse_long(opt_idx, opt);
+ if (retval != 0)
+ return retval;
+ break;
+ default:
+ return -EINVAL;
+ }
+ }
+
+ return 0;
+}
+
+void
+ml_options_dump(struct ml_options *opt)
+{
+ struct rte_ml_dev_info dev_info;
+
+ rte_ml_dev_info_get(opt->dev_id, &dev_info);
+
+ ml_dump("driver", "%s", dev_info.driver_name);
+ ml_dump("test", "%s", opt->test_name);
+ ml_dump("dev_id", "%d", opt->dev_id);
+
+ if (opt->socket_id == SOCKET_ID_ANY)
+ ml_dump("socket_id", "%d (SOCKET_ID_ANY)", opt->socket_id);
+ else
+ ml_dump("socket_id", "%d", opt->socket_id);
+
+ ml_dump("debug", "%s", (opt->debug ? "true" : "false"));
+}
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
new file mode 100644
index 0000000000..05311a9a47
--- /dev/null
+++ b/app/test-mldev/ml_options.h
@@ -0,0 +1,31 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_OPTIONS_
+#define _ML_OPTIONS_
+
+#include <stdbool.h>
+#include <stdint.h>
+
+#define ML_TEST_NAME_MAX_LEN 32
+
+/* Options names */
+#define ML_TEST ("test")
+#define ML_DEVICE_ID ("dev_id")
+#define ML_SOCKET_ID ("socket_id")
+#define ML_DEBUG ("debug")
+#define ML_HELP ("help")
+
+struct ml_options {
+ char test_name[ML_TEST_NAME_MAX_LEN];
+ int16_t dev_id;
+ int socket_id;
+ bool debug;
+};
+
+void ml_options_default(struct ml_options *opt);
+int ml_options_parse(struct ml_options *opt, int argc, char **argv);
+void ml_options_dump(struct ml_options *opt);
+
+#endif /* _ML_OPTIONS_ */
diff --git a/app/test-mldev/ml_test.c b/app/test-mldev/ml_test.c
new file mode 100644
index 0000000000..dcdeabe0bd
--- /dev/null
+++ b/app/test-mldev/ml_test.c
@@ -0,0 +1,41 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include "ml_test.h"
+
+static STAILQ_HEAD(, ml_test_entry) head = STAILQ_HEAD_INITIALIZER(head);
+
+void
+ml_test_register(struct ml_test_entry *entry)
+{
+ STAILQ_INSERT_TAIL(&head, entry, next);
+}
+
+struct ml_test *
+ml_test_get(const char *name)
+{
+ struct ml_test_entry *entry;
+
+ if (!name)
+ return NULL;
+
+ STAILQ_FOREACH(entry, &head, next)
+ if (!strncmp(entry->test.name, name, strlen(name)))
+ return &entry->test;
+
+ return NULL;
+}
+
+void
+ml_test_dump_names(void (*f)(const char *name))
+{
+ struct ml_test_entry *entry;
+
+ STAILQ_FOREACH(entry, &head, next)
+ {
+ if (entry->test.name)
+ printf("\t %s\n", entry->test.name);
+ f(entry->test.name);
+ }
+}
diff --git a/app/test-mldev/ml_test.h b/app/test-mldev/ml_test.h
new file mode 100644
index 0000000000..f7e6bf2f44
--- /dev/null
+++ b/app/test-mldev/ml_test.h
@@ -0,0 +1,76 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_TEST_
+#define _ML_TEST_
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/queue.h>
+
+#include <rte_common.h>
+
+#include "ml_options.h"
+
+#define ML_TEST_MAX_POOL_SIZE 256
+
+enum ml_test_result {
+ ML_TEST_SUCCESS,
+ ML_TEST_FAILED,
+ ML_TEST_UNSUPPORTED,
+};
+
+struct ml_test;
+
+typedef bool (*ml_test_capability_check_t)(struct ml_options *opt);
+typedef int (*ml_test_options_check_t)(struct ml_options *opt);
+typedef void (*ml_test_options_dump_t)(struct ml_options *opt);
+typedef int (*ml_test_setup_t)(struct ml_test *test, struct ml_options *opt);
+typedef void (*ml_test_destroy_t)(struct ml_test *test, struct ml_options *opt);
+typedef int (*ml_test_driver_t)(struct ml_test *test, struct ml_options *opt);
+typedef int (*ml_test_result_t)(struct ml_test *test, struct ml_options *opt);
+
+struct ml_test_ops {
+ ml_test_capability_check_t cap_check;
+ ml_test_options_check_t opt_check;
+ ml_test_options_dump_t opt_dump;
+ ml_test_setup_t test_setup;
+ ml_test_destroy_t test_destroy;
+ ml_test_driver_t test_driver;
+ ml_test_result_t test_result;
+};
+
+struct ml_test {
+ const char *name;
+ void *test_priv;
+ struct ml_test_ops ops;
+};
+
+struct ml_test_entry {
+ struct ml_test test;
+
+ STAILQ_ENTRY(ml_test_entry) next;
+};
+
+static inline void *
+ml_test_priv(struct ml_test *test)
+{
+ return test->test_priv;
+}
+
+struct ml_test *ml_test_get(const char *name);
+void ml_test_register(struct ml_test_entry *test);
+void ml_test_dump_names(void (*f)(const char *));
+
+#define ML_TEST_REGISTER(nm) \
+ static struct ml_test_entry _ml_test_entry_##nm; \
+ RTE_INIT(ml_test_##nm) \
+ { \
+ _ml_test_entry_##nm.test.name = RTE_STR(nm); \
+ memcpy(&_ml_test_entry_##nm.test.ops, &nm, sizeof(struct ml_test_ops)); \
+ ml_test_register(&_ml_test_entry_##nm); \
+ }
+
+#endif /* _ML_TEST_ */
diff --git a/app/test-mldev/parser.c b/app/test-mldev/parser.c
new file mode 100644
index 0000000000..0b7fb63fe5
--- /dev/null
+++ b/app/test-mldev/parser.c
@@ -0,0 +1,380 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2016 Intel Corporation.
+ * Copyright (c) 2017 Cavium, Inc.
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <ctype.h>
+#include <errno.h>
+#include <stdbool.h>
+#include <stdint.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include <rte_common.h>
+
+#include "parser.h"
+
+static uint32_t
+get_hex_val(char c)
+{
+ switch (c) {
+ case '0':
+ case '1':
+ case '2':
+ case '3':
+ case '4':
+ case '5':
+ case '6':
+ case '7':
+ case '8':
+ case '9':
+ return c - '0';
+ case 'A':
+ case 'B':
+ case 'C':
+ case 'D':
+ case 'E':
+ case 'F':
+ return c - 'A' + 10;
+ case 'a':
+ case 'b':
+ case 'c':
+ case 'd':
+ case 'e':
+ case 'f':
+ return c - 'a' + 10;
+ default:
+ return 0;
+ }
+}
+
+int
+parser_read_arg_bool(const char *p)
+{
+ p = skip_white_spaces(p);
+ int result = -EINVAL;
+
+ if (((p[0] == 'y') && (p[1] == 'e') && (p[2] == 's')) ||
+ ((p[0] == 'Y') && (p[1] == 'E') && (p[2] == 'S'))) {
+ p += 3;
+ result = 1;
+ }
+
+ if (((p[0] == 'o') && (p[1] == 'n')) || ((p[0] == 'O') && (p[1] == 'N'))) {
+ p += 2;
+ result = 1;
+ }
+
+ if (((p[0] == 'n') && (p[1] == 'o')) || ((p[0] == 'N') && (p[1] == 'O'))) {
+ p += 2;
+ result = 0;
+ }
+
+ if (((p[0] == 'o') && (p[1] == 'f') && (p[2] == 'f')) ||
+ ((p[0] == 'O') && (p[1] == 'F') && (p[2] == 'F'))) {
+ p += 3;
+ result = 0;
+ }
+
+ p = skip_white_spaces(p);
+
+ if (p[0] != '\0')
+ return -EINVAL;
+
+ return result;
+}
+
+int
+parser_read_uint64(uint64_t *value, const char *p)
+{
+ char *next;
+ uint64_t val;
+
+ p = skip_white_spaces(p);
+ if (!isdigit(*p))
+ return -EINVAL;
+
+ val = strtoul(p, &next, 10);
+ if (p == next)
+ return -EINVAL;
+
+ p = next;
+ switch (*p) {
+ case 'T':
+ val *= 1024ULL;
+ /* fall through */
+ case 'G':
+ val *= 1024ULL;
+ /* fall through */
+ case 'M':
+ val *= 1024ULL;
+ /* fall through */
+ case 'k':
+ case 'K':
+ val *= 1024ULL;
+ p++;
+ break;
+ }
+
+ p = skip_white_spaces(p);
+ if (*p != '\0')
+ return -EINVAL;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_int32(int32_t *value, const char *p)
+{
+ char *next;
+ int32_t val;
+
+ p = skip_white_spaces(p);
+ if (!isdigit(*p))
+ return -EINVAL;
+
+ val = strtol(p, &next, 10);
+ if (p == next)
+ return -EINVAL;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_int16(int16_t *value, const char *p)
+{
+ char *next;
+ int16_t val;
+
+ p = skip_white_spaces(p);
+ if (!isdigit(*p))
+ return -EINVAL;
+
+ val = strtol(p, &next, 10);
+ if (p == next)
+ return -EINVAL;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint64_hex(uint64_t *value, const char *p)
+{
+ char *next;
+ uint64_t val;
+
+ p = skip_white_spaces(p);
+
+ val = strtoul(p, &next, 16);
+ if (p == next)
+ return -EINVAL;
+
+ p = skip_white_spaces(next);
+ if (*p != '\0')
+ return -EINVAL;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint32(uint32_t *value, const char *p)
+{
+ uint64_t val = 0;
+ int ret = parser_read_uint64(&val, p);
+
+ if (ret < 0)
+ return ret;
+
+ if (val > UINT32_MAX)
+ return -ERANGE;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint32_hex(uint32_t *value, const char *p)
+{
+ uint64_t val = 0;
+ int ret = parser_read_uint64_hex(&val, p);
+
+ if (ret < 0)
+ return ret;
+
+ if (val > UINT32_MAX)
+ return -ERANGE;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint16(uint16_t *value, const char *p)
+{
+ uint64_t val = 0;
+ int ret = parser_read_uint64(&val, p);
+
+ if (ret < 0)
+ return ret;
+
+ if (val > UINT16_MAX)
+ return -ERANGE;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint16_hex(uint16_t *value, const char *p)
+{
+ uint64_t val = 0;
+ int ret = parser_read_uint64_hex(&val, p);
+
+ if (ret < 0)
+ return ret;
+
+ if (val > UINT16_MAX)
+ return -ERANGE;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint8(uint8_t *value, const char *p)
+{
+ uint64_t val = 0;
+ int ret = parser_read_uint64(&val, p);
+
+ if (ret < 0)
+ return ret;
+
+ if (val > UINT8_MAX)
+ return -ERANGE;
+
+ *value = val;
+ return 0;
+}
+
+int
+parser_read_uint8_hex(uint8_t *value, const char *p)
+{
+ uint64_t val = 0;
+ int ret = parser_read_uint64_hex(&val, p);
+
+ if (ret < 0)
+ return ret;
+
+ if (val > UINT8_MAX)
+ return -ERANGE;
+
+ *value = val;
+ return 0;
+}
+
+int
+parse_tokenize_string(char *string, char *tokens[], uint32_t *n_tokens)
+{
+ uint32_t i;
+
+ if ((string == NULL) || (tokens == NULL) || (*n_tokens < 1))
+ return -EINVAL;
+
+ for (i = 0; i < *n_tokens; i++) {
+ tokens[i] = strtok_r(string, PARSE_DELIMITER, &string);
+ if (tokens[i] == NULL)
+ break;
+ }
+
+ if ((i == *n_tokens) && (strtok_r(string, PARSE_DELIMITER, &string) != NULL))
+ return -E2BIG;
+
+ *n_tokens = i;
+ return 0;
+}
+
+int
+parse_hex_string(char *src, uint8_t *dst, uint32_t *size)
+{
+ char *c;
+ uint32_t len, i;
+
+ /* Check input parameters */
+ if ((src == NULL) || (dst == NULL) || (size == NULL) || (*size == 0))
+ return -1;
+
+ len = strlen(src);
+ if (((len & 3) != 0) || (len > (*size) * 2))
+ return -1;
+ *size = len / 2;
+
+ for (c = src; *c != 0; c++) {
+ if ((((*c) >= '0') && ((*c) <= '9')) || (((*c) >= 'A') && ((*c) <= 'F')) ||
+ (((*c) >= 'a') && ((*c) <= 'f')))
+ continue;
+
+ return -1;
+ }
+
+ /* Convert chars to bytes */
+ for (i = 0; i < *size; i++)
+ dst[i] = get_hex_val(src[2 * i]) * 16 + get_hex_val(src[2 * i + 1]);
+
+ return 0;
+}
+
+int
+parse_lcores_list(bool lcores[], int lcores_num, const char *corelist)
+{
+ int i, idx = 0;
+ int min, max;
+ char *end = NULL;
+
+ if (corelist == NULL)
+ return -1;
+ while (isblank(*corelist))
+ corelist++;
+ i = strlen(corelist);
+ while ((i > 0) && isblank(corelist[i - 1]))
+ i--;
+
+ /* Get list of lcores */
+ min = RTE_MAX_LCORE;
+ do {
+ while (isblank(*corelist))
+ corelist++;
+ if (*corelist == '\0')
+ return -1;
+ idx = strtoul(corelist, &end, 10);
+ if (idx < 0 || idx > lcores_num)
+ return -1;
+
+ if (end == NULL)
+ return -1;
+ while (isblank(*end))
+ end++;
+ if (*end == '-') {
+ min = idx;
+ } else if ((*end == ',') || (*end == '\0')) {
+ max = idx;
+ if (min == RTE_MAX_LCORE)
+ min = idx;
+ for (idx = min; idx <= max; idx++) {
+ if (lcores[idx] == 1)
+ return -E2BIG;
+ lcores[idx] = 1;
+ }
+
+ min = RTE_MAX_LCORE;
+ } else
+ return -1;
+ corelist = end + 1;
+ } while (*end != '\0');
+
+ return 0;
+}
diff --git a/app/test-mldev/parser.h b/app/test-mldev/parser.h
new file mode 100644
index 0000000000..f0d5e79e4b
--- /dev/null
+++ b/app/test-mldev/parser.h
@@ -0,0 +1,55 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2010-2016 Intel Corporation.
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef __INCLUDE_PARSER_H__
+#define __INCLUDE_PARSER_H__
+
+#include <ctype.h>
+#include <stdint.h>
+#include <stdlib.h>
+
+#define PARSE_DELIMITER " \f\n\r\t\v"
+
+#define skip_white_spaces(pos) \
+ ({ \
+ __typeof__(pos) _p = (pos); \
+ for (; isspace(*_p); _p++) \
+ ; \
+ _p; \
+ })
+
+static inline size_t
+skip_digits(const char *src)
+{
+ size_t i;
+
+ for (i = 0; isdigit(src[i]); i++)
+ ;
+
+ return i;
+}
+
+int parser_read_arg_bool(const char *p);
+
+int parser_read_uint64(uint64_t *value, const char *p);
+int parser_read_uint32(uint32_t *value, const char *p);
+int parser_read_uint16(uint16_t *value, const char *p);
+int parser_read_uint8(uint8_t *value, const char *p);
+
+int parser_read_uint64_hex(uint64_t *value, const char *p);
+int parser_read_uint32_hex(uint32_t *value, const char *p);
+int parser_read_uint16_hex(uint16_t *value, const char *p);
+int parser_read_uint8_hex(uint8_t *value, const char *p);
+
+int parser_read_int32(int32_t *value, const char *p);
+int parser_read_int16(int16_t *value, const char *p);
+
+int parse_hex_string(char *src, uint8_t *dst, uint32_t *size);
+
+int parse_tokenize_string(char *string, char *tokens[], uint32_t *n_tokens);
+
+int parse_lcores_list(bool lcores[], int lcores_num, const char *corelist);
+
+#endif /* __INCLUDE_PARSER_H__ */
diff --git a/doc/guides/rel_notes/release_23_03.rst b/doc/guides/rel_notes/release_23_03.rst
index af6f37389c..072f2c9482 100644
--- a/doc/guides/rel_notes/release_23_03.rst
+++ b/doc/guides/rel_notes/release_23_03.rst
@@ -226,6 +226,14 @@ New Features
* Added support to capture packets at each graph node with packet metadata and
node name.
+* **Added test application for machine learning inference device library.**
+
+ * Added test application for mldev library with support for multiple test cases.
+ * Test case for device operations.
+ * Test case for model operations.
+ * Inference test case using multiple models in ordered mode.
+ * Inference test case using multiple models in interleave mode.
+
Removed Items
-------------
diff --git a/doc/guides/tools/index.rst b/doc/guides/tools/index.rst
index f1f5b94c8c..6f84fc31ff 100644
--- a/doc/guides/tools/index.rst
+++ b/doc/guides/tools/index.rst
@@ -21,4 +21,5 @@ DPDK Tools User Guides
comp_perf
testeventdev
testregex
+ testmldev
dts
diff --git a/doc/guides/tools/testmldev.rst b/doc/guides/tools/testmldev.rst
new file mode 100644
index 0000000000..9dd7606d0b
--- /dev/null
+++ b/doc/guides/tools/testmldev.rst
@@ -0,0 +1,70 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright (c) 2022 Marvell.
+
+dpdk-test-mldev Application
+===========================
+
+The ``dpdk-test-mldev`` tool is a Data Plane Development Kit (DPDK) application that allows testing
+various mldev use cases. This application has a generic framework to add new mldev based test cases
+to verify functionality and measure the performance of inference execution on DPDK ML devices.
+
+
+Application and Options
+-----------------------
+
+The application has a number of command line options:
+
+.. code-block:: console
+
+ dpdk-test-mldev [EAL Options] -- [application options]
+
+EAL Options
+~~~~~~~~~~~
+
+The following are the EAL command-line options that can be used with the ``dpdk-test-mldev``
+application. See the DPDK Getting Started Guides for more information on these options.
+
+* ``-c <COREMASK>`` or ``-l <CORELIST>``
+
+ Set the hexadecimal bitmask of the cores to run on. The corelist is a list of cores to use.
+
+* ``-a <PCI_ID>``
+
+ Attach a PCI based ML device. Specific to drivers using a PCI based ML devices.
+
+* ``--vdev <driver>``
+
+ Add a virtual mldev device. Specific to drivers using a ML virtual device.
+
+
+Application Options
+~~~~~~~~~~~~~~~~~~~
+
+The following are the command-line options supported by the test application.
+
+* ``--test <name>``
+
+ Name of the test to execute.
+
+* ``--dev_id <n>``
+
+ Set the device id of the ML device to be used for the test. Default value is `0`.
+
+* ``--socket_id <n>``
+
+ Set the socket id of the application resources. Default value is `SOCKET_ID_ANY`.
+
+* ``--debug``
+
+ Enable the tests to run in debug mode.
+
+* ``--help``
+
+ Print help message.
+
+
+Debug mode
+----------
+
+ML tests can be executed in debug mode by enabling the option ``--debug``. Execution of tests in
+debug mode would enable additional prints.
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v7 02/11] app/mldev: add common test functions
2023-03-16 21:14 ` [PATCH v7 00/11] Implementation of mldev test application Srikanth Yalavarthi
2023-03-16 21:14 ` [PATCH v7 01/11] app/mldev: implement test framework for mldev Srikanth Yalavarthi
@ 2023-03-16 21:14 ` Srikanth Yalavarthi
2023-03-16 21:14 ` [PATCH v7 03/11] app/mldev: add test case to validate device ops Srikanth Yalavarthi
` (9 subsequent siblings)
11 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-16 21:14 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Added common functions used by all tests. Common code
includes functions to check capabilities, options, and
handle ML devices.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
Acked-by: Anup Prabhu <aprabhu@marvell.com>
---
app/test-mldev/meson.build | 1 +
app/test-mldev/test_common.c | 136 +++++++++++++++++++++++++++++++++++
app/test-mldev/test_common.h | 27 +++++++
3 files changed, 164 insertions(+)
create mode 100644 app/test-mldev/test_common.c
create mode 100644 app/test-mldev/test_common.h
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
index 8ca2e1a1c1..964bb9ddc4 100644
--- a/app/test-mldev/meson.build
+++ b/app/test-mldev/meson.build
@@ -12,6 +12,7 @@ sources = files(
'ml_options.c',
'ml_test.c',
'parser.c',
+ 'test_common.c',
)
deps += ['mldev']
diff --git a/app/test-mldev/test_common.c b/app/test-mldev/test_common.c
new file mode 100644
index 0000000000..8c4da4609a
--- /dev/null
+++ b/app/test-mldev/test_common.c
@@ -0,0 +1,136 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <errno.h>
+
+#include <rte_common.h>
+#include <rte_memory.h>
+#include <rte_mldev.h>
+
+#include "ml_common.h"
+#include "test_common.h"
+
+bool
+ml_test_cap_check(struct ml_options *opt)
+{
+ struct rte_ml_dev_info dev_info;
+
+ rte_ml_dev_info_get(opt->dev_id, &dev_info);
+ if (dev_info.max_models == 0) {
+ ml_err("Not enough mldev models supported = %u", dev_info.max_models);
+ return false;
+ }
+
+ return true;
+}
+
+int
+ml_test_opt_check(struct ml_options *opt)
+{
+ uint16_t dev_count;
+ int socket_id;
+
+ RTE_SET_USED(opt);
+
+ dev_count = rte_ml_dev_count();
+ if (dev_count == 0) {
+ ml_err("No ML devices found");
+ return -ENODEV;
+ }
+
+ if (opt->dev_id >= dev_count) {
+ ml_err("Invalid option dev_id = %d", opt->dev_id);
+ return -EINVAL;
+ }
+
+ socket_id = rte_ml_dev_socket_id(opt->dev_id);
+ if (!((opt->socket_id != SOCKET_ID_ANY) || (opt->socket_id != socket_id))) {
+ ml_err("Invalid option, socket_id = %d\n", opt->socket_id);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+void
+ml_test_opt_dump(struct ml_options *opt)
+{
+ ml_options_dump(opt);
+}
+
+int
+ml_test_device_configure(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_common *t = ml_test_priv(test);
+ struct rte_ml_dev_config dev_config;
+ int ret;
+
+ ret = rte_ml_dev_info_get(opt->dev_id, &t->dev_info);
+ if (ret != 0) {
+ ml_err("Failed to get mldev info, dev_id = %d\n", opt->dev_id);
+ return ret;
+ }
+
+ /* configure device */
+ dev_config.socket_id = opt->socket_id;
+ dev_config.nb_models = t->dev_info.max_models;
+ dev_config.nb_queue_pairs = t->dev_info.max_queue_pairs;
+ ret = rte_ml_dev_configure(opt->dev_id, &dev_config);
+ if (ret != 0) {
+ ml_err("Failed to configure ml device, dev_id = %d\n", opt->dev_id);
+ return ret;
+ }
+
+ return 0;
+}
+
+int
+ml_test_device_close(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_common *t = ml_test_priv(test);
+ int ret = 0;
+
+ RTE_SET_USED(t);
+
+ /* close device */
+ ret = rte_ml_dev_close(opt->dev_id);
+ if (ret != 0)
+ ml_err("Failed to close ML device, dev_id = %d\n", opt->dev_id);
+
+ return ret;
+}
+
+int
+ml_test_device_start(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_common *t = ml_test_priv(test);
+ int ret;
+
+ RTE_SET_USED(t);
+
+ /* start device */
+ ret = rte_ml_dev_start(opt->dev_id);
+ if (ret != 0) {
+ ml_err("Failed to start ml device, dev_id = %d\n", opt->dev_id);
+ return ret;
+ }
+
+ return 0;
+}
+
+int
+ml_test_device_stop(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_common *t = ml_test_priv(test);
+ int ret = 0;
+
+ RTE_SET_USED(t);
+
+ /* stop device */
+ ret = rte_ml_dev_stop(opt->dev_id);
+ if (ret != 0)
+ ml_err("Failed to stop ML device, dev_id = %d\n", opt->dev_id);
+
+ return ret;
+}
diff --git a/app/test-mldev/test_common.h b/app/test-mldev/test_common.h
new file mode 100644
index 0000000000..cb286adcd6
--- /dev/null
+++ b/app/test-mldev/test_common.h
@@ -0,0 +1,27 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_TEST_COMMON_
+#define _ML_TEST_COMMON_
+
+#include <rte_mldev.h>
+
+#include "ml_common.h"
+#include "ml_test.h"
+
+struct test_common {
+ struct ml_options *opt;
+ enum ml_test_result result;
+ struct rte_ml_dev_info dev_info;
+};
+
+bool ml_test_cap_check(struct ml_options *opt);
+int ml_test_opt_check(struct ml_options *opt);
+void ml_test_opt_dump(struct ml_options *opt);
+int ml_test_device_configure(struct ml_test *test, struct ml_options *opt);
+int ml_test_device_close(struct ml_test *test, struct ml_options *opt);
+int ml_test_device_start(struct ml_test *test, struct ml_options *opt);
+int ml_test_device_stop(struct ml_test *test, struct ml_options *opt);
+
+#endif /* _ML_TEST_COMMON_ */
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v7 03/11] app/mldev: add test case to validate device ops
2023-03-16 21:14 ` [PATCH v7 00/11] Implementation of mldev test application Srikanth Yalavarthi
2023-03-16 21:14 ` [PATCH v7 01/11] app/mldev: implement test framework for mldev Srikanth Yalavarthi
2023-03-16 21:14 ` [PATCH v7 02/11] app/mldev: add common test functions Srikanth Yalavarthi
@ 2023-03-16 21:14 ` Srikanth Yalavarthi
2023-03-16 21:14 ` [PATCH v7 04/11] app/mldev: add test case to validate model ops Srikanth Yalavarthi
` (8 subsequent siblings)
11 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-16 21:14 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Added test case to validate device handling operations. Device ops
test is a collection of multiple sub-tests. Enabled sub-test to
validate device reconfiguration. Set device_ops as the default test.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
Acked-by: Anup Prabhu <aprabhu@marvell.com>
---
app/test-mldev/meson.build | 1 +
app/test-mldev/ml_options.c | 5 +-
app/test-mldev/ml_options.h | 2 +
app/test-mldev/test_device_ops.c | 228 +++++++++++++++++++++++++++++++
app/test-mldev/test_device_ops.h | 17 +++
doc/guides/tools/testmldev.rst | 42 +++++-
6 files changed, 292 insertions(+), 3 deletions(-)
create mode 100644 app/test-mldev/test_device_ops.c
create mode 100644 app/test-mldev/test_device_ops.h
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
index 964bb9ddc4..60ea23d142 100644
--- a/app/test-mldev/meson.build
+++ b/app/test-mldev/meson.build
@@ -13,6 +13,7 @@ sources = files(
'ml_test.c',
'parser.c',
'test_common.c',
+ 'test_device_ops.c',
)
deps += ['mldev']
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 37559614b1..854d316521 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -19,7 +19,7 @@ void
ml_options_default(struct ml_options *opt)
{
memset(opt, 0, sizeof(*opt));
- strlcpy(opt->test_name, "ml_test", ML_TEST_NAME_MAX_LEN);
+ strlcpy(opt->test_name, "device_ops", ML_TEST_NAME_MAX_LEN);
opt->dev_id = 0;
opt->socket_id = SOCKET_ID_ANY;
opt->debug = false;
@@ -61,7 +61,8 @@ ml_parse_socket_id(struct ml_options *opt, const char *arg)
static void
ml_dump_test_options(const char *testname)
{
- RTE_SET_USED(testname);
+ if (strcmp(testname, "device_ops") == 0)
+ printf("\n");
}
static void
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index 05311a9a47..6899cc2f88 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -9,6 +9,7 @@
#include <stdint.h>
#define ML_TEST_NAME_MAX_LEN 32
+#define ML_TEST_MAX_MODELS 8
/* Options names */
#define ML_TEST ("test")
@@ -21,6 +22,7 @@ struct ml_options {
char test_name[ML_TEST_NAME_MAX_LEN];
int16_t dev_id;
int socket_id;
+ uint8_t nb_filelist;
bool debug;
};
diff --git a/app/test-mldev/test_device_ops.c b/app/test-mldev/test_device_ops.c
new file mode 100644
index 0000000000..be0c5f0780
--- /dev/null
+++ b/app/test-mldev/test_device_ops.c
@@ -0,0 +1,228 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <errno.h>
+
+#include <rte_common.h>
+#include <rte_malloc.h>
+#include <rte_mldev.h>
+
+#include "test_device_ops.h"
+
+static bool
+test_device_cap_check(struct ml_options *opt)
+{
+ if (!ml_test_cap_check(opt))
+ return false;
+
+ return true;
+}
+
+static int
+test_device_opt_check(struct ml_options *opt)
+{
+ int ret;
+
+ /* check common opts */
+ ret = ml_test_opt_check(opt);
+ if (ret != 0)
+ return ret;
+
+ return 0;
+}
+
+static void
+test_device_opt_dump(struct ml_options *opt)
+{
+ /* dump common opts */
+ ml_test_opt_dump(opt);
+}
+
+static int
+test_device_setup(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_device *t;
+ void *test_device;
+ int ret = 0;
+
+ /* allocate for test structure */
+ test_device = rte_zmalloc_socket(test->name, sizeof(struct test_device),
+ RTE_CACHE_LINE_SIZE, opt->socket_id);
+ if (test_device == NULL) {
+ ml_err("failed to allocate memory for test_model");
+ ret = -ENOMEM;
+ goto error;
+ }
+ test->test_priv = test_device;
+ t = ml_test_priv(test);
+
+ t->cmn.result = ML_TEST_FAILED;
+ t->cmn.opt = opt;
+
+ /* get device info */
+ ret = rte_ml_dev_info_get(opt->dev_id, &t->cmn.dev_info);
+ if (ret < 0) {
+ ml_err("failed to get device info");
+ goto error;
+ }
+
+ return 0;
+
+error:
+ if (test_device != NULL)
+ rte_free(test_device);
+
+ return ret;
+}
+
+static void
+test_device_destroy(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_device *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+ if (t != NULL)
+ rte_free(t);
+}
+
+static int
+test_device_reconfigure(struct ml_test *test, struct ml_options *opt)
+{
+ struct rte_ml_dev_config dev_config;
+ struct rte_ml_dev_qp_conf qp_conf;
+ struct test_device *t;
+ uint16_t qp_id = 0;
+ int ret = 0;
+
+ t = ml_test_priv(test);
+
+ /* configure with default options */
+ ret = ml_test_device_configure(test, opt);
+ if (ret != 0)
+ return ret;
+
+ /* setup one queue pair with nb_desc = 1 */
+ qp_conf.nb_desc = 1;
+ qp_conf.cb = NULL;
+
+ ret = rte_ml_dev_queue_pair_setup(opt->dev_id, qp_id, &qp_conf, opt->socket_id);
+ if (ret != 0) {
+ ml_err("Failed to setup ML device queue-pair, dev_id = %d, qp_id = %u\n",
+ opt->dev_id, qp_id);
+ goto error;
+ }
+
+ /* start device */
+ ret = ml_test_device_start(test, opt);
+ if (ret != 0)
+ goto error;
+
+ /* stop device */
+ ret = ml_test_device_stop(test, opt);
+ if (ret != 0) {
+ ml_err("Failed to stop device");
+ goto error;
+ }
+
+ /* reconfigure device based on dev_info */
+ dev_config.socket_id = opt->socket_id;
+ dev_config.nb_models = t->cmn.dev_info.max_models;
+ dev_config.nb_queue_pairs = t->cmn.dev_info.max_queue_pairs;
+ ret = rte_ml_dev_configure(opt->dev_id, &dev_config);
+ if (ret != 0) {
+ ml_err("Failed to reconfigure ML device, dev_id = %d\n", opt->dev_id);
+ return ret;
+ }
+
+ /* setup queue pairs */
+ for (qp_id = 0; qp_id < t->cmn.dev_info.max_queue_pairs; qp_id++) {
+ qp_conf.nb_desc = t->cmn.dev_info.max_desc;
+ qp_conf.cb = NULL;
+
+ ret = rte_ml_dev_queue_pair_setup(opt->dev_id, qp_id, &qp_conf, opt->socket_id);
+ if (ret != 0) {
+ ml_err("Failed to setup ML device queue-pair, dev_id = %d, qp_id = %u\n",
+ opt->dev_id, qp_id);
+ goto error;
+ }
+ }
+
+ /* start device */
+ ret = ml_test_device_start(test, opt);
+ if (ret != 0)
+ goto error;
+
+ /* stop device */
+ ret = ml_test_device_stop(test, opt);
+ if (ret != 0)
+ goto error;
+
+ /* close device */
+ ret = ml_test_device_close(test, opt);
+ if (ret != 0)
+ return ret;
+
+ return 0;
+
+error:
+ ml_test_device_close(test, opt);
+
+ return ret;
+}
+
+static int
+test_device_driver(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_device *t;
+ int ret = 0;
+
+ t = ml_test_priv(test);
+
+ /* sub-test: device reconfigure */
+ ret = test_device_reconfigure(test, opt);
+ if (ret != 0) {
+ printf("\n");
+ printf("Model Device Reconfigure Test: " CLRED "%s" CLNRM "\n", "Failed");
+ goto error;
+ } else {
+ printf("\n");
+ printf("Model Device Reconfigure Test: " CLYEL "%s" CLNRM "\n", "Passed");
+ }
+
+ printf("\n");
+
+ t->cmn.result = ML_TEST_SUCCESS;
+
+ return 0;
+
+error:
+ t->cmn.result = ML_TEST_FAILED;
+ return -1;
+}
+
+static int
+test_device_result(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_device *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+
+ return t->cmn.result;
+}
+
+static const struct ml_test_ops device_ops = {
+ .cap_check = test_device_cap_check,
+ .opt_check = test_device_opt_check,
+ .opt_dump = test_device_opt_dump,
+ .test_setup = test_device_setup,
+ .test_destroy = test_device_destroy,
+ .test_driver = test_device_driver,
+ .test_result = test_device_result,
+};
+
+ML_TEST_REGISTER(device_ops);
diff --git a/app/test-mldev/test_device_ops.h b/app/test-mldev/test_device_ops.h
new file mode 100644
index 0000000000..115b1072a2
--- /dev/null
+++ b/app/test-mldev/test_device_ops.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_TEST_DEVICE_OPS_
+#define _ML_TEST_DEVICE_OPS_
+
+#include <rte_common.h>
+
+#include "test_common.h"
+
+struct test_device {
+ /* common data */
+ struct test_common cmn;
+} __rte_cache_aligned;
+
+#endif /* _ML_TEST_DEVICE_OPS_ */
diff --git a/doc/guides/tools/testmldev.rst b/doc/guides/tools/testmldev.rst
index 9dd7606d0b..1771c49fb4 100644
--- a/doc/guides/tools/testmldev.rst
+++ b/doc/guides/tools/testmldev.rst
@@ -44,7 +44,12 @@ The following are the command-line options supported by the test application.
* ``--test <name>``
- Name of the test to execute.
+ Name of the test to execute. ML tests supported include device tests. Test name should be
+ one of the following supported tests.
+
+ **ML Device Tests** ::
+
+ device_ops
* ``--dev_id <n>``
@@ -63,6 +68,41 @@ The following are the command-line options supported by the test application.
Print help message.
+ML Device Tests
+-------------------------
+
+ML device tests are functional tests to validate ML device APIs. Device tests validate the ML device
+handling APIs configure, close, start and stop APIs.
+
+
+Application Options
+~~~~~~~~~~~~~~~~~~~
+
+Supported command line options for the `device_ops` test are following::
+
+ --debug
+ --test
+ --dev_id
+ --socket_id
+
+
+DEVICE_OPS Test
+~~~~~~~~~~~~~~~
+
+Device ops test validates the device configuration and reconfiguration.
+
+
+Example
+^^^^^^^
+
+Command to run device_ops test:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=device_ops
+
+
Debug mode
----------
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v7 04/11] app/mldev: add test case to validate model ops
2023-03-16 21:14 ` [PATCH v7 00/11] Implementation of mldev test application Srikanth Yalavarthi
` (2 preceding siblings ...)
2023-03-16 21:14 ` [PATCH v7 03/11] app/mldev: add test case to validate device ops Srikanth Yalavarthi
@ 2023-03-16 21:14 ` Srikanth Yalavarthi
2023-03-16 21:14 ` [PATCH v7 05/11] app/mldev: add ordered inference test case Srikanth Yalavarthi
` (7 subsequent siblings)
11 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-16 21:14 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Added test case to validate model operations. Model ops test
is a collection of sub-tests. Each sub-test invokes the model
operations in a specific order.
Sub-test A: (load -> start -> stop -> unload) x n
Sub-test B: load x n -> start x n -> stop x n -> unload x n
Sub-test C: load x n + (start + stop) x n + unload x n
Sub-test D: (load + start) x n -> (stop + unload) x n
Added internal functions to handle model load, start, stop and
unload. List of models to be used for testing can be specified
through application argument "--models"
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
Acked-by: Anup Prabhu <aprabhu@marvell.com>
---
app/test-mldev/meson.build | 2 +
app/test-mldev/ml_options.c | 39 ++
app/test-mldev/ml_options.h | 6 +
app/test-mldev/test_model_common.c | 158 +++++++
app/test-mldev/test_model_common.h | 34 ++
app/test-mldev/test_model_ops.c | 428 ++++++++++++++++++
app/test-mldev/test_model_ops.h | 20 +
.../tools/img/mldev_model_ops_subtest_a.svg | 420 +++++++++++++++++
.../tools/img/mldev_model_ops_subtest_b.svg | 423 +++++++++++++++++
.../tools/img/mldev_model_ops_subtest_c.svg | 366 +++++++++++++++
.../tools/img/mldev_model_ops_subtest_d.svg | 424 +++++++++++++++++
doc/guides/tools/testmldev.rst | 97 +++-
12 files changed, 2415 insertions(+), 2 deletions(-)
create mode 100644 app/test-mldev/test_model_common.c
create mode 100644 app/test-mldev/test_model_common.h
create mode 100644 app/test-mldev/test_model_ops.c
create mode 100644 app/test-mldev/test_model_ops.h
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_a.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_b.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_c.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_d.svg
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
index 60ea23d142..b09e1ccc8a 100644
--- a/app/test-mldev/meson.build
+++ b/app/test-mldev/meson.build
@@ -14,6 +14,8 @@ sources = files(
'parser.c',
'test_common.c',
'test_device_ops.c',
+ 'test_model_common.c',
+ 'test_model_ops.c',
)
deps += ['mldev']
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 854d316521..8ffbab7f75 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -22,6 +22,7 @@ ml_options_default(struct ml_options *opt)
strlcpy(opt->test_name, "device_ops", ML_TEST_NAME_MAX_LEN);
opt->dev_id = 0;
opt->socket_id = SOCKET_ID_ANY;
+ opt->nb_filelist = 0;
opt->debug = false;
}
@@ -58,11 +59,47 @@ ml_parse_socket_id(struct ml_options *opt, const char *arg)
return 0;
}
+static int
+ml_parse_models(struct ml_options *opt, const char *arg)
+{
+ const char *delim = ",";
+ char models[PATH_MAX];
+ char *token;
+ int ret = 0;
+
+ strlcpy(models, arg, PATH_MAX);
+
+ token = strtok(models, delim);
+ while (token != NULL) {
+ strlcpy(opt->filelist[opt->nb_filelist].model, token, PATH_MAX);
+ opt->nb_filelist++;
+
+ if (opt->nb_filelist >= ML_TEST_MAX_MODELS) {
+ ml_err("Exceeded model count, max = %d\n", ML_TEST_MAX_MODELS);
+ ret = -EINVAL;
+ break;
+ }
+ token = strtok(NULL, delim);
+ }
+
+ if (opt->nb_filelist == 0) {
+ ml_err("Models list is empty. Need at least one model for the test");
+ ret = -EINVAL;
+ }
+
+ return ret;
+}
+
static void
ml_dump_test_options(const char *testname)
{
if (strcmp(testname, "device_ops") == 0)
printf("\n");
+
+ if (strcmp(testname, "model_ops") == 0) {
+ printf("\t\t--models : comma separated list of models\n");
+ printf("\n");
+ }
}
static void
@@ -84,6 +121,7 @@ static struct option lgopts[] = {
{ML_TEST, 1, 0, 0},
{ML_DEVICE_ID, 1, 0, 0},
{ML_SOCKET_ID, 1, 0, 0},
+ {ML_MODELS, 1, 0, 0},
{ML_DEBUG, 0, 0, 0},
{ML_HELP, 0, 0, 0},
{NULL, 0, 0, 0}};
@@ -97,6 +135,7 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
{ML_TEST, ml_parse_test_name},
{ML_DEVICE_ID, ml_parse_dev_id},
{ML_SOCKET_ID, ml_parse_socket_id},
+ {ML_MODELS, ml_parse_models},
};
for (i = 0; i < RTE_DIM(parsermap); i++) {
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index 6899cc2f88..61e938d2e2 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -15,13 +15,19 @@
#define ML_TEST ("test")
#define ML_DEVICE_ID ("dev_id")
#define ML_SOCKET_ID ("socket_id")
+#define ML_MODELS ("models")
#define ML_DEBUG ("debug")
#define ML_HELP ("help")
+struct ml_filelist {
+ char model[PATH_MAX];
+};
+
struct ml_options {
char test_name[ML_TEST_NAME_MAX_LEN];
int16_t dev_id;
int socket_id;
+ struct ml_filelist filelist[ML_TEST_MAX_MODELS];
uint8_t nb_filelist;
bool debug;
};
diff --git a/app/test-mldev/test_model_common.c b/app/test-mldev/test_model_common.c
new file mode 100644
index 0000000000..b94d46154d
--- /dev/null
+++ b/app/test-mldev/test_model_common.c
@@ -0,0 +1,158 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <errno.h>
+
+#include <rte_common.h>
+#include <rte_malloc.h>
+#include <rte_mldev.h>
+
+#include "ml_common.h"
+#include "test_model_common.h"
+
+int
+ml_model_load(struct ml_test *test, struct ml_options *opt, struct ml_model *model, uint16_t fid)
+{
+ struct test_common *t = ml_test_priv(test);
+ struct rte_ml_model_params model_params;
+ FILE *fp;
+ int ret;
+
+ if (model->state == MODEL_LOADED)
+ return 0;
+
+ if (model->state != MODEL_INITIAL)
+ return -EINVAL;
+
+ /* read model binary */
+ fp = fopen(opt->filelist[fid].model, "r");
+ if (fp == NULL) {
+ ml_err("Failed to open model file : %s\n", opt->filelist[fid].model);
+ return -1;
+ }
+
+ fseek(fp, 0, SEEK_END);
+ model_params.size = ftell(fp);
+ fseek(fp, 0, SEEK_SET);
+
+ model_params.addr = rte_malloc_socket("ml_model", model_params.size,
+ t->dev_info.min_align_size, opt->socket_id);
+ if (model_params.addr == NULL) {
+ ml_err("Failed to allocate memory for model: %s\n", opt->filelist[fid].model);
+ fclose(fp);
+ return -ENOMEM;
+ }
+
+ if (fread(model_params.addr, 1, model_params.size, fp) != model_params.size) {
+ ml_err("Failed to read model file : %s\n", opt->filelist[fid].model);
+ rte_free(model_params.addr);
+ fclose(fp);
+ return -1;
+ }
+ fclose(fp);
+
+ /* load model to device */
+ ret = rte_ml_model_load(opt->dev_id, &model_params, &model->id);
+ if (ret != 0) {
+ ml_err("Failed to load model : %s\n", opt->filelist[fid].model);
+ model->state = MODEL_ERROR;
+ rte_free(model_params.addr);
+ return ret;
+ }
+
+ /* release mz */
+ rte_free(model_params.addr);
+
+ /* get model info */
+ ret = rte_ml_model_info_get(opt->dev_id, model->id, &model->info);
+ if (ret != 0) {
+ ml_err("Failed to get model info : %s\n", opt->filelist[fid].model);
+ return ret;
+ }
+
+ model->state = MODEL_LOADED;
+
+ return 0;
+}
+
+int
+ml_model_unload(struct ml_test *test, struct ml_options *opt, struct ml_model *model, uint16_t fid)
+{
+ struct test_common *t = ml_test_priv(test);
+ int ret;
+
+ RTE_SET_USED(t);
+
+ if (model->state == MODEL_INITIAL)
+ return 0;
+
+ if (model->state != MODEL_LOADED)
+ return -EINVAL;
+
+ /* unload model */
+ ret = rte_ml_model_unload(opt->dev_id, model->id);
+ if (ret != 0) {
+ ml_err("Failed to unload model: %s\n", opt->filelist[fid].model);
+ model->state = MODEL_ERROR;
+ return ret;
+ }
+
+ model->state = MODEL_INITIAL;
+
+ return 0;
+}
+
+int
+ml_model_start(struct ml_test *test, struct ml_options *opt, struct ml_model *model, uint16_t fid)
+{
+ struct test_common *t = ml_test_priv(test);
+ int ret;
+
+ RTE_SET_USED(t);
+
+ if (model->state == MODEL_STARTED)
+ return 0;
+
+ if (model->state != MODEL_LOADED)
+ return -EINVAL;
+
+ /* start model */
+ ret = rte_ml_model_start(opt->dev_id, model->id);
+ if (ret != 0) {
+ ml_err("Failed to start model : %s\n", opt->filelist[fid].model);
+ model->state = MODEL_ERROR;
+ return ret;
+ }
+
+ model->state = MODEL_STARTED;
+
+ return 0;
+}
+
+int
+ml_model_stop(struct ml_test *test, struct ml_options *opt, struct ml_model *model, uint16_t fid)
+{
+ struct test_common *t = ml_test_priv(test);
+ int ret;
+
+ RTE_SET_USED(t);
+
+ if (model->state == MODEL_LOADED)
+ return 0;
+
+ if (model->state != MODEL_STARTED)
+ return -EINVAL;
+
+ /* stop model */
+ ret = rte_ml_model_stop(opt->dev_id, model->id);
+ if (ret != 0) {
+ ml_err("Failed to stop model: %s\n", opt->filelist[fid].model);
+ model->state = MODEL_ERROR;
+ return ret;
+ }
+
+ model->state = MODEL_LOADED;
+
+ return 0;
+}
diff --git a/app/test-mldev/test_model_common.h b/app/test-mldev/test_model_common.h
new file mode 100644
index 0000000000..74aec0a797
--- /dev/null
+++ b/app/test-mldev/test_model_common.h
@@ -0,0 +1,34 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_TEST_MODEL_COMMON_
+#define _ML_TEST_MODEL_COMMON_
+
+#include <rte_mldev.h>
+
+#include "test_common.h"
+
+enum model_state {
+ MODEL_INITIAL,
+ MODEL_LOADED,
+ MODEL_STARTED,
+ MODEL_ERROR,
+};
+
+struct ml_model {
+ uint16_t id;
+ struct rte_ml_model_info info;
+ enum model_state state;
+};
+
+int ml_model_load(struct ml_test *test, struct ml_options *opt, struct ml_model *model,
+ uint16_t fid);
+int ml_model_unload(struct ml_test *test, struct ml_options *opt, struct ml_model *model,
+ uint16_t fid);
+int ml_model_start(struct ml_test *test, struct ml_options *opt, struct ml_model *model,
+ uint16_t fid);
+int ml_model_stop(struct ml_test *test, struct ml_options *opt, struct ml_model *model,
+ uint16_t fid);
+
+#endif /* _ML_TEST_MODEL_COMMON_ */
diff --git a/app/test-mldev/test_model_ops.c b/app/test-mldev/test_model_ops.c
new file mode 100644
index 0000000000..0202b31190
--- /dev/null
+++ b/app/test-mldev/test_model_ops.c
@@ -0,0 +1,428 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <unistd.h>
+
+#include <rte_common.h>
+#include <rte_malloc.h>
+#include <rte_mldev.h>
+
+#include "test_model_ops.h"
+
+static bool
+test_model_ops_cap_check(struct ml_options *opt)
+{
+ if (!ml_test_cap_check(opt))
+ return false;
+
+ return true;
+}
+
+static int
+test_model_ops_opt_check(struct ml_options *opt)
+{
+ uint32_t i;
+ int ret;
+
+ /* check common opts */
+ ret = ml_test_opt_check(opt);
+ if (ret != 0)
+ return ret;
+
+ /* check model file availability */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ if (access(opt->filelist[i].model, F_OK) == -1) {
+ ml_err("Model file not available: id = %u, file = %s", i,
+ opt->filelist[i].model);
+ return -ENOENT;
+ }
+ }
+
+ return 0;
+}
+
+static void
+test_model_ops_opt_dump(struct ml_options *opt)
+{
+ uint32_t i;
+
+ /* dump common opts */
+ ml_test_opt_dump(opt);
+
+ /* dump test specific opts */
+ ml_dump_begin("models");
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_dump_list("model", i, opt->filelist[i].model);
+ ml_dump_end;
+}
+
+static int
+test_model_ops_setup(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+ void *test_model_ops;
+ int ret = 0;
+ uint32_t i;
+
+ /* allocate model ops test structure */
+ test_model_ops = rte_zmalloc_socket(test->name, sizeof(struct test_model_ops),
+ RTE_CACHE_LINE_SIZE, opt->socket_id);
+ if (test_model_ops == NULL) {
+ ml_err("Failed to allocate memory for test_model");
+ ret = -ENOMEM;
+ goto error;
+ }
+ test->test_priv = test_model_ops;
+ t = ml_test_priv(test);
+
+ t->cmn.result = ML_TEST_FAILED;
+ t->cmn.opt = opt;
+
+ /* get device info */
+ ret = rte_ml_dev_info_get(opt->dev_id, &t->cmn.dev_info);
+ if (ret < 0) {
+ ml_err("Failed to get device info");
+ goto error;
+ }
+
+ /* set model initial state */
+ for (i = 0; i < opt->nb_filelist; i++)
+ t->model[i].state = MODEL_INITIAL;
+
+ return 0;
+
+error:
+ if (test_model_ops != NULL)
+ rte_free(test_model_ops);
+
+ return ret;
+}
+
+static void
+test_model_ops_destroy(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+ if (t != NULL)
+ rte_free(t);
+}
+
+static int
+test_model_ops_mldev_setup(struct ml_test *test, struct ml_options *opt)
+{
+ int ret;
+
+ ret = ml_test_device_configure(test, opt);
+ if (ret != 0)
+ return ret;
+
+ ret = ml_test_device_start(test, opt);
+ if (ret != 0)
+ goto error;
+
+ return 0;
+
+error:
+ ml_test_device_close(test, opt);
+
+ return ret;
+}
+
+static int
+test_model_ops_mldev_destroy(struct ml_test *test, struct ml_options *opt)
+{
+ int ret;
+
+ ret = ml_test_device_stop(test, opt);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_test_device_close(test, opt);
+ if (ret != 0)
+ return ret;
+
+ return 0;
+
+error:
+ ml_test_device_close(test, opt);
+
+ return ret;
+}
+
+/* Sub-test A: (load -> start -> stop -> unload) x n */
+static int
+test_model_ops_subtest_a(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+ int ret = 0;
+ uint32_t i;
+
+ t = ml_test_priv(test);
+
+ /* load + start + stop + unload */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_load(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_start(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_stop(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_unload(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+error:
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_stop(test, opt, &t->model[i], i);
+
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_unload(test, opt, &t->model[i], i);
+
+ return ret;
+}
+
+/* Sub-test B: load x n -> start x n -> stop x n -> unload x n */
+static int
+test_model_ops_subtest_b(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+ int ret = 0;
+ uint32_t i;
+
+ t = ml_test_priv(test);
+
+ /* load */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_load(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* start */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_start(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* stop */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_stop(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* unload */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_unload(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ return 0;
+
+error:
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_stop(test, opt, &t->model[i], i);
+
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_unload(test, opt, &t->model[i], i);
+
+ return ret;
+}
+
+/* Sub-test C: load x n + (start + stop) x n + unload x n */
+static int
+test_model_ops_subtest_c(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+ int ret = 0;
+ uint32_t i;
+
+ t = ml_test_priv(test);
+
+ /* load */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_load(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* start + stop */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_start(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_stop(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* unload */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_unload(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ return 0;
+
+error:
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_stop(test, opt, &t->model[i], i);
+
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_unload(test, opt, &t->model[i], i);
+
+ return ret;
+}
+
+/* Sub-test D: (load + start) x n -> (stop + unload) x n */
+static int
+test_model_ops_subtest_d(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+ int ret = 0;
+ uint32_t i;
+
+ t = ml_test_priv(test);
+
+ /* load + start */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_load(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_start(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* stop + unload */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ret = ml_model_stop(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_unload(test, opt, &t->model[i], i);
+ if (ret != 0)
+ goto error;
+ }
+
+ return 0;
+
+error:
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_stop(test, opt, &t->model[i], i);
+
+ for (i = 0; i < opt->nb_filelist; i++)
+ ml_model_unload(test, opt, &t->model[i], i);
+
+ return ret;
+}
+
+static int
+test_model_ops_driver(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+ int ret = 0;
+
+ t = ml_test_priv(test);
+
+ /* device setup */
+ ret = test_model_ops_mldev_setup(test, opt);
+ if (ret != 0)
+ return ret;
+
+ printf("\n");
+
+ /* sub-test A */
+ ret = test_model_ops_subtest_a(test, opt);
+ if (ret != 0) {
+ printf("Model Ops Sub-test A: " CLRED "%s" CLNRM "\n", "Failed");
+ goto error;
+ } else {
+ printf("Model Ops Sub-test A: " CLYEL "%s" CLNRM "\n", "Passed");
+ }
+
+ /* sub-test B */
+ ret = test_model_ops_subtest_b(test, opt);
+ if (ret != 0) {
+ printf("Model Ops Sub-test B: " CLRED "%s" CLNRM "\n", "Failed");
+ goto error;
+ } else {
+ printf("Model Ops Sub-test B: " CLYEL "%s" CLNRM "\n", "Passed");
+ }
+
+ /* sub-test C */
+ ret = test_model_ops_subtest_c(test, opt);
+ if (ret != 0) {
+ printf("Model Ops Sub-test C: " CLRED "%s" CLNRM "\n", "Failed");
+ goto error;
+ } else {
+ printf("Model Ops Sub-test C: " CLYEL "%s" CLNRM "\n", "Passed");
+ }
+
+ /* sub-test D */
+ ret = test_model_ops_subtest_d(test, opt);
+ if (ret != 0) {
+ printf("Model Ops Sub-test D: " CLRED "%s" CLNRM "\n", "Failed");
+ goto error;
+ } else {
+ printf("Model Ops Sub-test D: " CLYEL "%s" CLNRM "\n", "Passed");
+ }
+
+ printf("\n");
+
+ /* device destroy */
+ ret = test_model_ops_mldev_destroy(test, opt);
+ if (ret != 0)
+ return ret;
+
+ t->cmn.result = ML_TEST_SUCCESS;
+
+ return 0;
+
+error:
+ test_model_ops_mldev_destroy(test, opt);
+
+ t->cmn.result = ML_TEST_FAILED;
+
+ return ret;
+}
+
+static int
+test_model_ops_result(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_model_ops *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+
+ return t->cmn.result;
+}
+
+static const struct ml_test_ops model_ops = {
+ .cap_check = test_model_ops_cap_check,
+ .opt_check = test_model_ops_opt_check,
+ .opt_dump = test_model_ops_opt_dump,
+ .test_setup = test_model_ops_setup,
+ .test_destroy = test_model_ops_destroy,
+ .test_driver = test_model_ops_driver,
+ .test_result = test_model_ops_result,
+};
+
+ML_TEST_REGISTER(model_ops);
diff --git a/app/test-mldev/test_model_ops.h b/app/test-mldev/test_model_ops.h
new file mode 100644
index 0000000000..c21f12b788
--- /dev/null
+++ b/app/test-mldev/test_model_ops.h
@@ -0,0 +1,20 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_TEST_MODEL_OPS_
+#define _ML_TEST_MODEL_OPS_
+
+#include <rte_common.h>
+
+#include "test_model_common.h"
+
+struct test_model_ops {
+ /* common data */
+ struct test_common cmn;
+
+ /* test specific data */
+ struct ml_model model[ML_TEST_MAX_MODELS];
+} __rte_cache_aligned;
+
+#endif /* _ML_TEST_MODEL_OPS_ */
diff --git a/doc/guides/tools/img/mldev_model_ops_subtest_a.svg b/doc/guides/tools/img/mldev_model_ops_subtest_a.svg
new file mode 100644
index 0000000000..ed12cc5a05
--- /dev/null
+++ b/doc/guides/tools/img/mldev_model_ops_subtest_a.svg
@@ -0,0 +1,420 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- SPDX-License-Identifier: BSD-3-Clause -->
+<!-- Copyright (c) 2022 Marvell. -->
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="275mm"
+ height="135mm"
+ viewBox="0 0 275 135"
+ version="1.1"
+ id="svg5"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="mldev_model_ops_subtest_d.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg">
+ <sodipodi:namedview
+ id="namedview7"
+ pagecolor="#ffffff"
+ bordercolor="#000000"
+ borderopacity="0.25"
+ inkscape:showpageshadow="2"
+ inkscape:pageopacity="0.0"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1"
+ inkscape:document-units="mm"
+ showgrid="false"
+ inkscape:zoom="0.72515432"
+ inkscape:cx="372.33454"
+ inkscape:cy="401.29389"
+ inkscape:window-width="1920"
+ inkscape:window-height="986"
+ inkscape:window-x="-11"
+ inkscape:window-y="-11"
+ inkscape:window-maximized="1"
+ inkscape:current-layer="layer1" />
+ <defs
+ id="defs2">
+ <marker
+ style="overflow:visible"
+ id="RoundedArrow"
+ refX="6"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="RoundedArrow"
+ markerWidth="6.1347523"
+ markerHeight="5.9304948"
+ viewBox="0 0 6.1347524 5.9304951"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.7)"
+ d="m -0.21114562,-4.1055728 6.42229122,3.21114561 a 1,1 90 0 1 0,1.78885438 L -0.21114562,4.1055728 A 1.236068,1.236068 31.717474 0 1 -2,3 v -6 a 1.236068,1.236068 148.28253 0 1 1.78885438,-1.1055728 z"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:none"
+ id="path1367" />
+ </marker>
+ </defs>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1">
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 62.203489,25.425124 H 80.823463"
+ id="path1916"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234"
+ inkscape:connection-end="#rect234-07" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 128.19001,25.425124 h 18.6197"
+ id="path1918"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-07"
+ inkscape:connection-end="#rect234-6" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 194.17681,25.425125 h 18.61942"
+ id="path1922"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-6"
+ inkscape:connection-end="#rect234-0-6-6" />
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.633453;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234"
+ width="47.366547"
+ height="17.366549"
+ x="14.836943"
+ y="16.74185"
+ ry="1.4985789"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-25.888845"
+ y="38.514706"
+ id="text290"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-25.888845"
+ y="38.514706">Model 0 / Load</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.633453;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-07"
+ width="47.366547"
+ height="17.366549"
+ x="80.823463"
+ y="16.74185"
+ ry="1.4985789"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-25.888845"
+ y="104.31795"
+ id="text290-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-11"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-25.888845"
+ y="104.31795">Model 0 / Start</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.6329;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-6"
+ width="47.3671"
+ height="17.33604"
+ x="146.80971"
+ y="16.757105"
+ ry="1.4959463" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-25.362436"
+ y="170.39679"
+ id="text290-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-15"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-25.362436"
+ y="170.39679">Model 0 / Stop</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6"
+ width="47.3671"
+ height="17.33604"
+ x="212.79623"
+ y="16.757105"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-25.888849"
+ y="236.47427"
+ id="text290-3-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-25.888849"
+ y="236.47427">Model 0 / Unload</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 236.47978,34.093145 V 44.957249 H 38.520216 v 10.84885"
+ id="path1924"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6"
+ inkscape:connection-end="#rect234-0-6-6-3" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 62.203766,64.474119 H 80.823189"
+ id="path1926"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-3"
+ inkscape:connection-end="#rect234-0-6-6-2" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 128.19029,64.474119 h 18.61942"
+ id="path1928"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-2"
+ inkscape:connection-end="#rect234-0-6-1" />
+ <rect
+ style="fill:#fff6d5;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-3"
+ width="47.3671"
+ height="17.33604"
+ x="14.836666"
+ y="55.806099"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-64.937843"
+ y="38.541786"
+ id="text290-3-2-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-64.937843"
+ y="38.541786">Model 1 / Load</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-2"
+ width="47.3671"
+ height="17.33604"
+ x="80.823189"
+ y="55.806099"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-64.937843"
+ y="104.31795"
+ id="text290-3-2-2-28"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-2"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-64.937843"
+ y="104.31795">Model 1 / Start</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6"
+ width="47.3671"
+ height="17.33604"
+ x="212.79623"
+ y="55.806099"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-64.937843"
+ y="236.47427"
+ id="text290-3-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-64.937843"
+ y="236.47427">Model 1 / Unload</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-1"
+ width="47.3671"
+ height="17.33604"
+ x="146.80971"
+ y="55.806099"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-64.41143"
+ y="170.39679"
+ id="text290-3-2-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-8"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-64.41143"
+ y="170.39679">Model 1 / Stop</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 194.17681,64.474119 h 18.61942"
+ id="path1930"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6"
+ inkscape:connection-start="#rect234-0-6-1" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 236.47978,73.142139 V 83.990988 H 38.520216 V 94.85511"
+ id="path1932"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6-7"
+ inkscape:connection-start="#rect234-0-6" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 62.203766,103.52313 h 9.309711 v 1.3e-4 h 9.309712"
+ id="path1934"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-7" />
+ <rect
+ style="fill:#fff6d5;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-7"
+ width="47.3671"
+ height="17.33604"
+ x="14.836666"
+ y="94.85511"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.98687"
+ y="38.541786"
+ id="text290-3-2-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.98687"
+ y="38.541786">Model N / Load</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0"
+ width="47.3671"
+ height="17.33604"
+ x="80.823189"
+ y="94.855164"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.9869"
+ y="104.31795"
+ id="text290-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.9869"
+ y="104.31795">Model N / Start</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-9"
+ width="47.3671"
+ height="17.33604"
+ x="146.80971"
+ y="94.855164"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.4605"
+ y="170.39679"
+ id="text290-3-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.4605"
+ y="170.39679">Model N / Stop</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.633452;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-5"
+ width="47.366547"
+ height="17.366549"
+ x="212.79651"
+ y="94.839836"
+ ry="1.4985789"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.98682"
+ y="236.47427"
+ id="text290-3-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-3"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.98682"
+ y="236.47427">Model N / Unload</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 128.19029,103.52326 h 18.61942"
+ id="path1936"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 194.17681,103.52326 h 9.30985 v -1.5e-4 h 9.30985"
+ id="path1938"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-5" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#fff6d5;fill-rule:evenodd;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-120.71075"
+ y="220.77164"
+ id="text392"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan390"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';fill:#000000;stroke-width:0.5"
+ x="-120.71075"
+ y="220.77164">mldev: model_ops / subtest D</tspan></text>
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.448724;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect2164"
+ width="259.55127"
+ height="119.55128"
+ x="7.7243652"
+ y="7.7243614"
+ ry="1.17494" />
+ </g>
+</svg>
diff --git a/doc/guides/tools/img/mldev_model_ops_subtest_b.svg b/doc/guides/tools/img/mldev_model_ops_subtest_b.svg
new file mode 100644
index 0000000000..173a2c6c05
--- /dev/null
+++ b/doc/guides/tools/img/mldev_model_ops_subtest_b.svg
@@ -0,0 +1,423 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- SPDX-License-Identifier: BSD-3-Clause -->
+<!-- Copyright (c) 2022 Marvell. -->
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="275mm"
+ height="135mm"
+ viewBox="0 0 275 135"
+ version="1.1"
+ id="svg5"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="mldev_model_ops_subtest_a.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg">
+ <sodipodi:namedview
+ id="namedview7"
+ pagecolor="#ffffff"
+ bordercolor="#000000"
+ borderopacity="0.25"
+ inkscape:showpageshadow="2"
+ inkscape:pageopacity="0.0"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1"
+ inkscape:document-units="mm"
+ showgrid="false"
+ inkscape:zoom="0.72515432"
+ inkscape:cx="277.87189"
+ inkscape:cy="401.29389"
+ inkscape:window-width="1920"
+ inkscape:window-height="986"
+ inkscape:window-x="-11"
+ inkscape:window-y="-11"
+ inkscape:window-maximized="1"
+ inkscape:current-layer="layer1" />
+ <defs
+ id="defs2">
+ <marker
+ style="overflow:visible"
+ id="RoundedArrow"
+ refX="6"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="RoundedArrow"
+ markerWidth="6.1347523"
+ markerHeight="5.9304948"
+ viewBox="0 0 6.1347524 5.9304951"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.7)"
+ d="m -0.21114562,-4.1055728 6.42229122,3.21114561 a 1,1 90 0 1 0,1.78885438 L -0.21114562,4.1055728 A 1.236068,1.236068 31.717474 0 1 -2,3 v -6 a 1.236068,1.236068 148.28253 0 1 1.78885438,-1.1055728 z"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:none"
+ id="path1367" />
+ </marker>
+ </defs>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1">
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.671403;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234"
+ width="53.328598"
+ height="17.328598"
+ x="15.144302"
+ y="12.878438"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-22.00646"
+ y="41.803089"
+ id="text290"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-22.00646"
+ y="41.803089">Model 0 / Load</tspan></text>
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.671403;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-07"
+ width="53.328598"
+ height="17.328598"
+ x="87.552399"
+ y="12.878438"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-22.00646"
+ y="114.21132"
+ id="text290-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-11"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-22.00646"
+ y="114.21132">Model 1 / Load</tspan></text>
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.671403;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-6"
+ width="53.328598"
+ height="17.328598"
+ x="206.5271"
+ y="12.878438"
+ ry="1.4953041" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-22.00646"
+ y="233.18588"
+ id="text290-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-15"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-22.00646"
+ y="233.18588">Model N / Load</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 68.4729,21.542737 H 87.552399"
+ id="path1916"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234"
+ inkscape:connection-end="#rect234-07" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 140.881,21.542737 h 65.6461"
+ id="path1918"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-07"
+ inkscape:connection-end="#rect234-6" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 233.1914,30.207036 0,4.918771 H 41.808601 v 4.918772"
+ id="path1922"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-6"
+ inkscape:connection-end="#rect234-0-6-6" />
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6"
+ width="53.328598"
+ height="17.328598"
+ x="15.144302"
+ y="40.044579"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-49.1726"
+ y="41.619808"
+ id="text290-3-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-49.1726"
+ y="41.619808">Model 0 / Start</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-3"
+ width="53.328598"
+ height="17.328598"
+ x="87.552399"
+ y="40.044579"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-49.1726"
+ y="114.02805"
+ id="text290-3-2-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-49.1726"
+ y="114.02805">Model 1 / Start</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-2"
+ width="53.328598"
+ height="17.328598"
+ x="206.5271"
+ y="40.044579"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-49.1726"
+ y="233.00261"
+ id="text290-3-2-2-28"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-2"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-49.1726"
+ y="233.00261">Model N / Start</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 68.4729,48.708878 H 87.552399"
+ id="path1924"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6"
+ inkscape:connection-end="#rect234-0-6-6-3" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 140.881,48.708878 h 65.6461"
+ id="path1926"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-3"
+ inkscape:connection-end="#rect234-0-6-6-2" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 233.1914,57.373177 V 62.29195 H 41.808599 v 4.918774"
+ id="path1928"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-2"
+ inkscape:connection-end="#rect234-0-6-1" />
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6"
+ width="53.328598"
+ height="17.328598"
+ x="87.552399"
+ y="67.210724"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-75.812294"
+ y="114.12037"
+ id="text290-3-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-75.812294"
+ y="114.12037">Model 1 / Stop</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-1"
+ width="53.328598"
+ height="17.328598"
+ x="15.1443"
+ y="67.210724"
+ ry="1.4953041" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-75.812294"
+ y="41.712139"
+ id="text290-3-2-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-8"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-75.812294"
+ y="41.712139">Model 0 / Stop</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-7"
+ width="53.328598"
+ height="17.328598"
+ x="206.5271"
+ y="67.210724"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-75.812294"
+ y="233.09494"
+ id="text290-3-2-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-75.812294"
+ y="233.09494">Model N / Stop</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 68.472898,75.875023 H 87.552399"
+ id="path1930"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6"
+ inkscape:connection-start="#rect234-0-6-1" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 140.881,75.875023 h 65.6461"
+ id="path1932"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6-7"
+ inkscape:connection-start="#rect234-0-6" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 233.1914,84.539322 0,4.91877 H 41.808602 v 4.91877"
+ id="path1934"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0"
+ inkscape:connection-start="#rect234-0-6-7" />
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0"
+ width="53.328598"
+ height="17.328598"
+ x="15.144303"
+ y="94.376862"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.50489"
+ y="41.803085"
+ id="text290-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.50489"
+ y="41.803085">Model 0 / Unload</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-9"
+ width="53.328598"
+ height="17.328598"
+ x="87.552399"
+ y="94.376862"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.50489"
+ y="114.2113"
+ id="text290-3-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.50489"
+ y="114.2113">Model 1 / Unload</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-5"
+ width="53.328598"
+ height="17.328598"
+ x="206.5271"
+ y="94.376862"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-103.50489"
+ y="233.18588"
+ id="text290-3-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-3"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-103.50489"
+ y="233.18588">Model N / Unload</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 68.472901,103.04116 H 87.552399"
+ id="path1936"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-9"
+ inkscape:connection-start="#rect234-0" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 140.881,103.04116 h 65.6461"
+ id="path1938"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-5"
+ inkscape:connection-start="#rect234-0-9" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#fff6d5;fill-rule:evenodd;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-119.78458"
+ y="220.32892"
+ id="text392"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan390"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';fill:#000000;stroke-width:0.5"
+ x="-119.78458"
+ y="220.32892">mldev: model_ops / subtest A</tspan></text>
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.442854;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect3967"
+ width="259.55716"
+ height="119.55714"
+ x="7.7214203"
+ y="7.7214317"
+ ry="1.1806604" />
+ </g>
+</svg>
diff --git a/doc/guides/tools/img/mldev_model_ops_subtest_c.svg b/doc/guides/tools/img/mldev_model_ops_subtest_c.svg
new file mode 100644
index 0000000000..f66f146d05
--- /dev/null
+++ b/doc/guides/tools/img/mldev_model_ops_subtest_c.svg
@@ -0,0 +1,366 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- SPDX-License-Identifier: BSD-3-Clause -->
+<!-- Copyright (c) 2022 Marvell. -->
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="275mm"
+ height="135mm"
+ viewBox="0 0 275 135"
+ version="1.1"
+ id="svg5"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="mldev_model_ops_subtest_c.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg">
+ <sodipodi:namedview
+ id="namedview7"
+ pagecolor="#ffffff"
+ bordercolor="#000000"
+ borderopacity="0.25"
+ inkscape:showpageshadow="2"
+ inkscape:pageopacity="0.0"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1"
+ inkscape:document-units="mm"
+ showgrid="false"
+ inkscape:zoom="0.72515432"
+ inkscape:cx="479.89785"
+ inkscape:cy="401.29389"
+ inkscape:window-width="1920"
+ inkscape:window-height="986"
+ inkscape:window-x="-11"
+ inkscape:window-y="-11"
+ inkscape:window-maximized="1"
+ inkscape:current-layer="layer1" />
+ <defs
+ id="defs2">
+ <marker
+ style="overflow:visible"
+ id="RoundedArrow"
+ refX="6"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="RoundedArrow"
+ markerWidth="6.1347523"
+ markerHeight="5.9304948"
+ viewBox="0 0 6.1347524 5.9304951"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.7)"
+ d="m -0.21114562,-4.1055728 6.42229122,3.21114561 a 1,1 90 0 1 0,1.78885438 L -0.21114562,4.1055728 A 1.236068,1.236068 31.717474 0 1 -2,3 v -6 a 1.236068,1.236068 148.28253 0 1 1.78885438,-1.1055728 z"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:none"
+ id="path1367" />
+ </marker>
+ </defs>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1">
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.671403;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234"
+ width="53.328598"
+ height="17.328598"
+ x="12.6443"
+ y="13.208546"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-22.336561"
+ y="39.303089"
+ id="text290"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-22.336561"
+ y="39.303089">Model 0 / Load</tspan></text>
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.671403;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-07"
+ width="53.328598"
+ height="17.328598"
+ x="85.052399"
+ y="13.208546"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-22.336561"
+ y="111.71132"
+ id="text290-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-11"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-22.336561"
+ y="111.71132">Model 1 / Load</tspan></text>
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.671403;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-6"
+ width="53.328598"
+ height="17.328598"
+ x="204.0271"
+ y="13.208546"
+ ry="1.4953041" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-22.336561"
+ y="230.68588"
+ id="text290-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-15"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-22.336561"
+ y="230.68588">Model N / Load</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 65.972898,21.872845 H 85.052399"
+ id="path1916"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234"
+ inkscape:connection-end="#rect234-07" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 138.381,21.872845 h 65.6461"
+ id="path1918"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-07"
+ inkscape:connection-end="#rect234-6" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 230.6914,30.537144 V 36.33787 H 39.308599 v 5.800726"
+ id="path1922"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-6"
+ inkscape:connection-end="#rect234-0-6-6" />
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6"
+ width="53.328598"
+ height="17.328598"
+ x="12.6443"
+ y="42.138596"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-51.266617"
+ y="39.119808"
+ id="text290-3-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-51.266617"
+ y="39.119808">Model 0 / Start</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-3"
+ width="53.328598"
+ height="17.328598"
+ x="85.052399"
+ y="42.138596"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-51.266617"
+ y="111.52805"
+ id="text290-3-2-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-51.266617"
+ y="111.52805">Model 0 / Stop</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 65.972898,50.802895 H 85.052399"
+ id="path1924"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6"
+ inkscape:connection-end="#rect234-0-6-6-3" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 184.94759,79.732941 H 204.0271"
+ id="path1930"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6"
+ inkscape:connection-start="#rect234-0-6-1" />
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0"
+ width="53.328598"
+ height="17.328598"
+ x="12.644301"
+ y="99.998688"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-109.12671"
+ y="39.303085"
+ id="text290-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-109.12671"
+ y="39.303085">Model 0 / Unload</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-9"
+ width="53.328598"
+ height="17.328598"
+ x="85.052399"
+ y="99.998688"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-109.12671"
+ y="111.7113"
+ id="text290-3-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-109.12671"
+ y="111.7113">Model 1 / Unload</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-5"
+ width="53.328598"
+ height="17.328598"
+ x="204.0271"
+ y="99.998688"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-109.12671"
+ y="230.68588"
+ id="text290-3-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-3"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-109.12671"
+ y="230.68588">Model N / Unload</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 65.972899,108.66299 h 19.0795"
+ id="path1936"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-9"
+ inkscape:connection-start="#rect234-0" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 138.381,108.66299 h 65.6461"
+ id="path1938"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-5"
+ inkscape:connection-start="#rect234-0-9" />
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6"
+ width="53.328598"
+ height="17.328598"
+ x="204.0271"
+ y="71.068642"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-79.670258"
+ y="230.59494"
+ id="text290-3-2-2-28"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-2"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-79.670258"
+ y="230.59494">Model N / Stop</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.671402;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-1"
+ width="53.328598"
+ height="17.328598"
+ x="131.61899"
+ y="71.068642"
+ ry="1.4953041"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-80.196663"
+ y="158.0945"
+ id="text290-3-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-80.196663"
+ y="158.0945">Model N / Start</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 138.381,50.802895 h 19.90229 v 20.265747"
+ id="path1158"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-3"
+ inkscape:connection-end="#rect234-0-6-1" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 230.6914,88.39724 v 5.800724 H 39.3086 v 5.800724"
+ id="path1160"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6"
+ inkscape:connection-end="#rect234-0" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#fff6d5;fill-rule:evenodd;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-123.28999"
+ y="217.99152"
+ id="text392"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan390"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';fill:#000000;stroke-width:0.5"
+ x="-123.28999"
+ y="217.99152">mldev: model_ops / subtest C</tspan></text>
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.441855;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect3015"
+ width="259.55814"
+ height="119.55814"
+ x="7.720932"
+ y="7.7209282"
+ ry="1.1638433" />
+ </g>
+</svg>
diff --git a/doc/guides/tools/img/mldev_model_ops_subtest_d.svg b/doc/guides/tools/img/mldev_model_ops_subtest_d.svg
new file mode 100644
index 0000000000..3e2b89ad25
--- /dev/null
+++ b/doc/guides/tools/img/mldev_model_ops_subtest_d.svg
@@ -0,0 +1,424 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- SPDX-License-Identifier: BSD-3-Clause -->
+<!-- Copyright (c) 2022 Marvell. -->
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="275mm"
+ height="135mm"
+ viewBox="0 0 275 135"
+ version="1.1"
+ id="svg5"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="mldev_model_ops_subtest_b.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg">
+ <sodipodi:namedview
+ id="namedview7"
+ pagecolor="#ffffff"
+ bordercolor="#000000"
+ borderopacity="0.25"
+ inkscape:showpageshadow="2"
+ inkscape:pageopacity="0.0"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1"
+ inkscape:document-units="mm"
+ showgrid="false"
+ inkscape:zoom="0.72515432"
+ inkscape:cx="324.06895"
+ inkscape:cy="295.1096"
+ inkscape:window-width="1920"
+ inkscape:window-height="986"
+ inkscape:window-x="-11"
+ inkscape:window-y="-11"
+ inkscape:window-maximized="1"
+ inkscape:current-layer="layer1" />
+ <defs
+ id="defs2">
+ <marker
+ style="overflow:visible"
+ id="RoundedArrow"
+ refX="6"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="RoundedArrow"
+ markerWidth="6.1347523"
+ markerHeight="5.9304948"
+ viewBox="0 0 6.1347524 5.9304951"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.7)"
+ d="m -0.21114562,-4.1055728 6.42229122,3.21114561 a 1,1 90 0 1 0,1.78885438 L -0.21114562,4.1055728 A 1.236068,1.236068 31.717474 0 1 -2,3 v -6 a 1.236068,1.236068 148.28253 0 1 1.78885438,-1.1055728 z"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:none"
+ id="path1367" />
+ </marker>
+ </defs>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1">
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.633453;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234"
+ width="47.366547"
+ height="17.366549"
+ x="14.864025"
+ y="14.24185"
+ ry="1.4985789"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-23.388845"
+ y="38.541786"
+ id="text290"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-23.388845"
+ y="38.541786">Model 0 / Load</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.633453;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-07"
+ width="47.366547"
+ height="17.366549"
+ x="81.026939"
+ y="14.24185"
+ ry="1.4985789"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-23.388845"
+ y="104.52142"
+ id="text290-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-11"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-23.388845"
+ y="104.52142">Model 0 / Start</tspan></text>
+ <rect
+ style="fill:#fff6d5;fill-rule:evenodd;stroke:#000000;stroke-width:0.6329;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-6"
+ width="47.3671"
+ height="17.33604"
+ x="147.18958"
+ y="14.257105"
+ ry="1.4959463" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-23.388849"
+ y="170.86761"
+ id="text290-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-15"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-23.388849"
+ y="170.86761">Model 1 / Load</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 62.230572,22.925124 H 81.026939"
+ id="path1916"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234"
+ inkscape:connection-end="#rect234-07" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 128.39349,22.925124 h 18.79609"
+ id="path1918"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-07"
+ inkscape:connection-end="#rect234-6" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 194.55667,22.925125 18.79581,0"
+ id="path1922"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-6"
+ inkscape:connection-end="#rect234-0-6-6" />
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6"
+ width="47.3671"
+ height="17.33604"
+ x="213.35248"
+ y="14.257105"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-23.388849"
+ y="236.84723"
+ id="text290-3-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-23.388849"
+ y="236.84723">Model 1 / Start</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 237.03603,31.593145 0,5.328675 H 38.547297 v 5.313421"
+ id="path1924"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6"
+ inkscape:connection-end="#rect234-0-6-6-3" />
+ <rect
+ style="fill:#fff6d5;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-3"
+ width="47.3671"
+ height="17.33604"
+ x="14.863747"
+ y="42.235241"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-51.366989"
+ y="38.541786"
+ id="text290-3-2-2-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-51.366989"
+ y="38.541786">Model N / Load</tspan></text>
+ <rect
+ style="fill:#00ffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-6-2"
+ width="47.3671"
+ height="17.33604"
+ x="81.026657"
+ y="42.235241"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-51.366989"
+ y="104.52142"
+ id="text290-3-2-2-28"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-1-2"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-51.366989"
+ y="104.52142">Model N / Start</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 62.230846,50.903261 H 81.026657"
+ id="path1926"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-3"
+ inkscape:connection-end="#rect234-0-6-6-2" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 128.39376,50.903261 h 42.47937 v 19.294584"
+ id="path1928"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect234-0-6-6-2"
+ inkscape:connection-end="#rect234-0-6-1" />
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6"
+ width="47.3671"
+ height="17.33604"
+ x="213.35248"
+ y="70.197845"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-79.32959"
+ y="237.03052"
+ id="text290-3-2"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-79.32959"
+ y="237.03052">Model N / Unload</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-1"
+ width="47.3671"
+ height="17.33604"
+ x="147.18958"
+ y="70.197845"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-78.803177"
+ y="170.77666"
+ id="text290-3-2-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-8"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-78.803177"
+ y="170.77666">Model N / Stop</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 194.55667,78.865866 h 18.79581"
+ id="path1930"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6"
+ inkscape:connection-start="#rect234-0-6-1" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 237.03603,87.533886 v 5.313417 H 38.547297 v 5.328677"
+ id="path1932"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-6-7"
+ inkscape:connection-start="#rect234-0-6" />
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-6-7"
+ width="47.3671"
+ height="17.33604"
+ x="14.863747"
+ y="98.17598"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-106.78132"
+ y="38.450832"
+ id="text290-3-2-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-7-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-106.78132"
+ y="38.450832">Model 1 / Stop</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="M 62.230846,106.844 H 81.026657"
+ id="path1934"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0"
+ inkscape:connection-start="#rect234-0-6-7" />
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0"
+ width="47.3671"
+ height="17.33604"
+ x="81.026657"
+ y="98.17598"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-107.30773"
+ y="104.7047"
+ id="text290-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-107.30773"
+ y="104.7047">Model 1 / Unload</tspan></text>
+ <rect
+ style="fill:#355eff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.632899;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-9"
+ width="47.3671"
+ height="17.33604"
+ x="147.18958"
+ y="98.17598"
+ ry="1.4959463"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-106.78133"
+ y="170.77666"
+ id="text290-3-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-0"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-106.78133"
+ y="170.77666">Model 0 / Stop</tspan></text>
+ <rect
+ style="fill:#fa7137;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.633452;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect234-0-5"
+ width="47.366547"
+ height="17.366549"
+ x="213.35277"
+ y="98.160721"
+ ry="1.4985789"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-107.30773"
+ y="237.03053"
+ id="text290-3-4"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan288-1-3"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;stroke:none;stroke-width:0.75"
+ x="-107.30773"
+ y="237.03053">Model 0 / Unload</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 128.39376,106.844 h 18.79582"
+ id="path1936"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-9"
+ inkscape:connection-start="#rect234-0" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:1.5, 1.5;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#RoundedArrow)"
+ d="m 194.55667,106.844 9.39805,0 v 0 h 9.39805"
+ id="path1938"
+ inkscape:connector-type="orthogonal"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect234-0-5"
+ inkscape:connection-start="#rect234-0-9" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#fff6d5;fill-rule:evenodd;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ x="-122.00021"
+ y="222.26495"
+ id="text392"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan390"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';fill:#000000;stroke-width:0.5"
+ x="-122.00021"
+ y="222.26495">mldev: model_ops / subest B</tspan></text>
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.462341;stroke-linecap:round;stroke-linejoin:round;paint-order:stroke fill markers"
+ id="rect3475"
+ width="259.53769"
+ height="119.53766"
+ x="7.7311554"
+ y="7.7311707"
+ ry="1.2186421" />
+ </g>
+</svg>
diff --git a/doc/guides/tools/testmldev.rst b/doc/guides/tools/testmldev.rst
index 1771c49fb4..b8a2a16ca2 100644
--- a/doc/guides/tools/testmldev.rst
+++ b/doc/guides/tools/testmldev.rst
@@ -44,13 +44,17 @@ The following are the command-line options supported by the test application.
* ``--test <name>``
- Name of the test to execute. ML tests supported include device tests. Test name should be
- one of the following supported tests.
+ Name of the test to execute. ML tests are divided into two groups, Device and Model
+ tests. Test name should be one of the following supported tests.
**ML Device Tests** ::
device_ops
+ **ML Model Tests** ::
+
+ model_ops
+
* ``--dev_id <n>``
Set the device id of the ML device to be used for the test. Default value is `0`.
@@ -59,6 +63,12 @@ The following are the command-line options supported by the test application.
Set the socket id of the application resources. Default value is `SOCKET_ID_ANY`.
+* ``--models <model_list>``
+
+ Set the list of model files to be used for the tests. Application expects the
+ ``model_list`` in comma separated form (i.e. ``--models model_A.bin,model_B.bin``).
+ Maximum number of models supported by the test is ``8``.
+
* ``--debug``
Enable the tests to run in debug mode.
@@ -103,6 +113,89 @@ Command to run device_ops test:
--test=device_ops
+ML Model Tests
+-------------------------
+
+Model tests are functional tests to validate ML model APIs. Model tests validate the functioning
+of APIs to load, start, stop and unload ML models.
+
+
+Application Options
+~~~~~~~~~~~~~~~~~~~
+
+Supported command line options for the `model_ops` test are following::
+
+ --debug
+ --test
+ --dev_id
+ --socket_id
+ --models
+
+
+List of model files to be used for the `model_ops` test can be specified through the option
+``--models <model_list>`` as a comma separated list. Maximum number of models supported in
+the test is `8`.
+
+.. Note::
+
+ * The ``--models <model_list>`` is a mandatory option for running this test.
+ * Options not supported by the test are ignored if specified.
+
+
+MODEL_OPS Test
+~~~~~~~~~~~~~~
+
+The test is a collection of multiple sub-tests, each with a different order of slow-path
+operations when handling with `N` number of models.
+
+
+**Sub-test A:** executes the sequence of load / start / stop / unload for a model in order,
+followed by next model.
+.. _figure_mldev_model_ops_subtest_a:
+
+.. figure:: img/mldev_model_ops_subtest_a.*
+
+ Execution sequence of model_ops subtest A.
+
+
+**Sub-test B:** executes load for all models, followed by a start for all models. Upon successful
+start of all models, stop is invoked for all models followed by unload.
+.. _figure_mldev_model_ops_subtest_b:
+
+.. figure:: img/mldev_model_ops_subtest_b.*
+
+ Execution sequence of model_ops subtest B.
+
+
+**Sub-test C:** loads all models, followed by a start and stop of all models in order. Upon
+completion of stop, unload is invoked for all models.
+.. _figure_mldev_model_ops_subtest_c:
+
+.. figure:: img/mldev_model_ops_subtest_c.*
+
+ Execution sequence of model_ops subtest C.
+
+
+**Sub-test D:** executes load and start for all models available. Upon successful start of all
+models, stop and stop is executed for the models.
+.. _figure_mldev_model_ops_subtest_d:
+
+.. figure:: img/mldev_model_ops_subtest_d.*
+
+ Execution sequence of model_ops subtest D.
+
+
+Example
+^^^^^^^
+
+Command to run model_ops test:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=model_ops --models model_1.bin,model_2.bin,model_3.bin, model_4.bin
+
+
Debug mode
----------
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v7 05/11] app/mldev: add ordered inference test case
2023-03-16 21:14 ` [PATCH v7 00/11] Implementation of mldev test application Srikanth Yalavarthi
` (3 preceding siblings ...)
2023-03-16 21:14 ` [PATCH v7 04/11] app/mldev: add test case to validate model ops Srikanth Yalavarthi
@ 2023-03-16 21:14 ` Srikanth Yalavarthi
2023-03-16 21:14 ` [PATCH v7 06/11] app/mldev: add test case to interleave inferences Srikanth Yalavarthi
` (6 subsequent siblings)
11 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-16 21:14 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Added an ordered test case to execute inferences with single
or multiple models. In this test case inference requests for
a model are enqueued after completion of all requests for
the previous model. Test supports inference repetitions.
Operations sequence when testing with N models and R reps,
(load -> start -> (enqueue + dequeue) x R -> stop -> unload) x N
Test case can be executed by selecting "inference_ordered" test
and repetitions can be specified through "--repetitions" argument.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
Acked-by: Anup Prabhu <aprabhu@marvell.com>
---
app/test-mldev/meson.build | 2 +
app/test-mldev/ml_options.c | 65 ++
app/test-mldev/ml_options.h | 17 +-
app/test-mldev/test_inference_common.c | 567 ++++++++++++++++++
app/test-mldev/test_inference_common.h | 61 ++
app/test-mldev/test_inference_ordered.c | 115 ++++
app/test-mldev/test_model_common.h | 10 +
.../tools/img/mldev_inference_ordered.svg | 528 ++++++++++++++++
doc/guides/tools/testmldev.rst | 88 ++-
9 files changed, 1445 insertions(+), 8 deletions(-)
create mode 100644 app/test-mldev/test_inference_common.c
create mode 100644 app/test-mldev/test_inference_common.h
create mode 100644 app/test-mldev/test_inference_ordered.c
create mode 100644 doc/guides/tools/img/mldev_inference_ordered.svg
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
index b09e1ccc8a..475d76d126 100644
--- a/app/test-mldev/meson.build
+++ b/app/test-mldev/meson.build
@@ -16,6 +16,8 @@ sources = files(
'test_device_ops.c',
'test_model_common.c',
'test_model_ops.c',
+ 'test_inference_common.c',
+ 'test_inference_ordered.c',
)
deps += ['mldev']
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 8ffbab7f75..7b56bca90e 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -23,6 +23,7 @@ ml_options_default(struct ml_options *opt)
opt->dev_id = 0;
opt->socket_id = SOCKET_ID_ANY;
opt->nb_filelist = 0;
+ opt->repetitions = 1;
opt->debug = false;
}
@@ -90,6 +91,60 @@ ml_parse_models(struct ml_options *opt, const char *arg)
return ret;
}
+static int
+ml_parse_filelist(struct ml_options *opt, const char *arg)
+{
+ const char *delim = ",";
+ char filelist[PATH_MAX];
+ char *token;
+
+ if (opt->nb_filelist >= ML_TEST_MAX_MODELS) {
+ ml_err("Exceeded filelist count, max = %d\n", ML_TEST_MAX_MODELS);
+ return -1;
+ }
+
+ strlcpy(filelist, arg, PATH_MAX);
+
+ /* model */
+ token = strtok(filelist, delim);
+ if (token == NULL) {
+ ml_err("Invalid filelist, model not specified = %s\n", arg);
+ return -EINVAL;
+ }
+ strlcpy(opt->filelist[opt->nb_filelist].model, token, PATH_MAX);
+
+ /* input */
+ token = strtok(NULL, delim);
+ if (token == NULL) {
+ ml_err("Invalid filelist, input not specified = %s\n", arg);
+ return -EINVAL;
+ }
+ strlcpy(opt->filelist[opt->nb_filelist].input, token, PATH_MAX);
+
+ /* output */
+ token = strtok(NULL, delim);
+ if (token == NULL) {
+ ml_err("Invalid filelist, output not specified = %s\n", arg);
+ return -EINVAL;
+ }
+ strlcpy(opt->filelist[opt->nb_filelist].output, token, PATH_MAX);
+
+ opt->nb_filelist++;
+
+ if (opt->nb_filelist == 0) {
+ ml_err("Empty filelist. Need at least one filelist entry for the test.");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int
+ml_parse_repetitions(struct ml_options *opt, const char *arg)
+{
+ return parser_read_uint64(&opt->repetitions, arg);
+}
+
static void
ml_dump_test_options(const char *testname)
{
@@ -100,6 +155,12 @@ ml_dump_test_options(const char *testname)
printf("\t\t--models : comma separated list of models\n");
printf("\n");
}
+
+ if (strcmp(testname, "inference_ordered") == 0) {
+ printf("\t\t--filelist : comma separated list of model, input and output\n"
+ "\t\t--repetitions : number of inference repetitions\n");
+ printf("\n");
+ }
}
static void
@@ -122,6 +183,8 @@ static struct option lgopts[] = {
{ML_DEVICE_ID, 1, 0, 0},
{ML_SOCKET_ID, 1, 0, 0},
{ML_MODELS, 1, 0, 0},
+ {ML_FILELIST, 1, 0, 0},
+ {ML_REPETITIONS, 1, 0, 0},
{ML_DEBUG, 0, 0, 0},
{ML_HELP, 0, 0, 0},
{NULL, 0, 0, 0}};
@@ -136,6 +199,8 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
{ML_DEVICE_ID, ml_parse_dev_id},
{ML_SOCKET_ID, ml_parse_socket_id},
{ML_MODELS, ml_parse_models},
+ {ML_FILELIST, ml_parse_filelist},
+ {ML_REPETITIONS, ml_parse_repetitions},
};
for (i = 0; i < RTE_DIM(parsermap); i++) {
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index 61e938d2e2..6a13f97a30 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -12,15 +12,19 @@
#define ML_TEST_MAX_MODELS 8
/* Options names */
-#define ML_TEST ("test")
-#define ML_DEVICE_ID ("dev_id")
-#define ML_SOCKET_ID ("socket_id")
-#define ML_MODELS ("models")
-#define ML_DEBUG ("debug")
-#define ML_HELP ("help")
+#define ML_TEST ("test")
+#define ML_DEVICE_ID ("dev_id")
+#define ML_SOCKET_ID ("socket_id")
+#define ML_MODELS ("models")
+#define ML_FILELIST ("filelist")
+#define ML_REPETITIONS ("repetitions")
+#define ML_DEBUG ("debug")
+#define ML_HELP ("help")
struct ml_filelist {
char model[PATH_MAX];
+ char input[PATH_MAX];
+ char output[PATH_MAX];
};
struct ml_options {
@@ -29,6 +33,7 @@ struct ml_options {
int socket_id;
struct ml_filelist filelist[ML_TEST_MAX_MODELS];
uint8_t nb_filelist;
+ uint64_t repetitions;
bool debug;
};
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
new file mode 100644
index 0000000000..6a6999d524
--- /dev/null
+++ b/app/test-mldev/test_inference_common.c
@@ -0,0 +1,567 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <errno.h>
+#include <unistd.h>
+
+#include <rte_common.h>
+#include <rte_launch.h>
+#include <rte_lcore.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_mldev.h>
+
+#include "ml_common.h"
+#include "test_inference_common.h"
+
+/* Enqueue inference requests with burst size equal to 1 */
+static int
+ml_enqueue_single(void *arg)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)arg);
+ struct ml_request *req = NULL;
+ struct rte_ml_op *op = NULL;
+ struct ml_core_args *args;
+ uint64_t model_enq = 0;
+ uint32_t burst_enq;
+ uint32_t lcore_id;
+ uint16_t fid;
+ int ret;
+
+ lcore_id = rte_lcore_id();
+ args = &t->args[lcore_id];
+ model_enq = 0;
+
+ if (args->nb_reqs == 0)
+ return 0;
+
+next_rep:
+ fid = args->start_fid;
+
+next_model:
+ ret = rte_mempool_get(t->op_pool, (void **)&op);
+ if (ret != 0)
+ goto next_model;
+
+retry:
+ ret = rte_mempool_get(t->model[fid].io_pool, (void **)&req);
+ if (ret != 0)
+ goto retry;
+
+ op->model_id = t->model[fid].id;
+ op->nb_batches = t->model[fid].info.batch_size;
+ op->mempool = t->op_pool;
+
+ op->input.addr = req->input;
+ op->input.length = t->model[fid].inp_qsize;
+ op->input.next = NULL;
+
+ op->output.addr = req->output;
+ op->output.length = t->model[fid].out_qsize;
+ op->output.next = NULL;
+
+ op->user_ptr = req;
+ req->niters++;
+ req->fid = fid;
+
+enqueue_req:
+ burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, 0, &op, 1);
+ if (burst_enq == 0)
+ goto enqueue_req;
+
+ fid++;
+ if (likely(fid <= args->end_fid))
+ goto next_model;
+
+ model_enq++;
+ if (likely(model_enq < args->nb_reqs))
+ goto next_rep;
+
+ return 0;
+}
+
+/* Dequeue inference requests with burst size equal to 1 */
+static int
+ml_dequeue_single(void *arg)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)arg);
+ struct rte_ml_op_error error;
+ struct rte_ml_op *op = NULL;
+ struct ml_core_args *args;
+ struct ml_request *req;
+ uint64_t total_deq = 0;
+ uint8_t nb_filelist;
+ uint32_t burst_deq;
+ uint32_t lcore_id;
+
+ lcore_id = rte_lcore_id();
+ args = &t->args[lcore_id];
+ nb_filelist = args->end_fid - args->start_fid + 1;
+
+ if (args->nb_reqs == 0)
+ return 0;
+
+dequeue_req:
+ burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, 0, &op, 1);
+
+ if (likely(burst_deq == 1)) {
+ total_deq += burst_deq;
+ if (unlikely(op->status == RTE_ML_OP_STATUS_ERROR)) {
+ rte_ml_op_error_get(t->cmn.opt->dev_id, op, &error);
+ ml_err("error_code = 0x%" PRIx64 ", error_message = %s\n", error.errcode,
+ error.message);
+ t->error_count[lcore_id]++;
+ }
+ req = (struct ml_request *)op->user_ptr;
+ rte_mempool_put(t->model[req->fid].io_pool, req);
+ rte_mempool_put(t->op_pool, op);
+ }
+
+ if (likely(total_deq < args->nb_reqs * nb_filelist))
+ goto dequeue_req;
+
+ return 0;
+}
+
+bool
+test_inference_cap_check(struct ml_options *opt)
+{
+ struct rte_ml_dev_info dev_info;
+
+ if (!ml_test_cap_check(opt))
+ return false;
+
+ rte_ml_dev_info_get(opt->dev_id, &dev_info);
+ if (opt->nb_filelist > dev_info.max_models) {
+ ml_err("Insufficient capabilities: Filelist count exceeded device limit, count = %u (max limit = %u)",
+ opt->nb_filelist, dev_info.max_models);
+ return false;
+ }
+
+ return true;
+}
+
+int
+test_inference_opt_check(struct ml_options *opt)
+{
+ uint32_t i;
+ int ret;
+
+ /* check common opts */
+ ret = ml_test_opt_check(opt);
+ if (ret != 0)
+ return ret;
+
+ /* check file availability */
+ for (i = 0; i < opt->nb_filelist; i++) {
+ if (access(opt->filelist[i].model, F_OK) == -1) {
+ ml_err("Model file not accessible: id = %u, file = %s", i,
+ opt->filelist[i].model);
+ return -ENOENT;
+ }
+
+ if (access(opt->filelist[i].input, F_OK) == -1) {
+ ml_err("Input file not accessible: id = %u, file = %s", i,
+ opt->filelist[i].input);
+ return -ENOENT;
+ }
+ }
+
+ if (opt->repetitions == 0) {
+ ml_err("Invalid option, repetitions = %" PRIu64 "\n", opt->repetitions);
+ return -EINVAL;
+ }
+
+ /* check number of available lcores. */
+ if (rte_lcore_count() < 3) {
+ ml_err("Insufficient lcores = %u\n", rte_lcore_count());
+ ml_err("Minimum lcores required to create %u queue-pairs = %u\n", 1, 3);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+void
+test_inference_opt_dump(struct ml_options *opt)
+{
+ uint32_t i;
+
+ /* dump common opts */
+ ml_test_opt_dump(opt);
+
+ /* dump test opts */
+ ml_dump("repetitions", "%" PRIu64, opt->repetitions);
+
+ ml_dump_begin("filelist");
+ for (i = 0; i < opt->nb_filelist; i++) {
+ ml_dump_list("model", i, opt->filelist[i].model);
+ ml_dump_list("input", i, opt->filelist[i].input);
+ ml_dump_list("output", i, opt->filelist[i].output);
+ }
+ ml_dump_end;
+}
+
+int
+test_inference_setup(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t;
+ void *test_inference;
+ int ret = 0;
+ uint32_t i;
+
+ test_inference = rte_zmalloc_socket(test->name, sizeof(struct test_inference),
+ RTE_CACHE_LINE_SIZE, opt->socket_id);
+ if (test_inference == NULL) {
+ ml_err("failed to allocate memory for test_model");
+ ret = -ENOMEM;
+ goto error;
+ }
+ test->test_priv = test_inference;
+ t = ml_test_priv(test);
+
+ t->nb_used = 0;
+ t->cmn.result = ML_TEST_FAILED;
+ t->cmn.opt = opt;
+ memset(t->error_count, 0, RTE_MAX_LCORE * sizeof(uint64_t));
+
+ /* get device info */
+ ret = rte_ml_dev_info_get(opt->dev_id, &t->cmn.dev_info);
+ if (ret < 0) {
+ ml_err("failed to get device info");
+ goto error;
+ }
+
+ t->enqueue = ml_enqueue_single;
+ t->dequeue = ml_dequeue_single;
+
+ /* set model initial state */
+ for (i = 0; i < opt->nb_filelist; i++)
+ t->model[i].state = MODEL_INITIAL;
+
+ return 0;
+
+error:
+ if (test_inference != NULL)
+ rte_free(test_inference);
+
+ return ret;
+}
+
+void
+test_inference_destroy(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+ if (t != NULL)
+ rte_free(t);
+}
+
+int
+ml_inference_mldev_setup(struct ml_test *test, struct ml_options *opt)
+{
+ struct rte_ml_dev_qp_conf qp_conf;
+ struct test_inference *t;
+ int ret;
+
+ t = ml_test_priv(test);
+
+ ret = ml_test_device_configure(test, opt);
+ if (ret != 0)
+ return ret;
+
+ /* setup queue pairs */
+ qp_conf.nb_desc = t->cmn.dev_info.max_desc;
+ qp_conf.cb = NULL;
+
+ ret = rte_ml_dev_queue_pair_setup(opt->dev_id, 0, &qp_conf, opt->socket_id);
+ if (ret != 0) {
+ ml_err("Failed to setup ml device queue-pair, dev_id = %d, qp_id = %u\n",
+ opt->dev_id, 0);
+ goto error;
+ }
+
+ ret = ml_test_device_start(test, opt);
+ if (ret != 0)
+ goto error;
+
+ return 0;
+
+error:
+ ml_test_device_close(test, opt);
+
+ return ret;
+}
+
+int
+ml_inference_mldev_destroy(struct ml_test *test, struct ml_options *opt)
+{
+ int ret;
+
+ ret = ml_test_device_stop(test, opt);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_test_device_close(test, opt);
+ if (ret != 0)
+ return ret;
+
+ return 0;
+
+error:
+ ml_test_device_close(test, opt);
+
+ return ret;
+}
+
+/* Callback for IO pool create. This function would compute the fields of ml_request
+ * structure and prepare the quantized input data.
+ */
+static void
+ml_request_initialize(struct rte_mempool *mp, void *opaque, void *obj, unsigned int obj_idx)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)opaque);
+ struct ml_request *req = (struct ml_request *)obj;
+
+ RTE_SET_USED(mp);
+ RTE_SET_USED(obj_idx);
+
+ req->input = (uint8_t *)obj +
+ RTE_ALIGN_CEIL(sizeof(struct ml_request), t->cmn.dev_info.min_align_size);
+ req->output = req->input +
+ RTE_ALIGN_CEIL(t->model[t->fid].inp_qsize, t->cmn.dev_info.min_align_size);
+ req->niters = 0;
+
+ /* quantize data */
+ rte_ml_io_quantize(t->cmn.opt->dev_id, t->model[t->fid].id,
+ t->model[t->fid].info.batch_size, t->model[t->fid].input, req->input);
+}
+
+int
+ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, uint16_t fid)
+{
+ struct test_inference *t = ml_test_priv(test);
+ char mz_name[RTE_MEMZONE_NAMESIZE];
+ char mp_name[RTE_MEMPOOL_NAMESIZE];
+ const struct rte_memzone *mz;
+ uint64_t nb_buffers;
+ uint32_t buff_size;
+ uint32_t mz_size;
+ uint32_t fsize;
+ FILE *fp;
+ int ret;
+
+ /* get input buffer size */
+ ret = rte_ml_io_input_size_get(opt->dev_id, t->model[fid].id, t->model[fid].info.batch_size,
+ &t->model[fid].inp_qsize, &t->model[fid].inp_dsize);
+ if (ret != 0) {
+ ml_err("Failed to get input size, model : %s\n", opt->filelist[fid].model);
+ return ret;
+ }
+
+ /* get output buffer size */
+ ret = rte_ml_io_output_size_get(opt->dev_id, t->model[fid].id,
+ t->model[fid].info.batch_size, &t->model[fid].out_qsize,
+ &t->model[fid].out_dsize);
+ if (ret != 0) {
+ ml_err("Failed to get input size, model : %s\n", opt->filelist[fid].model);
+ return ret;
+ }
+
+ /* allocate buffer for user data */
+ mz_size = t->model[fid].inp_dsize + t->model[fid].out_dsize;
+ sprintf(mz_name, "ml_user_data_%d", fid);
+ mz = rte_memzone_reserve(mz_name, mz_size, opt->socket_id, 0);
+ if (mz == NULL) {
+ ml_err("Memzone allocation failed for ml_user_data\n");
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ t->model[fid].input = mz->addr;
+ t->model[fid].output = t->model[fid].input + t->model[fid].inp_dsize;
+
+ /* load input file */
+ fp = fopen(opt->filelist[fid].input, "r");
+ if (fp == NULL) {
+ ml_err("Failed to open input file : %s\n", opt->filelist[fid].input);
+ ret = -errno;
+ goto error;
+ }
+
+ fseek(fp, 0, SEEK_END);
+ fsize = ftell(fp);
+ fseek(fp, 0, SEEK_SET);
+ if (fsize != t->model[fid].inp_dsize) {
+ ml_err("Invalid input file, size = %u (expected size = %" PRIu64 ")\n", fsize,
+ t->model[fid].inp_dsize);
+ ret = -EINVAL;
+ fclose(fp);
+ goto error;
+ }
+
+ if (fread(t->model[fid].input, 1, t->model[fid].inp_dsize, fp) != t->model[fid].inp_dsize) {
+ ml_err("Failed to read input file : %s\n", opt->filelist[fid].input);
+ ret = -errno;
+ fclose(fp);
+ goto error;
+ }
+ fclose(fp);
+
+ /* create mempool for quantized input and output buffers. ml_request_initialize is
+ * used as a callback for object creation.
+ */
+ buff_size = RTE_ALIGN_CEIL(sizeof(struct ml_request), t->cmn.dev_info.min_align_size) +
+ RTE_ALIGN_CEIL(t->model[fid].inp_qsize, t->cmn.dev_info.min_align_size) +
+ RTE_ALIGN_CEIL(t->model[fid].out_qsize, t->cmn.dev_info.min_align_size);
+ nb_buffers = RTE_MIN((uint64_t)ML_TEST_MAX_POOL_SIZE, opt->repetitions);
+
+ t->fid = fid;
+ sprintf(mp_name, "ml_io_pool_%d", fid);
+ t->model[fid].io_pool = rte_mempool_create(mp_name, nb_buffers, buff_size, 0, 0, NULL, NULL,
+ ml_request_initialize, test, opt->socket_id, 0);
+ if (t->model[fid].io_pool == NULL) {
+ ml_err("Failed to create io pool : %s\n", "ml_io_pool");
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ return 0;
+
+error:
+ if (mz != NULL)
+ rte_memzone_free(mz);
+
+ if (t->model[fid].io_pool != NULL) {
+ rte_mempool_free(t->model[fid].io_pool);
+ t->model[fid].io_pool = NULL;
+ }
+
+ return ret;
+}
+
+void
+ml_inference_iomem_destroy(struct ml_test *test, struct ml_options *opt, uint16_t fid)
+{
+ char mz_name[RTE_MEMZONE_NAMESIZE];
+ char mp_name[RTE_MEMPOOL_NAMESIZE];
+ const struct rte_memzone *mz;
+ struct rte_mempool *mp;
+
+ RTE_SET_USED(test);
+ RTE_SET_USED(opt);
+
+ /* release user data memzone */
+ sprintf(mz_name, "ml_user_data_%d", fid);
+ mz = rte_memzone_lookup(mz_name);
+ if (mz != NULL)
+ rte_memzone_free(mz);
+
+ /* destroy io pool */
+ sprintf(mp_name, "ml_io_pool_%d", fid);
+ mp = rte_mempool_lookup(mp_name);
+ if (mp != NULL)
+ rte_mempool_free(mp);
+}
+
+int
+ml_inference_mem_setup(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t = ml_test_priv(test);
+
+ /* create op pool */
+ t->op_pool = rte_ml_op_pool_create("ml_test_op_pool", ML_TEST_MAX_POOL_SIZE, 0, 0,
+ opt->socket_id);
+ if (t->op_pool == NULL) {
+ ml_err("Failed to create op pool : %s\n", "ml_op_pool");
+ return -ENOMEM;
+ }
+
+ return 0;
+}
+
+void
+ml_inference_mem_destroy(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t = ml_test_priv(test);
+
+ RTE_SET_USED(opt);
+
+ /* release op pool */
+ if (t->op_pool != NULL)
+ rte_mempool_free(t->op_pool);
+}
+
+/* Callback for mempool object iteration. This call would dequantize output data. */
+static void
+ml_request_finish(struct rte_mempool *mp, void *opaque, void *obj, unsigned int obj_idx)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)opaque);
+ struct ml_request *req = (struct ml_request *)obj;
+ struct ml_model *model = &t->model[req->fid];
+
+ RTE_SET_USED(mp);
+ RTE_SET_USED(obj_idx);
+
+ if (req->niters == 0)
+ return;
+
+ t->nb_used++;
+ rte_ml_io_dequantize(t->cmn.opt->dev_id, model->id, t->model[req->fid].info.batch_size,
+ req->output, model->output);
+}
+
+int
+ml_inference_result(struct ml_test *test, struct ml_options *opt, uint16_t fid)
+{
+ struct test_inference *t = ml_test_priv(test);
+ uint64_t error_count = 0;
+ uint32_t i;
+
+ RTE_SET_USED(opt);
+
+ /* check for errors */
+ for (i = 0; i < RTE_MAX_LCORE; i++)
+ error_count += t->error_count[i];
+
+ rte_mempool_obj_iter(t->model[fid].io_pool, ml_request_finish, test);
+
+ if ((t->nb_used > 0) && (error_count == 0))
+ t->cmn.result = ML_TEST_SUCCESS;
+ else
+ t->cmn.result = ML_TEST_FAILED;
+
+ return t->cmn.result;
+}
+
+int
+ml_inference_launch_cores(struct ml_test *test, struct ml_options *opt, uint16_t start_fid,
+ uint16_t end_fid)
+{
+ struct test_inference *t = ml_test_priv(test);
+ uint32_t lcore_id;
+ uint32_t id = 0;
+
+ RTE_LCORE_FOREACH_WORKER(lcore_id)
+ {
+ if (id == 2)
+ break;
+
+ t->args[lcore_id].nb_reqs = opt->repetitions;
+ t->args[lcore_id].start_fid = start_fid;
+ t->args[lcore_id].end_fid = end_fid;
+
+ if (id % 2 == 0)
+ rte_eal_remote_launch(t->enqueue, test, lcore_id);
+ else
+ rte_eal_remote_launch(t->dequeue, test, lcore_id);
+
+ id++;
+ }
+
+ return 0;
+}
diff --git a/app/test-mldev/test_inference_common.h b/app/test-mldev/test_inference_common.h
new file mode 100644
index 0000000000..abb20fc9fb
--- /dev/null
+++ b/app/test-mldev/test_inference_common.h
@@ -0,0 +1,61 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#ifndef _ML_TEST_INFERENCE_COMMON_
+#define _ML_TEST_INFERENCE_COMMON_
+
+#include <rte_common.h>
+#include <rte_mempool.h>
+#include <rte_mldev.h>
+
+#include "test_model_common.h"
+
+struct ml_request {
+ uint8_t *input;
+ uint8_t *output;
+ uint16_t fid;
+ uint64_t niters;
+};
+
+struct ml_core_args {
+ uint64_t nb_reqs;
+ uint16_t start_fid;
+ uint16_t end_fid;
+};
+
+struct test_inference {
+ /* common data */
+ struct test_common cmn;
+
+ /* test specific data */
+ struct ml_model model[ML_TEST_MAX_MODELS];
+ struct rte_mempool *op_pool;
+
+ uint64_t nb_used;
+ uint16_t fid;
+
+ int (*enqueue)(void *arg);
+ int (*dequeue)(void *arg);
+
+ struct ml_core_args args[RTE_MAX_LCORE];
+ uint64_t error_count[RTE_MAX_LCORE];
+} __rte_cache_aligned;
+
+bool test_inference_cap_check(struct ml_options *opt);
+int test_inference_opt_check(struct ml_options *opt);
+void test_inference_opt_dump(struct ml_options *opt);
+int test_inference_setup(struct ml_test *test, struct ml_options *opt);
+void test_inference_destroy(struct ml_test *test, struct ml_options *opt);
+
+int ml_inference_mldev_setup(struct ml_test *test, struct ml_options *opt);
+int ml_inference_mldev_destroy(struct ml_test *test, struct ml_options *opt);
+int ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, uint16_t fid);
+void ml_inference_iomem_destroy(struct ml_test *test, struct ml_options *opt, uint16_t fid);
+int ml_inference_mem_setup(struct ml_test *test, struct ml_options *opt);
+void ml_inference_mem_destroy(struct ml_test *test, struct ml_options *opt);
+int ml_inference_result(struct ml_test *test, struct ml_options *opt, uint16_t fid);
+int ml_inference_launch_cores(struct ml_test *test, struct ml_options *opt, uint16_t start_fid,
+ uint16_t end_fid);
+
+#endif /* _ML_TEST_INFERENCE_COMMON_ */
diff --git a/app/test-mldev/test_inference_ordered.c b/app/test-mldev/test_inference_ordered.c
new file mode 100644
index 0000000000..1cd91dc3d3
--- /dev/null
+++ b/app/test-mldev/test_inference_ordered.c
@@ -0,0 +1,115 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <rte_common.h>
+#include <rte_launch.h>
+
+#include "ml_common.h"
+#include "test_inference_common.h"
+
+static int
+test_inference_ordered_driver(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t;
+ uint16_t fid = 0;
+ int ret = 0;
+
+ t = ml_test_priv(test);
+
+ ret = ml_inference_mldev_setup(test, opt);
+ if (ret != 0)
+ return ret;
+
+ ret = ml_inference_mem_setup(test, opt);
+ if (ret != 0)
+ return ret;
+
+next_model:
+ /* load model */
+ ret = ml_model_load(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ /* start model */
+ ret = ml_model_start(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_inference_iomem_setup(test, opt, fid);
+ if (ret != 0)
+ goto error;
+
+ /* launch inferences for one model using available queue pairs */
+ ret = ml_inference_launch_cores(test, opt, fid, fid);
+ if (ret != 0) {
+ ml_err("failed to launch cores");
+ goto error;
+ }
+
+ rte_eal_mp_wait_lcore();
+
+ ret = ml_inference_result(test, opt, fid);
+ if (ret != ML_TEST_SUCCESS)
+ goto error;
+
+ ml_inference_iomem_destroy(test, opt, fid);
+
+ /* stop model */
+ ret = ml_model_stop(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ /* unload model */
+ ret = ml_model_unload(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ fid++;
+ if (fid < opt->nb_filelist)
+ goto next_model;
+
+ ml_inference_mem_destroy(test, opt);
+
+ ret = ml_inference_mldev_destroy(test, opt);
+ if (ret != 0)
+ return ret;
+
+ t->cmn.result = ML_TEST_SUCCESS;
+
+ return 0;
+
+error:
+ ml_inference_iomem_destroy(test, opt, fid);
+ ml_inference_mem_destroy(test, opt);
+ ml_model_stop(test, opt, &t->model[fid], fid);
+ ml_model_unload(test, opt, &t->model[fid], fid);
+
+ t->cmn.result = ML_TEST_FAILED;
+
+ return ret;
+}
+
+static int
+test_inference_ordered_result(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+
+ return t->cmn.result;
+}
+
+static const struct ml_test_ops inference_ordered = {
+ .cap_check = test_inference_cap_check,
+ .opt_check = test_inference_opt_check,
+ .opt_dump = test_inference_opt_dump,
+ .test_setup = test_inference_setup,
+ .test_destroy = test_inference_destroy,
+ .test_driver = test_inference_ordered_driver,
+ .test_result = test_inference_ordered_result,
+};
+
+ML_TEST_REGISTER(inference_ordered);
diff --git a/app/test-mldev/test_model_common.h b/app/test-mldev/test_model_common.h
index 74aec0a797..5ee975109d 100644
--- a/app/test-mldev/test_model_common.h
+++ b/app/test-mldev/test_model_common.h
@@ -20,6 +20,16 @@ struct ml_model {
uint16_t id;
struct rte_ml_model_info info;
enum model_state state;
+
+ uint64_t inp_dsize;
+ uint64_t inp_qsize;
+ uint64_t out_dsize;
+ uint64_t out_qsize;
+
+ uint8_t *input;
+ uint8_t *output;
+
+ struct rte_mempool *io_pool;
};
int ml_model_load(struct ml_test *test, struct ml_options *opt, struct ml_model *model,
diff --git a/doc/guides/tools/img/mldev_inference_ordered.svg b/doc/guides/tools/img/mldev_inference_ordered.svg
new file mode 100644
index 0000000000..12fa6acaec
--- /dev/null
+++ b/doc/guides/tools/img/mldev_inference_ordered.svg
@@ -0,0 +1,528 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- SPDX-License-Identifier: BSD-3-Clause -->
+<!-- Copyright (c) 2022 Marvell. -->
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="243mm"
+ height="144mm"
+ viewBox="0 0 243 144"
+ version="1.1"
+ id="svg5"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="inference_ordered.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns:xlink="http://www.w3.org/1999/xlink"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg">
+ <sodipodi:namedview
+ id="namedview7"
+ pagecolor="#ffffff"
+ bordercolor="#000000"
+ borderopacity="0.25"
+ inkscape:showpageshadow="2"
+ inkscape:pageopacity="0.0"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1"
+ inkscape:document-units="mm"
+ showgrid="false"
+ inkscape:zoom="0.74564394"
+ inkscape:cx="488.83922"
+ inkscape:cy="234.69647"
+ inkscape:window-width="1920"
+ inkscape:window-height="1017"
+ inkscape:window-x="1912"
+ inkscape:window-y="-8"
+ inkscape:window-maximized="1"
+ inkscape:current-layer="layer1" />
+ <defs
+ id="defs2">
+ <marker
+ style="overflow:visible"
+ id="RoundedArrow"
+ refX="5"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="RoundedArrow"
+ markerWidth="6.1347523"
+ markerHeight="5.9304948"
+ viewBox="0 0 6.1347524 5.9304951"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.7)"
+ d="m -0.21114562,-4.1055728 6.42229122,3.21114561 a 1,1 90 0 1 0,1.78885438 L -0.21114562,4.1055728 A 1.236068,1.236068 31.717474 0 1 -2,3 v -6 a 1.236068,1.236068 148.28253 0 1 1.78885438,-1.1055728 z"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:none"
+ id="path1367" />
+ </marker>
+ <linearGradient
+ inkscape:collect="always"
+ id="linearGradient31002">
+ <stop
+ style="stop-color:#fff6d5;stop-opacity:1;"
+ offset="0"
+ id="stop30998" />
+ <stop
+ style="stop-color:#fff6d5;stop-opacity:0;"
+ offset="1"
+ id="stop31000" />
+ </linearGradient>
+ <marker
+ style="overflow:visible"
+ id="TriangleStart"
+ refX="4"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="TriangleStart"
+ markerWidth="5.3244081"
+ markerHeight="6.155385"
+ viewBox="0 0 5.3244081 6.1553851"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.5)"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:context-stroke;stroke-width:1pt"
+ d="M 5.77,0 -2.88,5 V -5 Z"
+ id="path135" />
+ </marker>
+ <linearGradient
+ inkscape:collect="always"
+ xlink:href="#linearGradient31002"
+ id="linearGradient31004"
+ x1="19.620968"
+ y1="102.90323"
+ x2="279.1532"
+ y2="102.90323"
+ gradientUnits="userSpaceOnUse"
+ gradientTransform="matrix(0.89215122,0,0,0.73190287,13.449912,42.668706)" />
+ </defs>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1">
+ <g
+ id="g1340"
+ transform="translate(-25.225796,-45.983871)">
+ <rect
+ style="fill:url(#linearGradient31004);fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.404032;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect27876"
+ width="231.09595"
+ height="132.45081"
+ x="31.177822"
+ y="51.758469"
+ ry="3.5071263" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1813"
+ width="38.408459"
+ height="45.86002"
+ x="34.901794"
+ y="99.14959"
+ ry="5.2246051"
+ inkscape:connector-avoid="true" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 73.310253,115.94935 36.498807,-11.6509"
+ id="path1906"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1813"
+ inkscape:connection-end="#rect1724-0" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 73.310253,122.0796 36.117817,1e-5"
+ id="path1908"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1813"
+ inkscape:connection-end="#rect1724-4-8" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="M 73.310253,128.20983 109.80905,139.8607"
+ id="path1910"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1813"
+ inkscape:connection-end="#rect1724-6-7" />
+ <path
+ style="display:inline;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 141.71839,99.266314 19.42262,-10e-7"
+ id="path1912"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1724-0"
+ inkscape:connection-end="#rect1679" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 197.14101,99.266313 19.42259,10e-7"
+ id="path1914"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1679"
+ inkscape:connection-end="#rect1724" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 141.71839,122.07961 19.42262,-1e-5"
+ id="path1916"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1724-4-8"
+ inkscape:connection-end="#rect1679-4" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 197.14101,122.0796 19.42259,1e-5"
+ id="path1918"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1679-4"
+ inkscape:connection-end="#rect1724-4" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 141.71839,144.89282 19.42262,0"
+ id="path1920"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1724-6-7"
+ inkscape:connection-end="#rect1679-8" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 197.14101,144.89282 19.42259,0"
+ id="path1922"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1679-8"
+ inkscape:connection-end="#rect1724-6" />
+ <text
+ xml:space="preserve"
+ style="font-size:8.46667px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-121.09793"
+ y="54.031597"
+ id="text4093"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4091"
+ style="font-size:8.46667px;stroke-width:0.75"
+ x="-121.09793"
+ y="54.031597">Model X</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:6.35px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-118.63563"
+ y="179.13635"
+ id="text4097"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4095"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-118.63563"
+ y="179.13635">Queue</tspan><tspan
+ sodipodi:role="line"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-126.57313"
+ y="179.13635"
+ id="tspan4099">Pair 1</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:6.35px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-141.44887"
+ y="179.13635"
+ id="text4097-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4095-6"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-141.44887"
+ y="179.13635">Queue</tspan><tspan
+ sodipodi:role="line"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-149.38637"
+ y="179.13635"
+ id="tspan4099-4">Pair 2</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.354849;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.709699, 0.709699;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924"
+ width="44.145252"
+ height="72.532341"
+ x="157.06865"
+ y="85.813438"
+ ry="4.31247" />
+ <g
+ id="g1224">
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1679"
+ width="36"
+ height="18"
+ x="161.14101"
+ y="90.266312"
+ ry="3" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1679-8"
+ width="36"
+ height="18"
+ x="161.14101"
+ y="135.89282"
+ ry="3" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1679-4"
+ width="36"
+ height="18"
+ x="161.14101"
+ y="113.07959"
+ ry="3" />
+ </g>
+ <text
+ xml:space="preserve"
+ style="font-size:6.35px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-95.820801"
+ y="179.13635"
+ id="text4097-8"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4095-4"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-95.820801"
+ y="179.13635">Queue</tspan><tspan
+ sodipodi:role="line"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-103.7583"
+ y="179.13635"
+ id="tspan4099-5">Pair 0</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.317648;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1811"
+ width="44.196934"
+ height="16.731901"
+ x="157.04254"
+ y="56.49292"
+ ry="2.761292" />
+ <text
+ xml:space="preserve"
+ style="font-size:3.5859px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.317649;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.952945, 0.317649;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-60.009941"
+ y="186.38451"
+ id="text4156"
+ transform="matrix(0,-1.040508,0.96106903,0,0,0)"><tspan
+ sodipodi:role="line"
+ id="tspan4154"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-family:Arial;-inkscape-font-specification:'Arial Bold';stroke-width:0.317648"
+ x="-60.009941"
+ y="186.38451">Machine Learning</tspan><tspan
+ sodipodi:role="line"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-family:Arial;-inkscape-font-specification:'Arial Bold';stroke-width:0.317648"
+ x="-64.492317"
+ y="186.38451"
+ id="tspan4158">Hardware Engine</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-99.705231"
+ y="125.91087"
+ id="text3708"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-99.705231"
+ y="125.91087">lcore 1</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-145.3221"
+ y="125.50572"
+ id="text3708-8"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-7"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-145.3221"
+ y="125.50572">lcore 5</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-122.51577"
+ y="125.52089"
+ id="text3708-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-87"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-122.51577"
+ y="125.52089">lcore 3</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-162.06549"
+ y="125.4589"
+ id="text4542"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4540"
+ style="font-size:5.64444px;stroke-width:0.75"
+ x="-162.06549"
+ y="125.4589">Enqueue Workers</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-99.705231"
+ y="232.67706"
+ id="text3708-9"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-9"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-99.705231"
+ y="232.67706">lcore 2</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-122.51025"
+ y="232.66466"
+ id="text3708-7"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-8"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-122.51025"
+ y="232.66466">lcore 4</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-145.33035"
+ y="232.65778"
+ id="text3708-78"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-0"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-145.33035"
+ y="232.65778">lcore 6</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-162.06549"
+ y="232.59988"
+ id="text4542-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4540-7"
+ style="font-size:5.64444px;stroke-width:0.75"
+ x="-162.06549"
+ y="232.59988">Dequeue Workers</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:6.35px;font-family:Arial;-inkscape-font-specification:'Arial Bold';text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-177.01665"
+ y="220.07283"
+ id="text5181"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan5179"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-177.01665"
+ y="220.07283">test: inference_ordered</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-weight:bold;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:'Arial Bold';text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-178.63324"
+ y="98.67057"
+ id="text15571"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan15569"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.5;stroke-opacity:1"
+ x="-178.63324"
+ y="98.67057">nb_worker_threads = 2 * MIN(nb_queue_pairs, (lcore_count - 1) / 2)</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-weight:bold;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:'Arial Bold';text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-171.18085"
+ y="89.26754"
+ id="text15571-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan15569-9"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.5;stroke-opacity:1"
+ x="-171.18085"
+ y="89.26754">inferences_per_queue_pair = repetitions / nb_queue_pairs</tspan></text>
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.75;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#RoundedArrow);marker-end:url(#RoundedArrow)"
+ d="m 179.14101,85.813438 0,-12.588618"
+ id="path31090"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1924" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724"
+ width="32.290321"
+ height="11.709678"
+ x="216.5636"
+ y="93.411476"
+ ry="2" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-4"
+ width="32.290321"
+ height="11.709678"
+ x="216.5636"
+ y="116.22478"
+ ry="2" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-6"
+ width="32.290321"
+ height="11.709678"
+ x="216.5636"
+ y="139.03798"
+ ry="2" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.354849;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.709699, 0.709699;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924-6"
+ width="44.145252"
+ height="72.532341"
+ x="210.6364"
+ y="85.813477"
+ ry="4.31247" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-0"
+ width="32.290321"
+ height="11.709678"
+ x="109.42807"
+ y="93.411476"
+ ry="2" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-6-7"
+ width="32.290321"
+ height="11.709678"
+ x="109.42807"
+ y="139.03798"
+ ry="2"
+ inkscape:connector-avoid="true" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-4-8"
+ width="32.290321"
+ height="11.709678"
+ x="109.42807"
+ y="116.22478"
+ ry="2"
+ inkscape:connector-avoid="true" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.354849;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.709699, 0.709699;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924-6-3"
+ width="44.145252"
+ height="72.532341"
+ x="103.50092"
+ y="85.813477"
+ ry="4.31247" />
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/tools/testmldev.rst b/doc/guides/tools/testmldev.rst
index b8a2a16ca2..164fbca64f 100644
--- a/doc/guides/tools/testmldev.rst
+++ b/doc/guides/tools/testmldev.rst
@@ -44,8 +44,8 @@ The following are the command-line options supported by the test application.
* ``--test <name>``
- Name of the test to execute. ML tests are divided into two groups, Device and Model
- tests. Test name should be one of the following supported tests.
+ Name of the test to execute. ML tests are divided into three groups, Device, Model
+ and Inference tests. Test name should be one of the following supported tests.
**ML Device Tests** ::
@@ -55,6 +55,10 @@ The following are the command-line options supported by the test application.
model_ops
+ **ML Inference Tests** ::
+
+ inference_ordered
+
* ``--dev_id <n>``
Set the device id of the ML device to be used for the test. Default value is `0`.
@@ -69,6 +73,23 @@ The following are the command-line options supported by the test application.
``model_list`` in comma separated form (i.e. ``--models model_A.bin,model_B.bin``).
Maximum number of models supported by the test is ``8``.
+* ``--filelist <file_list>``
+
+ Set the list of model, input, output and reference files to be used for the tests.
+ Application expects the ``file_list`` to be in comma separated form
+ (i.e. ``--filelist <model,input,output>[,reference]``).
+
+ Multiple filelist entries can be specified when running the tests with multiple models.
+ Both quantized and dequantized outputs are written to the disk. Dequantized output file
+ would have the name specified by the user through ``--filelist`` option. A suffix ``.q``
+ is appended to quantized output filename. Maximum number of filelist entries supported
+ by the test is ``8``.
+
+* ``--repetitions <n>``
+
+ Set the number of inference repetitions to be executed in the test per each model. Default
+ value is `1`.
+
* ``--debug``
Enable the tests to run in debug mode.
@@ -196,6 +217,69 @@ Command to run model_ops test:
--test=model_ops --models model_1.bin,model_2.bin,model_3.bin, model_4.bin
+ML Inference Tests
+------------------
+
+Inference tests are a set of tests to validate end-to-end inference execution on ML device.
+These tests executes the full sequence of operations required to run inferences with one or
+multiple models.
+
+Application Options
+~~~~~~~~~~~~~~~~~~~
+
+Supported command line options for inference tests are following::
+
+ --debug
+ --test
+ --dev_id
+ --socket_id
+ --filelist
+ --repetitions
+
+
+List of files to be used for the inference tests can be specified through the option
+``--filelist <file_list>`` as a comma separated list. A filelist entry would be of the format
+``--filelist <model_file,input_file,output_file>[,reference_file]`` and is used to specify the
+list of files required to test with a single model. Multiple filelist entries are supported by
+the test, one entry per model. Maximum number of file entries supported by the test is `8`.
+
+.. Note::
+
+ * The ``--filelist <file_list>`` is a mandatory option for running inference tests.
+ * Options not supported by the tests are ignored if specified.
+
+
+INFERENCE_ORDERED Test
+~~~~~~~~~~~~~~~~~~~~~~
+
+This is a functional test for validating the end-to-end inference execution on ML device. This
+test configures ML device and queue pairs as per the queue-pair related options (queue_pairs and
+queue_size) specified by the user. Upon successful configuration of the device and queue pairs,
+the first model specified through the filelist is loaded to the device and inferences are enqueued
+by a pool of worker threads to the ML device. Total number of inferences enqueued for the model
+are equal to the repetitions specified. A dedicated pool of worker threads would dequeue the
+inferences from the device. The model is unloaded upon completion of all inferences for the model.
+The test would continue loading and executing inference requests for all models specified
+through ``filelist`` option in an ordered manner.
+
+.. _figure_mldev_inference_ordered:
+
+.. figure:: img/mldev_inference_ordered.*
+
+ Execution of inference_ordered on single model.
+
+
+Example
+^^^^^^^
+
+Example command to run inference_ordered test:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_ordered --filelist model.bin,input.bin,output.bin
+
+
Debug mode
----------
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v7 06/11] app/mldev: add test case to interleave inferences
2023-03-16 21:14 ` [PATCH v7 00/11] Implementation of mldev test application Srikanth Yalavarthi
` (4 preceding siblings ...)
2023-03-16 21:14 ` [PATCH v7 05/11] app/mldev: add ordered inference test case Srikanth Yalavarthi
@ 2023-03-16 21:14 ` Srikanth Yalavarthi
2023-03-16 21:14 ` [PATCH v7 07/11] app/mldev: enable support for burst inferences Srikanth Yalavarthi
` (5 subsequent siblings)
11 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-16 21:14 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Added test case to interleave inference requests from multiple
models. Interleaving would load and start all models and launch
inference requests for the models using available queue-pairs
Operations sequence when testing with N models and R reps,
(load + start) x N -> (enqueue + dequeue) x N x R ...
-> (stop + unload) x N
Test can be executed by selecting "inference_interleave" test.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
Acked-by: Anup Prabhu <aprabhu@marvell.com>
---
app/test-mldev/meson.build | 1 +
app/test-mldev/ml_options.c | 3 +-
app/test-mldev/test_inference_interleave.c | 114 +++
.../tools/img/mldev_inference_interleave.svg | 669 ++++++++++++++++++
doc/guides/tools/testmldev.rst | 41 ++
5 files changed, 827 insertions(+), 1 deletion(-)
create mode 100644 app/test-mldev/test_inference_interleave.c
create mode 100644 doc/guides/tools/img/mldev_inference_interleave.svg
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
index 475d76d126..41d22fb22c 100644
--- a/app/test-mldev/meson.build
+++ b/app/test-mldev/meson.build
@@ -18,6 +18,7 @@ sources = files(
'test_model_ops.c',
'test_inference_common.c',
'test_inference_ordered.c',
+ 'test_inference_interleave.c',
)
deps += ['mldev']
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 7b56bca90e..649cb9d8d1 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -156,7 +156,8 @@ ml_dump_test_options(const char *testname)
printf("\n");
}
- if (strcmp(testname, "inference_ordered") == 0) {
+ if ((strcmp(testname, "inference_ordered") == 0) ||
+ (strcmp(testname, "inference_interleave") == 0)) {
printf("\t\t--filelist : comma separated list of model, input and output\n"
"\t\t--repetitions : number of inference repetitions\n");
printf("\n");
diff --git a/app/test-mldev/test_inference_interleave.c b/app/test-mldev/test_inference_interleave.c
new file mode 100644
index 0000000000..9cf4cfa197
--- /dev/null
+++ b/app/test-mldev/test_inference_interleave.c
@@ -0,0 +1,114 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 Marvell.
+ */
+
+#include <rte_common.h>
+#include <rte_launch.h>
+
+#include "ml_common.h"
+#include "test_inference_common.h"
+
+static int
+test_inference_interleave_driver(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t;
+ uint16_t fid = 0;
+ int ret = 0;
+
+ t = ml_test_priv(test);
+
+ ret = ml_inference_mldev_setup(test, opt);
+ if (ret != 0)
+ return ret;
+
+ ret = ml_inference_mem_setup(test, opt);
+ if (ret != 0)
+ return ret;
+
+ /* load and start all models */
+ for (fid = 0; fid < opt->nb_filelist; fid++) {
+ ret = ml_model_load(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_start(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_inference_iomem_setup(test, opt, fid);
+ if (ret != 0)
+ goto error;
+ }
+
+ /* launch inference requests */
+ ret = ml_inference_launch_cores(test, opt, 0, opt->nb_filelist - 1);
+ if (ret != 0) {
+ ml_err("failed to launch cores");
+ goto error;
+ }
+
+ rte_eal_mp_wait_lcore();
+
+ /* stop and unload all models */
+ for (fid = 0; fid < opt->nb_filelist; fid++) {
+ ret = ml_inference_result(test, opt, fid);
+ if (ret != ML_TEST_SUCCESS)
+ goto error;
+
+ ml_inference_iomem_destroy(test, opt, fid);
+
+ ret = ml_model_stop(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+
+ ret = ml_model_unload(test, opt, &t->model[fid], fid);
+ if (ret != 0)
+ goto error;
+ }
+
+ ml_inference_mem_destroy(test, opt);
+
+ ret = ml_inference_mldev_destroy(test, opt);
+ if (ret != 0)
+ return ret;
+
+ t->cmn.result = ML_TEST_SUCCESS;
+
+ return 0;
+
+error:
+ ml_inference_mem_destroy(test, opt);
+ for (fid = 0; fid < opt->nb_filelist; fid++) {
+ ml_inference_iomem_destroy(test, opt, fid);
+ ml_model_stop(test, opt, &t->model[fid], fid);
+ ml_model_unload(test, opt, &t->model[fid], fid);
+ }
+
+ t->cmn.result = ML_TEST_FAILED;
+
+ return ret;
+}
+
+static int
+test_inference_interleave_result(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t;
+
+ RTE_SET_USED(opt);
+
+ t = ml_test_priv(test);
+
+ return t->cmn.result;
+}
+
+static const struct ml_test_ops inference_interleave = {
+ .cap_check = test_inference_cap_check,
+ .opt_check = test_inference_opt_check,
+ .opt_dump = test_inference_opt_dump,
+ .test_setup = test_inference_setup,
+ .test_destroy = test_inference_destroy,
+ .test_driver = test_inference_interleave_driver,
+ .test_result = test_inference_interleave_result,
+};
+
+ML_TEST_REGISTER(inference_interleave);
diff --git a/doc/guides/tools/img/mldev_inference_interleave.svg b/doc/guides/tools/img/mldev_inference_interleave.svg
new file mode 100644
index 0000000000..3a741ea627
--- /dev/null
+++ b/doc/guides/tools/img/mldev_inference_interleave.svg
@@ -0,0 +1,669 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- SPDX-License-Identifier: BSD-3-Clause -->
+<!-- Copyright (c) 2022 Marvell. -->
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+ width="224mm"
+ height="150mm"
+ viewBox="0 0 224 150"
+ version="1.1"
+ id="svg5369"
+ inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
+ sodipodi:docname="inference_interleave.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns:xlink="http://www.w3.org/1999/xlink"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg">
+ <sodipodi:namedview
+ id="namedview5371"
+ pagecolor="#ffffff"
+ bordercolor="#000000"
+ borderopacity="0.25"
+ inkscape:showpageshadow="2"
+ inkscape:pageopacity="0.0"
+ inkscape:pagecheckerboard="0"
+ inkscape:deskcolor="#d1d1d1"
+ inkscape:document-units="mm"
+ showgrid="false"
+ inkscape:zoom="0.74564394"
+ inkscape:cx="415.07747"
+ inkscape:cy="348.6919"
+ inkscape:window-width="1920"
+ inkscape:window-height="1017"
+ inkscape:window-x="1912"
+ inkscape:window-y="-8"
+ inkscape:window-maximized="1"
+ inkscape:current-layer="layer1" />
+ <defs
+ id="defs5366">
+ <linearGradient
+ inkscape:collect="always"
+ id="linearGradient19189">
+ <stop
+ style="stop-color:#ffeeaa;stop-opacity:1;"
+ offset="0"
+ id="stop19185" />
+ <stop
+ style="stop-color:#ffeeaa;stop-opacity:0;"
+ offset="1"
+ id="stop19187" />
+ </linearGradient>
+ <marker
+ style="overflow:visible"
+ id="TriangleStart"
+ refX="4"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="TriangleStart"
+ markerWidth="5.3244081"
+ markerHeight="6.155385"
+ viewBox="0 0 5.3244081 6.1553851"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.5)"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:context-stroke;stroke-width:1pt"
+ d="M 5.77,0 -2.88,5 V -5 Z"
+ id="path135" />
+ </marker>
+ <marker
+ style="overflow:visible"
+ id="RoundedArrow"
+ refX="5"
+ refY="0"
+ orient="auto-start-reverse"
+ inkscape:stockid="RoundedArrow"
+ markerWidth="6.1347523"
+ markerHeight="5.9304948"
+ viewBox="0 0 6.1347524 5.9304951"
+ inkscape:isstock="true"
+ inkscape:collect="always"
+ preserveAspectRatio="xMidYMid">
+ <path
+ transform="scale(0.7)"
+ d="m -0.21114562,-4.1055728 6.42229122,3.21114561 a 1,1 90 0 1 0,1.78885438 L -0.21114562,4.1055728 A 1.236068,1.236068 31.717474 0 1 -2,3 v -6 a 1.236068,1.236068 148.28253 0 1 1.78885438,-1.1055728 z"
+ style="fill:context-stroke;fill-rule:evenodd;stroke:none"
+ id="path1367" />
+ </marker>
+ <linearGradient
+ inkscape:collect="always"
+ xlink:href="#linearGradient19189"
+ id="linearGradient19191"
+ x1="12.169352"
+ y1="105"
+ x2="284.83066"
+ y2="105"
+ gradientUnits="userSpaceOnUse"
+ gradientTransform="matrix(0.79055445,0,0,0.74078976,29.505892,28.991272)" />
+ </defs>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1">
+ <g
+ id="g1477"
+ transform="translate(-34.903236,-31.774189)">
+ <rect
+ style="fill:url(#linearGradient19191);fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.396267;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect16635"
+ width="215.18147"
+ height="139.60078"
+ x="39.3125"
+ y="36.973797"
+ ry="2.2354064" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-169.3954"
+ y="214.97237"
+ id="text5181"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan5179"
+ style="font-size:5.64444px;stroke-width:0.75"
+ x="-169.3954"
+ y="214.97237">test: inference_interleave</tspan></text>
+ <path
+ style="display:inline;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 138.05133,87.886263 17.45982,-10e-7"
+ id="path1912"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1724-0"
+ inkscape:connection-end="#rect1679" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 191.51115,87.886262 17.45981,10e-7"
+ id="path1914"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1679"
+ inkscape:connection-end="#rect1724" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 138.05133,109.78102 17.45982,-1e-5"
+ id="path1916"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1724-4-8"
+ inkscape:connection-end="#rect1679-4" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 191.51115,109.78101 17.45981,1e-5"
+ id="path1918"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1679-4"
+ inkscape:connection-end="#rect1724-4" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 138.05133,131.67576 17.45982,-1e-5"
+ id="path1920"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1724-6-7"
+ inkscape:connection-end="#rect1679-8" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 191.51115,131.67575 17.45981,1e-5"
+ id="path1922"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1679-8"
+ inkscape:connection-end="#rect1724-6" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.75;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;marker-start:url(#RoundedArrow);marker-end:url(#RoundedArrow)"
+ d="m 173.51116,60.08164 0,12.907336"
+ id="path1933"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect1811"
+ inkscape:connection-end="#rect1924" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.368668;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.737336, 0.737336;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924"
+ width="46.97015"
+ height="73.58287"
+ x="150.02565"
+ y="72.988976"
+ ry="2.4685853" />
+ <text
+ xml:space="preserve"
+ style="font-size:6.35px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-84.44075"
+ y="173.5065"
+ id="text4097"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4095"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-84.44075"
+ y="173.5065">Queue</tspan><tspan
+ sodipodi:role="line"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-92.37825"
+ y="173.5065"
+ id="tspan4099">Pair 0</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:6.35px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-128.2318"
+ y="173.5065"
+ id="text4097-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4095-6"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-128.2318"
+ y="173.5065">Queue</tspan><tspan
+ sodipodi:role="line"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-136.1693"
+ y="173.5065"
+ id="tspan4099-4">Pair 2</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1679"
+ width="36"
+ height="18"
+ x="155.51115"
+ y="78.886261"
+ ry="3" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1679-8"
+ width="36"
+ height="18"
+ x="155.51115"
+ y="122.67575"
+ ry="3" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1679-4"
+ width="36"
+ height="18"
+ x="155.51115"
+ y="100.78101"
+ ry="3" />
+ <text
+ xml:space="preserve"
+ style="font-size:6.35px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-106.33705"
+ y="173.5065"
+ id="text4097-8"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4095-4"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-106.33705"
+ y="173.5065">Queue</tspan><tspan
+ sodipodi:role="line"
+ style="font-size:6.35px;stroke-width:0.75"
+ x="-114.27455"
+ y="173.5065"
+ id="tspan4099-5">Pair 1</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.388863;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1811"
+ width="58.909527"
+ height="18.812746"
+ x="144.0564"
+ y="41.268894"
+ ry="2.2255962" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-47.017281"
+ y="173.49187"
+ id="text4156"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4154"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';stroke-width:0.75"
+ x="-47.017281"
+ y="173.49187">Machine Learning</tspan><tspan
+ sodipodi:role="line"
+ style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444px;font-family:Arial;-inkscape-font-specification:'Arial Bold';stroke-width:0.75"
+ x="-54.07283"
+ y="173.49187"
+ id="tspan4158">Hardware Engine</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.368668;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.737336, 0.737336;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924-0"
+ width="46.97015"
+ height="73.58287"
+ x="98.42067"
+ y="72.988976"
+ ry="2.4685853" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-88.32518"
+ y="122.24379"
+ id="text3708"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-88.32518"
+ y="122.24379">lcore 1</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-132.10504"
+ y="121.83865"
+ id="text3708-8"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-7"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-132.10504"
+ y="121.83865">lcore 5</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-0"
+ width="32.290321"
+ height="11.709678"
+ x="105.76101"
+ y="82.031425"
+ ry="3.0161259" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-6-7"
+ width="32.290321"
+ height="11.709678"
+ x="105.76101"
+ y="125.82092"
+ ry="3.0161259"
+ inkscape:connector-avoid="true" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff8500;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-4-8"
+ width="32.290321"
+ height="11.709678"
+ x="105.76101"
+ y="103.92618"
+ ry="3.0161259"
+ inkscape:connector-avoid="true" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-110.21718"
+ y="121.85381"
+ id="text3708-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-87"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-110.21718"
+ y="121.85381">lcore 3</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-150.82878"
+ y="121.79179"
+ id="text4542"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4540"
+ style="font-size:5.64444px;stroke-width:0.75"
+ x="-150.82878"
+ y="121.79179">Enqueue Workers</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.368668;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.737336, 0.737336;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924-7"
+ width="46.97015"
+ height="73.58287"
+ x="201.63062"
+ y="72.988976"
+ ry="2.4685853" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-88.32518"
+ y="225.08443"
+ id="text3708-9"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-9"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-88.32518"
+ y="225.08443">lcore 2</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-110.21167"
+ y="225.07202"
+ id="text3708-7"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-8"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-110.21167"
+ y="225.07202">lcore 4</tspan></text>
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724"
+ width="32.290321"
+ height="11.709678"
+ x="208.97096"
+ y="82.031425"
+ ry="3.0161259" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-4"
+ width="32.290321"
+ height="11.709678"
+ x="208.97096"
+ y="103.92618"
+ ry="3.0161259" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00fb00;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1724-6"
+ width="32.290321"
+ height="11.709678"
+ x="208.97096"
+ y="125.82092"
+ ry="3.0161259" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-132.1133"
+ y="225.06514"
+ id="text3708-78"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-0"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-132.1133"
+ y="225.06514">lcore 6</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-150.82878"
+ y="225.00725"
+ id="text4542-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan4540-7"
+ style="font-size:5.64444px;stroke-width:0.75"
+ x="-150.82878"
+ y="225.00725">Dequeue Workers</tspan></text>
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 79.368074,78.962117 26.440446,5.546991"
+ id="path6677"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-0"
+ inkscape:connection-start="#rect6252" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="M 78.978262,84.050781 112.13805,103.92618"
+ id="path6679"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-4-8"
+ inkscape:connection-start="#rect6252" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 73.959073,84.25738 42.026977,41.56354"
+ id="path6681"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-6-7"
+ inkscape:connection-start="#rect6252" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="M 79.368074,95.959838 105.76913,90.949016"
+ id="path6683"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-0"
+ inkscape:connection-start="#rect6252-8" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 79.368074,101.28215 26.416716,5.27791"
+ id="path7830"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-4-8"
+ inkscape:connection-start="#rect6252-8" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 79.069199,106.4283 32.903161,19.39262"
+ id="path7862"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-6-7"
+ inkscape:connection-start="#rect6252-8" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="M 79.069201,113.13371 111.97235,93.741103"
+ id="path7900"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-0"
+ inkscape:connection-start="#rect6252-2" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 79.368074,118.27987 26.416716,-5.2779"
+ id="path7932"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-4-8"
+ inkscape:connection-start="#rect6252-2" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 79.368074,123.60218 26.401056,5.01083"
+ id="path7998"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-6-7"
+ inkscape:connection-start="#rect6252-2" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="M 73.959074,135.30464 115.98605,93.741103"
+ id="path8000"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-0"
+ inkscape:connection-start="#rect6252-6" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="M 78.978264,135.51124 112.13805,115.63586"
+ id="path8002"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-4-8"
+ inkscape:connection-start="#rect6252-6" />
+ <path
+ style="display:inline;fill:none;fill-rule:evenodd;stroke:#5d36ff;stroke-width:0.5;stroke-linecap:butt;stroke-linejoin:miter;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleStart)"
+ d="m 79.368074,140.59991 26.440446,-5.54699"
+ id="path8004"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-end="#rect1724-6-7"
+ inkscape:connection-start="#rect6252-6" />
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-76.53363"
+ y="65.63237"
+ id="text3708-84"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-4"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-76.53363"
+ y="65.63237">Model 0</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-98.991623"
+ y="66.015465"
+ id="text3708-84-1"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-4-6"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-98.991623"
+ y="66.015465">Model 1</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-121.44823"
+ y="65.646149"
+ id="text3708-84-9"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-4-1"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-121.44823"
+ y="65.646149">Model 2</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-size:5.64444px;font-family:Arial;-inkscape-font-specification:Arial;text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.25, 0.750001;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-143.9021"
+ y="65.625481"
+ id="text3708-84-5"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan3706-4-8"
+ style="font-size:5.64444px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75"
+ x="-143.9021"
+ y="65.625481">Model 3</tspan></text>
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect6252"
+ width="27.32258"
+ height="16.32258"
+ x="52.045494"
+ y="67.934799"
+ ry="1" />
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect6252-8"
+ width="27.32258"
+ height="16.32258"
+ x="52.045494"
+ y="90.391411"
+ ry="1" />
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect6252-2"
+ width="27.32258"
+ height="16.32258"
+ x="52.045494"
+ y="112.84802"
+ ry="1" />
+ <rect
+ style="fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.750001;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect6252-6"
+ width="27.32258"
+ height="16.32258"
+ x="52.045494"
+ y="135.30464"
+ ry="1" />
+ <rect
+ style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#00a6fb;stroke-width:0.399886;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:0.799773, 0.799773;stroke-dashoffset:0;stroke-opacity:1;paint-order:stroke fill markers"
+ id="rect1924-1-3"
+ width="43.035683"
+ height="94.487152"
+ x="44.188942"
+ y="62.536991"
+ ry="3.1694498" />
+ <g
+ id="g1010"
+ transform="translate(21.464467,-15.875004)">
+ <text
+ xml:space="preserve"
+ style="font-weight:bold;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:'Arial Bold';text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-185.88483"
+ y="86.542366"
+ id="text15571"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan15569"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.5;stroke-opacity:1"
+ x="-185.88483"
+ y="86.542366">nb_worker_threads = 2 * MIN(nb_queue_pairs, (lcore_count - 1) / 2)</tspan></text>
+ <text
+ xml:space="preserve"
+ style="font-weight:bold;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:'Arial Bold';text-align:center;writing-mode:tb-rl;text-anchor:middle;fill:none;fill-rule:evenodd;stroke:#00d7fb;stroke-width:0.499999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke fill markers"
+ x="-178.43243"
+ y="90.903854"
+ id="text15571-3"
+ transform="rotate(-90)"><tspan
+ sodipodi:role="line"
+ id="tspan15569-9"
+ style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:4.23333px;font-family:Arial;-inkscape-font-specification:Arial;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.5;stroke-opacity:1"
+ x="-178.43243"
+ y="90.903854">inferences_per_queue_pair = nb_models * (repetitions / nb_queue_pairs)</tspan></text>
+ </g>
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/tools/testmldev.rst b/doc/guides/tools/testmldev.rst
index 164fbca64f..1a1ab7d2bf 100644
--- a/doc/guides/tools/testmldev.rst
+++ b/doc/guides/tools/testmldev.rst
@@ -58,6 +58,7 @@ The following are the command-line options supported by the test application.
**ML Inference Tests** ::
inference_ordered
+ inference_interleave
* ``--dev_id <n>``
@@ -280,6 +281,46 @@ Example command to run inference_ordered test:
--test=inference_ordered --filelist model.bin,input.bin,output.bin
+INFERENCE_INTERLEAVE Test
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This is a stress test for validating the end-to-end inference execution on ML device. The test
+configures the ML device and queue pairs as per the queue-pair related options (queue_pairs
+and queue_size) specified by the user. Upon successful configuration of the device and queue
+pairs, all models specified through the filelist are loaded to the device. Inferences for multiple
+models are enqueued by a pool of worker threads in parallel. Inference execution by the device is
+interleaved between multiple models. Total number of inferences enqueued for a model are equal to
+the repetitions specified. An additional pool of threads would dequeue the inferences from the
+device. Models would be unloaded upon completion of inferences for all models loaded.
+
+
+.. _figure_mldev_inference_interleave:
+
+.. figure:: img/mldev_inference_interleave.*
+
+ Execution of inference_interleave on single model.
+
+
+Example
+^^^^^^^
+
+Example command to run inference_interleave test:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_interleave --filelist model.bin,input.bin,output.bin
+
+
+Example command to run inference_interleave test with multiple models:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_interleave --filelist model_A.bin,input_A.bin,output_A.bin \
+ --filelist model_B.bin,input_B.bin,output_B.bin
+
+
Debug mode
----------
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v7 07/11] app/mldev: enable support for burst inferences
2023-03-16 21:14 ` [PATCH v7 00/11] Implementation of mldev test application Srikanth Yalavarthi
` (5 preceding siblings ...)
2023-03-16 21:14 ` [PATCH v7 06/11] app/mldev: add test case to interleave inferences Srikanth Yalavarthi
@ 2023-03-16 21:14 ` Srikanth Yalavarthi
2023-03-16 21:14 ` [PATCH v7 08/11] app/mldev: enable support for queue pairs and size Srikanth Yalavarthi
` (4 subsequent siblings)
11 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-16 21:14 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Added 'burst_size' support for inference tests. Burst size
controls the number of inference requests handled during
the burst enqueue and dequeue operations of the test case.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
Acked-by: Anup Prabhu <aprabhu@marvell.com>
---
app/test-mldev/ml_options.c | 12 +-
app/test-mldev/ml_options.h | 2 +
app/test-mldev/test_inference_common.c | 160 ++++++++++++++++++++++++-
app/test-mldev/test_inference_common.h | 4 +
doc/guides/tools/testmldev.rst | 24 ++++
5 files changed, 199 insertions(+), 3 deletions(-)
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 649cb9d8d1..19f2e1279e 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -24,6 +24,7 @@ ml_options_default(struct ml_options *opt)
opt->socket_id = SOCKET_ID_ANY;
opt->nb_filelist = 0;
opt->repetitions = 1;
+ opt->burst_size = 1;
opt->debug = false;
}
@@ -145,6 +146,12 @@ ml_parse_repetitions(struct ml_options *opt, const char *arg)
return parser_read_uint64(&opt->repetitions, arg);
}
+static int
+ml_parse_burst_size(struct ml_options *opt, const char *arg)
+{
+ return parser_read_uint16(&opt->burst_size, arg);
+}
+
static void
ml_dump_test_options(const char *testname)
{
@@ -159,7 +166,8 @@ ml_dump_test_options(const char *testname)
if ((strcmp(testname, "inference_ordered") == 0) ||
(strcmp(testname, "inference_interleave") == 0)) {
printf("\t\t--filelist : comma separated list of model, input and output\n"
- "\t\t--repetitions : number of inference repetitions\n");
+ "\t\t--repetitions : number of inference repetitions\n"
+ "\t\t--burst_size : inference burst size\n");
printf("\n");
}
}
@@ -186,6 +194,7 @@ static struct option lgopts[] = {
{ML_MODELS, 1, 0, 0},
{ML_FILELIST, 1, 0, 0},
{ML_REPETITIONS, 1, 0, 0},
+ {ML_BURST_SIZE, 1, 0, 0},
{ML_DEBUG, 0, 0, 0},
{ML_HELP, 0, 0, 0},
{NULL, 0, 0, 0}};
@@ -202,6 +211,7 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
{ML_MODELS, ml_parse_models},
{ML_FILELIST, ml_parse_filelist},
{ML_REPETITIONS, ml_parse_repetitions},
+ {ML_BURST_SIZE, ml_parse_burst_size},
};
for (i = 0; i < RTE_DIM(parsermap); i++) {
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index 6a13f97a30..00342d8a0c 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -18,6 +18,7 @@
#define ML_MODELS ("models")
#define ML_FILELIST ("filelist")
#define ML_REPETITIONS ("repetitions")
+#define ML_BURST_SIZE ("burst_size")
#define ML_DEBUG ("debug")
#define ML_HELP ("help")
@@ -34,6 +35,7 @@ struct ml_options {
struct ml_filelist filelist[ML_TEST_MAX_MODELS];
uint8_t nb_filelist;
uint64_t repetitions;
+ uint16_t burst_size;
bool debug;
};
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
index 6a6999d524..35323306de 100644
--- a/app/test-mldev/test_inference_common.c
+++ b/app/test-mldev/test_inference_common.c
@@ -124,6 +124,132 @@ ml_dequeue_single(void *arg)
return 0;
}
+/* Enqueue inference requests with burst size greater than 1 */
+static int
+ml_enqueue_burst(void *arg)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)arg);
+ struct ml_core_args *args;
+ uint16_t ops_count;
+ uint64_t model_enq;
+ uint16_t burst_enq;
+ uint32_t lcore_id;
+ uint16_t pending;
+ uint16_t idx;
+ uint16_t fid;
+ uint16_t i;
+ int ret;
+
+ lcore_id = rte_lcore_id();
+ args = &t->args[lcore_id];
+ model_enq = 0;
+
+ if (args->nb_reqs == 0)
+ return 0;
+
+next_rep:
+ fid = args->start_fid;
+
+next_model:
+ ops_count = RTE_MIN(t->cmn.opt->burst_size, args->nb_reqs - model_enq);
+ ret = rte_mempool_get_bulk(t->op_pool, (void **)args->enq_ops, ops_count);
+ if (ret != 0)
+ goto next_model;
+
+retry:
+ ret = rte_mempool_get_bulk(t->model[fid].io_pool, (void **)args->reqs, ops_count);
+ if (ret != 0)
+ goto retry;
+
+ for (i = 0; i < ops_count; i++) {
+ args->enq_ops[i]->model_id = t->model[fid].id;
+ args->enq_ops[i]->nb_batches = t->model[fid].info.batch_size;
+ args->enq_ops[i]->mempool = t->op_pool;
+
+ args->enq_ops[i]->input.addr = args->reqs[i]->input;
+ args->enq_ops[i]->input.length = t->model[fid].inp_qsize;
+ args->enq_ops[i]->input.next = NULL;
+
+ args->enq_ops[i]->output.addr = args->reqs[i]->output;
+ args->enq_ops[i]->output.length = t->model[fid].out_qsize;
+ args->enq_ops[i]->output.next = NULL;
+
+ args->enq_ops[i]->user_ptr = args->reqs[i];
+ args->reqs[i]->niters++;
+ args->reqs[i]->fid = fid;
+ }
+
+ idx = 0;
+ pending = ops_count;
+
+enqueue_reqs:
+ burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, 0, &args->enq_ops[idx], pending);
+ pending = pending - burst_enq;
+
+ if (pending > 0) {
+ idx = idx + burst_enq;
+ goto enqueue_reqs;
+ }
+
+ fid++;
+ if (fid <= args->end_fid)
+ goto next_model;
+
+ model_enq = model_enq + ops_count;
+ if (model_enq < args->nb_reqs)
+ goto next_rep;
+
+ return 0;
+}
+
+/* Dequeue inference requests with burst size greater than 1 */
+static int
+ml_dequeue_burst(void *arg)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)arg);
+ struct rte_ml_op_error error;
+ struct ml_core_args *args;
+ struct ml_request *req;
+ uint64_t total_deq = 0;
+ uint16_t burst_deq = 0;
+ uint8_t nb_filelist;
+ uint32_t lcore_id;
+ uint32_t i;
+
+ lcore_id = rte_lcore_id();
+ args = &t->args[lcore_id];
+ nb_filelist = args->end_fid - args->start_fid + 1;
+
+ if (args->nb_reqs == 0)
+ return 0;
+
+dequeue_burst:
+ burst_deq =
+ rte_ml_dequeue_burst(t->cmn.opt->dev_id, 0, args->deq_ops, t->cmn.opt->burst_size);
+
+ if (likely(burst_deq > 0)) {
+ total_deq += burst_deq;
+
+ for (i = 0; i < burst_deq; i++) {
+ if (unlikely(args->deq_ops[i]->status == RTE_ML_OP_STATUS_ERROR)) {
+ rte_ml_op_error_get(t->cmn.opt->dev_id, args->deq_ops[i], &error);
+ ml_err("error_code = 0x%" PRIx64 ", error_message = %s\n",
+ error.errcode, error.message);
+ t->error_count[lcore_id]++;
+ }
+ req = (struct ml_request *)args->deq_ops[i]->user_ptr;
+ if (req != NULL)
+ rte_mempool_put(t->model[req->fid].io_pool, req);
+ }
+ rte_mempool_put_bulk(t->op_pool, (void *)args->deq_ops, burst_deq);
+ }
+
+ if (total_deq < args->nb_reqs * nb_filelist)
+ goto dequeue_burst;
+
+ return 0;
+}
+
bool
test_inference_cap_check(struct ml_options *opt)
{
@@ -173,6 +299,17 @@ test_inference_opt_check(struct ml_options *opt)
return -EINVAL;
}
+ if (opt->burst_size == 0) {
+ ml_err("Invalid option, burst_size = %u\n", opt->burst_size);
+ return -EINVAL;
+ }
+
+ if (opt->burst_size > ML_TEST_MAX_POOL_SIZE) {
+ ml_err("Invalid option, burst_size = %u (> max supported = %d)\n", opt->burst_size,
+ ML_TEST_MAX_POOL_SIZE);
+ return -EINVAL;
+ }
+
/* check number of available lcores. */
if (rte_lcore_count() < 3) {
ml_err("Insufficient lcores = %u\n", rte_lcore_count());
@@ -193,6 +330,7 @@ test_inference_opt_dump(struct ml_options *opt)
/* dump test opts */
ml_dump("repetitions", "%" PRIu64, opt->repetitions);
+ ml_dump("burst_size", "%u", opt->burst_size);
ml_dump_begin("filelist");
for (i = 0; i < opt->nb_filelist; i++) {
@@ -208,6 +346,7 @@ test_inference_setup(struct ml_test *test, struct ml_options *opt)
{
struct test_inference *t;
void *test_inference;
+ uint32_t lcore_id;
int ret = 0;
uint32_t i;
@@ -233,13 +372,30 @@ test_inference_setup(struct ml_test *test, struct ml_options *opt)
goto error;
}
- t->enqueue = ml_enqueue_single;
- t->dequeue = ml_dequeue_single;
+ if (opt->burst_size == 1) {
+ t->enqueue = ml_enqueue_single;
+ t->dequeue = ml_dequeue_single;
+ } else {
+ t->enqueue = ml_enqueue_burst;
+ t->dequeue = ml_dequeue_burst;
+ }
/* set model initial state */
for (i = 0; i < opt->nb_filelist; i++)
t->model[i].state = MODEL_INITIAL;
+ for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) {
+ t->args[lcore_id].enq_ops = rte_zmalloc_socket(
+ "ml_test_enq_ops", opt->burst_size * sizeof(struct rte_ml_op *),
+ RTE_CACHE_LINE_SIZE, opt->socket_id);
+ t->args[lcore_id].deq_ops = rte_zmalloc_socket(
+ "ml_test_deq_ops", opt->burst_size * sizeof(struct rte_ml_op *),
+ RTE_CACHE_LINE_SIZE, opt->socket_id);
+ t->args[lcore_id].reqs = rte_zmalloc_socket(
+ "ml_test_requests", opt->burst_size * sizeof(struct ml_request *),
+ RTE_CACHE_LINE_SIZE, opt->socket_id);
+ }
+
return 0;
error:
diff --git a/app/test-mldev/test_inference_common.h b/app/test-mldev/test_inference_common.h
index abb20fc9fb..da800f2bd4 100644
--- a/app/test-mldev/test_inference_common.h
+++ b/app/test-mldev/test_inference_common.h
@@ -22,6 +22,10 @@ struct ml_core_args {
uint64_t nb_reqs;
uint16_t start_fid;
uint16_t end_fid;
+
+ struct rte_ml_op **enq_ops;
+ struct rte_ml_op **deq_ops;
+ struct ml_request **reqs;
};
struct test_inference {
diff --git a/doc/guides/tools/testmldev.rst b/doc/guides/tools/testmldev.rst
index 1a1ab7d2bf..eb9081723b 100644
--- a/doc/guides/tools/testmldev.rst
+++ b/doc/guides/tools/testmldev.rst
@@ -91,6 +91,10 @@ The following are the command-line options supported by the test application.
Set the number of inference repetitions to be executed in the test per each model. Default
value is `1`.
+* ``--burst_size <n>``
+
+ Set the burst size to be used when enqueuing / dequeuing inferences. Default value is `1`.
+
* ``--debug``
Enable the tests to run in debug mode.
@@ -236,6 +240,7 @@ Supported command line options for inference tests are following::
--socket_id
--filelist
--repetitions
+ --burst_size
List of files to be used for the inference tests can be specified through the option
@@ -244,6 +249,9 @@ List of files to be used for the inference tests can be specified through the op
list of files required to test with a single model. Multiple filelist entries are supported by
the test, one entry per model. Maximum number of file entries supported by the test is `8`.
+When ``--burst_size <num>`` option is specified for the test, enqueue and dequeue burst would
+try to enqueue or dequeue ``num`` number of inferences per each call respectively.
+
.. Note::
* The ``--filelist <file_list>`` is a mandatory option for running inference tests.
@@ -280,6 +288,14 @@ Example command to run inference_ordered test:
sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
--test=inference_ordered --filelist model.bin,input.bin,output.bin
+Example command to run inference_ordered test with a specific burst size:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_ordered --filelist model.bin,input.bin,output.bin \
+ --burst_size 12
+
INFERENCE_INTERLEAVE Test
~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -320,6 +336,14 @@ Example command to run inference_interleave test with multiple models:
--test=inference_interleave --filelist model_A.bin,input_A.bin,output_A.bin \
--filelist model_B.bin,input_B.bin,output_B.bin
+Example command to run inference_interleave test with a specific burst size:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_interleave --filelist model.bin,input.bin,output.bin \
+ --burst_size 16
+
Debug mode
----------
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v7 08/11] app/mldev: enable support for queue pairs and size
2023-03-16 21:14 ` [PATCH v7 00/11] Implementation of mldev test application Srikanth Yalavarthi
` (6 preceding siblings ...)
2023-03-16 21:14 ` [PATCH v7 07/11] app/mldev: enable support for burst inferences Srikanth Yalavarthi
@ 2023-03-16 21:14 ` Srikanth Yalavarthi
2023-03-16 21:14 ` [PATCH v7 09/11] app/mldev: enable support for inference batches Srikanth Yalavarthi
` (3 subsequent siblings)
11 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-16 21:14 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Added support to create multiple queue-pairs per device to
enqueue and dequeue inference requests. Number of queue pairs
to be created can be specified through "--queue_pairs" option.
Support is also enabled to control the number of descriptors
per each queue pair through "--queue_size" option. Inference
requests for a model are distributed across all available
queue-pairs.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
Acked-by: Anup Prabhu <aprabhu@marvell.com>
---
app/test-mldev/ml_options.c | 31 +++++++++-
app/test-mldev/ml_options.h | 4 ++
app/test-mldev/test_common.c | 2 +-
app/test-mldev/test_inference_common.c | 79 +++++++++++++++++++++-----
app/test-mldev/test_inference_common.h | 1 +
doc/guides/tools/testmldev.rst | 44 +++++++++++++-
6 files changed, 140 insertions(+), 21 deletions(-)
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 19f2e1279e..901adaed33 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -25,6 +25,8 @@ ml_options_default(struct ml_options *opt)
opt->nb_filelist = 0;
opt->repetitions = 1;
opt->burst_size = 1;
+ opt->queue_pairs = 1;
+ opt->queue_size = 1;
opt->debug = false;
}
@@ -152,11 +154,30 @@ ml_parse_burst_size(struct ml_options *opt, const char *arg)
return parser_read_uint16(&opt->burst_size, arg);
}
+static int
+ml_parse_queue_pairs(struct ml_options *opt, const char *arg)
+{
+ int ret;
+
+ ret = parser_read_uint16(&opt->queue_pairs, arg);
+
+ return ret;
+}
+
+static int
+ml_parse_queue_size(struct ml_options *opt, const char *arg)
+{
+ return parser_read_uint16(&opt->queue_size, arg);
+}
+
static void
ml_dump_test_options(const char *testname)
{
- if (strcmp(testname, "device_ops") == 0)
+ if (strcmp(testname, "device_ops") == 0) {
+ printf("\t\t--queue_pairs : number of queue pairs to create\n"
+ "\t\t--queue_size : size fo queue-pair\n");
printf("\n");
+ }
if (strcmp(testname, "model_ops") == 0) {
printf("\t\t--models : comma separated list of models\n");
@@ -167,7 +188,9 @@ ml_dump_test_options(const char *testname)
(strcmp(testname, "inference_interleave") == 0)) {
printf("\t\t--filelist : comma separated list of model, input and output\n"
"\t\t--repetitions : number of inference repetitions\n"
- "\t\t--burst_size : inference burst size\n");
+ "\t\t--burst_size : inference burst size\n"
+ "\t\t--queue_pairs : number of queue pairs to create\n"
+ "\t\t--queue_size : size fo queue-pair\n");
printf("\n");
}
}
@@ -195,6 +218,8 @@ static struct option lgopts[] = {
{ML_FILELIST, 1, 0, 0},
{ML_REPETITIONS, 1, 0, 0},
{ML_BURST_SIZE, 1, 0, 0},
+ {ML_QUEUE_PAIRS, 1, 0, 0},
+ {ML_QUEUE_SIZE, 1, 0, 0},
{ML_DEBUG, 0, 0, 0},
{ML_HELP, 0, 0, 0},
{NULL, 0, 0, 0}};
@@ -212,6 +237,8 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
{ML_FILELIST, ml_parse_filelist},
{ML_REPETITIONS, ml_parse_repetitions},
{ML_BURST_SIZE, ml_parse_burst_size},
+ {ML_QUEUE_PAIRS, ml_parse_queue_pairs},
+ {ML_QUEUE_SIZE, ml_parse_queue_size},
};
for (i = 0; i < RTE_DIM(parsermap); i++) {
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index 00342d8a0c..c4018ee9d1 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -19,6 +19,8 @@
#define ML_FILELIST ("filelist")
#define ML_REPETITIONS ("repetitions")
#define ML_BURST_SIZE ("burst_size")
+#define ML_QUEUE_PAIRS ("queue_pairs")
+#define ML_QUEUE_SIZE ("queue_size")
#define ML_DEBUG ("debug")
#define ML_HELP ("help")
@@ -36,6 +38,8 @@ struct ml_options {
uint8_t nb_filelist;
uint64_t repetitions;
uint16_t burst_size;
+ uint16_t queue_pairs;
+ uint16_t queue_size;
bool debug;
};
diff --git a/app/test-mldev/test_common.c b/app/test-mldev/test_common.c
index 8c4da4609a..016b31c6ba 100644
--- a/app/test-mldev/test_common.c
+++ b/app/test-mldev/test_common.c
@@ -75,7 +75,7 @@ ml_test_device_configure(struct ml_test *test, struct ml_options *opt)
/* configure device */
dev_config.socket_id = opt->socket_id;
dev_config.nb_models = t->dev_info.max_models;
- dev_config.nb_queue_pairs = t->dev_info.max_queue_pairs;
+ dev_config.nb_queue_pairs = opt->queue_pairs;
ret = rte_ml_dev_configure(opt->dev_id, &dev_config);
if (ret != 0) {
ml_err("Failed to configure ml device, dev_id = %d\n", opt->dev_id);
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
index 35323306de..b4ad3c4b72 100644
--- a/app/test-mldev/test_inference_common.c
+++ b/app/test-mldev/test_inference_common.c
@@ -66,7 +66,7 @@ ml_enqueue_single(void *arg)
req->fid = fid;
enqueue_req:
- burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, 0, &op, 1);
+ burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, args->qp_id, &op, 1);
if (burst_enq == 0)
goto enqueue_req;
@@ -103,7 +103,7 @@ ml_dequeue_single(void *arg)
return 0;
dequeue_req:
- burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, 0, &op, 1);
+ burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, args->qp_id, &op, 1);
if (likely(burst_deq == 1)) {
total_deq += burst_deq;
@@ -183,7 +183,8 @@ ml_enqueue_burst(void *arg)
pending = ops_count;
enqueue_reqs:
- burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, 0, &args->enq_ops[idx], pending);
+ burst_enq =
+ rte_ml_enqueue_burst(t->cmn.opt->dev_id, args->qp_id, &args->enq_ops[idx], pending);
pending = pending - burst_enq;
if (pending > 0) {
@@ -224,8 +225,8 @@ ml_dequeue_burst(void *arg)
return 0;
dequeue_burst:
- burst_deq =
- rte_ml_dequeue_burst(t->cmn.opt->dev_id, 0, args->deq_ops, t->cmn.opt->burst_size);
+ burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, args->qp_id, args->deq_ops,
+ t->cmn.opt->burst_size);
if (likely(burst_deq > 0)) {
total_deq += burst_deq;
@@ -259,6 +260,19 @@ test_inference_cap_check(struct ml_options *opt)
return false;
rte_ml_dev_info_get(opt->dev_id, &dev_info);
+
+ if (opt->queue_pairs > dev_info.max_queue_pairs) {
+ ml_err("Insufficient capabilities: queue_pairs = %u, max_queue_pairs = %u",
+ opt->queue_pairs, dev_info.max_queue_pairs);
+ return false;
+ }
+
+ if (opt->queue_size > dev_info.max_desc) {
+ ml_err("Insufficient capabilities: queue_size = %u, max_desc = %u", opt->queue_size,
+ dev_info.max_desc);
+ return false;
+ }
+
if (opt->nb_filelist > dev_info.max_models) {
ml_err("Insufficient capabilities: Filelist count exceeded device limit, count = %u (max limit = %u)",
opt->nb_filelist, dev_info.max_models);
@@ -310,10 +324,21 @@ test_inference_opt_check(struct ml_options *opt)
return -EINVAL;
}
+ if (opt->queue_pairs == 0) {
+ ml_err("Invalid option, queue_pairs = %u\n", opt->queue_pairs);
+ return -EINVAL;
+ }
+
+ if (opt->queue_size == 0) {
+ ml_err("Invalid option, queue_size = %u\n", opt->queue_size);
+ return -EINVAL;
+ }
+
/* check number of available lcores. */
- if (rte_lcore_count() < 3) {
+ if (rte_lcore_count() < (uint32_t)(opt->queue_pairs * 2 + 1)) {
ml_err("Insufficient lcores = %u\n", rte_lcore_count());
- ml_err("Minimum lcores required to create %u queue-pairs = %u\n", 1, 3);
+ ml_err("Minimum lcores required to create %u queue-pairs = %u\n", opt->queue_pairs,
+ (opt->queue_pairs * 2 + 1));
return -EINVAL;
}
@@ -331,6 +356,8 @@ test_inference_opt_dump(struct ml_options *opt)
/* dump test opts */
ml_dump("repetitions", "%" PRIu64, opt->repetitions);
ml_dump("burst_size", "%u", opt->burst_size);
+ ml_dump("queue_pairs", "%u", opt->queue_pairs);
+ ml_dump("queue_size", "%u", opt->queue_size);
ml_dump_begin("filelist");
for (i = 0; i < opt->nb_filelist; i++) {
@@ -422,23 +449,31 @@ ml_inference_mldev_setup(struct ml_test *test, struct ml_options *opt)
{
struct rte_ml_dev_qp_conf qp_conf;
struct test_inference *t;
+ uint16_t qp_id;
int ret;
t = ml_test_priv(test);
+ RTE_SET_USED(t);
+
ret = ml_test_device_configure(test, opt);
if (ret != 0)
return ret;
/* setup queue pairs */
- qp_conf.nb_desc = t->cmn.dev_info.max_desc;
+ qp_conf.nb_desc = opt->queue_size;
qp_conf.cb = NULL;
- ret = rte_ml_dev_queue_pair_setup(opt->dev_id, 0, &qp_conf, opt->socket_id);
- if (ret != 0) {
- ml_err("Failed to setup ml device queue-pair, dev_id = %d, qp_id = %u\n",
- opt->dev_id, 0);
- goto error;
+ for (qp_id = 0; qp_id < opt->queue_pairs; qp_id++) {
+ qp_conf.nb_desc = opt->queue_size;
+ qp_conf.cb = NULL;
+
+ ret = rte_ml_dev_queue_pair_setup(opt->dev_id, qp_id, &qp_conf, opt->socket_id);
+ if (ret != 0) {
+ ml_err("Failed to setup ml device queue-pair, dev_id = %d, qp_id = %u\n",
+ opt->dev_id, qp_id);
+ return ret;
+ }
}
ret = ml_test_device_start(test, opt);
@@ -700,14 +735,28 @@ ml_inference_launch_cores(struct ml_test *test, struct ml_options *opt, uint16_t
{
struct test_inference *t = ml_test_priv(test);
uint32_t lcore_id;
+ uint32_t nb_reqs;
uint32_t id = 0;
+ uint32_t qp_id;
+
+ nb_reqs = opt->repetitions / opt->queue_pairs;
RTE_LCORE_FOREACH_WORKER(lcore_id)
{
- if (id == 2)
+ if (id >= opt->queue_pairs * 2)
break;
- t->args[lcore_id].nb_reqs = opt->repetitions;
+ qp_id = id / 2;
+ t->args[lcore_id].qp_id = qp_id;
+ t->args[lcore_id].nb_reqs = nb_reqs;
+ if (qp_id == 0)
+ t->args[lcore_id].nb_reqs += opt->repetitions - nb_reqs * opt->queue_pairs;
+
+ if (t->args[lcore_id].nb_reqs == 0) {
+ id++;
+ break;
+ }
+
t->args[lcore_id].start_fid = start_fid;
t->args[lcore_id].end_fid = end_fid;
diff --git a/app/test-mldev/test_inference_common.h b/app/test-mldev/test_inference_common.h
index da800f2bd4..81d9b07d41 100644
--- a/app/test-mldev/test_inference_common.h
+++ b/app/test-mldev/test_inference_common.h
@@ -22,6 +22,7 @@ struct ml_core_args {
uint64_t nb_reqs;
uint16_t start_fid;
uint16_t end_fid;
+ uint32_t qp_id;
struct rte_ml_op **enq_ops;
struct rte_ml_op **deq_ops;
diff --git a/doc/guides/tools/testmldev.rst b/doc/guides/tools/testmldev.rst
index eb9081723b..17ee2cf7fb 100644
--- a/doc/guides/tools/testmldev.rst
+++ b/doc/guides/tools/testmldev.rst
@@ -95,6 +95,17 @@ The following are the command-line options supported by the test application.
Set the burst size to be used when enqueuing / dequeuing inferences. Default value is `1`.
+* ``--queue_pairs <n>``
+
+ Set the number of queue-pairs to be used for inference enqueue and dequeue operations.
+ Default value is `1`.
+
+* ``--queue_size <n>``
+
+ Set the size of queue-pair to be created for inference enqueue / dequeue operations.
+ Queue size would translate into `rte_ml_dev_qp_conf::nb_desc` field during queue-pair
+ creation. Default value is `1`.
+
* ``--debug``
Enable the tests to run in debug mode.
@@ -120,12 +131,17 @@ Supported command line options for the `device_ops` test are following::
--test
--dev_id
--socket_id
+ --queue_pairs
+ --queue_size
DEVICE_OPS Test
~~~~~~~~~~~~~~~
-Device ops test validates the device configuration and reconfiguration.
+Device ops test validates the device configuration and reconfiguration support. The test configures
+ML device based on the option ``--queue_pairs`` and ``--queue_size`` specified by the user, and
+later reconfigures the ML device with the number of queue pairs and queue size based the maximum
+specified through the device info.
Example
@@ -139,6 +155,14 @@ Command to run device_ops test:
--test=device_ops
+Command to run device_ops test with user options:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=device_ops --queue_pairs <M> --queue_size <N>
+
+
ML Model Tests
-------------------------
@@ -241,6 +265,8 @@ Supported command line options for inference tests are following::
--filelist
--repetitions
--burst_size
+ --queue_pairs
+ --queue_size
List of files to be used for the inference tests can be specified through the option
@@ -252,6 +278,9 @@ the test, one entry per model. Maximum number of file entries supported by the t
When ``--burst_size <num>`` option is specified for the test, enqueue and dequeue burst would
try to enqueue or dequeue ``num`` number of inferences per each call respectively.
+In the inference test, a pair of lcores are mapped to each queue pair. Minimum number of lcores
+required for the tests is equal to ``(queue_pairs * 2 + 1)``.
+
.. Note::
* The ``--filelist <file_list>`` is a mandatory option for running inference tests.
@@ -296,6 +325,14 @@ Example command to run inference_ordered test with a specific burst size:
--test=inference_ordered --filelist model.bin,input.bin,output.bin \
--burst_size 12
+Example command to run inference_ordered test with multiple queue-pairs and queue size:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_ordered --filelist model.bin,input.bin,output.bin \
+ --queue_pairs 4 --queue_size 16
+
INFERENCE_INTERLEAVE Test
~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -336,13 +373,14 @@ Example command to run inference_interleave test with multiple models:
--test=inference_interleave --filelist model_A.bin,input_A.bin,output_A.bin \
--filelist model_B.bin,input_B.bin,output_B.bin
-Example command to run inference_interleave test with a specific burst size:
+Example command to run inference_interleave test with a specific burst size, multiple queue-pairs
+and queue size:
.. code-block:: console
sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
--test=inference_interleave --filelist model.bin,input.bin,output.bin \
- --burst_size 16
+ --queue_pairs 8 --queue_size 12 --burst_size 16
Debug mode
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v7 09/11] app/mldev: enable support for inference batches
2023-03-16 21:14 ` [PATCH v7 00/11] Implementation of mldev test application Srikanth Yalavarthi
` (7 preceding siblings ...)
2023-03-16 21:14 ` [PATCH v7 08/11] app/mldev: enable support for queue pairs and size Srikanth Yalavarthi
@ 2023-03-16 21:14 ` Srikanth Yalavarthi
2023-03-16 21:14 ` [PATCH v7 10/11] app/mldev: enable support for inference validation Srikanth Yalavarthi
` (2 subsequent siblings)
11 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-16 21:14 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Enabled support to execute multiple batches of inferences
per each enqueue request. Input and reference for the test
should be appropriately provided for multi-batch run. Number
of batches can be specified through "--batches" option.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
Acked-by: Anup Prabhu <aprabhu@marvell.com>
---
app/test-mldev/ml_options.c | 12 +++++++++++-
app/test-mldev/ml_options.h | 2 ++
app/test-mldev/test_inference_common.c | 22 +++++++++++++---------
app/test-mldev/test_model_common.c | 6 ++++++
app/test-mldev/test_model_common.h | 1 +
doc/guides/tools/testmldev.rst | 6 ++++++
6 files changed, 39 insertions(+), 10 deletions(-)
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 901adaed33..44df44991b 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -27,6 +27,7 @@ ml_options_default(struct ml_options *opt)
opt->burst_size = 1;
opt->queue_pairs = 1;
opt->queue_size = 1;
+ opt->batches = 0;
opt->debug = false;
}
@@ -170,6 +171,12 @@ ml_parse_queue_size(struct ml_options *opt, const char *arg)
return parser_read_uint16(&opt->queue_size, arg);
}
+static int
+ml_parse_batches(struct ml_options *opt, const char *arg)
+{
+ return parser_read_uint16(&opt->batches, arg);
+}
+
static void
ml_dump_test_options(const char *testname)
{
@@ -190,7 +197,8 @@ ml_dump_test_options(const char *testname)
"\t\t--repetitions : number of inference repetitions\n"
"\t\t--burst_size : inference burst size\n"
"\t\t--queue_pairs : number of queue pairs to create\n"
- "\t\t--queue_size : size fo queue-pair\n");
+ "\t\t--queue_size : size fo queue-pair\n"
+ "\t\t--batches : number of batches of input\n");
printf("\n");
}
}
@@ -220,6 +228,7 @@ static struct option lgopts[] = {
{ML_BURST_SIZE, 1, 0, 0},
{ML_QUEUE_PAIRS, 1, 0, 0},
{ML_QUEUE_SIZE, 1, 0, 0},
+ {ML_BATCHES, 1, 0, 0},
{ML_DEBUG, 0, 0, 0},
{ML_HELP, 0, 0, 0},
{NULL, 0, 0, 0}};
@@ -239,6 +248,7 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
{ML_BURST_SIZE, ml_parse_burst_size},
{ML_QUEUE_PAIRS, ml_parse_queue_pairs},
{ML_QUEUE_SIZE, ml_parse_queue_size},
+ {ML_BATCHES, ml_parse_batches},
};
for (i = 0; i < RTE_DIM(parsermap); i++) {
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index c4018ee9d1..48fe064150 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -21,6 +21,7 @@
#define ML_BURST_SIZE ("burst_size")
#define ML_QUEUE_PAIRS ("queue_pairs")
#define ML_QUEUE_SIZE ("queue_size")
+#define ML_BATCHES ("batches")
#define ML_DEBUG ("debug")
#define ML_HELP ("help")
@@ -40,6 +41,7 @@ struct ml_options {
uint16_t burst_size;
uint16_t queue_pairs;
uint16_t queue_size;
+ uint16_t batches;
bool debug;
};
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
index b4ad3c4b72..0f281aed6c 100644
--- a/app/test-mldev/test_inference_common.c
+++ b/app/test-mldev/test_inference_common.c
@@ -50,7 +50,7 @@ ml_enqueue_single(void *arg)
goto retry;
op->model_id = t->model[fid].id;
- op->nb_batches = t->model[fid].info.batch_size;
+ op->nb_batches = t->model[fid].nb_batches;
op->mempool = t->op_pool;
op->input.addr = req->input;
@@ -163,7 +163,7 @@ ml_enqueue_burst(void *arg)
for (i = 0; i < ops_count; i++) {
args->enq_ops[i]->model_id = t->model[fid].id;
- args->enq_ops[i]->nb_batches = t->model[fid].info.batch_size;
+ args->enq_ops[i]->nb_batches = t->model[fid].nb_batches;
args->enq_ops[i]->mempool = t->op_pool;
args->enq_ops[i]->input.addr = args->reqs[i]->input;
@@ -359,6 +359,11 @@ test_inference_opt_dump(struct ml_options *opt)
ml_dump("queue_pairs", "%u", opt->queue_pairs);
ml_dump("queue_size", "%u", opt->queue_size);
+ if (opt->batches == 0)
+ ml_dump("batches", "%u (default)", opt->batches);
+ else
+ ml_dump("batches", "%u", opt->batches);
+
ml_dump_begin("filelist");
for (i = 0; i < opt->nb_filelist; i++) {
ml_dump_list("model", i, opt->filelist[i].model);
@@ -528,8 +533,8 @@ ml_request_initialize(struct rte_mempool *mp, void *opaque, void *obj, unsigned
req->niters = 0;
/* quantize data */
- rte_ml_io_quantize(t->cmn.opt->dev_id, t->model[t->fid].id,
- t->model[t->fid].info.batch_size, t->model[t->fid].input, req->input);
+ rte_ml_io_quantize(t->cmn.opt->dev_id, t->model[t->fid].id, t->model[t->fid].nb_batches,
+ t->model[t->fid].input, req->input);
}
int
@@ -547,7 +552,7 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, uint16_t
int ret;
/* get input buffer size */
- ret = rte_ml_io_input_size_get(opt->dev_id, t->model[fid].id, t->model[fid].info.batch_size,
+ ret = rte_ml_io_input_size_get(opt->dev_id, t->model[fid].id, t->model[fid].nb_batches,
&t->model[fid].inp_qsize, &t->model[fid].inp_dsize);
if (ret != 0) {
ml_err("Failed to get input size, model : %s\n", opt->filelist[fid].model);
@@ -555,9 +560,8 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, uint16_t
}
/* get output buffer size */
- ret = rte_ml_io_output_size_get(opt->dev_id, t->model[fid].id,
- t->model[fid].info.batch_size, &t->model[fid].out_qsize,
- &t->model[fid].out_dsize);
+ ret = rte_ml_io_output_size_get(opt->dev_id, t->model[fid].id, t->model[fid].nb_batches,
+ &t->model[fid].out_qsize, &t->model[fid].out_dsize);
if (ret != 0) {
ml_err("Failed to get input size, model : %s\n", opt->filelist[fid].model);
return ret;
@@ -702,7 +706,7 @@ ml_request_finish(struct rte_mempool *mp, void *opaque, void *obj, unsigned int
return;
t->nb_used++;
- rte_ml_io_dequantize(t->cmn.opt->dev_id, model->id, t->model[req->fid].info.batch_size,
+ rte_ml_io_dequantize(t->cmn.opt->dev_id, model->id, t->model[req->fid].nb_batches,
req->output, model->output);
}
diff --git a/app/test-mldev/test_model_common.c b/app/test-mldev/test_model_common.c
index b94d46154d..c28e452f29 100644
--- a/app/test-mldev/test_model_common.c
+++ b/app/test-mldev/test_model_common.c
@@ -71,6 +71,12 @@ ml_model_load(struct ml_test *test, struct ml_options *opt, struct ml_model *mod
return ret;
}
+ /* Update number of batches */
+ if (opt->batches == 0)
+ model->nb_batches = model->info.batch_size;
+ else
+ model->nb_batches = opt->batches;
+
model->state = MODEL_LOADED;
return 0;
diff --git a/app/test-mldev/test_model_common.h b/app/test-mldev/test_model_common.h
index 5ee975109d..19429ce142 100644
--- a/app/test-mldev/test_model_common.h
+++ b/app/test-mldev/test_model_common.h
@@ -30,6 +30,7 @@ struct ml_model {
uint8_t *output;
struct rte_mempool *io_pool;
+ uint32_t nb_batches;
};
int ml_model_load(struct ml_test *test, struct ml_options *opt, struct ml_model *model,
diff --git a/doc/guides/tools/testmldev.rst b/doc/guides/tools/testmldev.rst
index 17ee2cf7fb..6632025fdf 100644
--- a/doc/guides/tools/testmldev.rst
+++ b/doc/guides/tools/testmldev.rst
@@ -106,6 +106,11 @@ The following are the command-line options supported by the test application.
Queue size would translate into `rte_ml_dev_qp_conf::nb_desc` field during queue-pair
creation. Default value is `1`.
+* ``--batches <n>``
+
+ Set the number batches in the input file provided for inference run. When not specified
+ the test would assume the number of batches is equal to the batch size of the model.
+
* ``--debug``
Enable the tests to run in debug mode.
@@ -267,6 +272,7 @@ Supported command line options for inference tests are following::
--burst_size
--queue_pairs
--queue_size
+ --batches
List of files to be used for the inference tests can be specified through the option
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v7 10/11] app/mldev: enable support for inference validation
2023-03-16 21:14 ` [PATCH v7 00/11] Implementation of mldev test application Srikanth Yalavarthi
` (8 preceding siblings ...)
2023-03-16 21:14 ` [PATCH v7 09/11] app/mldev: enable support for inference batches Srikanth Yalavarthi
@ 2023-03-16 21:14 ` Srikanth Yalavarthi
2023-03-16 21:14 ` [PATCH v7 11/11] app/mldev: enable reporting stats in mldev app Srikanth Yalavarthi
2023-03-19 22:08 ` [PATCH v7 00/11] Implementation of mldev test application Thomas Monjalon
11 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-16 21:14 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Enabled support to validate inference output with reference
output provided by the user. Validation would be successful
only when the inference outputs are within the 'tolerance'
specified through command line option "--tolerance".
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
Acked-by: Anup Prabhu <aprabhu@marvell.com>
---
app/test-mldev/meson.build | 2 +-
app/test-mldev/ml_options.c | 24 ++-
app/test-mldev/ml_options.h | 3 +
app/test-mldev/test_inference_common.c | 216 ++++++++++++++++++++++++-
app/test-mldev/test_inference_common.h | 1 +
app/test-mldev/test_model_common.h | 1 +
doc/guides/tools/testmldev.rst | 39 +++++
7 files changed, 281 insertions(+), 5 deletions(-)
diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build
index 41d22fb22c..15db534dc2 100644
--- a/app/test-mldev/meson.build
+++ b/app/test-mldev/meson.build
@@ -21,4 +21,4 @@ sources = files(
'test_inference_interleave.c',
)
-deps += ['mldev']
+deps += ['mldev', 'hash']
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index 44df44991b..da30796a6b 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -4,6 +4,7 @@
#include <errno.h>
#include <getopt.h>
+#include <math.h>
#include <rte_memory.h>
#include <rte_mldev.h>
@@ -28,6 +29,7 @@ ml_options_default(struct ml_options *opt)
opt->queue_pairs = 1;
opt->queue_size = 1;
opt->batches = 0;
+ opt->tolerance = 0.0;
opt->debug = false;
}
@@ -133,6 +135,13 @@ ml_parse_filelist(struct ml_options *opt, const char *arg)
}
strlcpy(opt->filelist[opt->nb_filelist].output, token, PATH_MAX);
+ /* reference - optional */
+ token = strtok(NULL, delim);
+ if (token != NULL)
+ strlcpy(opt->filelist[opt->nb_filelist].reference, token, PATH_MAX);
+ else
+ memset(opt->filelist[opt->nb_filelist].reference, 0, PATH_MAX);
+
opt->nb_filelist++;
if (opt->nb_filelist == 0) {
@@ -177,6 +186,14 @@ ml_parse_batches(struct ml_options *opt, const char *arg)
return parser_read_uint16(&opt->batches, arg);
}
+static int
+ml_parse_tolerance(struct ml_options *opt, const char *arg)
+{
+ opt->tolerance = fabs(atof(arg));
+
+ return 0;
+}
+
static void
ml_dump_test_options(const char *testname)
{
@@ -193,12 +210,13 @@ ml_dump_test_options(const char *testname)
if ((strcmp(testname, "inference_ordered") == 0) ||
(strcmp(testname, "inference_interleave") == 0)) {
- printf("\t\t--filelist : comma separated list of model, input and output\n"
+ printf("\t\t--filelist : comma separated list of model, input, output and reference\n"
"\t\t--repetitions : number of inference repetitions\n"
"\t\t--burst_size : inference burst size\n"
"\t\t--queue_pairs : number of queue pairs to create\n"
"\t\t--queue_size : size fo queue-pair\n"
- "\t\t--batches : number of batches of input\n");
+ "\t\t--batches : number of batches of input\n"
+ "\t\t--tolerance : maximum tolerance (%%) for output validation\n");
printf("\n");
}
}
@@ -229,6 +247,7 @@ static struct option lgopts[] = {
{ML_QUEUE_PAIRS, 1, 0, 0},
{ML_QUEUE_SIZE, 1, 0, 0},
{ML_BATCHES, 1, 0, 0},
+ {ML_TOLERANCE, 1, 0, 0},
{ML_DEBUG, 0, 0, 0},
{ML_HELP, 0, 0, 0},
{NULL, 0, 0, 0}};
@@ -249,6 +268,7 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
{ML_QUEUE_PAIRS, ml_parse_queue_pairs},
{ML_QUEUE_SIZE, ml_parse_queue_size},
{ML_BATCHES, ml_parse_batches},
+ {ML_TOLERANCE, ml_parse_tolerance},
};
for (i = 0; i < RTE_DIM(parsermap); i++) {
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index 48fe064150..7f3db29656 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -22,6 +22,7 @@
#define ML_QUEUE_PAIRS ("queue_pairs")
#define ML_QUEUE_SIZE ("queue_size")
#define ML_BATCHES ("batches")
+#define ML_TOLERANCE ("tolerance")
#define ML_DEBUG ("debug")
#define ML_HELP ("help")
@@ -29,6 +30,7 @@ struct ml_filelist {
char model[PATH_MAX];
char input[PATH_MAX];
char output[PATH_MAX];
+ char reference[PATH_MAX];
};
struct ml_options {
@@ -42,6 +44,7 @@ struct ml_options {
uint16_t queue_pairs;
uint16_t queue_size;
uint16_t batches;
+ float tolerance;
bool debug;
};
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
index 0f281aed6c..b605c1f5d3 100644
--- a/app/test-mldev/test_inference_common.c
+++ b/app/test-mldev/test_inference_common.c
@@ -6,6 +6,7 @@
#include <unistd.h>
#include <rte_common.h>
+#include <rte_hash_crc.h>
#include <rte_launch.h>
#include <rte_lcore.h>
#include <rte_malloc.h>
@@ -15,6 +16,27 @@
#include "ml_common.h"
#include "test_inference_common.h"
+#define ML_TEST_READ_TYPE(buffer, type) (*((type *)buffer))
+
+#define ML_TEST_CHECK_OUTPUT(output, reference, tolerance) \
+ (((float)output - (float)reference) <= (((float)reference * tolerance) / 100.0))
+
+#define ML_OPEN_WRITE_GET_ERR(name, buffer, size, err) \
+ do { \
+ FILE *fp = fopen(name, "w+"); \
+ if (fp == NULL) { \
+ ml_err("Unable to create file: %s, error: %s", name, strerror(errno)); \
+ err = true; \
+ } else { \
+ if (fwrite(buffer, 1, size, fp) != size) { \
+ ml_err("Error writing output, file: %s, error: %s", name, \
+ strerror(errno)); \
+ err = true; \
+ } \
+ fclose(fp); \
+ } \
+ } while (0)
+
/* Enqueue inference requests with burst size equal to 1 */
static int
ml_enqueue_single(void *arg)
@@ -358,6 +380,7 @@ test_inference_opt_dump(struct ml_options *opt)
ml_dump("burst_size", "%u", opt->burst_size);
ml_dump("queue_pairs", "%u", opt->queue_pairs);
ml_dump("queue_size", "%u", opt->queue_size);
+ ml_dump("tolerance", "%-7.3f", opt->tolerance);
if (opt->batches == 0)
ml_dump("batches", "%u (default)", opt->batches);
@@ -369,6 +392,8 @@ test_inference_opt_dump(struct ml_options *opt)
ml_dump_list("model", i, opt->filelist[i].model);
ml_dump_list("input", i, opt->filelist[i].input);
ml_dump_list("output", i, opt->filelist[i].output);
+ if (strcmp(opt->filelist[i].reference, "\0") != 0)
+ ml_dump_list("reference", i, opt->filelist[i].reference);
}
ml_dump_end;
}
@@ -393,6 +418,7 @@ test_inference_setup(struct ml_test *test, struct ml_options *opt)
t = ml_test_priv(test);
t->nb_used = 0;
+ t->nb_valid = 0;
t->cmn.result = ML_TEST_FAILED;
t->cmn.opt = opt;
memset(t->error_count, 0, RTE_MAX_LCORE * sizeof(uint64_t));
@@ -569,6 +595,9 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, uint16_t
/* allocate buffer for user data */
mz_size = t->model[fid].inp_dsize + t->model[fid].out_dsize;
+ if (strcmp(opt->filelist[fid].reference, "\0") != 0)
+ mz_size += t->model[fid].out_dsize;
+
sprintf(mz_name, "ml_user_data_%d", fid);
mz = rte_memzone_reserve(mz_name, mz_size, opt->socket_id, 0);
if (mz == NULL) {
@@ -579,6 +608,10 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, uint16_t
t->model[fid].input = mz->addr;
t->model[fid].output = t->model[fid].input + t->model[fid].inp_dsize;
+ if (strcmp(opt->filelist[fid].reference, "\0") != 0)
+ t->model[fid].reference = t->model[fid].output + t->model[fid].out_dsize;
+ else
+ t->model[fid].reference = NULL;
/* load input file */
fp = fopen(opt->filelist[fid].input, "r");
@@ -607,6 +640,27 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, uint16_t
}
fclose(fp);
+ /* load reference file */
+ if (t->model[fid].reference != NULL) {
+ fp = fopen(opt->filelist[fid].reference, "r");
+ if (fp == NULL) {
+ ml_err("Failed to open reference file : %s\n",
+ opt->filelist[fid].reference);
+ ret = -errno;
+ goto error;
+ }
+
+ if (fread(t->model[fid].reference, 1, t->model[fid].out_dsize, fp) !=
+ t->model[fid].out_dsize) {
+ ml_err("Failed to read reference file : %s\n",
+ opt->filelist[fid].reference);
+ ret = -errno;
+ fclose(fp);
+ goto error;
+ }
+ fclose(fp);
+ }
+
/* create mempool for quantized input and output buffers. ml_request_initialize is
* used as a callback for object creation.
*/
@@ -691,6 +745,121 @@ ml_inference_mem_destroy(struct ml_test *test, struct ml_options *opt)
rte_mempool_free(t->op_pool);
}
+static bool
+ml_inference_validation(struct ml_test *test, struct ml_request *req)
+{
+ struct test_inference *t = ml_test_priv((struct ml_test *)test);
+ struct ml_model *model;
+ uint32_t nb_elements;
+ uint8_t *reference;
+ uint8_t *output;
+ bool match;
+ uint32_t i;
+ uint32_t j;
+
+ model = &t->model[req->fid];
+
+ /* compare crc when tolerance is 0 */
+ if (t->cmn.opt->tolerance == 0.0) {
+ match = (rte_hash_crc(model->output, model->out_dsize, 0) ==
+ rte_hash_crc(model->reference, model->out_dsize, 0));
+ } else {
+ output = model->output;
+ reference = model->reference;
+
+ i = 0;
+next_output:
+ nb_elements =
+ model->info.output_info[i].shape.w * model->info.output_info[i].shape.x *
+ model->info.output_info[i].shape.y * model->info.output_info[i].shape.z;
+ j = 0;
+next_element:
+ match = false;
+ switch (model->info.output_info[i].dtype) {
+ case RTE_ML_IO_TYPE_INT8:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, int8_t),
+ ML_TEST_READ_TYPE(reference, int8_t),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(int8_t);
+ reference += sizeof(int8_t);
+ break;
+ case RTE_ML_IO_TYPE_UINT8:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, uint8_t),
+ ML_TEST_READ_TYPE(reference, uint8_t),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(float);
+ reference += sizeof(float);
+ break;
+ case RTE_ML_IO_TYPE_INT16:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, int16_t),
+ ML_TEST_READ_TYPE(reference, int16_t),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(int16_t);
+ reference += sizeof(int16_t);
+ break;
+ case RTE_ML_IO_TYPE_UINT16:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, uint16_t),
+ ML_TEST_READ_TYPE(reference, uint16_t),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(uint16_t);
+ reference += sizeof(uint16_t);
+ break;
+ case RTE_ML_IO_TYPE_INT32:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, int32_t),
+ ML_TEST_READ_TYPE(reference, int32_t),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(int32_t);
+ reference += sizeof(int32_t);
+ break;
+ case RTE_ML_IO_TYPE_UINT32:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, uint32_t),
+ ML_TEST_READ_TYPE(reference, uint32_t),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(uint32_t);
+ reference += sizeof(uint32_t);
+ break;
+ case RTE_ML_IO_TYPE_FP32:
+ if (ML_TEST_CHECK_OUTPUT(ML_TEST_READ_TYPE(output, float),
+ ML_TEST_READ_TYPE(reference, float),
+ t->cmn.opt->tolerance))
+ match = true;
+
+ output += sizeof(float);
+ reference += sizeof(float);
+ break;
+ default: /* other types, fp8, fp16, bfloat16 */
+ match = true;
+ }
+
+ if (!match)
+ goto done;
+ j++;
+ if (j < nb_elements)
+ goto next_element;
+
+ i++;
+ if (i < model->info.nb_outputs)
+ goto next_output;
+ }
+done:
+ if (match)
+ t->nb_valid++;
+
+ return match;
+}
+
/* Callback for mempool object iteration. This call would dequantize output data. */
static void
ml_request_finish(struct rte_mempool *mp, void *opaque, void *obj, unsigned int obj_idx)
@@ -698,9 +867,10 @@ ml_request_finish(struct rte_mempool *mp, void *opaque, void *obj, unsigned int
struct test_inference *t = ml_test_priv((struct ml_test *)opaque);
struct ml_request *req = (struct ml_request *)obj;
struct ml_model *model = &t->model[req->fid];
+ char str[PATH_MAX];
+ bool error = false;
RTE_SET_USED(mp);
- RTE_SET_USED(obj_idx);
if (req->niters == 0)
return;
@@ -708,6 +878,48 @@ ml_request_finish(struct rte_mempool *mp, void *opaque, void *obj, unsigned int
t->nb_used++;
rte_ml_io_dequantize(t->cmn.opt->dev_id, model->id, t->model[req->fid].nb_batches,
req->output, model->output);
+
+ if (model->reference == NULL) {
+ t->nb_valid++;
+ goto dump_output_pass;
+ }
+
+ if (!ml_inference_validation(opaque, req))
+ goto dump_output_fail;
+ else
+ goto dump_output_pass;
+
+dump_output_pass:
+ if (obj_idx == 0) {
+ /* write quantized output */
+ snprintf(str, PATH_MAX, "%s.q", t->cmn.opt->filelist[req->fid].output);
+ ML_OPEN_WRITE_GET_ERR(str, req->output, model->out_qsize, error);
+ if (error)
+ return;
+
+ /* write dequantized output */
+ snprintf(str, PATH_MAX, "%s", t->cmn.opt->filelist[req->fid].output);
+ ML_OPEN_WRITE_GET_ERR(str, model->output, model->out_dsize, error);
+ if (error)
+ return;
+ }
+
+ return;
+
+dump_output_fail:
+ if (t->cmn.opt->debug) {
+ /* dump quantized output buffer */
+ snprintf(str, PATH_MAX, "%s.q.%d", t->cmn.opt->filelist[req->fid].output, obj_idx);
+ ML_OPEN_WRITE_GET_ERR(str, req->output, model->out_qsize, error);
+ if (error)
+ return;
+
+ /* dump dequantized output buffer */
+ snprintf(str, PATH_MAX, "%s.%d", t->cmn.opt->filelist[req->fid].output, obj_idx);
+ ML_OPEN_WRITE_GET_ERR(str, model->output, model->out_dsize, error);
+ if (error)
+ return;
+ }
}
int
@@ -725,7 +937,7 @@ ml_inference_result(struct ml_test *test, struct ml_options *opt, uint16_t fid)
rte_mempool_obj_iter(t->model[fid].io_pool, ml_request_finish, test);
- if ((t->nb_used > 0) && (error_count == 0))
+ if ((t->nb_used == t->nb_valid) && (error_count == 0))
t->cmn.result = ML_TEST_SUCCESS;
else
t->cmn.result = ML_TEST_FAILED;
diff --git a/app/test-mldev/test_inference_common.h b/app/test-mldev/test_inference_common.h
index 81d9b07d41..2e4889e1f7 100644
--- a/app/test-mldev/test_inference_common.h
+++ b/app/test-mldev/test_inference_common.h
@@ -38,6 +38,7 @@ struct test_inference {
struct rte_mempool *op_pool;
uint64_t nb_used;
+ uint64_t nb_valid;
uint16_t fid;
int (*enqueue)(void *arg);
diff --git a/app/test-mldev/test_model_common.h b/app/test-mldev/test_model_common.h
index 19429ce142..b2a75a3261 100644
--- a/app/test-mldev/test_model_common.h
+++ b/app/test-mldev/test_model_common.h
@@ -28,6 +28,7 @@ struct ml_model {
uint8_t *input;
uint8_t *output;
+ uint8_t *reference;
struct rte_mempool *io_pool;
uint32_t nb_batches;
diff --git a/doc/guides/tools/testmldev.rst b/doc/guides/tools/testmldev.rst
index 6632025fdf..25dc878f25 100644
--- a/doc/guides/tools/testmldev.rst
+++ b/doc/guides/tools/testmldev.rst
@@ -111,6 +111,11 @@ The following are the command-line options supported by the test application.
Set the number batches in the input file provided for inference run. When not specified
the test would assume the number of batches is equal to the batch size of the model.
+* ``--tolerance <n>``
+
+ Set the tolerance value in percentage to be used for output validation. Default value
+ is `0`.
+
* ``--debug``
Enable the tests to run in debug mode.
@@ -273,6 +278,7 @@ Supported command line options for inference tests are following::
--queue_pairs
--queue_size
--batches
+ --tolerance
List of files to be used for the inference tests can be specified through the option
@@ -287,10 +293,20 @@ try to enqueue or dequeue ``num`` number of inferences per each call respectivel
In the inference test, a pair of lcores are mapped to each queue pair. Minimum number of lcores
required for the tests is equal to ``(queue_pairs * 2 + 1)``.
+Output validation of inference would be enabled only when a reference file is specified through
+the ``--filelist`` option. Application would additionally consider the tolerance value provided
+through ``--tolerance`` option during validation. When the tolerance values is 0, CRC32 hash of
+inference output and reference output are compared. When the tolerance is non-zero, element wise
+comparison of output is performed. Validation is considered as successful only when all the
+elements of the output tensor are with in the tolerance range specified.
+
.. Note::
* The ``--filelist <file_list>`` is a mandatory option for running inference tests.
* Options not supported by the tests are ignored if specified.
+ * Element wise comparison is not supported when the output dtype is either fp8, fp16
+ or bfloat16. This is applicable only when the tolerance is greater than zero and for
+ pre-quantized models only.
INFERENCE_ORDERED Test
@@ -339,6 +355,14 @@ Example command to run inference_ordered test with multiple queue-pairs and queu
--test=inference_ordered --filelist model.bin,input.bin,output.bin \
--queue_pairs 4 --queue_size 16
+Example command to run inference_ordered with output validation using tolerance of `1%``:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_ordered --filelist model.bin,input.bin,output.bin,reference.bin \
+ --tolerance 1.0
+
INFERENCE_INTERLEAVE Test
~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -388,9 +412,24 @@ and queue size:
--test=inference_interleave --filelist model.bin,input.bin,output.bin \
--queue_pairs 8 --queue_size 12 --burst_size 16
+Example command to run inference_interleave test with multiple models ad output validation
+using tolerance of `2.0%``:
+
+.. code-block:: console
+
+ sudo <build_dir>/app/dpdk-test-mldev -c 0xf -a <PCI_ID> -- \
+ --test=inference_interleave \
+ --filelist model_A.bin,input_A.bin,output_A.bin,reference_A.bin \
+ --filelist model_B.bin,input_B.bin,output_B.bin,reference_B.bin \
+ --tolerance 2.0
+
Debug mode
----------
ML tests can be executed in debug mode by enabling the option ``--debug``. Execution of tests in
debug mode would enable additional prints.
+
+When a validation failure is observed, output from that buffer is written to the disk, with the
+filenames having similar convention when the test has passed. Additionally index of the buffer
+would be appended to the filenames.
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* [PATCH v7 11/11] app/mldev: enable reporting stats in mldev app
2023-03-16 21:14 ` [PATCH v7 00/11] Implementation of mldev test application Srikanth Yalavarthi
` (9 preceding siblings ...)
2023-03-16 21:14 ` [PATCH v7 10/11] app/mldev: enable support for inference validation Srikanth Yalavarthi
@ 2023-03-16 21:14 ` Srikanth Yalavarthi
2023-03-19 22:08 ` [PATCH v7 00/11] Implementation of mldev test application Thomas Monjalon
11 siblings, 0 replies; 122+ messages in thread
From: Srikanth Yalavarthi @ 2023-03-16 21:14 UTC (permalink / raw)
To: Srikanth Yalavarthi; +Cc: dev, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
Enable reporting driver xstats and inference end-to-end
latency and throughput in mldev inference tests. Reporting
of stats can be enabled using "--stats" option.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
Acked-by: Anup Prabhu <aprabhu@marvell.com>
---
app/test-mldev/ml_options.c | 10 +-
app/test-mldev/ml_options.h | 2 +
app/test-mldev/test_inference_common.c | 140 +++++++++++++++++++++
app/test-mldev/test_inference_common.h | 8 ++
app/test-mldev/test_inference_interleave.c | 4 +
app/test-mldev/test_inference_ordered.c | 1 +
doc/guides/tools/testmldev.rst | 7 ++
7 files changed, 171 insertions(+), 1 deletion(-)
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index da30796a6b..2efcc3532c 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -30,6 +30,7 @@ ml_options_default(struct ml_options *opt)
opt->queue_size = 1;
opt->batches = 0;
opt->tolerance = 0.0;
+ opt->stats = false;
opt->debug = false;
}
@@ -216,7 +217,8 @@ ml_dump_test_options(const char *testname)
"\t\t--queue_pairs : number of queue pairs to create\n"
"\t\t--queue_size : size fo queue-pair\n"
"\t\t--batches : number of batches of input\n"
- "\t\t--tolerance : maximum tolerance (%%) for output validation\n");
+ "\t\t--tolerance : maximum tolerance (%%) for output validation\n"
+ "\t\t--stats : enable reporting performance statistics\n");
printf("\n");
}
}
@@ -248,6 +250,7 @@ static struct option lgopts[] = {
{ML_QUEUE_SIZE, 1, 0, 0},
{ML_BATCHES, 1, 0, 0},
{ML_TOLERANCE, 1, 0, 0},
+ {ML_STATS, 0, 0, 0},
{ML_DEBUG, 0, 0, 0},
{ML_HELP, 0, 0, 0},
{NULL, 0, 0, 0}};
@@ -290,6 +293,11 @@ ml_options_parse(struct ml_options *opt, int argc, char **argv)
while ((opts = getopt_long(argc, argv, "", lgopts, &opt_idx)) != EOF) {
switch (opts) {
case 0: /* parse long options */
+ if (!strcmp(lgopts[opt_idx].name, "stats")) {
+ opt->stats = true;
+ break;
+ }
+
if (!strcmp(lgopts[opt_idx].name, "debug")) {
opt->debug = true;
break;
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index 7f3db29656..beb0fe69c6 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -23,6 +23,7 @@
#define ML_QUEUE_SIZE ("queue_size")
#define ML_BATCHES ("batches")
#define ML_TOLERANCE ("tolerance")
+#define ML_STATS ("stats")
#define ML_DEBUG ("debug")
#define ML_HELP ("help")
@@ -45,6 +46,7 @@ struct ml_options {
uint16_t queue_size;
uint16_t batches;
float tolerance;
+ bool stats;
bool debug;
};
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
index b605c1f5d3..e85f32be60 100644
--- a/app/test-mldev/test_inference_common.c
+++ b/app/test-mldev/test_inference_common.c
@@ -6,6 +6,7 @@
#include <unistd.h>
#include <rte_common.h>
+#include <rte_cycles.h>
#include <rte_hash_crc.h>
#include <rte_launch.h>
#include <rte_lcore.h>
@@ -37,6 +38,17 @@
} \
} while (0)
+static void
+print_line(uint16_t len)
+{
+ uint16_t i;
+
+ for (i = 0; i < len; i++)
+ printf("-");
+
+ printf("\n");
+}
+
/* Enqueue inference requests with burst size equal to 1 */
static int
ml_enqueue_single(void *arg)
@@ -46,6 +58,7 @@ ml_enqueue_single(void *arg)
struct rte_ml_op *op = NULL;
struct ml_core_args *args;
uint64_t model_enq = 0;
+ uint64_t start_cycle;
uint32_t burst_enq;
uint32_t lcore_id;
uint16_t fid;
@@ -53,6 +66,7 @@ ml_enqueue_single(void *arg)
lcore_id = rte_lcore_id();
args = &t->args[lcore_id];
+ args->start_cycles = 0;
model_enq = 0;
if (args->nb_reqs == 0)
@@ -88,10 +102,12 @@ ml_enqueue_single(void *arg)
req->fid = fid;
enqueue_req:
+ start_cycle = rte_get_tsc_cycles();
burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, args->qp_id, &op, 1);
if (burst_enq == 0)
goto enqueue_req;
+ args->start_cycles += start_cycle;
fid++;
if (likely(fid <= args->end_fid))
goto next_model;
@@ -115,10 +131,12 @@ ml_dequeue_single(void *arg)
uint64_t total_deq = 0;
uint8_t nb_filelist;
uint32_t burst_deq;
+ uint64_t end_cycle;
uint32_t lcore_id;
lcore_id = rte_lcore_id();
args = &t->args[lcore_id];
+ args->end_cycles = 0;
nb_filelist = args->end_fid - args->start_fid + 1;
if (args->nb_reqs == 0)
@@ -126,9 +144,11 @@ ml_dequeue_single(void *arg)
dequeue_req:
burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, args->qp_id, &op, 1);
+ end_cycle = rte_get_tsc_cycles();
if (likely(burst_deq == 1)) {
total_deq += burst_deq;
+ args->end_cycles += end_cycle;
if (unlikely(op->status == RTE_ML_OP_STATUS_ERROR)) {
rte_ml_op_error_get(t->cmn.opt->dev_id, op, &error);
ml_err("error_code = 0x%" PRIx64 ", error_message = %s\n", error.errcode,
@@ -152,6 +172,7 @@ ml_enqueue_burst(void *arg)
{
struct test_inference *t = ml_test_priv((struct ml_test *)arg);
struct ml_core_args *args;
+ uint64_t start_cycle;
uint16_t ops_count;
uint64_t model_enq;
uint16_t burst_enq;
@@ -164,6 +185,7 @@ ml_enqueue_burst(void *arg)
lcore_id = rte_lcore_id();
args = &t->args[lcore_id];
+ args->start_cycles = 0;
model_enq = 0;
if (args->nb_reqs == 0)
@@ -205,8 +227,10 @@ ml_enqueue_burst(void *arg)
pending = ops_count;
enqueue_reqs:
+ start_cycle = rte_get_tsc_cycles();
burst_enq =
rte_ml_enqueue_burst(t->cmn.opt->dev_id, args->qp_id, &args->enq_ops[idx], pending);
+ args->start_cycles += burst_enq * start_cycle;
pending = pending - burst_enq;
if (pending > 0) {
@@ -236,11 +260,13 @@ ml_dequeue_burst(void *arg)
uint64_t total_deq = 0;
uint16_t burst_deq = 0;
uint8_t nb_filelist;
+ uint64_t end_cycle;
uint32_t lcore_id;
uint32_t i;
lcore_id = rte_lcore_id();
args = &t->args[lcore_id];
+ args->end_cycles = 0;
nb_filelist = args->end_fid - args->start_fid + 1;
if (args->nb_reqs == 0)
@@ -249,9 +275,11 @@ ml_dequeue_burst(void *arg)
dequeue_burst:
burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, args->qp_id, args->deq_ops,
t->cmn.opt->burst_size);
+ end_cycle = rte_get_tsc_cycles();
if (likely(burst_deq > 0)) {
total_deq += burst_deq;
+ args->end_cycles += burst_deq * end_cycle;
for (i = 0; i < burst_deq; i++) {
if (unlikely(args->deq_ops[i]->status == RTE_ML_OP_STATUS_ERROR)) {
@@ -381,6 +409,7 @@ test_inference_opt_dump(struct ml_options *opt)
ml_dump("queue_pairs", "%u", opt->queue_pairs);
ml_dump("queue_size", "%u", opt->queue_size);
ml_dump("tolerance", "%-7.3f", opt->tolerance);
+ ml_dump("stats", "%s", (opt->stats ? "true" : "false"));
if (opt->batches == 0)
ml_dump("batches", "%u (default)", opt->batches);
@@ -454,6 +483,11 @@ test_inference_setup(struct ml_test *test, struct ml_options *opt)
RTE_CACHE_LINE_SIZE, opt->socket_id);
}
+ for (i = 0; i < RTE_MAX_LCORE; i++) {
+ t->args[i].start_cycles = 0;
+ t->args[i].end_cycles = 0;
+ }
+
return 0;
error:
@@ -986,3 +1020,109 @@ ml_inference_launch_cores(struct ml_test *test, struct ml_options *opt, uint16_t
return 0;
}
+
+int
+ml_inference_stats_get(struct ml_test *test, struct ml_options *opt)
+{
+ struct test_inference *t = ml_test_priv(test);
+ uint64_t total_cycles = 0;
+ uint32_t nb_filelist;
+ uint64_t throughput;
+ uint64_t avg_e2e;
+ uint32_t qp_id;
+ uint64_t freq;
+ int ret;
+ int i;
+
+ if (!opt->stats)
+ return 0;
+
+ /* get xstats size */
+ t->xstats_size = rte_ml_dev_xstats_names_get(opt->dev_id, NULL, 0);
+ if (t->xstats_size >= 0) {
+ /* allocate for xstats_map and values */
+ t->xstats_map = rte_malloc(
+ "ml_xstats_map", t->xstats_size * sizeof(struct rte_ml_dev_xstats_map), 0);
+ if (t->xstats_map == NULL) {
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ t->xstats_values =
+ rte_malloc("ml_xstats_values", t->xstats_size * sizeof(uint64_t), 0);
+ if (t->xstats_values == NULL) {
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ ret = rte_ml_dev_xstats_names_get(opt->dev_id, t->xstats_map, t->xstats_size);
+ if (ret != t->xstats_size) {
+ printf("Unable to get xstats names, ret = %d\n", ret);
+ ret = -1;
+ goto error;
+ }
+
+ for (i = 0; i < t->xstats_size; i++)
+ rte_ml_dev_xstats_get(opt->dev_id, &t->xstats_map[i].id,
+ &t->xstats_values[i], 1);
+ }
+
+ /* print xstats*/
+ printf("\n");
+ print_line(80);
+ printf(" ML Device Extended Statistics\n");
+ print_line(80);
+ for (i = 0; i < t->xstats_size; i++)
+ printf(" %-64s = %" PRIu64 "\n", t->xstats_map[i].name, t->xstats_values[i]);
+ print_line(80);
+
+ /* release buffers */
+ if (t->xstats_map)
+ rte_free(t->xstats_map);
+
+ if (t->xstats_values)
+ rte_free(t->xstats_values);
+
+ /* print end-to-end stats */
+ freq = rte_get_tsc_hz();
+ for (qp_id = 0; qp_id < RTE_MAX_LCORE; qp_id++)
+ total_cycles += t->args[qp_id].end_cycles - t->args[qp_id].start_cycles;
+ avg_e2e = total_cycles / opt->repetitions;
+
+ if (freq == 0) {
+ avg_e2e = total_cycles / opt->repetitions;
+ printf(" %-64s = %" PRIu64 "\n", "Average End-to-End Latency (cycles)", avg_e2e);
+ } else {
+ avg_e2e = (total_cycles * NS_PER_S) / (opt->repetitions * freq);
+ printf(" %-64s = %" PRIu64 "\n", "Average End-to-End Latency (ns)", avg_e2e);
+ }
+
+ /* print inference throughput */
+ if (strcmp(opt->test_name, "inference_ordered") == 0)
+ nb_filelist = 1;
+ else
+ nb_filelist = opt->nb_filelist;
+
+ if (freq == 0) {
+ throughput = (nb_filelist * t->cmn.opt->repetitions * 1000000) / total_cycles;
+ printf(" %-64s = %" PRIu64 "\n", "Average Throughput (inferences / million cycles)",
+ throughput);
+ } else {
+ throughput = (nb_filelist * t->cmn.opt->repetitions * freq) / total_cycles;
+ printf(" %-64s = %" PRIu64 "\n", "Average Throughput (inferences / second)",
+ throughput);
+ }
+
+ print_line(80);
+
+ return 0;
+
+error:
+ if (t->xstats_map)
+ rte_free(t->xstats_map);
+
+ if (t->xstats_values)
+ rte_free(t->xstats_values);
+
+ return ret;
+}
diff --git a/app/test-mldev/test_inference_common.h b/app/test-mldev/test_inference_common.h
index 2e4889e1f7..0b4fba78e0 100644
--- a/app/test-mldev/test_inference_common.h
+++ b/app/test-mldev/test_inference_common.h
@@ -27,6 +27,9 @@ struct ml_core_args {
struct rte_ml_op **enq_ops;
struct rte_ml_op **deq_ops;
struct ml_request **reqs;
+
+ uint64_t start_cycles;
+ uint64_t end_cycles;
};
struct test_inference {
@@ -46,6 +49,10 @@ struct test_inference {
struct ml_core_args args[RTE_MAX_LCORE];
uint64_t error_count[RTE_MAX_LCORE];
+
+ struct rte_ml_dev_xstats_map *xstats_map;
+ uint64_t *xstats_values;
+ int xstats_size;
} __rte_cache_aligned;
bool test_inference_cap_check(struct ml_options *opt);
@@ -63,5 +70,6 @@ void ml_inference_mem_destroy(struct ml_test *test, struct ml_options *opt);
int ml_inference_result(struct ml_test *test, struct ml_options *opt, uint16_t fid);
int ml_inference_launch_cores(struct ml_test *test, struct ml_options *opt, uint16_t start_fid,
uint16_t end_fid);
+int ml_inference_stats_get(struct ml_test *test, struct ml_options *opt);
#endif /* _ML_TEST_INFERENCE_COMMON_ */
diff --git a/app/test-mldev/test_inference_interleave.c b/app/test-mldev/test_inference_interleave.c
index 9cf4cfa197..bd2c286737 100644
--- a/app/test-mldev/test_inference_interleave.c
+++ b/app/test-mldev/test_inference_interleave.c
@@ -56,7 +56,11 @@ test_inference_interleave_driver(struct ml_test *test, struct ml_options *opt)
goto error;
ml_inference_iomem_destroy(test, opt, fid);
+ }
+
+ ml_inference_stats_get(test, opt);
+ for (fid = 0; fid < opt->nb_filelist; fid++) {
ret = ml_model_stop(test, opt, &t->model[fid], fid);
if (ret != 0)
goto error;
diff --git a/app/test-mldev/test_inference_ordered.c b/app/test-mldev/test_inference_ordered.c
index 1cd91dc3d3..8992358936 100644
--- a/app/test-mldev/test_inference_ordered.c
+++ b/app/test-mldev/test_inference_ordered.c
@@ -54,6 +54,7 @@ test_inference_ordered_driver(struct ml_test *test, struct ml_options *opt)
goto error;
ml_inference_iomem_destroy(test, opt, fid);
+ ml_inference_stats_get(test, opt);
/* stop model */
ret = ml_model_stop(test, opt, &t->model[fid], fid);
diff --git a/doc/guides/tools/testmldev.rst b/doc/guides/tools/testmldev.rst
index 25dc878f25..e51d780cd5 100644
--- a/doc/guides/tools/testmldev.rst
+++ b/doc/guides/tools/testmldev.rst
@@ -116,6 +116,10 @@ The following are the command-line options supported by the test application.
Set the tolerance value in percentage to be used for output validation. Default value
is `0`.
+* ``--stats``
+
+ Enable reporting device extended stats.
+
* ``--debug``
Enable the tests to run in debug mode.
@@ -279,6 +283,7 @@ Supported command line options for inference tests are following::
--queue_size
--batches
--tolerance
+ --stats
List of files to be used for the inference tests can be specified through the option
@@ -300,6 +305,8 @@ inference output and reference output are compared. When the tolerance is non-ze
comparison of output is performed. Validation is considered as successful only when all the
elements of the output tensor are with in the tolerance range specified.
+Enabling ``--stats`` would print the extended stats supported by the driver.
+
.. Note::
* The ``--filelist <file_list>`` is a mandatory option for running inference tests.
--
2.17.1
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [PATCH v7 00/11] Implementation of mldev test application
2023-03-16 21:14 ` [PATCH v7 00/11] Implementation of mldev test application Srikanth Yalavarthi
` (10 preceding siblings ...)
2023-03-16 21:14 ` [PATCH v7 11/11] app/mldev: enable reporting stats in mldev app Srikanth Yalavarthi
@ 2023-03-19 22:08 ` Thomas Monjalon
11 siblings, 0 replies; 122+ messages in thread
From: Thomas Monjalon @ 2023-03-19 22:08 UTC (permalink / raw)
To: Srikanth Yalavarthi
Cc: dev, syalavarthi, sshankarnara, jerinj, aprabhu, ptakkar, pshukla
16/03/2023 22:14, Srikanth Yalavarthi:
> Srikanth Yalavarthi (11):
> app/mldev: implement test framework for mldev
> app/mldev: add common test functions
> app/mldev: add test case to validate device ops
> app/mldev: add test case to validate model ops
> app/mldev: add ordered inference test case
> app/mldev: add test case to interleave inferences
> app/mldev: enable support for burst inferences
> app/mldev: enable support for queue pairs and size
> app/mldev: enable support for inference batches
> app/mldev: enable support for inference validation
> app/mldev: enable reporting stats in mldev app
Reworked the documentation a little and other details, and applied, thanks.
^ permalink raw reply [flat|nested] 122+ messages in thread
