From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E0DFA41EB5; Thu, 16 Mar 2023 22:15:23 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8DDB042FA2; Thu, 16 Mar 2023 22:14:50 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id BB58B40DF6 for ; Thu, 16 Mar 2023 22:14:43 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 32GKcQtb004129 for ; Thu, 16 Mar 2023 14:14:43 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=Ncym7/ylaNb6HPNHhufsRMQH1wpL8nJHtncntrlREnU=; b=Hgeif2L5ByBKxY3MHGFUCkl/dpn7KCgq3wcA0fqpG/JwiUtSO3yjRIQhTGEDsHkkD3gB 9ZTJaqYwTkKEFvVywT25oMf1L0XcMI2IrleuyQBigSpPKZY3Xp7Z0UVQTk2J21VT7tGJ FJ8GR75VMjR3J9knkfK4ZXyqx9qr+Gjjl3rk3BU3jRtqbVkLOorUCE0w3QcBxyGzPzKH kckfUFVBhPZWcr6Xe9SkNrDsTyzc6zpWtIlHnMzqSkohEOrGbyX9Fh/nKWCQt6rZYoZ4 iALUZNIRfCMEw//Tb2rCn/YHePX+7IJ60CXsAJjN80G7fnARdL5AMc0C+/rPToNhMbrq 8g== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3pbxq2ub11-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 16 Mar 2023 14:14:42 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.42; Thu, 16 Mar 2023 14:14:40 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.42 via Frontend Transport; Thu, 16 Mar 2023 14:14:40 -0700 Received: from ml-host-33.caveonetworks.com (unknown [10.110.143.233]) by maili.marvell.com (Postfix) with ESMTP id 21F313F7081; Thu, 16 Mar 2023 14:14:40 -0700 (PDT) From: Srikanth Yalavarthi To: Srikanth Yalavarthi CC: , , , , , Subject: [PATCH v7 04/11] app/mldev: add test case to validate model ops Date: Thu, 16 Mar 2023 14:14:27 -0700 Message-ID: <20230316211434.13409-5-syalavarthi@marvell.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230316211434.13409-1-syalavarthi@marvell.com> References: <20221129070746.20396-1-syalavarthi@marvell.com> <20230316211434.13409-1-syalavarthi@marvell.com> MIME-Version: 1.0 Content-Type: text/plain X-Proofpoint-GUID: qUY-7EUybbYRJWsdKNYkuXCPKJ3s8amz X-Proofpoint-ORIG-GUID: qUY-7EUybbYRJWsdKNYkuXCPKJ3s8amz X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-16_14,2023-03-16_02,2023-02-09_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Added test case to validate model operations. Model ops test is a collection of sub-tests. Each sub-test invokes the model operations in a specific order. Sub-test A: (load -> start -> stop -> unload) x n Sub-test B: load x n -> start x n -> stop x n -> unload x n Sub-test C: load x n + (start + stop) x n + unload x n Sub-test D: (load + start) x n -> (stop + unload) x n Added internal functions to handle model load, start, stop and unload. List of models to be used for testing can be specified through application argument "--models" Signed-off-by: Srikanth Yalavarthi Acked-by: Anup Prabhu --- app/test-mldev/meson.build | 2 + app/test-mldev/ml_options.c | 39 ++ app/test-mldev/ml_options.h | 6 + app/test-mldev/test_model_common.c | 158 +++++++ app/test-mldev/test_model_common.h | 34 ++ app/test-mldev/test_model_ops.c | 428 ++++++++++++++++++ app/test-mldev/test_model_ops.h | 20 + .../tools/img/mldev_model_ops_subtest_a.svg | 420 +++++++++++++++++ .../tools/img/mldev_model_ops_subtest_b.svg | 423 +++++++++++++++++ .../tools/img/mldev_model_ops_subtest_c.svg | 366 +++++++++++++++ .../tools/img/mldev_model_ops_subtest_d.svg | 424 +++++++++++++++++ doc/guides/tools/testmldev.rst | 97 +++- 12 files changed, 2415 insertions(+), 2 deletions(-) create mode 100644 app/test-mldev/test_model_common.c create mode 100644 app/test-mldev/test_model_common.h create mode 100644 app/test-mldev/test_model_ops.c create mode 100644 app/test-mldev/test_model_ops.h create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_a.svg create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_b.svg create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_c.svg create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_d.svg diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build index 60ea23d142..b09e1ccc8a 100644 --- a/app/test-mldev/meson.build +++ b/app/test-mldev/meson.build @@ -14,6 +14,8 @@ sources = files( 'parser.c', 'test_common.c', 'test_device_ops.c', + 'test_model_common.c', + 'test_model_ops.c', ) deps += ['mldev'] diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c index 854d316521..8ffbab7f75 100644 --- a/app/test-mldev/ml_options.c +++ b/app/test-mldev/ml_options.c @@ -22,6 +22,7 @@ ml_options_default(struct ml_options *opt) strlcpy(opt->test_name, "device_ops", ML_TEST_NAME_MAX_LEN); opt->dev_id = 0; opt->socket_id = SOCKET_ID_ANY; + opt->nb_filelist = 0; opt->debug = false; } @@ -58,11 +59,47 @@ ml_parse_socket_id(struct ml_options *opt, const char *arg) return 0; } +static int +ml_parse_models(struct ml_options *opt, const char *arg) +{ + const char *delim = ","; + char models[PATH_MAX]; + char *token; + int ret = 0; + + strlcpy(models, arg, PATH_MAX); + + token = strtok(models, delim); + while (token != NULL) { + strlcpy(opt->filelist[opt->nb_filelist].model, token, PATH_MAX); + opt->nb_filelist++; + + if (opt->nb_filelist >= ML_TEST_MAX_MODELS) { + ml_err("Exceeded model count, max = %d\n", ML_TEST_MAX_MODELS); + ret = -EINVAL; + break; + } + token = strtok(NULL, delim); + } + + if (opt->nb_filelist == 0) { + ml_err("Models list is empty. Need at least one model for the test"); + ret = -EINVAL; + } + + return ret; +} + static void ml_dump_test_options(const char *testname) { if (strcmp(testname, "device_ops") == 0) printf("\n"); + + if (strcmp(testname, "model_ops") == 0) { + printf("\t\t--models : comma separated list of models\n"); + printf("\n"); + } } static void @@ -84,6 +121,7 @@ static struct option lgopts[] = { {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0}, + {ML_MODELS, 1, 0, 0}, {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}}; @@ -97,6 +135,7 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt) {ML_TEST, ml_parse_test_name}, {ML_DEVICE_ID, ml_parse_dev_id}, {ML_SOCKET_ID, ml_parse_socket_id}, + {ML_MODELS, ml_parse_models}, }; for (i = 0; i < RTE_DIM(parsermap); i++) { diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h index 6899cc2f88..61e938d2e2 100644 --- a/app/test-mldev/ml_options.h +++ b/app/test-mldev/ml_options.h @@ -15,13 +15,19 @@ #define ML_TEST ("test") #define ML_DEVICE_ID ("dev_id") #define ML_SOCKET_ID ("socket_id") +#define ML_MODELS ("models") #define ML_DEBUG ("debug") #define ML_HELP ("help") +struct ml_filelist { + char model[PATH_MAX]; +}; + struct ml_options { char test_name[ML_TEST_NAME_MAX_LEN]; int16_t dev_id; int socket_id; + struct ml_filelist filelist[ML_TEST_MAX_MODELS]; uint8_t nb_filelist; bool debug; }; diff --git a/app/test-mldev/test_model_common.c b/app/test-mldev/test_model_common.c new file mode 100644 index 0000000000..b94d46154d --- /dev/null +++ b/app/test-mldev/test_model_common.c @@ -0,0 +1,158 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2022 Marvell. + */ + +#include + +#include +#include +#include + +#include "ml_common.h" +#include "test_model_common.h" + +int +ml_model_load(struct ml_test *test, struct ml_options *opt, struct ml_model *model, uint16_t fid) +{ + struct test_common *t = ml_test_priv(test); + struct rte_ml_model_params model_params; + FILE *fp; + int ret; + + if (model->state == MODEL_LOADED) + return 0; + + if (model->state != MODEL_INITIAL) + return -EINVAL; + + /* read model binary */ + fp = fopen(opt->filelist[fid].model, "r"); + if (fp == NULL) { + ml_err("Failed to open model file : %s\n", opt->filelist[fid].model); + return -1; + } + + fseek(fp, 0, SEEK_END); + model_params.size = ftell(fp); + fseek(fp, 0, SEEK_SET); + + model_params.addr = rte_malloc_socket("ml_model", model_params.size, + t->dev_info.min_align_size, opt->socket_id); + if (model_params.addr == NULL) { + ml_err("Failed to allocate memory for model: %s\n", opt->filelist[fid].model); + fclose(fp); + return -ENOMEM; + } + + if (fread(model_params.addr, 1, model_params.size, fp) != model_params.size) { + ml_err("Failed to read model file : %s\n", opt->filelist[fid].model); + rte_free(model_params.addr); + fclose(fp); + return -1; + } + fclose(fp); + + /* load model to device */ + ret = rte_ml_model_load(opt->dev_id, &model_params, &model->id); + if (ret != 0) { + ml_err("Failed to load model : %s\n", opt->filelist[fid].model); + model->state = MODEL_ERROR; + rte_free(model_params.addr); + return ret; + } + + /* release mz */ + rte_free(model_params.addr); + + /* get model info */ + ret = rte_ml_model_info_get(opt->dev_id, model->id, &model->info); + if (ret != 0) { + ml_err("Failed to get model info : %s\n", opt->filelist[fid].model); + return ret; + } + + model->state = MODEL_LOADED; + + return 0; +} + +int +ml_model_unload(struct ml_test *test, struct ml_options *opt, struct ml_model *model, uint16_t fid) +{ + struct test_common *t = ml_test_priv(test); + int ret; + + RTE_SET_USED(t); + + if (model->state == MODEL_INITIAL) + return 0; + + if (model->state != MODEL_LOADED) + return -EINVAL; + + /* unload model */ + ret = rte_ml_model_unload(opt->dev_id, model->id); + if (ret != 0) { + ml_err("Failed to unload model: %s\n", opt->filelist[fid].model); + model->state = MODEL_ERROR; + return ret; + } + + model->state = MODEL_INITIAL; + + return 0; +} + +int +ml_model_start(struct ml_test *test, struct ml_options *opt, struct ml_model *model, uint16_t fid) +{ + struct test_common *t = ml_test_priv(test); + int ret; + + RTE_SET_USED(t); + + if (model->state == MODEL_STARTED) + return 0; + + if (model->state != MODEL_LOADED) + return -EINVAL; + + /* start model */ + ret = rte_ml_model_start(opt->dev_id, model->id); + if (ret != 0) { + ml_err("Failed to start model : %s\n", opt->filelist[fid].model); + model->state = MODEL_ERROR; + return ret; + } + + model->state = MODEL_STARTED; + + return 0; +} + +int +ml_model_stop(struct ml_test *test, struct ml_options *opt, struct ml_model *model, uint16_t fid) +{ + struct test_common *t = ml_test_priv(test); + int ret; + + RTE_SET_USED(t); + + if (model->state == MODEL_LOADED) + return 0; + + if (model->state != MODEL_STARTED) + return -EINVAL; + + /* stop model */ + ret = rte_ml_model_stop(opt->dev_id, model->id); + if (ret != 0) { + ml_err("Failed to stop model: %s\n", opt->filelist[fid].model); + model->state = MODEL_ERROR; + return ret; + } + + model->state = MODEL_LOADED; + + return 0; +} diff --git a/app/test-mldev/test_model_common.h b/app/test-mldev/test_model_common.h new file mode 100644 index 0000000000..74aec0a797 --- /dev/null +++ b/app/test-mldev/test_model_common.h @@ -0,0 +1,34 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2022 Marvell. + */ + +#ifndef _ML_TEST_MODEL_COMMON_ +#define _ML_TEST_MODEL_COMMON_ + +#include + +#include "test_common.h" + +enum model_state { + MODEL_INITIAL, + MODEL_LOADED, + MODEL_STARTED, + MODEL_ERROR, +}; + +struct ml_model { + uint16_t id; + struct rte_ml_model_info info; + enum model_state state; +}; + +int ml_model_load(struct ml_test *test, struct ml_options *opt, struct ml_model *model, + uint16_t fid); +int ml_model_unload(struct ml_test *test, struct ml_options *opt, struct ml_model *model, + uint16_t fid); +int ml_model_start(struct ml_test *test, struct ml_options *opt, struct ml_model *model, + uint16_t fid); +int ml_model_stop(struct ml_test *test, struct ml_options *opt, struct ml_model *model, + uint16_t fid); + +#endif /* _ML_TEST_MODEL_COMMON_ */ diff --git a/app/test-mldev/test_model_ops.c b/app/test-mldev/test_model_ops.c new file mode 100644 index 0000000000..0202b31190 --- /dev/null +++ b/app/test-mldev/test_model_ops.c @@ -0,0 +1,428 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2022 Marvell. + */ + +#include + +#include +#include +#include + +#include "test_model_ops.h" + +static bool +test_model_ops_cap_check(struct ml_options *opt) +{ + if (!ml_test_cap_check(opt)) + return false; + + return true; +} + +static int +test_model_ops_opt_check(struct ml_options *opt) +{ + uint32_t i; + int ret; + + /* check common opts */ + ret = ml_test_opt_check(opt); + if (ret != 0) + return ret; + + /* check model file availability */ + for (i = 0; i < opt->nb_filelist; i++) { + if (access(opt->filelist[i].model, F_OK) == -1) { + ml_err("Model file not available: id = %u, file = %s", i, + opt->filelist[i].model); + return -ENOENT; + } + } + + return 0; +} + +static void +test_model_ops_opt_dump(struct ml_options *opt) +{ + uint32_t i; + + /* dump common opts */ + ml_test_opt_dump(opt); + + /* dump test specific opts */ + ml_dump_begin("models"); + for (i = 0; i < opt->nb_filelist; i++) + ml_dump_list("model", i, opt->filelist[i].model); + ml_dump_end; +} + +static int +test_model_ops_setup(struct ml_test *test, struct ml_options *opt) +{ + struct test_model_ops *t; + void *test_model_ops; + int ret = 0; + uint32_t i; + + /* allocate model ops test structure */ + test_model_ops = rte_zmalloc_socket(test->name, sizeof(struct test_model_ops), + RTE_CACHE_LINE_SIZE, opt->socket_id); + if (test_model_ops == NULL) { + ml_err("Failed to allocate memory for test_model"); + ret = -ENOMEM; + goto error; + } + test->test_priv = test_model_ops; + t = ml_test_priv(test); + + t->cmn.result = ML_TEST_FAILED; + t->cmn.opt = opt; + + /* get device info */ + ret = rte_ml_dev_info_get(opt->dev_id, &t->cmn.dev_info); + if (ret < 0) { + ml_err("Failed to get device info"); + goto error; + } + + /* set model initial state */ + for (i = 0; i < opt->nb_filelist; i++) + t->model[i].state = MODEL_INITIAL; + + return 0; + +error: + if (test_model_ops != NULL) + rte_free(test_model_ops); + + return ret; +} + +static void +test_model_ops_destroy(struct ml_test *test, struct ml_options *opt) +{ + struct test_model_ops *t; + + RTE_SET_USED(opt); + + t = ml_test_priv(test); + if (t != NULL) + rte_free(t); +} + +static int +test_model_ops_mldev_setup(struct ml_test *test, struct ml_options *opt) +{ + int ret; + + ret = ml_test_device_configure(test, opt); + if (ret != 0) + return ret; + + ret = ml_test_device_start(test, opt); + if (ret != 0) + goto error; + + return 0; + +error: + ml_test_device_close(test, opt); + + return ret; +} + +static int +test_model_ops_mldev_destroy(struct ml_test *test, struct ml_options *opt) +{ + int ret; + + ret = ml_test_device_stop(test, opt); + if (ret != 0) + goto error; + + ret = ml_test_device_close(test, opt); + if (ret != 0) + return ret; + + return 0; + +error: + ml_test_device_close(test, opt); + + return ret; +} + +/* Sub-test A: (load -> start -> stop -> unload) x n */ +static int +test_model_ops_subtest_a(struct ml_test *test, struct ml_options *opt) +{ + struct test_model_ops *t; + int ret = 0; + uint32_t i; + + t = ml_test_priv(test); + + /* load + start + stop + unload */ + for (i = 0; i < opt->nb_filelist; i++) { + ret = ml_model_load(test, opt, &t->model[i], i); + if (ret != 0) + goto error; + + ret = ml_model_start(test, opt, &t->model[i], i); + if (ret != 0) + goto error; + + ret = ml_model_stop(test, opt, &t->model[i], i); + if (ret != 0) + goto error; + + ret = ml_model_unload(test, opt, &t->model[i], i); + if (ret != 0) + goto error; + } + +error: + for (i = 0; i < opt->nb_filelist; i++) + ml_model_stop(test, opt, &t->model[i], i); + + for (i = 0; i < opt->nb_filelist; i++) + ml_model_unload(test, opt, &t->model[i], i); + + return ret; +} + +/* Sub-test B: load x n -> start x n -> stop x n -> unload x n */ +static int +test_model_ops_subtest_b(struct ml_test *test, struct ml_options *opt) +{ + struct test_model_ops *t; + int ret = 0; + uint32_t i; + + t = ml_test_priv(test); + + /* load */ + for (i = 0; i < opt->nb_filelist; i++) { + ret = ml_model_load(test, opt, &t->model[i], i); + if (ret != 0) + goto error; + } + + /* start */ + for (i = 0; i < opt->nb_filelist; i++) { + ret = ml_model_start(test, opt, &t->model[i], i); + if (ret != 0) + goto error; + } + + /* stop */ + for (i = 0; i < opt->nb_filelist; i++) { + ret = ml_model_stop(test, opt, &t->model[i], i); + if (ret != 0) + goto error; + } + + /* unload */ + for (i = 0; i < opt->nb_filelist; i++) { + ret = ml_model_unload(test, opt, &t->model[i], i); + if (ret != 0) + goto error; + } + + return 0; + +error: + for (i = 0; i < opt->nb_filelist; i++) + ml_model_stop(test, opt, &t->model[i], i); + + for (i = 0; i < opt->nb_filelist; i++) + ml_model_unload(test, opt, &t->model[i], i); + + return ret; +} + +/* Sub-test C: load x n + (start + stop) x n + unload x n */ +static int +test_model_ops_subtest_c(struct ml_test *test, struct ml_options *opt) +{ + struct test_model_ops *t; + int ret = 0; + uint32_t i; + + t = ml_test_priv(test); + + /* load */ + for (i = 0; i < opt->nb_filelist; i++) { + ret = ml_model_load(test, opt, &t->model[i], i); + if (ret != 0) + goto error; + } + + /* start + stop */ + for (i = 0; i < opt->nb_filelist; i++) { + ret = ml_model_start(test, opt, &t->model[i], i); + if (ret != 0) + goto error; + + ret = ml_model_stop(test, opt, &t->model[i], i); + if (ret != 0) + goto error; + } + + /* unload */ + for (i = 0; i < opt->nb_filelist; i++) { + ret = ml_model_unload(test, opt, &t->model[i], i); + if (ret != 0) + goto error; + } + + return 0; + +error: + for (i = 0; i < opt->nb_filelist; i++) + ml_model_stop(test, opt, &t->model[i], i); + + for (i = 0; i < opt->nb_filelist; i++) + ml_model_unload(test, opt, &t->model[i], i); + + return ret; +} + +/* Sub-test D: (load + start) x n -> (stop + unload) x n */ +static int +test_model_ops_subtest_d(struct ml_test *test, struct ml_options *opt) +{ + struct test_model_ops *t; + int ret = 0; + uint32_t i; + + t = ml_test_priv(test); + + /* load + start */ + for (i = 0; i < opt->nb_filelist; i++) { + ret = ml_model_load(test, opt, &t->model[i], i); + if (ret != 0) + goto error; + + ret = ml_model_start(test, opt, &t->model[i], i); + if (ret != 0) + goto error; + } + + /* stop + unload */ + for (i = 0; i < opt->nb_filelist; i++) { + ret = ml_model_stop(test, opt, &t->model[i], i); + if (ret != 0) + goto error; + + ret = ml_model_unload(test, opt, &t->model[i], i); + if (ret != 0) + goto error; + } + + return 0; + +error: + for (i = 0; i < opt->nb_filelist; i++) + ml_model_stop(test, opt, &t->model[i], i); + + for (i = 0; i < opt->nb_filelist; i++) + ml_model_unload(test, opt, &t->model[i], i); + + return ret; +} + +static int +test_model_ops_driver(struct ml_test *test, struct ml_options *opt) +{ + struct test_model_ops *t; + int ret = 0; + + t = ml_test_priv(test); + + /* device setup */ + ret = test_model_ops_mldev_setup(test, opt); + if (ret != 0) + return ret; + + printf("\n"); + + /* sub-test A */ + ret = test_model_ops_subtest_a(test, opt); + if (ret != 0) { + printf("Model Ops Sub-test A: " CLRED "%s" CLNRM "\n", "Failed"); + goto error; + } else { + printf("Model Ops Sub-test A: " CLYEL "%s" CLNRM "\n", "Passed"); + } + + /* sub-test B */ + ret = test_model_ops_subtest_b(test, opt); + if (ret != 0) { + printf("Model Ops Sub-test B: " CLRED "%s" CLNRM "\n", "Failed"); + goto error; + } else { + printf("Model Ops Sub-test B: " CLYEL "%s" CLNRM "\n", "Passed"); + } + + /* sub-test C */ + ret = test_model_ops_subtest_c(test, opt); + if (ret != 0) { + printf("Model Ops Sub-test C: " CLRED "%s" CLNRM "\n", "Failed"); + goto error; + } else { + printf("Model Ops Sub-test C: " CLYEL "%s" CLNRM "\n", "Passed"); + } + + /* sub-test D */ + ret = test_model_ops_subtest_d(test, opt); + if (ret != 0) { + printf("Model Ops Sub-test D: " CLRED "%s" CLNRM "\n", "Failed"); + goto error; + } else { + printf("Model Ops Sub-test D: " CLYEL "%s" CLNRM "\n", "Passed"); + } + + printf("\n"); + + /* device destroy */ + ret = test_model_ops_mldev_destroy(test, opt); + if (ret != 0) + return ret; + + t->cmn.result = ML_TEST_SUCCESS; + + return 0; + +error: + test_model_ops_mldev_destroy(test, opt); + + t->cmn.result = ML_TEST_FAILED; + + return ret; +} + +static int +test_model_ops_result(struct ml_test *test, struct ml_options *opt) +{ + struct test_model_ops *t; + + RTE_SET_USED(opt); + + t = ml_test_priv(test); + + return t->cmn.result; +} + +static const struct ml_test_ops model_ops = { + .cap_check = test_model_ops_cap_check, + .opt_check = test_model_ops_opt_check, + .opt_dump = test_model_ops_opt_dump, + .test_setup = test_model_ops_setup, + .test_destroy = test_model_ops_destroy, + .test_driver = test_model_ops_driver, + .test_result = test_model_ops_result, +}; + +ML_TEST_REGISTER(model_ops); diff --git a/app/test-mldev/test_model_ops.h b/app/test-mldev/test_model_ops.h new file mode 100644 index 0000000000..c21f12b788 --- /dev/null +++ b/app/test-mldev/test_model_ops.h @@ -0,0 +1,20 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2022 Marvell. + */ + +#ifndef _ML_TEST_MODEL_OPS_ +#define _ML_TEST_MODEL_OPS_ + +#include + +#include "test_model_common.h" + +struct test_model_ops { + /* common data */ + struct test_common cmn; + + /* test specific data */ + struct ml_model model[ML_TEST_MAX_MODELS]; +} __rte_cache_aligned; + +#endif /* _ML_TEST_MODEL_OPS_ */ diff --git a/doc/guides/tools/img/mldev_model_ops_subtest_a.svg b/doc/guides/tools/img/mldev_model_ops_subtest_a.svg new file mode 100644 index 0000000000..ed12cc5a05 --- /dev/null +++ b/doc/guides/tools/img/mldev_model_ops_subtest_a.svg @@ -0,0 +1,420 @@ + + + + + + + + + + + + + + + + + + Model 0 / Load + + Model 0 / Start + + Model 0 / Stop + + Model 0 / Unload + + + + + Model 1 / Load + + Model 1 / Start + + Model 1 / Unload + + Model 1 / Stop + + + + + Model N / Load + + Model N / Start + + Model N / Stop + + Model N / Unload + + + mldev: model_ops / subtest D + + + diff --git a/doc/guides/tools/img/mldev_model_ops_subtest_b.svg b/doc/guides/tools/img/mldev_model_ops_subtest_b.svg new file mode 100644 index 0000000000..173a2c6c05 --- /dev/null +++ b/doc/guides/tools/img/mldev_model_ops_subtest_b.svg @@ -0,0 +1,423 @@ + + + + + + + + + + + + + + + Model 0 / Load + + Model 1 / Load + + Model N / Load + + + + + Model 0 / Start + + Model 1 / Start + + Model N / Start + + + + + Model 1 / Stop + + Model 0 / Stop + + Model N / Stop + + + + + Model 0 / Unload + + Model 1 / Unload + + Model N / Unload + + + mldev: model_ops / subtest A + + + diff --git a/doc/guides/tools/img/mldev_model_ops_subtest_c.svg b/doc/guides/tools/img/mldev_model_ops_subtest_c.svg new file mode 100644 index 0000000000..f66f146d05 --- /dev/null +++ b/doc/guides/tools/img/mldev_model_ops_subtest_c.svg @@ -0,0 +1,366 @@ + + + + + + + + + + + + + + + Model 0 / Load + + Model 1 / Load + + Model N / Load + + + + + Model 0 / Start + + Model 0 / Stop + + + + Model 0 / Unload + + Model 1 / Unload + + Model N / Unload + + + + Model N / Stop + + Model N / Start + + + mldev: model_ops / subtest C + + + diff --git a/doc/guides/tools/img/mldev_model_ops_subtest_d.svg b/doc/guides/tools/img/mldev_model_ops_subtest_d.svg new file mode 100644 index 0000000000..3e2b89ad25 --- /dev/null +++ b/doc/guides/tools/img/mldev_model_ops_subtest_d.svg @@ -0,0 +1,424 @@ + + + + + + + + + + + + + + + Model 0 / Load + + Model 0 / Start + + Model 1 / Load + + + + + Model 1 / Start + + + Model N / Load + + Model N / Start + + + + Model N / Unload + + Model N / Stop + + + + Model 1 / Stop + + + Model 1 / Unload + + Model 0 / Stop + + Model 0 / Unload + + + mldev: model_ops / subest B + + + diff --git a/doc/guides/tools/testmldev.rst b/doc/guides/tools/testmldev.rst index 1771c49fb4..b8a2a16ca2 100644 --- a/doc/guides/tools/testmldev.rst +++ b/doc/guides/tools/testmldev.rst @@ -44,13 +44,17 @@ The following are the command-line options supported by the test application. * ``--test `` - Name of the test to execute. ML tests supported include device tests. Test name should be - one of the following supported tests. + Name of the test to execute. ML tests are divided into two groups, Device and Model + tests. Test name should be one of the following supported tests. **ML Device Tests** :: device_ops + **ML Model Tests** :: + + model_ops + * ``--dev_id `` Set the device id of the ML device to be used for the test. Default value is `0`. @@ -59,6 +63,12 @@ The following are the command-line options supported by the test application. Set the socket id of the application resources. Default value is `SOCKET_ID_ANY`. +* ``--models `` + + Set the list of model files to be used for the tests. Application expects the + ``model_list`` in comma separated form (i.e. ``--models model_A.bin,model_B.bin``). + Maximum number of models supported by the test is ``8``. + * ``--debug`` Enable the tests to run in debug mode. @@ -103,6 +113,89 @@ Command to run device_ops test: --test=device_ops +ML Model Tests +------------------------- + +Model tests are functional tests to validate ML model APIs. Model tests validate the functioning +of APIs to load, start, stop and unload ML models. + + +Application Options +~~~~~~~~~~~~~~~~~~~ + +Supported command line options for the `model_ops` test are following:: + + --debug + --test + --dev_id + --socket_id + --models + + +List of model files to be used for the `model_ops` test can be specified through the option +``--models `` as a comma separated list. Maximum number of models supported in +the test is `8`. + +.. Note:: + + * The ``--models `` is a mandatory option for running this test. + * Options not supported by the test are ignored if specified. + + +MODEL_OPS Test +~~~~~~~~~~~~~~ + +The test is a collection of multiple sub-tests, each with a different order of slow-path +operations when handling with `N` number of models. + + +**Sub-test A:** executes the sequence of load / start / stop / unload for a model in order, +followed by next model. +.. _figure_mldev_model_ops_subtest_a: + +.. figure:: img/mldev_model_ops_subtest_a.* + + Execution sequence of model_ops subtest A. + + +**Sub-test B:** executes load for all models, followed by a start for all models. Upon successful +start of all models, stop is invoked for all models followed by unload. +.. _figure_mldev_model_ops_subtest_b: + +.. figure:: img/mldev_model_ops_subtest_b.* + + Execution sequence of model_ops subtest B. + + +**Sub-test C:** loads all models, followed by a start and stop of all models in order. Upon +completion of stop, unload is invoked for all models. +.. _figure_mldev_model_ops_subtest_c: + +.. figure:: img/mldev_model_ops_subtest_c.* + + Execution sequence of model_ops subtest C. + + +**Sub-test D:** executes load and start for all models available. Upon successful start of all +models, stop and stop is executed for the models. +.. _figure_mldev_model_ops_subtest_d: + +.. figure:: img/mldev_model_ops_subtest_d.* + + Execution sequence of model_ops subtest D. + + +Example +^^^^^^^ + +Command to run model_ops test: + +.. code-block:: console + + sudo /app/dpdk-test-mldev -c 0xf -a -- \ + --test=model_ops --models model_1.bin,model_2.bin,model_3.bin, model_4.bin + + Debug mode ---------- -- 2.17.1