From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
by inbox.dpdk.org (Postfix) with ESMTP id 9AB5EA0093;
Tue, 29 Nov 2022 09:21:56 +0100 (CET)
Received: from mails.dpdk.org (localhost [127.0.0.1])
by mails.dpdk.org (Postfix) with ESMTP id 5F6A442D3A;
Tue, 29 Nov 2022 09:21:24 +0100 (CET)
Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com
[67.231.156.173])
by mails.dpdk.org (Postfix) with ESMTP id 4A2B542BD9
for <dev@dpdk.org>; Tue, 29 Nov 2022 09:21:18 +0100 (CET)
Received: from pps.filterd (m0045851.ppops.net [127.0.0.1])
by mx0b-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id
2AT7sjLx020517 for <dev@dpdk.org>; Tue, 29 Nov 2022 00:21:17 -0800
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com;
h=from : to : cc :
subject : date : message-id : in-reply-to : references : mime-version :
content-type; s=pfpt0220; bh=Xi7pl1cHzE0Utv0E3WfPGCb+I0qTnvjBaMMOww/JGCE=;
b=Hk/EoUynpc2xDIuXQlz4+ONO2xbrIL2YQMCYHjmzWkVPujW7/pONAf8fzRLPiPgko2EF
P4md7a9qbJ0UWCd0W167Ypspy3+KbCoTDJbMbjAUhJfO/0ZjqWs6BU94AxvMaj7CGk0E
UU80j7unfYQn0Pweh2Vb75WVghbq+xCNiAmdPbgYcq9+lwWjb7IyhUtUu4ipo689pCED
fYILVKeJWHACl2jF8ZGG+0hVtNFdpu9MFT45lB0LUK+kcNrskA3bW0X3lzGYZ0bdPuwp
Lz91thMKprR+PEmEHrYUY2Qvpl+yIAbIOgKEF5Aohb+a4IRQYw2i0k/YKAMFIhujSuqq dg==
Received: from dc5-exch01.marvell.com ([199.233.59.181])
by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3m3k6wac9h-3
(version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT)
for <dev@dpdk.org>; Tue, 29 Nov 2022 00:21:17 -0800
Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com
(10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2;
Tue, 29 Nov 2022 00:21:16 -0800
Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com
(10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend
Transport; Tue, 29 Nov 2022 00:21:16 -0800
Received: from ml-host-33.caveonetworks.com (unknown [10.110.143.233])
by maili.marvell.com (Postfix) with ESMTP id C29E53F7086;
Tue, 29 Nov 2022 00:21:15 -0800 (PST)
From: Srikanth Yalavarthi <syalavarthi@marvell.com>
To: Srikanth Yalavarthi <syalavarthi@marvell.com>
CC: <dev@dpdk.org>, <sshankarnara@marvell.com>, <jerinj@marvell.com>
Subject: [PATCH v2 09/12] app/mldev: enable support for inference batches
Date: Tue, 29 Nov 2022 00:21:06 -0800
Message-ID: <20221129082109.6809-9-syalavarthi@marvell.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <20221129082109.6809-1-syalavarthi@marvell.com>
References: <20221129070746.20396-2-syalavarthi@marvell.com>
<20221129082109.6809-1-syalavarthi@marvell.com>
MIME-Version: 1.0
Content-Type: text/plain
X-Proofpoint-GUID: MK5p6DbAeBnRitO_Ps9r8-tGQ4orOCeN
X-Proofpoint-ORIG-GUID: MK5p6DbAeBnRitO_Ps9r8-tGQ4orOCeN
X-Proofpoint-Virus-Version: vendor=baseguard
engine=ICAP:2.0.219,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1
definitions=2022-11-29_06,2022-11-28_02,2022-06-22_01
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Enabled support to execute multiple batches of inferences
per each enqueue request. Input and reference for the test
should be appropriately provided for multi-batch run. Number
of batches can be specified through "--batches" option.
Signed-off-by: Srikanth Yalavarthi <syalavarthi@marvell.com>
---
app/test-mldev/ml_options.c | 15 ++++++++++++---
app/test-mldev/ml_options.h | 2 ++
app/test-mldev/test_inference_common.c | 22 +++++++++++++---------
app/test-mldev/test_model_common.c | 6 ++++++
app/test-mldev/test_model_common.h | 1 +
5 files changed, 34 insertions(+), 12 deletions(-)
diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c
index d5182a1701..331ec1704c 100644
--- a/app/test-mldev/ml_options.c
+++ b/app/test-mldev/ml_options.c
@@ -33,6 +33,7 @@ ml_options_default(struct ml_options *opt)
opt->burst_size = 1;
opt->queue_pairs = 1;
opt->queue_size = 1;
+ opt->batches = 0;
opt->debug = false;
}
@@ -176,6 +177,12 @@ ml_parse_queue_size(struct ml_options *opt, const char *arg)
return parser_read_uint16(&opt->queue_size, arg);
}
+static int
+ml_parse_batches(struct ml_options *opt, const char *arg)
+{
+ return parser_read_uint16(&opt->batches, arg);
+}
+
static void
ml_dump_test_options(const char *testname)
{
@@ -196,7 +203,8 @@ ml_dump_test_options(const char *testname)
"\t\t--repetitions : number of inference repetitions\n"
"\t\t--burst_size : inference burst size\n"
"\t\t--queue_pairs : number of queue pairs to create\n"
- "\t\t--queue_size : size fo queue-pair\n");
+ "\t\t--queue_size : size fo queue-pair\n"
+ "\t\t--batches : number of batches of input\n");
printf("\n");
}
}
@@ -220,7 +228,8 @@ static struct option lgopts[] = {
{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0},
{ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0},
{ML_BURST_SIZE, 1, 0, 0}, {ML_QUEUE_PAIRS, 1, 0, 0}, {ML_QUEUE_SIZE, 1, 0, 0},
- {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}};
+ {ML_BATCHES, 1, 0, 0}, {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0},
+ {NULL, 0, 0, 0}};
static int
ml_opts_parse_long(int opt_idx, struct ml_options *opt)
@@ -232,7 +241,7 @@ ml_opts_parse_long(int opt_idx, struct ml_options *opt)
{ML_SOCKET_ID, ml_parse_socket_id}, {ML_MODELS, ml_parse_models},
{ML_FILELIST, ml_parse_filelist}, {ML_REPETITIONS, ml_parse_repetitions},
{ML_BURST_SIZE, ml_parse_burst_size}, {ML_QUEUE_PAIRS, ml_parse_queue_pairs},
- {ML_QUEUE_SIZE, ml_parse_queue_size},
+ {ML_QUEUE_SIZE, ml_parse_queue_size}, {ML_BATCHES, ml_parse_batches},
};
for (i = 0; i < RTE_DIM(parsermap); i++) {
diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h
index 6bfef1b979..d23e842895 100644
--- a/app/test-mldev/ml_options.h
+++ b/app/test-mldev/ml_options.h
@@ -22,6 +22,7 @@
#define ML_BURST_SIZE ("burst_size")
#define ML_QUEUE_PAIRS ("queue_pairs")
#define ML_QUEUE_SIZE ("queue_size")
+#define ML_BATCHES ("batches")
#define ML_DEBUG ("debug")
#define ML_HELP ("help")
@@ -41,6 +42,7 @@ struct ml_options {
uint16_t burst_size;
uint16_t queue_pairs;
uint16_t queue_size;
+ uint16_t batches;
bool debug;
};
diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c
index d680d68f3d..4e29f6c7eb 100644
--- a/app/test-mldev/test_inference_common.c
+++ b/app/test-mldev/test_inference_common.c
@@ -56,7 +56,7 @@ ml_enqueue_single(void *arg)
goto retry;
op->model_id = t->model[fid].id;
- op->nb_batches = t->model[fid].info.batch_size;
+ op->nb_batches = t->model[fid].nb_batches;
op->mempool = t->op_pool;
op->input.addr = req->input;
@@ -168,7 +168,7 @@ ml_enqueue_burst(void *arg)
for (i = 0; i < ops_count; i++) {
args->enq_ops[i]->model_id = t->model[fid].id;
- args->enq_ops[i]->nb_batches = t->model[fid].info.batch_size;
+ args->enq_ops[i]->nb_batches = t->model[fid].nb_batches;
args->enq_ops[i]->mempool = t->op_pool;
args->enq_ops[i]->input.addr = args->reqs[i]->input;
@@ -363,6 +363,11 @@ test_inference_opt_dump(struct ml_options *opt)
ml_dump("queue_pairs", "%u", opt->queue_pairs);
ml_dump("queue_size", "%u", opt->queue_size);
+ if (opt->batches == 0)
+ ml_dump("batches", "%u (default)", opt->batches);
+ else
+ ml_dump("batches", "%u", opt->batches);
+
ml_dump_begin("filelist");
for (i = 0; i < opt->nb_filelist; i++) {
ml_dump_list("model", i, opt->filelist[i].model);
@@ -531,8 +536,8 @@ ml_request_initialize(struct rte_mempool *mp, void *opaque, void *obj, unsigned
req->niters = 0;
/* quantize data */
- rte_ml_io_quantize(t->cmn.opt->dev_id, t->model[t->fid].id,
- t->model[t->fid].info.batch_size, t->model[t->fid].input, req->input);
+ rte_ml_io_quantize(t->cmn.opt->dev_id, t->model[t->fid].id, t->model[t->fid].nb_batches,
+ t->model[t->fid].input, req->input);
}
int
@@ -550,7 +555,7 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, int16_t f
int ret;
/* get input buffer size */
- ret = rte_ml_io_input_size_get(opt->dev_id, t->model[fid].id, t->model[fid].info.batch_size,
+ ret = rte_ml_io_input_size_get(opt->dev_id, t->model[fid].id, t->model[fid].nb_batches,
&t->model[fid].inp_qsize, &t->model[fid].inp_dsize);
if (ret != 0) {
ml_err("Failed to get input size, model : %s\n", opt->filelist[fid].model);
@@ -558,9 +563,8 @@ ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, int16_t f
}
/* get output buffer size */
- ret = rte_ml_io_output_size_get(opt->dev_id, t->model[fid].id,
- t->model[fid].info.batch_size, &t->model[fid].out_qsize,
- &t->model[fid].out_dsize);
+ ret = rte_ml_io_output_size_get(opt->dev_id, t->model[fid].id, t->model[fid].nb_batches,
+ &t->model[fid].out_qsize, &t->model[fid].out_dsize);
if (ret != 0) {
ml_err("Failed to get input size, model : %s\n", opt->filelist[fid].model);
return ret;
@@ -705,7 +709,7 @@ ml_request_finish(struct rte_mempool *mp, void *opaque, void *obj, unsigned int
return;
t->nb_used++;
- rte_ml_io_dequantize(t->cmn.opt->dev_id, model->id, t->model[req->fid].info.batch_size,
+ rte_ml_io_dequantize(t->cmn.opt->dev_id, model->id, t->model[req->fid].nb_batches,
req->output, model->output);
}
diff --git a/app/test-mldev/test_model_common.c b/app/test-mldev/test_model_common.c
index 5368be17fe..51260c0789 100644
--- a/app/test-mldev/test_model_common.c
+++ b/app/test-mldev/test_model_common.c
@@ -75,6 +75,12 @@ ml_model_load(struct ml_test *test, struct ml_options *opt, struct ml_model *mod
return ret;
}
+ /* Update number of batches */
+ if (opt->batches == 0)
+ model->nb_batches = model->info.batch_size;
+ else
+ model->nb_batches = opt->batches;
+
model->state = MODEL_LOADED;
return 0;
diff --git a/app/test-mldev/test_model_common.h b/app/test-mldev/test_model_common.h
index c45ae80853..dfbf568f0b 100644
--- a/app/test-mldev/test_model_common.h
+++ b/app/test-mldev/test_model_common.h
@@ -33,6 +33,7 @@ struct ml_model {
uint8_t *output;
struct rte_mempool *io_pool;
+ uint32_t nb_batches;
};
int ml_model_load(struct ml_test *test, struct ml_options *opt, struct ml_model *model,
--
2.17.1