From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6F42C41D3D; Fri, 10 Mar 2023 09:11:28 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0258342D51; Fri, 10 Mar 2023 09:10:17 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 383E1410D0 for ; Fri, 10 Mar 2023 09:10:05 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 32A7ajpO009832 for ; Fri, 10 Mar 2023 00:10:04 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=JI/38CmbQyhHQlExl/U0wBIMDdncDqMq077WimXAlAQ=; b=lFyd6L/7cLSYb5o0WIOr6WdcdNa1AXjP3MuosO2qPJ+JEqPrfGb5ji5/PFkU+EuTik6s q/ImqndvNFyItEJkppUc19WaTmA/C8bxJvqcj6fSVzM73DacRXc23y1HlPstvFk4W5xM 6FtTscZCc3dSTdLAmNzRwnGaEhoYg/wexaFLpz1ZhPpu/I3PA3G+E1uMRpWomr/FgPiu UFwNjQoGPmw2R6p9jYHpNb2skTa1XYLJhkhvcVkueVpFeDk3xTM4RmX1xx05R4jXPLFx I8KWsCbeJMeh6C175BGy0BpCEMYkCGYgasqxJmiBEfIE/s8U2UCz/KOdR++sToOa32nR fA== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3p7n7dhy4p-8 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Fri, 10 Mar 2023 00:10:04 -0800 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.42; Fri, 10 Mar 2023 00:09:41 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.42 via Frontend Transport; Fri, 10 Mar 2023 00:09:42 -0800 Received: from ml-host-33.caveonetworks.com (unknown [10.110.143.233]) by maili.marvell.com (Postfix) with ESMTP id E46FB3F708D; Fri, 10 Mar 2023 00:09:41 -0800 (PST) From: Srikanth Yalavarthi To: Srikanth Yalavarthi CC: , , , , , Subject: [PATCH v5 11/12] app/mldev: enable reporting stats in mldev app Date: Fri, 10 Mar 2023 00:09:33 -0800 Message-ID: <20230310080935.2460-12-syalavarthi@marvell.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230310080935.2460-1-syalavarthi@marvell.com> References: <20221129070746.20396-1-syalavarthi@marvell.com> <20230310080935.2460-1-syalavarthi@marvell.com> MIME-Version: 1.0 Content-Type: text/plain X-Proofpoint-GUID: atSskfVhXoRd8tuL95Vt4onIInyjKmoE X-Proofpoint-ORIG-GUID: atSskfVhXoRd8tuL95Vt4onIInyjKmoE X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-10_02,2023-03-09_01,2023-02-09_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Enable reporting driver xstats and inference end-to-end latency and throughput in mldev inference tests. Reporting of stats can be enabled using "--stats" option. Signed-off-by: Srikanth Yalavarthi --- app/test-mldev/ml_options.c | 22 ++-- app/test-mldev/ml_options.h | 2 + app/test-mldev/test_inference_common.c | 140 +++++++++++++++++++++ app/test-mldev/test_inference_common.h | 8 ++ app/test-mldev/test_inference_interleave.c | 4 + app/test-mldev/test_inference_ordered.c | 1 + 6 files changed, 169 insertions(+), 8 deletions(-) diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c index 9dfa5ec0d8..769c3bffd1 100644 --- a/app/test-mldev/ml_options.c +++ b/app/test-mldev/ml_options.c @@ -30,6 +30,7 @@ ml_options_default(struct ml_options *opt) opt->queue_size = 1; opt->batches = 0; opt->tolerance = 0.0; + opt->stats = false; opt->debug = false; } @@ -216,7 +217,8 @@ ml_dump_test_options(const char *testname) "\t\t--queue_pairs : number of queue pairs to create\n" "\t\t--queue_size : size fo queue-pair\n" "\t\t--batches : number of batches of input\n" - "\t\t--tolerance : maximum tolerance (%%) for output validation\n"); + "\t\t--tolerance : maximum tolerance (%%) for output validation\n" + "\t\t--stats : enable reporting performance statistics\n"); printf("\n"); } } @@ -236,13 +238,12 @@ print_usage(char *program) ml_test_dump_names(ml_dump_test_options); } -static struct option lgopts[] = {{ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, - {ML_SOCKET_ID, 1, 0, 0}, {ML_MODELS, 1, 0, 0}, - {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0}, - {ML_BURST_SIZE, 1, 0, 0}, {ML_QUEUE_PAIRS, 1, 0, 0}, - {ML_QUEUE_SIZE, 1, 0, 0}, {ML_BATCHES, 1, 0, 0}, - {ML_TOLERANCE, 1, 0, 0}, {ML_DEBUG, 0, 0, 0}, - {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}}; +static struct option lgopts[] = { + {ML_TEST, 1, 0, 0}, {ML_DEVICE_ID, 1, 0, 0}, {ML_SOCKET_ID, 1, 0, 0}, + {ML_MODELS, 1, 0, 0}, {ML_FILELIST, 1, 0, 0}, {ML_REPETITIONS, 1, 0, 0}, + {ML_BURST_SIZE, 1, 0, 0}, {ML_QUEUE_PAIRS, 1, 0, 0}, {ML_QUEUE_SIZE, 1, 0, 0}, + {ML_BATCHES, 1, 0, 0}, {ML_TOLERANCE, 1, 0, 0}, {ML_STATS, 0, 0, 0}, + {ML_DEBUG, 0, 0, 0}, {ML_HELP, 0, 0, 0}, {NULL, 0, 0, 0}}; static int ml_opts_parse_long(int opt_idx, struct ml_options *opt) @@ -277,6 +278,11 @@ ml_options_parse(struct ml_options *opt, int argc, char **argv) while ((opts = getopt_long(argc, argv, "", lgopts, &opt_idx)) != EOF) { switch (opts) { case 0: /* parse long options */ + if (!strcmp(lgopts[opt_idx].name, "stats")) { + opt->stats = true; + break; + } + if (!strcmp(lgopts[opt_idx].name, "debug")) { opt->debug = true; break; diff --git a/app/test-mldev/ml_options.h b/app/test-mldev/ml_options.h index 7f3db29656..beb0fe69c6 100644 --- a/app/test-mldev/ml_options.h +++ b/app/test-mldev/ml_options.h @@ -23,6 +23,7 @@ #define ML_QUEUE_SIZE ("queue_size") #define ML_BATCHES ("batches") #define ML_TOLERANCE ("tolerance") +#define ML_STATS ("stats") #define ML_DEBUG ("debug") #define ML_HELP ("help") @@ -45,6 +46,7 @@ struct ml_options { uint16_t queue_size; uint16_t batches; float tolerance; + bool stats; bool debug; }; diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c index 87e9efbb37..46bd20f1a5 100644 --- a/app/test-mldev/test_inference_common.c +++ b/app/test-mldev/test_inference_common.c @@ -6,6 +6,7 @@ #include #include +#include #include #include #include @@ -37,6 +38,17 @@ } \ } while (0) +static void +print_line(uint16_t len) +{ + uint16_t i; + + for (i = 0; i < len; i++) + printf("-"); + + printf("\n"); +} + /* Enqueue inference requests with burst size equal to 1 */ static int ml_enqueue_single(void *arg) @@ -46,6 +58,7 @@ ml_enqueue_single(void *arg) struct rte_ml_op *op = NULL; struct ml_core_args *args; uint64_t model_enq = 0; + uint64_t start_cycle; uint32_t burst_enq; uint32_t lcore_id; uint16_t fid; @@ -53,6 +66,7 @@ ml_enqueue_single(void *arg) lcore_id = rte_lcore_id(); args = &t->args[lcore_id]; + args->start_cycles = 0; model_enq = 0; if (args->nb_reqs == 0) @@ -88,10 +102,12 @@ ml_enqueue_single(void *arg) req->fid = fid; enqueue_req: + start_cycle = rte_get_tsc_cycles(); burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, args->qp_id, &op, 1); if (burst_enq == 0) goto enqueue_req; + args->start_cycles += start_cycle; fid++; if (likely(fid <= args->end_fid)) goto next_model; @@ -115,10 +131,12 @@ ml_dequeue_single(void *arg) uint64_t total_deq = 0; uint8_t nb_filelist; uint32_t burst_deq; + uint64_t end_cycle; uint32_t lcore_id; lcore_id = rte_lcore_id(); args = &t->args[lcore_id]; + args->end_cycles = 0; nb_filelist = args->end_fid - args->start_fid + 1; if (args->nb_reqs == 0) @@ -126,9 +144,11 @@ ml_dequeue_single(void *arg) dequeue_req: burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, args->qp_id, &op, 1); + end_cycle = rte_get_tsc_cycles(); if (likely(burst_deq == 1)) { total_deq += burst_deq; + args->end_cycles += end_cycle; if (unlikely(op->status == RTE_ML_OP_STATUS_ERROR)) { rte_ml_op_error_get(t->cmn.opt->dev_id, op, &error); ml_err("error_code = 0x%" PRIx64 ", error_message = %s\n", error.errcode, @@ -151,6 +171,7 @@ ml_enqueue_burst(void *arg) { struct test_inference *t = ml_test_priv((struct ml_test *)arg); struct ml_core_args *args; + uint64_t start_cycle; uint16_t ops_count; uint64_t model_enq; uint16_t burst_enq; @@ -163,6 +184,7 @@ ml_enqueue_burst(void *arg) lcore_id = rte_lcore_id(); args = &t->args[lcore_id]; + args->start_cycles = 0; model_enq = 0; if (args->nb_reqs == 0) @@ -204,8 +226,10 @@ ml_enqueue_burst(void *arg) pending = ops_count; enqueue_reqs: + start_cycle = rte_get_tsc_cycles(); burst_enq = rte_ml_enqueue_burst(t->cmn.opt->dev_id, args->qp_id, &args->enq_ops[idx], pending); + args->start_cycles += burst_enq * start_cycle; pending = pending - burst_enq; if (pending > 0) { @@ -235,11 +259,13 @@ ml_dequeue_burst(void *arg) uint64_t total_deq = 0; uint16_t burst_deq = 0; uint8_t nb_filelist; + uint64_t end_cycle; uint32_t lcore_id; uint32_t i; lcore_id = rte_lcore_id(); args = &t->args[lcore_id]; + args->end_cycles = 0; nb_filelist = args->end_fid - args->start_fid + 1; if (args->nb_reqs == 0) @@ -248,9 +274,11 @@ ml_dequeue_burst(void *arg) dequeue_burst: burst_deq = rte_ml_dequeue_burst(t->cmn.opt->dev_id, args->qp_id, args->deq_ops, t->cmn.opt->burst_size); + end_cycle = rte_get_tsc_cycles(); if (likely(burst_deq > 0)) { total_deq += burst_deq; + args->end_cycles += burst_deq * end_cycle; for (i = 0; i < burst_deq; i++) { if (unlikely(args->deq_ops[i]->status == RTE_ML_OP_STATUS_ERROR)) { @@ -379,6 +407,7 @@ test_inference_opt_dump(struct ml_options *opt) ml_dump("queue_pairs", "%u", opt->queue_pairs); ml_dump("queue_size", "%u", opt->queue_size); ml_dump("tolerance", "%-7.3f", opt->tolerance); + ml_dump("stats", "%s", (opt->stats ? "true" : "false")); if (opt->batches == 0) ml_dump("batches", "%u (default)", opt->batches); @@ -451,6 +480,11 @@ test_inference_setup(struct ml_test *test, struct ml_options *opt) RTE_CACHE_LINE_SIZE, opt->socket_id); } + for (i = 0; i < RTE_MAX_LCORE; i++) { + t->args[i].start_cycles = 0; + t->args[i].end_cycles = 0; + } + return 0; error: @@ -980,3 +1014,109 @@ ml_inference_launch_cores(struct ml_test *test, struct ml_options *opt, uint16_t return 0; } + +int +ml_inference_stats_get(struct ml_test *test, struct ml_options *opt) +{ + struct test_inference *t = ml_test_priv(test); + uint64_t total_cycles = 0; + uint32_t nb_filelist; + uint64_t throughput; + uint64_t avg_e2e; + uint32_t qp_id; + uint64_t freq; + int ret; + int i; + + if (!opt->stats) + return 0; + + /* get xstats size */ + t->xstats_size = rte_ml_dev_xstats_names_get(opt->dev_id, NULL, 0); + if (t->xstats_size >= 0) { + /* allocate for xstats_map and values */ + t->xstats_map = rte_malloc( + "ml_xstats_map", t->xstats_size * sizeof(struct rte_ml_dev_xstats_map), 0); + if (t->xstats_map == NULL) { + ret = -ENOMEM; + goto error; + } + + t->xstats_values = + rte_malloc("ml_xstats_values", t->xstats_size * sizeof(uint64_t), 0); + if (t->xstats_values == NULL) { + ret = -ENOMEM; + goto error; + } + + ret = rte_ml_dev_xstats_names_get(opt->dev_id, t->xstats_map, t->xstats_size); + if (ret != t->xstats_size) { + printf("Unable to get xstats names, ret = %d\n", ret); + ret = -1; + goto error; + } + + for (i = 0; i < t->xstats_size; i++) + rte_ml_dev_xstats_get(opt->dev_id, &t->xstats_map[i].id, + &t->xstats_values[i], 1); + } + + /* print xstats*/ + printf("\n"); + print_line(80); + printf(" ML Device Extended Statistics\n"); + print_line(80); + for (i = 0; i < t->xstats_size; i++) + printf(" %-64s = %" PRIu64 "\n", t->xstats_map[i].name, t->xstats_values[i]); + print_line(80); + + /* release buffers */ + if (t->xstats_map) + rte_free(t->xstats_map); + + if (t->xstats_values) + rte_free(t->xstats_values); + + /* print end-to-end stats */ + freq = rte_get_tsc_hz(); + for (qp_id = 0; qp_id < RTE_MAX_LCORE; qp_id++) + total_cycles += t->args[qp_id].end_cycles - t->args[qp_id].start_cycles; + avg_e2e = total_cycles / opt->repetitions; + + if (freq == 0) { + avg_e2e = total_cycles / opt->repetitions; + printf(" %-64s = %" PRIu64 "\n", "Average End-to-End Latency (cycles)", avg_e2e); + } else { + avg_e2e = (total_cycles * NS_PER_S) / (opt->repetitions * freq); + printf(" %-64s = %" PRIu64 "\n", "Average End-to-End Latency (ns)", avg_e2e); + } + + /* print inference throughput */ + if (strcmp(opt->test_name, "inference_ordered") == 0) + nb_filelist = 1; + else + nb_filelist = opt->nb_filelist; + + if (freq == 0) { + throughput = (nb_filelist * t->cmn.opt->repetitions * 1000000) / total_cycles; + printf(" %-64s = %" PRIu64 "\n", "Average Throughput (inferences / million cycles)", + throughput); + } else { + throughput = (nb_filelist * t->cmn.opt->repetitions * freq) / total_cycles; + printf(" %-64s = %" PRIu64 "\n", "Average Throughput (inferences / second)", + throughput); + } + + print_line(80); + + return 0; + +error: + if (t->xstats_map) + rte_free(t->xstats_map); + + if (t->xstats_values) + rte_free(t->xstats_values); + + return ret; +} diff --git a/app/test-mldev/test_inference_common.h b/app/test-mldev/test_inference_common.h index 3f7373adf5..6f67e94ef8 100644 --- a/app/test-mldev/test_inference_common.h +++ b/app/test-mldev/test_inference_common.h @@ -27,6 +27,9 @@ struct ml_core_args { struct rte_ml_op **enq_ops; struct rte_ml_op **deq_ops; struct ml_request **reqs; + + uint64_t start_cycles; + uint64_t end_cycles; }; struct test_inference { @@ -45,6 +48,10 @@ struct test_inference { int (*dequeue)(void *arg); struct ml_core_args args[RTE_MAX_LCORE]; + + struct rte_ml_dev_xstats_map *xstats_map; + uint64_t *xstats_values; + int xstats_size; } __rte_cache_aligned; bool test_inference_cap_check(struct ml_options *opt); @@ -62,5 +69,6 @@ void ml_inference_mem_destroy(struct ml_test *test, struct ml_options *opt); int ml_inference_result(struct ml_test *test, struct ml_options *opt, uint16_t fid); int ml_inference_launch_cores(struct ml_test *test, struct ml_options *opt, uint16_t start_fid, uint16_t end_fid); +int ml_inference_stats_get(struct ml_test *test, struct ml_options *opt); #endif /* _ML_TEST_INFERENCE_COMMON_ */ diff --git a/app/test-mldev/test_inference_interleave.c b/app/test-mldev/test_inference_interleave.c index 9cf4cfa197..bd2c286737 100644 --- a/app/test-mldev/test_inference_interleave.c +++ b/app/test-mldev/test_inference_interleave.c @@ -56,7 +56,11 @@ test_inference_interleave_driver(struct ml_test *test, struct ml_options *opt) goto error; ml_inference_iomem_destroy(test, opt, fid); + } + + ml_inference_stats_get(test, opt); + for (fid = 0; fid < opt->nb_filelist; fid++) { ret = ml_model_stop(test, opt, &t->model[fid], fid); if (ret != 0) goto error; diff --git a/app/test-mldev/test_inference_ordered.c b/app/test-mldev/test_inference_ordered.c index 1cd91dc3d3..8992358936 100644 --- a/app/test-mldev/test_inference_ordered.c +++ b/app/test-mldev/test_inference_ordered.c @@ -54,6 +54,7 @@ test_inference_ordered_driver(struct ml_test *test, struct ml_options *opt) goto error; ml_inference_iomem_destroy(test, opt, fid); + ml_inference_stats_get(test, opt); /* stop model */ ret = ml_model_stop(test, opt, &t->model[fid], fid); -- 2.17.1