From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 780D0A0093; Tue, 29 Nov 2022 08:07:56 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 235654067E; Tue, 29 Nov 2022 08:07:56 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 4D53A4021E for ; Tue, 29 Nov 2022 08:07:55 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2ASNXEDC020462 for ; Mon, 28 Nov 2022 23:07:54 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : mime-version : content-type; s=pfpt0220; bh=2afx0r0uZ+o2OTCiX+Vx79gmUp+oBOcyCEK8WYZkJOY=; b=B0p3hbjsljhaKJURG/V66Ew500hhlLN+eiSV4VCSIYisULEmfJSCxMqrfzpW7fNBNN1c hdfbA/NXs1L7Rs0JgOiviQV8kQPV5yz0ULAF5iiDB8ngC3FLKseOXx9YF/Vs9kGmS5lJ acJ7MWQV0S45ynRHcfn/e6bU3B7+u/JNZRFouDF5GRw7cokvA6IQp9UFz3Ykz/CW8plX nHX+zSw1NGlQ/YQ0lYSq6Nxmx8ZY5x4Wtgri8pkZstbSD7G7DpXg+2KLrZ8XKAVuYz5s nhk1t8xDo35DTiNzGPnBOS55+6456xzWFMBgpEdoMd0BkZnSM+znpg+1jrmdpu+So8RR ww== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3m3k6wa3qh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Mon, 28 Nov 2022 23:07:54 -0800 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Mon, 28 Nov 2022 23:07:52 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Mon, 28 Nov 2022 23:07:52 -0800 Received: from ml-host-33.caveonetworks.com (unknown [10.110.143.233]) by maili.marvell.com (Postfix) with ESMTP id 0BA2E3F7079; Mon, 28 Nov 2022 23:07:52 -0800 (PST) From: Srikanth Yalavarthi To: CC: , , , "Srikanth Yalavarthi" Subject: [PATCH v1 00/12] implement mldev test application Date: Mon, 28 Nov 2022 23:07:34 -0800 Message-ID: <20221129070746.20396-1-syalavarthi@marvell.com> X-Mailer: git-send-email 2.17.1 MIME-Version: 1.0 Content-Type: text/plain X-Proofpoint-GUID: PDutwuBq5N9K12cmLiQnrdnISm61PX-y X-Proofpoint-ORIG-GUID: PDutwuBq5N9K12cmLiQnrdnISm61PX-y X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-11-29_06,2022-11-28_02,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Machine learning device APIs test application ============================================= This series of patches introduces a test application for machine learning device APIs. A test framework is implemented with multiple test enabled, to validate the device, model and fast-path functions. New tests can be added using the test framework. List of tests supported ----------------------- 1) device_ops: Test case to validate device re-configuration 2) model_ops: Collection of 4 sub-tests to validate model slow APIs. Each sub-test would invoke the slow path model APIs (load / start / stop / unload) in different order. 3) inference_ordered: Test case to validate execution of end-to-end inferences on ML device with one active model at a time. This test can execute inference requests for multiple models, with inferences for a model executed after completion of inferences for a previously loaded model. 4) inference_interleave: Test case to validate end-to-end inferences with multiple active models concurrently. This case would work as a stress test to validate ML device. Options supported the by tests include burst size for enqueuing and dequeuing inference requests, enabling multiple queue pairs with a user specified value for queue size. Support is also enabled for batch inferencing, output validation and statistics. Srikanth Yalavarthi (12): app/mldev: implement test framework for mldev app/mldev: add common test functions app/mldev: add test case to validate device ops app/mldev: add test case to validate model ops app/mldev: add ordered inference test case app/mldev: add test case to interleave inferences app/mldev: enable support for burst inferences app/mldev: enable support for queue pairs and size app/mldev: enable support for inference batches app/mldev: enable support for inference validation app/mldev: enable reporting stats in mldev app app/mldev: add documentation for mldev test cases MAINTAINERS | 2 + app/meson.build | 1 + app/test-mldev/meson.build | 24 + app/test-mldev/ml_common.h | 29 + app/test-mldev/ml_main.c | 118 ++ app/test-mldev/ml_options.c | 331 +++++ app/test-mldev/ml_options.h | 58 + app/test-mldev/ml_test.c | 45 + app/test-mldev/ml_test.h | 75 ++ app/test-mldev/parser.c | 380 ++++++ app/test-mldev/parser.h | 55 + app/test-mldev/test_common.c | 139 ++ app/test-mldev/test_common.h | 27 + app/test-mldev/test_device_ops.c | 234 ++++ app/test-mldev/test_device_ops.h | 17 + app/test-mldev/test_inference_common.c | 1126 +++++++++++++++++ app/test-mldev/test_inference_common.h | 79 ++ app/test-mldev/test_inference_interleave.c | 122 ++ app/test-mldev/test_inference_ordered.c | 120 ++ app/test-mldev/test_model_common.c | 168 +++ app/test-mldev/test_model_common.h | 49 + app/test-mldev/test_model_ops.c | 433 +++++++ app/test-mldev/test_model_ops.h | 21 + .../tools/img/mldev_inference_interleave.svg | 667 ++++++++++ .../tools/img/mldev_inference_ordered.svg | 526 ++++++++ .../tools/img/mldev_model_ops_subtest_a.svg | 418 ++++++ .../tools/img/mldev_model_ops_subtest_b.svg | 421 ++++++ .../tools/img/mldev_model_ops_subtest_c.svg | 364 ++++++ .../tools/img/mldev_model_ops_subtest_d.svg | 422 ++++++ doc/guides/tools/index.rst | 1 + doc/guides/tools/testmldev.rst | 441 +++++++ 31 files changed, 6913 insertions(+) create mode 100644 app/test-mldev/meson.build create mode 100644 app/test-mldev/ml_common.h create mode 100644 app/test-mldev/ml_main.c create mode 100644 app/test-mldev/ml_options.c create mode 100644 app/test-mldev/ml_options.h create mode 100644 app/test-mldev/ml_test.c create mode 100644 app/test-mldev/ml_test.h create mode 100644 app/test-mldev/parser.c create mode 100644 app/test-mldev/parser.h create mode 100644 app/test-mldev/test_common.c create mode 100644 app/test-mldev/test_common.h create mode 100644 app/test-mldev/test_device_ops.c create mode 100644 app/test-mldev/test_device_ops.h create mode 100644 app/test-mldev/test_inference_common.c create mode 100644 app/test-mldev/test_inference_common.h create mode 100644 app/test-mldev/test_inference_interleave.c create mode 100644 app/test-mldev/test_inference_ordered.c create mode 100644 app/test-mldev/test_model_common.c create mode 100644 app/test-mldev/test_model_common.h create mode 100644 app/test-mldev/test_model_ops.c create mode 100644 app/test-mldev/test_model_ops.h create mode 100644 doc/guides/tools/img/mldev_inference_interleave.svg create mode 100644 doc/guides/tools/img/mldev_inference_ordered.svg create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_a.svg create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_b.svg create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_c.svg create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_d.svg create mode 100644 doc/guides/tools/testmldev.rst -- 2.17.1