From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9697141C30; Tue, 7 Feb 2023 16:49:50 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 75A52410DF; Tue, 7 Feb 2023 16:49:50 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 7BC5F4021F for ; Tue, 7 Feb 2023 16:49:49 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 317EbMc3006803 for ; Tue, 7 Feb 2023 07:49:48 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=xNkSrLDLs0F2lZDhk+F4ba2uHqAOIRF2BEuM5NNNzZ4=; b=M5imxqsy5Oah48eVNzCY+tdWPSkFmdsGc6RWBVWCuHRjNvrNuRaLiMUu+MBeS1si0Hwt pimUIKbaPI/2BFY2q5iZsy41YYpzMNMEREtubMTAc9CtFmCqI2GWMwq9xdVOwoS1pQyk 5OlAzH+0o7KN7qFclOfeZX2b3A7F1cekxQmEizjL3qMXar4rDNtbJN7AaI3J1XkWv2Yi D3ncCHcirpXYwle8xaCGYamoPoaMDWtv57HTVb0ylmXL3m6d/ez1/ozxLyfTWajRz8cj u6RsLVO2VU2IVO8CTAfPGIr61MbCKi9BjRp9PUIU47ZecNuJvmeRGCuDvfNiWc6O1IRH iQ== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3nkdyrspmy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 07 Feb 2023 07:49:48 -0800 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.42; Tue, 7 Feb 2023 07:49:46 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.42 via Frontend Transport; Tue, 7 Feb 2023 07:49:46 -0800 Received: from ml-host-33.caveonetworks.com (unknown [10.110.143.233]) by maili.marvell.com (Postfix) with ESMTP id B265B3F7043; Tue, 7 Feb 2023 07:49:46 -0800 (PST) From: Srikanth Yalavarthi To: CC: , , , , , , "Srikanth Yalavarthi" Subject: [PATCH v4 00/12] Implementation of mldev test application Date: Tue, 7 Feb 2023 07:49:31 -0800 Message-ID: <20230207154943.18779-1-syalavarthi@marvell.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20221129070746.20396-1-syalavarthi@marvell.com> References: <20221129070746.20396-1-syalavarthi@marvell.com> MIME-Version: 1.0 Content-Type: text/plain X-Proofpoint-ORIG-GUID: eRanukgig1RUrWulXdEjsbYtn-WExdu1 X-Proofpoint-GUID: eRanukgig1RUrWulXdEjsbYtn-WExdu1 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.930,Hydra:6.0.562,FMLib:17.11.122.1 definitions=2023-02-07_07,2023-02-06_03,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Machine learning device APIs test application ============================================= This series of patches introduces a test application for machine learning device APIs. A test framework is implemented with multiple test enabled, to validate the device, model and fast-path functions. New tests can be added using the test framework. List of tests supported ----------------------- 1) device_ops: Test case to validate device re-configuration 2) model_ops: Collection of 4 sub-tests to validate model slow APIs. Each sub-test would invoke the slow path model APIs (load / start / stop / unload) in different order. 3) inference_ordered: Test case to validate execution of end-to-end inferences on ML device with one active model at a time. This test can execute inference requests for multiple models, with inferences for a model executed after completion of inferences for a previously loaded model. 4) inference_interleave: Test case to validate end-to-end inferences with multiple active models concurrently. This case would work as a stress test to validate ML device. Options supported the by tests include burst size for enqueuing and dequeuing inference requests, enabling multiple queue pairs with a user specified value for queue size. Support is also enabled for batch inferencing, output validation and statistics. v4: * Updated model_id as uint16_t * Updated license info in SVG files * Updated release notes v3: * Code rebase Srikanth Yalavarthi (12): app/mldev: implement test framework for mldev app/mldev: add common test functions app/mldev: add test case to validate device ops app/mldev: add test case to validate model ops app/mldev: add ordered inference test case app/mldev: add test case to interleave inferences app/mldev: enable support for burst inferences app/mldev: enable support for queue pairs and size app/mldev: enable support for inference batches app/mldev: enable support for inference validation app/mldev: enable reporting stats in mldev app app/mldev: add documentation for mldev test cases MAINTAINERS | 2 + app/meson.build | 1 + app/test-mldev/meson.build | 24 + app/test-mldev/ml_common.h | 29 + app/test-mldev/ml_main.c | 118 ++ app/test-mldev/ml_options.c | 331 +++++ app/test-mldev/ml_options.h | 58 + app/test-mldev/ml_test.c | 45 + app/test-mldev/ml_test.h | 75 ++ app/test-mldev/parser.c | 380 ++++++ app/test-mldev/parser.h | 55 + app/test-mldev/test_common.c | 139 +++ app/test-mldev/test_common.h | 27 + app/test-mldev/test_device_ops.c | 234 ++++ app/test-mldev/test_device_ops.h | 17 + app/test-mldev/test_inference_common.c | 1111 +++++++++++++++++ app/test-mldev/test_inference_common.h | 79 ++ app/test-mldev/test_inference_interleave.c | 122 ++ app/test-mldev/test_inference_ordered.c | 120 ++ app/test-mldev/test_model_common.c | 168 +++ app/test-mldev/test_model_common.h | 49 + app/test-mldev/test_model_ops.c | 433 +++++++ app/test-mldev/test_model_ops.h | 21 + doc/guides/rel_notes/release_23_03.rst | 8 + .../tools/img/mldev_inference_interleave.svg | 669 ++++++++++ .../tools/img/mldev_inference_ordered.svg | 528 ++++++++ .../tools/img/mldev_model_ops_subtest_a.svg | 420 +++++++ .../tools/img/mldev_model_ops_subtest_b.svg | 423 +++++++ .../tools/img/mldev_model_ops_subtest_c.svg | 366 ++++++ .../tools/img/mldev_model_ops_subtest_d.svg | 424 +++++++ doc/guides/tools/index.rst | 1 + doc/guides/tools/testmldev.rst | 441 +++++++ 32 files changed, 6918 insertions(+) create mode 100644 app/test-mldev/meson.build create mode 100644 app/test-mldev/ml_common.h create mode 100644 app/test-mldev/ml_main.c create mode 100644 app/test-mldev/ml_options.c create mode 100644 app/test-mldev/ml_options.h create mode 100644 app/test-mldev/ml_test.c create mode 100644 app/test-mldev/ml_test.h create mode 100644 app/test-mldev/parser.c create mode 100644 app/test-mldev/parser.h create mode 100644 app/test-mldev/test_common.c create mode 100644 app/test-mldev/test_common.h create mode 100644 app/test-mldev/test_device_ops.c create mode 100644 app/test-mldev/test_device_ops.h create mode 100644 app/test-mldev/test_inference_common.c create mode 100644 app/test-mldev/test_inference_common.h create mode 100644 app/test-mldev/test_inference_interleave.c create mode 100644 app/test-mldev/test_inference_ordered.c create mode 100644 app/test-mldev/test_model_common.c create mode 100644 app/test-mldev/test_model_common.h create mode 100644 app/test-mldev/test_model_ops.c create mode 100644 app/test-mldev/test_model_ops.h create mode 100644 doc/guides/tools/img/mldev_inference_interleave.svg create mode 100644 doc/guides/tools/img/mldev_inference_ordered.svg create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_a.svg create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_b.svg create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_c.svg create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_d.svg create mode 100644 doc/guides/tools/testmldev.rst -- 2.17.1