From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <dev-bounces@dpdk.org> Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A3DBD41D3D; Fri, 10 Mar 2023 09:10:03 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8F94D40A81; Fri, 10 Mar 2023 09:10:03 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 6677840150 for <dev@dpdk.org>; Fri, 10 Mar 2023 09:10:01 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 32A7ajpB009832 for <dev@dpdk.org>; Fri, 10 Mar 2023 00:10:00 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=uZwyLzccoqJ1VloXjWS2CLPJ/xOf175IBQDc5g9h3Cs=; b=Z9gGF6E60iApodB7cdAbLYygk8fSVBnvuWPwnZ7q9o9bpOCSmGXxWoKD1JaFjHrMfpIX OMaBQrK7VTDvuoHgrPAUYx6wjm+PwOB/Ekrh0CW2YNA0wmpRKFo9G4RZF50Odfwz39ZP v18SddeotySqyR+kZFEfBWy9+jU2MgAT3w252c0wT7TZe36pqqdbrBbn35gi9fWgkkFN +R/cMDuEuecL2Gqk+gaHBIMQ0NfK2iBy5cm8Gy6bktKc/6qNCYjtBpZPyMeFcN/sAxSW tunlIpXxdxzTofUvgrgqQdjBJy9ue8FjIw4c8lqtfczMIX+1Cp2rRrT+KQw6L9t0+4+u dw== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3p7n7dhy4p-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for <dev@dpdk.org>; Fri, 10 Mar 2023 00:10:00 -0800 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.42; Fri, 10 Mar 2023 00:09:39 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.42 via Frontend Transport; Fri, 10 Mar 2023 00:09:39 -0800 Received: from ml-host-33.caveonetworks.com (unknown [10.110.143.233]) by maili.marvell.com (Postfix) with ESMTP id C2FFB3F708D; Fri, 10 Mar 2023 00:09:39 -0800 (PST) From: Srikanth Yalavarthi <syalavarthi@marvell.com> To: CC: <dev@dpdk.org>, <syalavarthi@marvell.com>, <sshankarnara@marvell.com>, <jerinj@marvell.com>, <aprabhu@marvell.com>, <ptakkar@marvell.com>, <pshukla@marvell.com> Subject: [PATCH v5 00/12] Implementation of mldev test application Date: Fri, 10 Mar 2023 00:09:22 -0800 Message-ID: <20230310080935.2460-1-syalavarthi@marvell.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20221129070746.20396-1-syalavarthi@marvell.com> References: <20221129070746.20396-1-syalavarthi@marvell.com> MIME-Version: 1.0 Content-Type: text/plain X-Proofpoint-GUID: pAx-uhkDfcXLjw_fKKuFTeCX8BB1StID X-Proofpoint-ORIG-GUID: pAx-uhkDfcXLjw_fKKuFTeCX8BB1StID X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-10_02,2023-03-09_01,2023-02-09_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org Machine learning device APIs test application ============================================= This series of patches introduces a test application for machine learning device APIs. A test framework is implemented with multiple test enabled, to validate the device, model and fast-path functions. New tests can be added using the test framework. List of tests supported ----------------------- 1) device_ops: Test case to validate device re-configuration 2) model_ops: Collection of 4 sub-tests to validate model slow APIs. Each sub-test would invoke the slow path model APIs (load / start / stop / unload) in different order. 3) inference_ordered: Test case to validate execution of end-to-end inferences on ML device with one active model at a time. This test can execute inference requests for multiple models, with inferences for a model executed after completion of inferences for a previously loaded model. 4) inference_interleave: Test case to validate end-to-end inferences with multiple active models concurrently. This case would work as a stress test to validate ML device. Options supported the by tests include burst size for enqueuing and dequeuing inference requests, enabling multiple queue pairs with a user specified value for queue size. Support is also enabled for batch inferencing, output validation and statistics. v5: * Cleanup of header includes * Addressed issues with unclean patches * Changed input/output size variables to uint32_t * Rebase over main branch v4: * Updated model_id as uint16_t * Updated license info in SVG files * Updated release notes v3: * Code rebase Srikanth Yalavarthi (12): app/mldev: implement test framework for mldev app/mldev: add common test functions app/mldev: add test case to validate device ops app/mldev: add test case to validate model ops app/mldev: add ordered inference test case app/mldev: add test case to interleave inferences app/mldev: enable support for burst inferences app/mldev: enable support for queue pairs and size app/mldev: enable support for inference batches app/mldev: enable support for inference validation app/mldev: enable reporting stats in mldev app app/mldev: add documentation for mldev test cases MAINTAINERS | 2 + app/meson.build | 1 + app/test-mldev/meson.build | 24 + app/test-mldev/ml_common.h | 29 + app/test-mldev/ml_main.c | 113 ++ app/test-mldev/ml_options.c | 325 +++++ app/test-mldev/ml_options.h | 57 + app/test-mldev/ml_test.c | 41 + app/test-mldev/ml_test.h | 76 ++ app/test-mldev/parser.c | 380 ++++++ app/test-mldev/parser.h | 55 + app/test-mldev/test_common.c | 136 ++ app/test-mldev/test_common.h | 27 + app/test-mldev/test_device_ops.c | 228 ++++ app/test-mldev/test_device_ops.h | 17 + app/test-mldev/test_inference_common.c | 1122 +++++++++++++++++ app/test-mldev/test_inference_common.h | 74 ++ app/test-mldev/test_inference_interleave.c | 118 ++ app/test-mldev/test_inference_ordered.c | 116 ++ app/test-mldev/test_model_common.c | 164 +++ app/test-mldev/test_model_common.h | 46 + app/test-mldev/test_model_ops.c | 428 +++++++ app/test-mldev/test_model_ops.h | 20 + doc/guides/rel_notes/release_23_03.rst | 8 + .../tools/img/mldev_inference_interleave.svg | 669 ++++++++++ .../tools/img/mldev_inference_ordered.svg | 528 ++++++++ .../tools/img/mldev_model_ops_subtest_a.svg | 420 ++++++ .../tools/img/mldev_model_ops_subtest_b.svg | 423 +++++++ .../tools/img/mldev_model_ops_subtest_c.svg | 366 ++++++ .../tools/img/mldev_model_ops_subtest_d.svg | 424 +++++++ doc/guides/tools/index.rst | 1 + doc/guides/tools/testmldev.rst | 441 +++++++ 32 files changed, 6879 insertions(+) create mode 100644 app/test-mldev/meson.build create mode 100644 app/test-mldev/ml_common.h create mode 100644 app/test-mldev/ml_main.c create mode 100644 app/test-mldev/ml_options.c create mode 100644 app/test-mldev/ml_options.h create mode 100644 app/test-mldev/ml_test.c create mode 100644 app/test-mldev/ml_test.h create mode 100644 app/test-mldev/parser.c create mode 100644 app/test-mldev/parser.h create mode 100644 app/test-mldev/test_common.c create mode 100644 app/test-mldev/test_common.h create mode 100644 app/test-mldev/test_device_ops.c create mode 100644 app/test-mldev/test_device_ops.h create mode 100644 app/test-mldev/test_inference_common.c create mode 100644 app/test-mldev/test_inference_common.h create mode 100644 app/test-mldev/test_inference_interleave.c create mode 100644 app/test-mldev/test_inference_ordered.c create mode 100644 app/test-mldev/test_model_common.c create mode 100644 app/test-mldev/test_model_common.h create mode 100644 app/test-mldev/test_model_ops.c create mode 100644 app/test-mldev/test_model_ops.h create mode 100644 doc/guides/tools/img/mldev_inference_interleave.svg create mode 100644 doc/guides/tools/img/mldev_inference_ordered.svg create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_a.svg create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_b.svg create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_c.svg create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_d.svg create mode 100644 doc/guides/tools/testmldev.rst -- 2.17.1