From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <dev-bounces@dpdk.org> Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E872141EB5; Thu, 16 Mar 2023 22:14:44 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 72ED742DB7; Thu, 16 Mar 2023 22:14:44 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 90D4640DDC for <dev@dpdk.org>; Thu, 16 Mar 2023 22:14:42 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 32GJMmie016125 for <dev@dpdk.org>; Thu, 16 Mar 2023 14:14:41 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=G8GVLMdT68dvLC3FRVCPSOPgj51oMQF/apXLo5OXWro=; b=BxfU0M7N4Ck0WAV7bi4CjP2CGlPfqhYIrFQpCee4IgFDeamMXc6G2tbXeqg8mjDY/UTY ml0355nr9KHHboPp10dvkvNG695MXe0Of0/DNla1vht6LjhK7et1/kmIEcQeOgPxw9rA Ob4lDpu6CtUCGXscVTH1gweZH6a9Xa0Tm357aqYFNoF6bWemLV4TXyVDLRoOaWrT7SGa 08WrqBqqwPW8bpAzcV78UtjzaDUJu7UxAlHgtUYqnyuQLKdukyf3IqyR6K2clktEv1x9 EMNU92r/Op5ULSnHEON5pKpBnVo2AgUjBticyFVXZhzkW7h0SB3xr9wk1lcQThEeLj1x pg== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3pbs2u3mk5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for <dev@dpdk.org>; Thu, 16 Mar 2023 14:14:41 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.42; Thu, 16 Mar 2023 14:14:39 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.42 via Frontend Transport; Thu, 16 Mar 2023 14:14:39 -0700 Received: from ml-host-33.caveonetworks.com (unknown [10.110.143.233]) by maili.marvell.com (Postfix) with ESMTP id 4E2753F7063; Thu, 16 Mar 2023 14:14:39 -0700 (PDT) From: Srikanth Yalavarthi <syalavarthi@marvell.com> To: CC: <dev@dpdk.org>, <syalavarthi@marvell.com>, <sshankarnara@marvell.com>, <jerinj@marvell.com>, <aprabhu@marvell.com>, <ptakkar@marvell.com>, <pshukla@marvell.com> Subject: [PATCH v7 00/11] Implementation of mldev test application Date: Thu, 16 Mar 2023 14:14:23 -0700 Message-ID: <20230316211434.13409-1-syalavarthi@marvell.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20221129070746.20396-1-syalavarthi@marvell.com> References: <20221129070746.20396-1-syalavarthi@marvell.com> MIME-Version: 1.0 Content-Type: text/plain X-Proofpoint-GUID: BVFqC8i-nnwmrInuNs8g6t1DIge6bPtO X-Proofpoint-ORIG-GUID: BVFqC8i-nnwmrInuNs8g6t1DIge6bPtO X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-16_14,2023-03-16_02,2023-02-09_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org Machine learning device APIs test application ============================================= This series of patches introduces a test application for machine learning device APIs. A test framework is implemented with multiple test enabled, to validate the device, model and fast-path functions. New tests can be added using the test framework. List of tests supported ----------------------- 1) device_ops: Test case to validate device re-configuration 2) model_ops: Collection of 4 sub-tests to validate model slow APIs. Each sub-test would invoke the slow path model APIs (load / start / stop / unload) in different order. 3) inference_ordered: Test case to validate execution of end-to-end inferences on ML device with one active model at a time. This test can execute inference requests for multiple models, with inferences for a model executed after completion of inferences for a previously loaded model. 4) inference_interleave: Test case to validate end-to-end inferences with multiple active models concurrently. This case would work as a stress test to validate ML device. Options supported the by tests include burst size for enqueuing and dequeuing inference requests, enabling multiple queue pairs with a user specified value for queue size. Support is also enabled for batch inferencing, output validation and statistics. v7: * Add documentation to individual commits * Documentation fixes * Update array initialization formatting v6: * Enable error count * Fix segfault due to incorrect i/o size types * Revert i/o size variables to uint64_t v5: * Cleanup of header includes * Addressed issues with unclean patches * Changed input/output size variables to uint32_t * Rebase over main branch v4: * Updated model_id as uint16_t * Updated license info in SVG files * Updated release notes v3: * Code rebase Srikanth Yalavarthi (11): app/mldev: implement test framework for mldev app/mldev: add common test functions app/mldev: add test case to validate device ops app/mldev: add test case to validate model ops app/mldev: add ordered inference test case app/mldev: add test case to interleave inferences app/mldev: enable support for burst inferences app/mldev: enable support for queue pairs and size app/mldev: enable support for inference batches app/mldev: enable support for inference validation app/mldev: enable reporting stats in mldev app .mailmap | 1 + MAINTAINERS | 2 + app/meson.build | 1 + app/test-mldev/meson.build | 24 + app/test-mldev/ml_common.h | 29 + app/test-mldev/ml_main.c | 113 ++ app/test-mldev/ml_options.c | 340 +++++ app/test-mldev/ml_options.h | 57 + app/test-mldev/ml_test.c | 41 + app/test-mldev/ml_test.h | 76 ++ app/test-mldev/parser.c | 380 ++++++ app/test-mldev/parser.h | 55 + app/test-mldev/test_common.c | 136 ++ app/test-mldev/test_common.h | 27 + app/test-mldev/test_device_ops.c | 228 ++++ app/test-mldev/test_device_ops.h | 17 + app/test-mldev/test_inference_common.c | 1128 +++++++++++++++++ app/test-mldev/test_inference_common.h | 75 ++ app/test-mldev/test_inference_interleave.c | 118 ++ app/test-mldev/test_inference_ordered.c | 116 ++ app/test-mldev/test_model_common.c | 164 +++ app/test-mldev/test_model_common.h | 46 + app/test-mldev/test_model_ops.c | 428 +++++++ app/test-mldev/test_model_ops.h | 20 + doc/guides/rel_notes/release_23_03.rst | 8 + .../tools/img/mldev_inference_interleave.svg | 669 ++++++++++ .../tools/img/mldev_inference_ordered.svg | 528 ++++++++ .../tools/img/mldev_model_ops_subtest_a.svg | 420 ++++++ .../tools/img/mldev_model_ops_subtest_b.svg | 423 +++++++ .../tools/img/mldev_model_ops_subtest_c.svg | 366 ++++++ .../tools/img/mldev_model_ops_subtest_d.svg | 424 +++++++ doc/guides/tools/index.rst | 1 + doc/guides/tools/testmldev.rst | 442 +++++++ 33 files changed, 6903 insertions(+) create mode 100644 app/test-mldev/meson.build create mode 100644 app/test-mldev/ml_common.h create mode 100644 app/test-mldev/ml_main.c create mode 100644 app/test-mldev/ml_options.c create mode 100644 app/test-mldev/ml_options.h create mode 100644 app/test-mldev/ml_test.c create mode 100644 app/test-mldev/ml_test.h create mode 100644 app/test-mldev/parser.c create mode 100644 app/test-mldev/parser.h create mode 100644 app/test-mldev/test_common.c create mode 100644 app/test-mldev/test_common.h create mode 100644 app/test-mldev/test_device_ops.c create mode 100644 app/test-mldev/test_device_ops.h create mode 100644 app/test-mldev/test_inference_common.c create mode 100644 app/test-mldev/test_inference_common.h create mode 100644 app/test-mldev/test_inference_interleave.c create mode 100644 app/test-mldev/test_inference_ordered.c create mode 100644 app/test-mldev/test_model_common.c create mode 100644 app/test-mldev/test_model_common.h create mode 100644 app/test-mldev/test_model_ops.c create mode 100644 app/test-mldev/test_model_ops.h create mode 100644 doc/guides/tools/img/mldev_inference_interleave.svg create mode 100644 doc/guides/tools/img/mldev_inference_ordered.svg create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_a.svg create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_b.svg create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_c.svg create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_d.svg create mode 100644 doc/guides/tools/testmldev.rst -- 2.17.1