From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 647B041EB2; Thu, 16 Mar 2023 18:52:16 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5147C40DF6; Thu, 16 Mar 2023 18:52:16 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 9D0EA40DDC for ; Thu, 16 Mar 2023 18:52:14 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 32GGkB6r007381; Thu, 16 Mar 2023 10:52:14 -0700 Received: from nam12-dm6-obe.outbound.protection.outlook.com (mail-dm6nam12lp2171.outbound.protection.outlook.com [104.47.59.171]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3pbs2u33rj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 16 Mar 2023 10:52:13 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=g94hbaSeLeoNNCSyHexIttPHJKLAsoLOkzVhH77o4x6ohxeIkLI+h8DYD8QhRJkEg8WaP56QLyo3x9quRLS1/MGJ441nMG/Rd8B6rFFwV3P9tPKGqsa79jYb4Jke0vAipQ4dRuW/KOfQ05Hk5Suzh4zgL+tbQ8R8TtiAViAQ3mbfSFh2fjGcvtSbdHRI5Cg8jB4LSFGq9aixHjsAshaZmTgc6VS9FGqBUCwNCnGn2C7G/uIpbrkaoIHheeHIs0LcHrxnhS4vP2qKQ//1kq0cMt+0v9eIZk4Dk7ntkclV+LOLEeINeUSL2L+eNTVe2RK7h62bI4RRQhs4IW9qv+FPww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=QvtUzV4fqVkqELWy3KMdACnmPlV5J9ctEqSnkgf/eHw=; b=odrL2XT4K2QxgRkPamycUNzMkMztExbGhD+FFQJKGDEHSwzkhcDwyJSVqZRwMuKvJlu4UPn/9sKc1oLbxXUIRvrggFRDln2JL27RpfYAFfibXSosQ9GfCglmySME/Y6Tzs7Xzf3a9vz4Vqc1QkCWz1hat1Mb7YmqjW6Uq+r473RCe0oBqPUoSNA49CgEsv1IG2VyykjLs9R0zkqBFS39gvbT1d1AZnwZLtzK6RbCj2plo1x10hgJBav+/teJ5iQmcMwHEEPKJWZPYfQ9K2V0O7t48YFrp55261sNEt9+aEsGfgTGA4U6UhYmn5qSvyJvFkDz7onjD8zdvH5kZmad5Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=QvtUzV4fqVkqELWy3KMdACnmPlV5J9ctEqSnkgf/eHw=; b=Hi1ZMoy7pwq+qGm4gIKB1BoXwXh9WIjavFwOv2Ce3z6dLda+Lvlb5/l7T8dOCOOuCi1b8FANLBycfUV9h7i2Hoj1wr0pq+zTvSKtTplw2koQk6+pBbVoQ2WsT7r93NkPhgPmrbqmXhV3N3ao8R2qXoTw6p9NDZ3Afhf1bEwbpVM= Received: from CO6PR18MB3939.namprd18.prod.outlook.com (2603:10b6:5:34d::18) by SA0PR18MB3614.namprd18.prod.outlook.com (2603:10b6:806:9c::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6178.26; Thu, 16 Mar 2023 17:52:10 +0000 Received: from CO6PR18MB3939.namprd18.prod.outlook.com ([fe80::66ce:a9db:b43b:5ba0]) by CO6PR18MB3939.namprd18.prod.outlook.com ([fe80::66ce:a9db:b43b:5ba0%3]) with mapi id 15.20.6178.033; Thu, 16 Mar 2023 17:52:10 +0000 From: Srikanth Yalavarthi To: Thomas Monjalon CC: "dev@dpdk.org" , Shivah Shankar Shankar Narayan Rao , Jerin Jacob Kollanukkaran , Anup Prabhu , Prince Takkar , Parijat Shukla , Srikanth Yalavarthi Subject: RE: [EXT] Re: [PATCH v6 09/12] app/mldev: enable support for inference batches Thread-Topic: [EXT] Re: [PATCH v6 09/12] app/mldev: enable support for inference batches Thread-Index: AQHZVCuXhcv8ME3w4E6+yoCTvUFuHK79tm6AgAAASMA= Date: Thu, 16 Mar 2023 17:52:10 +0000 Message-ID: References: <20221129070746.20396-1-syalavarthi@marvell.com> <20230311150905.26824-1-syalavarthi@marvell.com> <20230311150905.26824-10-syalavarthi@marvell.com> <4252176.ejJDZkT8p0@thomas> In-Reply-To: <4252176.ejJDZkT8p0@thomas> Accept-Language: en-IN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-dg-ref: =?us-ascii?Q?PG1ldGE+PGF0IG5tPSJib2R5LnR4dCIgcD0iYzpcdXNlcnNcc3lhbGF2YXJ0?= =?us-ascii?Q?aGlcYXBwZGF0YVxyb2FtaW5nXDA5ZDg0OWI2LTMyZDMtNGE0MC04NWVlLTZi?= =?us-ascii?Q?ODRiYTI5ZTM1Ylxtc2dzXG1zZy00NDY2MTZjMi1jNDIzLTExZWQtOWVmMS05?= =?us-ascii?Q?OGZhOWI4YTVlZTJcYW1lLXRlc3RcNDQ2NjE2YzQtYzQyMy0xMWVkLTllZjEt?= =?us-ascii?Q?OThmYTliOGE1ZWUyYm9keS50eHQiIHN6PSIzMzI4IiB0PSIxMzMyMzQ2Mjcy?= =?us-ascii?Q?Nzk3NjI4NDMiIGg9ImF1MDRuK0hqRlRGeEJXQ0dHNjNxcEMxa1hxUT0iIGlk?= =?us-ascii?Q?PSIiIGJsPSIwIiBibz0iMSIgY2k9ImNBQUFBRVJIVTFSU1JVRk5DZ1VBQUhZ?= =?us-ascii?Q?SUFBQ2J0Y29HTUZqWkFkSVVlZXhZNW5DTjBoUjU3RmptY0kwTkFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFIQUFBQUFHQ0FBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFFQUFRQUJBQUFBMUZIM2FBQUFBQUFBQUFBQUFBQUFBSjRBQUFCaEFH?= =?us-ascii?Q?UUFaQUJ5QUdVQWN3QnpBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUVBQUFBQUFBQUFBZ0FBQUFBQW5nQUFBR01BZFFCekFIUUFid0J0QUY4?= =?us-ascii?Q?QWNBQmxBSElBY3dCdkFHNEFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFRQUFBQUFBQUFB?= =?us-ascii?Q?Q0FBQUFBQUNlQUFBQVl3QjFBSE1BZEFCdkFHMEFYd0J3QUdnQWJ3QnVBR1VB?= =?us-ascii?Q?YmdCMUFHMEFZZ0JsQUhJQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFCQUFBQUFBQUFBQUlBQUFBQUFKNEFBQUJq?= =?us-ascii?Q?QUhVQWN3QjBBRzhBYlFCZkFITUFjd0J1QUY4QVpBQmhBSE1BYUFCZkFIWUFN?= =?us-ascii?Q?QUF5QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= x-dg-refone: =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBRUFBQUFBQUFBQUFnQUFBQUFBbmdB?= =?us-ascii?Q?QUFHTUFkUUJ6QUhRQWJ3QnRBRjhBY3dCekFHNEFYd0JyQUdVQWVRQjNBRzhB?= =?us-ascii?Q?Y2dCa0FITUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQVFBQUFBQUFBQUFDQUFBQUFBQ2VBQUFBWXdCMUFITUFkQUJ2?= =?us-ascii?Q?QUcwQVh3QnpBSE1BYmdCZkFHNEFid0JrQUdVQWJBQnBBRzBBYVFCMEFHVUFj?= =?us-ascii?Q?Z0JmQUhZQU1BQXlBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUJBQUFB?= =?us-ascii?Q?QUFBQUFBSUFBQUFBQUo0QUFBQmpBSFVBY3dCMEFHOEFiUUJmQUhNQWN3QnVB?= =?us-ascii?Q?RjhBY3dCd0FHRUFZd0JsQUY4QWRnQXdBRElBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFFQUFBQUFBQUFBQWdBQUFBQUFu?= =?us-ascii?Q?Z0FBQUdRQWJBQndBRjhBY3dCckFIa0FjQUJsQUY4QVl3Qm9BR0VBZEFCZkFH?= =?us-ascii?Q?MEFaUUJ6QUhNQVlRQm5BR1VBWHdCMkFEQUFNZ0FBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBUUFBQUFBQUFBQUNBQUFBQUFDZUFBQUFaQUJzQUhBQVh3?= =?us-ascii?Q?QnpBR3dBWVFCakFHc0FYd0JqQUdnQVlRQjBBRjhBYlFCbEFITUFjd0JoQUdj?= =?us-ascii?Q?QVpRQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= x-dg-reftwo: =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQkFBQUFBQUFBQUFJQUFBQUFBSjRBQUFCa0FHd0Fj?= =?us-ascii?Q?QUJmQUhRQVpRQmhBRzBBY3dCZkFHOEFiZ0JsQUdRQWNnQnBBSFlBWlFCZkFH?= =?us-ascii?Q?WUFhUUJzQUdVQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUVBQUFBQUFBQUFBZ0FBQUFBQW5nQUFBR1VBYlFCaEFHa0FiQUJmQUdFQVpB?= =?us-ascii?Q?QmtBSElBWlFCekFITUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQ0FBQUFBQUFBQUFBQUFBQUFRQUFBQUFBQUFBQ0FB?= =?us-ascii?Q?QUFBQUNlQUFBQWJRQmhBSElBZGdCbEFHd0FiQUJmQUhBQWNnQnZBR29BWlFC?= =?us-ascii?Q?akFIUUFYd0JqQUc4QVpBQmxBSE1BQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFCQUFBQUFBQUFBQUlBQUFBQUFKNEFBQUJ0QUdF?= =?us-ascii?Q?QWNnQjJBR1VBYkFCc0FGOEFkQUJsQUhJQWJRQnBBRzRBZFFCekFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBRUFBQUFBQUFBQUFnQUFBQUFBIi8+PC9tZXRhPg=3D=3D?= x-dg-rorf: true x-ms-publictraffictype: Email x-ms-traffictypediagnostic: CO6PR18MB3939:EE_|SA0PR18MB3614:EE_ x-ms-office365-filtering-correlation-id: 8eaa60a6-0e3d-49dd-4855-08db26472ad0 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: DWHuXgTpgIAMcu3nCIBfE7lzydRIKvufdNcdbCCIg6crEpa1VNAWiiRmd4rx8Y6BNte6nUs9dk3L7XrzM1Q2ReqC8YhZF8ZhOM6KSAnEUpFt9WcyVKI667AQnCoDR36fE91OqkUlvDtRgVSVLO1AeRt/tzbVsYbAzdvDOCD12EZAsOvdT99WfD4UJ86QumWbq5rqihooNohMDdCgexFGuO3ZVTQL0hh6f7f4PY4cne3VoAeqs8RLrhnsd2jC9Nx+qrCH4A/8WuOrR8VnaXHesP/mTrYbLg3QaZMecwTlcPfO8OLnFqwl59Ruedc9GGZCuZt+EnutjvdsMa9d80iSBcorDlgC8a+YLaVbudZmpHdeQu/sku7yrF6hVCQnVMjMe1329nNo8hYaguYPbnt49Xv52qF+LKUaVknGYvTrqYoPwnl1gbT9jh0aPB0Jgty0F11wpexlfULtX26ZOfNWHL11/c4y/73K7z4CzwQy9f4duOHVplEtLE9/P7Q/I/LbTBnnHnApaqay/mQMMkKt8oJeC4/KttxIRLK7s1ua92mGKzQF3+tjG0AimnHVNTCdJ5iaWKyseV53gzXnyN7U1+vcYjoStkenWMJRLr9AFUrdi2UA/EiFhZorjdrmBVV881I1m5YlMHXbADFqdEFG0dODZtsKrDkJb8jomRRfAxltIP0Kk87lQn6fPO2YiDH7+kOFEv73klB22r/dTKT1JQ== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CO6PR18MB3939.namprd18.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230025)(4636009)(396003)(346002)(39860400002)(376002)(366004)(136003)(451199018)(5660300002)(8936002)(41300700001)(52536014)(2906002)(38100700002)(33656002)(38070700005)(86362001)(7696005)(122000001)(6916009)(66556008)(107886003)(64756008)(66446008)(8676002)(71200400001)(66476007)(76116006)(66946007)(478600001)(26005)(4326008)(83380400001)(55016003)(316002)(9686003)(54906003)(186003)(6506007)(53546011); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?qKf8JxpPgK3Ak/H5rdFPj48Hrp+Xobcjul8PYCoeRnr06o3bSf6Q75hTP8Ki?= =?us-ascii?Q?hZ525Qq0SPm6Qd0qQkLR446kXC/vC52g2HwIKK6BA4bYvQKhkO7UtYQF24k8?= =?us-ascii?Q?/X96UdI+obImHoejFwxwiv8COpvXDdhYlj/liyilMYR4pcT9EwbphwERZKUp?= =?us-ascii?Q?gPQjFOKkgm7UdseE8BxSDHiIrRQQ4bzs5aFkBlwR8rybopUaFTHcbTKOZ9LG?= =?us-ascii?Q?k8E6jw2+VOOFXRchUc5sv4IlS1u9nnACTYavvl4y8AS1WpdpttuPI2Jf33If?= =?us-ascii?Q?j8ErMes6hdsMJlQtSMp1yfPrAMsMyExNqvzgsOtozXOanb0b7wV7gxnlVKMx?= =?us-ascii?Q?jefIhGdnsK+f36saJGWOPlKVN6KnHE4J/mmLZFhQ7G08fZNKTPpXDj7Pq16m?= =?us-ascii?Q?yx3jzo/DqfdPxQeDtGMPn+Yw7GGNCXMhKYWhXFQnsEomqJlDyi5wV/jlfthx?= =?us-ascii?Q?uCCy6oCR5tx2msaiBq6sJXXMsOXRfKvG+zN42SZ8H3b+jbAndZdgleO1uQjF?= =?us-ascii?Q?Qd+cj6DLYskZMxqhlaAuqw+mtp3dplzFesRCqAZRbq03mVQVRDkQPgsP3gPK?= =?us-ascii?Q?wyHHWtbfxXLoIDJMrmnx9co+SyqZyvLQJNO15pacBIb2wz8nX4jA1mCeiq50?= =?us-ascii?Q?uwwpa6mFjBih2TwHfGb9N8kNS7QXaeP5gcR4iKFaQqivo3hAVLIV2WlHoCP/?= =?us-ascii?Q?J7nlPGU3dNsb8D6q0jkm7pkn8jJ0BgWegvqSKrMCWWWA5Sgz3CKtFzyY587o?= =?us-ascii?Q?zJxfX7JJ5mtlh/Tg4iPZvgGEFqO0E6rvRTNjI42Ju8gHKiySkqt5kQXcJ0sY?= =?us-ascii?Q?i5eRyGA2KN1cxjLnKuKzpJDvHeRLlNtUd9ESyg1OnPHjD1Tb2Sv1s74e2Kly?= =?us-ascii?Q?xdKj2ZC4x4D+iL9gJaSLuBplsyjhN0i92KWgT684MZK/98/pNveZHYW+Ztrk?= =?us-ascii?Q?7+61Z7ivP3xUFH+g3j13Hb+YApIy8x4PfnxidKcvOhFyxFrADi/fIHtVW+yJ?= =?us-ascii?Q?eljNHoWUuxjNyxUflYuYxi1+t9yMJridWpuww30qq0qSO+g4La52o1hCRZb3?= =?us-ascii?Q?0iYSHrx7TANeoRWaMs7JHpbk8NhiiwBusWb4bSAXLVHujAIVNnfG0tGhK7VJ?= =?us-ascii?Q?PHJjoytRZGMLN0cuetCrZbFmXWda1FbDdHqLhC1HfnToUwOoV+tFaV29IWWE?= =?us-ascii?Q?rwZhEJ35fW6z3j9sL82JunHeuc4PGO4+w2n38jVZeM05EJ1+qDMlIhVO2bRt?= =?us-ascii?Q?YTnSBGE3nkBs0zqRXtwnfaL3ST5PcnaNvJqaKeqeNjQNniJh0hnFhBhUuv0l?= =?us-ascii?Q?PHI57+C5+6Z9O4x9nNs84wv2n2logw2SudFev5FccU00yGaiiQE6bFNydYFz?= =?us-ascii?Q?LYmk8C2j4f26wR54EZvMCnssxAsdIg9c3ZwILYC4npsRoSoGO1wKbaAOi9gi?= =?us-ascii?Q?oAULitSAttCdONqCw1yJ4V6cbQaiubXnBBpMNazwA45Vd0bg2HCKCbijx8zG?= =?us-ascii?Q?wFFZuJHTtLvZC22rjnNPHmiLHE5uir/og5kEjWYMsyCVwHXp3wojgFvYbKD9?= =?us-ascii?Q?vBAz1Adk/JpyjG6XAgytMDcbK6w4qY0ynak56DhC?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: marvell.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: CO6PR18MB3939.namprd18.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8eaa60a6-0e3d-49dd-4855-08db26472ad0 X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Mar 2023 17:52:10.5571 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: lfM5Hs2DiH7pObQcwpsvIPUdlX9/57T9x4mQ6yTAKPeRpGXJDn+3OA3hAYfeOc2oK2SsGBiqJULmMcSLZxLXYg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR18MB3614 X-Proofpoint-GUID: LewAZ0wlHiSmpeWXNHbp-j2V5I643gXK X-Proofpoint-ORIG-GUID: LewAZ0wlHiSmpeWXNHbp-j2V5I643gXK X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-16_11,2023-03-16_01,2023-02-09_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org > -----Original Message----- > From: Thomas Monjalon > Sent: 16 March 2023 23:18 > To: Srikanth Yalavarthi > Cc: dev@dpdk.org; Shivah Shankar Shankar Narayan Rao > ; Jerin Jacob Kollanukkaran > ; Anup Prabhu ; Prince Takkar > ; Parijat Shukla > Subject: [EXT] Re: [PATCH v6 09/12] app/mldev: enable support for inferen= ce > batches >=20 > External Email >=20 > ---------------------------------------------------------------------- > 11/03/2023 16:09, Srikanth Yalavarthi: > > @@ -528,8 +533,8 @@ ml_request_initialize(struct rte_mempool *mp, > void *opaque, void *obj, unsigned > > req->niters =3D 0; > > > > /* quantize data */ > > - rte_ml_io_quantize(t->cmn.opt->dev_id, t->model[t->fid].id, > > - t->model[t->fid].info.batch_size, t->model[t- > >fid].input, req->input); > > + rte_ml_io_quantize(t->cmn.opt->dev_id, t->model[t->fid].id, t- > >model[t->fid].nb_batches, > > + t->model[t->fid].input, req->input); > > } > > > > int > > @@ -547,7 +552,7 @@ ml_inference_iomem_setup(struct ml_test *test, > struct ml_options *opt, uint16_t > > int ret; > > > > /* get input buffer size */ > > - ret =3D rte_ml_io_input_size_get(opt->dev_id, t->model[fid].id, t- > >model[fid].info.batch_size, > > + ret =3D rte_ml_io_input_size_get(opt->dev_id, t->model[fid].id, > > +t->model[fid].nb_batches, > > &t->model[fid].inp_qsize, &t- > >model[fid].inp_dsize); > > if (ret !=3D 0) { > > ml_err("Failed to get input size, model : %s\n", > > opt->filelist[fid].model); @@ -555,9 +560,8 @@ > ml_inference_iomem_setup(struct ml_test *test, struct ml_options *opt, > uint16_t > > } > > > > /* get output buffer size */ > > - ret =3D rte_ml_io_output_size_get(opt->dev_id, t->model[fid].id, > > - t->model[fid].info.batch_size, &t- > >model[fid].out_qsize, > > - &t->model[fid].out_dsize); > > + ret =3D rte_ml_io_output_size_get(opt->dev_id, t->model[fid].id, t- > >model[fid].nb_batches, > > + &t->model[fid].out_qsize, &t- > >model[fid].out_dsize); > > if (ret !=3D 0) { > > ml_err("Failed to get input size, model : %s\n", opt- > >filelist[fid].model); > > return ret; > > @@ -702,7 +706,7 @@ ml_request_finish(struct rte_mempool *mp, void > *opaque, void *obj, unsigned int > > return; > > > > t->nb_used++; > > - rte_ml_io_dequantize(t->cmn.opt->dev_id, model->id, t->model[req- > >fid].info.batch_size, > > + rte_ml_io_dequantize(t->cmn.opt->dev_id, model->id, > > +t->model[req->fid].nb_batches, > > req->output, model->output); >=20 > These changes look unrelated with the topic of the patch. > You should probably fix it when adding those lines at first. The changes are related to the patch. Initially the number of batches run p= er inference is set to the default batch_size value of the mode, which is r= eported to the user through rte_ml_model_info_get. This patch adds support to specify the number of batches to be run per infe= rence. Hence, the default batch_size is replace with nb_batches values spec= ified by the user. >=20