From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 680C442486; Wed, 25 Jan 2023 14:45:33 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1FF5C42D3E; Wed, 25 Jan 2023 14:45:33 +0100 (CET) Received: from new4-smtp.messagingengine.com (new4-smtp.messagingengine.com [66.111.4.230]) by mails.dpdk.org (Postfix) with ESMTP id 4E2E142D31; Wed, 25 Jan 2023 14:45:31 +0100 (CET) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailnew.nyi.internal (Postfix) with ESMTP id 7DCA3581E33; Wed, 25 Jan 2023 08:45:30 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Wed, 25 Jan 2023 08:45:30 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= cc:cc:content-transfer-encoding:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to; s=fm3; t=1674654330; x= 1674661530; bh=FXzXjnx2QL3vadx1saxU1cB+zWwJaJrRqI5jeeE71ak=; b=e XI+Ku4ItxSiud/r476JHJ5G25ec8rFtsAYXfPiN9XCwiKEH/WhALPG47ZlvMjG/j rWc3X3xOLxMzYCvBWiKjVFumGmoxlDsMQnA5An+Bi8fzBBaBN7uuD0V9TzRkL3wc qlTMeRKl9pjcAZRG+5Y+wRMauPFHMpk4LzWy5PBH9XkoLsYpYHEssSUmBB4qaQlh GCRfxGO1gUASMbBHpcwveYW+TnPL877G7c6RbbHTkqwGdRfzE26V1gg5xjXLCjBT k5OKNlpQvQLtfpYOvYXlVUfuBoqbt8uSYHikx6L918oQtziTDQ5jkcHmLJUJiMXf s3MPu+sQNS5BjAJ/gsRXA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1674654330; x= 1674661530; bh=FXzXjnx2QL3vadx1saxU1cB+zWwJaJrRqI5jeeE71ak=; b=Z e2PkGNBW9zd1ksBzaHkQPOxwVBXCZuGBTO1ig8mfYW4Zpxxind9uiRYOimjY8oFu nhpDdyQ8ghfR9IJ7+i0UyGIClf7ZcTQCjGOOviXNDnV3grv6GuZiBxudafrNzupt 1k3aYeoh7+dXSh7phlLB+uwogLzHQ9kZOdkI45iQz4EN8lo8HIskf5e68DTQrn1l lGOdmoufHp6XxSn26jVat5sQD4BmhyTlwLQA3/WvH4lRjlrOlCtmwzxNt7POoO6K PrTmL0D5gtG0SiTcHLpm9qtqrf9L6DKODnKhsAzdxP08GyzYOGJ32IDeA7gppnFk +hrb3zSl4asGaVX2Pto8A== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedruddvvddgheehucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkjghfggfgtgesthhqredttddtudenucfhrhhomhepvfhhohhm rghsucfoohhnjhgrlhhonhcuoehthhhomhgrshesmhhonhhjrghlohhnrdhnvghtqeenuc ggtffrrghtthgvrhhnpeekjeefgfehveejjeehheejtdeijedvudelueelheeijedtueej iedttdeuudfhveenucffohhmrghinheprghprggthhgvrdhorhhgnecuvehluhhsthgvrh fuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepthhhohhmrghssehmohhnjhgr lhhonhdrnhgvth X-ME-Proxy: Feedback-ID: i47234305:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 25 Jan 2023 08:45:18 -0500 (EST) From: Thomas Monjalon To: Jerin Jacob , dev@dpdk.org Cc: Jerin Jacob , dpdk-dev , Ferruh Yigit , Ajit Khaparde , Andrew Boyer , Andrew Rybchenko , Beilei Xing , "Richardson, Bruce" , Chas Williams , "Xia, Chenbo" , Ciara Loftus , Devendra Singh Rawat , Ed Czeck , Evgeny Schemeilin , Gaetan Rivet , Gagandeep Singh , Guoyang Zhou , Haiyue Wang , Harman Kalra , Heinrich Kuhn , Hemant Agrawal , Hyong Youb Kim , Igor Chauskin , Igor Russkikh , Jakub Grajciar , Jasvinder Singh , Jian Wang , Jiawen Wu , Jingjing Wu , John Daley , John Miller , "John W. Linville" , "Wiles, Keith" , Kiran Kumar K , Lijun Ou , Liron Himi , Long Li , Marcin Wojtas , Martin Spinler , Matan Azrad , Matt Peters , Maxime Coquelin , Michal Krawczyk , "Min Hu (Connor" , Pradeep Kumar Nalla , Nithin Dabilpuram , Qiming Yang , Qi Zhang , Radha Mohan Chintakuntla , Rahul Lakkireddy , Rasesh Mody , Rosen Xu , Sachin Saxena , Satha Koteswara Rao Kottidi , Shahed Shaikh , Shai Brandes , Shepard Siegel , Somalapuram Amaranath , Somnath Kotur , Stephen Hemminger , Steven Webster , Sunil Kumar Kori , Tetsuya Mukawa , Veerasenareddy Burru , Viacheslav Ovsiienko , Xiao Wang , Xiaoyun Wang , Yisen Zhuang , Yong Wang , Ziyang Xuan , Prasun Kapoor , nadavh@marvell.com, Satananda Burla , Narayana Prasad , Akhil Goyal , Ray Kinsella , Dmitry Kozlyuk , Anatoly Burakov , Cristian Dumitrescu , Honnappa Nagarahalli , Mattias =?ISO-8859-1?Q?R=F6nnblom?= , "Ruifeng Wang (Arm Technology China)" , David Christensen , "Ananyev, Konstantin" , Olivier Matz , "Jayatheerthan, Jay" , Ashwin Sekhar Thalakalath Kottilveetil , Pavan Nikhilesh , Elena Agostini , Srikanth Yalavarthi , dchickles@marvell.com, sshankarnara@marvell.com, John McNamara , Stephen Hemminger , Morten =?ISO-8859-1?Q?Br=F8rup?= , techboard@dpdk.org Subject: Re: [dpdk-dev] [RFC PATCH 0/1] mldev: introduce machine learning device library Date: Wed, 25 Jan 2023 14:45:16 +0100 Message-ID: <4426124.uADA5c2rLh@thomas> In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC35D87273@smartserver.smartshare.dk> References: <20220803132839.2747858-1-jerinj@marvell.com> <98CBD80474FA8B44BF855DF32C47DC35D87273@smartserver.smartshare.dk> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="iso-8859-1" X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org 17/08/2022 08:58, Morten Br=F8rup: > > From: Jerin Jacob [mailto:jerinjacobk@gmail.com] > > Sent: Wednesday, 17 August 2022 07.37 > >=20 > > On Tue, Aug 16, 2022 at 9:15 PM Morten Br=F8rup > > wrote: > > > > > > > From: Jerin Jacob [mailto:jerinjacobk@gmail.com] > > > > Sent: Tuesday, 16 August 2022 15.13 > > > > > > > > On Wed, Aug 3, 2022 at 8:49 PM Stephen Hemminger > > > > wrote: > > > > > > > > > > On Wed, 3 Aug 2022 18:58:37 +0530 > > > > > wrote: > > > > > > > > > > > Roadmap > > > > > > ------- > > > > > > 1) Address the comments for this RFC. > > > > > > 2) Common code for mldev > > > > > > 3) SW mldev driver based on TVM (https://tvm.apache.org/) > > > > > > > > > > Having a SW implementation is important because then it can be > > > > covered > > > > > by tests. > > > > > > > > Yes. That reason for adding TVM based SW driver as item (3). > > > > > > > > Is there any other high level or API level comments before > > proceeding > > > > with v1 and implementation. > > > > > > Have you seriously considered if the DPDK Project is the best home > > for this project? I can easily imagine the DPDK development process > > being a hindrance in many aspects for an evolving AI/ML library. Off > > the top of my head, it would probably be better off as a separate > > project, like SPDK. > >=20 > > Yes. The reasons are following > >=20 > > # AI/ML compiler libraries more focused on model creation and > > training etc (Thats where actual value addition the AI/ML libraries > > can offer) and minimal part for interference(It is just added for > > testing the model) > > # Considering the inference is the scope of the DPDK. DPDK is ideal > > place for following reasons > >=20 > > a) Inference scope is very limited. > > b) Avoid memcpy of interference data (Use directly from network or > > other class of device like cryptodev, regexdev) > > c) Reuse highspeed IO interface like PCI backed driver etc > > d) Integration with other DPDK subsystems like eventdev etc for job > > completion. > > e) Also support more inline offloads by merging two device classes > > like rte_secuity. > > f) Run the inference model from different AI/ML compiler frameworks or > > abstract the inference usage. > > Similar concept is already applied to other DPDK device classes like > > 1) In Regexdev, The compiler generates the rule database which is out > > of scope of DPDK. DPDK API just loads the rule database > > 2) In Gpudev, The GPU kernel etc out of scope of DPDK.DPDK cares about > > IO interface. >=20 > Thank you for the detailed reply, Jerin. >=20 > These are good reasons for adding the new device class to the DPDK projec= t - especially the Regexdev comparison got me convinced. >=20 > >=20 > > > If all this stuff can be completely omitted at build time, I have no > > objections. > >=20 > > Yes, It can be completely omitted at build time. >=20 > Perfect. >=20 > > Also no plan to > > integrate to testpmd and other existing application. Planning to add > > only app/test-mldev application. >=20 > +1 to that >=20 > >=20 > > > > > > A small note about naming (not intending to start a flame war, so > > please feel free to ignore!): I haven't worked seriously with ML/AI > > since university three decades ago, so I'm quite rusty in the domain. > > However, I don't see any Machine Learning functions proposed by this > > API. The library provides an API to an Inference Engine - but nobody > > says the inference model stems from Machine Learning; it might as well > > be a hand crafted model. Do you plan to propose APIs for training the > > models? If not, the name of the library could confuse some potential > > users. > >=20 > > No, scope is only inference and it is documented in the programing > > guide and API header file. I am trying to keep name similar to > > regexdev, gpudev etc which have similar scope. But I am open to other > > shortname/name if you have something in mind. >=20 > The AI(Artificial Intelligence)/ML(Machine Learning)/IE(Inference Engine)= chip market still seems immature and fragmented, so I can't find any conse= nsus on generic names for such hardware accelerator devices. >=20 > Some of the chip vendors represented on the DPDK mailing list offer AI/ML= /IE accelerator chips. Perhaps their marketing department could propose alt= ernatives to "Machine Learning Device"/"mldev" for inference engine devices= (with no acceleration for training the models). If not, the initially prop= osed name is good enough. >=20 > So: Everyone ask your marketing departments and speak up now, or the name= "mldev" will be set in stone. ;-) >=20 > I'm thinking: While "Inference Engine Device"/iedev might be technically = more correct, it doesn't have same value as "Machine Learning Device"/"mlde= v" on a marketing scale. And we should choose a name that we expect might b= ecome industry standard consensus. I don't why but I like mldev and dislike iedev. I could be OK with aidev as well.