From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8BB63A0032; Wed, 17 Aug 2022 16:54:05 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 73B584068E; Wed, 17 Aug 2022 16:54:05 +0200 (CEST) Received: from mail-qv1-f44.google.com (mail-qv1-f44.google.com [209.85.219.44]) by mails.dpdk.org (Postfix) with ESMTP id 3D3FD40685 for ; Wed, 17 Aug 2022 16:54:04 +0200 (CEST) Received: by mail-qv1-f44.google.com with SMTP id e4so559669qvr.2 for ; Wed, 17 Aug 2022 07:54:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc; bh=EKpHcMZiLWKX5Ed/4Ui5kQNoMaahCSM6+4fMOBSvBCM=; b=K4OEuLS5L+M7En1S82U1O9acN6I8KuIBe5PNipcU0fYILhTdImnza79zXzh2LUn1hR URYJ2M5+doHpI1YJbYBsCQIp//Mnjjsy9RcR/9OaXzbT5zniRHyKw0nk6zTFyCqW9lcC fqidUPz1WTQiR4YaRRJ/Kc4eqVw0YZ2YRcLLxHTVMk+x3vdX6HbR+a+DLq89s8NsXd6v gfKRwcWwqUkQFt0lZs6vhExARVRMpaa/+SweX8vGklaHxfVdSdy5hx5HeG4pK7K09aE+ VbO2AvEmsK++oKAdoGgqiz81Jrag31QCdd+AQ75L8ApAzoUe1pLeEvq1lJo3XxM3knje XZgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc; bh=EKpHcMZiLWKX5Ed/4Ui5kQNoMaahCSM6+4fMOBSvBCM=; b=nXpmEvxCUyL68kOHbt9p7cL+xmPC9VP72ngWrUT2cfbmvgFnnMBi8sgnWtv14cJV3u lX20hviGHeMeRdJ2ejyHjW4/z6D3nMRLPFFg0jwsBb4tjPI0nfkhsCQiCORgw6Br5OXO hVWUhmPWSghSVPJBHJ+DNkw7Lm5yhSg6Lf5Nhbhxa89XQWL5uPtjd33a11Aj/+xmk8qy 4zukvo5JKqHXAxi60K0RzK/r3t7eKc80gN3nbeZo6bYmY74iYPsWfGwLeySGQEGvzas/ v7cnMgy2Ok4xmnp6JXWXFbqDJtEKD4QASlmh67aLIbFc6V0qrD5OvuCJ2ApH0LNXXXGD UIug== X-Gm-Message-State: ACgBeo0gfWlPKH+/2LW0L+agB8Zc3CGfDFrG82Dbp/nFi0YUs4aVr+Iz p6KdRXJXEmYUs+pDE9kKzcFDEZJifPufj9K/noY= X-Google-Smtp-Source: AA6agR5uvujrbjPmJfWwNlQqWgjjyqIqa46straKsi+F2ZRGSwsaDlZLIFbMXgcrKildxmkMVOHYb1OJFDJPyhhpDrI= X-Received: by 2002:a05:6214:624:b0:474:7ec6:feea with SMTP id a4-20020a056214062400b004747ec6feeamr22926850qvx.0.1660748043493; Wed, 17 Aug 2022 07:54:03 -0700 (PDT) MIME-Version: 1.0 References: <20220803132839.2747858-1-jerinj@marvell.com> <20220803081916.3f03dd38@hermes.local> <98CBD80474FA8B44BF855DF32C47DC35D87271@smartserver.smartshare.dk> In-Reply-To: From: Jerin Jacob Date: Wed, 17 Aug 2022 20:23:37 +0530 Message-ID: Subject: Re: [dpdk-dev] [RFC PATCH 0/1] mldev: introduce machine learning device library To: Honnappa Nagarahalli Cc: =?UTF-8?Q?Morten_Br=C3=B8rup?= , "jerinj@marvell.com" , dpdk-dev , "thomas@monjalon.net" , Ferruh Yigit , "Ajit Khaparde (ajit.khaparde@broadcom.com)" , Andrew Boyer , Andrew Rybchenko , Beilei Xing , "Richardson, Bruce" , Chas Williams , "Xia, Chenbo" , Ciara Loftus , Devendra Singh Rawat , Ed Czeck , Evgeny Schemeilin , Gaetan Rivet , Gagandeep Singh , Guoyang Zhou , Haiyue Wang , Harman Kalra , Heinrich Kuhn , "hemant.agrawal@nxp.com" , Hyong Youb Kim , Igor Chauskin , Igor Russkikh , Jakub Grajciar , Jasvinder Singh , Jian Wang , Jiawen Wu , Jingjing Wu , John Daley , John Miller , "John W. Linville" , "Wiles, Keith" , Kiran Kumar K , Lijun Ou , Liron Himi , Long Li , Marcin Wojtas , Martin Spinler , Matan Azrad , Matt Peters , Maxime Coquelin , Michal Krawczyk , "Min Hu (Connor" , Pradeep Kumar Nalla , Nithin Dabilpuram , Qiming Yang , Qi Zhang , Radha Mohan Chintakuntla , Rahul Lakkireddy , Rasesh Mody , Rosen Xu , Sachin Saxena , Satha Koteswara Rao Kottidi , Shahed Shaikh , Shai Brandes , Shepard Siegel , Somalapuram Amaranath , Somnath Kotur , Stephen Hemminger , Steven Webster , Sunil Kumar Kori , Tetsuya Mukawa , Veerasenareddy Burru , nd , Viacheslav Ovsiienko , Xiao Wang , Xiaoyun Wang , Yisen Zhuang , Yong Wang , Ziyang Xuan , Prasun Kapoor , "nadavh@marvell.com" , Satananda Burla , Narayana Prasad , Akhil Goyal , Ray Kinsella , Dmitry Kozlyuk , Anatoly Burakov , Cristian Dumitrescu , =?UTF-8?Q?Mattias_R=C3=B6nnblom?= , Ruifeng Wang , David Christensen , "Ananyev, Konstantin" , Olivier Matz , "Jayatheerthan, Jay" , Ashwin Sekhar Thalakalath Kottilveetil , Pavan Nikhilesh , Elena Agostini , Srikanth Yalavarthi , "dchickles@marvell.com" , "sshankarnara@marvell.com" , John McNamara , Stephen Hemminger Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Tue, Aug 16, 2022 at 10:04 PM Honnappa Nagarahalli wrote: > > > > > > From: Jerin Jacob [mailto:jerinjacobk@gmail.com] > > > Sent: Tuesday, 16 August 2022 15.13 > > > > > > On Wed, Aug 3, 2022 at 8:49 PM Stephen Hemminger > > > wrote: > > > > > > > > On Wed, 3 Aug 2022 18:58:37 +0530 > > > > wrote: > > > > > > > > > Roadmap > > > > > ------- > > > > > 1) Address the comments for this RFC. > > > > > 2) Common code for mldev > > > > > 3) SW mldev driver based on TVM (https://tvm.apache.org/) > > > > > > > > Having a SW implementation is important because then it can be > > > covered > > > > by tests. > > > > > > Yes. That reason for adding TVM based SW driver as item (3). > > > > > > Is there any other high level or API level comments before proceeding > > > with v1 and implementation. > > > > Have you seriously considered if the DPDK Project is the best home for = this > > project? I can easily imagine the DPDK development process being a hind= rance > > in many aspects for an evolving AI/ML library. Off the top of my head, = it would > > probably be better off as a separate project, like SPDK. > There is a lot of talk about using ML in networking workloads. Although, = I am not very sure on how the use case looks like. For ex: is the inference= engine going to be inline (i.e. the packet goes through the inference engi= ne before coming to the CPU and provide some data (what sort of data?)), lo= ok aside (does it require the packets to be sent to the inference engine or= is it some other data?), what would be an end to end use case? A sample ap= plication using these APIs would be helpful. Simple application for the inference usage is added in the cover letter. Regarding the use cases, There are many like firewall, intrusion detection etc. Most of the use cases are driven by product requirements and SW IP vendors try to keep it to themselves as a product differentiate factor. That is the prime reason for DPDK scope only for inference where IO is involved. Model creation and training etc will heavily vary based on use case but not the inference model. > > IMO, if we need to share the packets with the inference engine, then it f= its into DPDK. Yes. Yes for networking or ORAN use cases the interface data comes over wire and result can go over wire. > > As I understand, there are many mature open source projects for ML/infere= nce outside of DPDK. Does it make sense for DPDK to adopt those projects ra= ther than inventing our own? # AI/ML compiler libraries more focused on model creation and training etc (Thats where actual value addition the AI/ML libraries can offer) and minimal part for inference (It is just added for testing the model) # Considering the inference is the scope of the DPDK. DPDK is ideal place for following reasons a) Inference scope is very limited. b) Avoid memcpy of inference data (Use directly from network or other class of device like cryptodev, regexdev) c) Reuse highspeed IO interface like PCI backed driver etc d) Integration with other DPDK subsystems like eventdev etc for job complet= ion. e) Also support more inline offloads by merging two device classes like rte_secuity. f) Run the inference model from different AI/ML compiler frameworks or abstract the inference usage. Similar concept is already applied to other DPDK device classes like 1) In Regexdev, The compiler generates the rule database which is out of scope of DPDK. DPDK API just loads the rule database 2) In Gpudev, The GPU kernel etc out of scope of DPDK.DPDK cares about IO interface. > > > > > If all this stuff can be completely omitted at build time, I have no ob= jections. > > > > A small note about naming (not intending to start a flame war, so pleas= e feel > > free to ignore!): I haven't worked seriously with ML/AI since universit= y three > > decades ago, so I'm quite rusty in the domain. However, I don't see any > > Machine Learning functions proposed by this API. The library provides a= n API to > > an Inference Engine - but nobody says the inference model stems from > > Machine Learning; it might as well be a hand crafted model. Do you plan= to > > propose APIs for training the models? If not, the name of the library c= ould > > confuse some potential users. > I think, at least on the edge devices, we need an inference device as ML = requires more cycles/power. > > > > > > Or Anyone else interested to review or contribute to this new DPDK > > > device class? >