From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6E94742421; Wed, 25 Jan 2023 14:54:35 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0FFFA42D3E; Wed, 25 Jan 2023 14:54:35 +0100 (CET) Received: from mail-ua1-f46.google.com (mail-ua1-f46.google.com [209.85.222.46]) by mails.dpdk.org (Postfix) with ESMTP id 8973942D31 for ; Wed, 25 Jan 2023 14:54:34 +0100 (CET) Received: by mail-ua1-f46.google.com with SMTP id r12so831087uaf.7 for ; Wed, 25 Jan 2023 05:54:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=L3kIbYsYLzsTTDPJ18ceRfRqqyKBZ0xelB8fFeJ37dQ=; b=HiTDMcf10/FTEC9LzSqcu+4Ez4iHlqJnMF08VXZ3FvvbbSbxRUOSQ7ZS3vswdBIB36 bjqDtfaxcXziYkxWlEO929iMa2auWdpztvkCLuyKqPml2ROXSaG09SRZKTH2Q5IUarHz OeDNhNK7ohClF8UNjAS18u/gNM+crCQiBTLXc5e7BmwA1Djvhqf46iabURycTcOlXWIE 5D9XjO43Pzx/DDbIF8s6u1MqDd80PHKuPn5Wc2pgRu3mC0H3xyKHNWQqBKmIUaA7c982 NThI9W+XJLVH/TrTExefDsnJwSgo5I7bsDY/0ulB+S551b9bB+Y932+y+rOMSjoxVZBB gbxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=L3kIbYsYLzsTTDPJ18ceRfRqqyKBZ0xelB8fFeJ37dQ=; b=izmhnFc/gmndyIWc+iwH3DRLQegmdo94P+2bbAO9S4/qLSi7wdFGPTD19AXE60MqeW y9UVTPJHOvVIugX5mZoDNvrJjqKIRzGBxe7ftiOgDrmUEYEt/hpRvDvNcXlg40KbhFJM rvkFYUxT94BrqWgb2qvIE9Gg+7Rt4+FGGpCa182fOHy8nQgjBEvzgKnDbH4DJ58mqwNp 11TSXSFBroC46aK8RYq6qw/ptW9tXPSyeho/rFe96IbHJ3yeGFbqTdJLnZgDwtrXDzc0 iDUhoEQf9BqdcmA4HpIodGToo8zAz3hTonyl34HfhAiJZ3ChqfqFwB4BuUrs3y/Q+57U 6/bw== X-Gm-Message-State: AO0yUKU0hDlzOSD0ZBs5o357K604xJlRe1rtDlWA/l4jOz98HKBptLrT LF+IPq/1BT2YBnDGz8q4i0KbOoBbJoQTkydW3X8= X-Google-Smtp-Source: AK7set/wjFUokafWaPkLgW3otblPrvVS3K73STlURAeTm7vsbMT4MEv/uCTntwe0wp/lgSp0ui92KbdqLwzHklD2Nbo= X-Received: by 2002:a9f:2bc6:0:b0:652:b4f6:d92d with SMTP id f6-20020a9f2bc6000000b00652b4f6d92dmr907297uaj.61.1674654873743; Wed, 25 Jan 2023 05:54:33 -0800 (PST) MIME-Version: 1.0 References: <20220803132839.2747858-1-jerinj@marvell.com> <4520101.QLehXeTyEo@thomas> In-Reply-To: <4520101.QLehXeTyEo@thomas> From: Jerin Jacob Date: Wed, 25 Jan 2023 19:24:07 +0530 Message-ID: Subject: Re: [dpdk-dev] [RFC PATCH 0/1] mldev: introduce machine learning device library To: Thomas Monjalon Cc: Honnappa Nagarahalli , dev@dpdk.org, =?UTF-8?Q?Morten_Br=C3=B8rup?= , "jerinj@marvell.com" , Ferruh Yigit , "Ajit Khaparde (ajit.khaparde@broadcom.com)" , Andrew Boyer , Andrew Rybchenko , Beilei Xing , "Richardson, Bruce" , Chas Williams , "Xia, Chenbo" , Ciara Loftus , Devendra Singh Rawat , Ed Czeck , Evgeny Schemeilin , Gaetan Rivet , Gagandeep Singh , Guoyang Zhou , Haiyue Wang , Harman Kalra , Heinrich Kuhn , "hemant.agrawal@nxp.com" , Hyong Youb Kim , Igor Chauskin , Igor Russkikh , Jakub Grajciar , Jasvinder Singh , Jian Wang , Jiawen Wu , Jingjing Wu , John Daley , John Miller , "John W. Linville" , "Wiles, Keith" , Kiran Kumar K , Lijun Ou , Liron Himi , Long Li , Marcin Wojtas , Martin Spinler , Matan Azrad , Matt Peters , Maxime Coquelin , Michal Krawczyk , "Min Hu (Connor" , Pradeep Kumar Nalla , Nithin Dabilpuram , Qiming Yang , Qi Zhang , Radha Mohan Chintakuntla , Rahul Lakkireddy , Rasesh Mody , Rosen Xu , Sachin Saxena , Satha Koteswara Rao Kottidi , Shahed Shaikh , Shai Brandes , Shepard Siegel , Somalapuram Amaranath , Somnath Kotur , Stephen Hemminger , Steven Webster , Sunil Kumar Kori , Tetsuya Mukawa , Veerasenareddy Burru , nd , Viacheslav Ovsiienko , Xiao Wang , Xiaoyun Wang , Yisen Zhuang , Yong Wang , Ziyang Xuan , Prasun Kapoor , "nadavh@marvell.com" , Satananda Burla , Narayana Prasad , Akhil Goyal , Ray Kinsella , Dmitry Kozlyuk , Anatoly Burakov , Cristian Dumitrescu , =?UTF-8?Q?Mattias_R=C3=B6nnblom?= , Ruifeng Wang , David Christensen , "Ananyev, Konstantin" , Olivier Matz , "Jayatheerthan, Jay" , Ashwin Sekhar Thalakalath Kottilveetil , Pavan Nikhilesh , Elena Agostini , Srikanth Yalavarthi , "dchickles@marvell.com" , "sshankarnara@marvell.com" , John McNamara , Stephen Hemminger Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Wed, Jan 25, 2023 at 7:17 PM Thomas Monjalon wrote= : > > 17/08/2022 16:53, Jerin Jacob: > > On Tue, Aug 16, 2022 at 10:04 PM Honnappa Nagarahalli > > wrote: > > > > > > > > > > > > > > From: Jerin Jacob [mailto:jerinjacobk@gmail.com] > > > > > Sent: Tuesday, 16 August 2022 15.13 > > > > > > > > > > On Wed, Aug 3, 2022 at 8:49 PM Stephen Hemminger > > > > > wrote: > > > > > > > > > > > > On Wed, 3 Aug 2022 18:58:37 +0530 > > > > > > wrote: > > > > > > > > > > > > > Roadmap > > > > > > > ------- > > > > > > > 1) Address the comments for this RFC. > > > > > > > 2) Common code for mldev > > > > > > > 3) SW mldev driver based on TVM (https://tvm.apache.org/) > > > > > > > > > > > > Having a SW implementation is important because then it can be > > > > > covered > > > > > > by tests. > > > > > > > > > > Yes. That reason for adding TVM based SW driver as item (3). > > > > > > > > > > Is there any other high level or API level comments before procee= ding > > > > > with v1 and implementation. > > > > > > > > Have you seriously considered if the DPDK Project is the best home = for this > > > > project? I can easily imagine the DPDK development process being a = hindrance > > > > in many aspects for an evolving AI/ML library. Off the top of my he= ad, it would > > > > probably be better off as a separate project, like SPDK. > > > There is a lot of talk about using ML in networking workloads. Althou= gh, I am not very sure on how the use case looks like. For ex: is the infer= ence engine going to be inline (i.e. the packet goes through the inference = engine before coming to the CPU and provide some data (what sort of data?))= , look aside (does it require the packets to be sent to the inference engin= e or is it some other data?), what would be an end to end use case? A sampl= e application using these APIs would be helpful. > > > > Simple application for the inference usage is added in the cover letter= . > > > > Regarding the use cases, There are many like firewall, intrusion > > detection etc. Most of the use cases are driven by product > > requirements and SW IP vendors try to keep it to themselves as a > > product differentiate factor. > > That is the prime reason for DPDK scope only for inference where IO is > > involved. Model creation and training etc will heavily vary based on > > use case but not the inference model. > > > > > > > > IMO, if we need to share the packets with the inference engine, then = it fits into DPDK. > > > > Yes. Yes for networking or ORAN use cases the interface data comes > > over wire and result can go over wire. > > > > > > > > As I understand, there are many mature open source projects for ML/in= ference outside of DPDK. Does it make sense for DPDK to adopt those project= s rather than inventing our own? > > > > # AI/ML compiler libraries more focused on model creation and > > training etc (Thats where actual value addition the AI/ML libraries > > can offer) and > > minimal part for inference (It is just added for testing the model) > > # Considering the inference is the scope of the DPDK. DPDK is ideal > > place for following reasons > > > > a) Inference scope is very limited. > > b) Avoid memcpy of inference data (Use directly from network or > > other class of device like cryptodev, regexdev) > > c) Reuse highspeed IO interface like PCI backed driver etc > > d) Integration with other DPDK subsystems like eventdev etc for job com= pletion. > > e) Also support more inline offloads by merging two device classes > > like rte_secuity. > > f) Run the inference model from different AI/ML compiler frameworks or > > abstract the inference usage. > > Similar concept is already applied to other DPDK device classes like > > 1) In Regexdev, The compiler generates the rule database which is out > > of scope of DPDK. DPDK API just loads the rule database > > 2) In Gpudev, The GPU kernel etc out of scope of DPDK.DPDK cares about > > IO interface. > > I think Honnappa was thinking about linking an existing inference library= . > What are the similar libraries? Not sure. Honnappa can tell if any which meets (a) to (f). > >