From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <dev-bounces@dpdk.org> Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6E94742421; Wed, 25 Jan 2023 14:54:35 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0FFFA42D3E; Wed, 25 Jan 2023 14:54:35 +0100 (CET) Received: from mail-ua1-f46.google.com (mail-ua1-f46.google.com [209.85.222.46]) by mails.dpdk.org (Postfix) with ESMTP id 8973942D31 for <dev@dpdk.org>; Wed, 25 Jan 2023 14:54:34 +0100 (CET) Received: by mail-ua1-f46.google.com with SMTP id r12so831087uaf.7 for <dev@dpdk.org>; Wed, 25 Jan 2023 05:54:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=L3kIbYsYLzsTTDPJ18ceRfRqqyKBZ0xelB8fFeJ37dQ=; b=HiTDMcf10/FTEC9LzSqcu+4Ez4iHlqJnMF08VXZ3FvvbbSbxRUOSQ7ZS3vswdBIB36 bjqDtfaxcXziYkxWlEO929iMa2auWdpztvkCLuyKqPml2ROXSaG09SRZKTH2Q5IUarHz OeDNhNK7ohClF8UNjAS18u/gNM+crCQiBTLXc5e7BmwA1Djvhqf46iabURycTcOlXWIE 5D9XjO43Pzx/DDbIF8s6u1MqDd80PHKuPn5Wc2pgRu3mC0H3xyKHNWQqBKmIUaA7c982 NThI9W+XJLVH/TrTExefDsnJwSgo5I7bsDY/0ulB+S551b9bB+Y932+y+rOMSjoxVZBB gbxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=L3kIbYsYLzsTTDPJ18ceRfRqqyKBZ0xelB8fFeJ37dQ=; b=izmhnFc/gmndyIWc+iwH3DRLQegmdo94P+2bbAO9S4/qLSi7wdFGPTD19AXE60MqeW y9UVTPJHOvVIugX5mZoDNvrJjqKIRzGBxe7ftiOgDrmUEYEt/hpRvDvNcXlg40KbhFJM rvkFYUxT94BrqWgb2qvIE9Gg+7Rt4+FGGpCa182fOHy8nQgjBEvzgKnDbH4DJ58mqwNp 11TSXSFBroC46aK8RYq6qw/ptW9tXPSyeho/rFe96IbHJ3yeGFbqTdJLnZgDwtrXDzc0 iDUhoEQf9BqdcmA4HpIodGToo8zAz3hTonyl34HfhAiJZ3ChqfqFwB4BuUrs3y/Q+57U 6/bw== X-Gm-Message-State: AO0yUKU0hDlzOSD0ZBs5o357K604xJlRe1rtDlWA/l4jOz98HKBptLrT LF+IPq/1BT2YBnDGz8q4i0KbOoBbJoQTkydW3X8= X-Google-Smtp-Source: AK7set/wjFUokafWaPkLgW3otblPrvVS3K73STlURAeTm7vsbMT4MEv/uCTntwe0wp/lgSp0ui92KbdqLwzHklD2Nbo= X-Received: by 2002:a9f:2bc6:0:b0:652:b4f6:d92d with SMTP id f6-20020a9f2bc6000000b00652b4f6d92dmr907297uaj.61.1674654873743; Wed, 25 Jan 2023 05:54:33 -0800 (PST) MIME-Version: 1.0 References: <20220803132839.2747858-1-jerinj@marvell.com> <DBAPR08MB5814B8F56B6E7DE75A85A8EF986B9@DBAPR08MB5814.eurprd08.prod.outlook.com> <CALBAE1PqO0-=foyAuhVuNNoS27LszNqXtmv3BUDGF-W83KHcfQ@mail.gmail.com> <4520101.QLehXeTyEo@thomas> In-Reply-To: <4520101.QLehXeTyEo@thomas> From: Jerin Jacob <jerinjacobk@gmail.com> Date: Wed, 25 Jan 2023 19:24:07 +0530 Message-ID: <CALBAE1MnHTyJSBQjpHFzN6m5FRd2aaC9cP91LqYbeZjBf1aE+w@mail.gmail.com> Subject: Re: [dpdk-dev] [RFC PATCH 0/1] mldev: introduce machine learning device library To: Thomas Monjalon <thomas@monjalon.net> Cc: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>, dev@dpdk.org, =?UTF-8?Q?Morten_Br=C3=B8rup?= <mb@smartsharesystems.com>, "jerinj@marvell.com" <jerinj@marvell.com>, Ferruh Yigit <ferruh.yigit@xilinx.com>, "Ajit Khaparde (ajit.khaparde@broadcom.com)" <ajit.khaparde@broadcom.com>, Andrew Boyer <aboyer@pensando.io>, Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>, Beilei Xing <beilei.xing@intel.com>, "Richardson, Bruce" <bruce.richardson@intel.com>, Chas Williams <chas3@att.com>, "Xia, Chenbo" <chenbo.xia@intel.com>, Ciara Loftus <ciara.loftus@intel.com>, Devendra Singh Rawat <dsinghrawat@marvell.com>, Ed Czeck <ed.czeck@atomicrules.com>, Evgeny Schemeilin <evgenys@amazon.com>, Gaetan Rivet <grive@u256.net>, Gagandeep Singh <g.singh@nxp.com>, Guoyang Zhou <zhouguoyang@huawei.com>, Haiyue Wang <haiyue.wang@intel.com>, Harman Kalra <hkalra@marvell.com>, Heinrich Kuhn <heinrich.kuhn@corigine.com>, "hemant.agrawal@nxp.com" <hemant.agrawal@nxp.com>, Hyong Youb Kim <hyonkim@cisco.com>, Igor Chauskin <igorch@amazon.com>, Igor Russkikh <irusskikh@marvell.com>, Jakub Grajciar <jgrajcia@cisco.com>, Jasvinder Singh <jasvinder.singh@intel.com>, Jian Wang <jianwang@trustnetic.com>, Jiawen Wu <jiawenwu@trustnetic.com>, Jingjing Wu <jingjing.wu@intel.com>, John Daley <johndale@cisco.com>, John Miller <john.miller@atomicrules.com>, "John W. Linville" <linville@tuxdriver.com>, "Wiles, Keith" <keith.wiles@intel.com>, Kiran Kumar K <kirankumark@marvell.com>, Lijun Ou <oulijun@huawei.com>, Liron Himi <lironh@marvell.com>, Long Li <longli@microsoft.com>, Marcin Wojtas <mw@semihalf.com>, Martin Spinler <spinler@cesnet.cz>, Matan Azrad <matan@nvidia.com>, Matt Peters <matt.peters@windriver.com>, Maxime Coquelin <maxime.coquelin@redhat.com>, Michal Krawczyk <mk@semihalf.com>, "Min Hu (Connor" <humin29@huawei.com>, Pradeep Kumar Nalla <pnalla@marvell.com>, Nithin Dabilpuram <ndabilpuram@marvell.com>, Qiming Yang <qiming.yang@intel.com>, Qi Zhang <qi.z.zhang@intel.com>, Radha Mohan Chintakuntla <radhac@marvell.com>, Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>, Rasesh Mody <rmody@marvell.com>, Rosen Xu <rosen.xu@intel.com>, Sachin Saxena <sachin.saxena@oss.nxp.com>, Satha Koteswara Rao Kottidi <skoteshwar@marvell.com>, Shahed Shaikh <shshaikh@marvell.com>, Shai Brandes <shaibran@amazon.com>, Shepard Siegel <shepard.siegel@atomicrules.com>, Somalapuram Amaranath <asomalap@amd.com>, Somnath Kotur <somnath.kotur@broadcom.com>, Stephen Hemminger <sthemmin@microsoft.com>, Steven Webster <steven.webster@windriver.com>, Sunil Kumar Kori <skori@marvell.com>, Tetsuya Mukawa <mtetsuyah@gmail.com>, Veerasenareddy Burru <vburru@marvell.com>, nd <nd@arm.com>, Viacheslav Ovsiienko <viacheslavo@nvidia.com>, Xiao Wang <xiao.w.wang@intel.com>, Xiaoyun Wang <cloud.wangxiaoyun@huawei.com>, Yisen Zhuang <yisen.zhuang@huawei.com>, Yong Wang <yongwang@vmware.com>, Ziyang Xuan <xuanziyang2@huawei.com>, Prasun Kapoor <pkapoor@marvell.com>, "nadavh@marvell.com" <nadavh@marvell.com>, Satananda Burla <sburla@marvell.com>, Narayana Prasad <pathreya@marvell.com>, Akhil Goyal <gakhil@marvell.com>, Ray Kinsella <mdr@ashroe.eu>, Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>, Anatoly Burakov <anatoly.burakov@intel.com>, Cristian Dumitrescu <cristian.dumitrescu@intel.com>, =?UTF-8?Q?Mattias_R=C3=B6nnblom?= <mattias.ronnblom@ericsson.com>, Ruifeng Wang <Ruifeng.Wang@arm.com>, David Christensen <drc@linux.vnet.ibm.com>, "Ananyev, Konstantin" <konstantin.ananyev@intel.com>, Olivier Matz <olivier.matz@6wind.com>, "Jayatheerthan, Jay" <jay.jayatheerthan@intel.com>, Ashwin Sekhar Thalakalath Kottilveetil <asekhar@marvell.com>, Pavan Nikhilesh <pbhagavatula@marvell.com>, Elena Agostini <eagostini@nvidia.com>, Srikanth Yalavarthi <syalavarthi@marvell.com>, "dchickles@marvell.com" <dchickles@marvell.com>, "sshankarnara@marvell.com" <sshankarnara@marvell.com>, John McNamara <john.mcnamara@intel.com>, Stephen Hemminger <stephen@networkplumber.org> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org On Wed, Jan 25, 2023 at 7:17 PM Thomas Monjalon <thomas@monjalon.net> wrote= : > > 17/08/2022 16:53, Jerin Jacob: > > On Tue, Aug 16, 2022 at 10:04 PM Honnappa Nagarahalli > > <Honnappa.Nagarahalli@arm.com> wrote: > > > > > > <snip> > > > > > > > > From: Jerin Jacob [mailto:jerinjacobk@gmail.com] > > > > > Sent: Tuesday, 16 August 2022 15.13 > > > > > > > > > > On Wed, Aug 3, 2022 at 8:49 PM Stephen Hemminger > > > > > <stephen@networkplumber.org> wrote: > > > > > > > > > > > > On Wed, 3 Aug 2022 18:58:37 +0530 > > > > > > <jerinj@marvell.com> wrote: > > > > > > > > > > > > > Roadmap > > > > > > > ------- > > > > > > > 1) Address the comments for this RFC. > > > > > > > 2) Common code for mldev > > > > > > > 3) SW mldev driver based on TVM (https://tvm.apache.org/) > > > > > > > > > > > > Having a SW implementation is important because then it can be > > > > > covered > > > > > > by tests. > > > > > > > > > > Yes. That reason for adding TVM based SW driver as item (3). > > > > > > > > > > Is there any other high level or API level comments before procee= ding > > > > > with v1 and implementation. > > > > > > > > Have you seriously considered if the DPDK Project is the best home = for this > > > > project? I can easily imagine the DPDK development process being a = hindrance > > > > in many aspects for an evolving AI/ML library. Off the top of my he= ad, it would > > > > probably be better off as a separate project, like SPDK. > > > There is a lot of talk about using ML in networking workloads. Althou= gh, I am not very sure on how the use case looks like. For ex: is the infer= ence engine going to be inline (i.e. the packet goes through the inference = engine before coming to the CPU and provide some data (what sort of data?))= , look aside (does it require the packets to be sent to the inference engin= e or is it some other data?), what would be an end to end use case? A sampl= e application using these APIs would be helpful. > > > > Simple application for the inference usage is added in the cover letter= . > > > > Regarding the use cases, There are many like firewall, intrusion > > detection etc. Most of the use cases are driven by product > > requirements and SW IP vendors try to keep it to themselves as a > > product differentiate factor. > > That is the prime reason for DPDK scope only for inference where IO is > > involved. Model creation and training etc will heavily vary based on > > use case but not the inference model. > > > > > > > > IMO, if we need to share the packets with the inference engine, then = it fits into DPDK. > > > > Yes. Yes for networking or ORAN use cases the interface data comes > > over wire and result can go over wire. > > > > > > > > As I understand, there are many mature open source projects for ML/in= ference outside of DPDK. Does it make sense for DPDK to adopt those project= s rather than inventing our own? > > > > # AI/ML compiler libraries more focused on model creation and > > training etc (Thats where actual value addition the AI/ML libraries > > can offer) and > > minimal part for inference (It is just added for testing the model) > > # Considering the inference is the scope of the DPDK. DPDK is ideal > > place for following reasons > > > > a) Inference scope is very limited. > > b) Avoid memcpy of inference data (Use directly from network or > > other class of device like cryptodev, regexdev) > > c) Reuse highspeed IO interface like PCI backed driver etc > > d) Integration with other DPDK subsystems like eventdev etc for job com= pletion. > > e) Also support more inline offloads by merging two device classes > > like rte_secuity. > > f) Run the inference model from different AI/ML compiler frameworks or > > abstract the inference usage. > > Similar concept is already applied to other DPDK device classes like > > 1) In Regexdev, The compiler generates the rule database which is out > > of scope of DPDK. DPDK API just loads the rule database > > 2) In Gpudev, The GPU kernel etc out of scope of DPDK.DPDK cares about > > IO interface. > > I think Honnappa was thinking about linking an existing inference library= . > What are the similar libraries? Not sure. Honnappa can tell if any which meets (a) to (f). > >