From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
	by inbox.dpdk.org (Postfix) with ESMTP id 6E94742421;
	Wed, 25 Jan 2023 14:54:35 +0100 (CET)
Received: from mails.dpdk.org (localhost [127.0.0.1])
	by mails.dpdk.org (Postfix) with ESMTP id 0FFFA42D3E;
	Wed, 25 Jan 2023 14:54:35 +0100 (CET)
Received: from mail-ua1-f46.google.com (mail-ua1-f46.google.com
 [209.85.222.46]) by mails.dpdk.org (Postfix) with ESMTP id 8973942D31
 for <dev@dpdk.org>; Wed, 25 Jan 2023 14:54:34 +0100 (CET)
Received: by mail-ua1-f46.google.com with SMTP id r12so831087uaf.7
 for <dev@dpdk.org>; Wed, 25 Jan 2023 05:54:34 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
 h=content-transfer-encoding:cc:to:subject:message-id:date:from
 :in-reply-to:references:mime-version:from:to:cc:subject:date
 :message-id:reply-to;
 bh=L3kIbYsYLzsTTDPJ18ceRfRqqyKBZ0xelB8fFeJ37dQ=;
 b=HiTDMcf10/FTEC9LzSqcu+4Ez4iHlqJnMF08VXZ3FvvbbSbxRUOSQ7ZS3vswdBIB36
 bjqDtfaxcXziYkxWlEO929iMa2auWdpztvkCLuyKqPml2ROXSaG09SRZKTH2Q5IUarHz
 OeDNhNK7ohClF8UNjAS18u/gNM+crCQiBTLXc5e7BmwA1Djvhqf46iabURycTcOlXWIE
 5D9XjO43Pzx/DDbIF8s6u1MqDd80PHKuPn5Wc2pgRu3mC0H3xyKHNWQqBKmIUaA7c982
 NThI9W+XJLVH/TrTExefDsnJwSgo5I7bsDY/0ulB+S551b9bB+Y932+y+rOMSjoxVZBB
 gbxQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20210112;
 h=content-transfer-encoding:cc:to:subject:message-id:date:from
 :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
 :subject:date:message-id:reply-to;
 bh=L3kIbYsYLzsTTDPJ18ceRfRqqyKBZ0xelB8fFeJ37dQ=;
 b=izmhnFc/gmndyIWc+iwH3DRLQegmdo94P+2bbAO9S4/qLSi7wdFGPTD19AXE60MqeW
 y9UVTPJHOvVIugX5mZoDNvrJjqKIRzGBxe7ftiOgDrmUEYEt/hpRvDvNcXlg40KbhFJM
 rvkFYUxT94BrqWgb2qvIE9Gg+7Rt4+FGGpCa182fOHy8nQgjBEvzgKnDbH4DJ58mqwNp
 11TSXSFBroC46aK8RYq6qw/ptW9tXPSyeho/rFe96IbHJ3yeGFbqTdJLnZgDwtrXDzc0
 iDUhoEQf9BqdcmA4HpIodGToo8zAz3hTonyl34HfhAiJZ3ChqfqFwB4BuUrs3y/Q+57U
 6/bw==
X-Gm-Message-State: AO0yUKU0hDlzOSD0ZBs5o357K604xJlRe1rtDlWA/l4jOz98HKBptLrT
 LF+IPq/1BT2YBnDGz8q4i0KbOoBbJoQTkydW3X8=
X-Google-Smtp-Source: AK7set/wjFUokafWaPkLgW3otblPrvVS3K73STlURAeTm7vsbMT4MEv/uCTntwe0wp/lgSp0ui92KbdqLwzHklD2Nbo=
X-Received: by 2002:a9f:2bc6:0:b0:652:b4f6:d92d with SMTP id
 f6-20020a9f2bc6000000b00652b4f6d92dmr907297uaj.61.1674654873743; Wed, 25 Jan
 2023 05:54:33 -0800 (PST)
MIME-Version: 1.0
References: <20220803132839.2747858-1-jerinj@marvell.com>
 <DBAPR08MB5814B8F56B6E7DE75A85A8EF986B9@DBAPR08MB5814.eurprd08.prod.outlook.com>
 <CALBAE1PqO0-=foyAuhVuNNoS27LszNqXtmv3BUDGF-W83KHcfQ@mail.gmail.com>
 <4520101.QLehXeTyEo@thomas>
In-Reply-To: <4520101.QLehXeTyEo@thomas>
From: Jerin Jacob <jerinjacobk@gmail.com>
Date: Wed, 25 Jan 2023 19:24:07 +0530
Message-ID: <CALBAE1MnHTyJSBQjpHFzN6m5FRd2aaC9cP91LqYbeZjBf1aE+w@mail.gmail.com>
Subject: Re: [dpdk-dev] [RFC PATCH 0/1] mldev: introduce machine learning
 device library
To: Thomas Monjalon <thomas@monjalon.net>
Cc: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>, dev@dpdk.org, 
 =?UTF-8?Q?Morten_Br=C3=B8rup?= <mb@smartsharesystems.com>, 
 "jerinj@marvell.com" <jerinj@marvell.com>,
 Ferruh Yigit <ferruh.yigit@xilinx.com>, 
 "Ajit Khaparde (ajit.khaparde@broadcom.com)" <ajit.khaparde@broadcom.com>,
 Andrew Boyer <aboyer@pensando.io>, 
 Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>,
 Beilei Xing <beilei.xing@intel.com>, 
 "Richardson, Bruce" <bruce.richardson@intel.com>, Chas Williams <chas3@att.com>,
 "Xia, Chenbo" <chenbo.xia@intel.com>, Ciara Loftus <ciara.loftus@intel.com>, 
 Devendra Singh Rawat <dsinghrawat@marvell.com>,
 Ed Czeck <ed.czeck@atomicrules.com>, 
 Evgeny Schemeilin <evgenys@amazon.com>, Gaetan Rivet <grive@u256.net>,
 Gagandeep Singh <g.singh@nxp.com>, 
 Guoyang Zhou <zhouguoyang@huawei.com>, Haiyue Wang <haiyue.wang@intel.com>, 
 Harman Kalra <hkalra@marvell.com>, Heinrich Kuhn <heinrich.kuhn@corigine.com>, 
 "hemant.agrawal@nxp.com" <hemant.agrawal@nxp.com>,
 Hyong Youb Kim <hyonkim@cisco.com>, 
 Igor Chauskin <igorch@amazon.com>, Igor Russkikh <irusskikh@marvell.com>, 
 Jakub Grajciar <jgrajcia@cisco.com>,
 Jasvinder Singh <jasvinder.singh@intel.com>, 
 Jian Wang <jianwang@trustnetic.com>, Jiawen Wu <jiawenwu@trustnetic.com>, 
 Jingjing Wu <jingjing.wu@intel.com>, John Daley <johndale@cisco.com>, 
 John Miller <john.miller@atomicrules.com>,
 "John W. Linville" <linville@tuxdriver.com>, 
 "Wiles, Keith" <keith.wiles@intel.com>, Kiran Kumar K <kirankumark@marvell.com>,
 Lijun Ou <oulijun@huawei.com>, Liron Himi <lironh@marvell.com>,
 Long Li <longli@microsoft.com>, 
 Marcin Wojtas <mw@semihalf.com>, Martin Spinler <spinler@cesnet.cz>,
 Matan Azrad <matan@nvidia.com>, Matt Peters <matt.peters@windriver.com>,
 Maxime Coquelin <maxime.coquelin@redhat.com>, 
 Michal Krawczyk <mk@semihalf.com>, "Min Hu (Connor" <humin29@huawei.com>, 
 Pradeep Kumar Nalla <pnalla@marvell.com>,
 Nithin Dabilpuram <ndabilpuram@marvell.com>, 
 Qiming Yang <qiming.yang@intel.com>, Qi Zhang <qi.z.zhang@intel.com>, 
 Radha Mohan Chintakuntla <radhac@marvell.com>,
 Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>, 
 Rasesh Mody <rmody@marvell.com>, Rosen Xu <rosen.xu@intel.com>, 
 Sachin Saxena <sachin.saxena@oss.nxp.com>, 
 Satha Koteswara Rao Kottidi <skoteshwar@marvell.com>,
 Shahed Shaikh <shshaikh@marvell.com>, Shai Brandes <shaibran@amazon.com>,
 Shepard Siegel <shepard.siegel@atomicrules.com>, 
 Somalapuram Amaranath <asomalap@amd.com>,
 Somnath Kotur <somnath.kotur@broadcom.com>, 
 Stephen Hemminger <sthemmin@microsoft.com>,
 Steven Webster <steven.webster@windriver.com>, 
 Sunil Kumar Kori <skori@marvell.com>, Tetsuya Mukawa <mtetsuyah@gmail.com>, 
 Veerasenareddy Burru <vburru@marvell.com>, nd <nd@arm.com>, 
 Viacheslav Ovsiienko <viacheslavo@nvidia.com>,
 Xiao Wang <xiao.w.wang@intel.com>, 
 Xiaoyun Wang <cloud.wangxiaoyun@huawei.com>,
 Yisen Zhuang <yisen.zhuang@huawei.com>, 
 Yong Wang <yongwang@vmware.com>, Ziyang Xuan <xuanziyang2@huawei.com>, 
 Prasun Kapoor <pkapoor@marvell.com>, "nadavh@marvell.com" <nadavh@marvell.com>,
 Satananda Burla <sburla@marvell.com>, Narayana Prasad <pathreya@marvell.com>, 
 Akhil Goyal <gakhil@marvell.com>, Ray Kinsella <mdr@ashroe.eu>, 
 Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>,
 Anatoly Burakov <anatoly.burakov@intel.com>, 
 Cristian Dumitrescu <cristian.dumitrescu@intel.com>, 
 =?UTF-8?Q?Mattias_R=C3=B6nnblom?= <mattias.ronnblom@ericsson.com>, 
 Ruifeng Wang <Ruifeng.Wang@arm.com>, David Christensen <drc@linux.vnet.ibm.com>,
 "Ananyev, Konstantin" <konstantin.ananyev@intel.com>,
 Olivier Matz <olivier.matz@6wind.com>, 
 "Jayatheerthan, Jay" <jay.jayatheerthan@intel.com>, 
 Ashwin Sekhar Thalakalath Kottilveetil <asekhar@marvell.com>,
 Pavan Nikhilesh <pbhagavatula@marvell.com>, 
 Elena Agostini <eagostini@nvidia.com>,
 Srikanth Yalavarthi <syalavarthi@marvell.com>, 
 "dchickles@marvell.com" <dchickles@marvell.com>, 
 "sshankarnara@marvell.com" <sshankarnara@marvell.com>,
 John McNamara <john.mcnamara@intel.com>, 
 Stephen Hemminger <stephen@networkplumber.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org

On Wed, Jan 25, 2023 at 7:17 PM Thomas Monjalon <thomas@monjalon.net> wrote=
:
>
> 17/08/2022 16:53, Jerin Jacob:
> > On Tue, Aug 16, 2022 at 10:04 PM Honnappa Nagarahalli
> > <Honnappa.Nagarahalli@arm.com> wrote:
> > >
> > > <snip>
> > >
> > > > > From: Jerin Jacob [mailto:jerinjacobk@gmail.com]
> > > > > Sent: Tuesday, 16 August 2022 15.13
> > > > >
> > > > > On Wed, Aug 3, 2022 at 8:49 PM Stephen Hemminger
> > > > > <stephen@networkplumber.org> wrote:
> > > > > >
> > > > > > On Wed, 3 Aug 2022 18:58:37 +0530
> > > > > > <jerinj@marvell.com> wrote:
> > > > > >
> > > > > > > Roadmap
> > > > > > > -------
> > > > > > > 1) Address the comments for this RFC.
> > > > > > > 2) Common code for mldev
> > > > > > > 3) SW mldev driver based on TVM (https://tvm.apache.org/)
> > > > > >
> > > > > > Having a SW implementation is important because then it can be
> > > > > covered
> > > > > > by tests.
> > > > >
> > > > > Yes. That reason for adding TVM based SW driver as item (3).
> > > > >
> > > > > Is there any other high level or API level comments before procee=
ding
> > > > > with v1 and implementation.
> > > >
> > > > Have you seriously considered if the DPDK Project is the best home =
for this
> > > > project? I can easily imagine the DPDK development process being a =
hindrance
> > > > in many aspects for an evolving AI/ML library. Off the top of my he=
ad, it would
> > > > probably be better off as a separate project, like SPDK.
> > > There is a lot of talk about using ML in networking workloads. Althou=
gh, I am not very sure on how the use case looks like. For ex: is the infer=
ence engine going to be inline (i.e. the packet goes through the inference =
engine before coming to the CPU and provide some data (what sort of data?))=
, look aside (does it require the packets to be sent to the inference engin=
e or is it some other data?), what would be an end to end use case? A sampl=
e application using these APIs would be helpful.
> >
> > Simple application for the inference usage is added in the cover letter=
.
> >
> > Regarding the use cases, There are many like firewall, intrusion
> > detection etc. Most of the use cases are driven by product
> > requirements and SW IP vendors try to keep it to themselves as a
> > product differentiate factor.
> > That is the prime reason for DPDK scope only for inference where IO is
> > involved. Model creation and training etc will heavily vary based on
> > use case but not the inference model.
> >
> > >
> > > IMO, if we need to share the packets with the inference engine, then =
it fits into DPDK.
> >
> > Yes. Yes for networking or ORAN use cases the interface data comes
> > over wire and result can go over wire.
> >
> > >
> > > As I understand, there are many mature open source projects for ML/in=
ference outside of DPDK. Does it make sense for DPDK to adopt those project=
s rather than inventing our own?
> >
> > #  AI/ML compiler libraries more focused on model creation and
> > training etc (Thats where actual value addition the AI/ML libraries
> > can offer) and
> > minimal part for inference (It is just added for testing the model)
> > # Considering the inference is the scope of the DPDK. DPDK is ideal
> > place for following reasons
> >
> > a) Inference scope is very limited.
> > b) Avoid memcpy of inference data (Use directly from network or
> > other class of device like cryptodev, regexdev)
> > c) Reuse highspeed IO interface like  PCI backed driver etc
> > d) Integration with other DPDK subsystems like eventdev etc for job com=
pletion.
> > e) Also support more inline offloads by merging two device classes
> > like rte_secuity.
> > f) Run the inference model from different AI/ML compiler frameworks or
> > abstract the inference usage.
> > Similar concept is already applied to other DPDK device classes like
> > 1) In Regexdev,  The compiler generates the rule database which is out
> > of scope of DPDK. DPDK API just loads the rule database
> > 2) In Gpudev, The GPU kernel etc out of scope of DPDK.DPDK cares about
> > IO interface.
>
> I think Honnappa was thinking about linking an existing inference library=
.
> What are the similar libraries?

Not sure. Honnappa can tell if any which meets (a) to (f).

>
>