From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 784AAA0525; Fri, 21 Feb 2020 17:14:32 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 5036D1BFAD; Fri, 21 Feb 2020 17:14:32 +0100 (CET) Received: from new3-smtp.messagingengine.com (new3-smtp.messagingengine.com [66.111.4.229]) by dpdk.org (Postfix) with ESMTP id 482A334F3; Fri, 21 Feb 2020 17:14:31 +0100 (CET) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailnew.nyi.internal (Postfix) with ESMTP id D894475FB; Fri, 21 Feb 2020 11:14:30 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute1.internal (MEProxy); Fri, 21 Feb 2020 11:14:30 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s=mesmtp; bh=VKy9akDGIShCQdMKdTNsv03aQNN4G9Hahv3HUesnfsE=; b=U1mJpsv1fuxQ kVdqeNU3coypBen47IEjof5PGDGY0qDd4iqkLPeowDgqTsmHZISO4tHzjQq+Ho2t 1cw1Iv8Bo2acuMi9xt1Ta+R+yJMF0Pe+kR3hWcpvubpWSFg0WvfcHzL9uouVKyRB 2j0GD9K7EG99vn18PdOS+VCWcJhs0Ys= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm2; bh=VKy9akDGIShCQdMKdTNsv03aQNN4G9Hahv3HUesnf sE=; b=LvY6goY+HAWnIYwf+dOBoClOAhh3/GAuSlWhmO+ll6pQ3EtsPX0n/mmR6 qXWXwM+5L0bDlQOxvae0aFv52QDcuNegWwU0SF3Q4LIr43myaweg2IiblSsbQrOz kkrKqHIUG2wBx5px1tNZpxcPwZxYW0rDqZw43KIkIfq9HND0j1nb7hLbGrgZTzvx 8nDmh/kukDlshbkB+SSzxmTCdgc0bT/7Ni7+EXLm2gefqhgJg0Ls3E5L5jcizdml D06xk/QHNkW56A1ibuZmkGx4vwFXKS4JtQo0QHGhY7Y2vQSgDZ0nIEp3rsmI53D3 c7cWMVnY5ZCxIjGNyESuN1IKA44SA== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedugedrkeeggdekhecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvffufffkjghfggfgtgesthfuredttddtvdenucfhrhhomhepvfhhohhmrghs ucfoohhnjhgrlhhonhcuoehthhhomhgrshesmhhonhhjrghlohhnrdhnvghtqeenucfkph epjeejrddufeegrddvtdefrddukeegnecuvehluhhsthgvrhfuihiivgeptdenucfrrghr rghmpehmrghilhhfrhhomhepthhhohhmrghssehmohhnjhgrlhhonhdrnhgvth X-ME-Proxy: Received: from xps.localnet (184.203.134.77.rev.sfr.net [77.134.203.184]) by mail.messagingengine.com (Postfix) with ESMTPA id EC4FC3060BD1; Fri, 21 Feb 2020 11:14:23 -0500 (EST) From: Thomas Monjalon To: Jerin Jacob Cc: Jerin Jacob , Ray Kinsella , dpdk-dev , Prasun Kapoor , Nithin Dabilpuram , Kiran Kumar K , Pavan Nikhilesh , Narayana Prasad , nsaxena@marvell.com, sshankarnara@marvell.com, Honnappa Nagarahalli , David Marchand , Ferruh Yigit , Andrew Rybchenko , Ajit Khaparde , "Ye, Xiaolong" , Raslan Darawsheh , Maxime Coquelin , Akhil Goyal , Cristian Dumitrescu , John McNamara , "Richardson, Bruce" , Anatoly Burakov , Gavin Hu , David Christensen , "Ananyev, Konstantin" , Pallavi Kadam , Olivier Matz , Gage Eads , "Rao, Nikhil" , Erik Gabriel Carrillo , Hemant Agrawal , "Artem V. Andreev" , Stephen Hemminger , Shahaf Shuler , "Wiles, Keith" , Mattias =?ISO-8859-1?Q?R=F6nnblom?= , Jasvinder Singh , Vladimir Medvedkin , techboard@dpdk.org, Stephen Hemminger , dave@barachs.net Date: Fri, 21 Feb 2020 17:14:22 +0100 Message-ID: <2267983.jE0xQCEvom@xps> In-Reply-To: References: <20200131170201.3236153-1-jerinj@marvell.com> <8553959.CDJkKcVGEf@xps> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" Subject: Re: [dpdk-dev] [RFC PATCH 0/5] graph: introduce graph subsystem X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" 21/02/2020 16:56, Jerin Jacob: > On Fri, Feb 21, 2020 at 4:40 PM Thomas Monjalon wrote: > > 21/02/2020 11:30, Jerin Jacob: > > > On Mon, Feb 17, 2020 at 4:28 PM Jerin Jacob wrote: > > > > On Mon, Feb 17, 2020 at 2:08 PM Thomas Monjalon wrote: > > > > > If we add rte_graph to DPDK, we will have 2 similar libraries. > > > > > > > > > > I already proposed several times to move rte_pipeline in a separate > > > > > repository for two reasons: > > > > > 1/ it is acting at a higher API layer level > > > > > > > > We need to define what is the higher layer API. Is it processing beyond L2? > > > > My opinion is that any API which is implemented differently > > for different hardware should be in DPDK. > > We need to define SIMD optimization(not HW specific to but > architecture-specific) > treatment as well, as the graph and node library will have SIMD > optimization as well. I think SIMD optimization is generic to any performance-related project, not specific to DPDK. > In general, by the above policy enforced, we need to split DPDK like below, > dpdk.git > ---------- > librte_compressdev > librte_bbdev > librte_eventdev > librte_pci > librte_rawdev > librte_eal > librte_security > librte_mempool > librte_mbuf > librte_cryptodev > librte_ethdev > > other repo(s). > ---------------- > librte_cmdline > librte_cfgfile > librte_bitratestats > librte_efd > librte_latencystats > librte_kvargs > librte_jobstats > librte_gso > librte_gro > librte_flow_classify > librte_pipeline > librte_net > librte_metrics > librte_meter > librte_member > librte_table > librte_stack > librte_sched > librte_rib > librte_reorder > librte_rcu > librte_power > librte_distributor > librte_bpf > librte_ip_frag > librte_hash > librte_fib > librte_timer > librte_telemetry > librte_port > librte_pdump > librte_kni > librte_acl > librte_vhost > librte_ring > librte_lpm > librte_ipsec I think it is a fair conclusion of the scope I am arguing, yes. > > Hardware devices can offload protocol processing higher than L2, > > so L2 does not look to be a good limit from my point of view. > > The node may use HW specific optimization if needed. That's an interesting argument. > > > > In the context of Graph library, it is a framework, not using any of > > > > the substem API > > > > other than EAL and it is under lib/librte_graph. > > > > Nodes library using graph and other subsystem components such as ethdev and > > > > it is under lib/lib_node/ > > > > > > > > > > > > Another interesting question would what would be an issue in DPDK supporting > > > > beyond L2. Or higher level protocols? > > > > Definitely higher than L2 is OK in DPDK as long as it is related to hardware > > capabilities, not software stack (which can be a DPDK application). > > The software stack is a vague term. librte_ipsec could be a software stack. I agree. > > > > > 2/ there can be different solutions in this layer > > > > > > > > Is there any issue with that? > > > > There is overlap with the distributor library and eventdev as well. > > > > ethdev and SW traffic manager libraries as well. That list goes on. > > > > I don't know how much it is an issue. > > But I think it shows that at least one implementation is not generic enough. > > I don't think, distributor lies there because of eventdev is not generic. > In fact, SW traffic manager is hooked to ethdev as well. It can work as both. > > > > > > > > > I think 1/ was commonly agreed in the community. > > > > > Now we see one more proof of the reason 2/. > > > > > > > > > > I believe it is time to move rte_pipeline (Packet Framework) > > > > > in a separate repository, and welcome rte_graph as well in another > > > > > separate repository. > > > > > > > > What would be gain out of this? > > > > The gain is to be clear about what should be the focus for contributors > > working on the main DPDK repository. > > Not sure how it can defocus if there is another code in the repo. > If that case, the Linux kernel is not focused at all. I see your point. > > What is expected to be maintained, tested, etc. > > We need to maintain and test other code in OTHER dpdk repo as well. Yes but the ones responsible are not the same. > > > > My concerns are: > > > > # Like packet-gen, The new code will be filled with unnecessary DPDK > > > > version checks > > > > and unnecessary compatibility issues. > > > > # Anything is not in main dpdk repo, it is a second class citizen. > > > > # Customer has the pain to use two repos and two releases. Internally, > > > > it can be two different > > > > repo but release needs to go through one repo. > > > > > > > > If we are focusing ONLY on the driver API then how can DPDK grow > > > > further? If linux kernel > > > > would be thought only have just the kernel and networking/storage as > > > > different repo it would > > > > not have grown up? > > > > Linux kernel is selecting what can enter in the focus or not. > > Sorry. This sentence is not very clear to me. I mean not everything proposed to Linux community is merged. > > And I wonder what is the desire of extending/growing the scope of a library? > > If the HW/Arch accelerated packet processing in the scope of DPDK this > library shall > come to that. > > IMO, As long as there is maintainer, who can give pull request in time > and contribute to > the technical decision of the specific library, I think, that should be enough > to add in dpdk.git. Yes, that's fair. > IMO, we can not get away from more contribution to dpdk. Assume, some set of > library goto pulled out main dpdk.git for some reason. One can still make > new releases say "dpdk-next" to including dpdk,git and various libraries. > Is that something, we are looking to enable as an end solution for > distros and/or > end-users. > > > > > > What is the real concern? Maintenance? > > > > > > > > > I think the original DPDK repository should focus on low-level features > > > > > which offer hardware offloads and optimizations. > > > > > > > > The nodes can be vendor-specific to optimize the specific use cases. > > > > As I mentioned in the cover letter, > > > > > > > > " > > > > 2) Based on our experience, NPU HW accelerates are so different than one vendor > > > > to another vendor. Going forward, We believe, API abstraction may not be enough > > > > abstract the difference in HW. The Vendor-specific nodes can abstract the HW > > > > differences and reuse generic the nodes as needed. > > > > This would help both the silicon vendors and DPDK end users. > > > > " > > > > > > > > Thoughts from other folks? > > > > > > > > > > > > > Consuming the low-level API in different abstractions, > > > > > and building applications, should be done on top of dpdk.git.