From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5622AA034F; Sun, 6 Jun 2021 07:28:37 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D00504069D; Sun, 6 Jun 2021 07:28:36 +0200 (CEST) Received: from mail-il1-f173.google.com (mail-il1-f173.google.com [209.85.166.173]) by mails.dpdk.org (Postfix) with ESMTP id 6EC564014E for ; Sun, 6 Jun 2021 07:28:35 +0200 (CEST) Received: by mail-il1-f173.google.com with SMTP id i17so13014337ilj.11 for ; Sat, 05 Jun 2021 22:28:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=GHuMmc+lZoPze8oqgoaayN6gVBWuTKWS0Fc3T8YBMj4=; b=VqOn5MGpZHth30Edbe9I9tkJFhu2Xj6dx8AZQibC0icOfUVIq5BmYaMqkUZoYNVKw7 N7foOjWJqFWlTgtck5kVrIElXhxMX/w2E5I160/sxxHhpCBRWQX1Lkhh2wS9+NX3mgRJ DDAoMOzbJf40+4HYJs3GvgNqPNlcOZGd0CBjnUOHZriAOO5L+WpqzFaE0Yk1no8r/vBu MIWRKG56c42C/gv0Fogg55ySEFWYb6u+v/jnNzvhHdQYdVYeJXngrvIFJTKAvOgxXqxA QhxqRCWjasptdV8ZydaYnJ51UHuY4uLdCYRqw+miyyhH3OGx0AfQWn6l7yeqW8P8m9TJ +3qw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=GHuMmc+lZoPze8oqgoaayN6gVBWuTKWS0Fc3T8YBMj4=; b=qzoiMflR3QkY9MUvtnFJ2v/vqNLzkfvj70V6NsTbUu0SFexrKLgbrpLEmjIZHF3EOq zufoHYliQ1DZkpv4oVzeHJXoQB9xTAEEw9GSXuUHNzWvGEx8tc03aQis+COTIL4REGlY t3ueB7rhqD2itMYhB8GWEAep5KHDSSJUG0Xkx3srG3Zddp0lo3JtIq2wdVCigYU17qyL n2c1FnZfkciu/S72Vs5d9hDHmD42lp+ZCzWI7WRUYgfCj7jurfoQ/BCN3e8tFm+7vKM7 k1jonL9CFrMfW6nBB9gmNQSz7mzGT2uGPhpvfHcyXVqWuXR8OC1vquSAYcre6FlRWO1w Xaqg== X-Gm-Message-State: AOAM533TPxuIlqRKeOmpZx67E2r2dRO59t9jkEAuGoCPdhfltHns8nOA +2O+6M/8Cu2RZkrg9FcHFo61lzQ50ymUazQgBP4= X-Google-Smtp-Source: ABdhPJy8h4G3jrJo1QS6ofjxWncZ7YmBGBX+Af6dHFX9dNVM4gJ6W2LpmdD59QmdBl/pXkuYrmkgl1FiCs+bromGNL0= X-Received: by 2002:a92:1310:: with SMTP id 16mr10970324ilt.60.1622957314597; Sat, 05 Jun 2021 22:28:34 -0700 (PDT) MIME-Version: 1.0 References: <20210602203531.2288645-1-thomas@monjalon.net> <1762355.HKltZAk3iZ@thomas> <2385250.mgn969muHi@thomas> In-Reply-To: From: Jerin Jacob Date: Sun, 6 Jun 2021 10:58:18 +0530 Message-ID: To: Honnappa Nagarahalli Cc: "Wang, Haiyue" , "thomas@monjalon.net" , Andrew Rybchenko , "Yigit, Ferruh" , dpdk-dev , Elena Agostini , David Marchand , nd Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [dpdk-dev] [PATCH] gpudev: introduce memory API X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Sun, Jun 6, 2021 at 6:44 AM Honnappa Nagarahalli wrote: > > > > > > > > > > > 04/06/2021 17:20, Jerin Jacob: > > > > > On Fri, Jun 4, 2021 at 7:39 PM Thomas Monjalon > > wrote: > > > > > > 04/06/2021 15:59, Andrew Rybchenko: > > > > > > > On 6/4/21 4:18 PM, Thomas Monjalon wrote: > > > > > > > > 04/06/2021 15:05, Andrew Rybchenko: > > > > > > > >> On 6/4/21 3:46 PM, Thomas Monjalon wrote: > > > > > > > >>> 04/06/2021 13:09, Jerin Jacob: > > > > > > > >>>> On Fri, Jun 4, 2021 at 3:58 PM Thomas Monjalon > > wrote: > > > > > > > >>>>> 03/06/2021 11:33, Ferruh Yigit: > > > > > > > >>>>>> On 6/3/2021 8:47 AM, Jerin Jacob wrote: > > > > > > > >>>>>>> On Thu, Jun 3, 2021 at 2:05 AM Thomas Monjalon > > wrote: > > > > > > > >>>>>>>> + [gpudev] (@ref rte_gpudev.h), > > > > > > > >>>>>>> > > > > > > > >>>>>>> Since this device does not have a queue etc? Shouldn'= t > > > > > > > >>>>>>> make it a library like mempool with vendor-defined op= s? > > > > > > > >>>>>> > > > > > > > >>>>>> +1 > > > > > > > >>>>>> > > > > > > > >>>>>> Current RFC announces additional memory allocation > > > > > > > >>>>>> capabilities, which can suits better as extension to > > > > > > > >>>>>> existing memory related library instead of a new devic= e > > abstraction library. > > > > > > > >>>>> > > > > > > > >>>>> It is not replacing mempool. > > > > > > > >>>>> It is more at the same level as EAL memory management: > > > > > > > >>>>> allocate simple buffer, but with the exception it is > > > > > > > >>>>> done on a specific device, so it requires a device ID. > > > > > > > >>>>> > > > > > > > >>>>> The other reason it needs to be a full library is that > > > > > > > >>>>> it will start a workload on the GPU and get completion > > > > > > > >>>>> notification so we can integrate the GPU workload in a = packet > > processing pipeline. > > > > > > > >>>> > > > > > > > >>>> I might have confused you. My intention is not to make t= o fit > > under mempool API. > > > > > > > >>>> > > > > > > > >>>> I agree that we need a separate library for this. My > > > > > > > >>>> objection is only to not call libgpudev and call it > > > > > > > >>>> libgpu. And have APIs with rte_gpu_ instead of > > > > > > > >>>> rte_gpu_dev as it not like existing "device libraries" i= n > > > > > > > >>>> DPDK and it like other "libraries" in DPDK. > > > > > > > >>> > > > > > > > >>> I think we should define a queue of processing actions, s= o > > > > > > > >>> it looks like other device libraries. > > > > > > > >>> And anyway I think a library managing a device class, and > > > > > > > >>> having some device drivers deserves the name of device li= brary. > > > > > > > >>> > > > > > > > >>> I would like to read more opinions. > > > > > > > >> > > > > > > > >> Since the library is an unified interface to GPU device > > > > > > > >> drivers I think it should be named as in the patch - gpude= v. > > > > > > > >> > > > > > > > >> Mempool looks like an exception here - initially it was > > > > > > > >> pure SW library, but not there are HW backends and > > > > > > > >> corresponding device drivers. > > > > > > > >> > > > > > > > >> What I don't understand where is GPU specifics here? > > > > > > > > > > > > > > > > That's an interesting question. > > > > > > > > Let's ask first what is a GPU for DPDK? > > > > > > > > I think it is like a sub-CPU with high parallel execution > > > > > > > > capabilities, and it is controlled by the CPU. > > > > > > > > > > > > > > I have no good ideas how to name it in accordance with above > > > > > > > description to avoid "G" which for "Graphics" if understand > > > > > > > correctly. However, may be it is not required. > > > > > > > No strong opinion on the topic, but unbinding from "Graphics" > > > > > > > would be nice. > > > > > > > > > > > > That's a question I ask myself for months now. > > > > > > I am not able to find a better name, and I start thinking that > > > > > > "GPU" is famous enough in high-load computing to convey the ide= a > > > > > > of what we can expect. > > > > > > > > > > > > > > > The closest I can think of is big-little architecture in ARM SoC. > > > > > https://www.arm.com/why-arm/technologies/big-little > From the application pov, big-little arch is nothing but SMT. Not sure ho= w it is similar to another device on PCIe. Yes. It may not be a device sitting on a PCIe bus, However, It can access it via some bus from the main CPU. > > > > > > > > > > > We do have similar architecture, Where the "coprocessor" is part > > > > > of the main CPU. > > > > > It is operations are: > > > > > - Download firmware > > > > > - Memory mapping for Main CPU memory by the co-processor > > > > > - Enq/Deq Jobs from/to Main CPU/Coprocessor CPU. > > > > > > > > Yes it looks like the exact same scope. > > > > I like the word "co-processor" in this context. > > > > > > > > > If your scope is something similar and No Graphics involved here > > > > > then we can remove G. > > > > > > > > Indeed no graphics in DPDK :) > > > > By removing the G, you mean keeping only PU? like "pudev"? > > > > We could also define the G as "General". > > > > > > > > > Coincidentally, Yesterday, I had an interaction with Elena for th= e > > > > > same for BaseBand related work in ORAN where GPU used as Baseband > > > > > processing instead of Graphics.(So I can understand the big > > > > > picture of this library) > This patch does not provide the big picture view of what the processing l= ooks like using GPU. It would be good to explain that. > For ex: > 1) Will the notion of GPU hidden from the application? i.e. is the applic= ation allowed to launch kernels? > 1a) Will DPDK provide abstract APIs to launch kernels? > This would require us to have the notion of GPU in DPDK and the appl= ication would depend on the availability of GPU in the system. > 2) Is launching kernels hidden? i.e. the application still calls DPDK abs= tract APIs (such as encryption/decryption APIs) without knowing that the en= cryption/decryption is happening on GPU. > This does not require us to have a notion of GPU in DPDK at the API = level I will leave this to Thomas. > > If we keep CXL in mind, I would imagine that in the future the devices on= PCIe could have their own local memory. May be some of the APIs could use = generic names. For ex: instead of calling it as "rte_gpu_malloc" may be we = could call it as "rte_dev_malloc". This way any future device which hosts i= ts own memory that need to be managed by the application, can use these API= s. That is a good thought. it is possible to hook the download firmware, memory management, Job management(as messages to/from device) to rte_device itself. I think, one needs to consider, how to integrate with the existing DPDK subsystem, for example: If one decided to implement bbdev or regexdev with such computing device, Need to consider, Is it better to have bbdev driver has depended gpudev or rte_device has this callback and use with bbdev driver. > > > > > > > > > > Yes baseband processing is one possible usage of GPU with DPDK. > > > > We could also imagine some security analysis, or any machine learni= ng... > > > > > > > > > I can think of "coprocessor-dev" as one of the name. > > > > > > > > "coprocessor" looks too long as prefix of the functions. > > > > Yes. Libray name can be lengthy, but API prefix should be 3 letters kin= d short > > form will be required. > > > > > > > > > > > > > We do have similar machine learning co-processors(for compute) if > > > > > we can keep a generic name and it is for the above functions we > > > > > may use this subsystem as well in the future. > > > > > > > > > > Accelerator, 'acce_dev' ? ;-) > > > > It may get confused with HW accelerators. > > > > > > Some of the options I can think of. Sorting in my preference. > > > > library name, API prefix > > 1) libhpc-dev, rte_hpc_ (hpc-> Heterogeneous processor compute) > > 2) libhc-dev, rte_hc_ > > (https://en.wikipedia.org/wiki/Heterogeneous_computing see: Example > > hardware) > > 3) libpu-dev, rte_pu_ (pu -> processing unit) > > 4) libhp-dev, rte_hp_ (hp->heterogeneous processor) > > 5) libcoprocessor-dev, rte_cps_ ? > > 6) libcompute-dev, rte_cpt_ ? > > 7) libgpu-dev, rte_gpu_ > These seem to assume that the application can launch its own workload on = the device? Does DPDK need to provide abstract APIs for launching work on a= device? > > > > > > > > > > > > > > > > > Yes that's the idea to share a common synchronization mechanism wit= h > > > > different HW. > > > > > > > > That's cool to have such a big interest in the community for this p= atch. > > > > > > >