Re: [dpdk-dev] [PATCH] gpudev: introduce memory API

[Jerin Jacob <jerinjacobk@gmail.com>]

On Thu, Jun 3, 2021 at 4:00 PM Thomas Monjalon <thomas@monjalon.net> wrote:
>
> 03/06/2021 12:04, Jerin Jacob:
> > On Thu, Jun 3, 2021 at 3:06 PM Thomas Monjalon <thomas@monjalon.net> wrote:
> > >
> > > 03/06/2021 11:20, Jerin Jacob:
> > > > On Thu, Jun 3, 2021 at 2:23 PM Thomas Monjalon <thomas@monjalon.net> wrote:
> > > > >
> > > > > 03/06/2021 10:47, Jerin Jacob:
> > > > > > On Thu, Jun 3, 2021 at 2:13 PM Thomas Monjalon <thomas@monjalon.net> wrote:
> > > > > > >
> > > > > > > 03/06/2021 10:41, Jerin Jacob:
> > > > > > > > On Thu, Jun 3, 2021 at 1:58 PM Thomas Monjalon <thomas@monjalon.net> wrote:
> > > > > > > > >
> > > > > > > > > 03/06/2021 09:47, Jerin Jacob:
> > > > > > > > > > On Thu, Jun 3, 2021 at 2:05 AM Thomas Monjalon <thomas@monjalon.net> wrote:
> > > > > > > > > > > --- a/doc/api/doxy-api-index.md
> > > > > > > > > > > +++ b/doc/api/doxy-api-index.md
> > > > > > > > > > > @@ -21,6 +21,7 @@ The public API headers are grouped by topics:
> > > > > > > > > > >    [compressdev]        (@ref rte_compressdev.h),
> > > > > > > > > > >    [compress]           (@ref rte_comp.h),
> > > > > > > > > > >    [regexdev]           (@ref rte_regexdev.h),
> > > > > > > > > > > +  [gpudev]             (@ref rte_gpudev.h),
> > > > > > > > > >
> > > > > > > > > > Since this device does not have a queue etc? Shouldn't make it a
> > > > > > > > > > library like mempool with vendor-defined ops?
> > > > > > > > > > Any specific reason for making it a device? The reason why I am asking
> > > > > > > > > > this is, as other DPDK devices as symmetry in queue(s), configure,
> > > > > > > > > > start, stop operation etc.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > +
> > > > > > > > > > > +struct rte_gpu_dev {
> > > > > > > > > > > +       /* Backing device. */
> > > > > > > > > > > +       struct rte_device *device;
> > > > > > > > > >
> > > > > > > > > > See above?
> > > > > > > > >
> > > > > > > > > There is a PCI device probed.
> > > > > > > > > I don't understand why it would not be represented as a device.
> > > > > > > >
> > > > > > > > All other DPDK device has symmetry in structures like queue and
> > > > > > > > symmetry in operation like it has configure, start, stop etc.
> > > > > > > > This one seems more like mempool to me all we want set of
> > > > > > > > vendor-defined ops. So any justification on
> > > > > > > > make it a device ? why not like mempool library?
> > > > > > > > (driver/mempool/octeontx2 Mempool HW is also PCI device, but
> > > > > > > > we don't take device path for mempool. So I would like to understand
> > > > > > > > any technical reason for making it a device).
> > > > > > >
> > > > > > > I don't understand what you mean by "symmetry".
> > > > > >
> > > > > > The common attributes. or similarity
> > > > >
> > > > > The common attributes of a device are:
> > > > >         - driver
> > > > >         - bus
> > > > >         - devargs
> > > > > We have these attributes for a GPU.
> > > >
> > > > Yes. Those are attributes of rte_device. That does not mean and
> > > > library can not use rte_device.(mempool library driver is using
> > > > rte_device which is backed by PCI)
> > > > In terms of similarity, all other device libraries(not devices) have
> > > > queue, enqueue() and dequeue() kind of scheme
> > > > in ethdev, cryptodev, compressdev, eventdev, bbdev, rawdev. regexdev.
> > > > i.e existing DPDK device libraries,
> > > > This one des not have have that, So question why to call it libgpudev vs libgpu.
> >
> > See below[1]
> >
> > > >
> > > > The functions you have are memory allocation etc. That's more of a
> > > > library candidate.
> > > >
> > > > > About configure/start/stop usual functions,
> > > > > I think we'll have something similar in the second step
> > > >
> > > > Do you think or it will be there?. I think, it is import decision.
> > >
> > > That's an important discussion we need to have.
> > > We are preparing a proposal.
> >
> > Ack.
> >
> > >
> > > > The device needs have a queue kind of structure
> > > > and it is mapping to core to have a notion of configure. queue_setup,
> > > > start and stop etc
> > >
> > > Why is it a requirement to call it a device API?
> >
> > Then we need to define what needs to call as device library vs library and how?
> > Why mempool is not called a  device library vs library?
>
> My view is simple:
> if it has drivers, it is a device API, except bus and mempool libs.

rte_secuity has drivers but it is not called a device library.

> About mempool, it started as a standard lib and got extended for HW support.

Yes. We did not change to device library as it was fundamentally
different than another DPDK deices
when we added the device support.

>
> > and why all
> > other device library has a common structure like queues and
> > it binding core etc. I tried to explain above the similar attributes
> > for dpdk device libraries[1] which I think, it a requirement so
> > that the end user will have familiarity with device libraries rather
> > than each one has separate General guidelines and principles.
> >
> > I think, it is more TB discussion topic and decides on this because I
> > don't see in technical issue in calling it a library.
>
> The naming is just a choice.

Not sure.

> Yesterday morning it was called lib/gpu/
> and in the evening it was renamed lib/gpudev/
> so no technical issue :)
>
> But the design of the API with queues or other paradigm
> is something I would like to discuss here.

Yeah, That is important. IMO, That defines what needs to be a device library.

> Note: there was no intent to publish GPU processing control
> in DPDK 21.08. We want to focus on GPU memory in 21.08,
> but I understand it is a key decision in the big picture.

if the scope is only memory allocation, IMO, it is better to make a library.

> What would be your need and would you design such API?

For me, there is no need for gpu library(as of now). May GPU consumers
can define what
they need to control using the library.


>
> > > > Something similar to
> > > > http://code.dpdk.org/dpdk/v21.05/source/lib/regexdev/rte_regexdev.h#L27
> > > > Could you share how "running tasks" translates to the above scheme
> > > > like other her dpdk device libraries?
> > >
> > > We will share our view soon but what to control in GPU execution
> > > must be a community discussed requirement.
> >
> > Makes sense.
>
>
>