Re: [dpdk-dev] [PATCH] gpudev: introduce memory API

[Thomas Monjalon <thomas@monjalon.net>]

03/06/2021 12:04, Jerin Jacob:
> On Thu, Jun 3, 2021 at 3:06 PM Thomas Monjalon <thomas@monjalon.net> wrote:
> >
> > 03/06/2021 11:20, Jerin Jacob:
> > > On Thu, Jun 3, 2021 at 2:23 PM Thomas Monjalon <thomas@monjalon.net> wrote:
> > > >
> > > > 03/06/2021 10:47, Jerin Jacob:
> > > > > On Thu, Jun 3, 2021 at 2:13 PM Thomas Monjalon <thomas@monjalon.net> wrote:
> > > > > >
> > > > > > 03/06/2021 10:41, Jerin Jacob:
> > > > > > > On Thu, Jun 3, 2021 at 1:58 PM Thomas Monjalon <thomas@monjalon.net> wrote:
> > > > > > > >
> > > > > > > > 03/06/2021 09:47, Jerin Jacob:
> > > > > > > > > On Thu, Jun 3, 2021 at 2:05 AM Thomas Monjalon <thomas@monjalon.net> wrote:
> > > > > > > > > > --- a/doc/api/doxy-api-index.md
> > > > > > > > > > +++ b/doc/api/doxy-api-index.md
> > > > > > > > > > @@ -21,6 +21,7 @@ The public API headers are grouped by topics:
> > > > > > > > > >    [compressdev]        (@ref rte_compressdev.h),
> > > > > > > > > >    [compress]           (@ref rte_comp.h),
> > > > > > > > > >    [regexdev]           (@ref rte_regexdev.h),
> > > > > > > > > > +  [gpudev]             (@ref rte_gpudev.h),
> > > > > > > > >
> > > > > > > > > Since this device does not have a queue etc? Shouldn't make it a
> > > > > > > > > library like mempool with vendor-defined ops?
> > > > > > > > > Any specific reason for making it a device? The reason why I am asking
> > > > > > > > > this is, as other DPDK devices as symmetry in queue(s), configure,
> > > > > > > > > start, stop operation etc.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > > +
> > > > > > > > > > +struct rte_gpu_dev {
> > > > > > > > > > +       /* Backing device. */
> > > > > > > > > > +       struct rte_device *device;
> > > > > > > > >
> > > > > > > > > See above?
> > > > > > > >
> > > > > > > > There is a PCI device probed.
> > > > > > > > I don't understand why it would not be represented as a device.
> > > > > > >
> > > > > > > All other DPDK device has symmetry in structures like queue and
> > > > > > > symmetry in operation like it has configure, start, stop etc.
> > > > > > > This one seems more like mempool to me all we want set of
> > > > > > > vendor-defined ops. So any justification on
> > > > > > > make it a device ? why not like mempool library?
> > > > > > > (driver/mempool/octeontx2 Mempool HW is also PCI device, but
> > > > > > > we don't take device path for mempool. So I would like to understand
> > > > > > > any technical reason for making it a device).
> > > > > >
> > > > > > I don't understand what you mean by "symmetry".
> > > > >
> > > > > The common attributes. or similarity
> > > >
> > > > The common attributes of a device are:
> > > >         - driver
> > > >         - bus
> > > >         - devargs
> > > > We have these attributes for a GPU.
> > >
> > > Yes. Those are attributes of rte_device. That does not mean and
> > > library can not use rte_device.(mempool library driver is using
> > > rte_device which is backed by PCI)
> > > In terms of similarity, all other device libraries(not devices) have
> > > queue, enqueue() and dequeue() kind of scheme
> > > in ethdev, cryptodev, compressdev, eventdev, bbdev, rawdev. regexdev.
> > > i.e existing DPDK device libraries,
> > > This one des not have have that, So question why to call it libgpudev vs libgpu.
> 
> See below[1]
> 
> > >
> > > The functions you have are memory allocation etc. That's more of a
> > > library candidate.
> > >
> > > > About configure/start/stop usual functions,
> > > > I think we'll have something similar in the second step
> > >
> > > Do you think or it will be there?. I think, it is import decision.
> >
> > That's an important discussion we need to have.
> > We are preparing a proposal.
> 
> Ack.
> 
> >
> > > The device needs have a queue kind of structure
> > > and it is mapping to core to have a notion of configure. queue_setup,
> > > start and stop etc
> >
> > Why is it a requirement to call it a device API?
> 
> Then we need to define what needs to call as device library vs library and how?
> Why mempool is not called a  device library vs library?

My view is simple:
if it has drivers, it is a device API, except bus and mempool libs.
About mempool, it started as a standard lib and got extended for HW support.

> and why all
> other device library has a common structure like queues and
> it binding core etc. I tried to explain above the similar attributes
> for dpdk device libraries[1] which I think, it a requirement so
> that the end user will have familiarity with device libraries rather
> than each one has separate General guidelines and principles.
> 
> I think, it is more TB discussion topic and decides on this because I
> don't see in technical issue in calling it a library.

The naming is just a choice.
Yesterday morning it was called lib/gpu/
and in the evening it was renamed lib/gpudev/
so no technical issue :)

But the design of the API with queues or other paradigm
is something I would like to discuss here.
Note: there was no intent to publish GPU processing control
in DPDK 21.08. We want to focus on GPU memory in 21.08,
but I understand it is a key decision in the big picture.
What would be your need and would you design such API?

> > > Something similar to
> > > http://code.dpdk.org/dpdk/v21.05/source/lib/regexdev/rte_regexdev.h#L27
> > > Could you share how "running tasks" translates to the above scheme
> > > like other her dpdk device libraries?
> >
> > We will share our view soon but what to control in GPU execution
> > must be a community discussed requirement.
> 
> Makes sense.