Re: [dpdk-dev] [PATCH v3 2/8] dmadev: add burst capacity API

[Jerin Jacob <jerinjacobk@gmail.com>]

On Tue, Sep 21, 2021 at 9:05 PM Pai G, Sunil <sunil.pai.g@intel.com> wrote:
>
> Hi Jerin,
>
> <snipped>
>
> > > > > The fact is  that it's very hard for apps to calculate the
> > > > > available space of a
> > > > DMA ring.
> > > >
> > > > Yes, I agree.
> > > >
> > > > My question is more why to calculate the space per burst and
> > > > introduce yet another fastpath API.
> > > > For example, the application needs to copy 8 segments to complete a
> > > > logical copy in the application perspective.
> > > > In case, when 8th copy is completed then only the application marks
> > > > the logical copy completed.
> > > > i.e why to check per burst, 8 segments are available or not? Even it
> > > > is available, there may be multiple reasons why any of the segment
> > > > copies can fail. So the application needs to track all the jobs completed or
> > not anyway.
> > > > Am I missing something in terms of vhost or OVS usage?
> > > >
> > >
> > > For the packets that do not entirely fit in the DMA ring , we have a SW copy
> > fallback in place.
> > > So, we would like to avoid scenario caused because of DMA ring full where
> > few parts of the packet are copied through DMA and other parts by CPU.
> > > Besides, this API would also help improve debuggability/device
> > introspection to check the occupancy rather than the app having to manually
> > track the state of every DMA device in use.
> >
> > To understand it better, Could you share more details on feedback
> > mechanism on your application enqueue
> >
> > app_enqueue_v1(.., nb_seg)
> > {
> >              /* Not enough space, Let application handle it by dropping or
> > resubmitting */
> >              if (rte_dmadev_burst_capacity() < nb_seg)
> >                         return -ENOSPC;
> >
> >             do rte_dma_op() in loop without checking error;
> >             return 0; // Success
> > }
> >
> > vs
> > app_enqueue_v2(.., nb_seg)
> > {
> >            int rc;
> >
> >             rc |= rte_dma_op() in loop without checking error;
> >             return rc; // return the actual status to application  if Not enough space,
> > Let application handle it by dropping or resubmitting */ }
> >
> > Is app_enqueue_v1() and app_enqueue_v2() logically the same from
> > application PoV. Right?
> >
> > If not, could you explain, the version you are planning to do for
> > app_enqueue()
>
> The exact version can be found here : http://patchwork.ozlabs.org/project/openvswitch/patch/20210907120021.4093604-2-sunil.pai.g@intel.com/
> Unfortunately, both versions are not same in our case because of the SW fallback we have for ring full scenario's.
> For a packet with 8 nb_segs, if the ring has only space for 4 , we would avoid this packet with app_enqueue_v1
> while going ahead with an enqueue with app_enqueue_v2, resulting in a mix of DMA and CPU copies for a packet which we would want to avoid.

Thanks for RFC link. Usage is clear now, Since you are checking the
space in the completion handler, I am not sure, what is hard to get
the remaining space,
Will following logic work to find the empty space. If not, please
discuss the issue with this approach. I am trying to avoid yet another
fastpath API
and complication in driver as there is element checking space in the
submit queue and completion queue at least in our hardware.

     max_count = nb_desc; (power of 2)
     mask = max_count - 1;

     for (i = 0; I < n; i++) {
          submit_idx = rte_dma_copy();
     }
     rc = rte_dma_completed(…, &completed_idx..);
     space_in_queue =  mask - ((submit_idx – completed_idx) & mask);

>
> <snipped>
>
> Thanks and regards,
> Sunil
>