Re: [dpdk-dev] [RFC PATCH] dmadev: introduce DMA device library

[Jerin Jacob <jerinjacobk@gmail.com>]

On Wed, Jun 23, 2021 at 9:20 AM fengchengwen <fengchengwen@huawei.com> wrote:
>

> >>
> >> So I prefer following prototype:
> >>   uint16_t rte_dmadev_completed(uint16_t dev_id, dma_cookie_t *cookie, uint16_t nb_cpls, bool *has_error)
> >>     -- nb_cpls: indicate max process operations number
> >>     -- has_error: indicate if there is an error
> >>     -- return value: the number of successful completed operations.
> >>     -- example:
> >>        1) If there are already 32 completed ops, and 4th is error, and nb_cpls is 32, then
> >>           the ret will be 3(because 1/2/3th is OK), and has_error will be true.
> >>        2) If there are already 32 completed ops, and all successful completed, then the ret
> >>           will be min(32, nb_cpls), and has_error will be false.
> >>        3) If there are already 32 completed ops, and all failed completed, then the ret will
> >>           be 0, and has_error will be true.
> >>   uint16_t rte_dmadev_completed_status(uint16_t dev_id, dma_cookie_t *cookie, uint16_t nb_status, uint32_t *status)
> >>     -- return value: the number of failed completed operations.
> >
> >
> >
> > In typical storage use cases etc, Sometimes application need to
> > provide scatter-gather list,
> > At least in our hardware sg list gives a "single completion result"
> > and it stops on the first failure to restart
> > the transfer by application. Have you thought of scatter-gather use
> > case and how it is in other  HW?
>
> cookie and request are in a one-to-one correspondence, whether the request is a single or sg-list.
> Kunpeng9x0 don't support sg-list, I'm still investigating other hardware.
>
> The above 'restart the transfer by application' mean re-schedule request (and have one new cookie) or
> just re-enable current failed request (this may introduce new API) ?
>
> >
> > prototype like the following works for us:
> > rte_dmadev_enq_sg(void **src, void **dest, unsigned int **length, int
> > nb_segments, cookie, ,,,)
>
> OK, we could define one scatter-list struct to wrap src/dest/length.


Inspired from following system call [1]

[1]
https://man7.org/linux/man-pages/man2/process_vm_readv.2.html

I propose the following style syntax for the sg list

struct rte_dma_iovec {
    void  *iov_base;    /* Starting address */
    size_t iov_len;     /* Number of bytes to transfer */
};

rte_dmadev_enq_sg(const struct rte_dma_iovec  *src_iov, unsigned int srcvcnt,
const struct rte_dma_iovec  *dst_iov, unsigned int dstvcnt, ....)

The reason for separating iov_len for src and dest is the many to one case and
one to many cases.  Example:
Copy of Multiple 2MB of 15 source segments to one 30MB dest. Quite use
full in storage use cases.


>
> >
> >
> >>
> >> The application use the following invocation order when polling:
> >>   has_error = false; // could be init to false by dmadev API, we need discuss
> >>   ret = rte_dmadev_completed(dev_id, &cookie, bust_size, &has_error);
> >>   // process successful completed case:
> >>   for (int i = 0; i < ret; i++) {
> >>   }
> >>   if (unlikely(has_error)) {
> >>     // process failed completed case
> >>     ret = rte_dmadev_completed_status(dev_id, &cookie, burst_size - ret, status_array);
> >>     for (int i = 0; i < ret; i++) {
> >>       // ...
> >>     }
> >>   }
> >>
> >>
> >>>
> >>> This two-function approach also allows future support for other DMA
> >>> functions such as comparison, where a status value is always required. Any
> >>> apps using that functionality would just always use the "_status" function
> >>> for completions.
> >>>
> >>> /Bruce
> >>>
> >>> .
> >>>
> >>
> >
> > .
> >
>