From: fengchengwen <fengchengwen@huawei.com>
To: Jerin Jacob <jerinjacobk@gmail.com>
Cc: "Bruce Richardson" <bruce.richardson@intel.com>,
"Jerin Jacob" <jerinj@marvell.com>,
"Morten Brørup" <mb@smartsharesystems.com>,
"Nipun Gupta" <nipun.gupta@nxp.com>,
"Thomas Monjalon" <thomas@monjalon.net>,
"Ferruh Yigit" <ferruh.yigit@intel.com>, dpdk-dev <dev@dpdk.org>,
"Hemant Agrawal" <hemant.agrawal@nxp.com>,
"Maxime Coquelin" <maxime.coquelin@redhat.com>,
"Honnappa Nagarahalli" <honnappa.nagarahalli@arm.com>,
"David Marchand" <david.marchand@redhat.com>,
"Satananda Burla" <sburla@marvell.com>,
"Prasun Kapoor" <pkapoor@marvell.com>
Subject: Re: [dpdk-dev] dmadev discussion summary
Date: Fri, 2 Jul 2021 21:45:15 +0800 [thread overview]
Message-ID: <e19b23b0-294f-5430-eaa3-067a3aadd13d@huawei.com> (raw)
In-Reply-To: <CALBAE1MJeoM8TC_b1LagS_qwPz9-qPr1Tw1Ay6AEjMvjbgOdZg@mail.gmail.com>
On 2021/7/1 23:01, Jerin Jacob wrote:
>> [key point]:
>> ----------- -----------
>> | channel | | channel |
>> ----------- -----------
>> \ /
>> \ /
>> \ /
>> ------------
>> | HW-queue |
>> ------------
>> |
>> --------
>> |rawdev|
>> --------
>> 1) User could create one channel by init context(dpi_dma_queue_ctx_s),
>> this interface is not standardized and needs to be implemented by
>> users.
>> 2) Different channels can support different transmissions, e.g. one for
>> inner m2m, and other for inbound copy.
>>
>> Overall, I think the 'channel' is similar the 'virt-queue' of dpaa2_qdma.
>> The difference is that dpaa2_qdma supports multiple hardware queues. The
>> 'channel' has following
>
> If dpaa2_qdma supports more than one HW queue, I think, it is good to
> have the queue notion
> in DPDK just like other DPDK device classes. It will be good to have
> confirmation from dpaa2 folks, @Hemant Agrawal,
> if there are really more than 1 HW queue in dppa device.
>
>
> IMO, Channel is a better name than a virtual queue. The reason is,
> virtual queue is more
> implementation-specific notation. No need to have this in API specification.
>
In the DPDK framework, many data-plane API names contain queues. e.g. eventdev/crypto..
The concept of virt queues has continuity.
>> [dma_copy/memset/sg]: all has vq_id input parameter.
>> Note: I notice dpaa can't support single and sg in one virt-queue, and
>> I think it's maybe software implement policy other than HW
>> restriction because virt-queue could share the same HW-queue.
>> Here we use vq_id to tackle different scenario, like local-to-local/
>> local-to-host and etc.
>
> IMO, The index representation has an additional overhead as one needs
> to translate it
> to memory pointer. I prefer to avoid by having object handle and use
> _lookup() API get to make it work
> in multi-process cases to avoid the additional indirection. Like mempool object.
>
This solution was first considered, similar to rte_hash returning a handle.
It is not intuitive and has no obvious performance advantage. The number of
jump times of the data-plane API index-driven callback function is not optimized.
>> 5) And the dmadev public data-plane API (just prototype):
>> dma_cookie_t rte_dmadev_memset(dev, vq_id, pattern, dst, len, flags)
>> -- flags: used as an extended parameter, it could be uint32_t
>> dma_cookie_t rte_dmadev_memcpy(dev, vq_id, src, dst, len, flags)
>> dma_cookie_t rte_dmadev_memcpy_sg(dev, vq_id, sg, sg_len, flags)
>> -- sg: struct dma_scatterlist array
>> uint16_t rte_dmadev_completed(dev, vq_id, dma_cookie_t *cookie,
>> uint16_t nb_cpls, bool *has_error)
>> -- nb_cpls: indicate max process operations number
>> -- has_error: indicate if there is an error
>> -- return value: the number of successful completed operations.
>> -- example:
>> 1) If there are already 32 completed ops, and 4th is error, and
>> nb_cpls is 32, then the ret will be 3(because 1/2/3th is OK), and
>> has_error will be true.
>> 2) If there are already 32 completed ops, and all successful
>> completed, then the ret will be min(32, nb_cpls), and has_error
>> will be false.
>> 3) If there are already 32 completed ops, and all failed completed,
>> then the ret will be 0, and has_error will be true.
>
> +1. IMO, it is better to call ring_idx instead of a cookie. To enforce
> that it the ring index.
>
>> uint16_t rte_dmadev_completed_status(dev_id, vq_id, dma_cookie_t *cookie,
>> uint16_t nb_status, uint32_t *status)
>> -- return value: the number of failed completed operations.
>
> See above. Here we are assuming it is an index otherwise we need to
> pass an array
> cookies.
>
>> And here I agree with Morten: we should design API which adapts to DPDK
>> service scenarios. So we don't support some sound-cards DMA, and 2D memory
>> copy which mainly used in video scenarios.
>> 6) The dma_cookie_t is signed int type, when <0 it mean error, it's
>> monotonically increasing base on HW-queue (other than virt-queue). The
>> driver needs to make sure this because the damdev framework don't manage
>> the dma_cookie's creation.
>
> +1 and see above.
>
>> 7) Because data-plane APIs are not thread-safe, and user could determine
>> virt-queue to HW-queue's map (at the queue-setup stage), so it is user's
>> duty to ensure thread-safe.
>
> +1. But I am not sure how easy for the fast-path application to have this logic,
> Instead, I think, it is better to tell the capa for queue by driver
> and in channel configuration,
> the application can request for requirement (Is multiple producers enq
> to the same HW queue or not).
> Based on the request, the implementation can pick the correct function
> pointer for enq.(lock vs lockless version if HW does not support
> lockless)
>
already redesigned. Please check the latest patch.
>
>> 8) One example:
>> vq_id = rte_dmadev_queue_setup(dev, config.{HW-queue-index=x, opaque});
>> if (vq_id < 0) {
>> // create virt-queue failed
>> return;
>> }
>> // submit memcpy task
>> cookit = rte_dmadev_memcpy(dev, vq_id, src, dst, len, flags);
>> if (cookie < 0) {
>> // submit failed
>> return;
>> }
>> // get complete task
>> ret = rte_dmadev_completed(dev, vq_id, &cookie, 1, has_error);
>> if (!has_error && ret == 1) {
>> // the memcpy successful complete
>> }
>
> +1
>
>> 9) As octeontx2_dma support sg-list which has many valid buffers in
>> dpi_dma_buf_ptr_s, it could call the rte_dmadev_memcpy_sg API.
>
> +1
>
>> 10) As ioat, it could delcare support one HW-queue at dev_configure stage, and
>> only support create one virt-queue.
>> 11) As dpaa2_qdma, I think it could migrate to new framework, but still wait
>> for dpaa2_qdma guys feedback.
>> 12) About the prototype src/dst parameters of rte_dmadev_memcpy API, we have
>> two candidates which are iova and void *, how about introduce dma_addr_t
>> type which could be va or iova ?
>
> I think, conversion looks ugly, better to have void * and share the
> constraints of void *
> as limitation/capability using flag. So that driver can update it.
>
already change to void *
>
>>
>
> .
>
next prev parent reply other threads:[~2021-07-02 13:45 UTC|newest]
Thread overview: 79+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-06-15 13:22 [dpdk-dev] [RFC PATCH] dmadev: introduce DMA device library Chengwen Feng
2021-06-15 16:38 ` Bruce Richardson
2021-06-16 7:09 ` Morten Brørup
2021-06-16 10:17 ` fengchengwen
2021-06-16 12:09 ` Morten Brørup
2021-06-16 13:06 ` Bruce Richardson
2021-06-16 14:37 ` Jerin Jacob
2021-06-17 9:15 ` Bruce Richardson
2021-06-18 5:52 ` Jerin Jacob
2021-06-18 9:41 ` fengchengwen
2021-06-22 17:25 ` Jerin Jacob
2021-06-23 3:30 ` fengchengwen
2021-06-23 7:21 ` Jerin Jacob
2021-06-23 9:37 ` Bruce Richardson
2021-06-23 11:40 ` Jerin Jacob
2021-06-23 14:19 ` Bruce Richardson
2021-06-24 6:49 ` Jerin Jacob
2021-06-23 9:41 ` Bruce Richardson
2021-06-23 10:10 ` Morten Brørup
2021-06-23 11:46 ` Jerin Jacob
2021-06-23 14:22 ` Bruce Richardson
2021-06-18 9:55 ` Bruce Richardson
2021-06-22 17:31 ` Jerin Jacob
2021-06-22 19:17 ` Bruce Richardson
2021-06-23 7:00 ` Jerin Jacob
2021-06-16 9:41 ` fengchengwen
2021-06-16 17:31 ` Bruce Richardson
2021-06-16 18:08 ` Jerin Jacob
2021-06-16 19:13 ` Bruce Richardson
2021-06-17 7:42 ` Jerin Jacob
2021-06-17 8:00 ` Bruce Richardson
2021-06-18 5:16 ` Jerin Jacob
2021-06-18 10:03 ` Bruce Richardson
2021-06-22 17:36 ` Jerin Jacob
2021-06-17 9:48 ` fengchengwen
2021-06-17 11:02 ` Bruce Richardson
2021-06-17 14:18 ` Bruce Richardson
2021-06-18 8:52 ` fengchengwen
2021-06-18 9:30 ` Bruce Richardson
2021-06-22 17:51 ` Jerin Jacob
2021-06-23 3:50 ` fengchengwen
2021-06-23 11:00 ` Jerin Jacob
2021-06-23 14:56 ` Bruce Richardson
2021-06-24 12:19 ` fengchengwen
2021-06-26 3:59 ` [dpdk-dev] dmadev discussion summary fengchengwen
2021-06-28 10:00 ` Bruce Richardson
2021-06-28 11:14 ` Ananyev, Konstantin
2021-06-28 12:53 ` Bruce Richardson
2021-07-02 13:31 ` fengchengwen
2021-07-01 15:01 ` Jerin Jacob
2021-07-01 16:33 ` Bruce Richardson
2021-07-02 7:39 ` Morten Brørup
2021-07-02 10:05 ` Bruce Richardson
2021-07-02 13:45 ` fengchengwen [this message]
2021-07-02 14:57 ` Morten Brørup
2021-07-03 0:32 ` fengchengwen
2021-07-03 8:53 ` Morten Brørup
2021-07-03 9:08 ` Jerin Jacob
2021-07-03 12:24 ` Morten Brørup
2021-07-04 7:43 ` Jerin Jacob
2021-07-05 10:28 ` Morten Brørup
2021-07-06 7:11 ` fengchengwen
2021-07-03 9:45 ` fengchengwen
2021-07-03 12:00 ` Morten Brørup
2021-07-04 7:34 ` Jerin Jacob
2021-07-02 7:07 ` Liang Ma
2021-07-02 13:59 ` fengchengwen
2021-06-24 7:03 ` [dpdk-dev] [RFC PATCH] dmadev: introduce DMA device library Jerin Jacob
2021-06-24 7:59 ` Morten Brørup
2021-06-24 8:05 ` Jerin Jacob
2021-06-23 5:34 ` Hu, Jiayu
2021-06-23 11:07 ` Jerin Jacob
2021-06-16 2:17 ` Wang, Haiyue
2021-06-16 8:04 ` Bruce Richardson
2021-06-16 8:16 ` Wang, Haiyue
2021-06-16 12:14 ` David Marchand
2021-06-16 13:11 ` Bruce Richardson
2021-06-16 16:48 ` Honnappa Nagarahalli
2021-06-16 19:10 ` Bruce Richardson
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=e19b23b0-294f-5430-eaa3-067a3aadd13d@huawei.com \
--to=fengchengwen@huawei.com \
--cc=bruce.richardson@intel.com \
--cc=david.marchand@redhat.com \
--cc=dev@dpdk.org \
--cc=ferruh.yigit@intel.com \
--cc=hemant.agrawal@nxp.com \
--cc=honnappa.nagarahalli@arm.com \
--cc=jerinj@marvell.com \
--cc=jerinjacobk@gmail.com \
--cc=maxime.coquelin@redhat.com \
--cc=mb@smartsharesystems.com \
--cc=nipun.gupta@nxp.com \
--cc=pkapoor@marvell.com \
--cc=sburla@marvell.com \
--cc=thomas@monjalon.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).