From: Anoob Joseph <anoobj@marvell.com>
To: Anatoly Burakov <anatoly.burakov@intel.com>
Cc: Vladimir Medvedkin <vladimir.medvedkin@intel.com>,
"dev@dpdk.org" <dev@dpdk.org>,
Chengwen Feng <fengchengwen@huawei.com>,
Kevin Laatz <kevin.laatz@intel.com>,
Bruce Richardson <bruce.richardson@intel.com>,
Jerin Jacob Kollanukkaran <jerinj@marvell.com>,
Vamsi Krishna Attunuru <vattunuru@marvell.com>,
Amit Prakash Shukla <amitprakashs@marvell.com>,
Vidya Sagar Velumuri <vvelumuri@marvell.com>
Subject: RE: [EXT] [PATCH v1 1/3] dmadev: add inter-domain operations
Date: Fri, 18 Aug 2023 08:08:10 +0000 [thread overview]
Message-ID: <PH0PR18MB467288F97D27C30221854EADDF1BA@PH0PR18MB4672.namprd18.prod.outlook.com> (raw)
In-Reply-To: <8866a5c7ea36e476b2a92e3e4cea6c2c127ab82f.1691768110.git.anatoly.burakov@intel.com>
Hi Anatoly,
Marvell CNXK DMA hardware also supports this feature, and it would be a good feature to add. Thanks for introducing the feature. Please see inline.
Thanks,
Anoob
> -----Original Message-----
> From: Anatoly Burakov <anatoly.burakov@intel.com>
> Sent: Friday, August 11, 2023 9:45 PM
> To: dev@dpdk.org; Chengwen Feng <fengchengwen@huawei.com>; Kevin
> Laatz <kevin.laatz@intel.com>; Bruce Richardson
> <bruce.richardson@intel.com>
> Cc: Vladimir Medvedkin <vladimir.medvedkin@intel.com>
> Subject: [EXT] [PATCH v1 1/3] dmadev: add inter-domain operations
>
> External Email
>
> ----------------------------------------------------------------------
> Add a flag to indicate that a specific device supports inter-domain operations,
> and add an API for inter-domain copy and fill.
>
> Inter-domain operation is an operation that is very similar to regular DMA
> operation, except either source or destination addresses can be in a
> different process's address space, indicated by source and destination
> handle values. These values are currently meant to be provided by private
> drivers' API's.
>
> This commit also adds a controller ID field into the DMA device API.
> This is an arbitrary value that may not be implemented by hardware, but it is
> meant to represent some kind of device hierarchy.
>
> Signed-off-by: Vladimir Medvedkin <vladimir.medvedkin@intel.com>
> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
> ---
> doc/guides/prog_guide/dmadev.rst | 18 +++++
> lib/dmadev/rte_dmadev.c | 2 +
> lib/dmadev/rte_dmadev.h | 133
> +++++++++++++++++++++++++++++++
> lib/dmadev/rte_dmadev_core.h | 12 +++
> 4 files changed, 165 insertions(+)
>
<snip>
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Enqueue an inter-domain copy operation.
> + *
> + * This queues up an inter-domain copy operation to be performed by
> +hardware, if
> + * the 'flags' parameter contains RTE_DMA_OP_FLAG_SUBMIT then trigger
> +doorbell
> + * to begin this operation, otherwise do not trigger doorbell.
> + *
> + * The source and destination handle parameters are arbitrary opaque
> +values,
> + * currently meant to be provided by private device driver API's. If
> +the source
> + * handle value is meaningful, RTE_DMA_OP_FLAG_SRC_HANDLE flag must
> be set.
> + * Similarly, if the destination handle value is meaningful,
> + * RTE_DMA_OP_FLAG_DST_HANDLE flag must be set. Source and
> destination
> +handle
> + * values are meant to provide information to the hardware about source
> +and/or
> + * destination PASID for the inter-domain copy operation.
> + *
> + * @param dev_id
> + * The identifier of the device.
> + * @param vchan
> + * The identifier of virtual DMA channel.
> + * @param src
> + * The address of the source buffer (if `src_handle` is set, source address
> + * will be in address space of process referred to by source handle).
> + * @param dst
> + * The address of the destination buffer (if `dst_handle` is set, destination
> + * address will be in address space of process referred to by destination
> + * handle).
> + * @param length
> + * The length of the data to be copied.
> + * @param src_handle
> + * Source handle value (if used, RTE_DMA_OP_FLAG_SRC_HANDLE flag
> must be set).
> + * @param dst_handle
> + * Destination handle value (if used, RTE_DMA_OP_FLAG_DST_HANDLE
> flag must be
> + * set).
> + * @param flags
> + * Flags for this operation.
> + * @return
> + * - 0..UINT16_MAX: index of enqueued job.
> + * - -ENOSPC: if no space left to enqueue.
> + * - other values < 0 on failure.
> + */
> +__rte_experimental
> +static inline int
> +rte_dma_copy_inter_dom(int16_t dev_id, uint16_t vchan, rte_iova_t src,
> + rte_iova_t dst, uint32_t length, uint16_t src_handle,
> + uint16_t dst_handle, uint64_t flags)
> +{
[Anoob] Won't this lead to duplication of all datapath APIs? Also, this approach assumes that 'inter-domain' operations always support run-time setting of 'src_handle' and 'dst_handle' within one DMA channel, which need not be supported by all platforms.
Can we move this 'src_handle' and 'dst_handle' registration to rte_dma_vchan_setup so that the 'src_handle' and 'dst_handle' can be configured in control path and the existing datapath APIs can work as is. The op flags (that is proposed) can be used to determine whether 'inter-domain' operation is requested. Having a fixed 'src_handle' & 'dst_handle' per vchan would be better for performance as well.
<snip>
next prev parent reply other threads:[~2023-08-18 8:08 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-08-11 16:14 [PATCH v1 0/3] Add support for inter-domain DMA operations Anatoly Burakov
2023-08-11 16:14 ` [PATCH v1 1/3] dmadev: add inter-domain operations Anatoly Burakov
2023-08-18 8:08 ` Anoob Joseph [this message]
2023-10-08 2:33 ` fengchengwen
2023-10-09 5:05 ` Jerin Jacob
2023-10-27 13:46 ` Medvedkin, Vladimir
2023-11-23 5:24 ` Jerin Jacob
2023-08-11 16:14 ` [PATCH v1 2/3] dma/idxd: implement " Anatoly Burakov
2023-08-11 16:14 ` [PATCH v1 3/3] dma/idxd: add API to create and attach to window Anatoly Burakov
2023-08-14 4:39 ` Jerin Jacob
2023-08-14 9:55 ` Burakov, Anatoly
2023-08-15 19:20 ` [EXT] [PATCH v1 0/3] Add support for inter-domain DMA operations Satananda Burla
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=PH0PR18MB467288F97D27C30221854EADDF1BA@PH0PR18MB4672.namprd18.prod.outlook.com \
--to=anoobj@marvell.com \
--cc=amitprakashs@marvell.com \
--cc=anatoly.burakov@intel.com \
--cc=bruce.richardson@intel.com \
--cc=dev@dpdk.org \
--cc=fengchengwen@huawei.com \
--cc=jerinj@marvell.com \
--cc=kevin.laatz@intel.com \
--cc=vattunuru@marvell.com \
--cc=vladimir.medvedkin@intel.com \
--cc=vvelumuri@marvell.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).