[PATCH v0 1/1] doc: announce inter-device DMA capability support in dmadev

[PATCH v0 1/1] doc: announce inter-device DMA capability support in dmadev

[Vamsi Krishna]

From: Vamsi Attunuru <vattunuru@marvell.com>

Modern DMA hardware supports data transfer between multiple
DMA devices, enabling data communication across isolated domains or
containers. To facilitate this, the ``dmadev`` library requires changes
to allow devices to register with or unregisters from DMA groups for
inter-device communication. This feature is planned for inclusion
in DPDK 25.11.

Signed-off-by: Vamsi Attunuru <vattunuru@marvell.com>
---
 doc/guides/rel_notes/deprecation.rst | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index e2d4125308..46836244dd 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -152,3 +152,10 @@ Deprecation Notices
 * bus/vmbus: Starting DPDK 25.11, all the vmbus API defined in
   ``drivers/bus/vmbus/rte_bus_vmbus.h`` will become internal to DPDK.
   Those API functions are used internally by DPDK core and netvsc PMD.
+
+* dmadev: a new capability flag ``RTE_DMA_CAPA_INTER_DEV`` will be added
+  to advertise DMA device's inter-device DMA copy capability. To enable
+  this functionality, a few dmadev APIs will be added to configure the DMA
+  access groups, facilitating coordinated data communication between devices.
+  A new ``dev_idx`` field will be added to the ``struct rte_dma_vchan_conf``
+  structure to configure a vchan for data transfers between any two DMA devices.
-- 
2.34.1

RE: [PATCH v0 1/1] doc: announce inter-device DMA capability support in dmadev

[Vamsi Krishna Attunuru]

Hi,

Gentle ping for reviews.

>-----Original Message-----
>From: Vamsi Krishna <vattunuru@marvell.com>
>Sent: Thursday, July 10, 2025 2:21 PM
>To: dev@dpdk.org; fengchengwen@huawei.com; Pavan Nikhilesh
>Bhagavatula <pbhagavatula@marvell.com>; kevin.laatz@intel.com;
>bruce.richardson@intel.com; mb@smartsharesystems.com
>Cc: Jerin Jacob <jerinj@marvell.com>; thomas@monjalon.net; Vamsi Krishna
>Attunuru <vattunuru@marvell.com>
>Subject: [PATCH v0 1/1] doc: announce inter-device DMA capability support in
>dmadev
>
>From: Vamsi Attunuru <vattunuru@marvell.com>
>
>Modern DMA hardware supports data transfer between multiple DMA
>devices, enabling data communication across isolated domains or containers.
>To facilitate this, the ``dmadev`` library requires changes to allow devices to
>register with or unregisters from DMA groups for inter-device communication.
>This feature is planned for inclusion in DPDK 25.11.
>
>Signed-off-by: Vamsi Attunuru <vattunuru@marvell.com>
>---
> doc/guides/rel_notes/deprecation.rst | 7 +++++++
> 1 file changed, 7 insertions(+)
>
>diff --git a/doc/guides/rel_notes/deprecation.rst
>b/doc/guides/rel_notes/deprecation.rst
>index e2d4125308..46836244dd 100644
>--- a/doc/guides/rel_notes/deprecation.rst
>+++ b/doc/guides/rel_notes/deprecation.rst
>@@ -152,3 +152,10 @@ Deprecation Notices
> * bus/vmbus: Starting DPDK 25.11, all the vmbus API defined in
>   ``drivers/bus/vmbus/rte_bus_vmbus.h`` will become internal to DPDK.
>   Those API functions are used internally by DPDK core and netvsc PMD.
>+
>+* dmadev: a new capability flag ``RTE_DMA_CAPA_INTER_DEV`` will be
>+added
>+  to advertise DMA device's inter-device DMA copy capability. To enable
>+  this functionality, a few dmadev APIs will be added to configure the
>+DMA
>+  access groups, facilitating coordinated data communication between
>devices.
>+  A new ``dev_idx`` field will be added to the ``struct
>+rte_dma_vchan_conf``
>+  structure to configure a vchan for data transfers between any two DMA
>devices.
>--
>2.34.1

RE: [EXTERNAL] [PATCH v0 1/1] doc: announce inter-device DMA capability support in dmadev

[Anoob Joseph]

> From: Vamsi Attunuru <mailto:vattunuru@marvell.com>
> 
> Modern DMA hardware supports data transfer between multiple
> DMA devices, enabling data communication across isolated domains or
> containers. To facilitate this, the ``dmadev`` library requires changes
> to allow devices to register with or unregisters from DMA groups for
> inter-device communication. This feature is planned for inclusion
> in DPDK 25.11.
> 
> Signed-off-by: Vamsi Attunuru <vattunuru@marvell.com>

Acked-by: Anoob Joseph <anoobj@marvell.com>

Re: [EXTERNAL] [PATCH v0 1/1] doc: announce inter-device DMA capability support in dmadev

[Pavan Nikhilesh Bhagavatula]

>> From: Vamsi Attunuru <mailto:vattunuru@marvell.com>
>>
>> Modern DMA hardware supports data transfer between multiple
>> DMA devices, enabling data communication across isolated domains or
>> containers. To facilitate this, the ``dmadev`` library requires changes
>> to allow devices to register with or unregisters from DMA groups for
>> inter-device communication. This feature is planned for inclusion
>> in DPDK 25.11.
>>
>> Signed-off-by: Vamsi Attunuru <vattunuru@marvell.com>
>
>Acked-by: Anoob Joseph <anoobj@marvell.com>

Acked-by: Pavan Nikhilesh <pbhagavatula@marvell.com>

>
>
>

Re: [PATCH v0 1/1] doc: announce inter-device DMA capability support in dmadev

[Medvedkin, Vladimir]

Hi Vamsi,

Could you please tell us more about usecases for this feature?

Thanks!

On 7/10/2025 9:51 AM, Vamsi Krishna wrote:
> From: Vamsi Attunuru <vattunuru@marvell.com>
>
> Modern DMA hardware supports data transfer between multiple
> DMA devices, enabling data communication across isolated domains or
> containers. To facilitate this, the ``dmadev`` library requires changes
> to allow devices to register with or unregisters from DMA groups for
> inter-device communication. This feature is planned for inclusion
> in DPDK 25.11.
>
> Signed-off-by: Vamsi Attunuru <vattunuru@marvell.com>
> ---
>   doc/guides/rel_notes/deprecation.rst | 7 +++++++
>   1 file changed, 7 insertions(+)
>
> diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
> index e2d4125308..46836244dd 100644
> --- a/doc/guides/rel_notes/deprecation.rst
> +++ b/doc/guides/rel_notes/deprecation.rst
> @@ -152,3 +152,10 @@ Deprecation Notices
>   * bus/vmbus: Starting DPDK 25.11, all the vmbus API defined in
>     ``drivers/bus/vmbus/rte_bus_vmbus.h`` will become internal to DPDK.
>     Those API functions are used internally by DPDK core and netvsc PMD.
> +
> +* dmadev: a new capability flag ``RTE_DMA_CAPA_INTER_DEV`` will be added
> +  to advertise DMA device's inter-device DMA copy capability. To enable
> +  this functionality, a few dmadev APIs will be added to configure the DMA
> +  access groups, facilitating coordinated data communication between devices.
> +  A new ``dev_idx`` field will be added to the ``struct rte_dma_vchan_conf``
> +  structure to configure a vchan for data transfers between any two DMA devices.

-- 
Regards,
Vladimir

RE: [PATCH v0 1/1] doc: announce inter-device DMA capability support in dmadev

[Vamsi Krishna Attunuru]

Hi Feng,

Kindly review below.

Regards
Vamsi

>-----Original Message-----
>From: Vamsi Krishna Attunuru <vattunuru@marvell.com>
>Sent: Friday, July 11, 2025 8:35 PM
>To: Vamsi Krishna Attunuru <vattunuru@marvell.com>; dev@dpdk.org;
>fengchengwen@huawei.com; Pavan Nikhilesh Bhagavatula
><pbhagavatula@marvell.com>; kevin.laatz@intel.com;
>bruce.richardson@intel.com; mb@smartsharesystems.com
>Cc: Jerin Jacob <jerinj@marvell.com>; thomas@monjalon.net
>Subject: RE: [PATCH v0 1/1] doc: announce inter-device DMA capability
>support in dmadev
>
>Hi,
>
>Gentle ping for reviews.
>
>>-----Original Message-----
>>From: Vamsi Krishna <vattunuru@marvell.com>
>>Sent: Thursday, July 10, 2025 2:21 PM
>>To: dev@dpdk.org; fengchengwen@huawei.com; Pavan Nikhilesh
>Bhagavatula
>><pbhagavatula@marvell.com>; kevin.laatz@intel.com;
>>bruce.richardson@intel.com; mb@smartsharesystems.com
>>Cc: Jerin Jacob <jerinj@marvell.com>; thomas@monjalon.net; Vamsi
>>Krishna Attunuru <vattunuru@marvell.com>
>>Subject: [PATCH v0 1/1] doc: announce inter-device DMA capability
>>support in dmadev
>>
>>From: Vamsi Attunuru <vattunuru@marvell.com>
>>
>>Modern DMA hardware supports data transfer between multiple DMA
>>devices, enabling data communication across isolated domains or containers.
>>To facilitate this, the ``dmadev`` library requires changes to allow
>>devices to register with or unregisters from DMA groups for inter-device
>communication.
>>This feature is planned for inclusion in DPDK 25.11.
>>
>>Signed-off-by: Vamsi Attunuru <vattunuru@marvell.com>
>>---
>> doc/guides/rel_notes/deprecation.rst | 7 +++++++
>> 1 file changed, 7 insertions(+)
>>
>>diff --git a/doc/guides/rel_notes/deprecation.rst
>>b/doc/guides/rel_notes/deprecation.rst
>>index e2d4125308..46836244dd 100644
>>--- a/doc/guides/rel_notes/deprecation.rst
>>+++ b/doc/guides/rel_notes/deprecation.rst
>>@@ -152,3 +152,10 @@ Deprecation Notices
>> * bus/vmbus: Starting DPDK 25.11, all the vmbus API defined in
>>   ``drivers/bus/vmbus/rte_bus_vmbus.h`` will become internal to DPDK.
>>   Those API functions are used internally by DPDK core and netvsc PMD.
>>+
>>+* dmadev: a new capability flag ``RTE_DMA_CAPA_INTER_DEV`` will be
>>+added
>>+  to advertise DMA device's inter-device DMA copy capability. To
>>+enable
>>+  this functionality, a few dmadev APIs will be added to configure the
>>+DMA
>>+  access groups, facilitating coordinated data communication between
>>devices.
>>+  A new ``dev_idx`` field will be added to the ``struct
>>+rte_dma_vchan_conf``
>>+  structure to configure a vchan for data transfers between any two
>>+DMA
>>devices.
>>--
>>2.34.1

RE: [EXTERNAL] Re: [PATCH v0 1/1] doc: announce inter-device DMA capability support in dmadev

[Vamsi Krishna Attunuru]

[-- Attachment #1: Type: text/plain, Size: 3146 bytes --]

Hi Vladimir,

Currently, mem-to-mem dma transfers happen within a single process address space. This feature aims to extend support for transfers across multiple process address spaces to meet the requirements of inter-process or inter-VM networking. In environments where share memory is not feasible due to security concerns, this feature allows controlled software to use DMA for packet transfers as needed.

Regards.

From: Medvedkin, Vladimir <vladimir.medvedkin@intel.com>
Sent: Monday, July 14, 2025 3:47 PM
To: dev@dpdk.org
Cc: Bruce Richardson <bruce.richardson@intel.com>
Subject: [EXTERNAL] Re: [PATCH v0 1/1] doc: announce inter-device DMA capability support in dmadev

Hi Vamsi, Could you please tell us more about usecases for this feature? Thanks! On 7/10/2025 9: 51 AM, Vamsi Krishna wrote: > From: Vamsi Attunuru <vattunuru@ marvell. com> > > Modern DMA hardware supports data transfer between
ZjQcmQRYFpfptBannerStart
Prioritize security for external emails:
Confirm sender and content safety before clicking links or opening attachments
    Report Suspicious  <https://us-phishalarm-ewt.proofpoint.com/EWT/v1/CRVmXkqW!ti3Yn35UY3X61OwcGRf6bRAsmnzalHb1b3IuGKm4KAmFKsaahO5TQ00KAcbC6zX9rZkmmbglmeBA6oy8diWgp0Bz2fptvfWCiA$>   ‌
ZjQcmQRYFpfptBannerEnd

Hi Vamsi,



Could you please tell us more about usecases for this feature?



Thanks!



On 7/10/2025 9:51 AM, Vamsi Krishna wrote:

> From: Vamsi Attunuru <vattunuru@marvell.com<mailto:vattunuru@marvell.com>>

>

> Modern DMA hardware supports data transfer between multiple

> DMA devices, enabling data communication across isolated domains or

> containers. To facilitate this, the ``dmadev`` library requires changes

> to allow devices to register with or unregisters from DMA groups for

> inter-device communication. This feature is planned for inclusion

> in DPDK 25.11.

>

> Signed-off-by: Vamsi Attunuru <vattunuru@marvell.com<mailto:vattunuru@marvell.com>>

> ---

>   doc/guides/rel_notes/deprecation.rst | 7 +++++++

>   1 file changed, 7 insertions(+)

>

> diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst

> index e2d4125308..46836244dd 100644

> --- a/doc/guides/rel_notes/deprecation.rst

> +++ b/doc/guides/rel_notes/deprecation.rst

> @@ -152,3 +152,10 @@ Deprecation Notices

>   * bus/vmbus: Starting DPDK 25.11, all the vmbus API defined in

>     ``drivers/bus/vmbus/rte_bus_vmbus.h`` will become internal to DPDK.

>     Those API functions are used internally by DPDK core and netvsc PMD.

> +

> +* dmadev: a new capability flag ``RTE_DMA_CAPA_INTER_DEV`` will be added

> +  to advertise DMA device's inter-device DMA copy capability. To enable

> +  this functionality, a few dmadev APIs will be added to configure the DMA

> +  access groups, facilitating coordinated data communication between devices.

> +  A new ``dev_idx`` field will be added to the ``struct rte_dma_vchan_conf``

> +  structure to configure a vchan for data transfers between any two DMA devices.



--

Regards,

Vladimir



[-- Attachment #2: Type: text/html, Size: 12230 bytes --]

Re: [PATCH v0 1/1] doc: announce inter-device DMA capability support in dmadev

[fengchengwen]

Hi Vamsi,

From the commit log, I guess this commit mainly want to meet following case:

     ---------------             ----------------
     |  Container  |             |  VirtMachine |
     |             |             |              |
     |  dmadev-1   |             |   dmadev2    |
     ---------------             ----------------
           |                            |
           ------------------------------

App run in the container could launch DMA transfer from local buffer to the VirtMachine by config
dmadev-1/2 (the dmadev-1/2 are passthrough to diffent OS domain).

Could you explain how to use it from application perspective (for example address translation) and
application & hardware restrictions?


BTW: In this case, there are two OS domain communication, and I remember there are also inter-process
DMA RFC, so maybe we could design more generic solution if you provide more info.

Thanks

On 2025/7/10 16:51, Vamsi Krishna wrote:
> From: Vamsi Attunuru <vattunuru@marvell.com>
> 
> Modern DMA hardware supports data transfer between multiple
> DMA devices, enabling data communication across isolated domains or
> containers. To facilitate this, the ``dmadev`` library requires changes
> to allow devices to register with or unregisters from DMA groups for
> inter-device communication. This feature is planned for inclusion
> in DPDK 25.11.
> 
> Signed-off-by: Vamsi Attunuru <vattunuru@marvell.com>
> ---
>  doc/guides/rel_notes/deprecation.rst | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
> index e2d4125308..46836244dd 100644
> --- a/doc/guides/rel_notes/deprecation.rst
> +++ b/doc/guides/rel_notes/deprecation.rst
> @@ -152,3 +152,10 @@ Deprecation Notices
>  * bus/vmbus: Starting DPDK 25.11, all the vmbus API defined in
>    ``drivers/bus/vmbus/rte_bus_vmbus.h`` will become internal to DPDK.
>    Those API functions are used internally by DPDK core and netvsc PMD.
> +
> +* dmadev: a new capability flag ``RTE_DMA_CAPA_INTER_DEV`` will be added
> +  to advertise DMA device's inter-device DMA copy capability. To enable
> +  this functionality, a few dmadev APIs will be added to configure the DMA
> +  access groups, facilitating coordinated data communication between devices.
> +  A new ``dev_idx`` field will be added to the ``struct rte_dma_vchan_conf``
> +  structure to configure a vchan for data transfers between any two DMA devices.

RE: [EXTERNAL] Re: [PATCH v0 1/1] doc: announce inter-device DMA capability support in dmadev

[Vamsi Krishna Attunuru]

[-- Attachment #1: Type: text/plain, Size: 4802 bytes --]

Hi Feng,

Thanks for depicting the feature use case.

From the application’s perspective, inter VM/process communication is required to exchange the src & dst buffer details, however the specifics of this communication mechanism are outside the scope in this context. Regarding the address translations, these buffer addresses can be either IOVA as PA or IOVA as VA. The DMA hardware must use the appropriate IOMMU stream IDs when initiating the DMA transfers. For example, in the use case shown in the diagram, dmadev-1 and dmadev-2 would join an access group managed by the kernel DMA controller driver. This controller driver will configure the access group on the DMA hardware, enabling the hardware to select the correct stream IDs for read/write operations. New rte_dma APIs could be introduced to join or leave the access group or to query the access group details. Additionally, a secure token mechanism (similar to vfio-pci token) can be implemented to validate any dmadev attempting to join the access group.

Regards.

From: fengchengwen <fengchengwen@huawei.com>
Sent: Tuesday, July 15, 2025 6:29 AM
To: Vamsi Krishna Attunuru <vattunuru@marvell.com>; dev@dpdk.org; Pavan Nikhilesh Bhagavatula <pbhagavatula@marvell.com>; kevin.laatz@intel.com; bruce.richardson@intel.com; mb@smartsharesystems.com
Cc: Jerin Jacob <jerinj@marvell.com>; thomas@monjalon.net
Subject: [EXTERNAL] Re: [PATCH v0 1/1] doc: announce inter-device DMA capability support in dmadev

Hi Vamsi, From the commit log, I guess this commit mainly want to meet following case: --------------- ---------------- | Container | | VirtMachine | | | | | | dmadev-1 | | dmadev2 | --------------- ---------------- | | ------------------------------
ZjQcmQRYFpfptBannerStart
Prioritize security for external emails:
Confirm sender and content safety before clicking links or opening attachments
    Report Suspicious  <https://us-phishalarm-ewt.proofpoint.com/EWT/v1/CRVmXkqW!tg3ZldV0Yr_wdSwWmT2aDdKMi-4rn2z58vFaxwfOeocS1P19w1BeRdyGs5sjnhV2rU_6m8MOWj4KFbuXKkKJIvcqwWD2WEwJW_0$>   ‌
ZjQcmQRYFpfptBannerEnd

Hi Vamsi,



From the commit log, I guess this commit mainly want to meet following case:



     ---------------             ----------------

     |  Container  |             |  VirtMachine |

     |             |             |              |

     |  dmadev-1   |             |   dmadev2    |

     ---------------             ----------------

           |                            |

           ------------------------------



App run in the container could launch DMA transfer from local buffer to the VirtMachine by config

dmadev-1/2 (the dmadev-1/2 are passthrough to diffent OS domain).



Could you explain how to use it from application perspective (for example address translation) and

application & hardware restrictions?





BTW: In this case, there are two OS domain communication, and I remember there are also inter-process

DMA RFC, so maybe we could design more generic solution if you provide more info.



Thanks



On 2025/7/10 16:51, Vamsi Krishna wrote:

> From: Vamsi Attunuru <vattunuru@marvell.com<mailto:vattunuru@marvell.com>>

>

> Modern DMA hardware supports data transfer between multiple

> DMA devices, enabling data communication across isolated domains or

> containers. To facilitate this, the ``dmadev`` library requires changes

> to allow devices to register with or unregisters from DMA groups for

> inter-device communication. This feature is planned for inclusion

> in DPDK 25.11.

>

> Signed-off-by: Vamsi Attunuru <vattunuru@marvell.com<mailto:vattunuru@marvell.com>>

> ---

>  doc/guides/rel_notes/deprecation.rst | 7 +++++++

>  1 file changed, 7 insertions(+)

>

> diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst

> index e2d4125308..46836244dd 100644

> --- a/doc/guides/rel_notes/deprecation.rst

> +++ b/doc/guides/rel_notes/deprecation.rst

> @@ -152,3 +152,10 @@ Deprecation Notices

>  * bus/vmbus: Starting DPDK 25.11, all the vmbus API defined in

>    ``drivers/bus/vmbus/rte_bus_vmbus.h`` will become internal to DPDK.

>    Those API functions are used internally by DPDK core and netvsc PMD.

> +

> +* dmadev: a new capability flag ``RTE_DMA_CAPA_INTER_DEV`` will be added

> +  to advertise DMA device's inter-device DMA copy capability. To enable

> +  this functionality, a few dmadev APIs will be added to configure the DMA

> +  access groups, facilitating coordinated data communication between devices.

> +  A new ``dev_idx`` field will be added to the ``struct rte_dma_vchan_conf``

> +  structure to configure a vchan for data transfers between any two DMA devices.



[-- Attachment #2: Type: text/html, Size: 16233 bytes --]