Re: [dpdk-dev] [PATCH 3/6] bus: introduce DMA memory mapping for external memory

[Shahaf Shuler <shahafs@mellanox.com>]

Thursday, February 14, 2019 4:01 PM, Gaëtan Rivet:
> Subject: Re: [PATCH 3/6] bus: introduce DMA memory mapping for external
> memory
> 
> On Wed, Feb 13, 2019 at 07:07:11PM +0000, Shahaf Shuler wrote:
> > Wednesday, February 13, 2019 1:17 PM, Gaëtan Rivet:
> > > Subject: Re: [PATCH 3/6] bus: introduce DMA memory mapping for
> > > external memory

[...]

> > >
> > > How are those other vendors' devices mapped initially right now? Are
> > > they using #2 scheme instead? Then the user will remap everything using
> #3?
> >
> > It is not a re-map, it is a completely different mode for the memory
> management.
> > The first question to ask is "how the application wants to manage its
> memory" ?
> > If it is either #1 or #2 above, no problem to make the mapping internal on
> the "other vendor devices" as they can register to the memory event
> callback which trigger every time new memory is added to the DPDK memory
> management system.
> > For #3 the memory does not exists in the DPDK memory management
> system, and no memory events. Hence the application needs to explicitly call
> the dma MAP.
> > The change on this patch is just to make it more generic than calling only
> VFIO.
> >
> 
> Right! I mostly used #1 ports and never really thought about other kind of
> memory management or how they might follow a different logic.
> 
> Do you think this could be used with a lot of sequential mapping/unmappings
> happening?

It much depends how efficient is the driver mapping and unmapping. 
In most cases, mapping is heavy operation. 

> 
> I'm thinking for example about a crypto app feeding crypto buffers, being
> able to directly map the result instead of copying it within buffers might be
> interesting. But then you'd have to unmap often.
> 
> - Is the unmap() simple from the app PoV?

Yes, just call rte_bus_dma_unmap. 

> 
> - Must the mapping remain available for a long time?

It must remain as long as you need the device to access the memory. On your example, it should remain till the crypto dev finished writing the buffers. 

> 
> - Does the app need to call tx_descriptor_status() a few times or
>   does dma_unmap() verify that the mapping is not in use before
> unmapping?

I think it is a matter of driver implementation. 
In general, it is application responsibly to make sure the memory is no longer needed before unmapping, just like I don't destroy today mempool being used by some rxq. It can be done by any means the application has, not only tx_descriptor_status.
Driver can protect bad application and warn + fail the call on such case, however it is not a must. 

> 
> --
> Gaëtan Rivet
> 6WIND