Re: [dpdk-dev] [PATCH 0/6] introduce DMA memory mapping for external memory

[Shahaf Shuler <shahafs@mellanox.com>]

Thursday, February 14, 2019 12:19 PM, Burakov, Anatoly:
> Subject: Re: [dpdk-dev] [PATCH 0/6] introduce DMA memory mapping for
> external memory
> 
> On 13-Feb-19 7:24 PM, Shahaf Shuler wrote:
> > Wednesday, February 13, 2019 1:43 PM, Alejandro Lucero:
> >> Subject: Re: [dpdk-dev] [PATCH 0/6] introduce DMA memory mapping for
> >> external memory
> >>
> >> On Wed, Feb 13, 2019 at 9:11 AM Shahaf Shuler
> <shahafs@mellanox.com>
> >> wrote:
> >>
> >>> This series is in continue to RFC[1].
> >>>
> >>> The DPDK APIs expose 3 different modes to work with memory used for
> >> DMA:
> >>>
> >>> 1. Use the DPDK owned memory (backed by the DPDK provided
> >> hugepages).
> >>> This memory is allocated by the DPDK libraries, included in the DPDK
> >>> memory system (memseg lists) and automatically DMA mapped by the
> >> DPDK
> >>> layers.
> >>>
> >>> 2. Use memory allocated by the user and register to the DPDK memory
> >>> systems. This is also referred as external memory. Upon registration
> >>> of the external memory, the DPDK layers will DMA map it to all
> >>> needed devices.
> >>>
> >>> 3. Use memory allocated by the user and not registered to the DPDK
> >>> memory system. This is for users who wants to have tight control on
> >>> this memory. The user will need to explicitly call DMA map function
> >>> in order to register such memory to the different devices.
> >>>
> >>> The scope of the patch focus on #3 above.
> >>>
> >>>
> >> Why can not we have case 2 covering case 3?
> >
> > Because it is not our choice rather the DPDK application.
> > We could not allow it, and force the application to register their external
> memory to the DPDK memory management system. However IMO it will be
> wrong.
> > The use case exists  - some application wants to manage their memory by
> themselves. w/o the extra overhead of rte_malloc, without creating a special
> socket to populate the memory and without redundant API calls to
> rte_extmem_*.
> >
> > Simply allocate chunk of memory, DMA map it to device and that’s it.
> 
> Just a small note: while this sounds good on paper, i should point out that at
> least *registering* the memory with DPDK is a necessity. You may see
> rte_extmem_* calls as redundant (and i agree, to an extent), but we don't
> advertise our PMD's capabilities in a way that makes it easy to determine
> whether a particular PMD will or will not work without registering external
> memory within DPDK (i.e. does it use
> rte_virt2memseg() internally, for example).
> 
> So, extmem register calls are a necessary evil in such case, and IMO should
> be called out as required for such external memory usage scenario.

If we are going to force all to use the extmem, then there is no need w/ this API. we can have the PMDs to register when the memory is registered. 
We can just drop the vfio_dma_map APIs and that's it.



> 
> --
> Thanks,
> Anatoly