Re: [dpdk-dev] [PATCH 0/6] introduce DMA memory mapping for external memory

["Burakov, Anatoly" <anatoly.burakov@intel.com>]

On 14-Feb-19 1:28 PM, Shahaf Shuler wrote:
> Thursday, February 14, 2019 12:19 PM, Burakov, Anatoly:
>> Subject: Re: [dpdk-dev] [PATCH 0/6] introduce DMA memory mapping for
>> external memory
>>
>> On 13-Feb-19 7:24 PM, Shahaf Shuler wrote:
>>> Wednesday, February 13, 2019 1:43 PM, Alejandro Lucero:
>>>> Subject: Re: [dpdk-dev] [PATCH 0/6] introduce DMA memory mapping for
>>>> external memory
>>>>
>>>> On Wed, Feb 13, 2019 at 9:11 AM Shahaf Shuler
>> <shahafs@mellanox.com>
>>>> wrote:
>>>>
>>>>> This series is in continue to RFC[1].
>>>>>
>>>>> The DPDK APIs expose 3 different modes to work with memory used for
>>>> DMA:
>>>>>
>>>>> 1. Use the DPDK owned memory (backed by the DPDK provided
>>>> hugepages).
>>>>> This memory is allocated by the DPDK libraries, included in the DPDK
>>>>> memory system (memseg lists) and automatically DMA mapped by the
>>>> DPDK
>>>>> layers.
>>>>>
>>>>> 2. Use memory allocated by the user and register to the DPDK memory
>>>>> systems. This is also referred as external memory. Upon registration
>>>>> of the external memory, the DPDK layers will DMA map it to all
>>>>> needed devices.
>>>>>
>>>>> 3. Use memory allocated by the user and not registered to the DPDK
>>>>> memory system. This is for users who wants to have tight control on
>>>>> this memory. The user will need to explicitly call DMA map function
>>>>> in order to register such memory to the different devices.
>>>>>
>>>>> The scope of the patch focus on #3 above.
>>>>>
>>>>>
>>>> Why can not we have case 2 covering case 3?
>>>
>>> Because it is not our choice rather the DPDK application.
>>> We could not allow it, and force the application to register their external
>> memory to the DPDK memory management system. However IMO it will be
>> wrong.
>>> The use case exists  - some application wants to manage their memory by
>> themselves. w/o the extra overhead of rte_malloc, without creating a special
>> socket to populate the memory and without redundant API calls to
>> rte_extmem_*.
>>>
>>> Simply allocate chunk of memory, DMA map it to device and that’s it.
>>
>> Just a small note: while this sounds good on paper, i should point out that at
>> least *registering* the memory with DPDK is a necessity. You may see
>> rte_extmem_* calls as redundant (and i agree, to an extent), but we don't
>> advertise our PMD's capabilities in a way that makes it easy to determine
>> whether a particular PMD will or will not work without registering external
>> memory within DPDK (i.e. does it use
>> rte_virt2memseg() internally, for example).
>>
>> So, extmem register calls are a necessary evil in such case, and IMO should
>> be called out as required for such external memory usage scenario.
> 
> If we are going to force all to use the extmem, then there is no need w/ this API. we can have the PMDs to register when the memory is registered.
> We can just drop the vfio_dma_map APIs and that's it.
> 

Well, whether we needed it or not is not really my call, but what i can 
say is that using extmem_register is _necessary_ if you're going to use 
the PMD's. You're right, we could just map memory for DMA at register 
time - that would save one API call to get the memory working. It makes 
it a bit weird semantically, but i think we can live with that :)

-- 
Thanks,
Anatoly