Re: [dpdk-dev] [PATCH 3/5] common/dpaax: add library for PA VA translation table

[Shreyansh Jain <shreyansh.jain@nxp.com>]

On Tuesday 25 September 2018 07:21 PM, Burakov, Anatoly wrote:
> On 25-Sep-18 2:39 PM, Shreyansh Jain wrote:
>> Hello Anatoly,
>>
>> On Tuesday 25 September 2018 06:58 PM, Burakov, Anatoly wrote:
>>> On 25-Sep-18 1:54 PM, Shreyansh Jain wrote:
>>>> A common library, valid for dpaaX drivers, which is used to maintain
>>>> a local copy of PA->VA translations.
>>>>
>>>> In case of physical addressing mode (one of the option for FSLMC, and
>>>> only option for DPAA bus), the addresses of descriptors Rx'd are
>>>> physical. These need to be converted into equivalent VA for rte_mbuf
>>>> and other similar calls.
>>>>
>>>> Using the rte_mem_virt2iova or rte_mem_virt2phy is expensive. This
>>>> library is an attempt to reduce the overall cost associated with
>>>> this translation.
>>>>
>>>> A small table is maintained, containing continuous entries
>>>> representing a continguous physical range. Each of these entries
>>>> stores the equivalent VA, which is fed during mempool creation, or
>>>> memory allocation/deallocation callbacks.
>>>>
>>>> Signed-off-by: Shreyansh Jain <shreyansh.jain@nxp.com>
>>>> ---
>>>
>>> Hi Shreyansh,
>>>
>>> So, basically, you're reimplementing old DPDK's memory view (storing 
>>> VA's in a PA-centric way). Makes sense :)
>>
>> Yes, and frankly, I couldn't come up with any other way.
>>
>>>
>>> I should caution you that right now, external memory allocator 
>>> implementation does *not* trigger any callbacks for newly added 
>>> memory. So, anything coming from external memory will not be 
>>> reflected in your table, unless it happens to be already there before 
>>> dpaax_iova_table_populate() gets called. This patchset makes a good 
>>> argument for why perhaps it should trigger callbacks. Thoughts?
>>
>> Oh. Then I must be finishing reading through your patches for external 
>> memory sooner. I didn't realize this.
> 
> To be clear, the current implementation of external memory allocators is 
> not necessarily final - it's not too late to add callbacks to enable 
> your use case better, if that's required (and it should be pretty easy 
> to implement as well).
> 

Is there any reason why external may not be raising call back right now? 
I might have missed any previous conversation on this. Or may be, it is 
just lack of need.

As for whether it is required - I do see a need. It is definitely 
possible that after rte_eal_init has been completed (and underlying 
probe), applications allocate memory. In which case, even existing 
memevent callbacks (like the one in fslmc_bus, which VFIO/DMA maps the 
area) would have issues. From the external memory patchset, I do see 
that it is assumed DMA mapping is caller's responsibility.

Having such callback would help drives reduce that throwback of 
responsibility.

(Speaking of external memory patches, I also realize that my memevent 
callback in this patch series need to handle msl->external).