Re: [dpdk-dev] [PATCH] eal/vfio: share the default container in multi process

["Burakov, Anatoly" <anatoly.burakov@intel.com>]

On 04-Oct-18 4:52 PM, Burakov, Anatoly wrote:
> On 03-Oct-18 1:39 PM, Darek Stojaczyk wrote:
>> So far each process in MP used to have a separate container
>> and relied on the primary process to register all memsegs.
>>
>> Mapping external memory via rte_vfio_container_dma_map()
>> in secondary processes was broken, because the default
>> (process-local) container had no groups bound. There was
>> even no way to bind any groups to it, because the container
>> fd was deeply encapsulated within EAL.
>>
>> This patch introduces a new SOCKET_REQ_DEFAULT_CONTAINER
>> message type for MP synchronization, makes all processes
>> within a MP party use a single default container, and hence
>> fixes rte_vfio_container_dma_map() for secondary processes.
>>
>>> From what I checked this behavior was always the same, but
>> started to be invalid/insufficient once mapping external
>> memory was allowed.
>>
>> Fixes: 73a639085938 ("vfio: allow to map other memory regions")
>> Cc: anatoly.burakov@intel.com
>> Cc: pawelx.wodkowski@intel.com
>> Cc: gowrishankar.m@linux.vnet.ibm.com
>> Cc: stable@dpdk.org
>>
>> While here, fix up the comment on rte_vfio_get_container_fd().
>> This function always opens a new container, never reuses
>> an old one.
>>
>> Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
>> ---
> 
> This makes things a bit funky when we consider external memory support. 
> The same address space may or may not be mapped into the process, so we 
> may or may not need to map things for DMA depending on whether we're 
> mapping shared memory for DMA, or not.
> 
> The previous behavior may be counter-intuitive, but it's IMO the correct 
> one - each process has its own container and performs its own DMA mappings.
> 

Re-reading the commit message, i can see that i've misinterpreted the 
commit a bit (or rather i missed the fact that we can't even map 
anything for DMA in the secondaries due to container being not the same 
container we use devices for).

However, my above comment regarding external memory still applies - if 
we share DMA mappings between processes, it will be possible for 
secondary to map something else into space occupied by external memory, 
and potentially overwrite the DMA mappings.

Unfortunately, we cannot do anything about it, as that memory isn't 
under our control. I will have to update my external memory patchset to 
account for this and map memory for DMA only once, in whatever process 
that creates the memory in the first place.

So, on the concept of this i have no objections here, as i think this 
fix is necessary. I'll have to review the code some more to make sure 
i'm not missing anything.

-- 
Thanks,
Anatoly