Re: [dpdk-dev] [RFC] ethdev: introduce DMA memory mapping for external memory

[Stephen Hemminger <stephen@networkplumber.org>]

On Thu, 15 Nov 2018 10:59:36 +0000
"Burakov, Anatoly" <anatoly.burakov@intel.com> wrote:

> > Unfortunately it cannot be done at least w/ Mellanox.
> > In Mellanox the kernel driver is the one which maps the memory. The mapping returns a key which identify a memory region which was just registered to the device.
> > There is a complete separation between the ports, meaning one port mapping cannot be used by in the other port, even if the key is known.
> > 
> > The separation is not only in ports, but also in processes (two primary ones, for secondary we have a way to share). If two process work on the same device, the must register the memory independently.  
> 
> Ah, OK.
> 
> So, we're right back to where we started. Right now, external memory 
> expects to behave the same way as all other memory - you don't need to 
> perform DMA mapping for it.
> 
> That said, part of the reason *why* it was done that way was because 
> there is no way to trigger VFIO DMA mapping for NXP (or was it MLX?) 
> devices. If you look at initial versions of the patchset, the DMA 
> mapping was actually done manually. Then, i became convinced that doing 
> this automatically is the way to go, both because it erases the 
> usability differences as far as memory types are concerned, and because 
> it enables whatever services that are subscribing to memory events to 
> receive notifications about external memory as well (i.e. consistency).
> 
> Given that it's still an experimental API, we can tinker with it all we 
> like, so it's not set in stone. However, i would really like to keep the 
> current automagic thing, because DMA mapping may not be the only user of 
> memory callbacks - they can be used for debug purposes, or for any other 
> things.

If you look at the netvsc PMD, you will discover it to has the kernel
setup the receive memory area. There is some flexibility about where it
is mmap'd but the memory is coming from pinned kernel memory.