Re: [dpdk-dev] Running DPDK as an unprivileged user

["Walker, Benjamin" <benjamin.walker@intel.com>]

On Sun, 2017-11-05 at 01:17 +0100, Thomas Monjalon wrote:
> Hi, restarting an old topic,
> 
> 05/01/2017 16:52, Tan, Jianfeng:
> > On 1/5/2017 5:34 AM, Walker, Benjamin wrote:
> > > > > Note that this
> > > > > probably means that using uio on recent kernels is subtly
> > > > > broken and cannot be supported going forward because there
> > > > > is no uio mechanism to pin the memory.
> > > > > 
> > > > > The first open question I have is whether DPDK should allow
> > > > > uio at all on recent (4.x) kernels. My current understanding
> > > > > is that there is no way to pin memory and hugepages can now
> > > > > be moved around, so uio would be unsafe. What does the
> > > > > community think here?
> > 
> > Back to this question, removing uio support in DPDK seems a little 
> > overkill to me. Can we just document it down? Like, firstly warn users 
> > do not invoke migrate_pages() or move_pages() to a DPDK process; as for 
> > the kcompactd daemon and some more cases (like compaction could be 
> > triggered by alloc_pages()), could we just recommend to disable 
> > CONFIG_COMPACTION?
> 
> We really need to better document the limitations of UIO.
> May we have some suggestions here?
> 
> > Another side, how does vfio pin those memory? Through memlock (from code 
> > in vfio_pin_pages())? So why not just mlock those hugepages?
> 
> Good question. Why not mlock the hugepages?

mlock just guarantees that a virtual page is always backed by *some* physical
page of memory. It does not guarantee that over the lifetime of the process a
virtual page is mapped to the *same* physical page. The kernel is free to
transparently move memory around, compress it, dedupe it, etc.

vfio is not pinning the memory, but instead is using the IOMMU (a piece of
hardware) to participate in the memory management on the platform. If a device
begins a DMA transfer to an I/O virtual address, the IOMMU will coordinate with
the main MMU to make sure that the data ends up in the correct location, even as
the virtual to physical mappings are being modified.