Re: [dpdk-dev] DDPK use of MAP_FIXED in mmap

[Bruce Richardson <bruce.richardson@intel.com>]

On Mon, Dec 08, 2014 at 07:02:38PM +0000, Karmarkar Suyash wrote:
> Hello,
> 
> In DPDK when we use mmap why are we passing the MAP_FIXED flag when Linux man page itself says that the option is discouraged? Any specific reason for passing the MAP_FIXED flag?
> 
> 
> http://linux.die.net/man/2/mmap
> 
> MAP_FIXED
> Don't interpret addr as a hint: place the mapping at exactly that address. addr must be a multiple of the page size. If the memory region specified by addr and len overlaps pages of any existing mapping(s), then the overlapped part of the existing mapping(s) will be discarded. If the specified address cannot be used, mmap() will fail. Because requiring a fixed address for a mapping is less portable, the use of this option is discouraged.
> 
> 
> Regards
> Suyash Karmarkar

I won't comment on every occurance of "MAP_FIXED" in DPDK, but it's main use is
when mapping the hugepages into memory inside EAL init. In this case, we are ok to
use it, as we take good care to ensure that our mapping space is free. What we do
is, once we know how many contiguous hugepages we need to map, we request a mapping
from /dev/zero for that particular size. We then record the address of the mapping
we get, and then unmap /dev/zero again - thereby freeing up the entire address
range. At this point, we then use MAP_FIXED to explicitly mmap in the hugepages
into this region that we have just freed up - thereby guaranteeing contiguous
hugepage mappings in the correct order. [The reason for doing things this way is
that we found on some systems - particularly with 32-bit code, the regular mmaps
of pages we being done in reverse order, meaning each page became it's own segment].

On the other hand, it's also good to note where we don't use MAP_FIXED. We don't
use map fixed when initializing a secondary process and are mapping the hugepage
memory into it. In this case, although we know where the memory has to be placed,
we don't know if it is safe to use or not. Instead of using MAP_FIXED, we instead
hint to the kernel our preferred address and check if the request was satisfied
at that address.

Hope this clarifies things a bit,
/Bruce