From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by dpdk.org (Postfix) with ESMTP id 2AFA28066 for ; Tue, 9 Dec 2014 10:55:09 +0100 (CET) Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga101.fm.intel.com with ESMTP; 09 Dec 2014 01:55:09 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.07,543,1413270000"; d="scan'208";a="634895129" Received: from bricha3-mobl3.ger.corp.intel.com ([10.243.20.39]) by fmsmga001.fm.intel.com with SMTP; 09 Dec 2014 01:55:07 -0800 Received: by (sSMTP sendmail emulation); Tue, 09 Dec 2014 09:55:06 +0025 Date: Tue, 9 Dec 2014 09:55:06 +0000 From: Bruce Richardson To: Karmarkar Suyash Message-ID: <20141209095506.GB9472@bricha3-MOBL3> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Organization: Intel Shannon Ltd. User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] DDPK use of MAP_FIXED in mmap X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Dec 2014 09:55:10 -0000 On Mon, Dec 08, 2014 at 07:02:38PM +0000, Karmarkar Suyash wrote: > Hello, > > In DPDK when we use mmap why are we passing the MAP_FIXED flag when Linux man page itself says that the option is discouraged? Any specific reason for passing the MAP_FIXED flag? > > > http://linux.die.net/man/2/mmap > > MAP_FIXED > Don't interpret addr as a hint: place the mapping at exactly that address. addr must be a multiple of the page size. If the memory region specified by addr and len overlaps pages of any existing mapping(s), then the overlapped part of the existing mapping(s) will be discarded. If the specified address cannot be used, mmap() will fail. Because requiring a fixed address for a mapping is less portable, the use of this option is discouraged. > > > Regards > Suyash Karmarkar I won't comment on every occurance of "MAP_FIXED" in DPDK, but it's main use is when mapping the hugepages into memory inside EAL init. In this case, we are ok to use it, as we take good care to ensure that our mapping space is free. What we do is, once we know how many contiguous hugepages we need to map, we request a mapping from /dev/zero for that particular size. We then record the address of the mapping we get, and then unmap /dev/zero again - thereby freeing up the entire address range. At this point, we then use MAP_FIXED to explicitly mmap in the hugepages into this region that we have just freed up - thereby guaranteeing contiguous hugepage mappings in the correct order. [The reason for doing things this way is that we found on some systems - particularly with 32-bit code, the regular mmaps of pages we being done in reverse order, meaning each page became it's own segment]. On the other hand, it's also good to note where we don't use MAP_FIXED. We don't use map fixed when initializing a secondary process and are mapping the hugepage memory into it. In this case, although we know where the memory has to be placed, we don't know if it is safe to use or not. Instead of using MAP_FIXED, we instead hint to the kernel our preferred address and check if the request was satisfied at that address. Hope this clarifies things a bit, /Bruce