From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by dpdk.org (Postfix) with ESMTP id 828961B1BE for ; Thu, 25 Jan 2018 17:18:47 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga105.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 25 Jan 2018 08:18:46 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.46,412,1511856000"; d="scan'208";a="29444298" Received: from nstojanx-wtg.amr.corp.intel.com (HELO [10.252.26.38]) ([10.252.26.38]) by orsmga002.jf.intel.com with ESMTP; 25 Jan 2018 08:18:44 -0800 To: Yongseok Koh Cc: "dev@dpdk.org" , "andras.kovacs@ericsson.com" , "laszlo.vadkeri@ericsson.com" , "keith.wiles@intel.com" , "benjamin.walker@intel.com" , "bruce.richardson@intel.com" , Thomas Monjalon References: <63712223-C6E5-4319-9958-86445D38F076@mellanox.com> From: "Burakov, Anatoly" Message-ID: Date: Thu, 25 Jan 2018 16:18:43 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.5.2 MIME-Version: 1.0 In-Reply-To: <63712223-C6E5-4319-9958-86445D38F076@mellanox.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [RFC v2 00/23] Dynamic memory allocation for DPDK X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Jan 2018 16:18:48 -0000 On 23-Jan-18 10:33 PM, Yongseok Koh wrote: > >> On Dec 19, 2017, at 3:14 AM, Anatoly Burakov wrote: > [...] >> Quick outline of all changes done as part of this patchset: >> >> * Malloc heap adjusted to handle holes in address space >> * Single memseg list replaced by multiple expandable memseg lists >> * VA space for hugepages is preallocated in advance > > Hi Anatoly, > > I haven't looked through your patchset yet but quick question. As far as I > understand, currently EAL remaps virtual addresses to make VA layout matches PA > layout. I'm not sure my expression is 100% correct. > > By your comment above, do you mean VA space for all available physical memory > will always be contiguous? > > I have been curious about why VA space is fragmented in DPDK. > > Thanks, > Yongseok > > Hi Yongseok, Yes and no. Currently, VA space is allocated opportunistically - EAL tries to allocate VA space to match PA space layout, but due to varying page sizes and contiguous segment sizes, it doesn't always turn out that way - hence possible VA fragmentation even if underlying PA space may be contiguous. With this patchset, we kind of do it the other way around - we allocate contiguous VA space segment per socket, per page size, and then we map physical memory into it, without regard for PA layout whatsoever. So, assuming all VA space is mapped, VA space is contiguous while PA space may or may not be (depending on underlying physical memory layout and if you're using IOMMU). However, since this is hotpluggable (and hot-unpluggable) memory, we can have holes in *mapped* VA space, even though *allocated* VA space would be contiguous within the boundaries of page size. You can think of it this way - we preallocate segments per page size, per socket, so e.g. on a machine with 2 sockets and 2MB and 1GB pages enabled, you'll get 4 contiguous chunks of VA space that is available for mapping. Underlying mappings may or may not be contiguous (i.e. mapped segments might become fragmented if you allocate/free memory all over the pace), but they will reside within the same contiguous chunk of VA space. Hope that answers your question :) -- Thanks, Anatoly