From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <anatoly.burakov@intel.com>
Received: from mga05.intel.com (mga05.intel.com [192.55.52.43])
 by dpdk.org (Postfix) with ESMTP id 21C842A62
 for <dev@dpdk.org>; Tue, 20 Mar 2018 11:07:55 +0100 (CET)
X-Amp-Result: SKIPPED(no attachment in message)
X-Amp-File-Uploaded: False
Received: from orsmga005.jf.intel.com ([10.7.209.41])
 by fmsmga105.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;
 20 Mar 2018 03:07:54 -0700
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.48,334,1517904000"; d="scan'208";a="209750076"
Received: from unknown (HELO [10.237.220.112]) ([10.237.220.112])
 by orsmga005.jf.intel.com with ESMTP; 20 Mar 2018 03:07:53 -0700
To: Shreyansh Jain <shreyansh.jain@nxp.com>
Cc: dev@dpdk.org, Thomas Monjalon <thomas@monjalon.net>
References: <cover.1520083504.git.anatoly.burakov@intel.com>
 <cover.1520428025.git.anatoly.burakov@intel.com>
 <CAJ5mUsVPwy1wc16CFxPxRMCsXDkgQ2=0uXDct327pbtj3rFr8A@mail.gmail.com>
From: "Burakov, Anatoly" <anatoly.burakov@intel.com>
Message-ID: <38cdd1c0-a92f-73ad-e534-292335481eb0@intel.com>
Date: Tue, 20 Mar 2018 10:07:53 +0000
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101
 Thunderbird/52.6.0
MIME-Version: 1.0
In-Reply-To: <CAJ5mUsVPwy1wc16CFxPxRMCsXDkgQ2=0uXDct327pbtj3rFr8A@mail.gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Subject: Re: [dpdk-dev] [PATCH v2 00/41] Memory Hotplug for DPDK
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://dpdk.org/ml/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://dpdk.org/ml/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
X-List-Received-Date: Tue, 20 Mar 2018 10:07:56 -0000

On 19-Mar-18 8:58 AM, Shreyansh Jain wrote:
> Hi Anatoly,
> 
> On Wed, Mar 7, 2018 at 10:26 PM, Anatoly Burakov
> <anatoly.burakov@intel.com> wrote:
>> This patchset introduces dynamic memory allocation for DPDK (aka memory
>> hotplug). Based upon RFC submitted in December [1].
>>
>> Dependencies (to be applied in specified order):
>> - IPC bugfixes patchset [2]
>> - IPC improvements patchset [3]
>> - IPC asynchronous request API patch [4]
>> - Function to return number of sockets [5]
>>
>> Deprecation notices relevant to this patchset:
>> - General outline of memory hotplug changes [6]
>> - EAL NUMA node count changes [7]
>>
>> The vast majority of changes are in the EAL and malloc, the external API
>> disruption is minimal: a new set of API's are added for contiguous memory
>> allocation for rte_memzone, and a few API additions in rte_memory due to
>> switch to memseg_lists as opposed to memsegs. Every other API change is
>> internal to EAL, and all of the memory allocation/freeing is handled
>> through rte_malloc, with no externally visible API changes.
>>
>> Quick outline of all changes done as part of this patchset:
>>
>>   * Malloc heap adjusted to handle holes in address space
>>   * Single memseg list replaced by multiple memseg lists
>>   * VA space for hugepages is preallocated in advance
>>   * Added alloc/free for pages happening as needed on rte_malloc/rte_free
>>   * Added contiguous memory allocation API's for rte_memzone
>>   * Integrated Pawel Wodkowski's patch for registering/unregistering memory
>>     with VFIO [8]
>>   * Callbacks for registering memory allocations
>>   * Multiprocess support done via DPDK IPC introduced in 18.02
>>
>> The biggest difference is a "memseg" now represents a single page (as opposed to
>> being a big contiguous block of pages). As a consequence, both memzones and
>> malloc elements are no longer guaranteed to be physically contiguous, unless
>> the user asks for it at reserve time. To preserve whatever functionality that
>> was dependent on previous behavior, a legacy memory option is also provided,
>> however it is expected (or perhaps vainly hoped) to be temporary solution.
>>
>> Why multiple memseg lists instead of one? Since memseg is a single page now,
>> the list of memsegs will get quite big, and we need to locate pages somehow
>> when we allocate and free them. We could of course just walk the list and
>> allocate one contiguous chunk of VA space for memsegs, but this
>> implementation uses separate lists instead in order to speed up many
>> operations with memseg lists.
>>
>> For v1 and v2, the following limitations are present:
>> - FreeBSD does not even compile, let alone run
>> - No 32-bit support
> 
> I just read on announce@dpdk.org [1] that an early merge to this
> series is expected. So, would this limitation be fixed before merge?
> Or, has it already been fixed in github repo?
> 
> [1] http://dpdk.org/ml/archives/announce/2018-March/000182.html
> 
> [...]
> 
> -
> Shreyansh
> 

Hi Shreyansh,

It will be fixed before merge, yes. I would expect this code to arrive 
in Github in the next few days.

-- 
Thanks,
Anatoly