From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by dpdk.org (Postfix) with ESMTP id 681364C70 for ; Thu, 8 Mar 2018 15:40:49 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Mar 2018 06:40:48 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.47,441,1515484800"; d="scan'208";a="206542076" Received: from aburakov-mobl.ger.corp.intel.com (HELO [10.237.220.141]) ([10.237.220.141]) by orsmga005.jf.intel.com with ESMTP; 08 Mar 2018 06:40:44 -0800 To: dev@dpdk.org Cc: keith.wiles@intel.com, jianfeng.tan@intel.com, andras.kovacs@ericsson.com, laszlo.vadkeri@ericsson.com, benjamin.walker@intel.com, bruce.richardson@intel.com, thomas@monjalon.net, konstantin.ananyev@intel.com, kuralamudhan.ramakrishnan@intel.com, louise.m.daly@intel.com, nelio.laranjeiro@6wind.com, yskoh@mellanox.com, pepperjo@japf.ch, jerin.jacob@caviumnetworks.com, hemant.agrawal@nxp.com, olivier.matz@6wind.com References: From: "Burakov, Anatoly" Message-ID: Date: Thu, 8 Mar 2018 14:40:44 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [PATCH 00/41] Memory Hotplug for DPDK X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Mar 2018 14:40:50 -0000 On 03-Mar-18 1:45 PM, Anatoly Burakov wrote: > This patchset introduces dynamic memory allocation for DPDK (aka memory > hotplug). Based upon RFC submitted in December [1]. > > Dependencies (to be applied in specified order): > - IPC bugfixes patchset [2] > - IPC improvements patchset [3] > - IPC asynchronous request API patch [4] > - Function to return number of sockets [5] > > Deprecation notices relevant to this patchset: > - General outline of memory hotplug changes [6] > - EAL NUMA node count changes [7] > > The vast majority of changes are in the EAL and malloc, the external API > disruption is minimal: a new set of API's are added for contiguous memory > allocation for rte_memzone, and a few API additions in rte_memory due to > switch to memseg_lists as opposed to memsegs. Every other API change is > internal to EAL, and all of the memory allocation/freeing is handled > through rte_malloc, with no externally visible API changes. > > Quick outline of all changes done as part of this patchset: > > * Malloc heap adjusted to handle holes in address space > * Single memseg list replaced by multiple memseg lists > * VA space for hugepages is preallocated in advance > * Added alloc/free for pages happening as needed on rte_malloc/rte_free > * Added contiguous memory allocation API's for rte_memzone > * Integrated Pawel Wodkowski's patch for registering/unregistering memory > with VFIO [8] > * Callbacks for registering memory allocations > * Multiprocess support done via DPDK IPC introduced in 18.02 > > The biggest difference is a "memseg" now represents a single page (as opposed to > being a big contiguous block of pages). As a consequence, both memzones and > malloc elements are no longer guaranteed to be physically contiguous, unless > the user asks for it at reserve time. To preserve whatever functionality that > was dependent on previous behavior, a legacy memory option is also provided, > however it is expected (or perhaps vainly hoped) to be temporary solution. > > Why multiple memseg lists instead of one? Since memseg is a single page now, > the list of memsegs will get quite big, and we need to locate pages somehow > when we allocate and free them. We could of course just walk the list and > allocate one contiguous chunk of VA space for memsegs, but this > implementation uses separate lists instead in order to speed up many > operations with memseg lists. > > For v1, the following limitations are present: > - FreeBSD does not even compile, let alone run > - No 32-bit support > - There are some minor quality-of-life improvements planned that aren't > ready yet and will be part of v2 > - VFIO support is only smoke-tested (but is expected to work), VFIO support > with secondary processes is not tested; work is ongoing to validate VFIO > for all use cases > - Dynamic mapping/unmapping memory with VFIO is not supported in sPAPR > IOMMU mode - help from sPAPR maintainers requested > > Nevertheless, this patchset should be testable under 64-bit Linux, and > should work for all use cases bar those mentioned above. > > [1] http://dpdk.org/dev/patchwork/bundle/aburakov/Memory_RFC/ > [2] http://dpdk.org/dev/patchwork/bundle/aburakov/IPC_Fixes/ > [3] http://dpdk.org/dev/patchwork/bundle/aburakov/IPC_Improvements/ > [4] http://dpdk.org/dev/patchwork/bundle/aburakov/IPC_Async_Request/ > [5] http://dpdk.org/dev/patchwork/bundle/aburakov/Num_Sockets/ > [6] http://dpdk.org/dev/patchwork/patch/34002/ > [7] http://dpdk.org/dev/patchwork/patch/33853/ > [8] http://dpdk.org/dev/patchwork/patch/24484/ > For those wanting to test this patchset, there is now a github tree with all of the dependent patches applied: https://github.com/anatolyburakov/dpdk -- Thanks, Anatoly