From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <stable-bounces@dpdk.org>
Received: from dpdk.org (dpdk.org [92.243.14.124])
by inbox.dpdk.org (Postfix) with ESMTP id 23372A046B
for <public@inbox.dpdk.org>; Thu, 9 Jan 2020 14:57:37 +0100 (CET)
Received: from [92.243.14.124] (localhost [127.0.0.1])
by dpdk.org (Postfix) with ESMTP id 01D301DDB4;
Thu, 9 Jan 2020 14:57:37 +0100 (CET)
Received: from mga01.intel.com (mga01.intel.com [192.55.52.88])
by dpdk.org (Postfix) with ESMTP id E70001DDA8;
Thu, 9 Jan 2020 14:57:33 +0100 (CET)
X-Amp-Result: SKIPPED(no attachment in message)
X-Amp-File-Uploaded: False
Received: from orsmga005.jf.intel.com ([10.7.209.41])
by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;
09 Jan 2020 05:57:33 -0800
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.69,413,1571727600"; d="scan'208";a="396097092"
Received: from aburakov-mobl1.ger.corp.intel.com (HELO [10.237.220.107])
([10.237.220.107])
by orsmga005.jf.intel.com with ESMTP; 09 Jan 2020 05:57:31 -0800
To: Olivier Matz <olivier.matz@6wind.com>, dev@dpdk.org
Cc: Andrew Rybchenko <arybchenko@solarflare.com>, stable@dpdk.org
References: <20200109132742.15828-1-olivier.matz@6wind.com>
From: "Burakov, Anatoly" <anatoly.burakov@intel.com>
Message-ID: <0e310d2d-3455-4201-6aa9-6f5c7b2032e3@intel.com>
Date: Thu, 9 Jan 2020 13:57:31 +0000
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101
Thunderbird/68.3.1
MIME-Version: 1.0
In-Reply-To: <20200109132742.15828-1-olivier.matz@6wind.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Subject: Re: [dpdk-stable] [PATCH] mempool: fix slow allocation of large
mempools
X-BeenThere: stable@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: patches for DPDK stable branches <stable.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/stable>,
<mailto:stable-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/stable/>
List-Post: <mailto:stable@dpdk.org>
List-Help: <mailto:stable-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/stable>,
<mailto:stable-request@dpdk.org?subject=subscribe>
Errors-To: stable-bounces@dpdk.org
Sender: "stable" <stable-bounces@dpdk.org>
On 09-Jan-20 1:27 PM, Olivier Matz wrote:
> When allocating a mempool which is larger than the largest
> available area, it can take a lot of time:
>
> a- the mempool calculate the required memory size, and tries
> to allocate it, it fails
> b- then it tries to allocate the largest available area (this
> does not request new huge pages)
> c- add this zone to the mempool, this triggers the allocation
> of a mem hdr, which request a new huge page
> d- back to a- until mempool is populated or until there is no
> more memory
>
> This can take a lot of time to finally fail (several minutes): in step
> a- it takes all available hugepages on the system, then release them
> after it fails.
>
> The problem appeared with commit eba11e364614 ("mempool: reduce wasted
> space on populate"), because smaller chunks are now allowed. Previously,
> it had to be at least one page size, which is not the case in step b-.
>
> To fix this, implement our own way to allocate the largest available
> area instead of using the feature from memzone: if an allocation fails,
> try to divide the size by 2 and retry. When the requested size falls
> below min_chunk_size, stop and return an error.
>
> Fixes: eba11e364614 ("mempool: reduce wasted space on populate")
> Cc: stable@dpdk.org
>
> Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
> ---
I don't particularly like the idea of working around this issue as
opposed to fixing it memzone-side, but since there's currently no plan
to address this in memzone allocator, this should work much better than
before.
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
--
Thanks,
Anatoly