From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id D574CA0528; Mon, 20 Jan 2020 11:12:40 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 26C8A2C6A; Mon, 20 Jan 2020 11:12:40 +0100 (CET) Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) by dpdk.org (Postfix) with ESMTP id 4593F2C2B; Mon, 20 Jan 2020 11:12:39 +0100 (CET) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id C82CA21EB2; Mon, 20 Jan 2020 05:12:38 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Mon, 20 Jan 2020 05:12:38 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s=mesmtp; bh=QwXX4n5yuT/S1DKNPX7DQFOJ2uPO6GvEUzamZv4tD4s=; b=ju06fwVzBWaZ fl3nwN8opEQ3EwTED1+2vl7hrGH66IdoVGUdUeErBYrUSrHaywObms3M1kwYBnZO r+OXJm+hDY4S6aIPuYSMtgK/vJ6iPcXBl7mvoVudEvIiOX9V7w/X4+Mohdr4dxWh 5DGrcmHWEWr+LYG4OMkMB4j6Tr0vFcI= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; bh=QwXX4n5yuT/S1DKNPX7DQFOJ2uPO6GvEUzamZv4tD 4s=; b=LMjnWQjtRP165n9KaOWGBOw4HEkF9UhdNr8sFAdSb6ENfGqk8cxyNXWLQ RExLwHq3r7yYQ7FPHVELj11jaJ9BkqGj4mhPf7sD4orHEFLsLxqgH+g7QtDelmP1 3Nx2I9s3sRVfkL2LAcJNE4cx8drWI/40PClj227CfaArJJRUmtqzYtVg0L6xPQfz RgWKbnu4bchhQS1Av9YX8r3pObOmhGbsmOj4wYJdPdeBhCludkBnV6sMrSSkOmy0 uOCJ4AYpAPQEL33SiizYBtqUfmFvChQTCfcjnsYIc+DTIK+VHlNP/V2NJdqxJNYM pjTzj/HD84+u/MeGL1PnNMU5SOZYA== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedugedrudehgddtlecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvffufffkjghfggfgtgesthfuredttddtvdenucfhrhhomhepvfhhohhmrghs ucfoohhnjhgrlhhonhcuoehthhhomhgrshesmhhonhhjrghlohhnrdhnvghtqeenucffoh hmrghinhepughpughkrdhorhhgnecukfhppeejjedrudefgedrvddtfedrudekgeenucfr rghrrghmpehmrghilhhfrhhomhepthhhohhmrghssehmohhnjhgrlhhonhdrnhgvthenuc evlhhushhtvghrufhiiigvpedt X-ME-Proxy: Received: from xps.localnet (184.203.134.77.rev.sfr.net [77.134.203.184]) by mail.messagingengine.com (Postfix) with ESMTPA id C103F80061; Mon, 20 Jan 2020 05:12:36 -0500 (EST) From: Thomas Monjalon To: Olivier Matz Cc: dev@dpdk.org, Ali Alnubani , Anatoly Burakov , Raslan Darawsheh , stable@dpdk.org, Andrew Rybchenko Date: Mon, 20 Jan 2020 11:12:35 +0100 Message-ID: <1863691.ZhSLo9btvA@xps> In-Reply-To: References: <20200109132742.15828-1-olivier.matz@6wind.com> <20200117100117.GX22738@platinum> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" Subject: Re: [dpdk-dev] [PATCH v2] mempool: fix slow allocation of large mempools X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" 17/01/2020 11:09, Andrew Rybchenko: > On 1/17/20 1:01 PM, Olivier Matz wrote: > > On Fri, Jan 17, 2020 at 10:51:49AM +0100, Olivier Matz wrote: > >> When allocating a mempool which is larger than the largest > >> available area, it can take a lot of time: > >> > >> a- the mempool calculate the required memory size, and tries > >> to allocate it, it fails > >> b- then it tries to allocate the largest available area (this > >> does not request new huge pages) > >> c- add this zone to the mempool, this triggers the allocation > >> of a mem hdr, which request a new huge page > >> d- back to a- until mempool is populated or until there is no > >> more memory > >> > >> This can take a lot of time to finally fail (several minutes): in step > >> a- it takes all available hugepages on the system, then release them > >> after it fails. > >> > >> The problem appeared with commit eba11e364614 ("mempool: reduce wasted > >> space on populate"), because smaller chunks are now allowed. Previously, > >> it had to be at least one page size, which is not the case in step b-. > >> > >> To fix this, implement our own way to allocate the largest available > >> area instead of using the feature from memzone: if an allocation fails, > >> try to divide the size by 2 and retry. When the requested size falls > >> below min_chunk_size, stop and return an error. > >> > >> Fixes: eba11e364614 ("mempool: reduce wasted space on populate") > >> Cc: stable@dpdk.org > >> > >> Signed-off-by: Olivier Matz > > > > Sorry I forgot to report Anatoly's ack on v1 > > http://patchwork.dpdk.org/patch/64370/ > > > > Acked-by: Anatoly Burakov > > Reviewed-by: Andrew Rybchenko Applied, thanks