From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id B5781A0531; Tue, 4 Feb 2020 11:55:16 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 1C8761C014; Tue, 4 Feb 2020 11:55:16 +0100 (CET) Received: from mail-pj1-f47.google.com (mail-pj1-f47.google.com [209.85.216.47]) by dpdk.org (Postfix) with ESMTP id D50341C00F for ; Tue, 4 Feb 2020 11:55:13 +0100 (CET) Received: by mail-pj1-f47.google.com with SMTP id fa20so1203305pjb.1 for ; Tue, 04 Feb 2020 02:55:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=zl2bzmq1CuO7gwGrWS2AlCOq0yGGeSZ5+IrA/zDHRlM=; b=JKgfOXQ/J4Z5Yd/iWpx98lKoW5ysY3VirnW2pbkh3Zlp7KJSc8jZvbRgMoXlS0q6O5 IwwfYzCHlpBRe7ZG7R8dx80goXLPafIeT4Pu7u/FVD1mtKq56/27AEC+xW3RrGODw0Pc vZej3iMqi2JDWfPaVhAESXApx3ysGUNHiLLBT3lZGGnoWnxW6p9tmwizlY21Yi/52RSO 63P0txbpghiJTS92+gGAfCqT7KTxgWUJERUVvSbjkwTjptnBhgcCidTYyHpCz+GoLido UB0PHRwt2SPNJ7JvvOBJjfxv4Jaf5sqWzwpZwcOACbi2q7HEHMPO52nQuiGvScVL5OwK 0cfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=zl2bzmq1CuO7gwGrWS2AlCOq0yGGeSZ5+IrA/zDHRlM=; b=amV7Mo2mxsdiAEwd7iHUnWtgZZY06yhGFWzkq/f/NW9il7OM3TBxSULKO1QR8Yew1T iVrk2WBMVmIg/MlbXTGhcvn29EExQBABbWGeWOf0D+06xBam0cl3PaalvSVcHS8h8yQW 7HblsmiExA7q8zmeX/7KIZzjOXe08BvDiMwKbeJBN+QqN2u5MBAjlthUnvFve8yUtIej fmVq/MWkrV/Qu0ZfvwX9dRN4T+sixJnYxkA8sG1JxSrh1W2Y8LUeS8trzvvB+oU4sQpO s1Y/1j8T+w+Vo0bETAUJz6LoTxqpwWzss1kwLr2KGv+7ki3sTKDY42V0NTrxH0EFvfYu VmLw== X-Gm-Message-State: APjAAAU/w7aE4HFfAv3K4fe2V73WQU4gtUi6WI38hcmQPk4PsZjazuPe ttS5obsvxV2MSdkHqA4eA+DAg/+53uRbxl9nhk5/hE2n X-Google-Smtp-Source: APXvYqyXaIAB5yEDHSbCaPH0xJBBr3EXN9gWH8LuTll4FQosBAzBNFO7DsjOrK9NMhJo+OV0hY0rErTNaCEF3rIPqE0= X-Received: by 2002:a17:902:9a86:: with SMTP id w6mr28365159plp.37.1580813712758; Tue, 04 Feb 2020 02:55:12 -0800 (PST) MIME-Version: 1.0 References: <6cebb805-91a3-c074-2380-8ec90ed6c132@intel.com> In-Reply-To: <6cebb805-91a3-c074-2380-8ec90ed6c132@intel.com> From: siddarth rai Date: Tue, 4 Feb 2020 16:25:02 +0530 Message-ID: To: "Burakov, Anatoly" Cc: David Marchand , dev Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] Big spike in DPDK VSZ X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Anatoly, I don't need a secondary process. I tried out Julien's suggestion and set the param 'RTE_MAX_MEM_MB' value to 8192 (the original value was over 500K). This works as a cap. The virtual size dropped down to less than 8G. So this seems to be working for me. I have a few queries/concerns though. Is it safe to reduce the RTE_MAX_MEM_MB to such a low value ? Can I reduce it further ? What will be the impact of doing so ? Will it limit the maximum size of mbuf pool which I create ? Regards, Siddarth On Tue, Feb 4, 2020 at 3:53 PM Burakov, Anatoly wrote: > On 30-Jan-20 8:51 AM, David Marchand wrote: > > On Thu, Jan 30, 2020 at 8:48 AM siddarth rai wrote: > >> I have been using DPDK 19.08 and I notice the process VSZ is huge. > >> > >> I tried running the test PMD. It takes 64G VSZ and if I use the > >> '--in-memory' option it takes up to 188G. > >> > >> Is there anyway to disable allocation of such huge VSZ in DPDK ? > > > > *Disclaimer* I don't know the arcanes of the mem subsystem. > > > > I suppose this is due to the memory allocator in dpdk that reserves > > unused virtual space (for memory hotplug + multiprocess). > > Yes, that's correct. In order to guarantee memory reservation succeeding > at all times, we need to reserve all possible memory in advance. > Otherwise we may end up in a situation where primary process has > allocated a page, but the secondary can't map it because the address > space is already occupied by something else. > > > > > If this is the case, maybe we could do something to enhance the > > situation for applications that won't care about multiprocess. > > Like inform dpdk that the application won't use multiprocess and skip > > those reservations. > > You're welcome to try this, but i assure you, avoiding these > reservations is a lot of work, because you'd be adding a yet another > path to an already overly complex allocator :) > > > > > Or another idea would be to limit those reservations to what is passed > > via --socket-limit. > > > > Anatoly? > > I have a patchset in the works that does this and was planning to submit > it to 19.08, but things got in the way and it's still sitting there > collecting bit rot. This may be reason enough to resurrect it and finish > it up :) > > > > > > > > > -- > > David Marchand > > > > > -- > Thanks, > Anatoly >