From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 69CACA0517; Wed, 10 Jun 2020 11:24:10 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 9B9722A5D; Wed, 10 Jun 2020 11:24:09 +0200 (CEST) Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by dpdk.org (Postfix) with ESMTP id DBBFD1DB8 for ; Wed, 10 Jun 2020 11:24:07 +0200 (CEST) IronPort-SDR: fdJlPdw6OzlSny0RXYDD5kspH+Z8i9KwtzVWdNmz9pgeNJrq71BvtoEMh24aeqwmDqGuKq7IMb 2xxWGs3qyW7w== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Jun 2020 02:24:06 -0700 IronPort-SDR: f95wg7Xo9Gk4OIGKT5NHj94G4Swaw9C8XXZqVT0SBcx74xhHvmv0dciu7qXBceNIYOJT8Yvktp rbPLqBVLE/CQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,495,1583222400"; d="scan'208";a="296144803" Received: from bdoole1-mobl.ger.corp.intel.com (HELO [10.213.200.247]) ([10.213.200.247]) by fmsmga004.fm.intel.com with ESMTP; 10 Jun 2020 02:24:05 -0700 To: Francesco Cc: dev@dpdk.org References: From: "Burakov, Anatoly" Message-ID: Date: Wed, 10 Jun 2020 10:24:04 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.8.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit Subject: Re: [dpdk-dev] very high VIRT memory usage X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 09-Jun-20 8:40 PM, Francesco wrote: > Hi Anatoly, > > Thanks a lot for the detailed response! > Good to know anyway there's a "fix" already done in 20.05... also > because I'm not interested in supporting secondary processes or having > shared memory... > > Looking forward for the backports in stable branches then! > > Thanks! > Francesco > Hi Francesco, Just to be clear - the "fix" i'm talking about is not about using less memory - it's about not including this memory in core dumps. The memory amounts used will stay the same (i.e. you'll still see the ~256GB used each time you run DPDK). > > Il mar 9 giu 2020, 14:46 Burakov, Anatoly > ha scritto: > > On 08-Jun-20 12:03 PM, Francesco wrote: > > Hi all, > > I upgraded an old DPDK-based app which was using DPDK 17.11 to > latest DPDK > > 20.05 and I noticed that if I look  at "top" I see that the VIRT > memory > > taken by my application is now 256.1GB while before it was <1GB. > > > > I've seen this same behavior with also "testpmd" example... is > this a known > > issue with latest DPDK versions? > > Can I tweak some setting to have VIRT memory usage more or less > similar to > > RSS ? > > > > I forgot to add I'm working on Linux, Centos7 > > > > Thanks, > > Francesco Montorsi > > > > There was a discussion on this not too long ago, but i can't seem to > find it for some reason. Anyway, long story short, that's not a bug, > that's by design. > > Since 18.11 (or 18.05 to be precise), there is a new memory > subsystem in > DPDK that allows growing and shrinking DPDK memory usage at runtime. > That means, you can start with zero hugepages preallocated, and then > allocate as you go, letting the memory subsystem decide how much memory > you need. > > The catch is that all of this hugepage memory is allocated into > somewhere, some virtual address space. And *that* address space is > preallocated at startup, to allow for secondary processes to duplicate > primary process's address space exactly, and allow dynamic > allocation of > *shared* memory at runtime. > > This memory will show up in top et al. but the truth is, it's zero > cost, > because it's anonymous memory. It isn't actually taking up any RAM. It > will show up in dumps (20.05 has already fixed that issue, and the > fixes > will probably be backported to stable, including 18.11), so unless you > have a very specific problem, i don't think that's anything you should > be concerned about. > > -- > Thanks, > Anatoly > -- Thanks, Anatoly