From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9B748A0517; Tue, 9 Jun 2020 15:39:59 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 00FA737AF; Tue, 9 Jun 2020 15:39:59 +0200 (CEST) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id 134BE1E34 for ; Tue, 9 Jun 2020 15:39:56 +0200 (CEST) IronPort-SDR: BtQOhII7jVzZ5INghRJlHGEZYjdg5IKoOD/snIqJG9thDFgnYu6YKapEO2bQyfH1K780rxo5if AnqW+RcfjxHA== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Jun 2020 06:39:55 -0700 IronPort-SDR: dexzxIkSda4fFIJ1XjW/JrssM9SqNSOgufZMRkB3iRSuk1wr2LVAfFWt9/lOfplQdyb4v+krO2 4Dm5jTxXzNkA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,492,1583222400"; d="scan'208";a="288833725" Received: from aburakov-mobl.ger.corp.intel.com (HELO [10.213.235.42]) ([10.213.235.42]) by orsmga002.jf.intel.com with ESMTP; 09 Jun 2020 06:39:54 -0700 To: Ferruh Yigit , Francesco , dev@dpdk.org References: <575e70e0-0e56-ed43-0411-df6215b1758f@intel.com> From: "Burakov, Anatoly" Message-ID: Date: Tue, 9 Jun 2020 14:39:54 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.8.1 MIME-Version: 1.0 In-Reply-To: <575e70e0-0e56-ed43-0411-df6215b1758f@intel.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] very high VIRT memory usage X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 09-Jun-20 2:13 PM, Ferruh Yigit wrote: > On 6/9/2020 1:46 PM, Burakov, Anatoly wrote: >> On 08-Jun-20 12:03 PM, Francesco wrote: >>> Hi all, >>> I upgraded an old DPDK-based app which was using DPDK 17.11 to latest DPDK >>> 20.05 and I noticed that if I look at "top" I see that the VIRT memory >>> taken by my application is now 256.1GB while before it was <1GB. >>> >>> I've seen this same behavior with also "testpmd" example... is this a known >>> issue with latest DPDK versions? >>> Can I tweak some setting to have VIRT memory usage more or less similar to >>> RSS ? >>> >>> I forgot to add I'm working on Linux, Centos7 >>> >>> Thanks, >>> Francesco Montorsi >>> >> >> There was a discussion on this not too long ago, but i can't seem to >> find it for some reason. > > Can it be "Big spike in DPDK VSZ" ? > http://inbox.dpdk.org/dev/CAGxAMwD6Wtfi=C2Txwjfk0zhFvRzeqBu7mFfE8ayh=EJi2aU-A@mail.gmail.com/#t > Yes, that's the one, thanks Ferruh :) >> Anyway, long story short, that's not a bug, >> that's by design. >> >> Since 18.11 (or 18.05 to be precise), there is a new memory subsystem in >> DPDK that allows growing and shrinking DPDK memory usage at runtime. >> That means, you can start with zero hugepages preallocated, and then >> allocate as you go, letting the memory subsystem decide how much memory >> you need. >> >> The catch is that all of this hugepage memory is allocated into >> somewhere, some virtual address space. And *that* address space is >> preallocated at startup, to allow for secondary processes to duplicate >> primary process's address space exactly, and allow dynamic allocation of >> *shared* memory at runtime. >> >> This memory will show up in top et al. but the truth is, it's zero cost, >> because it's anonymous memory. It isn't actually taking up any RAM. It >> will show up in dumps (20.05 has already fixed that issue, and the fixes >> will probably be backported to stable, including 18.11), so unless you >> have a very specific problem, i don't think that's anything you should >> be concerned about. >> > -- Thanks, Anatoly