From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0E6A9A0517; Tue, 9 Jun 2020 14:46:32 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 8475A1F1C; Tue, 9 Jun 2020 14:46:31 +0200 (CEST) Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by dpdk.org (Postfix) with ESMTP id 0301C1E34 for ; Tue, 9 Jun 2020 14:46:29 +0200 (CEST) IronPort-SDR: fCIKqmdwdNsnFeR8Jyd9gjgWXVauCdqYt0xX10vS2vErB1EKwS5P1R56w4HluldC2TjY7y81kO IN+jxfhq25fw== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Jun 2020 05:46:28 -0700 IronPort-SDR: xmXAv0dv8u5JWmDZvp1AbYmasqA+JgMm67nHZwL5qF86/wmkXGZCSJqjiZmVvopO08VC7nq0G7 43KOjsw0gkEQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,492,1583222400"; d="scan'208";a="288818846" Received: from aburakov-mobl.ger.corp.intel.com (HELO [10.213.235.42]) ([10.213.235.42]) by orsmga002.jf.intel.com with ESMTP; 09 Jun 2020 05:46:28 -0700 To: Francesco , dev@dpdk.org References: From: "Burakov, Anatoly" Message-ID: Date: Tue, 9 Jun 2020 13:46:27 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.8.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] very high VIRT memory usage X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 08-Jun-20 12:03 PM, Francesco wrote: > Hi all, > I upgraded an old DPDK-based app which was using DPDK 17.11 to latest DPDK > 20.05 and I noticed that if I look at "top" I see that the VIRT memory > taken by my application is now 256.1GB while before it was <1GB. > > I've seen this same behavior with also "testpmd" example... is this a known > issue with latest DPDK versions? > Can I tweak some setting to have VIRT memory usage more or less similar to > RSS ? > > I forgot to add I'm working on Linux, Centos7 > > Thanks, > Francesco Montorsi > There was a discussion on this not too long ago, but i can't seem to find it for some reason. Anyway, long story short, that's not a bug, that's by design. Since 18.11 (or 18.05 to be precise), there is a new memory subsystem in DPDK that allows growing and shrinking DPDK memory usage at runtime. That means, you can start with zero hugepages preallocated, and then allocate as you go, letting the memory subsystem decide how much memory you need. The catch is that all of this hugepage memory is allocated into somewhere, some virtual address space. And *that* address space is preallocated at startup, to allow for secondary processes to duplicate primary process's address space exactly, and allow dynamic allocation of *shared* memory at runtime. This memory will show up in top et al. but the truth is, it's zero cost, because it's anonymous memory. It isn't actually taking up any RAM. It will show up in dumps (20.05 has already fixed that issue, and the fixes will probably be backported to stable, including 18.11), so unless you have a very specific problem, i don't think that's anything you should be concerned about. -- Thanks, Anatoly