From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9179CA04FE; Tue, 9 Jun 2020 21:40:30 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A38C41DB9; Tue, 9 Jun 2020 21:40:29 +0200 (CEST) Received: from mail-io1-f65.google.com (mail-io1-f65.google.com [209.85.166.65]) by dpdk.org (Postfix) with ESMTP id 35C9C1B53 for ; Tue, 9 Jun 2020 21:40:29 +0200 (CEST) Received: by mail-io1-f65.google.com with SMTP id c8so24175782iob.6 for ; Tue, 09 Jun 2020 12:40:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=ha/v3Q8RaTIcJOR6n0oqpn6/VxeRywyJrt0u2g0W9P0=; b=cAfKD5zOyOEW6YKOO4x8KKlDCv91uMDfelHc+XFa4ws+HkWayURWfOgZcvs0exLDnk jbSeI1d+mfB7efm0w7h+QtTii/h3KIVc5/CK1czqMs4oFK2ppuxpdhgyiJnDtNrIX2qp sNc6U5zbMaWP+cQzl+LlF5t0AojFa9uD5AxbelRlZT3wiCGx/WzGaMjG1fGpjs/TKISL WYyTkyolQUqOVsZu26H/MhQynQEvY64cqAH7fjA/dncZ+kWnMvKJ7u5Q0DU8JcmBSzKE 3ob1HJ4oG0wgO9EquZp/cIRVq12GzN3Mosf6oAA+ydmPdR5FjSCckPkwi+SWkW/6+oRN GvQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=ha/v3Q8RaTIcJOR6n0oqpn6/VxeRywyJrt0u2g0W9P0=; b=BaFP1erckmlrdipnQf7Wio8gpwG0WscpTxGKCsaXKu/+GiFTBhenOh6Ofqt6TW0NS8 zHkBOlfdnrHhfpdkKEsCRXLPPPLnGYOUcmu31i/BjEHSkJroTSI/uT4/neeZNOxxvPNx OBQYuoJXmKVXbvK9LH6aRnip6Z4EX1AALDeZi8yJDB4xlcBsMDX+kzDreQiZeEywsWbL rrFm4TTDdSTSTXlqQG2w/fnmJEaV0NUrN7Glx5rpMBc9oVOVi2ig89Prfz/YILRb92tY OYvEulCAN5ggGlkPJp5i/wUFt+4HinSQxCKT1erCeIeDlFXi62Aj0vKtNI4QNZeGl8mA CoXA== X-Gm-Message-State: AOAM533PdKg0VpKF9KuGMD/Pi6b9UmPZijW3gz0/C47JJvLLGYzTWgZw H73ABnp5wwVHg6a7a4XHkwPr16rLIPOJnWB22zE= X-Google-Smtp-Source: ABdhPJwca87I6nq5bWIjPdnwtXTdGUGByx9ZG6PwkBXUkZf5F1bOiS5m/YCk+a1J2fRA1ZKQwh+vcwFbd2enjWIrWeI= X-Received: by 2002:a02:998b:: with SMTP id a11mr28070142jal.117.1591731628418; Tue, 09 Jun 2020 12:40:28 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Francesco Date: Tue, 9 Jun 2020 21:40:17 +0200 Message-ID: To: "Burakov, Anatoly" Cc: dev@dpdk.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] very high VIRT memory usage X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Anatoly, Thanks a lot for the detailed response! Good to know anyway there's a "fix" already done in 20.05... also because I'm not interested in supporting secondary processes or having shared memory... Looking forward for the backports in stable branches then! Thanks! Francesco Il mar 9 giu 2020, 14:46 Burakov, Anatoly ha scritto: > On 08-Jun-20 12:03 PM, Francesco wrote: > > Hi all, > > I upgraded an old DPDK-based app which was using DPDK 17.11 to latest > DPDK > > 20.05 and I noticed that if I look at "top" I see that the VIRT memory > > taken by my application is now 256.1GB while before it was <1GB. > > > > I've seen this same behavior with also "testpmd" example... is this a > known > > issue with latest DPDK versions? > > Can I tweak some setting to have VIRT memory usage more or less similar > to > > RSS ? > > > > I forgot to add I'm working on Linux, Centos7 > > > > Thanks, > > Francesco Montorsi > > > > There was a discussion on this not too long ago, but i can't seem to > find it for some reason. Anyway, long story short, that's not a bug, > that's by design. > > Since 18.11 (or 18.05 to be precise), there is a new memory subsystem in > DPDK that allows growing and shrinking DPDK memory usage at runtime. > That means, you can start with zero hugepages preallocated, and then > allocate as you go, letting the memory subsystem decide how much memory > you need. > > The catch is that all of this hugepage memory is allocated into > somewhere, some virtual address space. And *that* address space is > preallocated at startup, to allow for secondary processes to duplicate > primary process's address space exactly, and allow dynamic allocation of > *shared* memory at runtime. > > This memory will show up in top et al. but the truth is, it's zero cost, > because it's anonymous memory. It isn't actually taking up any RAM. It > will show up in dumps (20.05 has already fixed that issue, and the fixes > will probably be backported to stable, including 18.11), so unless you > have a very specific problem, i don't think that's anything you should > be concerned about. > > -- > Thanks, > Anatoly >