[dpdk-dev] very high VIRT memory usage

[dpdk-dev] very high VIRT memory usage

[Francesco]

Hi all,
I upgraded an old DPDK-based app which was using DPDK 17.11 to latest DPDK
20.05 and I noticed that if I look  at "top" I see that the VIRT memory
taken by my application is now 256.1GB while before it was <1GB.

I've seen this same behavior with also "testpmd" example... is this a known
issue with latest DPDK versions?
Can I tweak some setting to have VIRT memory usage more or less similar to
RSS ?

I forgot to add I'm working on Linux, Centos7

Thanks,
Francesco Montorsi

Re: [dpdk-dev] very high VIRT memory usage

[Burakov, Anatoly]

On 08-Jun-20 12:03 PM, Francesco wrote:
> Hi all,
> I upgraded an old DPDK-based app which was using DPDK 17.11 to latest DPDK
> 20.05 and I noticed that if I look  at "top" I see that the VIRT memory
> taken by my application is now 256.1GB while before it was <1GB.
> 
> I've seen this same behavior with also "testpmd" example... is this a known
> issue with latest DPDK versions?
> Can I tweak some setting to have VIRT memory usage more or less similar to
> RSS ?
> 
> I forgot to add I'm working on Linux, Centos7
> 
> Thanks,
> Francesco Montorsi
> 

There was a discussion on this not too long ago, but i can't seem to 
find it for some reason. Anyway, long story short, that's not a bug, 
that's by design.

Since 18.11 (or 18.05 to be precise), there is a new memory subsystem in 
DPDK that allows growing and shrinking DPDK memory usage at runtime. 
That means, you can start with zero hugepages preallocated, and then 
allocate as you go, letting the memory subsystem decide how much memory 
you need.

The catch is that all of this hugepage memory is allocated into 
somewhere, some virtual address space. And *that* address space is 
preallocated at startup, to allow for secondary processes to duplicate 
primary process's address space exactly, and allow dynamic allocation of 
*shared* memory at runtime.

This memory will show up in top et al. but the truth is, it's zero cost, 
because it's anonymous memory. It isn't actually taking up any RAM. It 
will show up in dumps (20.05 has already fixed that issue, and the fixes 
will probably be backported to stable, including 18.11), so unless you 
have a very specific problem, i don't think that's anything you should 
be concerned about.

-- 
Thanks,
Anatoly

Re: [dpdk-dev] very high VIRT memory usage

[Ferruh Yigit]

On 6/9/2020 1:46 PM, Burakov, Anatoly wrote:
> On 08-Jun-20 12:03 PM, Francesco wrote:
>> Hi all,
>> I upgraded an old DPDK-based app which was using DPDK 17.11 to latest DPDK
>> 20.05 and I noticed that if I look  at "top" I see that the VIRT memory
>> taken by my application is now 256.1GB while before it was <1GB.
>>
>> I've seen this same behavior with also "testpmd" example... is this a known
>> issue with latest DPDK versions?
>> Can I tweak some setting to have VIRT memory usage more or less similar to
>> RSS ?
>>
>> I forgot to add I'm working on Linux, Centos7
>>
>> Thanks,
>> Francesco Montorsi
>>
> 
> There was a discussion on this not too long ago, but i can't seem to 
> find it for some reason. 

Can it be "Big spike in DPDK VSZ" ?
http://inbox.dpdk.org/dev/CAGxAMwD6Wtfi=C2Txwjfk0zhFvRzeqBu7mFfE8ayh=EJi2aU-A@mail.gmail.com/#t

> Anyway, long story short, that's not a bug, 
> that's by design.
> 
> Since 18.11 (or 18.05 to be precise), there is a new memory subsystem in 
> DPDK that allows growing and shrinking DPDK memory usage at runtime. 
> That means, you can start with zero hugepages preallocated, and then 
> allocate as you go, letting the memory subsystem decide how much memory 
> you need.
> 
> The catch is that all of this hugepage memory is allocated into 
> somewhere, some virtual address space. And *that* address space is 
> preallocated at startup, to allow for secondary processes to duplicate 
> primary process's address space exactly, and allow dynamic allocation of 
> *shared* memory at runtime.
> 
> This memory will show up in top et al. but the truth is, it's zero cost, 
> because it's anonymous memory. It isn't actually taking up any RAM. It 
> will show up in dumps (20.05 has already fixed that issue, and the fixes 
> will probably be backported to stable, including 18.11), so unless you 
> have a very specific problem, i don't think that's anything you should 
> be concerned about.
> 

Re: [dpdk-dev] very high VIRT memory usage

[Burakov, Anatoly]

On 09-Jun-20 2:13 PM, Ferruh Yigit wrote:
> On 6/9/2020 1:46 PM, Burakov, Anatoly wrote:
>> On 08-Jun-20 12:03 PM, Francesco wrote:
>>> Hi all,
>>> I upgraded an old DPDK-based app which was using DPDK 17.11 to latest DPDK
>>> 20.05 and I noticed that if I look  at "top" I see that the VIRT memory
>>> taken by my application is now 256.1GB while before it was <1GB.
>>>
>>> I've seen this same behavior with also "testpmd" example... is this a known
>>> issue with latest DPDK versions?
>>> Can I tweak some setting to have VIRT memory usage more or less similar to
>>> RSS ?
>>>
>>> I forgot to add I'm working on Linux, Centos7
>>>
>>> Thanks,
>>> Francesco Montorsi
>>>
>>
>> There was a discussion on this not too long ago, but i can't seem to
>> find it for some reason.
> 
> Can it be "Big spike in DPDK VSZ" ?
> http://inbox.dpdk.org/dev/CAGxAMwD6Wtfi=C2Txwjfk0zhFvRzeqBu7mFfE8ayh=EJi2aU-A@mail.gmail.com/#t
> 

Yes, that's the one, thanks Ferruh :)

>> Anyway, long story short, that's not a bug,
>> that's by design.
>>
>> Since 18.11 (or 18.05 to be precise), there is a new memory subsystem in
>> DPDK that allows growing and shrinking DPDK memory usage at runtime.
>> That means, you can start with zero hugepages preallocated, and then
>> allocate as you go, letting the memory subsystem decide how much memory
>> you need.
>>
>> The catch is that all of this hugepage memory is allocated into
>> somewhere, some virtual address space. And *that* address space is
>> preallocated at startup, to allow for secondary processes to duplicate
>> primary process's address space exactly, and allow dynamic allocation of
>> *shared* memory at runtime.
>>
>> This memory will show up in top et al. but the truth is, it's zero cost,
>> because it's anonymous memory. It isn't actually taking up any RAM. It
>> will show up in dumps (20.05 has already fixed that issue, and the fixes
>> will probably be backported to stable, including 18.11), so unless you
>> have a very specific problem, i don't think that's anything you should
>> be concerned about.
>>
> 


-- 
Thanks,
Anatoly

Re: [dpdk-dev] very high VIRT memory usage

[Stephen Hemminger]

On Tue, 9 Jun 2020 14:39:54 +0100
"Burakov, Anatoly" <anatoly.burakov@intel.com> wrote:

> On 09-Jun-20 2:13 PM, Ferruh Yigit wrote:
> > On 6/9/2020 1:46 PM, Burakov, Anatoly wrote:  
> >> On 08-Jun-20 12:03 PM, Francesco wrote:  
> >>> Hi all,
> >>> I upgraded an old DPDK-based app which was using DPDK 17.11 to latest DPDK
> >>> 20.05 and I noticed that if I look  at "top" I see that the VIRT memory
> >>> taken by my application is now 256.1GB while before it was <1GB.
> >>>
> >>> I've seen this same behavior with also "testpmd" example... is this a known
> >>> issue with latest DPDK versions?
> >>> Can I tweak some setting to have VIRT memory usage more or less similar to
> >>> RSS ?
> >>>
> >>> I forgot to add I'm working on Linux, Centos7
> >>>
> >>> Thanks,
> >>> Francesco Montorsi
> >>>  
> >>
> >> There was a discussion on this not too long ago, but i can't seem to
> >> find it for some reason.  
> > 
> > Can it be "Big spike in DPDK VSZ" ?
> > http://inbox.dpdk.org/dev/CAGxAMwD6Wtfi=C2Txwjfk0zhFvRzeqBu7mFfE8ayh=EJi2aU-A@mail.gmail.com/#t
> >   
> 
> Yes, that's the one, thanks Ferruh :)
> 
> >> Anyway, long story short, that's not a bug,
> >> that's by design.
> >>
> >> Since 18.11 (or 18.05 to be precise), there is a new memory subsystem in
> >> DPDK that allows growing and shrinking DPDK memory usage at runtime.
> >> That means, you can start with zero hugepages preallocated, and then
> >> allocate as you go, letting the memory subsystem decide how much memory
> >> you need.
> >>
> >> The catch is that all of this hugepage memory is allocated into
> >> somewhere, some virtual address space. And *that* address space is
> >> preallocated at startup, to allow for secondary processes to duplicate
> >> primary process's address space exactly, and allow dynamic allocation of
> >> *shared* memory at runtime.
> >>
> >> This memory will show up in top et al. but the truth is, it's zero cost,
> >> because it's anonymous memory. It isn't actually taking up any RAM. It
> >> will show up in dumps (20.05 has already fixed that issue, and the fixes
> >> will probably be backported to stable, including 18.11), so unless you
> >> have a very specific problem, i don't think that's anything you should
> >> be concerned about.

The one concern is for cases like cgroup memory accounting thinking
the process is huge and OOM killing it.

Re: [dpdk-dev] very high VIRT memory usage

[Francesco]

Hi Anatoly,

Thanks a lot for the detailed response!
Good to know anyway there's a "fix" already done in 20.05... also because
I'm not interested in supporting secondary processes or having shared
memory...

Looking forward for the backports in stable branches then!

Thanks!
Francesco


Il mar 9 giu 2020, 14:46 Burakov, Anatoly <anatoly.burakov@intel.com> ha
scritto:

> On 08-Jun-20 12:03 PM, Francesco wrote:
> > Hi all,
> > I upgraded an old DPDK-based app which was using DPDK 17.11 to latest
> DPDK
> > 20.05 and I noticed that if I look  at "top" I see that the VIRT memory
> > taken by my application is now 256.1GB while before it was <1GB.
> >
> > I've seen this same behavior with also "testpmd" example... is this a
> known
> > issue with latest DPDK versions?
> > Can I tweak some setting to have VIRT memory usage more or less similar
> to
> > RSS ?
> >
> > I forgot to add I'm working on Linux, Centos7
> >
> > Thanks,
> > Francesco Montorsi
> >
>
> There was a discussion on this not too long ago, but i can't seem to
> find it for some reason. Anyway, long story short, that's not a bug,
> that's by design.
>
> Since 18.11 (or 18.05 to be precise), there is a new memory subsystem in
> DPDK that allows growing and shrinking DPDK memory usage at runtime.
> That means, you can start with zero hugepages preallocated, and then
> allocate as you go, letting the memory subsystem decide how much memory
> you need.
>
> The catch is that all of this hugepage memory is allocated into
> somewhere, some virtual address space. And *that* address space is
> preallocated at startup, to allow for secondary processes to duplicate
> primary process's address space exactly, and allow dynamic allocation of
> *shared* memory at runtime.
>
> This memory will show up in top et al. but the truth is, it's zero cost,
> because it's anonymous memory. It isn't actually taking up any RAM. It
> will show up in dumps (20.05 has already fixed that issue, and the fixes
> will probably be backported to stable, including 18.11), so unless you
> have a very specific problem, i don't think that's anything you should
> be concerned about.
>
> --
> Thanks,
> Anatoly
>

Re: [dpdk-dev] very high VIRT memory usage

[Burakov, Anatoly]

On 09-Jun-20 8:40 PM, Francesco wrote:
> Hi Anatoly,
> 
> Thanks a lot for the detailed response!
> Good to know anyway there's a "fix" already done in 20.05... also 
> because I'm not interested in supporting secondary processes or having 
> shared memory...
> 
> Looking forward for the backports in stable branches then!
> 
> Thanks!
> Francesco
> 

Hi Francesco,

Just to be clear - the "fix" i'm talking about is not about using less 
memory - it's about not including this memory in core dumps. The memory 
amounts used will stay the same (i.e. you'll still see the ~256GB used 
each time you run DPDK).

> 
> Il mar 9 giu 2020, 14:46 Burakov, Anatoly <anatoly.burakov@intel.com 
> <mailto:anatoly.burakov@intel.com>> ha scritto:
> 
>     On 08-Jun-20 12:03 PM, Francesco wrote:
>      > Hi all,
>      > I upgraded an old DPDK-based app which was using DPDK 17.11 to
>     latest DPDK
>      > 20.05 and I noticed that if I look  at "top" I see that the VIRT
>     memory
>      > taken by my application is now 256.1GB while before it was <1GB.
>      >
>      > I've seen this same behavior with also "testpmd" example... is
>     this a known
>      > issue with latest DPDK versions?
>      > Can I tweak some setting to have VIRT memory usage more or less
>     similar to
>      > RSS ?
>      >
>      > I forgot to add I'm working on Linux, Centos7
>      >
>      > Thanks,
>      > Francesco Montorsi
>      >
> 
>     There was a discussion on this not too long ago, but i can't seem to
>     find it for some reason. Anyway, long story short, that's not a bug,
>     that's by design.
> 
>     Since 18.11 (or 18.05 to be precise), there is a new memory
>     subsystem in
>     DPDK that allows growing and shrinking DPDK memory usage at runtime.
>     That means, you can start with zero hugepages preallocated, and then
>     allocate as you go, letting the memory subsystem decide how much memory
>     you need.
> 
>     The catch is that all of this hugepage memory is allocated into
>     somewhere, some virtual address space. And *that* address space is
>     preallocated at startup, to allow for secondary processes to duplicate
>     primary process's address space exactly, and allow dynamic
>     allocation of
>     *shared* memory at runtime.
> 
>     This memory will show up in top et al. but the truth is, it's zero
>     cost,
>     because it's anonymous memory. It isn't actually taking up any RAM. It
>     will show up in dumps (20.05 has already fixed that issue, and the
>     fixes
>     will probably be backported to stable, including 18.11), so unless you
>     have a very specific problem, i don't think that's anything you should
>     be concerned about.
> 
>     -- 
>     Thanks,
>     Anatoly
> 


-- 
Thanks,
Anatoly

Re: [dpdk-dev] very high VIRT memory usage

[Burakov, Anatoly]

On 09-Jun-20 4:35 PM, Stephen Hemminger wrote:
> On Tue, 9 Jun 2020 14:39:54 +0100
> "Burakov, Anatoly" <anatoly.burakov@intel.com> wrote:
> 
>> On 09-Jun-20 2:13 PM, Ferruh Yigit wrote:
>>> On 6/9/2020 1:46 PM, Burakov, Anatoly wrote:
>>>> On 08-Jun-20 12:03 PM, Francesco wrote:
>>>>> Hi all,
>>>>> I upgraded an old DPDK-based app which was using DPDK 17.11 to latest DPDK
>>>>> 20.05 and I noticed that if I look  at "top" I see that the VIRT memory
>>>>> taken by my application is now 256.1GB while before it was <1GB.
>>>>>
>>>>> I've seen this same behavior with also "testpmd" example... is this a known
>>>>> issue with latest DPDK versions?
>>>>> Can I tweak some setting to have VIRT memory usage more or less similar to
>>>>> RSS ?
>>>>>
>>>>> I forgot to add I'm working on Linux, Centos7
>>>>>
>>>>> Thanks,
>>>>> Francesco Montorsi
>>>>>   
>>>>
>>>> There was a discussion on this not too long ago, but i can't seem to
>>>> find it for some reason.
>>>
>>> Can it be "Big spike in DPDK VSZ" ?
>>> http://inbox.dpdk.org/dev/CAGxAMwD6Wtfi=C2Txwjfk0zhFvRzeqBu7mFfE8ayh=EJi2aU-A@mail.gmail.com/#t
>>>    
>>
>> Yes, that's the one, thanks Ferruh :)
>>
>>>> Anyway, long story short, that's not a bug,
>>>> that's by design.
>>>>
>>>> Since 18.11 (or 18.05 to be precise), there is a new memory subsystem in
>>>> DPDK that allows growing and shrinking DPDK memory usage at runtime.
>>>> That means, you can start with zero hugepages preallocated, and then
>>>> allocate as you go, letting the memory subsystem decide how much memory
>>>> you need.
>>>>
>>>> The catch is that all of this hugepage memory is allocated into
>>>> somewhere, some virtual address space. And *that* address space is
>>>> preallocated at startup, to allow for secondary processes to duplicate
>>>> primary process's address space exactly, and allow dynamic allocation of
>>>> *shared* memory at runtime.
>>>>
>>>> This memory will show up in top et al. but the truth is, it's zero cost,
>>>> because it's anonymous memory. It isn't actually taking up any RAM. It
>>>> will show up in dumps (20.05 has already fixed that issue, and the fixes
>>>> will probably be backported to stable, including 18.11), so unless you
>>>> have a very specific problem, i don't think that's anything you should
>>>> be concerned about.
> 
> The one concern is for cases like cgroup memory accounting thinking
> the process is huge and OOM killing it.
> 

Is there any way to know the *actual* memory usage of the process (i.e. 
not including anonymous memory)?

-- 
Thanks,
Anatoly

Re: [dpdk-dev] very high VIRT memory usage

[Francesco]

Hi Anatoly

Il giorno mer 10 giu 2020 alle ore 11:24 Burakov, Anatoly <
anatoly.burakov@intel.com> ha scritto:

> On 09-Jun-20 8:40 PM, Francesco wrote:
> > Hi Anatoly,
> >
> > Thanks a lot for the detailed response!
> > Good to know anyway there's a "fix" already done in 20.05... also
> > because I'm not interested in supporting secondary processes or having
> > shared memory...
> >
> > Looking forward for the backports in stable branches then!
> >
> > Thanks!
> > Francesco
> >
>
> Hi Francesco,
>
> Just to be clear - the "fix" i'm talking about is not about using less
> memory - it's about not including this memory in core dumps. The memory
> amounts used will stay the same (i.e. you'll still see the ~256GB used
> each time you run DPDK).


Ouch ok I see.
My issue is that I have tools that look at the health of my server and will
report this high VIRT memory usage as anomalous - I guess I will have to
work around them some way.

Thanks for the clarification

Francesco

Re: [dpdk-dev] very high VIRT memory usage

[Burakov, Anatoly]

On 10-Jun-20 11:14 AM, Francesco wrote:
> Hi Anatoly
> 
> Il giorno mer 10 giu 2020 alle ore 11:24 Burakov, Anatoly 
> <anatoly.burakov@intel.com <mailto:anatoly.burakov@intel.com>> ha scritto:
> 
>     On 09-Jun-20 8:40 PM, Francesco wrote:
>      > Hi Anatoly,
>      >
>      > Thanks a lot for the detailed response!
>      > Good to know anyway there's a "fix" already done in 20.05... also
>      > because I'm not interested in supporting secondary processes or
>     having
>      > shared memory...
>      >
>      > Looking forward for the backports in stable branches then!
>      >
>      > Thanks!
>      > Francesco
>      >
> 
>     Hi Francesco,
> 
>     Just to be clear - the "fix" i'm talking about is not about using less
>     memory - it's about not including this memory in core dumps. The memory
>     amounts used will stay the same (i.e. you'll still see the ~256GB used
>     each time you run DPDK).
> 
> 
> Ouch ok I see.
> My issue is that I have tools that look at the health of my server and 
> will report this high VIRT memory usage as anomalous - I guess I will 
> have to work around them some way.
> 
> Thanks for the clarification
> 
> Francesco
> 

Yep. Like i said earlier, this is a design decision. I understand that 
not everyone wants or needs secondary process support, but we have to 
have defaults that cover the most amount of use cases. Plus, it make 
internals very complex if we had two different init (and runtime!) paths 
for DPDK with and without secondary process support. So, there's little 
that can be done about that, short of lowering that limit at compile 
time. You can use the CONFIG_RTE_MAX_MEM_MB and similar options in the 
DPDK config file, but if you got your DPDK from a distro, you're out of 
luck, i'm afraid.

-- 
Thanks,
Anatoly

Re: [dpdk-dev] very high VIRT memory usage

[Stephen Hemminger]

On Wed, 10 Jun 2020 10:24:44 +0100
"Burakov, Anatoly" <anatoly.burakov@intel.com> wrote:

> On 09-Jun-20 4:35 PM, Stephen Hemminger wrote:
> > On Tue, 9 Jun 2020 14:39:54 +0100
> > "Burakov, Anatoly" <anatoly.burakov@intel.com> wrote:
> >   
> >> On 09-Jun-20 2:13 PM, Ferruh Yigit wrote:  
> >>> On 6/9/2020 1:46 PM, Burakov, Anatoly wrote:  
> >>>> On 08-Jun-20 12:03 PM, Francesco wrote:  
> >>>>> Hi all,
> >>>>> I upgraded an old DPDK-based app which was using DPDK 17.11 to latest DPDK
> >>>>> 20.05 and I noticed that if I look  at "top" I see that the VIRT memory
> >>>>> taken by my application is now 256.1GB while before it was <1GB.
> >>>>>
> >>>>> I've seen this same behavior with also "testpmd" example... is this a known
> >>>>> issue with latest DPDK versions?
> >>>>> Can I tweak some setting to have VIRT memory usage more or less similar to
> >>>>> RSS ?
> >>>>>
> >>>>> I forgot to add I'm working on Linux, Centos7
> >>>>>
> >>>>> Thanks,
> >>>>> Francesco Montorsi
> >>>>>     
> >>>>
> >>>> There was a discussion on this not too long ago, but i can't seem to
> >>>> find it for some reason.  
> >>>
> >>> Can it be "Big spike in DPDK VSZ" ?
> >>> http://inbox.dpdk.org/dev/CAGxAMwD6Wtfi=C2Txwjfk0zhFvRzeqBu7mFfE8ayh=EJi2aU-A@mail.gmail.com/#t
> >>>      
> >>
> >> Yes, that's the one, thanks Ferruh :)
> >>  
> >>>> Anyway, long story short, that's not a bug,
> >>>> that's by design.
> >>>>
> >>>> Since 18.11 (or 18.05 to be precise), there is a new memory subsystem in
> >>>> DPDK that allows growing and shrinking DPDK memory usage at runtime.
> >>>> That means, you can start with zero hugepages preallocated, and then
> >>>> allocate as you go, letting the memory subsystem decide how much memory
> >>>> you need.
> >>>>
> >>>> The catch is that all of this hugepage memory is allocated into
> >>>> somewhere, some virtual address space. And *that* address space is
> >>>> preallocated at startup, to allow for secondary processes to duplicate
> >>>> primary process's address space exactly, and allow dynamic allocation of
> >>>> *shared* memory at runtime.
> >>>>
> >>>> This memory will show up in top et al. but the truth is, it's zero cost,
> >>>> because it's anonymous memory. It isn't actually taking up any RAM. It
> >>>> will show up in dumps (20.05 has already fixed that issue, and the fixes
> >>>> will probably be backported to stable, including 18.11), so unless you
> >>>> have a very specific problem, i don't think that's anything you should
> >>>> be concerned about.  
> > 
> > The one concern is for cases like cgroup memory accounting thinking
> > the process is huge and OOM killing it.
> >   
> 
> Is there any way to know the *actual* memory usage of the process (i.e. 
> not including anonymous memory)?
> 

Huge pages do not count against the normal memory in cgroup.
There is a separate hugeTLB controller that limits that.

Re: [dpdk-dev] very high VIRT memory usage

[Burakov, Anatoly]

On 10-Jun-20 4:28 PM, Stephen Hemminger wrote:
> On Wed, 10 Jun 2020 10:24:44 +0100
> "Burakov, Anatoly" <anatoly.burakov@intel.com> wrote:
> 
>> On 09-Jun-20 4:35 PM, Stephen Hemminger wrote:
>>> On Tue, 9 Jun 2020 14:39:54 +0100
>>> "Burakov, Anatoly" <anatoly.burakov@intel.com> wrote:
>>>    
>>>> On 09-Jun-20 2:13 PM, Ferruh Yigit wrote:
>>>>> On 6/9/2020 1:46 PM, Burakov, Anatoly wrote:
>>>>>> On 08-Jun-20 12:03 PM, Francesco wrote:
>>>>>>> Hi all,
>>>>>>> I upgraded an old DPDK-based app which was using DPDK 17.11 to latest DPDK
>>>>>>> 20.05 and I noticed that if I look  at "top" I see that the VIRT memory
>>>>>>> taken by my application is now 256.1GB while before it was <1GB.
>>>>>>>
>>>>>>> I've seen this same behavior with also "testpmd" example... is this a known
>>>>>>> issue with latest DPDK versions?
>>>>>>> Can I tweak some setting to have VIRT memory usage more or less similar to
>>>>>>> RSS ?
>>>>>>>
>>>>>>> I forgot to add I'm working on Linux, Centos7
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Francesco Montorsi
>>>>>>>      
>>>>>>
>>>>>> There was a discussion on this not too long ago, but i can't seem to
>>>>>> find it for some reason.
>>>>>
>>>>> Can it be "Big spike in DPDK VSZ" ?
>>>>> http://inbox.dpdk.org/dev/CAGxAMwD6Wtfi=C2Txwjfk0zhFvRzeqBu7mFfE8ayh=EJi2aU-A@mail.gmail.com/#t
>>>>>       
>>>>
>>>> Yes, that's the one, thanks Ferruh :)
>>>>   
>>>>>> Anyway, long story short, that's not a bug,
>>>>>> that's by design.
>>>>>>
>>>>>> Since 18.11 (or 18.05 to be precise), there is a new memory subsystem in
>>>>>> DPDK that allows growing and shrinking DPDK memory usage at runtime.
>>>>>> That means, you can start with zero hugepages preallocated, and then
>>>>>> allocate as you go, letting the memory subsystem decide how much memory
>>>>>> you need.
>>>>>>
>>>>>> The catch is that all of this hugepage memory is allocated into
>>>>>> somewhere, some virtual address space. And *that* address space is
>>>>>> preallocated at startup, to allow for secondary processes to duplicate
>>>>>> primary process's address space exactly, and allow dynamic allocation of
>>>>>> *shared* memory at runtime.
>>>>>>
>>>>>> This memory will show up in top et al. but the truth is, it's zero cost,
>>>>>> because it's anonymous memory. It isn't actually taking up any RAM. It
>>>>>> will show up in dumps (20.05 has already fixed that issue, and the fixes
>>>>>> will probably be backported to stable, including 18.11), so unless you
>>>>>> have a very specific problem, i don't think that's anything you should
>>>>>> be concerned about.
>>>
>>> The one concern is for cases like cgroup memory accounting thinking
>>> the process is huge and OOM killing it.
>>>    
>>
>> Is there any way to know the *actual* memory usage of the process (i.e.
>> not including anonymous memory)?
>>
> 
> Huge pages do not count against the normal memory in cgroup.
> There is a separate hugeTLB controller that limits that.
> 

But hugepages only get mapped when they're required - the rest of the 
memory is mapped anonymously with PROT_NONE. Would that count against 
the limits?

-- 
Thanks,
Anatoly