DPDK patches and discussions
 help / color / mirror / Atom feed
* [dpdk-dev] Zeroing out memory on free
@ 2021-08-24 10:58 Dmitry Kozlyuk
  2021-08-25  7:26 ` Thomas Monjalon
  0 siblings, 1 reply; 4+ messages in thread
From: Dmitry Kozlyuk @ 2021-08-24 10:58 UTC (permalink / raw)
  To: dev, Anatoly Burakov; +Cc: Xueming(Steven) Li, Thomas Monjalon

Hello,

Me and Xueming are wondering why DPDK clears the memory on free
and not only when it's explicitly requested (rte_zmalloc).

It's been so for a while:

commit ea0bddbd14e68fb42d9774bc3543e51b510e48d3
Author: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
Date:   Tue Jul 5 12:01:15 2016 +0100

    mem: zero out memory on free
    
    Since commit fafcc11985a2, memzones are not guaranteed to be zeroed out.
    This could potentially cause issues as applications might have been
    relying on the allocated memory being zeroed out.
    
    On init all allocated memory is zeroed by the kernel, so by zeroing out
    memory on free, all available dpdk memory is always zeroed.
    
    Fixes: fafcc11985a2 ("mem: rework memzone to be allocated by malloc")
    
    Signed-off-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>

At present this explanation doesn't look satisfying:
1. Memzone API does not promise they will be zeroed out.
   Memzones are mostly used for DMA, so their content will likely be overwritten.
2. If application assumptions are wrong, DPDK should not try to fix it.
   Memory manager poisons memory in debug mode to detect such errors.

Zeroing memory is quite slow, but in many cases unneeded.
It looks attractive to only do it in rte_zmalloc().
AFAIK what prohibits moving memset() there unconditionally
is that the kernel already gives us cleared pages,
so the first allocation of each piece of memory would do unnecessary work.
This can be solved by giving the user a choice in EAL options
or with more elaborate tracking of memory dirtiness in MM.
Are there any other reasons why clearing is done the current way?

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [dpdk-dev] Zeroing out memory on free
  2021-08-24 10:58 [dpdk-dev] Zeroing out memory on free Dmitry Kozlyuk
@ 2021-08-25  7:26 ` Thomas Monjalon
  2021-08-25 11:47   ` Burakov, Anatoly
  0 siblings, 1 reply; 4+ messages in thread
From: Thomas Monjalon @ 2021-08-25  7:26 UTC (permalink / raw)
  To: Dmitry Kozlyuk
  Cc: dev, Anatoly Burakov, Xueming(Steven) Li, david.marchand,
	bruce.richardson

24/08/2021 12:58, Dmitry Kozlyuk:
> Hello,
> 
> Me and Xueming are wondering why DPDK clears the memory on free
> and not only when it's explicitly requested (rte_zmalloc).
> 
> It's been so for a while:
> 
> commit ea0bddbd14e68fb42d9774bc3543e51b510e48d3
> Author: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
> Date:   Tue Jul 5 12:01:15 2016 +0100
> 
>     mem: zero out memory on free
>     
>     Since commit fafcc11985a2, memzones are not guaranteed to be zeroed out.
>     This could potentially cause issues as applications might have been
>     relying on the allocated memory being zeroed out.
>     
>     On init all allocated memory is zeroed by the kernel, so by zeroing out
>     memory on free, all available dpdk memory is always zeroed.
>     
>     Fixes: fafcc11985a2 ("mem: rework memzone to be allocated by malloc")
>     
>     Signed-off-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
> 
> At present this explanation doesn't look satisfying:
> 1. Memzone API does not promise they will be zeroed out.
>    Memzones are mostly used for DMA, so their content will likely be overwritten.
> 2. If application assumptions are wrong, DPDK should not try to fix it.
>    Memory manager poisons memory in debug mode to detect such errors.
> 
> Zeroing memory is quite slow, but in many cases unneeded.
> It looks attractive to only do it in rte_zmalloc().
> AFAIK what prohibits moving memset() there unconditionally
> is that the kernel already gives us cleared pages,
> so the first allocation of each piece of memory would do unnecessary work.
> This can be solved by giving the user a choice in EAL options

No I don't think it should be workarounded in EAL options.

> or with more elaborate tracking of memory dirtiness in MM.

Looks a good idea.

> Are there any other reasons why clearing is done the current way?

I think the only reason was to avoid doing reset for the first usage.
You're right it is not optimal when reusing memory
without any zeroing need.



^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [dpdk-dev] Zeroing out memory on free
  2021-08-25  7:26 ` Thomas Monjalon
@ 2021-08-25 11:47   ` Burakov, Anatoly
  2021-08-25 12:30     ` Thomas Monjalon
  0 siblings, 1 reply; 4+ messages in thread
From: Burakov, Anatoly @ 2021-08-25 11:47 UTC (permalink / raw)
  To: Thomas Monjalon, Dmitry Kozlyuk
  Cc: dev, Xueming(Steven) Li, david.marchand, bruce.richardson

On 25-Aug-21 8:26 AM, Thomas Monjalon wrote:
> 24/08/2021 12:58, Dmitry Kozlyuk:
>> Hello,
>>
>> Me and Xueming are wondering why DPDK clears the memory on free
>> and not only when it's explicitly requested (rte_zmalloc).
>>
>> It's been so for a while:
>>
>> commit ea0bddbd14e68fb42d9774bc3543e51b510e48d3
>> Author: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
>> Date:   Tue Jul 5 12:01:15 2016 +0100
>>
>>      mem: zero out memory on free
>>      
>>      Since commit fafcc11985a2, memzones are not guaranteed to be zeroed out.
>>      This could potentially cause issues as applications might have been
>>      relying on the allocated memory being zeroed out.
>>      
>>      On init all allocated memory is zeroed by the kernel, so by zeroing out
>>      memory on free, all available dpdk memory is always zeroed.
>>      
>>      Fixes: fafcc11985a2 ("mem: rework memzone to be allocated by malloc")
>>      
>>      Signed-off-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
>>
>> At present this explanation doesn't look satisfying:
>> 1. Memzone API does not promise they will be zeroed out.
>>     Memzones are mostly used for DMA, so their content will likely be overwritten.
>> 2. If application assumptions are wrong, DPDK should not try to fix it.
>>     Memory manager poisons memory in debug mode to detect such errors.
>>
>> Zeroing memory is quite slow, but in many cases unneeded.
>> It looks attractive to only do it in rte_zmalloc().
>> AFAIK what prohibits moving memset() there unconditionally
>> is that the kernel already gives us cleared pages,
>> so the first allocation of each piece of memory would do unnecessary work.
>> This can be solved by giving the user a choice in EAL options
> 
> No I don't think it should be workarounded in EAL options.
> 
>> or with more elaborate tracking of memory dirtiness in MM.
> 
> Looks a good idea.
> 
>> Are there any other reasons why clearing is done the current way?
> 
> I think the only reason was to avoid doing reset for the first usage.
> You're right it is not optimal when reusing memory
> without any zeroing need.
> 

You said it yourself: zeroing out memory is quite slow. This was pretty 
much done for performance reasons - memory is released way less often 
than allocated, and we get memory from the kernel that is *always* 
zeroed out in case of hugepages, so we don't need to zero them out 
initially. So, in essence, this allows us to always keeps memory zeroed 
out without worrying about doing that for every allocation.

-- 
Thanks,
Anatoly

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [dpdk-dev] Zeroing out memory on free
  2021-08-25 11:47   ` Burakov, Anatoly
@ 2021-08-25 12:30     ` Thomas Monjalon
  0 siblings, 0 replies; 4+ messages in thread
From: Thomas Monjalon @ 2021-08-25 12:30 UTC (permalink / raw)
  To: Burakov, Anatoly
  Cc: Dmitry Kozlyuk, dev, Xueming(Steven) Li, david.marchand,
	bruce.richardson

25/08/2021 13:47, Burakov, Anatoly:
> On 25-Aug-21 8:26 AM, Thomas Monjalon wrote:
> > 24/08/2021 12:58, Dmitry Kozlyuk:
> >> Hello,
> >>
> >> Me and Xueming are wondering why DPDK clears the memory on free
> >> and not only when it's explicitly requested (rte_zmalloc).
> >>
> >> It's been so for a while:
> >>
> >> commit ea0bddbd14e68fb42d9774bc3543e51b510e48d3
> >> Author: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
> >> Date:   Tue Jul 5 12:01:15 2016 +0100
> >>
> >>      mem: zero out memory on free
> >>      
> >>      Since commit fafcc11985a2, memzones are not guaranteed to be zeroed out.
> >>      This could potentially cause issues as applications might have been
> >>      relying on the allocated memory being zeroed out.
> >>      
> >>      On init all allocated memory is zeroed by the kernel, so by zeroing out
> >>      memory on free, all available dpdk memory is always zeroed.
> >>      
> >>      Fixes: fafcc11985a2 ("mem: rework memzone to be allocated by malloc")
> >>      
> >>      Signed-off-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
> >>
> >> At present this explanation doesn't look satisfying:
> >> 1. Memzone API does not promise they will be zeroed out.
> >>     Memzones are mostly used for DMA, so their content will likely be overwritten.
> >> 2. If application assumptions are wrong, DPDK should not try to fix it.
> >>     Memory manager poisons memory in debug mode to detect such errors.
> >>
> >> Zeroing memory is quite slow, but in many cases unneeded.
> >> It looks attractive to only do it in rte_zmalloc().
> >> AFAIK what prohibits moving memset() there unconditionally
> >> is that the kernel already gives us cleared pages,
> >> so the first allocation of each piece of memory would do unnecessary work.
> >> This can be solved by giving the user a choice in EAL options
> > 
> > No I don't think it should be workarounded in EAL options.
> > 
> >> or with more elaborate tracking of memory dirtiness in MM.
> > 
> > Looks a good idea.
> > 
> >> Are there any other reasons why clearing is done the current way?
> > 
> > I think the only reason was to avoid doing reset for the first usage.
> > You're right it is not optimal when reusing memory
> > without any zeroing need.
> > 
> 
> You said it yourself: zeroing out memory is quite slow. This was pretty 
> much done for performance reasons - memory is released way less often 
> than allocated, and we get memory from the kernel that is *always* 
> zeroed out in case of hugepages, so we don't need to zero them out 
> initially. So, in essence, this allows us to always keeps memory zeroed 
> out without worrying about doing that for every allocation.

Anatoly, what do you think about the idea above
(marking initial state as clean, and zero on allocations only if needed)?



^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2021-08-25 12:30 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-08-24 10:58 [dpdk-dev] Zeroing out memory on free Dmitry Kozlyuk
2021-08-25  7:26 ` Thomas Monjalon
2021-08-25 11:47   ` Burakov, Anatoly
2021-08-25 12:30     ` Thomas Monjalon

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).