* [dpdk-dev] Is contiguous hugepages memory still required in latest DPDKs?
@ 2017-03-21 14:20 Ruslan Bilovol
2017-03-21 14:41 ` Sergio Gonzalez Monroy
0 siblings, 1 reply; 5+ messages in thread
From: Ruslan Bilovol @ 2017-03-21 14:20 UTC (permalink / raw)
To: dev
Hi,
Recently after moving to 4.4 Linux kernel we found that DPDK v16.07
can't find physically contiguous chunk of hugepages memory.
I've tracked this down to kernel commits
81c0a2bb515f ("mm: page_alloc: fair zone allocator policy")
fff4068cba48 ("mm: page_alloc: revert NUMA aspect of fair allocation policy")
4ffeaf3560a5 ("mm: page_alloc: reduce cost of the fair zone allocation policy")
These commits changed default page allocator behavior so it now allocates
memory proportionally from preferred and lower zones. Hugepages
are scattered proportionally among few memory zones, so possibility
to find big physically contiguous chunk of hugepages memory is much lower.
I see that there were some attempts to move from contiguous hugepages
approach, like http://dpdk.org/ml/archives/dev/2016-March/035201.html
Also some discussion here: http://dpdk.org/ml/archives/users/2016-October/001050.html
The question: is contiguous hugepages memory still required in latest DPDKs,
and if not, since which version?
Thanks,
Ruslan
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [dpdk-dev] Is contiguous hugepages memory still required in latest DPDKs?
2017-03-21 14:20 [dpdk-dev] Is contiguous hugepages memory still required in latest DPDKs? Ruslan Bilovol
@ 2017-03-21 14:41 ` Sergio Gonzalez Monroy
2017-03-22 12:08 ` Ruslan Bilovol
0 siblings, 1 reply; 5+ messages in thread
From: Sergio Gonzalez Monroy @ 2017-03-21 14:41 UTC (permalink / raw)
To: Ruslan Bilovol, dev
The DPDK will still try to allocate contiguous hugepages.
Since DPDK v16.07 the mempool library is able to create a mempool with
multiple chunks of memory.
You would still have the contig memory restriction for any other DPDK
library or app.
Are you pre-allocating hugepages on boot (kernel cmdline) or runtime?
Sergio
On 21/03/2017 14:20, Ruslan Bilovol wrote:
> Hi,
>
> Recently after moving to 4.4 Linux kernel we found that DPDK v16.07
> can't find physically contiguous chunk of hugepages memory.
>
> I've tracked this down to kernel commits
> 81c0a2bb515f ("mm: page_alloc: fair zone allocator policy")
> fff4068cba48 ("mm: page_alloc: revert NUMA aspect of fair allocation
> policy")
> 4ffeaf3560a5 ("mm: page_alloc: reduce cost of the fair zone
> allocation policy")
>
> These commits changed default page allocator behavior so it now allocates
> memory proportionally from preferred and lower zones. Hugepages
> are scattered proportionally among few memory zones, so possibility
> to find big physically contiguous chunk of hugepages memory is much
> lower.
>
> I see that there were some attempts to move from contiguous hugepages
> approach, like http://dpdk.org/ml/archives/dev/2016-March/035201.html
> Also some discussion here:
> http://dpdk.org/ml/archives/users/2016-October/001050.html
>
> The question: is contiguous hugepages memory still required in latest
> DPDKs,
> and if not, since which version?
>
> Thanks,
> Ruslan
>
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [dpdk-dev] Is contiguous hugepages memory still required in latest DPDKs?
2017-03-21 14:41 ` Sergio Gonzalez Monroy
@ 2017-03-22 12:08 ` Ruslan Bilovol
2017-03-23 16:24 ` Ruslan Bilovol
0 siblings, 1 reply; 5+ messages in thread
From: Ruslan Bilovol @ 2017-03-22 12:08 UTC (permalink / raw)
To: Sergio Gonzalez Monroy, dev
On 03/21/2017 04:41 PM, Sergio Gonzalez Monroy wrote:
> The DPDK will still try to allocate contiguous hugepages.
> Since DPDK v16.07 the mempool library is able to create a mempool with multiple chunks of memory.
> You would still have the contig memory restriction for any other DPDK library or app.
>
> Are you pre-allocating hugepages on boot (kernel cmdline) or runtime?
I tried both cases, and there is no much difference between them.
(I used debug prints in kernel to see physical address of preallocated hugepages
Main issue is that linux kernel memory allocation algorithm was changed
with "fair zone allocator policy" commits, and in my case need to
pre-allocate 2x or even 3x more hugepages now.
Regards,
Ruslan
>
> Sergio
>
> On 21/03/2017 14:20, Ruslan Bilovol wrote:
>> Hi,
>>
>> Recently after moving to 4.4 Linux kernel we found that DPDK v16.07
>> can't find physically contiguous chunk of hugepages memory.
>>
>> I've tracked this down to kernel commits
>> 81c0a2bb515f ("mm: page_alloc: fair zone allocator policy")
>> fff4068cba48 ("mm: page_alloc: revert NUMA aspect of fair allocation policy")
>> 4ffeaf3560a5 ("mm: page_alloc: reduce cost of the fair zone allocation policy")
>>
>> These commits changed default page allocator behavior so it now allocates
>> memory proportionally from preferred and lower zones. Hugepages
>> are scattered proportionally among few memory zones, so possibility
>> to find big physically contiguous chunk of hugepages memory is much lower.
>>
>> I see that there were some attempts to move from contiguous hugepages
>> approach, like http://dpdk.org/ml/archives/dev/2016-March/035201.html
>> Also some discussion here: http://dpdk.org/ml/archives/users/2016-October/001050.html
>>
>> The question: is contiguous hugepages memory still required in latest DPDKs,
>> and if not, since which version?
>>
>> Thanks,
>> Ruslan
>>
>
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [dpdk-dev] Is contiguous hugepages memory still required in latest DPDKs?
2017-03-22 12:08 ` Ruslan Bilovol
@ 2017-03-23 16:24 ` Ruslan Bilovol
2017-03-23 16:49 ` Sergio Gonzalez Monroy
0 siblings, 1 reply; 5+ messages in thread
From: Ruslan Bilovol @ 2017-03-23 16:24 UTC (permalink / raw)
To: Sergio Gonzalez Monroy, dev
On 03/22/2017 02:08 PM, Ruslan Bilovol wrote:
> On 03/21/2017 04:41 PM, Sergio Gonzalez Monroy wrote:
>> The DPDK will still try to allocate contiguous hugepages.
>> Since DPDK v16.07 the mempool library is able to create a mempool with multiple chunks of memory.
>> You would still have the contig memory restriction for any other DPDK library or app.
Another question: are there any improvements in contiguous hugepages requirements in latest DPDK (v17.02)?
Thanks,
Ruslan
>>
>> Are you pre-allocating hugepages on boot (kernel cmdline) or runtime?
> I tried both cases, and there is no much difference between them.
> (I used debug prints in kernel to see physical address of preallocated hugepages
>
> Main issue is that linux kernel memory allocation algorithm was changed
> with "fair zone allocator policy" commits, and in my case need to
> pre-allocate 2x or even 3x more hugepages now.
>
> Regards,
> Ruslan
>
>>
>> Sergio
>>
>> On 21/03/2017 14:20, Ruslan Bilovol wrote:
>>> Hi,
>>>
>>> Recently after moving to 4.4 Linux kernel we found that DPDK v16.07
>>> can't find physically contiguous chunk of hugepages memory.
>>>
>>> I've tracked this down to kernel commits
>>> 81c0a2bb515f ("mm: page_alloc: fair zone allocator policy")
>>> fff4068cba48 ("mm: page_alloc: revert NUMA aspect of fair allocation policy")
>>> 4ffeaf3560a5 ("mm: page_alloc: reduce cost of the fair zone allocation policy")
>>>
>>> These commits changed default page allocator behavior so it now allocates
>>> memory proportionally from preferred and lower zones. Hugepages
>>> are scattered proportionally among few memory zones, so possibility
>>> to find big physically contiguous chunk of hugepages memory is much lower.
>>>
>>> I see that there were some attempts to move from contiguous hugepages
>>> approach, like http://dpdk.org/ml/archives/dev/2016-March/035201.html
>>> Also some discussion here: http://dpdk.org/ml/archives/users/2016-October/001050.html
>>>
>>> The question: is contiguous hugepages memory still required in latest DPDKs,
>>> and if not, since which version?
>>>
>>> Thanks,
>>> Ruslan
>>>
>>
>
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [dpdk-dev] Is contiguous hugepages memory still required in latest DPDKs?
2017-03-23 16:24 ` Ruslan Bilovol
@ 2017-03-23 16:49 ` Sergio Gonzalez Monroy
0 siblings, 0 replies; 5+ messages in thread
From: Sergio Gonzalez Monroy @ 2017-03-23 16:49 UTC (permalink / raw)
To: Ruslan Bilovol, dev
On 23/03/2017 16:24, Ruslan Bilovol wrote:
> On 03/22/2017 02:08 PM, Ruslan Bilovol wrote:
>> On 03/21/2017 04:41 PM, Sergio Gonzalez Monroy wrote:
>>> The DPDK will still try to allocate contiguous hugepages.
>>> Since DPDK v16.07 the mempool library is able to create a mempool
>>> with multiple chunks of memory.
>>> You would still have the contig memory restriction for any other
>>> DPDK library or app.
>
> Another question: are there any improvements in contiguous hugepages
> requirements in latest DPDK (v17.02)?
>
Not that I can think of.
Sergio
> Thanks,
> Ruslan
>
>
>>>
>>> Are you pre-allocating hugepages on boot (kernel cmdline) or runtime?
>> I tried both cases, and there is no much difference between them.
>> (I used debug prints in kernel to see physical address of
>> preallocated hugepages
>>
>> Main issue is that linux kernel memory allocation algorithm was changed
>> with "fair zone allocator policy" commits, and in my case need to
>> pre-allocate 2x or even 3x more hugepages now.
>>
>> Regards,
>> Ruslan
>>
>>>
>>> Sergio
>>>
>>> On 21/03/2017 14:20, Ruslan Bilovol wrote:
>>>> Hi,
>>>>
>>>> Recently after moving to 4.4 Linux kernel we found that DPDK v16.07
>>>> can't find physically contiguous chunk of hugepages memory.
>>>>
>>>> I've tracked this down to kernel commits
>>>> 81c0a2bb515f ("mm: page_alloc: fair zone allocator policy")
>>>> fff4068cba48 ("mm: page_alloc: revert NUMA aspect of fair
>>>> allocation policy")
>>>> 4ffeaf3560a5 ("mm: page_alloc: reduce cost of the fair zone
>>>> allocation policy")
>>>>
>>>> These commits changed default page allocator behavior so it now
>>>> allocates
>>>> memory proportionally from preferred and lower zones. Hugepages
>>>> are scattered proportionally among few memory zones, so possibility
>>>> to find big physically contiguous chunk of hugepages memory is much
>>>> lower.
>>>>
>>>> I see that there were some attempts to move from contiguous hugepages
>>>> approach, like http://dpdk.org/ml/archives/dev/2016-March/035201.html
>>>> Also some discussion here:
>>>> http://dpdk.org/ml/archives/users/2016-October/001050.html
>>>>
>>>> The question: is contiguous hugepages memory still required in
>>>> latest DPDKs,
>>>> and if not, since which version?
>>>>
>>>> Thanks,
>>>> Ruslan
>>>>
>>>
>>
>
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2017-03-23 16:49 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-03-21 14:20 [dpdk-dev] Is contiguous hugepages memory still required in latest DPDKs? Ruslan Bilovol
2017-03-21 14:41 ` Sergio Gonzalez Monroy
2017-03-22 12:08 ` Ruslan Bilovol
2017-03-23 16:24 ` Ruslan Bilovol
2017-03-23 16:49 ` Sergio Gonzalez Monroy
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).