[dpdk-users] No free hugepages reported in hugepages when running DPDK helloworld example

[dpdk-users] No free hugepages reported in hugepages when running DPDK helloworld example

[gyp sud]

I downloaded and built dpdk-stable-16.11.4 version (using
x86_64-native-linuxapp-gcc target).  I'm running Ubuntu 16.04.3 LTS.
After setting up hugepages according to http://dpdk.org/doc/quick-start or
http://dpdk.org/doc/guides-16.04/linux_gsg/sys_reqs.html

mkdir -p /mnt/huge
mount -t hugetlbfs nodev /mnt/huge
echo 64 >
/sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages

I can see the hugepages fine.

#cat /proc/meminfo | grep
HugeAnonHugePages:    284672 kB
ShmemHugePages:        0 kB
HugePages_Total:      64
HugePages_Free:       64
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB

But when I run the helloWorld example, it complained about no free
hugepages, see below.

#./build/helloworld -l 0-3 -n 2
EAL: Detected 4 lcore(s)
EAL: No free hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
hello from core 1
hello from core 2
hello from core 3
hello from core 0

Also, the memory info showed free hugepages were all gone.
#cat /proc/meminfo | grep Huge
AnonHugePages:    380928 kB
ShmemHugePages:        0 kB
HugePages_Total:      64
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB

Any idea why this is happening?

Gyp

Re: [dpdk-users] No free hugepages reported in hugepages when running DPDK helloworld example

[James Bensley]

On 23 December 2017 at 21:44, gyp sud <gypsud@gmail.com> wrote:
> I downloaded and built dpdk-stable-16.11.4 version (using
> x86_64-native-linuxapp-gcc target).  I'm running Ubuntu 16.04.3 LTS.
> After setting up hugepages according to http://dpdk.org/doc/quick-start or
> http://dpdk.org/doc/guides-16.04/linux_gsg/sys_reqs.html
>
> mkdir -p /mnt/huge
> mount -t hugetlbfs nodev /mnt/huge
> echo 64 >
> /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
>
> I can see the hugepages fine.
>
> #cat /proc/meminfo | grep
> HugeAnonHugePages:    284672 kB
> ShmemHugePages:        0 kB
> HugePages_Total:      64
> HugePages_Free:       64
> HugePages_Rsvd:        0
> HugePages_Surp:        0
> Hugepagesize:       2048 kB
>
> But when I run the helloWorld example, it complained about no free
> hugepages, see below.
>
> #./build/helloworld -l 0-3 -n 2
> EAL: Detected 4 lcore(s)
> EAL: No free hugepages reported in hugepages-1048576kB
> EAL: Probing VFIO support...
> hello from core 1
> hello from core 2
> hello from core 3
> hello from core 0
>
> Also, the memory info showed free hugepages were all gone.
> #cat /proc/meminfo | grep Huge
> AnonHugePages:    380928 kB
> ShmemHugePages:        0 kB
> HugePages_Total:      64
> HugePages_Free:        0
> HugePages_Rsvd:        0
> HugePages_Surp:        0
> Hugepagesize:       2048 kB
>
> Any idea why this is happening?
>
> Gyp

Note the size of the HugePages in the EAL output
"hugepages-1048576kB", you allocated 2MB HugePages, this message is
about 1G HugePages.

1GB HugePages can only be allocated at boot up. It's "OK" though, 1GB
HugePages aren't required, as you saw the application used the 2GB
HugePages you allocated. For some applications and workloads that
require lots of memory fewer 1GB pages can have a performance increase
over more smaller 2MB HugePages.

Obviously it goes without saying it's usually best to install the most
recent DPDK version and check the most recent documentation URL, here
you will find info about setting up 1GB HugePages if you want to do
that:
dpdk.org/doc/guides/linux_gsg/sys_reqs.html

Cheers,
James.

Re: [dpdk-users] No free hugepages reported in hugepages when running DPDK helloworld example

[gyp sud]

Thanks, James.  I looked through some kernel documentation regarding
hugetlbpage support and DPDK source code (eal_hugepage_info.c in
particular), it appears the code was looking in
/sys/kernel/mm/hugepages the root huge page control directory.  I
guess since my system supports both pse and pdpe1gb flag, so 2MB and
1GB pages are supported.  Thus my /sys/kernel/mm/hugepages has
hugepages-1048576kB and hugepages-2048kB directories.  helloworld was
searching through both directories but complained about 1GB because
I've only allocated pages for 2MB as you pointed out.  So that warning
about 1GB is benign.

Cheers,

Gyp

On Sun, Dec 24, 2017 at 5:53 AM, James Bensley <jwbensley@gmail.com> wrote:
> On 23 December 2017 at 21:44, gyp sud <gypsud@gmail.com> wrote:
>> I downloaded and built dpdk-stable-16.11.4 version (using
>> x86_64-native-linuxapp-gcc target).  I'm running Ubuntu 16.04.3 LTS.
>> After setting up hugepages according to http://dpdk.org/doc/quick-start or
>> http://dpdk.org/doc/guides-16.04/linux_gsg/sys_reqs.html
>>
>> mkdir -p /mnt/huge
>> mount -t hugetlbfs nodev /mnt/huge
>> echo 64 >
>> /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
>>
>> I can see the hugepages fine.
>>
>> #cat /proc/meminfo | grep
>> HugeAnonHugePages:    284672 kB
>> ShmemHugePages:        0 kB
>> HugePages_Total:      64
>> HugePages_Free:       64
>> HugePages_Rsvd:        0
>> HugePages_Surp:        0
>> Hugepagesize:       2048 kB
>>
>> But when I run the helloWorld example, it complained about no free
>> hugepages, see below.
>>
>> #./build/helloworld -l 0-3 -n 2
>> EAL: Detected 4 lcore(s)
>> EAL: No free hugepages reported in hugepages-1048576kB
>> EAL: Probing VFIO support...
>> hello from core 1
>> hello from core 2
>> hello from core 3
>> hello from core 0
>>
>> Also, the memory info showed free hugepages were all gone.
>> #cat /proc/meminfo | grep Huge
>> AnonHugePages:    380928 kB
>> ShmemHugePages:        0 kB
>> HugePages_Total:      64
>> HugePages_Free:        0
>> HugePages_Rsvd:        0
>> HugePages_Surp:        0
>> Hugepagesize:       2048 kB
>>
>> Any idea why this is happening?
>>
>> Gyp
>
> Note the size of the HugePages in the EAL output
> "hugepages-1048576kB", you allocated 2MB HugePages, this message is
> about 1G HugePages.
>
> 1GB HugePages can only be allocated at boot up. It's "OK" though, 1GB
> HugePages aren't required, as you saw the application used the 2GB
> HugePages you allocated. For some applications and workloads that
> require lots of memory fewer 1GB pages can have a performance increase
> over more smaller 2MB HugePages.
>
> Obviously it goes without saying it's usually best to install the most
> recent DPDK version and check the most recent documentation URL, here
> you will find info about setting up 1GB HugePages if you want to do
> that:
> dpdk.org/doc/guides/linux_gsg/sys_reqs.html
>
> Cheers,
> James.