* [dpdk-dev] mmap fails with more than 40000 hugepages
@ 2015-02-05 12:00 Damjan Marion (damarion)
2015-02-05 12:59 ` Neil Horman
` (3 more replies)
0 siblings, 4 replies; 13+ messages in thread
From: Damjan Marion (damarion) @ 2015-02-05 12:00 UTC (permalink / raw)
To: dev
Hi,
I have system with 2 NUMA nodes and 256G RAM total. I noticed that DPDK crashes in rte_eal_init()
when number of available hugepages is around 40000 or above.
Everything works fine with lower values (i.e. 30000).
I also tried with allocating 40000 on node0 and 0 on node1, same crash happens.
Any idea what might be causing this?
Thanks,
Damjan
$ cat /sys/devices/system/node/node[01]/hugepages/hugepages-2048kB/nr_hugepages
20000
20000
$ grep -i huge /proc/meminfo
AnonHugePages: 706560 kB
HugePages_Total: 40000
HugePages_Free: 40000
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
$ sudo ~/src/dpdk/x86_64-native-linuxapp-gcc/app/testpmd -l 5-7 -n 3 --socket-mem 512,512
EAL: Detected lcore 0 as core 0 on socket 0
EAL: Detected lcore 1 as core 1 on socket 0
EAL: Detected lcore 2 as core 2 on socket 0
EAL: Detected lcore 3 as core 3 on socket 0
EAL: Detected lcore 4 as core 4 on socket 0
EAL: Detected lcore 5 as core 5 on socket 0
EAL: Detected lcore 6 as core 6 on socket 0
EAL: Detected lcore 7 as core 7 on socket 0
EAL: Detected lcore 8 as core 0 on socket 1
EAL: Detected lcore 9 as core 1 on socket 1
EAL: Detected lcore 10 as core 2 on socket 1
EAL: Detected lcore 11 as core 3 on socket 1
EAL: Detected lcore 12 as core 4 on socket 1
EAL: Detected lcore 13 as core 5 on socket 1
EAL: Detected lcore 14 as core 6 on socket 1
EAL: Detected lcore 15 as core 7 on socket 1
EAL: Detected lcore 16 as core 0 on socket 0
EAL: Detected lcore 17 as core 1 on socket 0
EAL: Detected lcore 18 as core 2 on socket 0
EAL: Detected lcore 19 as core 3 on socket 0
EAL: Detected lcore 20 as core 4 on socket 0
EAL: Detected lcore 21 as core 5 on socket 0
EAL: Detected lcore 22 as core 6 on socket 0
EAL: Detected lcore 23 as core 7 on socket 0
EAL: Detected lcore 24 as core 0 on socket 1
EAL: Detected lcore 25 as core 1 on socket 1
EAL: Detected lcore 26 as core 2 on socket 1
EAL: Detected lcore 27 as core 3 on socket 1
EAL: Detected lcore 28 as core 4 on socket 1
EAL: Detected lcore 29 as core 5 on socket 1
EAL: Detected lcore 30 as core 6 on socket 1
EAL: Detected lcore 31 as core 7 on socket 1
EAL: Support maximum 128 logical core(s) by configuration.
EAL: Detected 32 lcore(s)
EAL: VFIO modules not all loaded, skip VFIO support...
EAL: Setting up memory...
EAL: Ask a virtual area of 0x800000 bytes
EAL: Virtual area found at 0x7fae2a200000 (size = 0x800000)
EAL: Ask a virtual area of 0x7600000 bytes
EAL: Virtual area found at 0x7fae22a00000 (size = 0x7600000)
EAL: Ask a virtual area of 0x1400000 bytes
EAL: Virtual area found at 0x7fae21400000 (size = 0x1400000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fae21000000 (size = 0x200000)
EAL: Ask a virtual area of 0x400000 bytes
EAL: Virtual area found at 0x7fae20a00000 (size = 0x400000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fae20600000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fae20200000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fae1fe00000 (size = 0x200000)
EAL: Ask a virtual area of 0x5800000 bytes
EAL: Virtual area found at 0x7fae1a400000 (size = 0x5800000)
EAL: Ask a virtual area of 0x3b200000 bytes
EAL: Virtual area found at 0x7faddf000000 (size = 0x3b200000)
EAL: Ask a virtual area of 0x400000 bytes
EAL: Virtual area found at 0x7faddea00000 (size = 0x400000)
EAL: Ask a virtual area of 0x7c00000 bytes
EAL: Virtual area found at 0x7fadd6c00000 (size = 0x7c00000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fadd6800000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fadd6400000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fadd6000000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fadd5c00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fadd5800000 (size = 0x200000)
EAL: Ask a virtual area of 0x9800000 bytes
EAL: Virtual area found at 0x7fadcbe00000 (size = 0x9800000)
EAL: Ask a virtual area of 0x19000000 bytes
EAL: Virtual area found at 0x7fadb2c00000 (size = 0x19000000)
EAL: Ask a virtual area of 0x24400000 bytes
EAL: Virtual area found at 0x7fad8e600000 (size = 0x24400000)
EAL: Ask a virtual area of 0xc800000 bytes
EAL: Virtual area found at 0x7fad81c00000 (size = 0xc800000)
EAL: Ask a virtual area of 0x32000000 bytes
EAL: Virtual area found at 0x7fad4fa00000 (size = 0x32000000)
EAL: Ask a virtual area of 0x3db800000 bytes
EAL: Virtual area found at 0x7fa974000000 (size = 0x3db800000)
EAL: Ask a virtual area of 0xfa000000 bytes
EAL: Virtual area found at 0x7fa879e00000 (size = 0xfa000000)
EAL: Ask a virtual area of 0x168400000 bytes
EAL: Virtual area found at 0x7fa711800000 (size = 0x168400000)
EAL: Ask a virtual area of 0x7d000000 bytes
EAL: Virtual area found at 0x7fa694600000 (size = 0x7d000000)
EAL: Ask a virtual area of 0x1f4000000 bytes
EAL: Virtual area found at 0x7fa4a0400000 (size = 0x1f4000000)
EAL: Ask a virtual area of 0x2e400000 bytes
EAL: Virtual area found at 0x7fa471e00000 (size = 0x2e400000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa471a00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa471600000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa471200000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa470e00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa470a00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa470600000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa470200000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa46fe00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa46fa00000 (size = 0x200000)
EAL: Ask a virtual area of 0x400000 bytes
EAL: Virtual area found at 0x7fa46f400000 (size = 0x400000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa46f000000 (size = 0x200000)
EAL: Ask a virtual area of 0x400000 bytes
EAL: Virtual area found at 0x7fa46ea00000 (size = 0x400000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa46e600000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa46e200000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa46de00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa46da00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa46d600000 (size = 0x200000)
EAL: Ask a virtual area of 0x400000 bytes
EAL: Virtual area found at 0x7fa46d000000 (size = 0x400000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa46cc00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa46c800000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa46c400000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa46c000000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa46bc00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa46b800000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa46b400000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa46b000000 (size = 0x200000)
EAL: Ask a virtual area of 0x400000 bytes
EAL: Virtual area found at 0x7fa46aa00000 (size = 0x400000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa46a600000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa46a200000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa469e00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa469a00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa469600000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa469200000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa468e00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa468a00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa468600000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa468200000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa467e00000 (size = 0x200000)
EAL: Ask a virtual area of 0x400000 bytes
EAL: Virtual area found at 0x7fa467800000 (size = 0x400000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa467400000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa467000000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa466c00000 (size = 0x200000)
EAL: Ask a virtual area of 0x400000 bytes
EAL: Virtual area found at 0x7fa466600000 (size = 0x400000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa466200000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa465e00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa465a00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa465600000 (size = 0x200000)
EAL: Ask a virtual area of 0x400000 bytes
EAL: Virtual area found at 0x7fa465000000 (size = 0x400000)
EAL: Ask a virtual area of 0x400000 bytes
EAL: Virtual area found at 0x7fa464a00000 (size = 0x400000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa464600000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa464200000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa463e00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa463a00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa463600000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa463200000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa462e00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa462a00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa462600000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa462200000 (size = 0x200000)
EAL: Ask a virtual area of 0x400000 bytes
EAL: Virtual area found at 0x7fa461c00000 (size = 0x400000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa461800000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa461400000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa461000000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa460c00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa460800000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa460400000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa460000000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa45fc00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa45f800000 (size = 0x200000)
EAL: Ask a virtual area of 0x400000 bytes
EAL: Virtual area found at 0x7fa45f200000 (size = 0x400000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa45ee00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa45ea00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa45e600000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa45e200000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa45de00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa45da00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa45d600000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa45d200000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa45ce00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa45ca00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa45c600000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa45c200000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa45be00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa45ba00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa45b600000 (size = 0x200000)
EAL: Ask a virtual area of 0x400000 bytes
EAL: Virtual area found at 0x7fa45b000000 (size = 0x400000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa45ac00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa45a800000 (size = 0x200000)
EAL: Ask a virtual area of 0x400000 bytes
EAL: Virtual area found at 0x7fa45a200000 (size = 0x400000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa459e00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa459a00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa459600000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa459200000 (size = 0x200000)
EAL: Ask a virtual area of 0x400000 bytes
EAL: Virtual area found at 0x7fa458c00000 (size = 0x400000)
EAL: Ask a virtual area of 0x400000 bytes
EAL: Virtual area found at 0x7fa458600000 (size = 0x400000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa458200000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa457e00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa457a00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa457600000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa457200000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa456e00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa456a00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fa456600000 (size = 0x200000)
EAL: Ask a virtual area of 0x139400000 bytes
EAL: Virtual area found at 0x7fa31d000000 (size = 0x139400000)
EAL: Ask a virtual area of 0xe9000000 bytes
EAL: Virtual area found at 0x7fa233e00000 (size = 0xe9000000)
EAL: Ask a virtual area of 0x188000000 bytes
EAL: Virtual area found at 0x7fa0abc00000 (size = 0x188000000)
EAL: map_all_hugepages(): mmap failed: Cannot allocate memory
EAL: Failed to remap 2 MB pages
PANIC in rte_eal_init():
Cannot init memory
Aborted (core dumped)
^ permalink raw reply [flat|nested] 13+ messages in thread* Re: [dpdk-dev] mmap fails with more than 40000 hugepages 2015-02-05 12:00 [dpdk-dev] mmap fails with more than 40000 hugepages Damjan Marion (damarion) @ 2015-02-05 12:59 ` Neil Horman 2015-02-05 13:20 ` Damjan Marion (damarion) 2015-02-05 13:22 ` Jay Rolette ` (2 subsequent siblings) 3 siblings, 1 reply; 13+ messages in thread From: Neil Horman @ 2015-02-05 12:59 UTC (permalink / raw) To: Damjan Marion (damarion); +Cc: dev On Thu, Feb 05, 2015 at 12:00:48PM +0000, Damjan Marion (damarion) wrote: > Hi, > > I have system with 2 NUMA nodes and 256G RAM total. I noticed that DPDK crashes in rte_eal_init() > when number of available hugepages is around 40000 or above. > Everything works fine with lower values (i.e. 30000). > > I also tried with allocating 40000 on node0 and 0 on node1, same crash happens. > > > Any idea what might be causing this? > > Thanks, > > Damjan > > > $ cat /sys/devices/system/node/node[01]/hugepages/hugepages-2048kB/nr_hugepages > 20000 > 20000 > > $ grep -i huge /proc/meminfo > AnonHugePages: 706560 kB > HugePages_Total: 40000 > HugePages_Free: 40000 > HugePages_Rsvd: 0 > HugePages_Surp: 0 > Hugepagesize: 2048 kB > Whats your shmmax value set to? 40000 2MB hugepages is way above the default setting for how much shared ram a system will allow. I've not done the math on your logs below, but judging by the size of some of the mapped segments, I'm betting your hitting the default limit of 4GB. Neil > > $ sudo ~/src/dpdk/x86_64-native-linuxapp-gcc/app/testpmd -l 5-7 -n 3 --socket-mem 512,512 > EAL: Detected lcore 0 as core 0 on socket 0 > EAL: Detected lcore 1 as core 1 on socket 0 > EAL: Detected lcore 2 as core 2 on socket 0 > EAL: Detected lcore 3 as core 3 on socket 0 > EAL: Detected lcore 4 as core 4 on socket 0 > EAL: Detected lcore 5 as core 5 on socket 0 > EAL: Detected lcore 6 as core 6 on socket 0 > EAL: Detected lcore 7 as core 7 on socket 0 > EAL: Detected lcore 8 as core 0 on socket 1 > EAL: Detected lcore 9 as core 1 on socket 1 > EAL: Detected lcore 10 as core 2 on socket 1 > EAL: Detected lcore 11 as core 3 on socket 1 > EAL: Detected lcore 12 as core 4 on socket 1 > EAL: Detected lcore 13 as core 5 on socket 1 > EAL: Detected lcore 14 as core 6 on socket 1 > EAL: Detected lcore 15 as core 7 on socket 1 > EAL: Detected lcore 16 as core 0 on socket 0 > EAL: Detected lcore 17 as core 1 on socket 0 > EAL: Detected lcore 18 as core 2 on socket 0 > EAL: Detected lcore 19 as core 3 on socket 0 > EAL: Detected lcore 20 as core 4 on socket 0 > EAL: Detected lcore 21 as core 5 on socket 0 > EAL: Detected lcore 22 as core 6 on socket 0 > EAL: Detected lcore 23 as core 7 on socket 0 > EAL: Detected lcore 24 as core 0 on socket 1 > EAL: Detected lcore 25 as core 1 on socket 1 > EAL: Detected lcore 26 as core 2 on socket 1 > EAL: Detected lcore 27 as core 3 on socket 1 > EAL: Detected lcore 28 as core 4 on socket 1 > EAL: Detected lcore 29 as core 5 on socket 1 > EAL: Detected lcore 30 as core 6 on socket 1 > EAL: Detected lcore 31 as core 7 on socket 1 > EAL: Support maximum 128 logical core(s) by configuration. > EAL: Detected 32 lcore(s) > EAL: VFIO modules not all loaded, skip VFIO support... > EAL: Setting up memory... > EAL: Ask a virtual area of 0x800000 bytes > EAL: Virtual area found at 0x7fae2a200000 (size = 0x800000) > EAL: Ask a virtual area of 0x7600000 bytes > EAL: Virtual area found at 0x7fae22a00000 (size = 0x7600000) > EAL: Ask a virtual area of 0x1400000 bytes > EAL: Virtual area found at 0x7fae21400000 (size = 0x1400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fae21000000 (size = 0x200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fae20a00000 (size = 0x400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fae20600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fae20200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fae1fe00000 (size = 0x200000) > EAL: Ask a virtual area of 0x5800000 bytes > EAL: Virtual area found at 0x7fae1a400000 (size = 0x5800000) > EAL: Ask a virtual area of 0x3b200000 bytes > EAL: Virtual area found at 0x7faddf000000 (size = 0x3b200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7faddea00000 (size = 0x400000) > EAL: Ask a virtual area of 0x7c00000 bytes > EAL: Virtual area found at 0x7fadd6c00000 (size = 0x7c00000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fadd6800000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fadd6400000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fadd6000000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fadd5c00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fadd5800000 (size = 0x200000) > EAL: Ask a virtual area of 0x9800000 bytes > EAL: Virtual area found at 0x7fadcbe00000 (size = 0x9800000) > EAL: Ask a virtual area of 0x19000000 bytes > EAL: Virtual area found at 0x7fadb2c00000 (size = 0x19000000) > EAL: Ask a virtual area of 0x24400000 bytes > EAL: Virtual area found at 0x7fad8e600000 (size = 0x24400000) > EAL: Ask a virtual area of 0xc800000 bytes > EAL: Virtual area found at 0x7fad81c00000 (size = 0xc800000) > EAL: Ask a virtual area of 0x32000000 bytes > EAL: Virtual area found at 0x7fad4fa00000 (size = 0x32000000) > EAL: Ask a virtual area of 0x3db800000 bytes > EAL: Virtual area found at 0x7fa974000000 (size = 0x3db800000) > EAL: Ask a virtual area of 0xfa000000 bytes > EAL: Virtual area found at 0x7fa879e00000 (size = 0xfa000000) > EAL: Ask a virtual area of 0x168400000 bytes > EAL: Virtual area found at 0x7fa711800000 (size = 0x168400000) > EAL: Ask a virtual area of 0x7d000000 bytes > EAL: Virtual area found at 0x7fa694600000 (size = 0x7d000000) > EAL: Ask a virtual area of 0x1f4000000 bytes > EAL: Virtual area found at 0x7fa4a0400000 (size = 0x1f4000000) > EAL: Ask a virtual area of 0x2e400000 bytes > EAL: Virtual area found at 0x7fa471e00000 (size = 0x2e400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa471a00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa471600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa471200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa470e00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa470a00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa470600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa470200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46fe00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46fa00000 (size = 0x200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fa46f400000 (size = 0x400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46f000000 (size = 0x200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fa46ea00000 (size = 0x400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46e600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46e200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46de00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46da00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46d600000 (size = 0x200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fa46d000000 (size = 0x400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46cc00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46c800000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46c400000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46c000000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46bc00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46b800000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46b400000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46b000000 (size = 0x200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fa46aa00000 (size = 0x400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46a600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46a200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa469e00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa469a00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa469600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa469200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa468e00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa468a00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa468600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa468200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa467e00000 (size = 0x200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fa467800000 (size = 0x400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa467400000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa467000000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa466c00000 (size = 0x200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fa466600000 (size = 0x400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa466200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa465e00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa465a00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa465600000 (size = 0x200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fa465000000 (size = 0x400000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fa464a00000 (size = 0x400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa464600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa464200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa463e00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa463a00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa463600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa463200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa462e00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa462a00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa462600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa462200000 (size = 0x200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fa461c00000 (size = 0x400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa461800000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa461400000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa461000000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa460c00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa460800000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa460400000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa460000000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45fc00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45f800000 (size = 0x200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fa45f200000 (size = 0x400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45ee00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45ea00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45e600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45e200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45de00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45da00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45d600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45d200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45ce00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45ca00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45c600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45c200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45be00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45ba00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45b600000 (size = 0x200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fa45b000000 (size = 0x400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45ac00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45a800000 (size = 0x200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fa45a200000 (size = 0x400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa459e00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa459a00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa459600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa459200000 (size = 0x200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fa458c00000 (size = 0x400000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fa458600000 (size = 0x400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa458200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa457e00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa457a00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa457600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa457200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa456e00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa456a00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa456600000 (size = 0x200000) > EAL: Ask a virtual area of 0x139400000 bytes > EAL: Virtual area found at 0x7fa31d000000 (size = 0x139400000) > EAL: Ask a virtual area of 0xe9000000 bytes > EAL: Virtual area found at 0x7fa233e00000 (size = 0xe9000000) > EAL: Ask a virtual area of 0x188000000 bytes > EAL: Virtual area found at 0x7fa0abc00000 (size = 0x188000000) > EAL: map_all_hugepages(): mmap failed: Cannot allocate memory > EAL: Failed to remap 2 MB pages > PANIC in rte_eal_init(): > Cannot init memory > Aborted (core dumped) > > > > ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [dpdk-dev] mmap fails with more than 40000 hugepages 2015-02-05 12:59 ` Neil Horman @ 2015-02-05 13:20 ` Damjan Marion (damarion) 2015-02-05 14:59 ` Neil Horman 0 siblings, 1 reply; 13+ messages in thread From: Damjan Marion (damarion) @ 2015-02-05 13:20 UTC (permalink / raw) To: Neil Horman; +Cc: dev > On 05 Feb 2015, at 13:59, Neil Horman <nhorman@tuxdriver.com> wrote: > > On Thu, Feb 05, 2015 at 12:00:48PM +0000, Damjan Marion (damarion) wrote: >> Hi, >> >> I have system with 2 NUMA nodes and 256G RAM total. I noticed that DPDK crashes in rte_eal_init() >> when number of available hugepages is around 40000 or above. >> Everything works fine with lower values (i.e. 30000). >> >> I also tried with allocating 40000 on node0 and 0 on node1, same crash happens. >> >> >> Any idea what might be causing this? >> >> Thanks, >> >> Damjan >> >> >> $ cat /sys/devices/system/node/node[01]/hugepages/hugepages-2048kB/nr_hugepages >> 20000 >> 20000 >> >> $ grep -i huge /proc/meminfo >> AnonHugePages: 706560 kB >> HugePages_Total: 40000 >> HugePages_Free: 40000 >> HugePages_Rsvd: 0 >> HugePages_Surp: 0 >> Hugepagesize: 2048 kB >> > Whats your shmmax value set to? 40000 2MB hugepages is way above the default > setting for how much shared ram a system will allow. I've not done the math on > your logs below, but judging by the size of some of the mapped segments, I'm > betting your hitting the default limit of 4GB. $ cat /proc/sys/kernel/shmmax 33554432 $ sysctl -w kernel.shmmax=8589934592 kernel.shmmax = 8589934592 same crash :( Thanks, Damjan ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [dpdk-dev] mmap fails with more than 40000 hugepages 2015-02-05 13:20 ` Damjan Marion (damarion) @ 2015-02-05 14:59 ` Neil Horman 2015-02-05 15:41 ` Damjan Marion (damarion) 0 siblings, 1 reply; 13+ messages in thread From: Neil Horman @ 2015-02-05 14:59 UTC (permalink / raw) To: Damjan Marion (damarion); +Cc: dev On Thu, Feb 05, 2015 at 01:20:01PM +0000, Damjan Marion (damarion) wrote: > > > On 05 Feb 2015, at 13:59, Neil Horman <nhorman@tuxdriver.com> wrote: > > > > On Thu, Feb 05, 2015 at 12:00:48PM +0000, Damjan Marion (damarion) wrote: > >> Hi, > >> > >> I have system with 2 NUMA nodes and 256G RAM total. I noticed that DPDK crashes in rte_eal_init() > >> when number of available hugepages is around 40000 or above. > >> Everything works fine with lower values (i.e. 30000). > >> > >> I also tried with allocating 40000 on node0 and 0 on node1, same crash happens. > >> > >> > >> Any idea what might be causing this? > >> > >> Thanks, > >> > >> Damjan > >> > >> > >> $ cat /sys/devices/system/node/node[01]/hugepages/hugepages-2048kB/nr_hugepages > >> 20000 > >> 20000 > >> > >> $ grep -i huge /proc/meminfo > >> AnonHugePages: 706560 kB > >> HugePages_Total: 40000 > >> HugePages_Free: 40000 > >> HugePages_Rsvd: 0 > >> HugePages_Surp: 0 > >> Hugepagesize: 2048 kB > >> > > Whats your shmmax value set to? 40000 2MB hugepages is way above the default > > setting for how much shared ram a system will allow. I've not done the math on > > your logs below, but judging by the size of some of the mapped segments, I'm > > betting your hitting the default limit of 4GB. > > $ cat /proc/sys/kernel/shmmax > 33554432 > > $ sysctl -w kernel.shmmax=8589934592 > kernel.shmmax = 8589934592 > > same crash :( > > Thanks, > > Damjan What about the shmmni and shmmax values. The shmmax value will also need to be set to at least 80G (more if you have other shared memory needs), and shmmni will need to be larger than 40,000 to handle all the segments your creating. Neil ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [dpdk-dev] mmap fails with more than 40000 hugepages 2015-02-05 14:59 ` Neil Horman @ 2015-02-05 15:41 ` Damjan Marion (damarion) 2015-02-05 16:35 ` Neil Horman 0 siblings, 1 reply; 13+ messages in thread From: Damjan Marion (damarion) @ 2015-02-05 15:41 UTC (permalink / raw) To: Neil Horman; +Cc: dev > On 05 Feb 2015, at 15:59, Neil Horman <nhorman@tuxdriver.com> wrote: > > On Thu, Feb 05, 2015 at 01:20:01PM +0000, Damjan Marion (damarion) wrote: >> >>> On 05 Feb 2015, at 13:59, Neil Horman <nhorman@tuxdriver.com> wrote: >>> >>> On Thu, Feb 05, 2015 at 12:00:48PM +0000, Damjan Marion (damarion) wrote: >>>> Hi, >>>> >>>> I have system with 2 NUMA nodes and 256G RAM total. I noticed that DPDK crashes in rte_eal_init() >>>> when number of available hugepages is around 40000 or above. >>>> Everything works fine with lower values (i.e. 30000). >>>> >>>> I also tried with allocating 40000 on node0 and 0 on node1, same crash happens. >>>> >>>> >>>> Any idea what might be causing this? >>>> >>>> Thanks, >>>> >>>> Damjan >>>> >>>> >>>> $ cat /sys/devices/system/node/node[01]/hugepages/hugepages-2048kB/nr_hugepages >>>> 20000 >>>> 20000 >>>> >>>> $ grep -i huge /proc/meminfo >>>> AnonHugePages: 706560 kB >>>> HugePages_Total: 40000 >>>> HugePages_Free: 40000 >>>> HugePages_Rsvd: 0 >>>> HugePages_Surp: 0 >>>> Hugepagesize: 2048 kB >>>> >>> Whats your shmmax value set to? 40000 2MB hugepages is way above the default >>> setting for how much shared ram a system will allow. I've not done the math on >>> your logs below, but judging by the size of some of the mapped segments, I'm >>> betting your hitting the default limit of 4GB. >> >> $ cat /proc/sys/kernel/shmmax >> 33554432 >> >> $ sysctl -w kernel.shmmax=8589934592 >> kernel.shmmax = 8589934592 >> >> same crash :( >> >> Thanks, >> >> Damjan > > What about the shmmni and shmmax values. The shmmax value will also need to be > set to at least 80G (more if you have other shared memory needs), and shmmni > will need to be larger than 40,000 to handle all the segments your creating. > Neil Hmm, if that is the reason, then 30000 will also not work. as both values are set by default to much much lower value. Anyway, I tried: $ sysctl kernel.shmmni kernel.shmmni = 100000 $ sysctl kernel.shmmax kernel.shmmax = 274877906944 $ cat /sys/devices/system/node/node[01]/hugepages/hugepages-2048kB/nr_hugepages 20000 20000 $ sudo ~/src/dpdk/x86_64-native-linuxapp-gcc/app/testpmd -l 5-7 -n 3 --socket-mem 512,512 EAL: Detected lcore 0 as core 0 on socket 0 EAL: Detected lcore 1 as core 1 on socket 0 EAL: Detected lcore 2 as core 2 on socket 0 EAL: Detected lcore 3 as core 3 on socket 0 EAL: Detected lcore 4 as core 4 on socket 0 EAL: Detected lcore 5 as core 5 on socket 0 EAL: Detected lcore 6 as core 6 on socket 0 EAL: Detected lcore 7 as core 7 on socket 0 EAL: Detected lcore 8 as core 0 on socket 1 EAL: Detected lcore 9 as core 1 on socket 1 EAL: Detected lcore 10 as core 2 on socket 1 EAL: Detected lcore 11 as core 3 on socket 1 EAL: Detected lcore 12 as core 4 on socket 1 EAL: Detected lcore 13 as core 5 on socket 1 EAL: Detected lcore 14 as core 6 on socket 1 EAL: Detected lcore 15 as core 7 on socket 1 EAL: Detected lcore 16 as core 0 on socket 0 EAL: Detected lcore 17 as core 1 on socket 0 EAL: Detected lcore 18 as core 2 on socket 0 EAL: Detected lcore 19 as core 3 on socket 0 EAL: Detected lcore 20 as core 4 on socket 0 EAL: Detected lcore 21 as core 5 on socket 0 EAL: Detected lcore 22 as core 6 on socket 0 EAL: Detected lcore 23 as core 7 on socket 0 EAL: Detected lcore 24 as core 0 on socket 1 EAL: Detected lcore 25 as core 1 on socket 1 EAL: Detected lcore 26 as core 2 on socket 1 EAL: Detected lcore 27 as core 3 on socket 1 EAL: Detected lcore 28 as core 4 on socket 1 EAL: Detected lcore 29 as core 5 on socket 1 EAL: Detected lcore 30 as core 6 on socket 1 EAL: Detected lcore 31 as core 7 on socket 1 EAL: Support maximum 128 logical core(s) by configuration. EAL: Detected 32 lcore(s) EAL: VFIO modules not all loaded, skip VFIO support... EAL: Setting up memory... EAL: Ask a virtual area of 0x80000000 bytes EAL: Virtual area found at 0x7fe280000000 (size = 0x80000000) EAL: Ask a virtual area of 0x80000000 bytes EAL: Virtual area found at 0x7fe1c0000000 (size = 0x80000000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe335c00000 (size = 0x200000) EAL: Ask a virtual area of 0x9400000 bytes EAL: Virtual area found at 0x7fe32c600000 (size = 0x9400000) EAL: Ask a virtual area of 0x5c00000 bytes EAL: Virtual area found at 0x7fe326800000 (size = 0x5c00000) EAL: Ask a virtual area of 0x38e00000 bytes EAL: Virtual area found at 0x7fe247000000 (size = 0x38e00000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe326400000 (size = 0x200000) EAL: Ask a virtual area of 0x8000000 bytes EAL: Virtual area found at 0x7fe31e200000 (size = 0x8000000) EAL: Ask a virtual area of 0x815800000 bytes EAL: Virtual area found at 0x7fd9aa600000 (size = 0x815800000) EAL: Ask a virtual area of 0x1f400000 bytes EAL: Virtual area found at 0x7fd98b000000 (size = 0x1f400000) EAL: Ask a virtual area of 0xfd400000 bytes EAL: Virtual area found at 0x7fd88da00000 (size = 0xfd400000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe31de00000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe31da00000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe31d600000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe31d200000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe31ce00000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe31ca00000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe31c600000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe31c200000 (size = 0x200000) EAL: Ask a virtual area of 0x400000 bytes EAL: Virtual area found at 0x7fe31bc00000 (size = 0x400000) EAL: Ask a virtual area of 0x400000 bytes EAL: Virtual area found at 0x7fe31b600000 (size = 0x400000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe31b200000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe31ae00000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe31aa00000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe31a600000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe31a200000 (size = 0x200000) EAL: Ask a virtual area of 0x400000 bytes EAL: Virtual area found at 0x7fe319c00000 (size = 0x400000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe319800000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe319400000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe319000000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe318c00000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe318800000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe318400000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe318000000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe317c00000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe317800000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe317400000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe317000000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe316c00000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe316800000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe316400000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe316000000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe315c00000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe315800000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe315400000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe315000000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe314c00000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe314800000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe314400000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe314000000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe313c00000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe313800000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe313400000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe313000000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe312c00000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe312800000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe312400000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe312000000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe311c00000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe311800000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe311400000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe311000000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe310c00000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe310800000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe310400000 (size = 0x200000) EAL: Ask a virtual area of 0x400000 bytes EAL: Virtual area found at 0x7fe30fe00000 (size = 0x400000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe30fa00000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe30f600000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe30f200000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe30ee00000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe30ea00000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe30e600000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe30e200000 (size = 0x200000) EAL: Ask a virtual area of 0x400000 bytes EAL: Virtual area found at 0x7fe30dc00000 (size = 0x400000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe30d800000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe30d400000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe30d000000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe30cc00000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe30c800000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe30c400000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe30c000000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe30bc00000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe30b800000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe30b400000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe30b000000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe30ac00000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe30a800000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe30a400000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe30a000000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe309c00000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe309800000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe309400000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe309000000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe308c00000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fe308800000 (size = 0x200000) EAL: Ask a virtual area of 0x2f800000 bytes EAL: Virtual area found at 0x7fd85e000000 (size = 0x2f800000) EAL: Ask a virtual area of 0x7400000 bytes EAL: Virtual area found at 0x7fe301200000 (size = 0x7400000) EAL: Ask a virtual area of 0x9b5400000 bytes EAL: Virtual area found at 0x7fcea8a00000 (size = 0x9b5400000) EAL: map_all_hugepages(): mmap failed: Cannot allocate memory EAL: Failed to remap 2 MB pages PANIC in rte_eal_init(): Cannot init memory ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [dpdk-dev] mmap fails with more than 40000 hugepages 2015-02-05 15:41 ` Damjan Marion (damarion) @ 2015-02-05 16:35 ` Neil Horman 0 siblings, 0 replies; 13+ messages in thread From: Neil Horman @ 2015-02-05 16:35 UTC (permalink / raw) To: Damjan Marion (damarion); +Cc: dev On Thu, Feb 05, 2015 at 03:41:25PM +0000, Damjan Marion (damarion) wrote: > > > On 05 Feb 2015, at 15:59, Neil Horman <nhorman@tuxdriver.com> wrote: > > > > On Thu, Feb 05, 2015 at 01:20:01PM +0000, Damjan Marion (damarion) wrote: > >> > >>> On 05 Feb 2015, at 13:59, Neil Horman <nhorman@tuxdriver.com> wrote: > >>> > >>> On Thu, Feb 05, 2015 at 12:00:48PM +0000, Damjan Marion (damarion) wrote: > >>>> Hi, > >>>> > >>>> I have system with 2 NUMA nodes and 256G RAM total. I noticed that DPDK crashes in rte_eal_init() > >>>> when number of available hugepages is around 40000 or above. > >>>> Everything works fine with lower values (i.e. 30000). > >>>> > >>>> I also tried with allocating 40000 on node0 and 0 on node1, same crash happens. > >>>> > >>>> > >>>> Any idea what might be causing this? > >>>> > >>>> Thanks, > >>>> > >>>> Damjan > >>>> > >>>> > >>>> $ cat /sys/devices/system/node/node[01]/hugepages/hugepages-2048kB/nr_hugepages > >>>> 20000 > >>>> 20000 > >>>> > >>>> $ grep -i huge /proc/meminfo > >>>> AnonHugePages: 706560 kB > >>>> HugePages_Total: 40000 > >>>> HugePages_Free: 40000 > >>>> HugePages_Rsvd: 0 > >>>> HugePages_Surp: 0 > >>>> Hugepagesize: 2048 kB > >>>> > >>> Whats your shmmax value set to? 40000 2MB hugepages is way above the default > >>> setting for how much shared ram a system will allow. I've not done the math on > >>> your logs below, but judging by the size of some of the mapped segments, I'm > >>> betting your hitting the default limit of 4GB. > >> > >> $ cat /proc/sys/kernel/shmmax > >> 33554432 > >> > >> $ sysctl -w kernel.shmmax=8589934592 > >> kernel.shmmax = 8589934592 > >> > >> same crash :( > >> > >> Thanks, > >> > >> Damjan > > > > What about the shmmni and shmmax values. The shmmax value will also need to be > > set to at least 80G (more if you have other shared memory needs), and shmmni > > will need to be larger than 40,000 to handle all the segments your creating. > > Neil > > Hmm, if that is the reason, then 30000 will also not work. as both values are set by default to much much lower value. > > Anyway, I tried: > > $ sysctl kernel.shmmni > kernel.shmmni = 100000 > > $ sysctl kernel.shmmax > kernel.shmmax = 274877906944 > > $ cat /sys/devices/system/node/node[01]/hugepages/hugepages-2048kB/nr_hugepages > 20000 > 20000 > Hmm, then you probably need to start looking at other failure possibilities. The segment that failed was requesting a size of about 38Gb. Is it possible that your virtual memory map doesn't actually have a hole that large? That would indicate a bug in the eal init path, since it searches for that. Time to start adding some printf and dumping your /proc/pid/maps file Neil > $ sudo ~/src/dpdk/x86_64-native-linuxapp-gcc/app/testpmd -l 5-7 -n 3 --socket-mem 512,512 > EAL: Detected lcore 0 as core 0 on socket 0 > EAL: Detected lcore 1 as core 1 on socket 0 > EAL: Detected lcore 2 as core 2 on socket 0 > EAL: Detected lcore 3 as core 3 on socket 0 > EAL: Detected lcore 4 as core 4 on socket 0 > EAL: Detected lcore 5 as core 5 on socket 0 > EAL: Detected lcore 6 as core 6 on socket 0 > EAL: Detected lcore 7 as core 7 on socket 0 > EAL: Detected lcore 8 as core 0 on socket 1 > EAL: Detected lcore 9 as core 1 on socket 1 > EAL: Detected lcore 10 as core 2 on socket 1 > EAL: Detected lcore 11 as core 3 on socket 1 > EAL: Detected lcore 12 as core 4 on socket 1 > EAL: Detected lcore 13 as core 5 on socket 1 > EAL: Detected lcore 14 as core 6 on socket 1 > EAL: Detected lcore 15 as core 7 on socket 1 > EAL: Detected lcore 16 as core 0 on socket 0 > EAL: Detected lcore 17 as core 1 on socket 0 > EAL: Detected lcore 18 as core 2 on socket 0 > EAL: Detected lcore 19 as core 3 on socket 0 > EAL: Detected lcore 20 as core 4 on socket 0 > EAL: Detected lcore 21 as core 5 on socket 0 > EAL: Detected lcore 22 as core 6 on socket 0 > EAL: Detected lcore 23 as core 7 on socket 0 > EAL: Detected lcore 24 as core 0 on socket 1 > EAL: Detected lcore 25 as core 1 on socket 1 > EAL: Detected lcore 26 as core 2 on socket 1 > EAL: Detected lcore 27 as core 3 on socket 1 > EAL: Detected lcore 28 as core 4 on socket 1 > EAL: Detected lcore 29 as core 5 on socket 1 > EAL: Detected lcore 30 as core 6 on socket 1 > EAL: Detected lcore 31 as core 7 on socket 1 > EAL: Support maximum 128 logical core(s) by configuration. > EAL: Detected 32 lcore(s) > EAL: VFIO modules not all loaded, skip VFIO support... > EAL: Setting up memory... > EAL: Ask a virtual area of 0x80000000 bytes > EAL: Virtual area found at 0x7fe280000000 (size = 0x80000000) > EAL: Ask a virtual area of 0x80000000 bytes > EAL: Virtual area found at 0x7fe1c0000000 (size = 0x80000000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe335c00000 (size = 0x200000) > EAL: Ask a virtual area of 0x9400000 bytes > EAL: Virtual area found at 0x7fe32c600000 (size = 0x9400000) > EAL: Ask a virtual area of 0x5c00000 bytes > EAL: Virtual area found at 0x7fe326800000 (size = 0x5c00000) > EAL: Ask a virtual area of 0x38e00000 bytes > EAL: Virtual area found at 0x7fe247000000 (size = 0x38e00000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe326400000 (size = 0x200000) > EAL: Ask a virtual area of 0x8000000 bytes > EAL: Virtual area found at 0x7fe31e200000 (size = 0x8000000) > EAL: Ask a virtual area of 0x815800000 bytes > EAL: Virtual area found at 0x7fd9aa600000 (size = 0x815800000) > EAL: Ask a virtual area of 0x1f400000 bytes > EAL: Virtual area found at 0x7fd98b000000 (size = 0x1f400000) > EAL: Ask a virtual area of 0xfd400000 bytes > EAL: Virtual area found at 0x7fd88da00000 (size = 0xfd400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe31de00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe31da00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe31d600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe31d200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe31ce00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe31ca00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe31c600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe31c200000 (size = 0x200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fe31bc00000 (size = 0x400000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fe31b600000 (size = 0x400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe31b200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe31ae00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe31aa00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe31a600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe31a200000 (size = 0x200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fe319c00000 (size = 0x400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe319800000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe319400000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe319000000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe318c00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe318800000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe318400000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe318000000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe317c00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe317800000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe317400000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe317000000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe316c00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe316800000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe316400000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe316000000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe315c00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe315800000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe315400000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe315000000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe314c00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe314800000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe314400000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe314000000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe313c00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe313800000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe313400000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe313000000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe312c00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe312800000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe312400000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe312000000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe311c00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe311800000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe311400000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe311000000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe310c00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe310800000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe310400000 (size = 0x200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fe30fe00000 (size = 0x400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe30fa00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe30f600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe30f200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe30ee00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe30ea00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe30e600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe30e200000 (size = 0x200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fe30dc00000 (size = 0x400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe30d800000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe30d400000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe30d000000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe30cc00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe30c800000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe30c400000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe30c000000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe30bc00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe30b800000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe30b400000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe30b000000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe30ac00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe30a800000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe30a400000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe30a000000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe309c00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe309800000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe309400000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe309000000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe308c00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fe308800000 (size = 0x200000) > EAL: Ask a virtual area of 0x2f800000 bytes > EAL: Virtual area found at 0x7fd85e000000 (size = 0x2f800000) > EAL: Ask a virtual area of 0x7400000 bytes > EAL: Virtual area found at 0x7fe301200000 (size = 0x7400000) > EAL: Ask a virtual area of 0x9b5400000 bytes > EAL: Virtual area found at 0x7fcea8a00000 (size = 0x9b5400000) > EAL: map_all_hugepages(): mmap failed: Cannot allocate memory > EAL: Failed to remap 2 MB pages > PANIC in rte_eal_init(): > Cannot init memory > > > ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [dpdk-dev] mmap fails with more than 40000 hugepages 2015-02-05 12:00 [dpdk-dev] mmap fails with more than 40000 hugepages Damjan Marion (damarion) 2015-02-05 12:59 ` Neil Horman @ 2015-02-05 13:22 ` Jay Rolette 2015-02-05 13:36 ` Damjan Marion (damarion) 2015-02-06 1:46 ` Linhaifeng 2015-02-06 2:04 ` Linhaifeng 3 siblings, 1 reply; 13+ messages in thread From: Jay Rolette @ 2015-02-05 13:22 UTC (permalink / raw) To: Damjan Marion (damarion); +Cc: dev On Thu, Feb 5, 2015 at 6:00 AM, Damjan Marion (damarion) <damarion@cisco.com > wrote: > Hi, > > I have system with 2 NUMA nodes and 256G RAM total. I noticed that DPDK > crashes in rte_eal_init() > when number of available hugepages is around 40000 or above. > Everything works fine with lower values (i.e. 30000). > > I also tried with allocating 40000 on node0 and 0 on node1, same crash > happens. > > > Any idea what might be causing this? > Any reason you can't switch to using 1GB hugepages? You'll get better performance and your init time will be shorter. The systems we run on are similar (256GB, 2 NUMA nodes) and that works fine for us. Not directly related, but if you have to stick with 2MB hugepages, you might want to take a look at a patch I submitted that fixes the O(n^2) algorithm used in initializing hugepages. Jay ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [dpdk-dev] mmap fails with more than 40000 hugepages 2015-02-05 13:22 ` Jay Rolette @ 2015-02-05 13:36 ` Damjan Marion (damarion) 2015-02-05 15:09 ` Jay Rolette 0 siblings, 1 reply; 13+ messages in thread From: Damjan Marion (damarion) @ 2015-02-05 13:36 UTC (permalink / raw) To: Jay Rolette; +Cc: dev On 05 Feb 2015, at 14:22, Jay Rolette <rolette@infiniteio.com<mailto:rolette@infiniteio.com>> wrote: On Thu, Feb 5, 2015 at 6:00 AM, Damjan Marion (damarion) <damarion@cisco.com<mailto:damarion@cisco.com>> wrote: Hi, I have system with 2 NUMA nodes and 256G RAM total. I noticed that DPDK crashes in rte_eal_init() when number of available hugepages is around 40000 or above. Everything works fine with lower values (i.e. 30000). I also tried with allocating 40000 on node0 and 0 on node1, same crash happens. Any idea what might be causing this? Any reason you can't switch to using 1GB hugepages? You'll get better performance and your init time will be shorter. The systems we run on are similar (256GB, 2 NUMA nodes) and that works fine for us. Yes, unfortunately some other consumers are needing smaller pages Not directly related, but if you have to stick with 2MB hugepages, you might want to take a look at a patch I submitted that fixes the O(n^2) algorithm used in initializing hugepages. I tried it hoping that it will change something, no luck… Thanks, Damjan ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [dpdk-dev] mmap fails with more than 40000 hugepages 2015-02-05 13:36 ` Damjan Marion (damarion) @ 2015-02-05 15:09 ` Jay Rolette 0 siblings, 0 replies; 13+ messages in thread From: Jay Rolette @ 2015-02-05 15:09 UTC (permalink / raw) To: Damjan Marion (damarion); +Cc: dev On Thu, Feb 5, 2015 at 7:36 AM, Damjan Marion (damarion) <damarion@cisco.com > wrote: > > On 05 Feb 2015, at 14:22, Jay Rolette <rolette@infiniteio.com> wrote: > > Not directly related, but if you have to stick with 2MB hugepages, you > might want to take a look at a patch I submitted that fixes the O(n^2) > algorithm used in initializing hugepages. > > > I tried it hoping that it will change something, no luck… > I suggested that patch only to speed up your init time. There's nothing in it that will help with the crash you are seeing. ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [dpdk-dev] mmap fails with more than 40000 hugepages 2015-02-05 12:00 [dpdk-dev] mmap fails with more than 40000 hugepages Damjan Marion (damarion) 2015-02-05 12:59 ` Neil Horman 2015-02-05 13:22 ` Jay Rolette @ 2015-02-06 1:46 ` Linhaifeng 2015-02-06 2:04 ` Linhaifeng 3 siblings, 0 replies; 13+ messages in thread From: Linhaifeng @ 2015-02-06 1:46 UTC (permalink / raw) To: Damjan Marion (damarion), dev On 2015/2/5 20:00, Damjan Marion (damarion) wrote: > Hi, > > I have system with 2 NUMA nodes and 256G RAM total. I noticed that DPDK crashes in rte_eal_init() > when number of available hugepages is around 40000 or above. > Everything works fine with lower values (i.e. 30000). > > I also tried with allocating 40000 on node0 and 0 on node1, same crash happens. > > > Any idea what might be causing this? > > Thanks, > > Damjan > Is there any other process trying to use hugepages? > > $ cat /sys/devices/system/node/node[01]/hugepages/hugepages-2048kB/nr_hugepages > 20000 > 20000 > > $ grep -i huge /proc/meminfo > AnonHugePages: 706560 kB > HugePages_Total: 40000 > HugePages_Free: 40000 > HugePages_Rsvd: 0 > HugePages_Surp: 0 > Hugepagesize: 2048 kB > > > $ sudo ~/src/dpdk/x86_64-native-linuxapp-gcc/app/testpmd -l 5-7 -n 3 --socket-mem 512,512 > EAL: Detected lcore 0 as core 0 on socket 0 > EAL: Detected lcore 1 as core 1 on socket 0 > EAL: Detected lcore 2 as core 2 on socket 0 > EAL: Detected lcore 3 as core 3 on socket 0 > EAL: Detected lcore 4 as core 4 on socket 0 > EAL: Detected lcore 5 as core 5 on socket 0 > EAL: Detected lcore 6 as core 6 on socket 0 > EAL: Detected lcore 7 as core 7 on socket 0 > EAL: Detected lcore 8 as core 0 on socket 1 > EAL: Detected lcore 9 as core 1 on socket 1 > EAL: Detected lcore 10 as core 2 on socket 1 > EAL: Detected lcore 11 as core 3 on socket 1 > EAL: Detected lcore 12 as core 4 on socket 1 > EAL: Detected lcore 13 as core 5 on socket 1 > EAL: Detected lcore 14 as core 6 on socket 1 > EAL: Detected lcore 15 as core 7 on socket 1 > EAL: Detected lcore 16 as core 0 on socket 0 > EAL: Detected lcore 17 as core 1 on socket 0 > EAL: Detected lcore 18 as core 2 on socket 0 > EAL: Detected lcore 19 as core 3 on socket 0 > EAL: Detected lcore 20 as core 4 on socket 0 > EAL: Detected lcore 21 as core 5 on socket 0 > EAL: Detected lcore 22 as core 6 on socket 0 > EAL: Detected lcore 23 as core 7 on socket 0 > EAL: Detected lcore 24 as core 0 on socket 1 > EAL: Detected lcore 25 as core 1 on socket 1 > EAL: Detected lcore 26 as core 2 on socket 1 > EAL: Detected lcore 27 as core 3 on socket 1 > EAL: Detected lcore 28 as core 4 on socket 1 > EAL: Detected lcore 29 as core 5 on socket 1 > EAL: Detected lcore 30 as core 6 on socket 1 > EAL: Detected lcore 31 as core 7 on socket 1 > EAL: Support maximum 128 logical core(s) by configuration. > EAL: Detected 32 lcore(s) > EAL: VFIO modules not all loaded, skip VFIO support... > EAL: Setting up memory... > EAL: Ask a virtual area of 0x800000 bytes > EAL: Virtual area found at 0x7fae2a200000 (size = 0x800000) > EAL: Ask a virtual area of 0x7600000 bytes > EAL: Virtual area found at 0x7fae22a00000 (size = 0x7600000) > EAL: Ask a virtual area of 0x1400000 bytes > EAL: Virtual area found at 0x7fae21400000 (size = 0x1400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fae21000000 (size = 0x200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fae20a00000 (size = 0x400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fae20600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fae20200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fae1fe00000 (size = 0x200000) > EAL: Ask a virtual area of 0x5800000 bytes > EAL: Virtual area found at 0x7fae1a400000 (size = 0x5800000) > EAL: Ask a virtual area of 0x3b200000 bytes > EAL: Virtual area found at 0x7faddf000000 (size = 0x3b200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7faddea00000 (size = 0x400000) > EAL: Ask a virtual area of 0x7c00000 bytes > EAL: Virtual area found at 0x7fadd6c00000 (size = 0x7c00000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fadd6800000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fadd6400000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fadd6000000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fadd5c00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fadd5800000 (size = 0x200000) > EAL: Ask a virtual area of 0x9800000 bytes > EAL: Virtual area found at 0x7fadcbe00000 (size = 0x9800000) > EAL: Ask a virtual area of 0x19000000 bytes > EAL: Virtual area found at 0x7fadb2c00000 (size = 0x19000000) > EAL: Ask a virtual area of 0x24400000 bytes > EAL: Virtual area found at 0x7fad8e600000 (size = 0x24400000) > EAL: Ask a virtual area of 0xc800000 bytes > EAL: Virtual area found at 0x7fad81c00000 (size = 0xc800000) > EAL: Ask a virtual area of 0x32000000 bytes > EAL: Virtual area found at 0x7fad4fa00000 (size = 0x32000000) > EAL: Ask a virtual area of 0x3db800000 bytes > EAL: Virtual area found at 0x7fa974000000 (size = 0x3db800000) > EAL: Ask a virtual area of 0xfa000000 bytes > EAL: Virtual area found at 0x7fa879e00000 (size = 0xfa000000) > EAL: Ask a virtual area of 0x168400000 bytes > EAL: Virtual area found at 0x7fa711800000 (size = 0x168400000) > EAL: Ask a virtual area of 0x7d000000 bytes > EAL: Virtual area found at 0x7fa694600000 (size = 0x7d000000) > EAL: Ask a virtual area of 0x1f4000000 bytes > EAL: Virtual area found at 0x7fa4a0400000 (size = 0x1f4000000) > EAL: Ask a virtual area of 0x2e400000 bytes > EAL: Virtual area found at 0x7fa471e00000 (size = 0x2e400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa471a00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa471600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa471200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa470e00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa470a00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa470600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa470200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46fe00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46fa00000 (size = 0x200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fa46f400000 (size = 0x400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46f000000 (size = 0x200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fa46ea00000 (size = 0x400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46e600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46e200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46de00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46da00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46d600000 (size = 0x200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fa46d000000 (size = 0x400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46cc00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46c800000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46c400000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46c000000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46bc00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46b800000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46b400000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46b000000 (size = 0x200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fa46aa00000 (size = 0x400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46a600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46a200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa469e00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa469a00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa469600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa469200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa468e00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa468a00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa468600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa468200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa467e00000 (size = 0x200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fa467800000 (size = 0x400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa467400000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa467000000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa466c00000 (size = 0x200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fa466600000 (size = 0x400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa466200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa465e00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa465a00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa465600000 (size = 0x200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fa465000000 (size = 0x400000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fa464a00000 (size = 0x400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa464600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa464200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa463e00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa463a00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa463600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa463200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa462e00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa462a00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa462600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa462200000 (size = 0x200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fa461c00000 (size = 0x400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa461800000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa461400000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa461000000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa460c00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa460800000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa460400000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa460000000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45fc00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45f800000 (size = 0x200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fa45f200000 (size = 0x400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45ee00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45ea00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45e600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45e200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45de00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45da00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45d600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45d200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45ce00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45ca00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45c600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45c200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45be00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45ba00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45b600000 (size = 0x200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fa45b000000 (size = 0x400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45ac00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45a800000 (size = 0x200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fa45a200000 (size = 0x400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa459e00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa459a00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa459600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa459200000 (size = 0x200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fa458c00000 (size = 0x400000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fa458600000 (size = 0x400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa458200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa457e00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa457a00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa457600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa457200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa456e00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa456a00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa456600000 (size = 0x200000) > EAL: Ask a virtual area of 0x139400000 bytes > EAL: Virtual area found at 0x7fa31d000000 (size = 0x139400000) > EAL: Ask a virtual area of 0xe9000000 bytes > EAL: Virtual area found at 0x7fa233e00000 (size = 0xe9000000) > EAL: Ask a virtual area of 0x188000000 bytes > EAL: Virtual area found at 0x7fa0abc00000 (size = 0x188000000) > EAL: map_all_hugepages(): mmap failed: Cannot allocate memory > EAL: Failed to remap 2 MB pages > PANIC in rte_eal_init(): > Cannot init memory > Aborted (core dumped) > > > > > . > -- Regards, Haifeng ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [dpdk-dev] mmap fails with more than 40000 hugepages 2015-02-05 12:00 [dpdk-dev] mmap fails with more than 40000 hugepages Damjan Marion (damarion) ` (2 preceding siblings ...) 2015-02-06 1:46 ` Linhaifeng @ 2015-02-06 2:04 ` Linhaifeng 2015-02-06 10:26 ` De Lara Guarch, Pablo 3 siblings, 1 reply; 13+ messages in thread From: Linhaifeng @ 2015-02-06 2:04 UTC (permalink / raw) To: Damjan Marion (damarion), dev On 2015/2/5 20:00, Damjan Marion (damarion) wrote: > Hi, > > I have system with 2 NUMA nodes and 256G RAM total. I noticed that DPDK crashes in rte_eal_init() > when number of available hugepages is around 40000 or above. > Everything works fine with lower values (i.e. 30000). > > I also tried with allocating 40000 on node0 and 0 on node1, same crash happens. > > > Any idea what might be causing this? > > Thanks, > > Damjan > cat /proc/sys/vm/max_map_count > > $ cat /sys/devices/system/node/node[01]/hugepages/hugepages-2048kB/nr_hugepages > 20000 > 20000 > > $ grep -i huge /proc/meminfo > AnonHugePages: 706560 kB > HugePages_Total: 40000 > HugePages_Free: 40000 > HugePages_Rsvd: 0 > HugePages_Surp: 0 > Hugepagesize: 2048 kB > > > $ sudo ~/src/dpdk/x86_64-native-linuxapp-gcc/app/testpmd -l 5-7 -n 3 --socket-mem 512,512 > EAL: Detected lcore 0 as core 0 on socket 0 > EAL: Detected lcore 1 as core 1 on socket 0 > EAL: Detected lcore 2 as core 2 on socket 0 > EAL: Detected lcore 3 as core 3 on socket 0 > EAL: Detected lcore 4 as core 4 on socket 0 > EAL: Detected lcore 5 as core 5 on socket 0 > EAL: Detected lcore 6 as core 6 on socket 0 > EAL: Detected lcore 7 as core 7 on socket 0 > EAL: Detected lcore 8 as core 0 on socket 1 > EAL: Detected lcore 9 as core 1 on socket 1 > EAL: Detected lcore 10 as core 2 on socket 1 > EAL: Detected lcore 11 as core 3 on socket 1 > EAL: Detected lcore 12 as core 4 on socket 1 > EAL: Detected lcore 13 as core 5 on socket 1 > EAL: Detected lcore 14 as core 6 on socket 1 > EAL: Detected lcore 15 as core 7 on socket 1 > EAL: Detected lcore 16 as core 0 on socket 0 > EAL: Detected lcore 17 as core 1 on socket 0 > EAL: Detected lcore 18 as core 2 on socket 0 > EAL: Detected lcore 19 as core 3 on socket 0 > EAL: Detected lcore 20 as core 4 on socket 0 > EAL: Detected lcore 21 as core 5 on socket 0 > EAL: Detected lcore 22 as core 6 on socket 0 > EAL: Detected lcore 23 as core 7 on socket 0 > EAL: Detected lcore 24 as core 0 on socket 1 > EAL: Detected lcore 25 as core 1 on socket 1 > EAL: Detected lcore 26 as core 2 on socket 1 > EAL: Detected lcore 27 as core 3 on socket 1 > EAL: Detected lcore 28 as core 4 on socket 1 > EAL: Detected lcore 29 as core 5 on socket 1 > EAL: Detected lcore 30 as core 6 on socket 1 > EAL: Detected lcore 31 as core 7 on socket 1 > EAL: Support maximum 128 logical core(s) by configuration. > EAL: Detected 32 lcore(s) > EAL: VFIO modules not all loaded, skip VFIO support... > EAL: Setting up memory... > EAL: Ask a virtual area of 0x800000 bytes > EAL: Virtual area found at 0x7fae2a200000 (size = 0x800000) > EAL: Ask a virtual area of 0x7600000 bytes > EAL: Virtual area found at 0x7fae22a00000 (size = 0x7600000) > EAL: Ask a virtual area of 0x1400000 bytes > EAL: Virtual area found at 0x7fae21400000 (size = 0x1400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fae21000000 (size = 0x200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fae20a00000 (size = 0x400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fae20600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fae20200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fae1fe00000 (size = 0x200000) > EAL: Ask a virtual area of 0x5800000 bytes > EAL: Virtual area found at 0x7fae1a400000 (size = 0x5800000) > EAL: Ask a virtual area of 0x3b200000 bytes > EAL: Virtual area found at 0x7faddf000000 (size = 0x3b200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7faddea00000 (size = 0x400000) > EAL: Ask a virtual area of 0x7c00000 bytes > EAL: Virtual area found at 0x7fadd6c00000 (size = 0x7c00000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fadd6800000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fadd6400000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fadd6000000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fadd5c00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fadd5800000 (size = 0x200000) > EAL: Ask a virtual area of 0x9800000 bytes > EAL: Virtual area found at 0x7fadcbe00000 (size = 0x9800000) > EAL: Ask a virtual area of 0x19000000 bytes > EAL: Virtual area found at 0x7fadb2c00000 (size = 0x19000000) > EAL: Ask a virtual area of 0x24400000 bytes > EAL: Virtual area found at 0x7fad8e600000 (size = 0x24400000) > EAL: Ask a virtual area of 0xc800000 bytes > EAL: Virtual area found at 0x7fad81c00000 (size = 0xc800000) > EAL: Ask a virtual area of 0x32000000 bytes > EAL: Virtual area found at 0x7fad4fa00000 (size = 0x32000000) > EAL: Ask a virtual area of 0x3db800000 bytes > EAL: Virtual area found at 0x7fa974000000 (size = 0x3db800000) > EAL: Ask a virtual area of 0xfa000000 bytes > EAL: Virtual area found at 0x7fa879e00000 (size = 0xfa000000) > EAL: Ask a virtual area of 0x168400000 bytes > EAL: Virtual area found at 0x7fa711800000 (size = 0x168400000) > EAL: Ask a virtual area of 0x7d000000 bytes > EAL: Virtual area found at 0x7fa694600000 (size = 0x7d000000) > EAL: Ask a virtual area of 0x1f4000000 bytes > EAL: Virtual area found at 0x7fa4a0400000 (size = 0x1f4000000) > EAL: Ask a virtual area of 0x2e400000 bytes > EAL: Virtual area found at 0x7fa471e00000 (size = 0x2e400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa471a00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa471600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa471200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa470e00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa470a00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa470600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa470200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46fe00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46fa00000 (size = 0x200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fa46f400000 (size = 0x400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46f000000 (size = 0x200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fa46ea00000 (size = 0x400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46e600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46e200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46de00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46da00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46d600000 (size = 0x200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fa46d000000 (size = 0x400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46cc00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46c800000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46c400000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46c000000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46bc00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46b800000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46b400000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46b000000 (size = 0x200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fa46aa00000 (size = 0x400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46a600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa46a200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa469e00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa469a00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa469600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa469200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa468e00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa468a00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa468600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa468200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa467e00000 (size = 0x200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fa467800000 (size = 0x400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa467400000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa467000000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa466c00000 (size = 0x200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fa466600000 (size = 0x400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa466200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa465e00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa465a00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa465600000 (size = 0x200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fa465000000 (size = 0x400000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fa464a00000 (size = 0x400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa464600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa464200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa463e00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa463a00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa463600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa463200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa462e00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa462a00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa462600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa462200000 (size = 0x200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fa461c00000 (size = 0x400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa461800000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa461400000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa461000000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa460c00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa460800000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa460400000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa460000000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45fc00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45f800000 (size = 0x200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fa45f200000 (size = 0x400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45ee00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45ea00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45e600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45e200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45de00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45da00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45d600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45d200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45ce00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45ca00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45c600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45c200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45be00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45ba00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45b600000 (size = 0x200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fa45b000000 (size = 0x400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45ac00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa45a800000 (size = 0x200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fa45a200000 (size = 0x400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa459e00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa459a00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa459600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa459200000 (size = 0x200000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fa458c00000 (size = 0x400000) > EAL: Ask a virtual area of 0x400000 bytes > EAL: Virtual area found at 0x7fa458600000 (size = 0x400000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa458200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa457e00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa457a00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa457600000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa457200000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa456e00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa456a00000 (size = 0x200000) > EAL: Ask a virtual area of 0x200000 bytes > EAL: Virtual area found at 0x7fa456600000 (size = 0x200000) > EAL: Ask a virtual area of 0x139400000 bytes > EAL: Virtual area found at 0x7fa31d000000 (size = 0x139400000) > EAL: Ask a virtual area of 0xe9000000 bytes > EAL: Virtual area found at 0x7fa233e00000 (size = 0xe9000000) > EAL: Ask a virtual area of 0x188000000 bytes > EAL: Virtual area found at 0x7fa0abc00000 (size = 0x188000000) > EAL: map_all_hugepages(): mmap failed: Cannot allocate memory > EAL: Failed to remap 2 MB pages > PANIC in rte_eal_init(): > Cannot init memory > Aborted (core dumped) > > > > > . > -- Regards, Haifeng ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [dpdk-dev] mmap fails with more than 40000 hugepages 2015-02-06 2:04 ` Linhaifeng @ 2015-02-06 10:26 ` De Lara Guarch, Pablo 2015-02-06 10:31 ` Damjan Marion (damarion) 0 siblings, 1 reply; 13+ messages in thread From: De Lara Guarch, Pablo @ 2015-02-06 10:26 UTC (permalink / raw) To: Linhaifeng, Damjan Marion (damarion), dev > -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Linhaifeng > Sent: Friday, February 06, 2015 2:04 AM > To: Damjan Marion (damarion); dev@dpdk.org > Subject: Re: [dpdk-dev] mmap fails with more than 40000 hugepages > > > > On 2015/2/5 20:00, Damjan Marion (damarion) wrote: > > Hi, > > > > I have system with 2 NUMA nodes and 256G RAM total. I noticed that DPDK > crashes in rte_eal_init() > > when number of available hugepages is around 40000 or above. > > Everything works fine with lower values (i.e. 30000). > > > > I also tried with allocating 40000 on node0 and 0 on node1, same crash > happens. > > > > > > Any idea what might be causing this? > > > > Thanks, > > > > Damjan > > > > cat /proc/sys/vm/max_map_count I just checked on my board, and having 40k hugepages, you need that value to be over the double of that (a value of 81k should work). Let us know if that fixed the problem! Pablo > > > > > > $ cat /sys/devices/system/node/node[01]/hugepages/hugepages- > 2048kB/nr_hugepages > > 20000 > > 20000 > > > > $ grep -i huge /proc/meminfo > > AnonHugePages: 706560 kB > > HugePages_Total: 40000 > > HugePages_Free: 40000 > > HugePages_Rsvd: 0 > > HugePages_Surp: 0 > > Hugepagesize: 2048 kB > > > > > > $ sudo ~/src/dpdk/x86_64-native-linuxapp-gcc/app/testpmd -l 5-7 -n 3 -- > socket-mem 512,512 > > EAL: Detected lcore 0 as core 0 on socket 0 > > EAL: Detected lcore 1 as core 1 on socket 0 > > EAL: Detected lcore 2 as core 2 on socket 0 > > EAL: Detected lcore 3 as core 3 on socket 0 > > EAL: Detected lcore 4 as core 4 on socket 0 > > EAL: Detected lcore 5 as core 5 on socket 0 > > EAL: Detected lcore 6 as core 6 on socket 0 > > EAL: Detected lcore 7 as core 7 on socket 0 > > EAL: Detected lcore 8 as core 0 on socket 1 > > EAL: Detected lcore 9 as core 1 on socket 1 > > EAL: Detected lcore 10 as core 2 on socket 1 > > EAL: Detected lcore 11 as core 3 on socket 1 > > EAL: Detected lcore 12 as core 4 on socket 1 > > EAL: Detected lcore 13 as core 5 on socket 1 > > EAL: Detected lcore 14 as core 6 on socket 1 > > EAL: Detected lcore 15 as core 7 on socket 1 > > EAL: Detected lcore 16 as core 0 on socket 0 > > EAL: Detected lcore 17 as core 1 on socket 0 > > EAL: Detected lcore 18 as core 2 on socket 0 > > EAL: Detected lcore 19 as core 3 on socket 0 > > EAL: Detected lcore 20 as core 4 on socket 0 > > EAL: Detected lcore 21 as core 5 on socket 0 > > EAL: Detected lcore 22 as core 6 on socket 0 > > EAL: Detected lcore 23 as core 7 on socket 0 > > EAL: Detected lcore 24 as core 0 on socket 1 > > EAL: Detected lcore 25 as core 1 on socket 1 > > EAL: Detected lcore 26 as core 2 on socket 1 > > EAL: Detected lcore 27 as core 3 on socket 1 > > EAL: Detected lcore 28 as core 4 on socket 1 > > EAL: Detected lcore 29 as core 5 on socket 1 > > EAL: Detected lcore 30 as core 6 on socket 1 > > EAL: Detected lcore 31 as core 7 on socket 1 > > EAL: Support maximum 128 logical core(s) by configuration. > > EAL: Detected 32 lcore(s) > > EAL: VFIO modules not all loaded, skip VFIO support... > > EAL: Setting up memory... > > EAL: Ask a virtual area of 0x800000 bytes > > EAL: Virtual area found at 0x7fae2a200000 (size = 0x800000) > > EAL: Ask a virtual area of 0x7600000 bytes > > EAL: Virtual area found at 0x7fae22a00000 (size = 0x7600000) > > EAL: Ask a virtual area of 0x1400000 bytes > > EAL: Virtual area found at 0x7fae21400000 (size = 0x1400000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fae21000000 (size = 0x200000) > > EAL: Ask a virtual area of 0x400000 bytes > > EAL: Virtual area found at 0x7fae20a00000 (size = 0x400000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fae20600000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fae20200000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fae1fe00000 (size = 0x200000) > > EAL: Ask a virtual area of 0x5800000 bytes > > EAL: Virtual area found at 0x7fae1a400000 (size = 0x5800000) > > EAL: Ask a virtual area of 0x3b200000 bytes > > EAL: Virtual area found at 0x7faddf000000 (size = 0x3b200000) > > EAL: Ask a virtual area of 0x400000 bytes > > EAL: Virtual area found at 0x7faddea00000 (size = 0x400000) > > EAL: Ask a virtual area of 0x7c00000 bytes > > EAL: Virtual area found at 0x7fadd6c00000 (size = 0x7c00000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fadd6800000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fadd6400000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fadd6000000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fadd5c00000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fadd5800000 (size = 0x200000) > > EAL: Ask a virtual area of 0x9800000 bytes > > EAL: Virtual area found at 0x7fadcbe00000 (size = 0x9800000) > > EAL: Ask a virtual area of 0x19000000 bytes > > EAL: Virtual area found at 0x7fadb2c00000 (size = 0x19000000) > > EAL: Ask a virtual area of 0x24400000 bytes > > EAL: Virtual area found at 0x7fad8e600000 (size = 0x24400000) > > EAL: Ask a virtual area of 0xc800000 bytes > > EAL: Virtual area found at 0x7fad81c00000 (size = 0xc800000) > > EAL: Ask a virtual area of 0x32000000 bytes > > EAL: Virtual area found at 0x7fad4fa00000 (size = 0x32000000) > > EAL: Ask a virtual area of 0x3db800000 bytes > > EAL: Virtual area found at 0x7fa974000000 (size = 0x3db800000) > > EAL: Ask a virtual area of 0xfa000000 bytes > > EAL: Virtual area found at 0x7fa879e00000 (size = 0xfa000000) > > EAL: Ask a virtual area of 0x168400000 bytes > > EAL: Virtual area found at 0x7fa711800000 (size = 0x168400000) > > EAL: Ask a virtual area of 0x7d000000 bytes > > EAL: Virtual area found at 0x7fa694600000 (size = 0x7d000000) > > EAL: Ask a virtual area of 0x1f4000000 bytes > > EAL: Virtual area found at 0x7fa4a0400000 (size = 0x1f4000000) > > EAL: Ask a virtual area of 0x2e400000 bytes > > EAL: Virtual area found at 0x7fa471e00000 (size = 0x2e400000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa471a00000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa471600000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa471200000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa470e00000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa470a00000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa470600000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa470200000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa46fe00000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa46fa00000 (size = 0x200000) > > EAL: Ask a virtual area of 0x400000 bytes > > EAL: Virtual area found at 0x7fa46f400000 (size = 0x400000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa46f000000 (size = 0x200000) > > EAL: Ask a virtual area of 0x400000 bytes > > EAL: Virtual area found at 0x7fa46ea00000 (size = 0x400000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa46e600000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa46e200000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa46de00000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa46da00000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa46d600000 (size = 0x200000) > > EAL: Ask a virtual area of 0x400000 bytes > > EAL: Virtual area found at 0x7fa46d000000 (size = 0x400000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa46cc00000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa46c800000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa46c400000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa46c000000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa46bc00000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa46b800000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa46b400000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa46b000000 (size = 0x200000) > > EAL: Ask a virtual area of 0x400000 bytes > > EAL: Virtual area found at 0x7fa46aa00000 (size = 0x400000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa46a600000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa46a200000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa469e00000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa469a00000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa469600000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa469200000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa468e00000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa468a00000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa468600000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa468200000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa467e00000 (size = 0x200000) > > EAL: Ask a virtual area of 0x400000 bytes > > EAL: Virtual area found at 0x7fa467800000 (size = 0x400000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa467400000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa467000000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa466c00000 (size = 0x200000) > > EAL: Ask a virtual area of 0x400000 bytes > > EAL: Virtual area found at 0x7fa466600000 (size = 0x400000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa466200000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa465e00000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa465a00000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa465600000 (size = 0x200000) > > EAL: Ask a virtual area of 0x400000 bytes > > EAL: Virtual area found at 0x7fa465000000 (size = 0x400000) > > EAL: Ask a virtual area of 0x400000 bytes > > EAL: Virtual area found at 0x7fa464a00000 (size = 0x400000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa464600000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa464200000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa463e00000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa463a00000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa463600000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa463200000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa462e00000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa462a00000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa462600000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa462200000 (size = 0x200000) > > EAL: Ask a virtual area of 0x400000 bytes > > EAL: Virtual area found at 0x7fa461c00000 (size = 0x400000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa461800000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa461400000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa461000000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa460c00000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa460800000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa460400000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa460000000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa45fc00000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa45f800000 (size = 0x200000) > > EAL: Ask a virtual area of 0x400000 bytes > > EAL: Virtual area found at 0x7fa45f200000 (size = 0x400000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa45ee00000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa45ea00000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa45e600000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa45e200000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa45de00000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa45da00000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa45d600000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa45d200000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa45ce00000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa45ca00000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa45c600000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa45c200000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa45be00000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa45ba00000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa45b600000 (size = 0x200000) > > EAL: Ask a virtual area of 0x400000 bytes > > EAL: Virtual area found at 0x7fa45b000000 (size = 0x400000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa45ac00000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa45a800000 (size = 0x200000) > > EAL: Ask a virtual area of 0x400000 bytes > > EAL: Virtual area found at 0x7fa45a200000 (size = 0x400000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa459e00000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa459a00000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa459600000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa459200000 (size = 0x200000) > > EAL: Ask a virtual area of 0x400000 bytes > > EAL: Virtual area found at 0x7fa458c00000 (size = 0x400000) > > EAL: Ask a virtual area of 0x400000 bytes > > EAL: Virtual area found at 0x7fa458600000 (size = 0x400000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa458200000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa457e00000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa457a00000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa457600000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa457200000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa456e00000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa456a00000 (size = 0x200000) > > EAL: Ask a virtual area of 0x200000 bytes > > EAL: Virtual area found at 0x7fa456600000 (size = 0x200000) > > EAL: Ask a virtual area of 0x139400000 bytes > > EAL: Virtual area found at 0x7fa31d000000 (size = 0x139400000) > > EAL: Ask a virtual area of 0xe9000000 bytes > > EAL: Virtual area found at 0x7fa233e00000 (size = 0xe9000000) > > EAL: Ask a virtual area of 0x188000000 bytes > > EAL: Virtual area found at 0x7fa0abc00000 (size = 0x188000000) > > EAL: map_all_hugepages(): mmap failed: Cannot allocate memory > > EAL: Failed to remap 2 MB pages > > PANIC in rte_eal_init(): > > Cannot init memory > > Aborted (core dumped) > > > > > > > > > > . > > > > -- > Regards, > Haifeng ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [dpdk-dev] mmap fails with more than 40000 hugepages 2015-02-06 10:26 ` De Lara Guarch, Pablo @ 2015-02-06 10:31 ` Damjan Marion (damarion) 0 siblings, 0 replies; 13+ messages in thread From: Damjan Marion (damarion) @ 2015-02-06 10:31 UTC (permalink / raw) To: De Lara Guarch, Pablo; +Cc: dev > On 06 Feb 2015, at 11:26, De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com> wrote: > > > >> -----Original Message----- >> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Linhaifeng >> Sent: Friday, February 06, 2015 2:04 AM >> To: Damjan Marion (damarion); dev@dpdk.org >> Subject: Re: [dpdk-dev] mmap fails with more than 40000 hugepages >> >> >> >> On 2015/2/5 20:00, Damjan Marion (damarion) wrote: >>> Hi, >>> >>> I have system with 2 NUMA nodes and 256G RAM total. I noticed that DPDK >> crashes in rte_eal_init() >>> when number of available hugepages is around 40000 or above. >>> Everything works fine with lower values (i.e. 30000). >>> >>> I also tried with allocating 40000 on node0 and 0 on node1, same crash >> happens. >>> >>> >>> Any idea what might be causing this? >>> >>> Thanks, >>> >>> Damjan >>> >> >> cat /proc/sys/vm/max_map_count > > I just checked on my board, and having 40k hugepages, you need that value to be over the double of that (a value of 81k should work). > > Let us know if that fixed the problem! Yes, it works now. Thanks! ^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2015-02-06 10:31 UTC | newest] Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2015-02-05 12:00 [dpdk-dev] mmap fails with more than 40000 hugepages Damjan Marion (damarion) 2015-02-05 12:59 ` Neil Horman 2015-02-05 13:20 ` Damjan Marion (damarion) 2015-02-05 14:59 ` Neil Horman 2015-02-05 15:41 ` Damjan Marion (damarion) 2015-02-05 16:35 ` Neil Horman 2015-02-05 13:22 ` Jay Rolette 2015-02-05 13:36 ` Damjan Marion (damarion) 2015-02-05 15:09 ` Jay Rolette 2015-02-06 1:46 ` Linhaifeng 2015-02-06 2:04 ` Linhaifeng 2015-02-06 10:26 ` De Lara Guarch, Pablo 2015-02-06 10:31 ` Damjan Marion (damarion)
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).