* [dpdk-dev] [Bug 95] eal/memory: testpmd failed to start with --socket-mem > 16384 with 2MB huge pages and with memory hotplug
@ 2018-10-16 16:38 bugzilla
0 siblings, 0 replies; only message in thread
From: bugzilla @ 2018-10-16 16:38 UTC (permalink / raw)
To: dev
https://bugs.dpdk.org/show_bug.cgi?id=95
Bug ID: 95
Summary: eal/memory: testpmd failed to start with --socket-mem
> 16384 with 2MB huge pages and with memory hotplug
Product: DPDK
Version: 18.11
Hardware: x86
OS: Linux
Status: CONFIRMED
Severity: normal
Priority: Normal
Component: core
Assignee: dev@dpdk.org
Reporter: maxime.leroy@6wind.com
Target Milestone: ---
With the lastest version on dpdk.org (i.e. commit id
94d31549c380ee8b9108c4621d6fad8d624a395e)
First configure 2Mb hugepages:
mkdir -p /mnt/huge
mount -t hugetlbfs nodev /mnt/huge
echo 12598 >
/sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
Start test pmd with 25Gb memory:
root@ubuntu1604:~/dpdk# ./build/app/testpmd -c7 --socket-mem 25196 -- -i
--nb-cores=2 - --total-num-mbufs=2048
EAL: Detected 24 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: No free hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL: eal_memalloc_alloc_seg_bulk(): couldn't find suitable memseg_list
EAL: FATAL: Cannot init memory
EAL: Cannot init memory
PANIC in main():
Cannot init EAL
5: [./build/app/testpmd(_start+0x29) [0x48b509]]
4: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0) [0x7f28f9adb830]]
3: [./build/app/testpmd(main+0xb0e) [0x4825ae]]
2: [./build/app/testpmd(__rte_panic+0xc3) [0x475920]]
1: [./build/app/testpmd(rte_dump_stack+0x2b) [0x5ba22b]]
Aborted (core dumped)
It failed.
With --legacy-mem option, it's work fine:
./build/app/testpmd -c7 --socket-mem 25196 --legacy-mem -- -i --nb-cores=2 -
--total-num-mbufs=2048
EAL: Detected 24 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: No free hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: VFIO support initialized
..
Interactive-mode selected
testpmd: create a new mbuf pool <mbuf_pool_socket_0>: n=2048, size=2176,
socket=0
testpmd: preferred mempool ops selected: ring_mp_mc
Done
testpmd>
testpmd> dump_physmem
Segment 0-0: IOVA:0x2400000, len:2097152, virt:0x7f3491a00000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:11
Segment 0-1: IOVA:0x2600000, len:2097152, virt:0x7f3491c00000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:12
Segment 0-2: IOVA:0x2800000, len:2097152, virt:0x7f3491e00000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:13
Segment 0-3: IOVA:0x2a00000, len:2097152, virt:0x7f3492000000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:14
Segment 0-4: IOVA:0x2c00000, len:2097152, virt:0x7f3492200000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:15
Segment 0-5: IOVA:0x2e00000, len:2097152, virt:0x7f3492400000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:16
Segment 0-6: IOVA:0x3000000, len:2097152, virt:0x7f3492600000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:17
Segment 0-7: IOVA:0x3200000, len:2097152, virt:0x7f3492800000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:18
Segment 0-8: IOVA:0x3400000, len:2097152, virt:0x7f3492a00000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:19
Segment 0-9: IOVA:0x3600000, len:2097152, virt:0x7f3492c00000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:20
Segment 0-10: IOVA:0x3800000, len:2097152, virt:0x7f3492e00000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:21
Segment 0-11: IOVA:0x3a00000, len:2097152, virt:0x7f3493000000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:22
Segment 0-12: IOVA:0x3c00000, len:2097152, virt:0x7f3493200000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:23
Segment 0-13: IOVA:0x3e00000, len:2097152, virt:0x7f3493400000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:24
Segment 0-14: IOVA:0x4000000, len:2097152, virt:0x7f3493600000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:25
...
Segment 0-8188: IOVA:0x539200000, len:2097152, virt:0x7f3891200000,
socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:7617
Segment 0-8189: IOVA:0x539400000, len:2097152, virt:0x7f3891400000,
socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:7618
Segment 0-8190: IOVA:0x539600000, len:2097152, virt:0x7f3891600000,
socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:7619
Segment 1-0: IOVA:0x514800000, len:2097152, virt:0x7f3091800000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:7335
Segment 1-1: IOVA:0x514a00000, len:2097152, virt:0x7f3091a00000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:7336
Segment 1-2: IOVA:0x514c00000, len:2097152, virt:0x7f3091c00000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:7337
Segment 1-3: IOVA:0x514e00000, len:2097152, virt:0x7f3091e00000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:7338
Segment 1-4: IOVA:0x515000000, len:2097152, virt:0x7f3092000000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:7339
...
Segment 1-5517: IOVA:0x7f3c00000, len:2097152, virt:0x7f3343200000,
socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:12602
Segment 1-5519: IOVA:0x7f4c00000, len:2097152, virt:0x7f3343600000,
socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:12603
Segment 1-5520: IOVA:0x7f4e00000, len:2097152, virt:0x7f3343800000,
socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:12604
Segment 1-5522: IOVA:0x7f7000000, len:2097152, virt:0x7f3343c00000,
socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:12605
Segment 1-5524: IOVA:0x7f7a00000, len:2097152, virt:0x7f3344000000,
socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:12606
Segment 1-5526: IOVA:0x7fc600000, len:2097152, virt:0x7f3344400000,
socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:12607
Segment 1-5528: IOVA:0x800c00000, len:2097152, virt:0x7f3344800000,
socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:12608
We have 2 segments. The first one has 8192 pages and the second one 5529
segment.
If we increaze the MAX_MEMSEG_LIST:
--- a/config/common_base
+++ b/config/common_base
@@ -65,7 +65,7 @@ CONFIG_RTE_MAX_HEAPS=32
CONFIG_RTE_MAX_MEMSEG_LISTS=64
# each memseg list will be limited to either RTE_MAX_MEMSEG_PER_LIST pages
# or RTE_MAX_MEM_MB_PER_LIST megabytes worth of memory, whichever is smaller
-CONFIG_RTE_MAX_MEMSEG_PER_LIST=8192
+CONFIG_RTE_MAX_MEMSEG_PER_LIST=16384
./build/app/testpmd -c7 --socket-mem 25196 -- -i --nb-cores=2 -
--total-num-mbufs=2048
Interactive-mode selected
testpmd: create a new mbuf pool <mbuf_pool_socket_0>: n=2048, size=2176,
socket=0
testpmd: preferred mempool ops selected: ring_mp_mc
Done
testpmd>
It's working fine.
testpmd> dump_physmem
Segment 0-0: IOVA:0x2400000, len:2097152, virt:0x7f3491a00000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:11
Segment 0-1: IOVA:0x2600000, len:2097152, virt:0x7f3491c00000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:12
Segment 0-2: IOVA:0x2800000, len:2097152, virt:0x7f3491e00000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:13
Segment 0-3: IOVA:0x2a00000, len:2097152, virt:0x7f3492000000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:14
Segment 0-4: IOVA:0x2c00000, len:2097152, virt:0x7f3492200000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:15
Segment 0-5: IOVA:0x2e00000, len:2097152, virt:0x7f3492400000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:16
Segment 0-6: IOVA:0x3000000, len:2097152, virt:0x7f3492600000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:17
Segment 0-7: IOVA:0x3200000, len:2097152, virt:0x7f3492800000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:18
Segment 0-8: IOVA:0x3400000, len:2097152, virt:0x7f3492a00000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:19
Segment 0-9: IOVA:0x3600000, len:2097152, virt:0x7f3492c00000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:20
Segment 0-10: IOVA:0x3800000, len:2097152, virt:0x7f3492e00000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:21
Segment 0-11: IOVA:0x3a00000, len:2097152, virt:0x7f3493000000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:22
Segment 0-12: IOVA:0x3c00000, len:2097152, virt:0x7f3493200000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:23
Segment 0-13: IOVA:0x3e00000, len:2097152, virt:0x7f3493400000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:24
Segment 0-14: IOVA:0x4000000, len:2097152, virt:0x7f3493600000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:25
Segment 0-15: IOVA:0x4200000, len:2097152, virt:0x7f3493800000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:26
..
Segment 0-12588: IOVA:0x4cd000000, len:2097152, virt:0x7fb892c00000,
socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:12599
Segment 0-12589: IOVA:0x4d2e00000, len:2097152, virt:0x7fb892e00000,
socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:12600
Segment 0-12590: IOVA:0x73ba00000, len:2097152, virt:0x7fb893000000,
socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:12601
Segment 0-12591: IOVA:0x4d4a00000, len:2097152, virt:0x7fb893200000,
socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:12602
Segment 0-12592: IOVA:0xc4600000, len:2097152, virt:0x7fb893400000,
socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:12603
Segment 0-12593: IOVA:0x514400000, len:2097152, virt:0x7fb893600000,
socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:12604
Segment 0-12594: IOVA:0x7e3000000, len:2097152, virt:0x7fb893800000,
socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:12605
Segment 0-12595: IOVA:0x4f9800000, len:2097152, virt:0x7fb893a00000,
socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:12606
Segment 0-12596: IOVA:0x294400000, len:2097152, virt:0x7fb893c00000,
socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:12607
Segment 0-12597: IOVA:0x4d5000000, len:2097152, virt:0x7fb893e00000,
socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:12608
But we have only one segment in this case.
Question
--------
1) It's normal that we don't support anymore to allocate more that 16GB with
hotplug memory and default config value ? It was working with previous dpdk
version.
2) Should we increase the default value of CONFIG_RTE_MAX_MEMSEG_PER_LIST ?
Note: This http://patches.dpdk.org/patch/46112/ patch doesn't fix the issue.
--
You are receiving this mail because:
You are the assignee for the bug.
^ permalink raw reply [flat|nested] only message in thread
only message in thread, other threads:[~2018-10-16 16:38 UTC | newest]
Thread overview: (only message) (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-10-16 16:38 [dpdk-dev] [Bug 95] eal/memory: testpmd failed to start with --socket-mem > 16384 with 2MB huge pages and with memory hotplug bugzilla
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).