RE: DPDK 22.11.2 requires too much VIRT memory, how to reduce

["Lombardo, Ed" <Ed.Lombardo@netscout.com>]

[-- Attachment #1: Type: text/plain, Size: 3301 bytes --]

Hi Demitry,
Thank you for your reply.

Tried many things with no success.
I have tried 1G hugepage size x 2 pages and the VIRT value went up from 66.0 G to 128G.
I used DPDK 17.11 with 1024 Hugepages of size 2MB and did not see VIRT more than 5.5G.

With 1G hugepage size:
[root@vstreamOR9-160 ~]# ./dpdk-hugepages.py --show
Node Pages Size Total
0    2     1Gb    2Gb

Hugepages mounted on /dev/hugepages /mnt/huge

You can see the 16 G and 32 G mappings with 1G hugepage size.  Any additional mmap we do in our application fails for “OUT OF MEMORY (12)”
pmap -d <appPID>
0000000100035000       4 rw-s- 0000000000000000 000:00019 fbarray_memseg-1048576k-0-0
0000000140000000 1048576 rw-s- 0000000000000000 000:00027 rtemap_0
0000000180000000 1048576 rw-s- 0000000000000000 000:00027 rtemap_1
00000001c0000000 31457280 ----- 0000000000000000 000:00000   [ anon ]
0000000940000000       4 rw-s- 0000000000000000 000:00019 fbarray_memseg-1048576k-0-1
0000000980000000 33554432 ----- 0000000000000000 000:00000   [ anon ]
0000001180000000     388 rw-s- 0000000000000000 000:00019 fbarray_memseg-2048k-0-0
0000001180200000 16777216 ----- 0000000000000000 000:00000   [ anon ]
0000001580200000     388 rw-s- 0000000000000000 000:00019 fbarray_memseg-2048k-0-1
0000001580400000 16777216 ----- 0000000000000000 000:00000   [ anon ]
0000001980400000     388 rw-s- 0000000000000000 000:00019 fbarray_memseg-2048k-0-2
0000001980600000 16777216 ----- 0000000000000000 000:00000   [ anon ]
0000001d80600000     388 rw-s- 0000000000000000 000:00019 fbarray_memseg-2048k-0-3
0000001d80800000 16777216 ----- 0000000000000000 000:00000   [ anon ]

I tried dpdk_testpmd (# ./dpdk-testpmd -l 0-3  -n 1 -- -i --portmask=0x01) on my VM and the VIRT memory jumps to 66 GB with 2MB hugepage size and 1024 huge pages.

I also tried adding –single-file-segments in the EAL init argument.

When you say VIRT memory is not important then what would you suggest I focus on to determine the resource hog that DPDK 22.11 injects in our application?
Is there too many mmaps done by DPDK in 22.11?

Thanks,
Ed

From: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
Sent: Sunday, November 5, 2023 1:23 PM
To: Lombardo, Ed <Ed.Lombardo@netscout.com>
Cc: users <users@dpdk.org>
Subject: Re: DPDK 22.11.2 requires too much VIRT memory, how to reduce

External Email: This message originated outside of NETSCOUT. Do not click links or open attachments unless you recognize the sender and know the content is safe.
Hi Ed,

DPDK indeed reserves a large amount of virtual address space at startup. This does not prevent DPDK from running on low-memory systems, since this reservation does not consume physical memory. So the large figure in VIRT is not an issue. Hugepages are mapped on demand. Each mapped hugepage consunes a file descriptor. If you use 2MB hugepages, the number of open files can exhaust the limit easily. If you don't want to increase it, please see: https://doc.dpdk.org/guides/prog_guide/env_abstraction_layer.html#segment-file-descriptors<https://urldefense.com/v3/__https:/doc.dpdk.org/guides/prog_guide/env_abstraction_layer.html*segment-file-descriptors__;Iw!!Nzg7nt7_!C6krMpuvUwO2CGGJ-X-hjU7gIsnPFVpoBW1CyXcNJrZAWfOP_oODYN5YQ1v0y7yy2Ac2ej2uicsBFMNaZYgBTmXOiOM$>

[-- Attachment #2: Type: text/html, Size: 7746 bytes --]