Re: [dpdk-users] 0MB available on socket when running pktgen-dpdk

["Wiles, Keith" <keith.wiles@intel.com>]

> On Jun 28, 2017, at 10:03 AM, McCullough, Harrison <harrison_mccullough@labs.att.com> wrote:
> 
> When I run pktgen-dpdk I always get the error that there are 0MB available (no
> matter how much I request).  However, if I run testpmd, then it seems to run
> without complaint.  Does anybody know what's going on?

Let me look at the rest of your email , but I figured I would get off a couple quick answers.

> 
> P.S. Why does /proc/meminfo always report that there are 0 hugepages free after
> I do anything with DPDK? Does it not clean up afterward and free them? If I rm
> /mnt/huge/* then /proc/meminfo reports that the hugepages were freed, but it
> seems like I shouldn't have to manually do that.

The quick answer DPDK does not clean up of release the huge pages, which are normally located in /dev/hugepages or where ever you have the huge pages mounted in your system. To release all of the huge pages back to the system, just remove the files located in the /dev/hugepages directory or your location.

> 
> P.P.S. If /proc/meminfo reports that there are 0 hugepages free, then why does
> testpmd work? It appears to be using them, since they are no longer free after
> running testpmd.

DPDK uses the files in the /dev/hugepages by creating or reusing these files, which at the mmap’d files for the huge pages.

> 
> root@ubuntu:/home/harrison/git/dpdk# grep -i huge /proc/meminfo
> AnonHugePages:    208896 kB
> HugePages_Total:    1024
> HugePages_Free:        0
> HugePages_Rsvd:        0
> HugePages_Surp:        0
> Hugepagesize:       2048 kB
> root@ubuntu:/home/harrison/git/dpdk# ./x86_64-native-linuxapp-gcc/app/testpmd
> EAL: Detected 8 lcore(s)
> EAL: Probing VFIO support...
> EAL: VFIO support initialized
> EAL: PCI device 0000:05:00.0 on NUMA socket -1
> EAL:   probe driver: 8086:10e8 net_e1000_igb
> EAL: PCI device 0000:05:00.1 on NUMA socket -1
> EAL:   probe driver: 8086:10e8 net_e1000_igb
> EAL: PCI device 0000:06:00.0 on NUMA socket -1
> EAL:   probe driver: 8086:10e8 net_e1000_igb
> EAL: PCI device 0000:06:00.1 on NUMA socket -1
> EAL:   probe driver: 8086:10e8 net_e1000_igb
> Warning: NUMA should be configured manually by using --port-numa-config and --ring-numa-config parameters along with --numa.
> USER1: create a new mbuf pool <mbuf_pool_socket_0>: n=203456, size=2176, socket=0
> Configuring Port 0 (socket 0)
> Port 0: 00:1B:21:6C:FC:9C
> Configuring Port 1 (socket 0)
> Port 1: 00:1B:21:6C:FC:9D
> Checking link statuses...
> Done
> No commandline core given, start packet forwarding
> io packet forwarding - ports=2 - cores=1 - streams=2 - NUMA support enabled, MP over anonymous pages disabled
> Logical Core 1 (socket 0) forwards packets on 2 streams:
>  RX P=0/Q=0 (socket 0) -> TX P=1/Q=0 (socket 0) peer=02:00:00:00:00:01
>  RX P=1/Q=0 (socket 0) -> TX P=0/Q=0 (socket 0) peer=02:00:00:00:00:00
> 
>  io packet forwarding - CRC stripping enabled - packets/burst=32
>  nb forwarding cores=1 - nb forwarding ports=2
>  RX queues=1 - RX desc=128 - RX free threshold=32
>  RX threshold registers: pthresh=8 hthresh=8 wthresh=1
>  TX queues=1 - TX desc=512 - TX free threshold=0
>  TX threshold registers: pthresh=8 hthresh=1 wthresh=1
>  TX RS bit threshold=0 - TXQ flags=0x0
> Press enter to exit
> 
> Telling cores to stop...
> Waiting for lcores to finish...
> 
>  ---------------------- Forward statistics for port 0  ----------------------
>  RX-packets: 0              RX-dropped: 0             RX-total: 0
>  TX-packets: 0              TX-dropped: 0             TX-total: 0
>  ----------------------------------------------------------------------------
> 
>  ---------------------- Forward statistics for port 1  ----------------------
>  RX-packets: 0              RX-dropped: 0             RX-total: 0
>  TX-packets: 0              TX-dropped: 0             TX-total: 0
>  ----------------------------------------------------------------------------
> 
>  +++++++++++++++ Accumulated forward statistics for all ports+++++++++++++++
>  RX-packets: 0              RX-dropped: 0             RX-total: 0
>  TX-packets: 0              TX-dropped: 0             TX-total: 0
>  ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> 
> Done.
> 
> Shutting down port 0...
> Stopping ports...
> Done
> Closing ports...
> Done
> 
> Shutting down port 1...
> Stopping ports...
> Done
> Closing ports...
> Done
> 
> Bye...
> root@ubuntu:/home/harrison/git/dpdk# grep -i huge /proc/meminfo
> AnonHugePages:    208896 kB
> HugePages_Total:    1024
> HugePages_Free:        0
> HugePages_Rsvd:        0
> HugePages_Surp:        0
> Hugepagesize:       2048 kB
> root@ubuntu:/home/harrison/git/dpdk# cd ../pktgen-dpdk/
> root@ubuntu:/home/harrison/git/pktgen-dpdk# ./tools/run.py harrison
> sudo ./app/x86_64-native-linuxapp-gcc/pktgen -l 0,1-4 -n 2 --proc-type auto --log-level 7 --socket-mem 512,512 --file-prefix pg -- -T -P --crc-strip -m [1:2].0 -m [3:4].1 -f themes/black-yellow.theme
> 
> Copyright (c) <2010-2017>, Intel Corporation. All rights reserved. Powered by Intel(r) DPDK
> EAL: Detected 8 lcore(s)
> EAL: Auto-detected process type: PRIMARY
> EAL: Probing VFIO support...
> EAL: VFIO support initialized
> EAL: Not enough memory available on socket 1! Requested: 512MB, available: 0MB
> EAL: FATAL: Cannot init memory
> 
> EAL: Cannot init memory
> 
> root@ubuntu:/home/harrison/git/pktgen-dpdk# grep -i huge /proc/meminfo
> AnonHugePages:    208896 kB
> HugePages_Total:    1024
> HugePages_Free:        0
> HugePages_Rsvd:        0
> HugePages_Surp:        0
> Hugepagesize:       2048 kB
> root@ubuntu:/home/harrison/git/pktgen-dpdk# sudo rm /mnt/huge/*
> root@ubuntu:/home/harrison/git/pktgen-dpdk# sudo rm /mnt/huge
> huge/     huge_1GB/
> root@ubuntu:/home/harrison/git/pktgen-dpdk# grep -i huge /proc/meminfo
> AnonHugePages:    208896 kB
> HugePages_Total:    1024
> HugePages_Free:     1024
> HugePages_Rsvd:        0
> HugePages_Surp:        0
> Hugepagesize:       2048 kB
> root@ubuntu:/home/harrison/git/pktgen-dpdk# ./tools/run.py harrison
> sudo ./app/x86_64-native-linuxapp-gcc/pktgen -l 0,1-4 -n 2 --proc-type auto --log-level 7 --socket-mem 512,512 --file-prefix pg -- -T -P --crc-strip -m [1:2].0 -m [3:4].1 -f themes/black-yellow.theme
> 
> Copyright (c) <2010-2017>, Intel Corporation. All rights reserved. Powered by Intel(r) DPDK
> EAL: Detected 8 lcore(s)
> EAL: Auto-detected process type: PRIMARY
> EAL: Probing VFIO support...
> EAL: VFIO support initialized
> EAL: Not enough memory available on socket 1! Requested: 512MB, available: 0MB
> EAL: FATAL: Cannot init memory
> 
> EAL: Cannot init memory
> 
> root@ubuntu:/home/harrison/git/pktgen-dpdk# grep -i huge /proc/meminfo
> AnonHugePages:    208896 kB
> HugePages_Total:    1024
> HugePages_Free:        0
> HugePages_Rsvd:        0
> HugePages_Surp:        0
> Hugepagesize:       2048 kB

Regards,
Keith