Re: Need help with reducing VIRT memory

[Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>]

Hi Ed,

I presume it's a revival of this thread:

	http://inbox.dpdk.org/users/CH3PR01MB8470C9675763E14954D6E3B88F292@CH3PR01MB8470.prod.exchangelabs.com/

2024-05-02 19:05 (UTC+0000), Lombardo, Ed:
[...]
> My situation is as follows:
> We were on DPDK 17.11.6 and upgraded to DPDK22.11.2 to support the Intel E810.  Also upgraded from CentOS7 to Oracle 91.
> In DPDK 22.11.2 the VIRT memory has skyrocketed and causes major issues with our application that runs on 16 GB of memory (virtual appliances and low end remote appliances).
> 
> I have also tried DPDK 23.11 and 24.03 and I also see across the DPDK versions that the VIRT memory has increase by a factor of 5 over DPDK 17.11
> 
> Our application process shows VIRT memory is 7.6 GB with DPDK 17.11.6,
> and with DPDK 22.11.2, 23.11.2 and 24.03 the VIRT memory is ~36.6GB
> 
> Our application minimum configuration needs to run with 16 GB memory when DPDK is enabled.

As explained in the linked thread,
high reserved VIRT does not mean high memory demand.
Running DPDK on a 16 GB machine is perfectly viable.
Something in your app or setup conflicts with the high VIRT reservation.

> Our application uses memory locking and rlimit settings to get optimal performance.

Does it use address space limit (prlimit -v)?
If so, this limit may be hit.
With default build options, DPDK reserves at least 32 GB of VIRT (one list)
per memory type (a combination of NUMA node + hugepage size).
This is controlled by defines in <rte_config.h>:

#define RTE_MAX_MEMSEG_LISTS 128
#define RTE_MAX_MEMSEG_PER_LIST 8192
#define RTE_MAX_MEM_MB_PER_LIST 32768
#define RTE_MAX_MEMSEG_PER_TYPE 32768
#define RTE_MAX_MEM_MB_PER_TYPE 65536

Try reducing those and rebuilding DPDK if the above is your case.
Reducing these values will limit the amount of memory available to DPDK
allocator in your app.