Re: DPDK 22.11.2 requires too much VIRT memory, how to reduce

[Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>]

2023-11-10 12:31 (UTC+0300), Dmitry Kozlyuk:
> Hi Ed,
> 
> 2023-11-10 00:16 (UTC+0000), Lombardo, Ed:
> > I finally finished testing all the options and found the VIRT value can be
> > reduced from 66 GB to 16 GB with --legacy-mem setting in EAL init argument.  
> 
> Right.
> By default, DPDK can use up to 64 GB of hugepage memory,
> so it reserves 64 GB of VIRT (but does not map most of it),
> RES should be low until the app actually allocates something.
> In legacy mode, DPDK maps all available hugepage memory at startup,
> in your case 16 GB, VIRT and RES should be close.
> 
> >  So I therefore had to increase the VM memory from 16 GB to 24 GB (instead
> > of 80 GB without this setting).  
> 
> I don't understand why you have to do that.
> Possible VIRT is not limited by available RAM.
> DPDK should be able to reserve 64 GB of VIRT on a machine with 16 GB of RAM,
> it will just be unable to map more than 16 GB (obviously).

Sorry, I've sent the message early by mistake.

> > I wonder what do we give up with this setting?

Most importantly, in legacy mode DPDK will consume all available hugepages at
startup and will not free them back to the system until the all is terminated.
The default dynamic mode allocates and frees physical RAM on demand.
Some advanced DPDK memory API don't work in legacy mode.

> > 
> > All the other settings I tried and combinations of these had no impact
> > (socket-limit=2048, single-file-segments, no-shconf, and no-telemetry) on
> > VIRT memory.

Right, they should not.
DPDK assumes that VIRT reservation is almost free and unlimited.
May it be that your system somehow limits it?