Re: Increase DPDK's virtual memory to more than 512 GiB

[Lucas <lucascrijns@gmail.com>]

[-- Attachment #1.1: Type: text/plain, Size: 1327 bytes --]

Hi Dmitry,

Excuse my late reply.
My system is configured with 1G huge page size upon boot and then later on
I issue `sysctl -w vm.nr_hugepages=700`
It appears that the swapping was caused by some daemon that decided to
start up and allocate quite some memory. I no longer observe this effect.
I included a FlameGraph (recorded with perf) from program startup that does
a memory allocation for 500 GB (MBUFS + priv data): mempool allocation
with 218'531'468 MBUFs.
Most time is spent in mmap and in memsets.

Let me know what you think and perhaps if there are ways to improve the
loading time.

On Thu, Feb 20, 2025 at 3:55 PM Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
wrote:

> 2025-02-20 15:19 (UTC+0100), Lucas:
> > As for the issue, at 550 GB so 252'757'352 MBUFs (using default mbuf buf
> > size), it works now, with ring allocation. However, the physical memory
> > usage now goes up a lot and I end up swapping. It appears that not all
> > memory is in hugepages (not all pages are filled) and that perhaps the
> > kernel also allocates more memory. I have 755 GiB RAM available, so 600
> GB
> > of mempool is pushing it.
>
> How is your system configured to reserve hugepages?
> DPDK always allocates hugepages; if none are available, allocation fails.
> Hugepages never swap AFAIK.
>

[-- Attachment #1.2: Type: text/html, Size: 1775 bytes --]

[-- Attachment #2: startup_flamegraph.svg --]
[-- Type: image/svg+xml, Size: 71899 bytes --]