Re: Increase DPDK's virtual memory to more than 512 GiB

[Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>]

2025-02-19 15:22 (UTC+0100), Lucas:
> I am creating an application where I want to cache packets in a ring for a
> certain duration. To do so at high speed (100 Gb/s) for tens of seconds, I
> would like to create a mempool with a size bigger than 500 GiB.
> I know that in rte_config.h, we have this excerpt:
> ```
> #define RTE_MAX_MEMSEG_LISTS 128
> ...
> #define RTE_MAX_MEM_MB_PER_LIST 32768
> ```
> 4'194'304 MiB of addressable memory, which is more than enough. However,
> there is a virtual memory limit on DPDK processes on x86_64, which appears
> to be 512 GiB.
> at https://doc.dpdk.org/guides/prog_guide/env_abstraction_layer.html it is
> listed that RTE_MAX_MEM_MB controls that.
> In the meson.build file in the config directory, we have:
> ```
> if dpdk_conf.get('RTE_ARCH_64')
>     dpdk_conf.set('RTE_MAX_MEM_MB', 524288)
> else # for 32-bit we need smaller reserved memory areas
>     dpdk_conf.set('RTE_MAX_MEM_MB', 2048)
> endif
> ```
> Changing this value does not seem to change the amount of virtual memory
> that DPDK allocates. It appears that no headers or C-files actually
> reference this value.

https://elixir.bootlin.com/dpdk/v24.11.1/source/lib/eal/common/eal_common_dynmem.c#L113

> What would I need to change to allow more virtual memory than 512 GiB and
> be able to allocate a mempool with a size bigger than that?

Increase RTE_MAX_MEM_MB_PER_LIST and RTE_MAX_MEM_MB_PER_TYPE to 512 GB.
See the big comment in the linked function code.
Note that RTE_MAX_MEM_MB must accommodate for all memory types,
e.g. for a 2-processor x86_64 system with 2M and 1G hugepages supported
it must be 2 NUMA nodes x 2 huggepage sizes x 512 GB = 2048 GB.