Windows mbuf pool memory allocation

Windows mbuf pool memory allocation

[Robert Hable]

[-- Attachment #1: Type: text/plain, Size: 1471 bytes --]

Hello,

I have a question about mbuf pool memory allocation. When I have my machine freshly started, it usually works flawless to run testpmd. But after a few days uptime sometimes the allocation of the mbuf pool fails. Currently I get the error "Not enough space".
What could be the reason behind this that it works after a reboot, but not after a few days uptime?
According to the windows taskmanager there is still 32.8 GB free memory, and only 30.3 GB is currently used.

c:\Dev\Extern\dpdk\build\app>dpdk-testpmd.exe
EAL: Detected CPU lcores: 20
EAL: Detected NUMA nodes: 1
EAL: Multi-process support is requested, but not available.
EAL: Probe PCI driver: mlx5_pci (15b3:1015) device: 0000:01:00.0 (socket 0)
mlx5_common: DevX read access NIC register=0X9055 failed errno=0 status=0 syndrome=0
mlx5_net: mlx5_os_dev_shared_handler_install: is not supported
mlx5_net: Rx CQE 128B compression is not supported.
EAL: Probe PCI driver: mlx5_pci (15b3:1015) device: 0000:01:00.1 (socket 0)
mlx5_common: DevX read access NIC register=0X9055 failed errno=0 status=0 syndrome=0
mlx5_net: mlx5_os_dev_shared_handler_install: is not supported
mlx5_net: Rx CQE 128B compression is not supported.
testpmd: create a new mbuf pool <mb_pool_0>: n=299456, size=2176, socket=0
testpmd: preferred mempool ops selected: ring_mp_mc
EAL: Error - exiting with code: 1
  Cause: Creation of mbuf pool for socket 0 failed: Not enough space

Regards,
Robert Hable


[-- Attachment #2: Type: text/html, Size: 4638 bytes --]

Re: Windows mbuf pool memory allocation

[Dmitry Kozlyuk]

Hi Robert,

2022-09-27 15:22 (UTC+0000), Robert Hable:
> Hello,
> 
> I have a question about mbuf pool memory allocation. When I have my machine freshly started, it usually works flawless to run testpmd. But after a few days uptime sometimes the allocation of the mbuf pool fails. Currently I get the error "Not enough space".
> What could be the reason behind this that it works after a reboot, but not after a few days uptime?
> According to the windows taskmanager there is still 32.8 GB free memory, and only 30.3 GB is currently used.

Looks like RAM fragmentation.
If the system is out of contiguous hugepage spans, --iova-mode=va should help.
If the system cannot find contiguous RAM for a single 2MB hugepage,
there's not solution that I know of, there's no option to reserve hugepages
(or none that I know of---Dmitry, Tyler?)

> 
> c:\Dev\Extern\dpdk\build\app>dpdk-testpmd.exe  
> EAL: Detected CPU lcores: 20
> EAL: Detected NUMA nodes: 1
> EAL: Multi-process support is requested, but not available.
> EAL: Probe PCI driver: mlx5_pci (15b3:1015) device: 0000:01:00.0 (socket 0)
> mlx5_common: DevX read access NIC register=0X9055 failed errno=0 status=0 syndrome=0
> mlx5_net: mlx5_os_dev_shared_handler_install: is not supported
> mlx5_net: Rx CQE 128B compression is not supported.
> EAL: Probe PCI driver: mlx5_pci (15b3:1015) device: 0000:01:00.1 (socket 0)
> mlx5_common: DevX read access NIC register=0X9055 failed errno=0 status=0 syndrome=0
> mlx5_net: mlx5_os_dev_shared_handler_install: is not supported
> mlx5_net: Rx CQE 128B compression is not supported.
> testpmd: create a new mbuf pool <mb_pool_0>: n=299456, size=2176, socket=0
> testpmd: preferred mempool ops selected: ring_mp_mc
> EAL: Error - exiting with code: 1
>   Cause: Creation of mbuf pool for socket 0 failed: Not enough space

--log-level=lib.eal:debug would show which allocation failed.