DPDK usage discussions
 help / color / mirror / Atom feed
* Dpdk allocates more memory, than available physically (hugepages)
@ 2023-02-08  2:43 Szymon Szozda
  2023-02-13 15:46 ` Dmitry Kozlyuk
  0 siblings, 1 reply; 3+ messages in thread
From: Szymon Szozda @ 2023-02-08  2:43 UTC (permalink / raw)
  To: users

[-- Attachment #1: Type: text/plain, Size: 684 bytes --]

Hey,
I'm running dpdk on a machine with 64GB of RAM. It is configured, so 16GiB
(16 x 1GiB chunks) of hugepage memory is reserved on boot. I was expecting
dpdk to consume only those 16GiB, but it seems it gets more than 30GiB of
virtual memory ( I base it on memory VSZ output of top command ). The
machine is 1 NUMA, 1 NIC. I did some debugging and I do not see any logic
which limits the memory consumption, basically it seems that
eal_dynmem_memseg_lists_init() will allocate the same amount, no matter how
much RAM is physically available.

Is it expected? How to know that setup will not crash due to
insufficient memory available? How to limit those memory consumption.by
dpdk?

[-- Attachment #2: Type: text/html, Size: 757 bytes --]

^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: Dpdk allocates more memory, than available physically (hugepages)
  2023-02-08  2:43 Dpdk allocates more memory, than available physically (hugepages) Szymon Szozda
@ 2023-02-13 15:46 ` Dmitry Kozlyuk
  2023-02-13 17:17   ` Stephen Hemminger
  0 siblings, 1 reply; 3+ messages in thread
From: Dmitry Kozlyuk @ 2023-02-13 15:46 UTC (permalink / raw)
  To: Szymon Szozda; +Cc: users

2023-02-08 03:43 (UTC+0100), Szymon Szozda:
> Hey,
> I'm running dpdk on a machine with 64GB of RAM. It is configured, so 16GiB
> (16 x 1GiB chunks) of hugepage memory is reserved on boot. I was expecting
> dpdk to consume only those 16GiB, but it seems it gets more than 30GiB of
> virtual memory ( I base it on memory VSZ output of top command ). The
> machine is 1 NUMA, 1 NIC. I did some debugging and I do not see any logic
> which limits the memory consumption, basically it seems that
> eal_dynmem_memseg_lists_init() will allocate the same amount, no matter how
> much RAM is physically available.
> 
> Is it expected? How to know that setup will not crash due to
> insufficient memory available? How to limit those memory consumption.by
> dpdk?

Hi,

DPDK always reserves a large chunk of virtual address space,
but this costs almost nothing and does not need to be limited.
Then DPDK maps and unmaps actual pages to those addresses as needed.
DPDK does not crash if it runs out of hugepages reserved in the system
but merely returns NULL from its allocation API (rte_malloc).
Real memory consumption can be limited with --socket-limit EAL option.
See also --socket-mem and -m to reserve hugepages at DPDK startup.

^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: Dpdk allocates more memory, than available physically (hugepages)
  2023-02-13 15:46 ` Dmitry Kozlyuk
@ 2023-02-13 17:17   ` Stephen Hemminger
  0 siblings, 0 replies; 3+ messages in thread
From: Stephen Hemminger @ 2023-02-13 17:17 UTC (permalink / raw)
  To: Dmitry Kozlyuk; +Cc: Szymon Szozda, users

On Mon, 13 Feb 2023 18:46:14 +0300
Dmitry Kozlyuk <dmitry.kozliuk@gmail.com> wrote:

> 2023-02-08 03:43 (UTC+0100), Szymon Szozda:
> > Hey,
> > I'm running dpdk on a machine with 64GB of RAM. It is configured, so 16GiB
> > (16 x 1GiB chunks) of hugepage memory is reserved on boot. I was expecting
> > dpdk to consume only those 16GiB, but it seems it gets more than 30GiB of
> > virtual memory ( I base it on memory VSZ output of top command ). The
> > machine is 1 NUMA, 1 NIC. I did some debugging and I do not see any logic
> > which limits the memory consumption, basically it seems that
> > eal_dynmem_memseg_lists_init() will allocate the same amount, no matter how
> > much RAM is physically available.
> > 
> > Is it expected? How to know that setup will not crash due to
> > insufficient memory available? How to limit those memory consumption.by
> > dpdk?  
> 
> Hi,
> 
> DPDK always reserves a large chunk of virtual address space,
> but this costs almost nothing and does not need to be limited.
> Then DPDK maps and unmaps actual pages to those addresses as needed.
> DPDK does not crash if it runs out of hugepages reserved in the system
> but merely returns NULL from its allocation API (rte_malloc).
> Real memory consumption can be limited with --socket-limit EAL option.
> See also --socket-mem and -m to reserve hugepages at DPDK startup.

Hugepages are mainly used for buffers and shared driver resources.
The normal text and data are not in hugepages (unless you figure
out how to use hugelbfs). Memory consumption limiting is best
done with cgroups. But most applications die if they can't get the
memory they need.  You can see where memory is used with /proc/XXX/maps

^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2023-02-13 17:17 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-02-08  2:43 Dpdk allocates more memory, than available physically (hugepages) Szymon Szozda
2023-02-13 15:46 ` Dmitry Kozlyuk
2023-02-13 17:17   ` Stephen Hemminger

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).