DPDK usage discussions
 help / color / Atom feed
* [dpdk-users] rte_eal_memseg_init(), virtual memory usage and coredump size
@ 2020-07-02 12:56 Valerii Malov
  2020-07-06  2:49 ` Li Feng
  0 siblings, 1 reply; 2+ messages in thread
From: Valerii Malov @ 2020-07-02 12:56 UTC (permalink / raw)
  To: users

Hi,



I've noticed since upgrading to one of the recent DPDK versions (I think 
it was DPDK 19), Linux application started to allocate rather big chunks 
of virtual memory. This causes coredump file size to blow up, and 
playing around with coredump_filter doesn't seem to help, since this is 
not hugepages memory.




After digging through initialization process, seems like 
rte_eal_memseg_init() is the culprit. It preallocates chunks for each 
memory "type", until it hits one of the ceilings, in my case it's 
default RTE_MAX_MEM_MB_PER_TYPE, which happens to be default 64 
gigabytes, so on 2 numa node system this results in nearly extra 128 
gigabytes of virtual memory.

Surprisingly enough, in FreeBSD implementation of rte_eal_memseg_init() 
virtual memory usage seems to be a concern due to coredump file size 
specifically.

Are there any ways to reduce the coredump file size apart from building 
DPDK with reduced RTE_MAX_MEM_MB_PER_TYPE? Am I missing something in 
system configuration and Linux is not supposed to dump those unused 
virtual memory pages? This behavior is expressed on CentOS7, but I 
*think* it was also reproducible on more modern distributions.

Thanks in advance,
Valerii.

^ permalink raw reply	[flat|nested] 2+ messages in thread

* Re: [dpdk-users] rte_eal_memseg_init(), virtual memory usage and coredump size
  2020-07-02 12:56 [dpdk-users] rte_eal_memseg_init(), virtual memory usage and coredump size Valerii Malov
@ 2020-07-06  2:49 ` Li Feng
  0 siblings, 0 replies; 2+ messages in thread
From: Li Feng @ 2020-07-06  2:49 UTC (permalink / raw)
  To: Valerii Malov; +Cc: users

Hi Valerii,

You could upgrade your DPDK to the latest version.
I have submitted a patch to fix this issue some months ago.

Thanks,

Feng Li

On Mon, Jul 6, 2020 at 4:53 AM Valerii Malov <jazzvoid@gmail.com> wrote:
>
> Hi,
>
>
>
> I've noticed since upgrading to one of the recent DPDK versions (I think
> it was DPDK 19), Linux application started to allocate rather big chunks
> of virtual memory. This causes coredump file size to blow up, and
> playing around with coredump_filter doesn't seem to help, since this is
> not hugepages memory.
>
>
>
>
> After digging through initialization process, seems like
> rte_eal_memseg_init() is the culprit. It preallocates chunks for each
> memory "type", until it hits one of the ceilings, in my case it's
> default RTE_MAX_MEM_MB_PER_TYPE, which happens to be default 64
> gigabytes, so on 2 numa node system this results in nearly extra 128
> gigabytes of virtual memory.
>
> Surprisingly enough, in FreeBSD implementation of rte_eal_memseg_init()
> virtual memory usage seems to be a concern due to coredump file size
> specifically.
>
> Are there any ways to reduce the coredump file size apart from building
> DPDK with reduced RTE_MAX_MEM_MB_PER_TYPE? Am I missing something in
> system configuration and Linux is not supposed to dump those unused
> virtual memory pages? This behavior is expressed on CentOS7, but I
> *think* it was also reproducible on more modern distributions.
>
> Thanks in advance,
> Valerii.

^ permalink raw reply	[flat|nested] 2+ messages in thread

end of thread, back to index

Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-07-02 12:56 [dpdk-users] rte_eal_memseg_init(), virtual memory usage and coredump size Valerii Malov
2020-07-06  2:49 ` Li Feng

DPDK usage discussions

Archives are clonable:
	git clone --mirror http://inbox.dpdk.org/users/0 users/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 users users/ http://inbox.dpdk.org/users \
		users@dpdk.org
	public-inbox-index users


Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.users


AGPL code for this site: git clone https://public-inbox.org/ public-inbox