From: <dkozlyuk@oss.nvidia.com>
To: <dev@dpdk.org>
Cc: Anatoly Burakov <anatoly.burakov@intel.com>,
Dmitry Kozlyuk <dkozlyuk@oss.nvidia.com>
Subject: [dpdk-dev] [PATCH v4 0/3] eal: add memory pre-allocation from existing files
Date: Mon, 20 Sep 2021 15:52:59 +0300 [thread overview]
Message-ID: <20210920125302.820273-1-dkozlyuk@nvidia.com> (raw)
In-Reply-To: <20210914103456.535427-1-dkozlyuk@nvidia.com>
From: Dmitry Kozlyuk <dkozlyuk@oss.nvidia.com>
Hugepage allocation from the system takes time, resulting in slow
startup or sporadic delays later. Most of the time spent in kernel
is zero-filling memory for security reasons, which may be irrelevant
in a controlled environment. The bottleneck is memory access speed,
so for speeduup the amount of memory cleared must be reduced.
We propose a new EAL option --mem-file FILE1,FILE2,... to quickly
allocate dirty pages from existing files and clean it as necessary.
A new malloc_perf_autotest is provided to estimate the impact.
More details are explained in relevant patches.
v4: getmntent() -> getmntent_r(), better error detection (John Levon)
v3: fix hugepage mount point detection
v2: fix CI failures
Dmitry Kozlyuk (2):
eal/linux: make hugetlbfs analysis reusable
app/test: add allocator performance autotest
Viacheslav Ovsiienko (1):
eal: add memory pre-allocation from existing files
app/test/meson.build | 2 +
app/test/test_malloc_perf.c | 157 +++++++++
doc/guides/linux_gsg/linux_eal_parameters.rst | 17 +
lib/eal/common/eal_common_dynmem.c | 6 +
lib/eal/common/eal_common_options.c | 23 ++
lib/eal/common/eal_internal_cfg.h | 4 +
lib/eal/common/eal_memalloc.h | 8 +-
lib/eal/common/eal_options.h | 2 +
lib/eal/common/malloc_elem.c | 5 +
lib/eal/common/malloc_heap.h | 8 +
lib/eal/common/rte_malloc.c | 16 +-
lib/eal/include/rte_memory.h | 4 +-
lib/eal/linux/eal.c | 28 ++
lib/eal/linux/eal_hugepage_info.c | 158 ++++++---
lib/eal/linux/eal_hugepage_info.h | 39 +++
lib/eal/linux/eal_memalloc.c | 328 +++++++++++++++++-
16 files changed, 735 insertions(+), 70 deletions(-)
create mode 100644 app/test/test_malloc_perf.c
create mode 100644 lib/eal/linux/eal_hugepage_info.h
--
2.25.1
next prev parent reply other threads:[~2021-09-20 12:53 UTC|newest]
Thread overview: 48+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-07-05 12:49 [dpdk-dev] [PATCH 21.11 " Dmitry Kozlyuk
2021-07-05 12:49 ` [dpdk-dev] [PATCH 21.11 1/3] eal/linux: make hugetlbfs analysis reusable Dmitry Kozlyuk
2021-07-05 12:49 ` [dpdk-dev] [PATCH 21.11 2/3] eal: add memory pre-allocation from existing files Dmitry Kozlyuk
2021-07-05 12:49 ` [dpdk-dev] [PATCH 21.11 3/3] app/test: add allocator performance autotest Dmitry Kozlyuk
2021-07-16 11:08 ` [dpdk-dev] [PATCH 21.11 v2 0/3] eal: add memory pre-allocation from existing files Dmitry Kozlyuk
2021-07-16 11:08 ` [dpdk-dev] [PATCH 21.11 v2 1/3] eal/linux: make hugetlbfs analysis reusable Dmitry Kozlyuk
2021-07-16 11:08 ` [dpdk-dev] [PATCH 21.11 v2 2/3] eal: add memory pre-allocation from existing files Dmitry Kozlyuk
2021-07-16 11:08 ` [dpdk-dev] [PATCH 21.11 v2 3/3] app/test: add allocator performance autotest Dmitry Kozlyuk
2021-08-09 9:45 ` [dpdk-dev] [PATCH 21.11 v2 0/3] eal: add memory pre-allocation from existing files Dmitry Kozlyuk
2021-08-30 8:21 ` Dmitry Kozlyuk
2021-09-14 10:34 ` [dpdk-dev] [PATCH v3 " Dmitry Kozlyuk
2021-09-14 10:34 ` [dpdk-dev] [PATCH v3 1/3] eal/linux: make hugetlbfs analysis reusable Dmitry Kozlyuk
2021-09-14 12:48 ` John Levon
2021-09-14 12:57 ` Dmitry Kozlyuk
2021-09-16 12:08 ` John Levon
2021-09-14 10:34 ` [dpdk-dev] [PATCH v3 2/3] eal: add memory pre-allocation from existing files Dmitry Kozlyuk
2021-09-14 10:34 ` [dpdk-dev] [PATCH v3 3/3] app/test: add allocator performance autotest Dmitry Kozlyuk
2021-09-20 12:52 ` dkozlyuk [this message]
2021-09-20 12:53 ` [dpdk-dev] [PATCH v4 1/3] eal/linux: make hugetlbfs analysis reusable dkozlyuk
2021-09-20 12:53 ` [dpdk-dev] [PATCH v4 2/3] eal: add memory pre-allocation from existing files dkozlyuk
2021-09-20 12:53 ` [dpdk-dev] [PATCH v4 3/3] app/test: add allocator performance autotest dkozlyuk
2021-09-21 8:16 ` [dpdk-dev] [PATCH v5 0/3] eal: add memory pre-allocation from existing files dkozlyuk
2021-09-21 8:16 ` [dpdk-dev] [PATCH v5 1/3] eal/linux: make hugetlbfs analysis reusable dkozlyuk
2021-09-22 13:52 ` John Levon
2021-10-05 17:36 ` Thomas Monjalon
2021-10-08 15:33 ` John Levon
2021-10-08 15:50 ` Dmitry Kozlyuk
2021-09-21 8:16 ` [dpdk-dev] [PATCH v5 2/3] eal: add memory pre-allocation from existing files dkozlyuk
2021-09-21 8:16 ` [dpdk-dev] [PATCH v5 3/3] app/test: add allocator performance autotest dkozlyuk
2021-10-11 8:56 ` [dpdk-dev] [PATCH v6 0/3] eal: add memory pre-allocation from existing files Dmitry Kozlyuk
2021-10-11 8:56 ` [dpdk-dev] [PATCH v6 1/3] eal/linux: make hugetlbfs analysis reusable Dmitry Kozlyuk
2021-10-13 8:16 ` David Marchand
2021-10-13 9:21 ` Dmitry Kozlyuk
2021-10-11 8:56 ` [dpdk-dev] [PATCH v6 2/3] eal: add memory pre-allocation from existing files Dmitry Kozlyuk
2021-10-12 15:37 ` David Marchand
2021-10-12 15:55 ` Dmitry Kozlyuk
2021-10-12 17:32 ` David Marchand
2021-10-12 21:09 ` Dmitry Kozlyuk
2021-10-13 10:18 ` David Marchand
2021-11-08 14:27 ` Dmitry Kozlyuk
2021-11-08 17:45 ` David Marchand
2021-10-11 8:56 ` [dpdk-dev] [PATCH v6 3/3] app/test: add allocator performance autotest Dmitry Kozlyuk
2021-10-12 13:53 ` Aaron Conole
2021-10-12 14:48 ` Dmitry Kozlyuk
2021-10-15 13:47 ` Aaron Conole
2021-10-11 18:52 ` [dpdk-dev] [PATCH v6 0/3] eal: add memory pre-allocation from existing files Thomas Monjalon
2021-10-11 21:12 ` [dpdk-dev] [dpdk-ci] " Lincoln Lavoie
2021-10-12 6:54 ` Thomas Monjalon
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210920125302.820273-1-dkozlyuk@nvidia.com \
--to=dkozlyuk@oss.nvidia.com \
--cc=anatoly.burakov@intel.com \
--cc=dev@dpdk.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).