DPDK patches and discussions
 help / color / mirror / Atom feed
From: Anatoly Burakov <anatoly.burakov@intel.com>
To: dev@dpdk.org
Cc: John McNamara <john.mcnamara@intel.com>,
	Marko Kovacevic <marko.kovacevic@intel.com>,
	ferruh.yigit@intel.com, bruce.richardson@intel.com,
	padraig.j.connolly@intel.com, stable@dpdk.org
Subject: [dpdk-dev] [PATCH v3 2/2] doc/linux_gsg: update information on using hugepages
Date: Tue, 25 Aug 2020 14:57:35 +0100	[thread overview]
Message-ID: <6f93d916bbceb67e09083a2be05f23a572a0d614.1598363848.git.anatoly.burakov@intel.com> (raw)
In-Reply-To: <242515b3b0d4ac57ee86cada96af90fb78e14997.1598363848.git.anatoly.burakov@intel.com>
In-Reply-To: <196e97d2802cf2250577aaa113b9093b0beadb3d.1598357863.git.anatoly.burakov@intel.com>

Current information regarding hugepage usage is a little out of date.
Update it to include information on in-memory mode, as well as on
default mountpoints provided by systemd.

Cc: stable@dpdk.org

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
---

Notes:
    v3:
    - Clarified wording around non-default hugepage sizes
    
    v2:
    - Reworked the description
    - Put runtime reservation first, and boot time as an alternative
    - Clarified wording and fixed typos
    - Mentioned that some kernel versions not supporting reserving 1G pages

 doc/guides/linux_gsg/sys_reqs.rst | 70 +++++++++++++++++++------------
 1 file changed, 44 insertions(+), 26 deletions(-)

diff --git a/doc/guides/linux_gsg/sys_reqs.rst b/doc/guides/linux_gsg/sys_reqs.rst
index a124656bcb..587f9e85e5 100644
--- a/doc/guides/linux_gsg/sys_reqs.rst
+++ b/doc/guides/linux_gsg/sys_reqs.rst
@@ -155,8 +155,35 @@ Without hugepages, high TLB miss rates would occur with the standard 4k page siz
 Reserving Hugepages for DPDK Use
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-The allocation of hugepages should be done at boot time or as soon as possible after system boot
-to prevent memory from being fragmented in physical memory.
+The reservation of hugepages can be performed at run time. This is done by
+echoing the number of hugepages required to a ``nr_hugepages`` file in the
+``/sys/kernel/`` directory corresponding to a specific page size (in
+Kilobytes). For a single-node system, the command to use is as follows
+(assuming that 1024 of 2MB pages are required)::
+
+    echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
+
+On a NUMA machine, the above command will usually divide the number of hugepages
+equally across all NUMA nodes (assuming there is enough memory on all NUMA
+nodes). However, pages can also be reserved explicitly on individual NUMA
+nodes using a ``nr_hugepages`` file in the ``/sys/devices/`` directory::
+
+    echo 1024 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
+    echo 1024 > /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages
+
+.. note::
+
+    Some kernel versions may not allow reserving 1 GB hugepages at run time, so
+    reserving them at boot time may be the only option. Please see below for
+    instructions.
+
+**Alternative:**
+
+In the general case, reserving hugepages at run time is perfectly fine, but in
+use cases where having lots of physically contiguous memory is required, it is
+preferable to reserve hugepages at boot time, as that will help in preventing
+physical memory from becoming heavily fragmented.
+
 To reserve hugepages at boot time, a parameter is passed to the Linux kernel on the kernel command line.
 
 For 2 MB pages, just pass the hugepages option to the kernel. For example, to reserve 1024 pages of 2 MB, use::
@@ -185,35 +212,26 @@ the number of hugepages reserved at boot time is generally divided equally betwe
 
 See the Documentation/admin-guide/kernel-parameters.txt file in your Linux source tree for further details of these and other kernel options.
 
-**Alternative:**
-
-For 2 MB pages, there is also the option of allocating hugepages after the system has booted.
-This is done by echoing the number of hugepages required to a nr_hugepages file in the ``/sys/devices/`` directory.
-For a single-node system, the command to use is as follows (assuming that 1024 pages are required)::
-
-    echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
-
-On a NUMA machine, pages should be allocated explicitly on separate nodes::
-
-    echo 1024 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
-    echo 1024 > /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages
-
-.. note::
-
-    For 1G pages, it is not possible to reserve the hugepage memory after the system has booted.
-
 Using Hugepages with the DPDK
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-Once the hugepage memory is reserved, to make the memory available for DPDK use, perform the following steps::
+If secondary process support is not required, DPDK is able to use hugepages
+without any configuration by using "in-memory" mode. Please see
+:ref:`linux_eal_parameters` for more details.
+
+If secondary process support is required, mount points for hugepages need to be
+created. On modern Linux distributions, a default mount point for hugepages is provided
+by the system and is located at ``/dev/hugepages``. This mount point will use the
+default hugepage size set by the kernel parameters as described above.
+
+However, in order to use hugepage sizes other than the default, it is necessary
+to manually create mount points for those hugepage sizes (e.g. 1GB pages).
+
+To make the hugepages of size 1GB available for DPDK use, perform the following steps::
 
     mkdir /mnt/huge
-    mount -t hugetlbfs nodev /mnt/huge
+    mount -t hugetlbfs pagesize=1GB /mnt/huge
 
 The mount point can be made permanent across reboots, by adding the following line to the ``/etc/fstab`` file::
 
-    nodev /mnt/huge hugetlbfs defaults 0 0
-
-For 1GB pages, the page size must be specified as a mount option::
-
-    nodev /mnt/huge_1GB hugetlbfs pagesize=1GB 0 0
+    nodev /mnt/huge hugetlbfs pagesize=1GB 0 0
-- 
2.17.1

  parent reply	other threads:[~2020-08-25 13:57 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-08-24 15:45 [dpdk-dev] [PATCH 1/2] doc/linux_gsg: clarify instructions on running as non-root Anatoly Burakov
2020-08-24 15:45 ` [dpdk-dev] [PATCH 2/2] doc/linux_gsg: update information on using hugepages Anatoly Burakov
2020-08-24 17:13   ` Bruce Richardson
2020-08-25  9:28     ` Burakov, Anatoly
2020-08-24 17:08 ` [dpdk-dev] [PATCH 1/2] doc/linux_gsg: clarify instructions on running as non-root Bruce Richardson
2020-08-25  9:29   ` Burakov, Anatoly
2020-08-25  7:47 ` Ferruh Yigit
2020-08-25 12:17 ` [dpdk-dev] [PATCH v2 " Anatoly Burakov
2020-08-25 13:06   ` Bruce Richardson
2020-08-25 13:57   ` [dpdk-dev] [PATCH v3 " Anatoly Burakov
2020-11-19 10:52     ` [dpdk-dev] [PATCH v4 1/2] doc: " Anatoly Burakov
2020-11-19 10:52     ` [dpdk-dev] [PATCH v4 2/2] doc/linux_gsg: update information on using hugepages Anatoly Burakov
2020-11-19 21:03       ` David Marchand
2020-11-20 10:50         ` Burakov, Anatoly
2020-11-27 15:23       ` [dpdk-dev] [dpdk-stable] " Thomas Monjalon
2020-08-25 13:57   ` Anatoly Burakov [this message]
2020-08-25 12:17 ` [dpdk-dev] [PATCH v2 " Anatoly Burakov
2020-08-25 13:10   ` Bruce Richardson

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=6f93d916bbceb67e09083a2be05f23a572a0d614.1598363848.git.anatoly.burakov@intel.com \
    --to=anatoly.burakov@intel.com \
    --cc=bruce.richardson@intel.com \
    --cc=dev@dpdk.org \
    --cc=ferruh.yigit@intel.com \
    --cc=john.mcnamara@intel.com \
    --cc=marko.kovacevic@intel.com \
    --cc=padraig.j.connolly@intel.com \
    --cc=stable@dpdk.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).