From: Anatoly Burakov <anatoly.burakov@intel.com> To: dev@dpdk.org Cc: John McNamara <john.mcnamara@intel.com>, Marko Kovacevic <marko.kovacevic@intel.com>, ferruh.yigit@intel.com, bruce.richardson@intel.com, padraig.j.connolly@intel.com, stable@dpdk.org Subject: [dpdk-stable] [PATCH v3 2/2] doc/linux_gsg: update information on using hugepages Date: Tue, 25 Aug 2020 14:57:35 +0100 Message-ID: <6f93d916bbceb67e09083a2be05f23a572a0d614.1598363848.git.anatoly.burakov@intel.com> (raw) In-Reply-To: <242515b3b0d4ac57ee86cada96af90fb78e14997.1598363848.git.anatoly.burakov@intel.com> In-Reply-To: <196e97d2802cf2250577aaa113b9093b0beadb3d.1598357863.git.anatoly.burakov@intel.com> Current information regarding hugepage usage is a little out of date. Update it to include information on in-memory mode, as well as on default mountpoints provided by systemd. Cc: stable@dpdk.org Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com> Acked-by: Bruce Richardson <bruce.richardson@intel.com> --- Notes: v3: - Clarified wording around non-default hugepage sizes v2: - Reworked the description - Put runtime reservation first, and boot time as an alternative - Clarified wording and fixed typos - Mentioned that some kernel versions not supporting reserving 1G pages doc/guides/linux_gsg/sys_reqs.rst | 70 +++++++++++++++++++------------ 1 file changed, 44 insertions(+), 26 deletions(-) diff --git a/doc/guides/linux_gsg/sys_reqs.rst b/doc/guides/linux_gsg/sys_reqs.rst index a124656bcb..587f9e85e5 100644 --- a/doc/guides/linux_gsg/sys_reqs.rst +++ b/doc/guides/linux_gsg/sys_reqs.rst @@ -155,8 +155,35 @@ Without hugepages, high TLB miss rates would occur with the standard 4k page siz Reserving Hugepages for DPDK Use ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -The allocation of hugepages should be done at boot time or as soon as possible after system boot -to prevent memory from being fragmented in physical memory. +The reservation of hugepages can be performed at run time. This is done by +echoing the number of hugepages required to a ``nr_hugepages`` file in the +``/sys/kernel/`` directory corresponding to a specific page size (in +Kilobytes). For a single-node system, the command to use is as follows +(assuming that 1024 of 2MB pages are required):: + + echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages + +On a NUMA machine, the above command will usually divide the number of hugepages +equally across all NUMA nodes (assuming there is enough memory on all NUMA +nodes). However, pages can also be reserved explicitly on individual NUMA +nodes using a ``nr_hugepages`` file in the ``/sys/devices/`` directory:: + + echo 1024 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages + echo 1024 > /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages + +.. note:: + + Some kernel versions may not allow reserving 1 GB hugepages at run time, so + reserving them at boot time may be the only option. Please see below for + instructions. + +**Alternative:** + +In the general case, reserving hugepages at run time is perfectly fine, but in +use cases where having lots of physically contiguous memory is required, it is +preferable to reserve hugepages at boot time, as that will help in preventing +physical memory from becoming heavily fragmented. + To reserve hugepages at boot time, a parameter is passed to the Linux kernel on the kernel command line. For 2 MB pages, just pass the hugepages option to the kernel. For example, to reserve 1024 pages of 2 MB, use:: @@ -185,35 +212,26 @@ the number of hugepages reserved at boot time is generally divided equally betwe See the Documentation/admin-guide/kernel-parameters.txt file in your Linux source tree for further details of these and other kernel options. -**Alternative:** - -For 2 MB pages, there is also the option of allocating hugepages after the system has booted. -This is done by echoing the number of hugepages required to a nr_hugepages file in the ``/sys/devices/`` directory. -For a single-node system, the command to use is as follows (assuming that 1024 pages are required):: - - echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages - -On a NUMA machine, pages should be allocated explicitly on separate nodes:: - - echo 1024 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages - echo 1024 > /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages - -.. note:: - - For 1G pages, it is not possible to reserve the hugepage memory after the system has booted. - Using Hugepages with the DPDK ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Once the hugepage memory is reserved, to make the memory available for DPDK use, perform the following steps:: +If secondary process support is not required, DPDK is able to use hugepages +without any configuration by using "in-memory" mode. Please see +:ref:`linux_eal_parameters` for more details. + +If secondary process support is required, mount points for hugepages need to be +created. On modern Linux distributions, a default mount point for hugepages is provided +by the system and is located at ``/dev/hugepages``. This mount point will use the +default hugepage size set by the kernel parameters as described above. + +However, in order to use hugepage sizes other than the default, it is necessary +to manually create mount points for those hugepage sizes (e.g. 1GB pages). + +To make the hugepages of size 1GB available for DPDK use, perform the following steps:: mkdir /mnt/huge - mount -t hugetlbfs nodev /mnt/huge + mount -t hugetlbfs pagesize=1GB /mnt/huge The mount point can be made permanent across reboots, by adding the following line to the ``/etc/fstab`` file:: - nodev /mnt/huge hugetlbfs defaults 0 0 - -For 1GB pages, the page size must be specified as a mount option:: - - nodev /mnt/huge_1GB hugetlbfs pagesize=1GB 0 0 + nodev /mnt/huge hugetlbfs pagesize=1GB 0 0 -- 2.17.1
next prev parent reply other threads:[~2020-08-25 13:57 UTC|newest] Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-08-24 15:45 [dpdk-stable] [PATCH 1/2] doc/linux_gsg: clarify instructions on running as non-root Anatoly Burakov 2020-08-24 15:45 ` [dpdk-stable] [PATCH 2/2] doc/linux_gsg: update information on using hugepages Anatoly Burakov 2020-08-24 17:13 ` Bruce Richardson 2020-08-25 9:28 ` Burakov, Anatoly 2020-08-24 17:08 ` [dpdk-stable] [PATCH 1/2] doc/linux_gsg: clarify instructions on running as non-root Bruce Richardson 2020-08-25 9:29 ` Burakov, Anatoly 2020-08-25 7:47 ` Ferruh Yigit 2020-08-25 12:17 ` [dpdk-stable] [PATCH v2 " Anatoly Burakov 2020-08-25 13:06 ` Bruce Richardson 2020-08-25 13:57 ` [dpdk-stable] [PATCH v3 " Anatoly Burakov 2020-11-19 10:52 ` [dpdk-stable] [PATCH v4 1/2] doc: " Anatoly Burakov 2020-11-19 10:52 ` [dpdk-stable] [PATCH v4 2/2] doc/linux_gsg: update information on using hugepages Anatoly Burakov 2020-11-19 21:03 ` [dpdk-stable] [dpdk-dev] " David Marchand 2020-11-20 10:50 ` Burakov, Anatoly 2020-11-27 15:23 ` [dpdk-stable] " Thomas Monjalon 2020-08-25 13:57 ` Anatoly Burakov [this message] 2020-08-25 12:17 ` [dpdk-stable] [PATCH v2 " Anatoly Burakov 2020-08-25 13:10 ` Bruce Richardson
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=6f93d916bbceb67e09083a2be05f23a572a0d614.1598363848.git.anatoly.burakov@intel.com \ --to=anatoly.burakov@intel.com \ --cc=bruce.richardson@intel.com \ --cc=dev@dpdk.org \ --cc=ferruh.yigit@intel.com \ --cc=john.mcnamara@intel.com \ --cc=marko.kovacevic@intel.com \ --cc=padraig.j.connolly@intel.com \ --cc=stable@dpdk.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
patches for DPDK stable branches This inbox may be cloned and mirrored by anyone: git clone --mirror http://inbox.dpdk.org/stable/0 stable/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 stable stable/ http://inbox.dpdk.org/stable \ stable@dpdk.org public-inbox-index stable Example config snippet for mirrors. Newsgroup available over NNTP: nntp://inbox.dpdk.org/inbox.dpdk.stable AGPL code for this site: git clone https://public-inbox.org/public-inbox.git