From: Bruce Richardson <bruce.richardson@intel.com>
To: Anatoly Burakov <anatoly.burakov@intel.com>
Cc: dev@dpdk.org, John McNamara <john.mcnamara@intel.com>,
Marko Kovacevic <marko.kovacevic@intel.com>,
ferruh.yigit@intel.com, padraig.j.connolly@intel.com,
stable@dpdk.org
Subject: Re: [dpdk-dev] [PATCH 2/2] doc/linux_gsg: update information on using hugepages
Date: Mon, 24 Aug 2020 18:13:07 +0100 [thread overview]
Message-ID: <20200824171307.GE547@bricha3-MOBL.ger.corp.intel.com> (raw)
In-Reply-To: <01efc239aaf6513b768829f7e44ad411cab881fe.1598283570.git.anatoly.burakov@intel.com>
On Mon, Aug 24, 2020 at 04:45:01PM +0100, Anatoly Burakov wrote:
> Current information regarding hugepage usage is a little out of date.
> Update it to include information on in-memory mode, as well as on
> default mountpoints provided by systemd.
>
> Cc: stable@dpdk.org
>
> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
> ---
> doc/guides/linux_gsg/sys_reqs.rst | 39 +++++++++++++++++++------------
> 1 file changed, 24 insertions(+), 15 deletions(-)
>
> diff --git a/doc/guides/linux_gsg/sys_reqs.rst b/doc/guides/linux_gsg/sys_reqs.rst
> index a124656bcb..2ddd7ed667 100644
> --- a/doc/guides/linux_gsg/sys_reqs.rst
> +++ b/doc/guides/linux_gsg/sys_reqs.rst
> @@ -155,8 +155,12 @@ Without hugepages, high TLB miss rates would occur with the standard 4k page siz
> Reserving Hugepages for DPDK Use
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> -The allocation of hugepages should be done at boot time or as soon as possible after system boot
> -to prevent memory from being fragmented in physical memory.
> +The allocation of hugepages can be performed either at run time or at boot time.
> +In the general case, reserving hugepages at run time is perfectly fine, but in
> +use cases where having lots of physically contiguous memory is required, it is
> +preferable to reserve hugepages at boot time, as that will help in preventing
> +physical memory from becoming heavily fragmented.
> +
Although we are removing the note about 1G pages requiring to be reserved
at boot time, I think we should still mention here that some older kernel
versions do not allow 1G reservations post-boot.
> To reserve hugepages at boot time, a parameter is passed to the Linux kernel on the kernel command line.
>
> For 2 MB pages, just pass the hugepages option to the kernel. For example, to reserve 1024 pages of 2 MB, use::
> @@ -187,9 +191,9 @@ See the Documentation/admin-guide/kernel-parameters.txt file in your Linux sourc
>
> **Alternative:**
>
> -For 2 MB pages, there is also the option of allocating hugepages after the system has booted.
> +There is also the option of allocating hugepages after the system has booted.
> This is done by echoing the number of hugepages required to a nr_hugepages file in the ``/sys/devices/`` directory.
> -For a single-node system, the command to use is as follows (assuming that 1024 pages are required)::
> +For a single-node system, the command to use is as follows (assuming that 1024 of 2MB pages are required)::
>
> echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
>
> @@ -198,22 +202,27 @@ On a NUMA machine, pages should be allocated explicitly on separate nodes::
> echo 1024 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
> echo 1024 > /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages
>
> -.. note::
> -
> - For 1G pages, it is not possible to reserve the hugepage memory after the system has booted.
> -
> Using Hugepages with the DPDK
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> -Once the hugepage memory is reserved, to make the memory available for DPDK use, perform the following steps::
> +If secondary process support is not required, DPDK is able to use hugepages
> +without any configuration by using "in-memory" mode. Please see
> +:ref:`linux_eal_parameters` for more details.
> +
> +If secondary process support is required, mount points for hugepages need to be
> +created. On modern Linux distributions, a default mount point for hugepages is provided
> +by the system and is located at ``/dev/hugepages``. This mount point will use the
> +default hugepage size set by the kernel parameters as described above.
> +
> +However, in order to use multiple hugepage sizes, it is necessary to manually
Rather than multiple hugepage sizes, I'd suggest changing this to hugepage
sizes other than the default.
Do we also want to add a line somewhere explaining that the default size
can be set a boot using a kernel parameter?
> +create mount points for hugepage sizes that are not provided by the system
> +(e.g. 1GB pages).
> +
> +To make the hugepages of size 1GB available for DPDK use, perform the following steps::
>
> mkdir /mnt/huge
> - mount -t hugetlbfs nodev /mnt/huge
> + mount -t hugetlbfs pagesize=1GB /mnt/huge
>
> The mount point can be made permanent across reboots, by adding the following line to the ``/etc/fstab`` file::
>
> - nodev /mnt/huge hugetlbfs defaults 0 0
> -
> -For 1GB pages, the page size must be specified as a mount option::
> -
> - nodev /mnt/huge_1GB hugetlbfs pagesize=1GB 0 0
> + nodev /mnt/huge hugetlbfs pagesize=1GB 0 0
> --
> 2.17.1
next prev parent reply other threads:[~2020-08-24 17:13 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-08-24 15:45 [dpdk-dev] [PATCH 1/2] doc/linux_gsg: clarify instructions on running as non-root Anatoly Burakov
2020-08-24 15:45 ` [dpdk-dev] [PATCH 2/2] doc/linux_gsg: update information on using hugepages Anatoly Burakov
2020-08-24 17:13 ` Bruce Richardson [this message]
2020-08-25 9:28 ` Burakov, Anatoly
2020-08-24 17:08 ` [dpdk-dev] [PATCH 1/2] doc/linux_gsg: clarify instructions on running as non-root Bruce Richardson
2020-08-25 9:29 ` Burakov, Anatoly
2020-08-25 7:47 ` Ferruh Yigit
2020-08-25 12:17 ` [dpdk-dev] [PATCH v2 " Anatoly Burakov
2020-08-25 13:06 ` Bruce Richardson
2020-08-25 13:57 ` [dpdk-dev] [PATCH v3 " Anatoly Burakov
2020-11-19 10:52 ` [dpdk-dev] [PATCH v4 1/2] doc: " Anatoly Burakov
2020-11-19 10:52 ` [dpdk-dev] [PATCH v4 2/2] doc/linux_gsg: update information on using hugepages Anatoly Burakov
2020-11-19 21:03 ` David Marchand
2020-11-20 10:50 ` Burakov, Anatoly
2020-11-27 15:23 ` [dpdk-dev] [dpdk-stable] " Thomas Monjalon
2020-08-25 13:57 ` [dpdk-dev] [PATCH v3 " Anatoly Burakov
2020-08-25 12:17 ` [dpdk-dev] [PATCH v2 " Anatoly Burakov
2020-08-25 13:10 ` Bruce Richardson
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200824171307.GE547@bricha3-MOBL.ger.corp.intel.com \
--to=bruce.richardson@intel.com \
--cc=anatoly.burakov@intel.com \
--cc=dev@dpdk.org \
--cc=ferruh.yigit@intel.com \
--cc=john.mcnamara@intel.com \
--cc=marko.kovacevic@intel.com \
--cc=padraig.j.connolly@intel.com \
--cc=stable@dpdk.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).