patches for DPDK stable branches
 help / color / mirror / Atom feed
From: Bruce Richardson <bruce.richardson@intel.com>
To: Anatoly Burakov <anatoly.burakov@intel.com>
Cc: dev@dpdk.org, John McNamara <john.mcnamara@intel.com>,
	Marko Kovacevic <marko.kovacevic@intel.com>,
	ferruh.yigit@intel.com, padraig.j.connolly@intel.com,
	stable@dpdk.org
Subject: Re: [dpdk-stable] [PATCH v2 2/2] doc/linux_gsg: update information on using hugepages
Date: Tue, 25 Aug 2020 14:10:40 +0100
Message-ID: <20200825131040.GB554@bricha3-MOBL.ger.corp.intel.com> (raw)
In-Reply-To: <5f68d1f573f9edee2aed8c3d81b655416c18dff0.1598357863.git.anatoly.burakov@intel.com>

On Tue, Aug 25, 2020 at 01:17:49PM +0100, Anatoly Burakov wrote:
> Current information regarding hugepage usage is a little out of date.
> Update it to include information on in-memory mode, as well as on
> default mountpoints provided by systemd.
> 
> Cc: stable@dpdk.org
> 
> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
> ---
> 
> Notes:
>     v2:
>     - Reworked the description
>     - Put runtime reservation first, and boot time as an alternative
>     - Clarified wording and fixed typos
>     - Mentioned that some kernel versions not supporting reserving 1G pages
> 
>  doc/guides/linux_gsg/sys_reqs.rst | 71 ++++++++++++++++++++-----------
>  1 file changed, 45 insertions(+), 26 deletions(-)
> 
> diff --git a/doc/guides/linux_gsg/sys_reqs.rst b/doc/guides/linux_gsg/sys_reqs.rst
> index a124656bcb..8782d05579 100644
> --- a/doc/guides/linux_gsg/sys_reqs.rst
> +++ b/doc/guides/linux_gsg/sys_reqs.rst
> @@ -155,8 +155,35 @@ Without hugepages, high TLB miss rates would occur with the standard 4k page siz
>  Reserving Hugepages for DPDK Use
>  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>  
> -The allocation of hugepages should be done at boot time or as soon as possible after system boot
> -to prevent memory from being fragmented in physical memory.
> +The reservation of hugepages can be performed at run time. This is done by
> +echoing the number of hugepages required to a ``nr_hugepages`` file in the
> +``/sys/kernel/`` directory corresponding to a specific page size (in
> +Kilobytes). For a single-node system, the command to use is as follows
> +(assuming that 1024 of 2MB pages are required)::
> +
> +    echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
> +
> +On a NUMA machine, the above command will usually divide the number of hugepages
> +equally across all NUMA nodes (assuming there is enough memory on all NUMA
> +nodes). However, pages can also be reserved explicitly on individual NUMA
> +nodes using a ``nr_hugepages`` file in the ``/sys/devices/`` directory::
> +
> +    echo 1024 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
> +    echo 1024 > /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages
> +
> +.. note::
> +
> +    Some kernel versions may not allow reserving 1 GB hugepages at run time, so
> +    reserving them at boot time may be the only option. Please see below for
> +    instructions.
> +
> +**Alternative:**
> +
> +In the general case, reserving hugepages at run time is perfectly fine, but in
> +use cases where having lots of physically contiguous memory is required, it is
> +preferable to reserve hugepages at boot time, as that will help in preventing
> +physical memory from becoming heavily fragmented.
> +
>  To reserve hugepages at boot time, a parameter is passed to the Linux kernel on the kernel command line.
>  
>  For 2 MB pages, just pass the hugepages option to the kernel. For example, to reserve 1024 pages of 2 MB, use::
> @@ -185,35 +212,27 @@ the number of hugepages reserved at boot time is generally divided equally betwe
>  
>  See the Documentation/admin-guide/kernel-parameters.txt file in your Linux source tree for further details of these and other kernel options.
>  
> -**Alternative:**
> -
> -For 2 MB pages, there is also the option of allocating hugepages after the system has booted.
> -This is done by echoing the number of hugepages required to a nr_hugepages file in the ``/sys/devices/`` directory.
> -For a single-node system, the command to use is as follows (assuming that 1024 pages are required)::
> -
> -    echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
> -
> -On a NUMA machine, pages should be allocated explicitly on separate nodes::
> -
> -    echo 1024 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
> -    echo 1024 > /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages
> -
> -.. note::
> -
> -    For 1G pages, it is not possible to reserve the hugepage memory after the system has booted.
> -
>  Using Hugepages with the DPDK
>  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>  
> -Once the hugepage memory is reserved, to make the memory available for DPDK use, perform the following steps::
> +If secondary process support is not required, DPDK is able to use hugepages
> +without any configuration by using "in-memory" mode. Please see
> +:ref:`linux_eal_parameters` for more details.
> +
> +If secondary process support is required, mount points for hugepages need to be
> +created. On modern Linux distributions, a default mount point for hugepages is provided
> +by the system and is located at ``/dev/hugepages``. This mount point will use the
> +default hugepage size set by the kernel parameters as described above.
> +
> +However, in order to use hugepage sizes other than default, it is necessary to
> +manually create mount points for hugepage sizes that are not provided by the
> +system (e.g. 1GB pages).

This reads a bit strangely, as it implies that the hugepage sizes are not
provided by the system, but I believe the intention is to say that the
mount points are not provided by the system, correct? Perhaps look to
reword.

> +
> +To make the hugepages of size 1GB available for DPDK use, perform the following steps::
>  
>      mkdir /mnt/huge
> -    mount -t hugetlbfs nodev /mnt/huge
> +    mount -t hugetlbfs pagesize=1GB /mnt/huge
>  
>  The mount point can be made permanent across reboots, by adding the following line to the ``/etc/fstab`` file::
>  
> -    nodev /mnt/huge hugetlbfs defaults 0 0
> -
> -For 1GB pages, the page size must be specified as a mount option::
> -
> -    nodev /mnt/huge_1GB hugetlbfs pagesize=1GB 0 0
> +    nodev /mnt/huge hugetlbfs pagesize=1GB 0 0
> -- 
> 2.17.1

Apart from the one note above, LGTM:

Acked-by: Bruce Richardson <bruce.richardson@intel.com>


      reply	other threads:[~2020-08-25 13:10 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-08-24 15:45 [dpdk-stable] [PATCH 1/2] doc/linux_gsg: clarify instructions on running as non-root Anatoly Burakov
2020-08-24 15:45 ` [dpdk-stable] [PATCH 2/2] doc/linux_gsg: update information on using hugepages Anatoly Burakov
2020-08-24 17:13   ` Bruce Richardson
2020-08-25  9:28     ` Burakov, Anatoly
2020-08-24 17:08 ` [dpdk-stable] [PATCH 1/2] doc/linux_gsg: clarify instructions on running as non-root Bruce Richardson
2020-08-25  9:29   ` Burakov, Anatoly
2020-08-25  7:47 ` Ferruh Yigit
2020-08-25 12:17 ` [dpdk-stable] [PATCH v2 " Anatoly Burakov
2020-08-25 13:06   ` Bruce Richardson
2020-08-25 13:57   ` [dpdk-stable] [PATCH v3 " Anatoly Burakov
2020-11-19 10:52     ` [dpdk-stable] [PATCH v4 1/2] doc: " Anatoly Burakov
2020-11-19 10:52     ` [dpdk-stable] [PATCH v4 2/2] doc/linux_gsg: update information on using hugepages Anatoly Burakov
2020-11-19 21:03       ` [dpdk-stable] [dpdk-dev] " David Marchand
2020-11-20 10:50         ` Burakov, Anatoly
2020-11-27 15:23       ` [dpdk-stable] " Thomas Monjalon
2020-08-25 13:57   ` [dpdk-stable] [PATCH v3 " Anatoly Burakov
2020-08-25 12:17 ` [dpdk-stable] [PATCH v2 " Anatoly Burakov
2020-08-25 13:10   ` Bruce Richardson [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200825131040.GB554@bricha3-MOBL.ger.corp.intel.com \
    --to=bruce.richardson@intel.com \
    --cc=anatoly.burakov@intel.com \
    --cc=dev@dpdk.org \
    --cc=ferruh.yigit@intel.com \
    --cc=john.mcnamara@intel.com \
    --cc=marko.kovacevic@intel.com \
    --cc=padraig.j.connolly@intel.com \
    --cc=stable@dpdk.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

patches for DPDK stable branches

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://inbox.dpdk.org/stable/0 stable/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 stable stable/ https://inbox.dpdk.org/stable \
		stable@dpdk.org
	public-inbox-index stable

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.stable


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git