From: "Burakov, Anatoly" <anatoly.burakov@intel.com>
To: Bruce Richardson <bruce.richardson@intel.com>
Cc: dev@dpdk.org, John McNamara <john.mcnamara@intel.com>,
Marko Kovacevic <marko.kovacevic@intel.com>,
ferruh.yigit@intel.com, padraig.j.connolly@intel.com,
stable@dpdk.org
Subject: Re: [dpdk-stable] [PATCH 2/2] doc/linux_gsg: update information on using hugepages
Date: Tue, 25 Aug 2020 10:28:53 +0100 [thread overview]
Message-ID: <f9aa59e3-2f51-a5c3-e404-423d78dfb56e@intel.com> (raw)
In-Reply-To: <20200824171307.GE547@bricha3-MOBL.ger.corp.intel.com>
On 24-Aug-20 6:13 PM, Bruce Richardson wrote:
> On Mon, Aug 24, 2020 at 04:45:01PM +0100, Anatoly Burakov wrote:
>> Current information regarding hugepage usage is a little out of date.
>> Update it to include information on in-memory mode, as well as on
>> default mountpoints provided by systemd.
>>
>> Cc: stable@dpdk.org
>>
>> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
>> ---
>> doc/guides/linux_gsg/sys_reqs.rst | 39 +++++++++++++++++++------------
>> 1 file changed, 24 insertions(+), 15 deletions(-)
>>
>> diff --git a/doc/guides/linux_gsg/sys_reqs.rst b/doc/guides/linux_gsg/sys_reqs.rst
>> index a124656bcb..2ddd7ed667 100644
>> --- a/doc/guides/linux_gsg/sys_reqs.rst
>> +++ b/doc/guides/linux_gsg/sys_reqs.rst
>> @@ -155,8 +155,12 @@ Without hugepages, high TLB miss rates would occur with the standard 4k page siz
>> Reserving Hugepages for DPDK Use
>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>
>> -The allocation of hugepages should be done at boot time or as soon as possible after system boot
>> -to prevent memory from being fragmented in physical memory.
>> +The allocation of hugepages can be performed either at run time or at boot time.
>> +In the general case, reserving hugepages at run time is perfectly fine, but in
>> +use cases where having lots of physically contiguous memory is required, it is
>> +preferable to reserve hugepages at boot time, as that will help in preventing
>> +physical memory from becoming heavily fragmented.
>> +
>
> Although we are removing the note about 1G pages requiring to be reserved
> at boot time, I think we should still mention here that some older kernel
> versions do not allow 1G reservations post-boot.
Agreed, will fix in v2.
>
>> To reserve hugepages at boot time, a parameter is passed to the Linux kernel on the kernel command line.
>>
>> For 2 MB pages, just pass the hugepages option to the kernel. For example, to reserve 1024 pages of 2 MB, use::
>> @@ -187,9 +191,9 @@ See the Documentation/admin-guide/kernel-parameters.txt file in your Linux sourc
>>
>> **Alternative:**
>>
>> -For 2 MB pages, there is also the option of allocating hugepages after the system has booted.
>> +There is also the option of allocating hugepages after the system has booted.
>> This is done by echoing the number of hugepages required to a nr_hugepages file in the ``/sys/devices/`` directory.
>> -For a single-node system, the command to use is as follows (assuming that 1024 pages are required)::
>> +For a single-node system, the command to use is as follows (assuming that 1024 of 2MB pages are required)::
>>
>> echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
>>
>> @@ -198,22 +202,27 @@ On a NUMA machine, pages should be allocated explicitly on separate nodes::
>> echo 1024 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
>> echo 1024 > /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages
>>
>> -.. note::
>> -
>> - For 1G pages, it is not possible to reserve the hugepage memory after the system has booted.
>> -
>> Using Hugepages with the DPDK
>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>
>> -Once the hugepage memory is reserved, to make the memory available for DPDK use, perform the following steps::
>> +If secondary process support is not required, DPDK is able to use hugepages
>> +without any configuration by using "in-memory" mode. Please see
>> +:ref:`linux_eal_parameters` for more details.
>> +
>> +If secondary process support is required, mount points for hugepages need to be
>> +created. On modern Linux distributions, a default mount point for hugepages is provided
>> +by the system and is located at ``/dev/hugepages``. This mount point will use the
>> +default hugepage size set by the kernel parameters as described above.
>> +
>> +However, in order to use multiple hugepage sizes, it is necessary to manually
>
> Rather than multiple hugepage sizes, I'd suggest changing this to hugepage
> sizes other than the default.
OK, will fix.
>
> Do we also want to add a line somewhere explaining that the default size
> can be set a boot using a kernel parameter?
It's already there, right above this :)
>
>> +create mount points for hugepage sizes that are not provided by the system
>> +(e.g. 1GB pages).
>> +
>> +To make the hugepages of size 1GB available for DPDK use, perform the following steps::
>>
>> mkdir /mnt/huge
>> - mount -t hugetlbfs nodev /mnt/huge
>> + mount -t hugetlbfs pagesize=1GB /mnt/huge
>>
>> The mount point can be made permanent across reboots, by adding the following line to the ``/etc/fstab`` file::
>>
>> - nodev /mnt/huge hugetlbfs defaults 0 0
>> -
>> -For 1GB pages, the page size must be specified as a mount option::
>> -
>> - nodev /mnt/huge_1GB hugetlbfs pagesize=1GB 0 0
>> + nodev /mnt/huge hugetlbfs pagesize=1GB 0 0
>> --
>> 2.17.1
--
Thanks,
Anatoly
next prev parent reply other threads:[~2020-08-25 9:29 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-08-24 15:45 [dpdk-stable] [PATCH 1/2] doc/linux_gsg: clarify instructions on running as non-root Anatoly Burakov
2020-08-24 15:45 ` [dpdk-stable] [PATCH 2/2] doc/linux_gsg: update information on using hugepages Anatoly Burakov
2020-08-24 17:13 ` Bruce Richardson
2020-08-25 9:28 ` Burakov, Anatoly [this message]
2020-08-24 17:08 ` [dpdk-stable] [PATCH 1/2] doc/linux_gsg: clarify instructions on running as non-root Bruce Richardson
2020-08-25 9:29 ` Burakov, Anatoly
2020-08-25 7:47 ` Ferruh Yigit
2020-08-25 12:17 ` [dpdk-stable] [PATCH v2 " Anatoly Burakov
2020-08-25 13:06 ` Bruce Richardson
2020-08-25 13:57 ` [dpdk-stable] [PATCH v3 " Anatoly Burakov
2020-11-19 10:52 ` [dpdk-stable] [PATCH v4 1/2] doc: " Anatoly Burakov
2020-11-19 10:52 ` [dpdk-stable] [PATCH v4 2/2] doc/linux_gsg: update information on using hugepages Anatoly Burakov
2020-11-19 21:03 ` [dpdk-stable] [dpdk-dev] " David Marchand
2020-11-20 10:50 ` Burakov, Anatoly
2020-11-27 15:23 ` [dpdk-stable] " Thomas Monjalon
2020-08-25 13:57 ` [dpdk-stable] [PATCH v3 " Anatoly Burakov
2020-08-25 12:17 ` [dpdk-stable] [PATCH v2 " Anatoly Burakov
2020-08-25 13:10 ` Bruce Richardson
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=f9aa59e3-2f51-a5c3-e404-423d78dfb56e@intel.com \
--to=anatoly.burakov@intel.com \
--cc=bruce.richardson@intel.com \
--cc=dev@dpdk.org \
--cc=ferruh.yigit@intel.com \
--cc=john.mcnamara@intel.com \
--cc=marko.kovacevic@intel.com \
--cc=padraig.j.connolly@intel.com \
--cc=stable@dpdk.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).