From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 14DD2A04B2 for ; Tue, 25 Aug 2020 11:29:02 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id DFC591C1A3; Tue, 25 Aug 2020 11:29:01 +0200 (CEST) Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by dpdk.org (Postfix) with ESMTP id 2F35F1C191; Tue, 25 Aug 2020 11:28:57 +0200 (CEST) IronPort-SDR: EryplNW2PzSVc7pjDe1OJex1EsLwVbmGJUbT53zu1fGlPo6e0LBXqK9sUHwWMDzz1kDv2caowH kZkJbuEpYfJQ== X-IronPort-AV: E=McAfee;i="6000,8403,9723"; a="156069447" X-IronPort-AV: E=Sophos;i="5.76,352,1592895600"; d="scan'208";a="156069447" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Aug 2020 02:28:56 -0700 IronPort-SDR: tIxrmLQr5rGRT4fK/4TWvpbQ1agYPTIYW50lfoOct29llkP8ga5dnHvnUYfn1Mb8xxJBf69aOA ieV3ClTt89wA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,352,1592895600"; d="scan'208";a="322695508" Received: from aburakov-mobl.ger.corp.intel.com (HELO [10.213.236.34]) ([10.213.236.34]) by fmsmga004.fm.intel.com with ESMTP; 25 Aug 2020 02:28:55 -0700 To: Bruce Richardson Cc: dev@dpdk.org, John McNamara , Marko Kovacevic , ferruh.yigit@intel.com, padraig.j.connolly@intel.com, stable@dpdk.org References: <01efc239aaf6513b768829f7e44ad411cab881fe.1598283570.git.anatoly.burakov@intel.com> <20200824171307.GE547@bricha3-MOBL.ger.corp.intel.com> From: "Burakov, Anatoly" Message-ID: Date: Tue, 25 Aug 2020 10:28:53 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.11.0 MIME-Version: 1.0 In-Reply-To: <20200824171307.GE547@bricha3-MOBL.ger.corp.intel.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-stable] [PATCH 2/2] doc/linux_gsg: update information on using hugepages X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Sender: "stable" Archived-At: List-Archive: On 24-Aug-20 6:13 PM, Bruce Richardson wrote: > On Mon, Aug 24, 2020 at 04:45:01PM +0100, Anatoly Burakov wrote: >> Current information regarding hugepage usage is a little out of date. >> Update it to include information on in-memory mode, as well as on >> default mountpoints provided by systemd. >> >> Cc: stable@dpdk.org >> >> Signed-off-by: Anatoly Burakov >> --- >> doc/guides/linux_gsg/sys_reqs.rst | 39 +++++++++++++++++++------------ >> 1 file changed, 24 insertions(+), 15 deletions(-) >> >> diff --git a/doc/guides/linux_gsg/sys_reqs.rst b/doc/guides/linux_gsg/sys_reqs.rst >> index a124656bcb..2ddd7ed667 100644 >> --- a/doc/guides/linux_gsg/sys_reqs.rst >> +++ b/doc/guides/linux_gsg/sys_reqs.rst >> @@ -155,8 +155,12 @@ Without hugepages, high TLB miss rates would occur with the standard 4k page siz >> Reserving Hugepages for DPDK Use >> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >> >> -The allocation of hugepages should be done at boot time or as soon as possible after system boot >> -to prevent memory from being fragmented in physical memory. >> +The allocation of hugepages can be performed either at run time or at boot time. >> +In the general case, reserving hugepages at run time is perfectly fine, but in >> +use cases where having lots of physically contiguous memory is required, it is >> +preferable to reserve hugepages at boot time, as that will help in preventing >> +physical memory from becoming heavily fragmented. >> + > > Although we are removing the note about 1G pages requiring to be reserved > at boot time, I think we should still mention here that some older kernel > versions do not allow 1G reservations post-boot. Agreed, will fix in v2. > >> To reserve hugepages at boot time, a parameter is passed to the Linux kernel on the kernel command line. >> >> For 2 MB pages, just pass the hugepages option to the kernel. For example, to reserve 1024 pages of 2 MB, use:: >> @@ -187,9 +191,9 @@ See the Documentation/admin-guide/kernel-parameters.txt file in your Linux sourc >> >> **Alternative:** >> >> -For 2 MB pages, there is also the option of allocating hugepages after the system has booted. >> +There is also the option of allocating hugepages after the system has booted. >> This is done by echoing the number of hugepages required to a nr_hugepages file in the ``/sys/devices/`` directory. >> -For a single-node system, the command to use is as follows (assuming that 1024 pages are required):: >> +For a single-node system, the command to use is as follows (assuming that 1024 of 2MB pages are required):: >> >> echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages >> >> @@ -198,22 +202,27 @@ On a NUMA machine, pages should be allocated explicitly on separate nodes:: >> echo 1024 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages >> echo 1024 > /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages >> >> -.. note:: >> - >> - For 1G pages, it is not possible to reserve the hugepage memory after the system has booted. >> - >> Using Hugepages with the DPDK >> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >> >> -Once the hugepage memory is reserved, to make the memory available for DPDK use, perform the following steps:: >> +If secondary process support is not required, DPDK is able to use hugepages >> +without any configuration by using "in-memory" mode. Please see >> +:ref:`linux_eal_parameters` for more details. >> + >> +If secondary process support is required, mount points for hugepages need to be >> +created. On modern Linux distributions, a default mount point for hugepages is provided >> +by the system and is located at ``/dev/hugepages``. This mount point will use the >> +default hugepage size set by the kernel parameters as described above. >> + >> +However, in order to use multiple hugepage sizes, it is necessary to manually > > Rather than multiple hugepage sizes, I'd suggest changing this to hugepage > sizes other than the default. OK, will fix. > > Do we also want to add a line somewhere explaining that the default size > can be set a boot using a kernel parameter? It's already there, right above this :) > >> +create mount points for hugepage sizes that are not provided by the system >> +(e.g. 1GB pages). >> + >> +To make the hugepages of size 1GB available for DPDK use, perform the following steps:: >> >> mkdir /mnt/huge >> - mount -t hugetlbfs nodev /mnt/huge >> + mount -t hugetlbfs pagesize=1GB /mnt/huge >> >> The mount point can be made permanent across reboots, by adding the following line to the ``/etc/fstab`` file:: >> >> - nodev /mnt/huge hugetlbfs defaults 0 0 >> - >> -For 1GB pages, the page size must be specified as a mount option:: >> - >> - nodev /mnt/huge_1GB hugetlbfs pagesize=1GB 0 0 >> + nodev /mnt/huge hugetlbfs pagesize=1GB 0 0 >> -- >> 2.17.1 -- Thanks, Anatoly