From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <stable-bounces@dpdk.org>
Received: from dpdk.org (dpdk.org [92.243.14.124])
	by inbox.dpdk.org (Postfix) with ESMTP id 14DD2A04B2
	for <public@inbox.dpdk.org>; Tue, 25 Aug 2020 11:29:02 +0200 (CEST)
Received: from [92.243.14.124] (localhost [127.0.0.1])
	by dpdk.org (Postfix) with ESMTP id DFC591C1A3;
	Tue, 25 Aug 2020 11:29:01 +0200 (CEST)
Received: from mga03.intel.com (mga03.intel.com [134.134.136.65])
 by dpdk.org (Postfix) with ESMTP id 2F35F1C191;
 Tue, 25 Aug 2020 11:28:57 +0200 (CEST)
IronPort-SDR: EryplNW2PzSVc7pjDe1OJex1EsLwVbmGJUbT53zu1fGlPo6e0LBXqK9sUHwWMDzz1kDv2caowH
 kZkJbuEpYfJQ==
X-IronPort-AV: E=McAfee;i="6000,8403,9723"; a="156069447"
X-IronPort-AV: E=Sophos;i="5.76,352,1592895600"; d="scan'208";a="156069447"
X-Amp-Result: SKIPPED(no attachment in message)
X-Amp-File-Uploaded: False
Received: from fmsmga004.fm.intel.com ([10.253.24.48])
 by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;
 25 Aug 2020 02:28:56 -0700
IronPort-SDR: tIxrmLQr5rGRT4fK/4TWvpbQ1agYPTIYW50lfoOct29llkP8ga5dnHvnUYfn1Mb8xxJBf69aOA
 ieV3ClTt89wA==
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.76,352,1592895600"; d="scan'208";a="322695508"
Received: from aburakov-mobl.ger.corp.intel.com (HELO [10.213.236.34])
 ([10.213.236.34])
 by fmsmga004.fm.intel.com with ESMTP; 25 Aug 2020 02:28:55 -0700
To: Bruce Richardson <bruce.richardson@intel.com>
Cc: dev@dpdk.org, John McNamara <john.mcnamara@intel.com>,
 Marko Kovacevic <marko.kovacevic@intel.com>, ferruh.yigit@intel.com,
 padraig.j.connolly@intel.com, stable@dpdk.org
References: <aca9a5986871ecb3aba7f476fa906a34dabc9e7e.1598283570.git.anatoly.burakov@intel.com>
 <01efc239aaf6513b768829f7e44ad411cab881fe.1598283570.git.anatoly.burakov@intel.com>
 <20200824171307.GE547@bricha3-MOBL.ger.corp.intel.com>
From: "Burakov, Anatoly" <anatoly.burakov@intel.com>
Message-ID: <f9aa59e3-2f51-a5c3-e404-423d78dfb56e@intel.com>
Date: Tue, 25 Aug 2020 10:28:53 +0100
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101
 Thunderbird/68.11.0
MIME-Version: 1.0
In-Reply-To: <20200824171307.GE547@bricha3-MOBL.ger.corp.intel.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Subject: Re: [dpdk-stable] [PATCH 2/2] doc/linux_gsg: update information on
	using hugepages
X-BeenThere: stable@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: patches for DPDK stable branches <stable.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/stable>,
 <mailto:stable-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/stable/>
List-Post: <mailto:stable@dpdk.org>
List-Help: <mailto:stable-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/stable>,
 <mailto:stable-request@dpdk.org?subject=subscribe>
Errors-To: stable-bounces@dpdk.org
Sender: "stable" <stable-bounces@dpdk.org>

On 24-Aug-20 6:13 PM, Bruce Richardson wrote:
> On Mon, Aug 24, 2020 at 04:45:01PM +0100, Anatoly Burakov wrote:
>> Current information regarding hugepage usage is a little out of date.
>> Update it to include information on in-memory mode, as well as on
>> default mountpoints provided by systemd.
>>
>> Cc: stable@dpdk.org
>>
>> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
>> ---
>>   doc/guides/linux_gsg/sys_reqs.rst | 39 +++++++++++++++++++------------
>>   1 file changed, 24 insertions(+), 15 deletions(-)
>>
>> diff --git a/doc/guides/linux_gsg/sys_reqs.rst b/doc/guides/linux_gsg/sys_reqs.rst
>> index a124656bcb..2ddd7ed667 100644
>> --- a/doc/guides/linux_gsg/sys_reqs.rst
>> +++ b/doc/guides/linux_gsg/sys_reqs.rst
>> @@ -155,8 +155,12 @@ Without hugepages, high TLB miss rates would occur with the standard 4k page siz
>>   Reserving Hugepages for DPDK Use
>>   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>   
>> -The allocation of hugepages should be done at boot time or as soon as possible after system boot
>> -to prevent memory from being fragmented in physical memory.
>> +The allocation of hugepages can be performed either at run time or at boot time.
>> +In the general case, reserving hugepages at run time is perfectly fine, but in
>> +use cases where having lots of physically contiguous memory is required, it is
>> +preferable to reserve hugepages at boot time, as that will help in preventing
>> +physical memory from becoming heavily fragmented.
>> +
> 
> Although we are removing the note about 1G pages requiring to be reserved
> at boot time, I think we should still mention here that some older kernel
> versions do not allow 1G reservations post-boot.

Agreed, will fix in v2.

> 
>>   To reserve hugepages at boot time, a parameter is passed to the Linux kernel on the kernel command line.
>>   
>>   For 2 MB pages, just pass the hugepages option to the kernel. For example, to reserve 1024 pages of 2 MB, use::
>> @@ -187,9 +191,9 @@ See the Documentation/admin-guide/kernel-parameters.txt file in your Linux sourc
>>   
>>   **Alternative:**
>>   
>> -For 2 MB pages, there is also the option of allocating hugepages after the system has booted.
>> +There is also the option of allocating hugepages after the system has booted.
>>   This is done by echoing the number of hugepages required to a nr_hugepages file in the ``/sys/devices/`` directory.
>> -For a single-node system, the command to use is as follows (assuming that 1024 pages are required)::
>> +For a single-node system, the command to use is as follows (assuming that 1024 of 2MB pages are required)::
>>   
>>       echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
>>   
>> @@ -198,22 +202,27 @@ On a NUMA machine, pages should be allocated explicitly on separate nodes::
>>       echo 1024 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
>>       echo 1024 > /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages
>>   
>> -.. note::
>> -
>> -    For 1G pages, it is not possible to reserve the hugepage memory after the system has booted.
>> -
>>   Using Hugepages with the DPDK
>>   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>   
>> -Once the hugepage memory is reserved, to make the memory available for DPDK use, perform the following steps::
>> +If secondary process support is not required, DPDK is able to use hugepages
>> +without any configuration by using "in-memory" mode. Please see
>> +:ref:`linux_eal_parameters` for more details.
>> +
>> +If secondary process support is required, mount points for hugepages need to be
>> +created. On modern Linux distributions, a default mount point for hugepages is provided
>> +by the system and is located at ``/dev/hugepages``. This mount point will use the
>> +default hugepage size set by the kernel parameters as described above.
>> +
>> +However, in order to use multiple hugepage sizes, it is necessary to manually
> 
> Rather than multiple hugepage sizes, I'd suggest changing this to hugepage
> sizes other than the default.

OK, will fix.

> 
> Do we also want to add a line somewhere explaining that the default size
> can be set a boot using a kernel parameter?

It's already there, right above this :)

> 
>> +create mount points for hugepage sizes that are not provided by the system
>> +(e.g. 1GB pages).
>> +
>> +To make the hugepages of size 1GB available for DPDK use, perform the following steps::
>>   
>>       mkdir /mnt/huge
>> -    mount -t hugetlbfs nodev /mnt/huge
>> +    mount -t hugetlbfs pagesize=1GB /mnt/huge
>>   
>>   The mount point can be made permanent across reboots, by adding the following line to the ``/etc/fstab`` file::
>>   
>> -    nodev /mnt/huge hugetlbfs defaults 0 0
>> -
>> -For 1GB pages, the page size must be specified as a mount option::
>> -
>> -    nodev /mnt/huge_1GB hugetlbfs pagesize=1GB 0 0
>> +    nodev /mnt/huge hugetlbfs pagesize=1GB 0 0
>> -- 
>> 2.17.1


-- 
Thanks,
Anatoly