From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id B95C2A04B1; Mon, 24 Aug 2020 19:13:15 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id D197CF72; Mon, 24 Aug 2020 19:13:14 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id 55C0CDE3; Mon, 24 Aug 2020 19:13:13 +0200 (CEST) IronPort-SDR: 8Z2tc8wg969/VFm/PaMttF6cW1j9oHuJeytAlm03CsMVRKKz1LRJ2OWwWNvPMOLvt3PMjLJRzq X2DHHSR2JRJg== X-IronPort-AV: E=McAfee;i="6000,8403,9723"; a="153532070" X-IronPort-AV: E=Sophos;i="5.76,349,1592895600"; d="scan'208";a="153532070" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Aug 2020 10:13:12 -0700 IronPort-SDR: 7tZVWqt+45Vg/fPJz4O6ydgk6nPOXJAVlWngrwd9c08k4PMU3+bpcTUDTrVXEnwJFAtE11G3XO mUXwe2Azr/WA== X-IronPort-AV: E=Sophos;i="5.76,349,1592895600"; d="scan'208";a="443296112" Received: from bricha3-mobl.ger.corp.intel.com ([10.252.20.200]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-SHA; 24 Aug 2020 10:13:10 -0700 Date: Mon, 24 Aug 2020 18:13:07 +0100 From: Bruce Richardson To: Anatoly Burakov Cc: dev@dpdk.org, John McNamara , Marko Kovacevic , ferruh.yigit@intel.com, padraig.j.connolly@intel.com, stable@dpdk.org Message-ID: <20200824171307.GE547@bricha3-MOBL.ger.corp.intel.com> References: <01efc239aaf6513b768829f7e44ad411cab881fe.1598283570.git.anatoly.burakov@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <01efc239aaf6513b768829f7e44ad411cab881fe.1598283570.git.anatoly.burakov@intel.com> Subject: Re: [dpdk-dev] [PATCH 2/2] doc/linux_gsg: update information on using hugepages X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Mon, Aug 24, 2020 at 04:45:01PM +0100, Anatoly Burakov wrote: > Current information regarding hugepage usage is a little out of date. > Update it to include information on in-memory mode, as well as on > default mountpoints provided by systemd. > > Cc: stable@dpdk.org > > Signed-off-by: Anatoly Burakov > --- > doc/guides/linux_gsg/sys_reqs.rst | 39 +++++++++++++++++++------------ > 1 file changed, 24 insertions(+), 15 deletions(-) > > diff --git a/doc/guides/linux_gsg/sys_reqs.rst b/doc/guides/linux_gsg/sys_reqs.rst > index a124656bcb..2ddd7ed667 100644 > --- a/doc/guides/linux_gsg/sys_reqs.rst > +++ b/doc/guides/linux_gsg/sys_reqs.rst > @@ -155,8 +155,12 @@ Without hugepages, high TLB miss rates would occur with the standard 4k page siz > Reserving Hugepages for DPDK Use > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > -The allocation of hugepages should be done at boot time or as soon as possible after system boot > -to prevent memory from being fragmented in physical memory. > +The allocation of hugepages can be performed either at run time or at boot time. > +In the general case, reserving hugepages at run time is perfectly fine, but in > +use cases where having lots of physically contiguous memory is required, it is > +preferable to reserve hugepages at boot time, as that will help in preventing > +physical memory from becoming heavily fragmented. > + Although we are removing the note about 1G pages requiring to be reserved at boot time, I think we should still mention here that some older kernel versions do not allow 1G reservations post-boot. > To reserve hugepages at boot time, a parameter is passed to the Linux kernel on the kernel command line. > > For 2 MB pages, just pass the hugepages option to the kernel. For example, to reserve 1024 pages of 2 MB, use:: > @@ -187,9 +191,9 @@ See the Documentation/admin-guide/kernel-parameters.txt file in your Linux sourc > > **Alternative:** > > -For 2 MB pages, there is also the option of allocating hugepages after the system has booted. > +There is also the option of allocating hugepages after the system has booted. > This is done by echoing the number of hugepages required to a nr_hugepages file in the ``/sys/devices/`` directory. > -For a single-node system, the command to use is as follows (assuming that 1024 pages are required):: > +For a single-node system, the command to use is as follows (assuming that 1024 of 2MB pages are required):: > > echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages > > @@ -198,22 +202,27 @@ On a NUMA machine, pages should be allocated explicitly on separate nodes:: > echo 1024 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages > echo 1024 > /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages > > -.. note:: > - > - For 1G pages, it is not possible to reserve the hugepage memory after the system has booted. > - > Using Hugepages with the DPDK > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > -Once the hugepage memory is reserved, to make the memory available for DPDK use, perform the following steps:: > +If secondary process support is not required, DPDK is able to use hugepages > +without any configuration by using "in-memory" mode. Please see > +:ref:`linux_eal_parameters` for more details. > + > +If secondary process support is required, mount points for hugepages need to be > +created. On modern Linux distributions, a default mount point for hugepages is provided > +by the system and is located at ``/dev/hugepages``. This mount point will use the > +default hugepage size set by the kernel parameters as described above. > + > +However, in order to use multiple hugepage sizes, it is necessary to manually Rather than multiple hugepage sizes, I'd suggest changing this to hugepage sizes other than the default. Do we also want to add a line somewhere explaining that the default size can be set a boot using a kernel parameter? > +create mount points for hugepage sizes that are not provided by the system > +(e.g. 1GB pages). > + > +To make the hugepages of size 1GB available for DPDK use, perform the following steps:: > > mkdir /mnt/huge > - mount -t hugetlbfs nodev /mnt/huge > + mount -t hugetlbfs pagesize=1GB /mnt/huge > > The mount point can be made permanent across reboots, by adding the following line to the ``/etc/fstab`` file:: > > - nodev /mnt/huge hugetlbfs defaults 0 0 > - > -For 1GB pages, the page size must be specified as a mount option:: > - > - nodev /mnt/huge_1GB hugetlbfs pagesize=1GB 0 0 > + nodev /mnt/huge hugetlbfs pagesize=1GB 0 0 > -- > 2.17.1