From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 45CC0A04DB for ; Mon, 30 Nov 2020 12:50:55 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 3E3ECC936; Mon, 30 Nov 2020 12:50:54 +0100 (CET) Received: from mail-wm1-f48.google.com (mail-wm1-f48.google.com [209.85.128.48]) by dpdk.org (Postfix) with ESMTP id 8C8D2C936 for ; Mon, 30 Nov 2020 12:50:52 +0100 (CET) Received: by mail-wm1-f48.google.com with SMTP id f190so21682851wme.1 for ; Mon, 30 Nov 2020 03:50:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=G9u6d64A2ptLCNxEohSt/Dt5xfm0L69clkFWb8mI/6s=; b=NpgUoFdMVlwkk7tgDRoiPBtHxOyEn5FqjZ2HPmwL9nH85EWHCZpn0hZe1bGTPiPW6E /EGVV0ljlKQxoReR4h5Bqa8CwArjXYtM6fTFWRE1gb5bKCEvd71dgYMRJrpsEM8RfPNp E1ejVtzBxvrMNfMOJ1ADr/bWsOZe18HlGLGIZ4zQZDm405OLTX8uiZqZwIfylWxiyOYx /L1gVfR1dWiEE7FGbBjp/PdNmRsWQNMTmOWj+qKlkxmk69LagG9ux9rmvZ7SFryBgcow oJ0+fgE1aGAfBGvq4BD7naJhS+Ml7uRHu8jhS0S/ouNGETsJ4MbCHPuawc2t2FjCmsqC 5H0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=G9u6d64A2ptLCNxEohSt/Dt5xfm0L69clkFWb8mI/6s=; b=DHPx85F2Rw8UNmQlBWf87y5FNphdpQnjU2wwVVmM5/55ntKrykRkY+JYH500AI3KyZ zSHOpzTacTjTqC0I8PIpfuVdNbr2iIHnyxpVPX5Tt6AZvAx9ITvUolUA20ta24kIF7hG YB2MnpX2HnH/O1C7yUUzh+mwbi7wcRjHRfZWFudPbq8hUbtsy1G/5vRw3WVXZS75S5YR u+4Qwud2pR3s7gRsIiXmvYYpCRwzz2Rql0kTSvLOccR44nqDorO2SxlRp2U7Uwg4Pw42 e80sPf9Kn9ETc89OegJJEDNlW8uU+3Y33y970OxFYARDSuyWP+z6WwV9CxR765wFE9N0 fbaw== X-Gm-Message-State: AOAM530Zfx57vgPvK36LZtENQiSzDsUwM0j0+IKNSnIvZPqROBhOyByi ZAZFCUSdtC7QvJBXbK25Tq0= X-Google-Smtp-Source: ABdhPJwHCZ9h5iMsCWrJqFbbGg5c2MaoNtx9NbyXK4TFpEg/Hro7wFDI5qNqIVj/SRCEvYsNh+ezeg== X-Received: by 2002:a7b:c154:: with SMTP id z20mr22594399wmi.160.1606737051361; Mon, 30 Nov 2020 03:50:51 -0800 (PST) Received: from localhost ([88.98.246.218]) by smtp.gmail.com with ESMTPSA id z11sm27013869wmc.39.2020.11.30.03.50.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Nov 2020 03:50:50 -0800 (PST) From: luca.boccassi@gmail.com To: Anatoly Burakov Cc: Bruce Richardson , dpdk stable Date: Mon, 30 Nov 2020 11:50:26 +0000 Message-Id: <20201130115027.64046-8-luca.boccassi@gmail.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201130115027.64046-1-luca.boccassi@gmail.com> References: <20201028104606.3504127-1-luca.boccassi@gmail.com> <20201130115027.64046-1-luca.boccassi@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [dpdk-stable] patch 'doc: update information on using hugepages' has been queued to stable release 19.11.6 X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Sender: "stable" Hi, FYI, your patch has been queued to stable release 19.11.6 Note it hasn't been pushed to http://dpdk.org/browse/dpdk-stable yet. It will be pushed if I get no objections before 12/02/20. So please shout if anyone has objections. Also note that after the patch there's a diff of the upstream commit vs the patch applied to the branch. This will indicate if there was any rebasing needed to apply to the stable branch. If there were code changes for rebasing (ie: not only metadata diffs), please double check that the rebase was correctly done. Queued patches are on a temporary branch at: https://github.com/bluca/dpdk-stable This queued commit can be viewed at: https://github.com/bluca/dpdk-stable/commit/cb3c1c1f2f969661e27536c607cb090bb126ca55 Thanks. Luca Boccassi --- >From cb3c1c1f2f969661e27536c607cb090bb126ca55 Mon Sep 17 00:00:00 2001 From: Anatoly Burakov Date: Thu, 19 Nov 2020 10:52:45 +0000 Subject: [PATCH] doc: update information on using hugepages [ upstream commit 8397cac725e43562df3ce7d230aa3b4390b64b10 ] Current information regarding hugepage usage is a little out of date. Update it to include information on in-memory mode, as well as on default mountpoints provided by systemd. Signed-off-by: Anatoly Burakov Acked-by: Bruce Richardson --- doc/guides/linux_gsg/sys_reqs.rst | 74 ++++++++++++++++++++----------- 1 file changed, 48 insertions(+), 26 deletions(-) diff --git a/doc/guides/linux_gsg/sys_reqs.rst b/doc/guides/linux_gsg/sys_reqs.rst index cf760a6c67..0af0b22813 100644 --- a/doc/guides/linux_gsg/sys_reqs.rst +++ b/doc/guides/linux_gsg/sys_reqs.rst @@ -157,8 +157,36 @@ Without hugepages, high TLB miss rates would occur with the standard 4k page siz Reserving Hugepages for DPDK Use ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -The allocation of hugepages should be done at boot time or as soon as possible after system boot -to prevent memory from being fragmented in physical memory. +The reservation of hugepages can be performed at run time. +This is done by echoing the number of hugepages required +to a ``nr_hugepages`` file in the ``/sys/kernel/`` directory +corresponding to a specific page size (in Kilobytes). +For a single-node system, the command to use is as follows +(assuming that 1024 of 2MB pages are required):: + + echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages + +On a NUMA machine, the above command will usually divide the number of hugepages +equally across all NUMA nodes (assuming there is enough memory on all NUMA nodes). +However, pages can also be reserved explicitly on individual NUMA nodes +using a ``nr_hugepages`` file in the ``/sys/devices/`` directory:: + + echo 1024 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages + echo 1024 > /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages + +.. note:: + + Some kernel versions may not allow reserving 1 GB hugepages at run time, + so reserving them at boot time may be the only option. + Please see below for instructions. + +**Alternative:** + +In the general case, reserving hugepages at run time is perfectly fine, +but in use cases where having lots of physically contiguous memory is required, +it is preferable to reserve hugepages at boot time, +as that will help in preventing physical memory from becoming heavily fragmented. + To reserve hugepages at boot time, a parameter is passed to the Linux kernel on the kernel command line. For 2 MB pages, just pass the hugepages option to the kernel. For example, to reserve 1024 pages of 2 MB, use:: @@ -187,35 +215,29 @@ the number of hugepages reserved at boot time is generally divided equally betwe See the Documentation/admin-guide/kernel-parameters.txt file in your Linux source tree for further details of these and other kernel options. -**Alternative:** - -For 2 MB pages, there is also the option of allocating hugepages after the system has booted. -This is done by echoing the number of hugepages required to a nr_hugepages file in the ``/sys/devices/`` directory. -For a single-node system, the command to use is as follows (assuming that 1024 pages are required):: - - echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages - -On a NUMA machine, pages should be allocated explicitly on separate nodes:: - - echo 1024 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages - echo 1024 > /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages - -.. note:: - - For 1G pages, it is not possible to reserve the hugepage memory after the system has booted. - Using Hugepages with the DPDK ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Once the hugepage memory is reserved, to make the memory available for DPDK use, perform the following steps:: +If secondary process support is not required, DPDK is able to use hugepages +without any configuration by using "in-memory" mode. +Please see :doc:`linux_eal_parameters` for more details. + +If secondary process support is required, +mount points for hugepages need to be created. +On modern Linux distributions, a default mount point for hugepages +is provided by the system and is located at ``/dev/hugepages``. +This mount point will use the default hugepage size +set by the kernel parameters as described above. + +However, in order to use hugepage sizes other than the default, it is necessary +to manually create mount points for those hugepage sizes (e.g. 1GB pages). + +To make the hugepages of size 1GB available for DPDK use, +following steps must be performed:: mkdir /mnt/huge - mount -t hugetlbfs nodev /mnt/huge + mount -t hugetlbfs pagesize=1GB /mnt/huge The mount point can be made permanent across reboots, by adding the following line to the ``/etc/fstab`` file:: - nodev /mnt/huge hugetlbfs defaults 0 0 - -For 1GB pages, the page size must be specified as a mount option:: - - nodev /mnt/huge_1GB hugetlbfs pagesize=1GB 0 0 + nodev /mnt/huge hugetlbfs pagesize=1GB 0 0 -- 2.27.0 --- Diff of the applied patch vs upstream commit (please double-check if non-empty: --- --- - 2020-11-30 11:49:30.640823023 +0000 +++ 0008-doc-update-information-on-using-hugepages.patch 2020-11-30 11:49:30.402271747 +0000 @@ -1 +1 @@ -From 8397cac725e43562df3ce7d230aa3b4390b64b10 Mon Sep 17 00:00:00 2001 +From cb3c1c1f2f969661e27536c607cb090bb126ca55 Mon Sep 17 00:00:00 2001 @@ -5,0 +6,2 @@ +[ upstream commit 8397cac725e43562df3ce7d230aa3b4390b64b10 ] + @@ -10,2 +11,0 @@ -Cc: stable@dpdk.org - @@ -19 +19 @@ -index e074faf514..be714adf22 100644 +index cf760a6c67..0af0b22813 100644 @@ -22 +22 @@ -@@ -158,8 +158,36 @@ Without hugepages, high TLB miss rates would occur with the standard 4k page siz +@@ -157,8 +157,36 @@ Without hugepages, high TLB miss rates would occur with the standard 4k page siz @@ -61 +61 @@ -@@ -188,35 +216,29 @@ the number of hugepages reserved at boot time is generally divided equally betwe +@@ -187,35 +215,29 @@ the number of hugepages reserved at boot time is generally divided equally betwe