DPDK patches and discussions
 help / color / mirror / Atom feed
From: Stephen Hemminger <stephen@networkplumber.org>
To: dev@dpdk.org
Cc: Stephen Hemminger <stephen@networkplumber.org>
Subject: [dpdk-dev] [PATCH] usertools: add huge page setup script
Date: Tue,  1 Sep 2020 09:56:43 -0700
Message-ID: <20200901165643.15668-1-stephen@networkplumber.org> (raw)
In-Reply-To: <2173565.OSGeAx7z5R@thomas>

This is an improved version of the setup of huge pages
bases on earlier DPDK setup. Differences are:
   * it autodetects NUMA vs non NUMA
   * it allows setting different page sizes
     recent kernels support multiple sizes.
   * it accepts a parameter in bytes (not pages).

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
This is lightly tested, it still needs testing on multiple architectures
etc.

 usertools/hugepage-setup.sh | 169 ++++++++++++++++++++++++++++++++++++
 1 file changed, 169 insertions(+)
 create mode 100755 usertools/hugepage-setup.sh

diff --git a/usertools/hugepage-setup.sh b/usertools/hugepage-setup.sh
new file mode 100755
index 000000000000..df132e2f8d64
--- /dev/null
+++ b/usertools/hugepage-setup.sh
@@ -0,0 +1,169 @@
+#! /bin/bash
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+#
+
+usage()
+{
+    echo "Usage: $0 size [pagesize]"
+    echo "  size is in bytes with optional M or G suffix"
+    echo "  pagesize is the pagesize to use"
+    exit 1
+}
+
+get_pagesize()
+{
+    SIZE="$1"
+
+    if [[ "$SIZE" =~ ^[0-9]+G$ ]]; then
+	echo $((${SIZE%%G} * 1024 * 1024))
+    elif [[ "$SIZE" =~ ^[0-9]+M$ ]]; then
+	echo $((${SIZE%%M} * 1024))
+    elif [[ "$SIZE" =~ ^[0-9]+K$ ]]; then
+	echo ${SIZE%%K}
+    elif [[ "$SIZE" =~ ^[0-9]+$ ]]; then
+	if [ $((SIZE % 1024)) -ne 0 ]; then
+	    exit 1
+	else
+	    echo $((SIZE / 1024))
+	fi
+    else
+	exit 1
+    fi
+}
+
+#
+# Creates hugepage filesystem.
+#
+create_mnt_huge()
+{
+    echo "Creating /mnt/huge and mounting as hugetlbfs"
+    mkdir -p /mnt/huge
+
+    grep -s '/mnt/huge' /proc/mounts > /dev/null
+    if [ $? -ne 0 ] ; then
+	mount -t hugetlbfs -o pagesize=${PAGESIZE} nodev /mnt/huge
+    fi
+}
+
+#
+# Removes hugepage filesystem.
+#
+remove_mnt_huge()
+{
+    echo "Unmounting /mnt/huge and removing directory"
+    grep -s '/mnt/huge' /proc/mounts > /dev/null
+    if [ $? -eq 0 ] ; then
+	umount /mnt/huge
+    fi
+
+    if [ -d /mnt/huge ] ; then
+	rm -R /mnt/huge
+    fi
+}
+#
+# Removes all reserved hugepages.
+#
+clear_huge_pages()
+{
+    echo > .echo_tmp
+    for d in /sys/devices/system/node/node? ; do
+	for sz in $d/hugepages/hugepages-* ; do
+	    echo "echo 0 > ${sz}/nr_hugepages" >> .echo_tmp
+	done
+    done
+    echo "Removing currently reserved hugepages"
+    sh .echo_tmp
+    rm -f .echo_tmp
+
+    remove_mnt_huge
+}
+
+#
+# Creates hugepages.
+#
+set_non_numa_pages()
+{
+    path=/sys/kernel/mm/hugepages/hugepages-${HUGEPGSZ}kB
+    if [ ! -d $path ]; then
+	>&2 echo "${HUGEPGSZ}K is not a valid huge page size"
+	exit 1
+    fi
+    for sz in /sys/kernel/mm/hugepages/hugepages-* ; do
+	echo "echo 0 > ${sz}/nr_hugepages" >> .echo_tmp
+    done
+
+    echo "Reserving $PAGES hugepages of size $HUGEPGSZ kB"
+    echo $PAGES > $path/nr_hugepages
+
+    create_mnt_huge
+}
+
+#
+# Creates hugepages on specific NUMA nodes.
+#
+set_numa_pages()
+{
+	clear_huge_pages
+
+	echo > .echo_tmp
+	for d in /sys/devices/system/node/node? ; do
+		node=$(basename $d)
+		path="$d/hugepages/hugepages-${HUGEPGSZ}kB"
+		if [ ! -d $path ]; then
+		    >&2 echo "${HUGEPGSZ}K is not a valid huge page size"
+		    exit 1
+		fi
+
+		echo "echo $Pages > $path" >> .echo_tmp
+	done
+	echo "Reserving $PAGES hugepages of size $HUGEPGSZ kB (numa)"
+	sh .echo_tmp
+	rm -f .echo_tmp
+
+	create_mnt_huge
+}
+
+#
+# Need size argument
+#
+[ $# -ge 1 ] || usage
+
+#
+# Convert from size to pages
+#
+KSIZE=$(get_pagesize $1)
+if [ $? -ne 0 ]; then
+    >&2 echo "Invalid huge area size: $1"
+    exit 1
+fi
+
+#
+# Optional second argument is pagesize
+#
+if [ $# -gt 1 ]; then
+    HUGEPGSZ=$(get_pagesize $2)
+    if [ $? -ne 0 ]; then
+	>&2 echo "Invalid huge page size: $2"
+	exit 1
+    fi
+else
+    HUGEPGSZ=$(awk '/^Hugepagesize/ { print $2 }' /proc/meminfo )
+fi
+
+if [ $((KSIZE % HUGEPGSZ)) -ne 0 ] ; then
+    echo "Invalid number of huge pages $KSIZE K, should be multiple of $HUGEPGSZ K"
+    exit 1
+fi
+
+PAGES=$((KSIZE / HUGEPGSZ))
+PAGESIZE=$((HUGEPGSZ * 1024))
+
+#
+# Do NUMA if necessary
+#
+if [ -e /sys/devices/numa/node ]; then
+    set_numa_pages
+else
+    set_non_numa_pages
+fi
-- 
2.27.0


  reply	other threads:[~2020-09-01 16:56 UTC|newest]

Thread overview: 37+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-08-18 12:39 [dpdk-dev] [RFC] usertools: Replace dpdk-setup with a python curses based script Sarosh Arif
2020-08-18 17:09 ` Stephen Hemminger
2020-09-01 13:30   ` Thomas Monjalon
2020-09-01 16:56     ` Stephen Hemminger [this message]
2020-09-02  9:47       ` [dpdk-dev] [PATCH] usertools: add huge page setup script Ferruh Yigit
2020-09-02  9:55       ` Bruce Richardson
2020-09-02 14:50         ` Stephen Hemminger
2020-09-03 22:48       ` [dpdk-dev] [PATCH v2] " Stephen Hemminger
2020-09-04  9:22         ` Bruce Richardson
2020-09-04 17:18           ` Stephen Hemminger
2020-09-04 14:58         ` Burakov, Anatoly
2020-09-04 15:10           ` Bruce Richardson
2020-09-04 18:35       ` [dpdk-dev] [PATCH] " Stephen Hemminger
2020-09-04 23:13         ` Ferruh Yigit
2020-09-04 23:30           ` Stephen Hemminger
2020-09-05  3:07       ` [dpdk-dev] [PATCH v4] " Stephen Hemminger
2020-09-06  3:42       ` [dpdk-dev] [PATCH v5] " Stephen Hemminger
2020-09-07  8:54         ` Ferruh Yigit
2020-09-07  8:58           ` Bruce Richardson
2020-09-07 17:20             ` Stephen Hemminger
2020-09-08  8:18               ` Bruce Richardson
2020-09-08 14:58                 ` Stephen Hemminger
2020-09-08 21:49             ` Thomas Monjalon
2020-09-08 15:17       ` [dpdk-dev] [PATCH v6] usertools: add a " Stephen Hemminger
2020-09-09 11:46         ` Ferruh Yigit
2020-09-09 19:26         ` Ajit Khaparde
2020-09-09 18:51       ` [dpdk-dev] [PATCH v7] " Stephen Hemminger
2020-09-14 15:31         ` Burakov, Anatoly
2020-10-20 18:01           ` Ferruh Yigit
2020-11-22 21:39             ` Thomas Monjalon
2020-09-24  4:31         ` Stephen Hemminger
2020-11-22 21:30           ` Thomas Monjalon
2020-11-23  0:12             ` Stephen Hemminger
2020-11-24 17:45             ` Stephen Hemminger
2020-11-24 21:37               ` Thomas Monjalon
2020-11-25  9:16                 ` Ferruh Yigit
2020-08-28 12:09 ` [dpdk-dev] [RFC] usertools: Replace dpdk-setup with a python curses based script Morten Brørup

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200901165643.15668-1-stephen@networkplumber.org \
    --to=stephen@networkplumber.org \
    --cc=dev@dpdk.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

DPDK patches and discussions

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://inbox.dpdk.org/dev/0 dev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dev dev/ https://inbox.dpdk.org/dev \
		dev@dpdk.org
	public-inbox-index dev

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.dev


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git