From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9F70FA04AC; Tue, 1 Sep 2020 18:56:55 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id D2BEE1C0B0; Tue, 1 Sep 2020 18:56:54 +0200 (CEST) Received: from mail-pj1-f68.google.com (mail-pj1-f68.google.com [209.85.216.68]) by dpdk.org (Postfix) with ESMTP id 2483B1C0AD for ; Tue, 1 Sep 2020 18:56:53 +0200 (CEST) Received: by mail-pj1-f68.google.com with SMTP id n3so882334pjq.1 for ; Tue, 01 Sep 2020 09:56:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=NfFGSb4KaPgrkOeGHqgadUeGmnE6LrZ+W1ZCEVONgi4=; b=LGAOx42nEhZjg3r33NkrmWCakXftc4q91GLjk8AOss8lfAzNTqWJjBaFNuNwQgfu9o TFyYPquv8QUrpRGdeA9YWNt0sOfehkWuc0lC7lxmk1+DBnlObvj/rNjRic1ds13bF1v8 sptygdbmENgDj4lrYWppMBw5w/mhzORWzMWU6QZk47noQim0wawodJ5SLXm+VgtVHbdw DTZWxAY1Tg8pFzeodmP7Wp4QhkMPoA/BBOlA2Zmv2QyUFo77rYhBZG6E1FEJxgFng1BI 58DgsdxP+R8E2xen2POn0RxUSBLLRem+GntTM1Sa4HOp8rDmlBneB93gzsabi/WmDjQ3 FCgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=NfFGSb4KaPgrkOeGHqgadUeGmnE6LrZ+W1ZCEVONgi4=; b=uJ2cxjR7/tTexsIILysKVoX+NWYTsZUQx3FsfdoJHPNMNX82o+swpKnsd5IUuNpiwR J8zXmHf8Z5GRJQKbyRV21PwDYegy0MbIpHIfGOvb5jlHN62y8jiCDbpFjgtmTQCKLUPO tfySvEqQLG4HMyA9KwJbrEpGsuqeJWe8WITmeSQf2aUfSYy0ENoLXZmMicIxEqb3FzKA 3MV+VW9PAay2kJ5JLBtcjzeUHWfKwhjzWKAKpQ3ztw3LJ4vdk8JouoG1ggG0886TWxQn UXA+9a3TXDn7b7fqkwHzp7E0j6l0JT9Iip2+oqWDDCL2n8PmMecamJQ9R2w+1BqNwXht U+Ng== X-Gm-Message-State: AOAM532MLrve0VfrsFv71VhrQ+bS4pWHw/YoUsCHdUVBPvajvd01Fnve PCJvm9dd+WfvJoBMhNESATpgUKM3DKBsOw== X-Google-Smtp-Source: ABdhPJwUp6x2vn1mEcdYFHKqidXqYpBs4NMLqRg/bALFDmr58QC9goV7Zf9Kvr4PijP+uajS/UokpA== X-Received: by 2002:a17:902:7c01:: with SMTP id x1mr2247255pll.34.1598979411280; Tue, 01 Sep 2020 09:56:51 -0700 (PDT) Received: from hermes.lan (204-195-22-127.wavecable.com. [204.195.22.127]) by smtp.gmail.com with ESMTPSA id b203sm2266499pfb.205.2020.09.01.09.56.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 01 Sep 2020 09:56:50 -0700 (PDT) From: Stephen Hemminger To: dev@dpdk.org Cc: Stephen Hemminger Date: Tue, 1 Sep 2020 09:56:43 -0700 Message-Id: <20200901165643.15668-1-stephen@networkplumber.org> X-Mailer: git-send-email 2.27.0 In-Reply-To: <2173565.OSGeAx7z5R@thomas> References: <2173565.OSGeAx7z5R@thomas> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [dpdk-dev] [PATCH] usertools: add huge page setup script X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This is an improved version of the setup of huge pages bases on earlier DPDK setup. Differences are: * it autodetects NUMA vs non NUMA * it allows setting different page sizes recent kernels support multiple sizes. * it accepts a parameter in bytes (not pages). Signed-off-by: Stephen Hemminger --- This is lightly tested, it still needs testing on multiple architectures etc. usertools/hugepage-setup.sh | 169 ++++++++++++++++++++++++++++++++++++ 1 file changed, 169 insertions(+) create mode 100755 usertools/hugepage-setup.sh diff --git a/usertools/hugepage-setup.sh b/usertools/hugepage-setup.sh new file mode 100755 index 000000000000..df132e2f8d64 --- /dev/null +++ b/usertools/hugepage-setup.sh @@ -0,0 +1,169 @@ +#! /bin/bash +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2010-2014 Intel Corporation +# + +usage() +{ + echo "Usage: $0 size [pagesize]" + echo " size is in bytes with optional M or G suffix" + echo " pagesize is the pagesize to use" + exit 1 +} + +get_pagesize() +{ + SIZE="$1" + + if [[ "$SIZE" =~ ^[0-9]+G$ ]]; then + echo $((${SIZE%%G} * 1024 * 1024)) + elif [[ "$SIZE" =~ ^[0-9]+M$ ]]; then + echo $((${SIZE%%M} * 1024)) + elif [[ "$SIZE" =~ ^[0-9]+K$ ]]; then + echo ${SIZE%%K} + elif [[ "$SIZE" =~ ^[0-9]+$ ]]; then + if [ $((SIZE % 1024)) -ne 0 ]; then + exit 1 + else + echo $((SIZE / 1024)) + fi + else + exit 1 + fi +} + +# +# Creates hugepage filesystem. +# +create_mnt_huge() +{ + echo "Creating /mnt/huge and mounting as hugetlbfs" + mkdir -p /mnt/huge + + grep -s '/mnt/huge' /proc/mounts > /dev/null + if [ $? -ne 0 ] ; then + mount -t hugetlbfs -o pagesize=${PAGESIZE} nodev /mnt/huge + fi +} + +# +# Removes hugepage filesystem. +# +remove_mnt_huge() +{ + echo "Unmounting /mnt/huge and removing directory" + grep -s '/mnt/huge' /proc/mounts > /dev/null + if [ $? -eq 0 ] ; then + umount /mnt/huge + fi + + if [ -d /mnt/huge ] ; then + rm -R /mnt/huge + fi +} +# +# Removes all reserved hugepages. +# +clear_huge_pages() +{ + echo > .echo_tmp + for d in /sys/devices/system/node/node? ; do + for sz in $d/hugepages/hugepages-* ; do + echo "echo 0 > ${sz}/nr_hugepages" >> .echo_tmp + done + done + echo "Removing currently reserved hugepages" + sh .echo_tmp + rm -f .echo_tmp + + remove_mnt_huge +} + +# +# Creates hugepages. +# +set_non_numa_pages() +{ + path=/sys/kernel/mm/hugepages/hugepages-${HUGEPGSZ}kB + if [ ! -d $path ]; then + >&2 echo "${HUGEPGSZ}K is not a valid huge page size" + exit 1 + fi + for sz in /sys/kernel/mm/hugepages/hugepages-* ; do + echo "echo 0 > ${sz}/nr_hugepages" >> .echo_tmp + done + + echo "Reserving $PAGES hugepages of size $HUGEPGSZ kB" + echo $PAGES > $path/nr_hugepages + + create_mnt_huge +} + +# +# Creates hugepages on specific NUMA nodes. +# +set_numa_pages() +{ + clear_huge_pages + + echo > .echo_tmp + for d in /sys/devices/system/node/node? ; do + node=$(basename $d) + path="$d/hugepages/hugepages-${HUGEPGSZ}kB" + if [ ! -d $path ]; then + >&2 echo "${HUGEPGSZ}K is not a valid huge page size" + exit 1 + fi + + echo "echo $Pages > $path" >> .echo_tmp + done + echo "Reserving $PAGES hugepages of size $HUGEPGSZ kB (numa)" + sh .echo_tmp + rm -f .echo_tmp + + create_mnt_huge +} + +# +# Need size argument +# +[ $# -ge 1 ] || usage + +# +# Convert from size to pages +# +KSIZE=$(get_pagesize $1) +if [ $? -ne 0 ]; then + >&2 echo "Invalid huge area size: $1" + exit 1 +fi + +# +# Optional second argument is pagesize +# +if [ $# -gt 1 ]; then + HUGEPGSZ=$(get_pagesize $2) + if [ $? -ne 0 ]; then + >&2 echo "Invalid huge page size: $2" + exit 1 + fi +else + HUGEPGSZ=$(awk '/^Hugepagesize/ { print $2 }' /proc/meminfo ) +fi + +if [ $((KSIZE % HUGEPGSZ)) -ne 0 ] ; then + echo "Invalid number of huge pages $KSIZE K, should be multiple of $HUGEPGSZ K" + exit 1 +fi + +PAGES=$((KSIZE / HUGEPGSZ)) +PAGESIZE=$((HUGEPGSZ * 1024)) + +# +# Do NUMA if necessary +# +if [ -e /sys/devices/numa/node ]; then + set_numa_pages +else + set_non_numa_pages +fi -- 2.27.0