From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 975E6A04C5; Fri, 4 Sep 2020 17:10:22 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 1478B1C0C6; Fri, 4 Sep 2020 17:10:22 +0200 (CEST) Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by dpdk.org (Postfix) with ESMTP id E084F1C0C2 for ; Fri, 4 Sep 2020 17:10:19 +0200 (CEST) IronPort-SDR: 6znmzb0Px2lZYGmvRgVu+G+rOjJGZBCh+qcEFng6KTI6GNjMZU44spD0P/4cSN3rJy/siPvV9O DIHbiEH/0v9w== X-IronPort-AV: E=McAfee;i="6000,8403,9734"; a="157763890" X-IronPort-AV: E=Sophos;i="5.76,389,1592895600"; d="scan'208";a="157763890" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Sep 2020 08:10:18 -0700 IronPort-SDR: VVyN4ATQQefekUtGNb14Mx4USgrJYLU7cWnCtC1XSTJTuicVi7rAmh2fWkA3gXz+WJRCRwkCkO paYE63Dbix5g== X-IronPort-AV: E=Sophos;i="5.76,389,1592895600"; d="scan'208";a="503538971" Received: from bricha3-mobl.ger.corp.intel.com ([10.252.4.76]) by fmsmga005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-SHA; 04 Sep 2020 08:10:17 -0700 Date: Fri, 4 Sep 2020 16:10:14 +0100 From: Bruce Richardson To: "Burakov, Anatoly" Cc: Stephen Hemminger , dev@dpdk.org Message-ID: <20200904151014.GB305@bricha3-MOBL.ger.corp.intel.com> References: <20200901165643.15668-1-stephen@networkplumber.org> <20200903224831.5932-1-stephen@networkplumber.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Subject: Re: [dpdk-dev] [PATCH v2] usertools: add huge page setup script X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Fri, Sep 04, 2020 at 03:58:03PM +0100, Burakov, Anatoly wrote: > On 03-Sep-20 11:48 PM, Stephen Hemminger wrote: > > This is an improved version of the setup of huge pages > > bases on earlier DPDK setup. Differences are: > > * it autodetects NUMA vs non NUMA > > * it allows setting different page sizes > > recent kernels support multiple sizes. > > * it accepts a parameter in bytes (not pages). > > > > If necessary the steps of clearing old settings and mounting/umounting > > can be done individually. > > > > > > Signed-off-by: Stephen Hemminger > > --- > > v2 -- rewrite in python > > The script is python3 only because supporting older versions > > no longer makes any sense. > > > > usertools/hugepage-setup.py | 317 ++++++++++++++++++++++++++++++++++++ > > 1 file changed, 317 insertions(+) > > create mode 100644 usertools/hugepage-setup.py > > > > diff --git a/usertools/hugepage-setup.py b/usertools/hugepage-setup.py > > new file mode 100644 > > index 000000000000..8e7642428d9e > > --- /dev/null > > +++ b/usertools/hugepage-setup.py > > @@ -0,0 +1,317 @@ > > +# Copyright (c) 2020 Microsoft Corporation > > +# > > +# Script to query and setup huge pages for DPDK applications. > > + > > +import sys > > +import os > > +import re > > +import getopt > > +import glob > > +from os.path import exists, basename > > + > > +# convention for where to mount huge pages > > +hugedir = '/dev/hugepages' > > This isn't a "convention", this is a default systemd mountpoint. > > > + > > +# command-line flags > > +show_flag = None > > +reserve_kb = None > > +clear_flag = None > > +hugepagesize_kb = None > > +mount_flag = None > > +unmount_flag = None > > + > > + > > +def usage(): > > + '''Print usage information for the program''' > > + global hugedir > > + mnt = hugedir > > + argv0 = basename(sys.argv[0]) > > + print(""" > > +Usage: > > +------ > > + %(argv0)s [options] > > + > > +Options: > > + --help, --usage: > > + Display usage information and quit > > + > > + -s, --show: > > + Print the current huge page configuration. > > + > > + --setup: > > + Simplified version of clear, umount, reserve, mount operations > > + > > + -c, --clear: > > + Remove all huge pages > > + > > + -r, --reserve: > > + Reserve huge pages. The size specified is in bytes, with > > + optional K, M or G suffix. The size must be a multiple > > + of the page size. > > + > > + -p, --pagesize > > + Choose page size to use. If not specified, the default > > + system page size will be used. > > + > > + -m, --mount > > + Mount the system huge page directory %(mnt)s > > + > > + -u, --umount > > + Unmount the system huge page directory %(mnt)s > > + > > + > > +Examples: > > +--------- > > + > > +To display current huge page settings: > > + %(argv0)s -s > > + > > +To a complete setup of with 2 Gigabyte of 1G huge pages: > > + %(argv0)s -p 1G --setup 2G > > + > > +Equivalent to: > > + %(argv0)s -p 1G -c -u -r 2G -m > > + > > +To clear existing huge page settings and umount %(mnt)s > > + %(argv0)s -c -u > > + > > + """ % locals()) > > + > > + > > +def fmt_memsize(sz): > > + '''Format memory size in conventional format''' > > + sz_kb = int(sz) > > + if sz_kb >= 1024 * 1024: > > + return '{}Gb'.format(sz_kb / (1024 * 1024)) > > + elif sz_kb >= 1024: > > + return '{}Mb'.format(sz_kb / 1024) > > + else: > > + return '{}Kb'.format(sz_kb) > > I've lost count how many times i've had to reimplement this code, but there > is an easier way :) Off the top of my head, > > idx = log2(sz) > # every 10th power of 2 > return '{}{}b'.format(sz, ' kMG'[int(idx) / 10]) > > or something close to that. > Another minor nit, since these are memory sizes, not bandwidth rates, it's bytes not bits, so the "b" should be "B" in all the prints, whatever way it's calculated.