From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 03377A0C57; Sun, 29 Aug 2021 00:17:41 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 81DBB40142; Sun, 29 Aug 2021 00:17:41 +0200 (CEST) Received: from mail-lj1-f179.google.com (mail-lj1-f179.google.com [209.85.208.179]) by mails.dpdk.org (Postfix) with ESMTP id AFEFB40042 for ; Sun, 29 Aug 2021 00:17:40 +0200 (CEST) Received: by mail-lj1-f179.google.com with SMTP id c12so18403428ljr.5 for ; Sat, 28 Aug 2021 15:17:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Mlh9nNsL3+wvwoEpCgZEXjc1Dpyf3dFXDWyIdkgvygg=; b=OkoFLKP9UVqRPApB/nc6Y47vE3aYgl1odGa2X5xIGzECYtajZHbZ68RxAKhuh2gC8u PJn3+RGw7IV79j05kPuHEzQkqzsG9fUtwCQUZE/LeU1V5eLRB4p+TULkFFwrkWgp8KkO /vurlOvCNio7Gmvo+226P3p8JAU+hbtJ4nUISCSFryrocNE1/RARqPPQ496AqODqR9Qy t0JmLU+1XZKKv7ncpS31eDM8c0benfvk58s08W+vPW8qgNDFhT8mEU9EaJIZ6VSw60Gi vfwq4f4vnC2ZJAHOhj2uU9g26LYfW3GjbVPTNM1aMGDYmfLk1Ds4YqdiVEdFAcGTaEDE z/Pw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Mlh9nNsL3+wvwoEpCgZEXjc1Dpyf3dFXDWyIdkgvygg=; b=sUSkD5y4RRgr344KVPkcfRze/90YCeG9H6uCS5lkKCH6fPbnofKnPdDqRSBWc83YTY YdaOaoE4YX14HUJG9W8vaep6jhtGkM6NvKE+jhzfSr1yHvTB7aoHLk3IF2wq5ZRhycgH PqT8ke1eLM9j4Zn08HhMPKGWa8ZkngG5QSCK/YNDBAjBSG3ZdtoCbOSP9bUYVzNCvMbd W4pzdQlSilmeGDrvSjNpTBXU+E61ksWTFX37yi8e8H9MhmIFa1t1msZzOfcrm/QSyz8T To8J3j7DyHXpixA04pWjcvFXhrUtaxlUEFm958IX065IcUdIQQREu74G25chRxteFDYG nPHA== X-Gm-Message-State: AOAM532QAUBh9T4f7gwKhNa55lv5IkDX8LKsItGuGDPOBhrFlBw8TwIE +QbgNewnRmkEo2NuXUaVCjk= X-Google-Smtp-Source: ABdhPJyGpCjjddtQ0y0UpVilWxzxDTMKjCPglVMYHn445KgjzP7rXg522NvxZ7PnudFabrhCiLyTBA== X-Received: by 2002:a2e:a726:: with SMTP id s38mr13389866lje.386.1630189060177; Sat, 28 Aug 2021 15:17:40 -0700 (PDT) Received: from sovereign (broadband-37-110-65-23.ip.moscow.rt.ru. [37.110.65.23]) by smtp.gmail.com with ESMTPSA id h15sm1211176ljc.96.2021.08.28.15.17.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 28 Aug 2021 15:17:39 -0700 (PDT) Date: Sun, 29 Aug 2021 01:17:38 +0300 From: Dmitry Kozlyuk To: Jie Zhou Cc: dev@dpdk.org, roretzla@microsoft.com, navasile@linux.microsoft.com, dmitrym@microsoft.com, pallavi.kadam@intel.com, talshn@nvidia.com, thomas@monjalon.net, aconole@redhat.com Message-ID: <20210829011738.4f78f2cd@sovereign> In-Reply-To: <1629306834-6277-13-git-send-email-jizh@linux.microsoft.com> References: <1629267476-901-1-git-send-email-jizh@linux.microsoft.com> <1629306834-6277-1-git-send-email-jizh@linux.microsoft.com> <1629306834-6277-13-git-send-email-jizh@linux.microsoft.com> X-Mailer: Claws Mail 3.17.8 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [PATCH v2 12/13] app/test: replace .sh scripts with .py scripts X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" 2021-08-18 10:13 (UTC-0700), Jie Zhou: > - Add python scripts to get coremask and check hugepage config > - Remove two corresponding .sh scripts > - Replace calling of .sh with corresponding .py in meson.build There are multiple violations of PEP8 in new files, please run a formatter on them, e.g. "black" or "yapf". I also suggest running "pylint" on those files. > Signed-off-by: Jie Zhou > --- > app/test/get-coremask.sh | 13 ------------- > app/test/get_coremask.py | 8 ++++++++ > app/test/has-hugepage.sh | 11 ----------- > app/test/has_hugepage.py | 41 ++++++++++++++++++++++++++++++++++++++++ > app/test/meson.build | 7 ++++--- > 5 files changed, 53 insertions(+), 27 deletions(-) > delete mode 100755 app/test/get-coremask.sh > create mode 100644 app/test/get_coremask.py > delete mode 100755 app/test/has-hugepage.sh > create mode 100644 app/test/has_hugepage.py > > diff --git a/app/test/get-coremask.sh b/app/test/get-coremask.sh > deleted file mode 100755 > index bb8cf404d2..0000000000 > --- a/app/test/get-coremask.sh > +++ /dev/null > @@ -1,13 +0,0 @@ > -#! /bin/sh -e > -# SPDX-License-Identifier: BSD-3-Clause > -# Copyright(c) 2019 Intel Corporation > - > -if [ "$(uname)" = "Linux" ] ; then > - cat /sys/devices/system/cpu/present > -elif [ "$(uname)" = "FreeBSD" ] ; then > - ncpus=$(/sbin/sysctl -n hw.ncpu) > - echo 0-$(expr $ncpus - 1) > -else > -# fallback > - echo 0-3 > -fi > diff --git a/app/test/get_coremask.py b/app/test/get_coremask.py > new file mode 100644 > index 0000000000..3d0947e8ff > --- /dev/null > +++ b/app/test/get_coremask.py > @@ -0,0 +1,8 @@ > +#! /usr/bin/env python3 > +# SPDX-License-Identifier: BSD-3-Clause > +# Copyright (c) 2021 Microsoft Corporation > + > +import multiprocessing > + > +c = multiprocessing.cpu_count() > +print("0-"+str(c-1)) Fallback case is lost, error handling missing. Or is it intended for this script to fail fast? > diff --git a/app/test/has-hugepage.sh b/app/test/has-hugepage.sh > deleted file mode 100755 > index d600fad319..0000000000 > --- a/app/test/has-hugepage.sh > +++ /dev/null > @@ -1,11 +0,0 @@ > -#! /bin/sh > -# SPDX-License-Identifier: BSD-3-Clause > -# Copyright 2020 Mellanox Technologies, Ltd > - > -if [ "$(uname)" = "Linux" ] ; then > - cat /proc/sys/vm/nr_hugepages || echo 0 > -elif [ "$(uname)" = "FreeBSD" ] ; then > - echo 1 # assume FreeBSD always has hugepages > -else > - echo 0 > -fi > diff --git a/app/test/has_hugepage.py b/app/test/has_hugepage.py > new file mode 100644 > index 0000000000..decf47df45 > --- /dev/null > +++ b/app/test/has_hugepage.py > @@ -0,0 +1,41 @@ > +#! /usr/bin/env python3 > +# SPDX-License-Identifier: BSD-3-Clause > +# Copyright (c) 2021 Microsoft Corporation > + > +import os > +import platform > + > +osName = platform.system() > +if osName == "Linux": > + file_o=open("/proc/sys/vm/nr_hugepages") > + content=file_o.read() > + print(content) > + file_o.close() > +elif osName == "FreeBSD": > + # Assume FreeBSD always has hugepages enabled > + print("1") > +elif osName == "Windows": > + # On Windows, check if the Administrator has "Lock pages in memory" security setting > + # to determine if large page is enabled or not This script is intended to check not whether there are sufficient privileges but whether the system supports hugepages at all. I suggest checking ctypes.windll.kernel32.GetLargePageMinimum() > 0. > + > + # Export the USER_RIGHTS security settings > + # Use os.popen instead of os.system to suppress the output of secedit to stdout > + userRightsfile = "userrights.inf" > + os.popen('secedit /export /areas USER_RIGHTS /cfg "' + userRightsfile + '"') > + > + # Parse the exported user rights setting to determine if Administrator > + # SeLockMemoryPrivilege being set or not > + largepageEnabledStr = 'SeLockMemoryPrivilege = Administrator' > + enabled = 0 > + # On different OS versions tested, the exported inf file has utf-16 encoding > + with open(userRightsfile, encoding = 'utf-16') as f: > + urcontent = f.readlines() > + for line in urcontent: > + if largepageEnabledStr in line: > + enabled = 1 > + break > + > + f.close() > + print(enabled) > +else: > + print("0") > diff --git a/app/test/meson.build b/app/test/meson.build > index a7611686ad..7240304916 100644 > --- a/app/test/meson.build > +++ b/app/test/meson.build > @@ -450,7 +450,7 @@ dpdk_test = executable('dpdk-test', > driver_install_path), > install: true) > > -has_hugepage = run_command('has-hugepage.sh').stdout().strip() != '0' > +has_hugepage = run_command('has_hugepage.py').stdout().strip() != '0' Python scripts should be called using `py3` variable, see buildtools/ for examples. > message('hugepage availability: @0@'.format(has_hugepage)) > > # some perf tests (eg: memcpy perf autotest)take very long > @@ -458,8 +458,9 @@ message('hugepage availability: @0@'.format(has_hugepage)) > timeout_seconds = 600 > timeout_seconds_fast = 10 > > -get_coremask = find_program('get-coremask.sh') > -num_cores_arg = '-l ' + run_command(get_coremask).stdout().strip() > +list_of_cores = run_command('get_coremask.py').stdout().strip() > +message('list of cores: @0@'.format(list_of_cores)) > +num_cores_arg = '-l ' + list_of_cores > > default_test_args = [num_cores_arg] >