From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by dpdk.org (Postfix) with ESMTP id B0D481D8A; Tue, 28 Aug 2018 04:57:59 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 27 Aug 2018 19:57:58 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,297,1531810800"; d="scan'208";a="257759822" Received: from fmsmsx105.amr.corp.intel.com ([10.18.124.203]) by fmsmga005.fm.intel.com with ESMTP; 27 Aug 2018 19:57:58 -0700 Received: from fmsmsx113.amr.corp.intel.com (10.18.116.7) by FMSMSX105.amr.corp.intel.com (10.18.124.203) with Microsoft SMTP Server (TLS) id 14.3.319.2; Mon, 27 Aug 2018 19:57:57 -0700 Received: from shsmsx152.ccr.corp.intel.com (10.239.6.52) by FMSMSX113.amr.corp.intel.com (10.18.116.7) with Microsoft SMTP Server (TLS) id 14.3.319.2; Mon, 27 Aug 2018 19:57:56 -0700 Received: from shsmsx103.ccr.corp.intel.com ([169.254.4.240]) by SHSMSX152.ccr.corp.intel.com ([169.254.6.150]) with mapi id 14.03.0319.002; Tue, 28 Aug 2018 10:57:39 +0800 From: "Liu, Yong" To: "Phil.Yang@arm.com" , dts , "dts@dpdk.org" CC: nd Thread-Topic: [dts] [PATCH] framework/dut: setup hugepage for all available NUMA nodes Thread-Index: AQHUPbPZN1lJ15W9BkW2yYEL65Uof6TTEN/QgADTfoCAAJQg4A== Date: Tue, 28 Aug 2018 02:57:39 +0000 Message-ID: <86228AFD5BCD8E4EBFD2B90117B5E81E6312DAB6@SHSMSX103.ccr.corp.intel.com> References: <1535332589-8090-1-git-send-email-phil.yang@arm.com> <86228AFD5BCD8E4EBFD2B90117B5E81E6312D154@SHSMSX103.ccr.corp.intel.com> In-Reply-To: Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiZDk0YWQwNDctODBkZS00OWU1LWI0ZmQtYWQ4ZDhlYzc5ZGUzIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoiN0pBSEw3RDRjOFdQWm04d215bGZwTWMzcFhJSFZEczVmd0krd2c1SE5kbm01bHBBUUZNYVJsc3NQeGM1U3F2QSJ9 x-ctpclassification: CTP_NT dlp-product: dlpe-windows dlp-version: 11.0.400.15 dlp-reaction: no-action x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dts] [PATCH] framework/dut: setup hugepage for all available NUMA nodes X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Aug 2018 02:58:00 -0000 You're welcome, Phil.=20 One more comment, It's better to make total_numa_nodes and numa_service_num= variables as integer before comparing them. Function "min" may not return the expect result when numa number over ten. Thanks, Marvin > -----Original Message----- > From: Phil.Yang@arm.com [mailto:Phil.Yang@arm.com] > Sent: Tuesday, August 28, 2018 10:01 AM > To: Liu, Yong ; dts ; dts@dpdk.= org > Cc: nd > Subject: RE: [dts] [PATCH] framework/dut: setup hugepage for all availabl= e > NUMA nodes >=20 > Hi Marvin, >=20 > Thanks for your comment. > I reworked this patch, please review it. >=20 > Thanks, > Phil Yang >=20 > > -----Original Message----- > > From: dts On Behalf Of Liu, Yong > > Sent: Monday, August 27, 2018 1:28 PM > > To: dts ; dts@dpdk.org > > Cc: nd > > Subject: Re: [dts] [PATCH] framework/dut: setup hugepage for all availa= ble > > NUMA nodes > > > > Hi Phil, > > One comment is inline. > > > > Thanks, > > Marvin > > > > > -----Original Message----- > > > From: dts [mailto:dts-bounces@dpdk.org] > > > Sent: Monday, August 27, 2018 9:16 AM > > > To: dts@dpdk.org > > > Cc: nd@arm.com > > > Subject: [dts] [PATCH] framework/dut: setup hugepage for all availabl= e > > > NUMA nodes > > > > > > On a NUMA machine, we need to setup hugepage for all NUMA nodes. > > > > > > The default approach is by setting /sys/kernel/mm/hugepages, but it > > > only works on a single-node system. > > > > > > This fix needs numactl tool support on Linux. > > > > > > Signed-off-by: Phil Yang > > > --- > > > framework/crb.py | 6 ++++-- > > > framework/dut.py | 9 ++++++++- > > > 2 files changed, 12 insertions(+), 3 deletions(-) > > > > > > diff --git a/framework/crb.py b/framework/crb.py index > > > 5c555db..97bebfe 100644 > > > --- a/framework/crb.py > > > +++ b/framework/crb.py > > > @@ -204,8 +204,10 @@ class Crb(object): > > > if numa =3D=3D -1: > > > self.send_expect('echo %d > > > > /sys/kernel/mm/hugepages/hugepages- > > > %skB/nr_hugepages' % (huge_pages, page_size), '# ', 5) > > > else: > > > - # sometimes we set hugepage on kernel cmdline, so we nee= d > clear > > > default hugepage > > > - self.send_expect('echo 0 > > /sys/kernel/mm/hugepages/hugepages- > > > %skB/nr_hugepages' % (page_size), '# ', 5) > > > + # sometimes we set hugepage on kernel cmdline, so we > > > + clear all > > > nodes' default hugepages at the first time. > > > + if numa =3D=3D 0: > > > + self.send_expect('echo 0 > > > > /sys/kernel/mm/hugepages/hugepages-%skB/nr_hugepages' % (page_size), > > > '# ', 5) > > > + > > > # some platform not support numa, example vm dut > > > try: > > > self.send_expect('echo %d > > > > /sys/devices/system/node/node%d/hugepages/hugepages- > > %skB/nr_hugepages' > > > % (huge_pages, numa, page_size), '# ', 5) diff --git > > > a/framework/dut.py b/framework/dut.py index 18f0b39..b358578 100644 > > > --- a/framework/dut.py > > > +++ b/framework/dut.py > > > @@ -305,6 +305,12 @@ class Dut(Crb): > > > return > > > hugepages_size =3D self.send_expect("awk '/Hugepagesize/ {pr= int > $2}' > > > /proc/meminfo", "# ") > > > total_huge_pages =3D self.get_total_huge_pages() > > > + total_numa_nodes =3D self.send_expect("numactl -H | awk > > > + '/available/ > > > {print $2}'", "# ") > > > > We may not expect that numactl is installed on the DUT. It will be more > portable > > by checking sysfs like "ls /sys/devices/system/node | grep node* | wc -= l". > > > > > + numa_service_num =3D > > > self.get_def_rte_config('CONFIG_RTE_MAX_NUMA_NODES') > > > + if numa_service_num: > > > + numa =3D min(total_numa_nodes, numa_service_num) > > > + else: > > > + numa =3D total_numa_nodes > > > force_socket =3D False > > > > > > if int(hugepages_size) < (1024 * 1024): > > > @@ -332,7 +338,8 @@ class Dut(Crb): > > > if force_socket: > > > self.set_huge_pages(arch_huge_pages, 0) > > > else: > > > - self.set_huge_pages(arch_huge_pages) > > > + for numa_id in range(0, int(numa)): > > > + self.set_huge_pages(arch_huge_pages, numa_id= ) > > > > > > self.mount_huge_pages() > > > self.hugepage_path =3D self.strip_hugepage_path() > > > -- > > > 2.7.4