From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR02-AM5-obe.outbound.protection.outlook.com (mail-eopbgr00052.outbound.protection.outlook.com [40.107.0.52]) by dpdk.org (Postfix) with ESMTP id 76217DED; Tue, 28 Aug 2018 05:10:58 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector1-arm-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Tae2JLT9y02WyezdF4fPLNGS61z45G+Vo+ndOsQOHRw=; b=IbzK1ZyngVPqwbfuyPM09kHB/5VfWJUcMuyZ47eOCC3vXc/a8YUSiE3z18o6bGyOZMDwPRiWzk1KV96b4lh0OgX+/97WzT8WE8jwAt6Gab8qTh9tP46VwUX0GLXSiUqjhNrnKwNHznqZr8c69hY08tYNFu+0lb25f3jQFHNLiOQ= Received: from DB7PR08MB3385.eurprd08.prod.outlook.com (20.176.238.90) by DB7PR08MB3674.eurprd08.prod.outlook.com (20.177.120.156) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1080.14; Tue, 28 Aug 2018 03:10:57 +0000 Received: from DB7PR08MB3385.eurprd08.prod.outlook.com ([fe80::8895:d40e:da45:f53c]) by DB7PR08MB3385.eurprd08.prod.outlook.com ([fe80::8895:d40e:da45:f53c%2]) with mapi id 15.20.1080.015; Tue, 28 Aug 2018 03:10:57 +0000 From: Phil Yang To: "Liu, Yong" , dts , "dts@dpdk.org" CC: nd Thread-Topic: [dts] [PATCH] framework/dut: setup hugepage for all available NUMA nodes Thread-Index: AQHUPbPKpTWzdfrdIky6aiJOuoBsB6TTEe4AgAFT2gCAABSTgIAAA4Yg Date: Tue, 28 Aug 2018 03:10:57 +0000 Message-ID: References: <1535332589-8090-1-git-send-email-phil.yang@arm.com> <86228AFD5BCD8E4EBFD2B90117B5E81E6312D154@SHSMSX103.ccr.corp.intel.com> <86228AFD5BCD8E4EBFD2B90117B5E81E6312DAB6@SHSMSX103.ccr.corp.intel.com> In-Reply-To: <86228AFD5BCD8E4EBFD2B90117B5E81E6312DAB6@SHSMSX103.ccr.corp.intel.com> Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Phil.Yang@arm.com; x-originating-ip: [113.29.88.7] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; DB7PR08MB3674; 6:T13uGJl1UXULAlmAlmRxqVylLEaeo+Xpv1poVv+RVqH0vHWyT2dEvYOY9leRJ83svvNICjxecWyTPCWVVLcPhp38qfqq/my7//iRH/mJ7rGdw/2CSKAsCXNQjRpue1mfzokW/ZAZOUbpbz8V483n4r+L+j7TmMYeT+bIM4FY83aVXFo5XElUjww+ON0H3xarCIwDjHIesSJITY5SLZv8UMfJYzM4uxEV1coZbN1YRXZBKOKYUCfl0+pLN4IFBc3qCXxmTEuKcHK4Qh1zYF6CrUpjLG4sUgAM11D9fs4Gk0NyrbkUOSvzqa6CQUCL/YC3UjPFbQiDMDDN9KVk0kFL35bHzbQcH6+eIw/53AYx1Jb7fpqGiMULPHnD5S/MlZg+pUADLBJy0WZE7/Wxdqjhv+GePdtlaHU6Dn1RtsjpM3W6lhI0bm2lnScMH2ivtrHRBIjKzIOvNAeM89A/WOHYDA==; 5:pkEgzAnaptkf22oZgoquauls25GOr0mgX5kk1gTwYE6jQEnUsTY2hP7mR7vIGsHRNa8zOyQH+ArGaKHFFFNgl/mPGUfs4SWF0TJ+VPyKkjOt61A9DdILCO9yS90NpfHJ+IAcc5CRob03gBhgQdA6uIb1iI9fRMVdbGXcJBIyj+w=; 7:PIJrpXo9O0ou+7h38dgk7Kolw4TZNDpN8PeR6gMaJkKJcsDIReyFv4eExe6u6RIStAtq3Nf9WtjSyq+ed7NAnWHw+6KHyBshcmr2ncBW7zriQ55fc1GYF/zLfTKI5gv0/vZ26YXPX8hxp1C9ckrroRfhzP3o8vjKaCG31aRKQqO0fIepoqWyAenUbcMH+4acbLyOB57nMjadxmLWmumLj39gI0tc7g4qOSNUYkTOXp109zbvyQXALfRjV0Obaov0 x-ms-exchange-antispam-srfa-diagnostics: SOS; x-ms-office365-filtering-correlation-id: 42334df9-8958-4a75-5d5e-08d60c93dff8 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989137)(4534165)(4627221)(201703031133081)(201702281549075)(8990107)(5600074)(711020)(4618075)(2017052603328)(7153060)(7193020); SRVR:DB7PR08MB3674; x-ms-traffictypediagnostic: DB7PR08MB3674: nodisclaimer: True x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(180628864354917)(228905959029699); x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(823301075)(3231311)(944501410)(52105095)(3002001)(93006095)(93001095)(10201501046)(6055026)(149027)(150027)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123564045)(20161123560045)(20161123558120)(20161123562045)(201708071742011)(7699016); SRVR:DB7PR08MB3674; BCL:0; PCL:0; RULEID:; SRVR:DB7PR08MB3674; x-forefront-prvs: 077884B8B5 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(39860400002)(366004)(136003)(346002)(396003)(376002)(13464003)(199004)(189003)(6506007)(4326008)(93886005)(110136005)(305945005)(5660300001)(105586002)(2501003)(5250100002)(53936002)(97736004)(186003)(26005)(106356001)(478600001)(8936002)(6436002)(53546011)(316002)(102836004)(55016002)(7696005)(229853002)(81156014)(81166006)(33656002)(9686003)(99286004)(76176011)(6246003)(14454004)(486006)(25786009)(476003)(2900100001)(7736002)(575784001)(86362001)(2906002)(66066001)(256004)(74316002)(446003)(11346002)(72206003)(6116002)(3846002)(68736007); DIR:OUT; SFP:1101; SCL:1; SRVR:DB7PR08MB3674; H:DB7PR08MB3385.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: IXNYvejrP+4WnHFDdwavFuzcriN4Zi2rQko/ajlTniMrpH0OMyyqOm9ZZ6kbwhFdzS2F+eXe+Sc+y3rS5+gv+IpNjHMtbNG90Yx9SLoNsdOtnD1N2Oueyif51VReSr0qhilyHY4blKeAQWSUdoxYpKVweBcneu8xqImtIIzA80e85Rdf7q5QL3uqjJEEsYrSlY8myI1hXmL9jhnShXx+j8e1Ey8j43MTgSTWe7wdhdAuVcFmEMg0X+AkrwwZ+QC94ziJm0Mhe/SUyRo1v1MpvTLa30ghHfLZ4nrBQZDkG3JaaFQnFIzLz142QMEQcvyS61n7Ge8dN4LhhepxLHIA+93bOanMslVmFnxv0SNI2pk= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-Network-Message-Id: 42334df9-8958-4a75-5d5e-08d60c93dff8 X-MS-Exchange-CrossTenant-originalarrivaltime: 28 Aug 2018 03:10:57.5975 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR08MB3674 Subject: Re: [dts] [PATCH] framework/dut: setup hugepage for all available NUMA nodes X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Aug 2018 03:10:58 -0000 Yes. That's very thoughtful. Reworked. Please review it. Thanks, Phil Yang > -----Original Message----- > From: Liu, Yong > Sent: Tuesday, August 28, 2018 10:58 AM > To: Phil Yang ; dts ; > dts@dpdk.org > Cc: nd > Subject: RE: [dts] [PATCH] framework/dut: setup hugepage for all availabl= e > NUMA nodes >=20 > You're welcome, Phil. > One more comment, It's better to make total_numa_nodes and > numa_service_num variables as integer before comparing them. > Function "min" may not return the expect result when numa number over ten= . >=20 > Thanks, > Marvin >=20 > > -----Original Message----- > > From: Phil.Yang@arm.com [mailto:Phil.Yang@arm.com] > > Sent: Tuesday, August 28, 2018 10:01 AM > > To: Liu, Yong ; dts ; > > dts@dpdk.org > > Cc: nd > > Subject: RE: [dts] [PATCH] framework/dut: setup hugepage for all > > available NUMA nodes > > > > Hi Marvin, > > > > Thanks for your comment. > > I reworked this patch, please review it. > > > > Thanks, > > Phil Yang > > > > > -----Original Message----- > > > From: dts On Behalf Of Liu, Yong > > > Sent: Monday, August 27, 2018 1:28 PM > > > To: dts ; dts@dpdk.org > > > Cc: nd > > > Subject: Re: [dts] [PATCH] framework/dut: setup hugepage for all > > > available NUMA nodes > > > > > > Hi Phil, > > > One comment is inline. > > > > > > Thanks, > > > Marvin > > > > > > > -----Original Message----- > > > > From: dts [mailto:dts-bounces@dpdk.org] > > > > Sent: Monday, August 27, 2018 9:16 AM > > > > To: dts@dpdk.org > > > > Cc: nd@arm.com > > > > Subject: [dts] [PATCH] framework/dut: setup hugepage for all > > > > available NUMA nodes > > > > > > > > On a NUMA machine, we need to setup hugepage for all NUMA nodes. > > > > > > > > The default approach is by setting /sys/kernel/mm/hugepages, but > > > > it only works on a single-node system. > > > > > > > > This fix needs numactl tool support on Linux. > > > > > > > > Signed-off-by: Phil Yang > > > > --- > > > > framework/crb.py | 6 ++++-- > > > > framework/dut.py | 9 ++++++++- > > > > 2 files changed, 12 insertions(+), 3 deletions(-) > > > > > > > > diff --git a/framework/crb.py b/framework/crb.py index > > > > 5c555db..97bebfe 100644 > > > > --- a/framework/crb.py > > > > +++ b/framework/crb.py > > > > @@ -204,8 +204,10 @@ class Crb(object): > > > > if numa =3D=3D -1: > > > > self.send_expect('echo %d > > > > > /sys/kernel/mm/hugepages/hugepages- > > > > %skB/nr_hugepages' % (huge_pages, page_size), '# ', 5) > > > > else: > > > > - # sometimes we set hugepage on kernel cmdline, so we n= eed > > clear > > > > default hugepage > > > > - self.send_expect('echo 0 > > > /sys/kernel/mm/hugepages/hugepages- > > > > %skB/nr_hugepages' % (page_size), '# ', 5) > > > > + # sometimes we set hugepage on kernel cmdline, so we > > > > + clear all > > > > nodes' default hugepages at the first time. > > > > + if numa =3D=3D 0: > > > > + self.send_expect('echo 0 > > > > > /sys/kernel/mm/hugepages/hugepages-%skB/nr_hugepages' % > > > > (page_size), '# ', 5) > > > > + > > > > # some platform not support numa, example vm dut > > > > try: > > > > self.send_expect('echo %d > > > > > /sys/devices/system/node/node%d/hugepages/hugepages- > > > %skB/nr_hugepages' > > > > % (huge_pages, numa, page_size), '# ', 5) diff --git > > > > a/framework/dut.py b/framework/dut.py index 18f0b39..b358578 > > > > 100644 > > > > --- a/framework/dut.py > > > > +++ b/framework/dut.py > > > > @@ -305,6 +305,12 @@ class Dut(Crb): > > > > return > > > > hugepages_size =3D self.send_expect("awk '/Hugepagesize/ > > > > {print > > $2}' > > > > /proc/meminfo", "# ") > > > > total_huge_pages =3D self.get_total_huge_pages() > > > > + total_numa_nodes =3D self.send_expect("numactl -H | awk > > > > + '/available/ > > > > {print $2}'", "# ") > > > > > > We may not expect that numactl is installed on the DUT. It will be > > > more > > portable > > > by checking sysfs like "ls /sys/devices/system/node | grep node* | wc= -l". > > > > > > > + numa_service_num =3D > > > > self.get_def_rte_config('CONFIG_RTE_MAX_NUMA_NODES') > > > > + if numa_service_num: > > > > + numa =3D min(total_numa_nodes, numa_service_num) > > > > + else: > > > > + numa =3D total_numa_nodes > > > > force_socket =3D False > > > > > > > > if int(hugepages_size) < (1024 * 1024): > > > > @@ -332,7 +338,8 @@ class Dut(Crb): > > > > if force_socket: > > > > self.set_huge_pages(arch_huge_pages, 0) > > > > else: > > > > - self.set_huge_pages(arch_huge_pages) > > > > + for numa_id in range(0, int(numa)): > > > > + self.set_huge_pages(arch_huge_pages, > > > > + numa_id) > > > > > > > > self.mount_huge_pages() > > > > self.hugepage_path =3D self.strip_hugepage_path() > > > > -- > > > > 2.7.4