From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 64801428BA; Tue, 4 Apr 2023 04:47:59 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E00994067E; Tue, 4 Apr 2023 04:47:58 +0200 (CEST) Received: from EUR04-HE1-obe.outbound.protection.outlook.com (mail-he1eur04on2047.outbound.protection.outlook.com [40.107.7.47]) by mails.dpdk.org (Postfix) with ESMTP id 4037140141 for ; Tue, 4 Apr 2023 04:47:57 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zOXIqh5z918kDa/G52AuQS65stzbmvtQe917p8A7AEc=; b=aeZzfMN6Cyi32xCrmxRmU6rDP8a8jYRwIT/KvaRg/rBdjSMP8xznnnYgUNOOBCO1FJm4NLfhl0wABErTfz+k/AzFkU2XOCrg+icYfk0DbWXD2lJLPPiiSJPpp0Y7YkrcoyVLo31y/PzYpFa6lDabq5ztscMwoxaTESq7go94ZgE= Received: from DB6PR07CA0005.eurprd07.prod.outlook.com (2603:10a6:6:2d::15) by DU0PR08MB9752.eurprd08.prod.outlook.com (2603:10a6:10:444::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6254.26; Tue, 4 Apr 2023 02:47:53 +0000 Received: from DBAEUR03FT043.eop-EUR03.prod.protection.outlook.com (2603:10a6:6:2d:cafe::13) by DB6PR07CA0005.outlook.office365.com (2603:10a6:6:2d::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6254.19 via Frontend Transport; Tue, 4 Apr 2023 02:47:53 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DBAEUR03FT043.mail.protection.outlook.com (100.127.143.24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6277.28 via Frontend Transport; Tue, 4 Apr 2023 02:47:53 +0000 Received: ("Tessian outbound e13c2446394c:v136"); Tue, 04 Apr 2023 02:47:53 +0000 X-CR-MTA-TID: 64aa7808 Received: from 091459795c66.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 09446407-543A-40E4-BDEB-06CAC39C9BFD.1; Tue, 04 Apr 2023 02:47:43 +0000 Received: from EUR05-VI1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 091459795c66.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Tue, 04 Apr 2023 02:47:43 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ZIgq/5OVofaK7njKam7kZa9kQ7yj5RgI9H/+J7hcFjpjKfHlTfAMleAs9P+B46CP1jOFO+4mi0phgZxj70bJMIZXc3oBa9OTZWcSwHc8DtsGSnRCU4dqOIGA1PrfYh7ve7TUjeYsmWO2aCaPOfh/6I7PdbFAmlEfTGyI/C+Q4wiLOW0lyiX8mTBbdWrOcLCFJ8IER4olg8QGLucZ+f+FZ0P8KObm/P9lC/X5Amva9pDTEUvZyflX8Zwyn3tptsaCID9/dvfV0N17DraAjUfwNuKeRC2KKpPJsFMLzKjCluZQWefdiZrLaA2YfVF8O0wNpL5QjmZQCp1pj/Fft85LXQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=zOXIqh5z918kDa/G52AuQS65stzbmvtQe917p8A7AEc=; b=MrAPw4qbJBHByGCZzyXzg8d7ukcCrrs/Wq/kvnWXnMJBG3EWwcHncEuYn7MvanAQISahI4Z6NBrgfdZBdiPiSGWVL4AdXu7YL9b0HbF28xnE5z9a4zYWxDQCwpnKgsVUwSSMyMePFAmuLNYuEXIlSv6tWSNNmuqrIRhfpQT2PfOJyAtOk2KfnVbxPypMDDdxufA+8MiruXN1H9ZBGTEXtoCbg3DtY+O/R1q3fjE/lal2urrmORDHQw7qxH18g+WKdX68ayP6B2KBG6LjFA9fR4YmK7dvjHRMnq/m83QP3CoF9UVGtL5RH9XOGfW+bmbo2+W7/QjZf0sh8hvQsMqDfg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zOXIqh5z918kDa/G52AuQS65stzbmvtQe917p8A7AEc=; b=aeZzfMN6Cyi32xCrmxRmU6rDP8a8jYRwIT/KvaRg/rBdjSMP8xznnnYgUNOOBCO1FJm4NLfhl0wABErTfz+k/AzFkU2XOCrg+icYfk0DbWXD2lJLPPiiSJPpp0Y7YkrcoyVLo31y/PzYpFa6lDabq5ztscMwoxaTESq7go94ZgE= Received: from AS8PR08MB7718.eurprd08.prod.outlook.com (2603:10a6:20b:50a::22) by PAXPR08MB7333.eurprd08.prod.outlook.com (2603:10a6:102:230::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6254.35; Tue, 4 Apr 2023 02:47:35 +0000 Received: from AS8PR08MB7718.eurprd08.prod.outlook.com ([fe80::95ea:4c82:b97f:5ff6]) by AS8PR08MB7718.eurprd08.prod.outlook.com ([fe80::95ea:4c82:b97f:5ff6%4]) with mapi id 15.20.6254.030; Tue, 4 Apr 2023 02:47:35 +0000 From: Feifei Wang To: Alexander Kozyrev , Ruifeng Wang CC: "dev@dpdk.org" , nd Subject: RE: ARM power-saving measurements Thread-Topic: ARM power-saving measurements Thread-Index: AdlV+DWpXWmoICSHRem/UCeySG5oQQN/b42QAKpsUKA= Date: Tue, 4 Apr 2023 02:47:35 +0000 Message-ID: References: In-Reply-To: Accept-Language: zh-CN, en-US Content-Language: zh-CN X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: D6A77E78F930CB4DAA20BC365EC70330.0 x-checkrecipientchecked: true Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; x-ms-traffictypediagnostic: AS8PR08MB7718:EE_|PAXPR08MB7333:EE_|DBAEUR03FT043:EE_|DU0PR08MB9752:EE_ X-MS-Office365-Filtering-Correlation-Id: 561cd751-6f40-4104-cfc0-08db34b6fce1 x-checkrecipientrouted: true nodisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: a70RCLX839lP6iZeNVv0SuIdZviY32LOqnOE5cZyroGLGi1vQ0lQ9E+34RUX3rU0TUCLddlWyPUZffnMjKhTCtcNgH6i+lj31u5bQvhbYWdDHHMFPDOqymrRoFh5hDtNLG0lJcJ1CIgHUtgbKRkM9E8Ijov0e/EmkRTcioWgVRx6uW2c/9f+uRLbdJLNZ3PnV326H0Tk5lxmzLOxH2D4VD83ZXpfORCW38vKIevaS6kCfn3Vh+Iwi8e+4LbIwMxj7hBJvCaiKtbIpUGThMD/yeZgmwq+yY1vA8uVKVa49ZLjeLAX9pyM/yhQc5gksqdNMxCIUTADOoVDRFm2itwoutfrKcy52KIKgcqUGiDTiJIcLHO5WgHidLUAo6ciPOtxEpKYVOBBeRqIrQ1Quhsx5/p9PBNOf4L6ERQSjKid4/66aDPisQT3t6M7x01i12rpRDnMg584nl2NVJXNHYjZHW49v4MXcIM677XMYpOB+Vgi820zA7K0nJFQBcQ/d2exCx0sDTAsDWfJ/euI4t4qE1agIPRat46tEx4B/JQqUB6T3aMzkMTWJlD/4okqhBLaZjZCnVcRWXjVyH2ZI8URFlltsTiYmdCxYfFz1sFMQRc= X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AS8PR08MB7718.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(4636009)(346002)(136003)(366004)(376002)(396003)(39860400002)(451199021)(66946007)(66476007)(66446008)(66556008)(41300700001)(64756008)(4326008)(8676002)(76116006)(52536014)(8936002)(5660300002)(6636002)(54906003)(478600001)(9686003)(7696005)(966005)(6506007)(26005)(53546011)(186003)(83380400001)(316002)(110136005)(33656002)(3480700007)(122000001)(86362001)(38100700002)(2906002)(71200400001)(38070700005)(55016003); DIR:OUT; SFP:1101; Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB7333 Original-Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DBAEUR03FT043.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: a2cce998-0cb5-4581-c179-08db34b6f23d X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ygbkwiuHsQrDZMbnAp7XsrI2uHnTv9qEm6bYJnPpmMsvt3ngACcrqGpnZgR5D73hMWqEYgO1UwmB55chOiT35neXucaE6GrSyvIt/L/lHuqvzgtc1Us7O4s7H1F+oLprxnKUp267K3d+CUjquU8hRq764ROLwvgq6q3eBcKn9hUWORAayuqNUj80ZIGuDGG+2ruVTeeiHe8VA4UYttMQj6y7y6ljOL0GXX2HVvK8WWL5ZOCnCt6gii58Pqoqsg6wjksRRX8JWUcf1AfC5XGbACfRsmzGy9r4aX4nBAwPWHfanWEZ2cm6C2pOObLdH/7oTXKXOKgsE+x+5FFBNlF/PK19weWrqIuPoznrcjbJD2jhrhQ1iWSkKSjz9GFpyRbCK3c+DcFINb6HDVd8DQdJ6iIsP5YAiK0CY6+PPW1jZUCc+Fo1luj88UKua+11bC1EDobcYoZXGJuQp1hekgh8sLJAqCBT8E9NBygVs1koG39aKzbxkrXnEeUjhaXQVIAEu+tZnN2mDyjREHigIeljEBfPz3ZOAn6cQ9B2CTrOH4M7XS1CTlge6YPoB2/aJ10yjkL1bRbHGEgEoA6gcTfhWpvjRwPTc9vQEH+sn2+OguBFF4X2u+PFGX1irhFgWEfxAcQJmQd1BhuMFXqzTZxA589zsWf1gi2hYSpiXkT0TapOa8IKGD8SVCYxgfKLkn9N X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(13230028)(4636009)(136003)(396003)(39860400002)(346002)(376002)(451199021)(40470700004)(36840700001)(46966006)(55016003)(9686003)(6506007)(186003)(26005)(41300700001)(40460700003)(110136005)(54906003)(6636002)(478600001)(316002)(8936002)(33656002)(8676002)(82740400003)(53546011)(7696005)(86362001)(70206006)(2906002)(5660300002)(966005)(70586007)(82310400005)(52536014)(40480700001)(4326008)(81166007)(47076005)(336012)(3480700007)(36860700001)(83380400001)(356005); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Apr 2023 02:47:53.4250 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 561cd751-6f40-4104-cfc0-08db34b6fce1 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: DBAEUR03FT043.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR08MB9752 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Hi, Alex Thanks for your attention. Following I will share how to measure the power savings in Arm: 1.Test Setup: 1.1 ensure the dpdk version has been applied the wfe patch: http://patches.dpdk.org/project/dpdk/cover/20230220085109.3463640-1-feifei.= wang2@arm.com/ This is due to dpdk cannot support l3fwd-power 'power monitor' in arm befo= re this patch 1.2 enable wfe instructions in arm: dpdk/config/arm/meson.build: 'RTE_ARM_USE_WFE', false -> 'RTE_ARM_USE_WFE', true 1.3 build l3fwd-power meson -Dexamples=3Dl3fwd-power build && cd build && ninja 2. Test Command: L3fwd-power uses 'power monitor' mode sudo ./examples/dpdk-l3fwd-power -l 14-15 -n 4 -a 0000:13:00.0 -a 0000:13:0= 0.1 -- -p 0x3 -P --config=3D'(0,0,14),(1,0,15)' --pmd-mgmt=3Dmonitor note: power monitor can just support more than 1 cores. 3. Power saving count: For arm, wfe instructions can let CPU go into sleeping mode if it is always= idle. Thus, we have 2 ways to collect power saving info. 3.1 Using perf to record dynamic instruction number within 1 second: $sudo perf stat -C , and then we can get CPU instructions number du= ring a fixed time slot. For examples: Performance counter stats for 'CPU(s) 14': 3,506.56 msec cpu-clock # 1.000 CPUs utilized 0 context-switches # 0.000 K/sec 0 cpu-migrations # 0.000 K/sec 0 page-faults # 0.000 K/sec 8,768,815,778 cycles # 2.501 GHz 26,071,802 instructions # 0.00 insn per cycle branches 104,982 branch-misses 3.506978873 seconds time elapsed Dynamic instructions number for 1 second =3D 26,071,802/3.506978873 If we disable wfe in arm, we can find 'dynamic instructions number for 1 se= cond' increases significantly. This also means CPU is running at full load. 3.2 Get CPU power form BMC: For ampere altra and thunderx2 server, we can look up CPU timely power from= BMC. For example, when we run l3fwd-power in thunderx2, server information->powe= r->consumption: ---------------------------------------------------------------------------= -------- CPU0 Current Power Consumption 69 W | 235 BTU/hr CPU1 Current Power Consumption 0 W | 0 BTU/hr CPU Power Limit Management Activated=09 CPU Power Limit in Watts(1-150)=09 ---------------------------------------------------------------------------= -------- If we disable wfe, CPU0 Current Power Consumption will increase, then we ca= n know how much power we can save. Best Regards Feifei > -----Original Message----- > From: Alexander Kozyrev > Sent: Saturday, April 1, 2023 1:29 AM > To: Feifei Wang ; Ruifeng Wang > > Cc: dev@dpdk.org > Subject: RE: ARM power-saving measurements >=20 > Hi Feifei/Ruifeng, any advice on power saving measurements on ARM? >=20 > Regards, > Alex >=20 > > -----Original Message----- > > From: Alexander Kozyrev > > Sent: March 13, 2023 18:08 > > To: Feifei Wang ; ruifeng.wang@arm.com > > Subject: ARM power-saving measurements > > > > Hi Feifei/ Ruifeng, I can see that power management support on ARM has > > been integrated successfully. > > Could you please share the knowledge on how to measure the power > > savings on this architecture? > > What are the tools available there? Unfortunately, turbostat is not > > supported on ARM. How did you test your feature? > > > > Regards, > > Alex