From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A3248A0544; Fri, 23 Sep 2022 15:15:17 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 96BB0406A2; Fri, 23 Sep 2022 15:15:17 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mails.dpdk.org (Postfix) with ESMTP id 82D014003F for ; Fri, 23 Sep 2022 15:15:15 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1663938915; x=1695474915; h=message-id:date:from:subject:to:cc:references: in-reply-to:content-transfer-encoding:mime-version; bh=idRPCELN1urKHBXCdQgv5MfJSUyvOEow96zDkJs3SmQ=; b=jv7VNpK3W4i8kklaBhEaQrKUuXhMSI97RIk1LZ1/mav448ky6dDjevZP yMWndU1nMPS6ZLV8pGepiG9tsz5Bg6LqsAry3wFKi3RGI3J7lQ4G0bJUJ QAWZtITEJ1FPpH1eLQI1p5B+MBjCzrDn23qAKBy8lyxnlCsyrmYAn9ql6 8Quh7ImEz1nJzBCgnn4MN5nZx923cmcR8aX7PzWvxH6RenCGpd57DUzXg hNHB4wCTXK1RjYYcDGWzBTtUM3ye2FOg2CVi4cfDG1+dXrB3YTEG8+hz9 3BPccxp6B88UpTTQSuQ9caSW8AQomw/bv57ZdqFnwYEMR3Id1u5a0mmCx g==; X-IronPort-AV: E=McAfee;i="6500,9779,10479"; a="326918084" X-IronPort-AV: E=Sophos;i="5.93,339,1654585200"; d="scan'208";a="326918084" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Sep 2022 06:15:14 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,339,1654585200"; d="scan'208";a="688711768" Received: from orsmsx601.amr.corp.intel.com ([10.22.229.14]) by fmsmga004.fm.intel.com with ESMTP; 23 Sep 2022 06:15:14 -0700 Received: from orsmsx612.amr.corp.intel.com (10.22.229.25) by ORSMSX601.amr.corp.intel.com (10.22.229.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Fri, 23 Sep 2022 06:15:14 -0700 Received: from ORSEDG601.ED.cps.intel.com (10.7.248.6) by orsmsx612.amr.corp.intel.com (10.22.229.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31 via Frontend Transport; Fri, 23 Sep 2022 06:15:14 -0700 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (104.47.58.109) by edgegateway.intel.com (134.134.137.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2375.31; Fri, 23 Sep 2022 06:15:13 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FXCRBfuNb5QTsikuS9PLvvaP6R3UC3EBaDKxnHNnKLQiMB31yDsrhWvffEBzEYA/ED5sUeC559EZthw0U1SUJEvxRFl2suCXfKhlrtZpFyWbecNuPNTQpRfjsYjP6K7a46wjeNnFL1PQi2cqmlmX5WKGafNTVnzxWkaqSgp9P/zDc9D8cGdiBF8eRGl8Spn6gojMhVtSbu3bZmUQ82mC23td6Pa0ojwAlf91+UlMalY2rf25rr22tGo2hBkxNYTz3BURF4PdALTr96ATHMeODQfdkAOGDiBrOGehhm3nGHee2sI57RAcaqSz+Vp9BA7cn1b1YNaJN2cZqAgY3dPiug== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=lj3euyHpdkmoW0FmXIJB/LH8zRNuQJ5eXXyZI92z2b0=; b=lIAdUKtveKIPfiK+/TqKttvlgJ5GArdyVq3B7LQ/Y8+rf6r88CSmMntjCaT6AOOoySWuQ/Qi0y5viWHOSMQuWLdUbLucsnRHtcsrL3wTAC3yk2Ic34TcTIFq4eyxA6Xw02Ehfy87yia7JKdj9BUGAduv4n0KE4KpI5e4J7TdsHfMf8iyxzJEWffzVVMFhlg+DAhVrdfL64sfn1TYybAJc5r6d4Gdy3J7tRhljHUL4YW3ZMFjZD6o6xwhq1sTbKgpZ7NFbyfx0sYCqDKITjuyD+Tgmz3F/Qiq49PuaYQC8pjbpPvwWyZWzb4rnzJ/4B7U86NJ+EkEb13Y04gzp/HWfA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from BYAPR11MB3799.namprd11.prod.outlook.com (2603:10b6:a03:fb::19) by SJ0PR11MB5119.namprd11.prod.outlook.com (2603:10b6:a03:2d6::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5632.21; Fri, 23 Sep 2022 13:15:11 +0000 Received: from BYAPR11MB3799.namprd11.prod.outlook.com ([fe80::1331:f80e:20cb:3546]) by BYAPR11MB3799.namprd11.prod.outlook.com ([fe80::1331:f80e:20cb:3546%5]) with mapi id 15.20.5654.019; Fri, 23 Sep 2022 13:15:11 +0000 Message-ID: <9a0daa0c-ca4d-2e25-d762-222a1a51e22b@intel.com> Date: Fri, 23 Sep 2022 14:15:06 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.13.0 From: "Hunt, David" Subject: Re: [PATCH v4 1/3] power: add uncore frequency control API to the power library To: Tadhg Kearney , CC: , References: <20220920094822.1403168-1-tadhg.kearney@intel.com> <20220920094822.1403168-2-tadhg.kearney@intel.com> Content-Language: en-US In-Reply-To: <20220920094822.1403168-2-tadhg.kearney@intel.com> Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit X-ClientProxiedBy: LO4P123CA0386.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:18f::13) To BYAPR11MB3799.namprd11.prod.outlook.com (2603:10b6:a03:fb::19) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BYAPR11MB3799:EE_|SJ0PR11MB5119:EE_ X-MS-Office365-Filtering-Correlation-Id: 7652e3b8-85db-4c07-a044-08da9d65a508 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: nJIdlUJ17XNf3DM9h+HjNIUaMBdSztqdy6M/m7xOtjQ4E1BTiKg6FLlzMYYdhJLPo++vp1mw1aiEye54cPmtemJTstpZPWMYyRPDuSroIgqFO7F9Utjc71Qz/Npw1uYLwBTeci2w2IGa4jwehm7hASnqtBJswKbK0TNeL/APU8pL+WrGHbHh6QtAuDGYlFxGg7ChNmt89zYOJmmBSvBA61ndvK2/CYJ6rS4L1X/nj3Iq/LMsM3X5tzXA/cPqNY9eVRDsBvNTjA4JCwOyvS5CmldtD2Y+POZJNWo0Y5CFNdKOiQQryo1s4eWh1DO9CqPyNFwGQ3MWnqsCVOWi90JfuoBwCFV4/SCj+ogCFBMrNChVJhpZ+LDU/mLcHseekc5blnC7ktYs+0GZXjc7iOOaJDLR0PyQBP/SHI/oY9hyrYxpEKojtYz3ZGwDkALLJzOnKOiS4Wti9kFyKJkH+ZqVDA7KgPcf8pe6xDzytyp5AWThS7SuykH0AnDvc+tECJ3Zz9m/ywwqX+kApCmjrDMNawy2wZQHx4ybPoS6Iajx18+ntQ4YW9LIo2nu06IwlGxQJtKy+eJng1KnGNfZI8xCpOc5KjWW+mivRo8EwwYjXc7Mu64+jNULkma9hGS3A459+QQ2mZKa84LE9LnBAGB+dkUuwDSn4DUxQ/a9tYoH/weEDy5khWsAbF+tjZkcskZ4mFgn8zdVzD1tEipluG9lrpsX1mMow/E59OF9jof0Th/kuG4DN6Sfz9y8SmsGQeyCaWCvH/cHN2h5zi/EWfRYnuAM0dBjedivwONwnHIcypU= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR11MB3799.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230022)(366004)(396003)(376002)(346002)(39860400002)(136003)(451199015)(38100700002)(6506007)(82960400001)(478600001)(6486002)(8676002)(316002)(26005)(83380400001)(6666004)(53546011)(2906002)(186003)(107886003)(30864003)(5660300002)(4326008)(66476007)(6512007)(66946007)(8936002)(41300700001)(2616005)(66556008)(36756003)(31696002)(31686004)(86362001)(43740500002)(45980500001); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?MW1ZTHFicTdmTk1QSzU4RUpvNUVHLzlldTZpYlY4cENGZGhFNWREVDhlTFor?= =?utf-8?B?V1VJWEs2eTNhWFJYdnpNamU5QTJ0VWhOL0QvOUR2dmRZQjQ2b0NkL0MvbFJi?= =?utf-8?B?WUJUZDRaclNpby8xOHRHN0IzYUVUOEczTnRjS29xREhSWjJOUnYzOHNIcmZj?= =?utf-8?B?RHVkU0ErcjA0Z05oYjNVTDJBcWVSeXRnRU9hZFJzeWhHV0pGQzQrWGpOVjVG?= =?utf-8?B?d3J0T3ArNzZwUElTUE5jYm5jcEx6R2xsV2NtbUVUV2ZOZTlSanJrKy9xUkY1?= =?utf-8?B?elo4NEMrM2JGWWc2U2dNUFlpazhVYVpPVWtGcWlCNTl5Yi9LU3huOFpJRGs2?= =?utf-8?B?ajVYNm93cnNPOGdCbTUwaEtCd3JDMWtia3c1d0NOdXpWbmNSaExORVdML0l1?= =?utf-8?B?R1MwT1ZRYU9WdTloSnZPZldsQXpDNjBObTJwSjZZc2FHVG5CNVJtZXI0Tzh5?= =?utf-8?B?L2ZUc3NHYlY3c2ZPUE14REhyZy9UZjJQWThoNVg5MW92VDF0V1JraFVtelhp?= =?utf-8?B?Ym1qTFZhYVdZUzBHdVVzeHp4aFBSOThpWGFIcUEvZUxDVkk4OFhOeTZmZmtQ?= =?utf-8?B?aDdDK0NEcGtGZ0FwZktqaEVCNnZNWVBuVHpRWmZhZVlONTgyU2s5L1hFcVM1?= =?utf-8?B?dml1aTFIY0NpWEVtZG85aFZrSUc0a1dXV2FYYm56UmZlM0dPQ0JHQ0xoK3ll?= =?utf-8?B?ZmtUQU4zN3VoOFg5WkpudE1vLzM5ZE9nR1FsZ2F0M2JYS3VWL1VuT0NGSnBZ?= =?utf-8?B?Ym5xM1hUU1FKa3piSGJKM2RoR2VtTFBnSzR6aHpZTFVoV0lEU1ZCL3VJQi9B?= =?utf-8?B?eUUyZ25kZ05FcG5tL3JGVjBxemczbFBFdlcxZklxYTdIbkFzVVViSmlQSFpw?= =?utf-8?B?SVo1MnZ3dTlQVkNWekNVTUc0ZmtRaFR1bXY2NHI5Y1pEajBBQTFJR0liblJP?= =?utf-8?B?TUJuc0J4U2xHbVZITmFYN0UvZEJFaWlPakNrQlZRV1lUYkF0R3kyMnMwd0VW?= =?utf-8?B?L1ZDcW81ZUtJMGlIdlJJeGNRWmdIdnZYa0xMQzBFSEIyWlppNFhNN2wxYTA2?= =?utf-8?B?Ym14NnRrb0YybUdheG9ZdWxmc3dHVEVJdCtoVHpteXZic2c1aUVNb25rcGc2?= =?utf-8?B?aXVWWFVBaDVIcllCWURoQ3F3ODJjUUhqNHI5TzA3NmY4QzJxeVB1dVBXaDlk?= =?utf-8?B?STBiUzN5bDBicVhNQXFvKzZRVEdWOEptYkxsWUcxYnUycnJOWU5PYVZwZFdU?= =?utf-8?B?V240RGZ4WTZDZWlUeVpnc2pNL296VXhHUUY2NTRVeUVoeUQ0V3ZqUDF3Rnhn?= =?utf-8?B?amYvSnVzTnJSelBRK2lHRVFKVVd6bU11Tkt1RVF4ZVNUZndOaDh3Z0huTEkz?= =?utf-8?B?aVozVDFpZXBNU2I2Q21MSk4xeTg1Z0h1bGZmMVVDdkdwMFFaaHJXNWEybFUv?= =?utf-8?B?RW8valByK1N5MXEwczdqaThCc1F3YkprZXhOdExwbHp5cUlGMEZnM0dQWTgr?= =?utf-8?B?RTdxRjN6cFBVSzUzNnQ0aGhzalp6OVZycXdnYUIyZm1lL3VpeWI0OCtPQU91?= =?utf-8?B?bU1CUG1oU01FUkdISUNZWHM2aVRjalRLVmNnMGxSTDJtODQ4ZUtmVkEwU042?= =?utf-8?B?WXh1SlJ5WldDeFczUDVPcEh4QlpTQzFQUG4vUUFxbTcyV21ZU1hib0FXV0tV?= =?utf-8?B?OGd4ZS9vQXlWRlZySXpYcDdLNFV4WkxweXpRL0xCMHN1SWJnaFg4TnNndS84?= =?utf-8?B?VGtjNTZ1ZGlUZmxrQmxqWGZhNTA4eWdOSzNyUW1wbkxTbVVaYVljNDVPL2tk?= =?utf-8?B?S2s0VExiS3VRMW8vVGRiRHkxZnV3NnRGWFpNQVlaN1BzLzZnZUJPMHlCcnlj?= =?utf-8?B?S3Yzck9tbGNiYkp2UTFXcGJvWGx4U3ViVkRiN2xrdXlkc01Sb2NJQmxUYWJu?= =?utf-8?B?TWRLZ1VnQWRrZ1pUUk14cTFZKzd6ZVFMb25SRjVZdlREM3RNaVZ2ajB4d1lU?= =?utf-8?B?QitjcVFoK0NzYVNDNkR0bE1ZbDlsenNiYnFRUW9odjhmSEU0YzdRSGJwTlI0?= =?utf-8?B?MDNraCtJWDRsTGpDcjVUOFpDOHV3OUI4aE1FMi90NGdsek5JVXV5WElYSlBk?= =?utf-8?B?Zjh5U0I5RVVvTjF3SjgvMTdjVzM4SVJpdlVWRTJwOU5sVTNKTndla2ZYMytR?= =?utf-8?B?dHc9PQ==?= X-MS-Exchange-CrossTenant-Network-Message-Id: 7652e3b8-85db-4c07-a044-08da9d65a508 X-MS-Exchange-CrossTenant-AuthSource: BYAPR11MB3799.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2022 13:15:11.5066 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: CD4cXo7dJbe1BHtmIL1WHTW8ENcxXcrMz2CnMWtdeAPWYEiVu16vlRcyHqvrd4OM0JejxBv1VISvg8/HDIQ2Ow== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR11MB5119 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 20/09/2022 10:48, Tadhg Kearney wrote: > Add API to allow uncore frequency adjustment. This is done through > manipulating related uncore frequency control sysfs entries to > adjust the minimum and maximum uncore frequency values. > Nine API's are being added that are all public and experimental. > > Signed-off-by: Tadhg Kearney > --- > doc/guides/prog_guide/power_man.rst | 37 ++ > doc/guides/rel_notes/release_22_11.rst | 5 + > lib/power/meson.build | 2 + > lib/power/rte_power_uncore.c | 447 +++++++++++++++++++++++++ > lib/power/rte_power_uncore.h | 194 +++++++++++ > lib/power/version.map | 11 + > 6 files changed, 696 insertions(+) > create mode 100644 lib/power/rte_power_uncore.c > create mode 100644 lib/power/rte_power_uncore.h > > diff --git a/doc/guides/prog_guide/power_man.rst b/doc/guides/prog_guide/power_man.rst > index 98cfd3c1f3..49ff3edef0 100644 > --- a/doc/guides/prog_guide/power_man.rst > +++ b/doc/guides/prog_guide/power_man.rst > @@ -276,6 +276,43 @@ API Overview for Ethernet PMD Power Management > * **Set Scaling Max Freq**: Set the maximum frequency (kHz) to be used in Frequency > Scaling mode. > > +Uncore API > +---------- > + > +Abstract > +~~~~~~~~ > + > +Uncore is a term used by Intel to describe the functions of a microprocessor that are > +not in the core, but which must be closely connected to the core to achieve high performance; > +L3 cache, on-die memory controller, etc. > +Significant power savings can be achieved by reducing the uncore frequency to its lowest value. > + > +The Linux kernel provides the driver “intel-uncore-frequency" to control the uncore frequency limits > +for x86 platform. The driver is available from kernel version 5.6 and above. > +This manipulates the contest of MSR 0x620, which sets min/max of the uncore for the SKU. Need to mention that the uncore frequency control is made available in the kernel by enabling the CONFIG_INTEL_UNCORE_FREQ_CONTROL kernel option which was added in 5.6 > + > + > +API Overview for Uncore > +~~~~~~~~~~~~~~~~~~~~~~~ > +* **Uncore Power Init**: Initialise uncore power, populate frequency array and record > + original min & max for pkg & die. > + > +* **Uncore Power Exit**: Exit uncore power, restoring original min & max for pkg & die. > + > +* **Get Uncore Power Freq**: Get current uncore freq index for pkg & die. > + > +* **Set Uncore Power Freq**: Set min & max uncore freq index for pkg & die (min and max will be the same). > + > +* **Uncore Power Max**: Set max uncore freq index for pkg & die. > + > +* **Uncore Power Min**: Set min uncore freq index for pkg & die. > + > +* **Get Num Freqs**: Get the number of frequencies in the index array. > + > +* **Get Num Pkgs**: Get the number of packages (CPUs) on the system. > + > +* **Get Num Dies**: Get the number of die's on a given package. > + > References > ---------- > > diff --git a/doc/guides/rel_notes/release_22_11.rst b/doc/guides/rel_notes/release_22_11.rst > index 8c021cf050..8e184034d8 100644 > --- a/doc/guides/rel_notes/release_22_11.rst > +++ b/doc/guides/rel_notes/release_22_11.rst > @@ -55,6 +55,11 @@ New Features > Also, make sure to start the actual text at the margin. > ======================================================= > > +* **Added uncore frequency control API to the power library.** > + > + Add api to allow uncore frequency adjustment. This is done through > + manipulating related uncore frequency control sysfs entries to > + adjust the minimum and maximum uncore frequency values. > > Removed Items > ------------- > diff --git a/lib/power/meson.build b/lib/power/meson.build > index ba8d66074b..80cdeb72d4 100644 > --- a/lib/power/meson.build > +++ b/lib/power/meson.build > @@ -21,12 +21,14 @@ sources = files( > 'rte_power.c', > 'rte_power_empty_poll.c', > 'rte_power_pmd_mgmt.c', > + 'rte_power_uncore.c', > ) > headers = files( > 'rte_power.h', > 'rte_power_empty_poll.h', > 'rte_power_pmd_mgmt.h', > 'rte_power_guest_channel.h', > + 'rte_power_uncore.h', > ) > if cc.has_argument('-Wno-cast-qual') > cflags += '-Wno-cast-qual' > diff --git a/lib/power/rte_power_uncore.c b/lib/power/rte_power_uncore.c > new file mode 100644 > index 0000000000..b3004e5bfc > --- /dev/null > +++ b/lib/power/rte_power_uncore.c > @@ -0,0 +1,447 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(c) 2022 Intel Corporation > + */ > + > +#include > +#include > +#include > + > +#include > + > +#include "rte_power_uncore.h" > +#include "power_common.h" > + > +#define MAX_UNCORE_FREQS 32 > +#define MAX_NUMA_DIE 8 > +#define BUS_FREQ 100000 > +#define FILTER_LENGTH 18 > +#define PACKAGE_FILTER "package_%02u_die_*" > +#define DIE_FILTER "package_%02u_die_%02u" > +#define UNCORE_FREQUENCY_DIR "/sys/devices/system/cpu/intel_uncore_frequency" > +#define POWER_GOVERNOR_PERF "performance" > +#define POWER_UNCORE_SYSFILE_MAX_FREQ \ > + "/sys/devices/system/cpu/intel_uncore_frequency/package_%02u_die_%02u/max_freq_khz" > +#define POWER_UNCORE_SYSFILE_MIN_FREQ \ > + "/sys/devices/system/cpu/intel_uncore_frequency/package_%02u_die_%02u/min_freq_khz" > +#define POWER_UNCORE_SYSFILE_BASE_MAX_FREQ \ > + "/sys/devices/system/cpu/intel_uncore_frequency/package_%02u_die_%02u/initial_max_freq_khz" > +#define POWER_UNCORE_SYSFILE_BASE_MIN_FREQ \ > + "/sys/devices/system/cpu/intel_uncore_frequency/package_%02u_die_%02u/initial_min_freq_khz" > + > + > +struct uncore_power_info { > + unsigned int die; /** Core die id */ > + unsigned int pkg; /** Package id */ > + uint32_t freqs[MAX_UNCORE_FREQS];/** Frequency array */ > + uint32_t nb_freqs; /** Number of available freqs */ > + FILE *f_cur_min; /** FD of scaling_min */ > + FILE *f_cur_max; /** FD of scaling_max */ > + uint32_t curr_idx; /** Freq index in freqs array */ > + uint32_t org_min_freq; /** Original min freq of uncore */ > + uint32_t org_max_freq; /** Original max freq of uncore */ > + uint32_t init_max_freq; /** System max uncore freq */ > + uint32_t init_min_freq; /** System min uncore freq */ > +} __rte_cache_aligned; > + > +static struct uncore_power_info uncore_info[RTE_MAX_NUMA_NODES][MAX_NUMA_DIE]; > + > +static int > +set_uncore_freq_internal(struct uncore_power_info *ui, uint32_t idx) > +{ > + uint32_t target_uncore_freq, curr_max_freq; > + int ret; > + > + if (idx >= MAX_UNCORE_FREQS || idx >= ui->nb_freqs) { > + RTE_LOG(DEBUG, POWER, "Invalid uncore frequency index %u, which " > + "should be less than %u\n", idx, ui->nb_freqs); > + return -1; > + } > + > + target_uncore_freq = ui->freqs[idx]; > + > + /* check current max freq, so that the value to be flushed first > + * can be accurately recorded > + */ > + open_core_sysfs_file(&ui->f_cur_max, "rw+", POWER_UNCORE_SYSFILE_MAX_FREQ, > + ui->pkg, ui->die); > + if (ui->f_cur_max == NULL) { > + RTE_LOG(DEBUG, POWER, "failed to open %s\n", > + POWER_UNCORE_SYSFILE_MAX_FREQ); > + return -1; > + } > + ret = read_core_sysfs_u32(ui->f_cur_max, &curr_max_freq); > + if (ret < 0) { > + RTE_LOG(DEBUG, POWER, "Failed to read %s\n", > + POWER_UNCORE_SYSFILE_MAX_FREQ); > + fclose(ui->f_cur_max); > + return -1; > + } > + > + /* check this value first before fprintf value to f_cur_max, so value isn't overwritten */ > + if (fprintf(ui->f_cur_min, "%u", target_uncore_freq) < 0) { > + RTE_LOG(ERR, POWER, "Fail to write new uncore frequency for " > + "pkg %02u die %02u\n", ui->pkg, ui->die); > + return -1; > + } > + > + if (fprintf(ui->f_cur_max, "%u", target_uncore_freq) < 0) { > + RTE_LOG(ERR, POWER, "Fail to write new uncore frequency for " > + "pkg %02u die %02u\n", ui->pkg, ui->die); > + return -1; > + } > + > + POWER_DEBUG_TRACE("Uncore frequency '%u' to be set for pkg %02u die %02u\n", > + target_uncore_freq, ui->pkg, ui->die); > + > + > + /* write the minimum value first if the target freq is less than current max */ > + if (target_uncore_freq <= curr_max_freq) { > + fflush(ui->f_cur_min); > + fflush(ui->f_cur_max); > + } else { > + fflush(ui->f_cur_max); > + fflush(ui->f_cur_min); > + } > + ui->curr_idx = idx; > + > + return 0; > +} > + > +/** > + * Fopen the sys file for the future setting of the uncore die frequency. > + */ > +static int > +power_init_for_setting_uncore_freq(struct uncore_power_info *ui) > +{ > + FILE *f_base_min = NULL, *f_base_max = NULL, *f_min = NULL, *f_max = NULL; > + uint32_t base_min_freq = 0, base_max_freq = 0, min_freq = 0, max_freq = 0; > + int ret; > + > + /* open and read all uncore sys files */ > + /* Base max */ > + open_core_sysfs_file(&f_base_max, "r", POWER_UNCORE_SYSFILE_BASE_MAX_FREQ, > + ui->pkg, ui->die); > + if (f_base_max == NULL) { > + RTE_LOG(DEBUG, POWER, "failed to open %s\n", > + POWER_UNCORE_SYSFILE_BASE_MAX_FREQ); > + goto err; > + } > + ret = read_core_sysfs_u32(f_base_max, &base_max_freq); > + if (ret < 0) { > + RTE_LOG(DEBUG, POWER, "Failed to read %s\n", > + POWER_UNCORE_SYSFILE_BASE_MAX_FREQ); > + goto err; > + } > + > + /* Base min */ > + open_core_sysfs_file(&f_base_min, "r", POWER_UNCORE_SYSFILE_BASE_MIN_FREQ, > + ui->pkg, ui->die); > + if (f_base_min == NULL) { > + RTE_LOG(DEBUG, POWER, "failed to open %s\n", > + POWER_UNCORE_SYSFILE_BASE_MIN_FREQ); > + goto err; > + } > + if (f_base_min != NULL) { > + ret = read_core_sysfs_u32(f_base_min, &base_min_freq); > + if (ret < 0) { > + RTE_LOG(DEBUG, POWER, "Failed to read %s\n", > + POWER_UNCORE_SYSFILE_BASE_MIN_FREQ); > + goto err; > + } > + } > + > + /* Curr min */ > + open_core_sysfs_file(&f_min, "rw+", POWER_UNCORE_SYSFILE_MIN_FREQ, > + ui->pkg, ui->die); > + if (f_min == NULL) { > + RTE_LOG(DEBUG, POWER, "failed to open %s\n", > + POWER_UNCORE_SYSFILE_MIN_FREQ); > + goto err; > + } > + if (f_min != NULL) { > + ret = read_core_sysfs_u32(f_min, &min_freq); > + if (ret < 0) { > + RTE_LOG(DEBUG, POWER, "Failed to read %s\n", > + POWER_UNCORE_SYSFILE_MIN_FREQ); > + goto err; > + } > + } > + > + /* Curr max */ > + open_core_sysfs_file(&f_max, "rw+", POWER_UNCORE_SYSFILE_MAX_FREQ, > + ui->pkg, ui->die); > + if (f_max == NULL) { > + RTE_LOG(DEBUG, POWER, "failed to open %s\n", > + POWER_UNCORE_SYSFILE_MAX_FREQ); > + goto err; > + } > + if (f_max != NULL) { > + ret = read_core_sysfs_u32(f_max, &max_freq); > + if (ret < 0) { > + RTE_LOG(DEBUG, POWER, "Failed to read %s\n", > + POWER_UNCORE_SYSFILE_MAX_FREQ); > + goto err; > + } > + } > + > + /* assign file handles */ > + ui->f_cur_min = f_min; > + ui->f_cur_max = f_max; > + /* save current min + max freq's so that they can be restored on exit */ > + ui->org_min_freq = min_freq; > + ui->org_max_freq = max_freq; > + ui->init_max_freq = base_max_freq; > + ui->init_min_freq = base_min_freq; > + > + return 0; > + > +err: > + if (f_base_min != NULL) > + fclose(f_base_min); > + if (f_base_max != NULL) > + fclose(f_base_max); > + if (f_min != NULL) > + fclose(f_min); > + if (f_max != NULL) > + fclose(f_max); > + return -1; > +} > + > +/** > + * Get the available uncore frequencies of the specific die by reading the > + * sys file. > + */ > +static int > +power_get_available_uncore_freqs(struct uncore_power_info *ui) > +{ > + int ret = -1; > + uint32_t i, num_uncore_freqs = 0; > + > + num_uncore_freqs = (ui->init_max_freq - ui->init_min_freq) / BUS_FREQ + 1; > + if (num_uncore_freqs >= MAX_UNCORE_FREQS) { > + RTE_LOG(ERR, POWER, "Too many available uncore frequencies: %d\n", > + num_uncore_freqs); > + goto out; > + } > + > + /* Generate the uncore freq bucket array. */ > + for (i = 0; i < num_uncore_freqs; i++) > + ui->freqs[i] = ui->init_max_freq - (i) * BUS_FREQ; > + > + ui->nb_freqs = num_uncore_freqs; > + > + ret = 0; > + > + POWER_DEBUG_TRACE("%d frequency(s) of pkg %02u die %02u are available\n", > + num_uncore_freqs, ui->pkg, ui->die); > + > +out: > + return ret; > +} > + > +static int > +check_pkg_die_values(unsigned int pkg, unsigned int die) > +{ > + unsigned int max_pkgs, max_dies; > + max_pkgs = rte_power_uncore_get_num_pkgs(); > + if (max_pkgs == 0) > + return -1; > + if (pkg >= max_pkgs) { > + RTE_LOG(DEBUG, POWER, "Package number %02u can not exceed %u\n", > + pkg, max_pkgs); > + return -1; > + } > + > + max_dies = rte_power_uncore_get_num_dies(pkg); > + if (max_dies == 0) > + return -1; > + if (die >= max_dies) { > + RTE_LOG(DEBUG, POWER, "Die number %02u can not exceed %u\n", > + die, max_dies); > + return -1; > + } > + > + return 0; > +} > + > +int > +rte_power_uncore_init(unsigned int pkg, unsigned int die) > +{ > + struct uncore_power_info *ui; > + > + int ret = check_pkg_die_values(pkg, die); > + if (ret < 0) > + return -1; > + > + ui = &uncore_info[pkg][die]; > + ui->die = die; > + ui->pkg = pkg; > + > + /* Init for setting uncore die frequency */ > + if (power_init_for_setting_uncore_freq(ui) < 0) { > + RTE_LOG(DEBUG, POWER, "Cannot init for setting uncore frequency for " > + "pkg %02u die %02u\n", pkg, die); > + return -1; > + } > + > + /* Get the available frequencies */ > + if (power_get_available_uncore_freqs(ui) < 0) { > + RTE_LOG(DEBUG, POWER, "Cannot get available uncore frequencies of " > + "pkg %02u die %02u\n", pkg, die); > + return -1; > + } > + > + return 0; > +} > + > +int > +rte_power_uncore_exit(unsigned int pkg, unsigned int die) > +{ > + struct uncore_power_info *ui; > + > + int ret = check_pkg_die_values(pkg, die); > + if (ret < 0) > + return -1; > + > + ui = &uncore_info[pkg][die]; > + > + if (fprintf(ui->f_cur_min, "%u", ui->org_min_freq) < 0) { > + RTE_LOG(ERR, POWER, "Fail to write original uncore frequency for " > + "pkg %02u die %02u\n", ui->pkg, ui->die); > + return -1; > + } > + > + if (fprintf(ui->f_cur_max, "%u", ui->org_max_freq) < 0) { > + RTE_LOG(ERR, POWER, "Fail to write original uncore frequency for " > + "pkg %02u die %02u\n", ui->pkg, ui->die); > + return -1; > + } > + > + fflush(ui->f_cur_min); > + fflush(ui->f_cur_max); > + > + /* Close FD of setting freq */ > + fclose(ui->f_cur_min); > + fclose(ui->f_cur_max); > + ui->f_cur_min = NULL; > + ui->f_cur_max = NULL; > + > + return 0; > +} > + > +uint32_t > +rte_power_get_uncore_freq(unsigned int pkg, unsigned int die) > +{ > + int ret = check_pkg_die_values(pkg, die); > + if (ret < 0) > + return -1; > + > + return uncore_info[pkg][die].curr_idx; > +} > + > +int > +rte_power_set_uncore_freq(unsigned int pkg, unsigned int die, uint32_t index) > +{ > + int ret = check_pkg_die_values(pkg, die); > + if (ret < 0) > + return -1; > + > + return set_uncore_freq_internal(&(uncore_info[pkg][die]), index); > +} > + > +int > +rte_power_uncore_freq_max(unsigned int pkg, unsigned int die) > +{ > + int ret = check_pkg_die_values(pkg, die); > + if (ret < 0) > + return -1; > + > + return set_uncore_freq_internal(&(uncore_info[pkg][die]), 0); > +} > + > + > +int > +rte_power_uncore_freq_min(unsigned int pkg, unsigned int die) > +{ > + int ret = check_pkg_die_values(pkg, die); > + if (ret < 0) > + return -1; > + > + struct uncore_power_info *ui = &uncore_info[pkg][die]; > + > + return set_uncore_freq_internal(&(uncore_info[pkg][die]), ui->nb_freqs - 1); > +} > + > +int > +rte_power_uncore_get_num_freqs(unsigned int pkg, unsigned int die) > +{ > + int ret = check_pkg_die_values(pkg, die); > + if (ret < 0) > + return -1; > + > + return uncore_info[pkg][die].nb_freqs; > +} > + > +unsigned int > +rte_power_uncore_get_num_pkgs(void) > +{ > + DIR *d; > + struct dirent *dir; > + unsigned int count = 0; > + char filter[FILTER_LENGTH]; > + > + d = opendir(UNCORE_FREQUENCY_DIR); > + if (d == NULL) { > + RTE_LOG(ERR, POWER, > + "Uncore frequency management only supported on x86 with linux kernel >= 5.6."); > + return 0; > + } Suggest re-wording this message. I'm running a 5.17 kernel, and still see this message because CONFIG_INTEL_UNCORE_FREQ_CONTROL is not set. Maybe "Uncore frequency management not supported/enabled on this kernel. Please enable CONFIG_INTEL_UNCORE_FREQ_CONTROL." Also, the '\n' is missing from the end of the line. > + > + /* search by incrementing file name for max pkg file value */ > + while ((dir = readdir(d)) != NULL) { > + snprintf(filter, FILTER_LENGTH, PACKAGE_FILTER, count); > + /* make sure filter string is in file name (don't include hidden files) */ > + if (fnmatch(filter, dir->d_name, 0) == 0) > + count++; > + } > + > + closedir(d); > + > + return count; > +} > + > +unsigned int > +rte_power_uncore_get_num_dies(unsigned int pkg) > +{ > + DIR *d; > + struct dirent *dir; > + unsigned int count = 0, max_pkgs; > + char filter[FILTER_LENGTH]; > + > + max_pkgs = rte_power_uncore_get_num_pkgs(); > + if (max_pkgs == 0) > + return 0; > + if (pkg >= max_pkgs) { > + RTE_LOG(DEBUG, POWER, "Invalid package number\n"); > + return 0; > + } > + > + d = opendir(UNCORE_FREQUENCY_DIR); > + if (d == NULL) { > + RTE_LOG(ERR, POWER, > + "Uncore frequency management only supported on x86 with linux kernel >= 5.6."); > + return 0; > + } Suggest re-wording this message. I'm running a 5.17 kernel, and still see this message because CONFIG_INTEL_UNCORE_FREQ_CONTROL is not set. Maybe "Uncore frequency management not supported/enabled on this kernel. Please enable CONFIG_INTEL_UNCORE_FREQ_CONTROL." Also, the '\n' is missing from the end of the line. > + > + /* search by incrementing file name for max die file value */ > + while ((dir = readdir(d)) != NULL) { > + snprintf(filter, FILTER_LENGTH, DIE_FILTER, pkg, count); > + /* make sure filter string is in file name (don't include hidden files) */ > + if (fnmatch(filter, dir->d_name, 0) == 0) > + count++; > + } > + > + closedir(d); > + > + return count; > +} > diff --git a/lib/power/rte_power_uncore.h b/lib/power/rte_power_uncore.h > new file mode 100644 > index 0000000000..2be6546f49 > --- /dev/null > +++ b/lib/power/rte_power_uncore.h > @@ -0,0 +1,194 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(c) 2022 Intel Corporation > + */ > + > +#ifndef _RTE_POWER_UNCORE_H > +#define _RTE_POWER_UNCORE_H > + > +/** > + * @file > + * RTE Uncore Frequency Management > + */ > + > +#include "rte_power.h" > + > +#ifdef __cplusplus > +extern "C" { > +#endif > + > +/** > + * Initialize uncore frequency management for specific die on a package. It will get the available > + * frequencies and prepare to set new die frequencies. > + * > + * This function should NOT be called in the fast path. > + * > + * @param pkg > + * Package number. > + * @param die > + * Die number. > + * > + * @return > + * - 0 on success. > + * - Negative on error. > + */ > +__rte_experimental > +int > +rte_power_uncore_init(unsigned int pkg, unsigned int die); > + > +/** > + * Exit uncore frequency management on a specific die on a package. It will restore uncore min and > + * max values to previous values before initialization of API. > + * > + * This function should NOT be called in the fast path. > + * > + * @param pkg > + * Package number. > + * @param die > + * Die number. > + * > + * @return > + * - 0 on success. > + * - Negative on error. > + */ > +__rte_experimental > +int > +rte_power_uncore_exit(unsigned int pkg, unsigned int die); > + > +/** > + * Return the current index of available frequencies of a specific die on a package. > + * It should be protected outside of this function for threadsafe. > + * > + * This function should NOT be called in the fast path. > + * > + * @param pkg > + * Package number. > + * @param die > + * Die number. > + * > + * @return > + * The current index of available frequencies. > + * If error, it will return 'RTE_POWER_INVALID_FREQ_INDEX = (~0)'. > + */ > +__rte_experimental > +uint32_t > +rte_power_get_uncore_freq(unsigned int pkg, unsigned int die); > + > +/** > + * Set the new frequency for a specific die on a package by indicating the index of > + * available frequencies. > + * It should be protected outside of this function for threadsafe. > + * > + * This function should NOT be called in the fast path. > + * > + * @param pkg > + * Package number. > + * @param die > + * Die number. > + * @param index > + * The index of available frequencies. > + * > + * @return > + * - 1 on success with frequency changed. > + * - 0 on success without frequency changed. > + * - Negative on error. > + */ > +__rte_experimental > +int > +rte_power_set_uncore_freq(unsigned int pkg, unsigned int die, uint32_t index); > + > +/** > + * Scale up the frequency of a specific die on a package to the highest according to the > + * available frequencies. > + * It should be protected outside of this function for threadsafe. > + * > + * This function should NOT be called in the fast path. > + * > + * @param pkg > + * Package number. > + * @param die > + * Die number. > + * > + * @return > + * - 1 on success with frequency changed. > + * - 0 on success without frequency changed. > + * - Negative on error. > + */ > +__rte_experimental > +int > +rte_power_uncore_freq_max(unsigned int pkg, unsigned int die); > + > +/** > + * Scale down the frequency of a specific die on a package to the lowest according to the > + * available frequencies. > + * It should be protected outside of this function for threadsafe. > + * > + * This function should NOT be called in the fast path. > + * > + * @param pkg > + * Package number. > + * @param die > + * Die number. > + * > + * @return > + * - 1 on success with frequency changed. > + * - 0 on success without frequency changed. > + * - Negative on error. > + */ > +__rte_experimental > +int > +rte_power_uncore_freq_min(unsigned int pkg, unsigned int die); > + > +/** > + * Return the list length of available frequencies in the index array. > + * > + * This function should NOT be called in the fast path. > + * > + * @param pkg > + * Package number. > + * @param die > + * Die number. > + * > + * @return > + * - The number of available index's in frequency array. > + * - Negative on error. > + */ > +__rte_experimental > +int > +rte_power_uncore_get_num_freqs(unsigned int pkg, unsigned int die); > + > +/** > + * Return the number of packages (CPUs) on a system by parsing the uncore > + * sysfs directory. > + * > + * This function should NOT be called in the fast path. > + * > + * @return > + * - Zero on error. > + * - Number of package on system on success. > + */ > +__rte_experimental > +unsigned int > +rte_power_uncore_get_num_pkgs(void); > + > +/** > + * Return the number of dies for pakckages (CPUs) specified from parsing > + * the uncore sysfs directory. > + * > + * This function should NOT be called in the fast path. > + * > + * @param pkg > + * Package number. > + * > + * @return > + * - Zero on error. > + * - Number of dies for package on sucecss. > + */ > +__rte_experimental > +unsigned int > +rte_power_uncore_get_num_dies(unsigned int pkg); > + > +#ifdef __cplusplus > +} > +#endif > + > +#endif > diff --git a/lib/power/version.map b/lib/power/version.map > index f9b2947adf..8fccbf20f7 100644 > --- a/lib/power/version.map > +++ b/lib/power/version.map > @@ -48,4 +48,15 @@ EXPERIMENTAL { > rte_power_pmd_mgmt_set_pause_duration; > rte_power_pmd_mgmt_set_scaling_freq_max; > rte_power_pmd_mgmt_set_scaling_freq_min; > + > + # added in 22.11 > + rte_power_get_uncore_freq; > + rte_power_set_uncore_freq; > + rte_power_uncore_exit; > + rte_power_uncore_freq_max; > + rte_power_uncore_freq_min; > + rte_power_uncore_get_num_dies; > + rte_power_uncore_get_num_freqs; > + rte_power_uncore_get_num_pkgs; > + rte_power_uncore_init; > }; Once the above requested changes are addressed: Reviewed-by: David Hunt