From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DA0054309C; Fri, 18 Aug 2023 15:49:02 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id AC95040ED9; Fri, 18 Aug 2023 15:49:02 +0200 (CEST) Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.43]) by mails.dpdk.org (Postfix) with ESMTP id 8547640395 for ; Fri, 18 Aug 2023 15:49:00 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1692366540; x=1723902540; h=date:from:to:cc:subject:message-id:references: in-reply-to:mime-version; bh=2jhfFoYKpyQjqI212V5W+40HA86TsmihvzokQOPHnjk=; b=H9ULkm2WQm8Vx6DGs458z9MkSSGbXk/6apX5b4iXkJdwYpJAKq9p4ssc cHGRDiz/Wd87ur8TEkskabsFdKWFTC4z1HE0ebifBB6R0gv0WZZ53Gf4p 2d+RtE2ZGY/KKoJwp0+S7MDc5eK9OEgvwVqL9G18W+SmaZ714fszimW8I Tu6m9InSICb3mh/Z+O51flHWU5VQUIh/IzPpHF3A8bqvNWWs9sjxLLM43 rLoAsB8IQ6qS49JNuhuKPkEW59lITt3hLHVAMHhticA1IiOhRliqM2pau QkE7qYoZFyZWvs1JVILKJY3p6cg9SapXBSdc98H8D441g0rTymVPJ9aYl Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10806"; a="459456631" X-IronPort-AV: E=Sophos;i="6.01,183,1684825200"; d="scan'208";a="459456631" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Aug 2023 06:48:59 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10806"; a="728612104" X-IronPort-AV: E=Sophos;i="6.01,183,1684825200"; d="scan'208";a="728612104" Received: from fmsmsx603.amr.corp.intel.com ([10.18.126.83]) by orsmga007.jf.intel.com with ESMTP; 18 Aug 2023 06:48:58 -0700 Received: from fmsmsx611.amr.corp.intel.com (10.18.126.91) by fmsmsx603.amr.corp.intel.com (10.18.126.83) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Fri, 18 Aug 2023 06:48:57 -0700 Received: from fmsmsx612.amr.corp.intel.com (10.18.126.92) by fmsmsx611.amr.corp.intel.com (10.18.126.91) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Fri, 18 Aug 2023 06:48:57 -0700 Received: from fmsedg601.ED.cps.intel.com (10.1.192.135) by fmsmsx612.amr.corp.intel.com (10.18.126.92) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27 via Frontend Transport; Fri, 18 Aug 2023 06:48:57 -0700 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (104.47.55.108) by edgegateway.intel.com (192.55.55.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.27; Fri, 18 Aug 2023 06:48:56 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=c+9BCBocTRwORSVGrhZFZqJFLDeaanLPum0Mi4DoitBimk3w6sHyqzFFzv7VwbPUbmr4IgT9G0JtZoWnEJhAstjD8JZ33QPthFfJIPQsdNDDWWyc6vwwjiWzNZqMJI/U2gv5BjhuiWnfwXdUfDlaES3FvL+b6r3O/YMzJxPAnWfgaGrHSI9/nIfmGyIz6Fu3WMLAbNSd03EJYfWsl42m/La8Kk4IzSp0kRBmI4y/7Gie2mx4utLOkp/iRVH7CFpzvx47lfITHh07owQ75uI+hLb6M3njGf95xNpSCgZMn+VMhd5app3xQHUbOjWzs7WrwBNZoJo3IrScl2eRZbxbrA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=iDbehLQbrt8bP85sT4TOBJUDMQiLOYfmQKAyLDetIFg=; b=EtYGrHwASkI8GmgfbZ+kyg8AXt/nfh6MBNzxBwgs0y11Qn1Ream3MlASpF9SX14hIP+7Uq0lQDZ7s8dA7QVc+7OYq1DFFNd2VTVQRg0UmofQTOHp+Sx1Q9OSe8ZdwsNYyxqO1DjHzZKm69OFF32uIa5PAu/miLHCxOezUvd/DmepNQbJkLTIUh5qxbIlTpMWg5NgLXOkRAikCe6U3aMh99pjo/h8SH+yq77N8WVUFDaC/wrGJbjcI2MmiCx3Sbsd92N/STrMz8G+O7gX+RQnsP1Dsqr3Ep1GFigFE71KVE+9HpsJbhaRxlg6ot1s1smGcur/3LtLubrAJQdtULmzGg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from DS0PR11MB7309.namprd11.prod.outlook.com (2603:10b6:8:13e::17) by IA0PR11MB7695.namprd11.prod.outlook.com (2603:10b6:208:400::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6678.26; Fri, 18 Aug 2023 13:48:49 +0000 Received: from DS0PR11MB7309.namprd11.prod.outlook.com ([fe80::9802:65f0:c441:780f]) by DS0PR11MB7309.namprd11.prod.outlook.com ([fe80::9802:65f0:c441:780f%7]) with mapi id 15.20.6678.031; Fri, 18 Aug 2023 13:48:49 +0000 Date: Fri, 18 Aug 2023 14:48:42 +0100 From: Bruce Richardson To: Ferruh Yigit CC: Konstantin Ananyev , "Tummala, Sivaprasad" , Tyler Retzlaff , "david.hunt@intel.com" , "anatoly.burakov@intel.com" , "david.marchand@redhat.com" , "thomas@monjalon.net" , "dev@dpdk.org" Subject: Re: [PATCH v5 3/3] power: amd power monitor support Message-ID: References: <20230418082529.544777-2-sivaprasad.tummala@amd.com> <20230816185959.1331336-1-sivaprasad.tummala@amd.com> <20230816185959.1331336-3-sivaprasad.tummala@amd.com> <20230816192758.GA12453@linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net> <92e7bd2e-b799-9658-c90e-f50638c6fdbd@amd.com> Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: <92e7bd2e-b799-9658-c90e-f50638c6fdbd@amd.com> X-ClientProxiedBy: DU2PR04CA0353.eurprd04.prod.outlook.com (2603:10a6:10:2b4::13) To DS0PR11MB7309.namprd11.prod.outlook.com (2603:10b6:8:13e::17) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR11MB7309:EE_|IA0PR11MB7695:EE_ X-MS-Office365-Filtering-Correlation-Id: ada8e1fc-ff05-490e-0dc7-08db9ff1d9e1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: P1iqetIObvse7ISJ4x8TkiXKeP1h/5vYnuM0EdBhyT+EjJd8jmH+xs7u59R5Gk0NRCfxP91fWm3Q1xswVIyfoTvTvbPi2pCJ3jKe/xxFYw5V8flvweP40uw+dVX6Wul0nCFBPcCRjUNtpW/AlfaMiO6wTHVdKgQ54Ki9YeMd8/024nEEDT0vShxhx/pSLKxJ4KyKw8LcXXLEU4JHDHUG9/xxSPLDs21AforoS6yUfEY8xJJnau1WLQOS4x/aYENLzFwH537B7VMuqhWH9asAIfp2b7cRPaSlsL0LzDtkxNAKBSSBFUi1F6d5ag0xPoPFsFcEIcgNHqg3RbbttL7V4ETfqq06Rqi/fmBuhjUtFc02lOfQxvtcnk7j3pYN1mimUKh7+qxbGnPAW4M1eVMoBPuFZ96h8oFU/XwnbCPTdUQbfWJo54kraL0rO7KJK+9jolTpKjSOBHKY/Y01Bt+914VuSqBw9oHl3UQQP0m9y2yKhrDbN0r54dCWtL+7n2uuJL3RmXc3J/fwLc7zoptcb+w0VqHnsYwoA2C8lo/etsLkBXnS+X2gUw/qImNFWCy8 X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DS0PR11MB7309.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(366004)(136003)(376002)(346002)(39860400002)(396003)(186009)(451199024)(1800799009)(2906002)(83380400001)(26005)(86362001)(478600001)(6506007)(6666004)(6486002)(6512007)(53546011)(44832011)(5660300002)(41300700001)(54906003)(66476007)(66946007)(66556008)(316002)(6916009)(4326008)(8936002)(8676002)(82960400001)(38100700002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?YQ7F2W8aRirSX3GGpsPGSNBWxCuIDzJnJCXBbkGrFWopG6nLNqk60CeEkws3?= =?us-ascii?Q?DPykjoxxMn47ro15/keL1VtMOXPvRxvU6tFERtF3h3A7mC3hdUjFHruNdEXP?= =?us-ascii?Q?OvOuD1OdyeMt1EYPJ35Iu6uQPyzUI6P7vWhYALtLVwa0X/2Ocx1z4+Zn9jVT?= =?us-ascii?Q?PGDYPpvSjxo3UlIUbctm0soA1TCC7aLRt0RQeUM//wrOorUMKR4mIyVc7+Dw?= =?us-ascii?Q?+nucY+E/b/vznrBzIaH0v+xZFahtqxCN+T2ERRE/wcDb7GpWifqH9JB0+ePW?= =?us-ascii?Q?Qb0eDwoPG9j69gV1gI0JXzI7D/beW/xHpOyrUmtN9gJAgc6GVeAMnYKyPXqZ?= =?us-ascii?Q?bLe5wYocDjsR4JAOJ8uP8YVKQOnj81Wx0ubnvwGtfjECvzY9Zo2Bq3J5sDbr?= =?us-ascii?Q?Vbi+KjhM8yEPj5gCHNWuj/0hl298/oe19EQbiYkLU4WK09r4uggCwUzN3Jrt?= =?us-ascii?Q?z+V23TB7+4XG/bSKdkTff8rQ1Q3omJv/Fv6B5rHPPa7OHqT5mNYSYKSjVFYT?= =?us-ascii?Q?KMevWt+HdPY/11XNFrBDFcdgk1/bZOlq11PtcVu7AjP5FRSWJ8i69PanHy75?= =?us-ascii?Q?15DYvxv/RooCxC5OwoFDEuCV1p5AbPvFQVi1UsD9QZQErNfw+6Jrxgssh88m?= =?us-ascii?Q?t6KFkd4xSqp9JCYUFonMUA9B8SoGa055YrFDXIhxFIOQ/pj08dIP/yYHA//U?= =?us-ascii?Q?q49EPZOd3q+LPK3dVO5NDFw+p8NWAxFH5eEB/QCrSUBZpCrIQGk82s8hBDfS?= =?us-ascii?Q?IRqedWpfIwTD3nUVVBW4QNoh6hoBdffJEb9NM5/YH4xrQ5Q+z6RxbiOfIolK?= =?us-ascii?Q?llJVgb4p2BqFA/TUaZoNi6SI2U4D5Rm7eA9Bh3g0ayUJRLozPeVdhovmh5XP?= =?us-ascii?Q?/+fci6GDqn+HaxOt6Ennk0Uo3EjLmT5eiv4csAmq+Q7vIcaA8OzDNjGO/KeS?= =?us-ascii?Q?4sYFH68sdtd6EG/WbWWvlsVNlBXPXWgsSLL6M9MDHCMpq3jVSpWalcArkbID?= =?us-ascii?Q?gT5Ppn6eWtxohwkr1C6cabdP/UQkn2EeYYtBrf+8w/7Cp+D+IQ96YLj6354I?= =?us-ascii?Q?d5yxAuGzb+GsDdFILK4ydx0q3CRRBKUuf7BC/eVDafNhPnRsO9yJ6aZcQYw2?= =?us-ascii?Q?O+LKlnWuJ4M8qQReL7LwmpEKEIyIrexdem+cCTfLpxkXPND3FLRs4wVzgiL9?= =?us-ascii?Q?vl5G4XzVioQ94C/NeAY2G4Bfi45qGgrlgu7DVPZsCpuTg/NBvgKsiQAUeCjy?= =?us-ascii?Q?BkorFu9L/ERRVckAddf5mZ9epRt8v0F0+gfCATT6kpIQpZr+74FDwTIutWwG?= =?us-ascii?Q?GTAjTq8Fdt6rgKvFjIqbn/CF8eEPJ+/lXGjSbe5wLzur86MfM8h+3WW8YPhk?= =?us-ascii?Q?P7Z3oAgy8gq+krVzZW2KEFuKhj33jLzncrg2lqVYVwOimkkipQ8y0P7j7WlJ?= =?us-ascii?Q?vc1opP3OUOTRrk6+SWFUmSoOXW+5MeKMSN4tzqBf6ihYg5G1qWZFqw+EztZ5?= =?us-ascii?Q?BGV1H4/i4KJ9eeREnR/sE5iI34HyIR/TO9j7J0s/SI8XJrbA8pbiN/C6cL8r?= =?us-ascii?Q?y08a5mzeKB/swnoHNKi4DSFmwbDLwot9OLMK7IuaJARfC8jbhcXhE0UoFmka?= =?us-ascii?Q?Fg=3D=3D?= X-MS-Exchange-CrossTenant-Network-Message-Id: ada8e1fc-ff05-490e-0dc7-08db9ff1d9e1 X-MS-Exchange-CrossTenant-AuthSource: DS0PR11MB7309.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Aug 2023 13:48:49.6671 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: fsRL3m1lon2+RalZhQLfmIsmNp4lWW+Mt4wY4VJiBsTAXYJfnNza/ud/duXtjdYRGv0gIrr0mSIh2XRcbPb4xtOa7ctM+v4I6fP6UslRS/E= X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR11MB7695 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Fri, Aug 18, 2023 at 02:25:14PM +0100, Ferruh Yigit wrote: > On 8/17/2023 3:18 PM, Konstantin Ananyev wrote: > > > >>> Caution: This message originated from an External Source. Use proper caution > >>> when opening attachments, clicking links, or responding. > >>> > >>> > >>> On Wed, Aug 16, 2023 at 11:59:59AM -0700, Sivaprasad Tummala wrote: > >>>> mwaitx allows EPYC processors to enter a implementation dependent > >>>> power/performance optimized state (C1 state) for a specific period or > >>>> until a store to the monitored address range. > >>>> > >>>> Signed-off-by: Sivaprasad Tummala > >>>> Acked-by: Anatoly Burakov > >>>> --- > >>>> lib/eal/x86/rte_power_intrinsics.c | 77 > >>>> +++++++++++++++++++++++++----- > >>>> 1 file changed, 66 insertions(+), 11 deletions(-) > >>>> > >>>> diff --git a/lib/eal/x86/rte_power_intrinsics.c > >>>> b/lib/eal/x86/rte_power_intrinsics.c > >>>> index 6eb9e50807..b4754e17da 100644 > >>>> --- a/lib/eal/x86/rte_power_intrinsics.c > >>>> +++ b/lib/eal/x86/rte_power_intrinsics.c > >>>> @@ -17,6 +17,60 @@ static struct power_wait_status { > >>>> volatile void *monitor_addr; /**< NULL if not currently sleeping > >>>> */ } __rte_cache_aligned wait_status[RTE_MAX_LCORE]; > >>>> > >>>> +/** > >>>> + * These functions uses UMONITOR/UMWAIT instructions and will enter C0.2 > >>> state. > >>>> + * For more information about usage of these instructions, please > >>>> +refer to > >>>> + * Intel(R) 64 and IA-32 Architectures Software Developer's Manual. > >>>> + */ > >>>> +static void intel_umonitor(volatile void *addr) { > >>>> + /* UMONITOR */ > >>>> + asm volatile(".byte 0xf3, 0x0f, 0xae, 0xf7;" > >>>> + : > >>>> + : "D"(addr)); > >>>> +} > >>>> + > >>>> +static void intel_umwait(const uint64_t timeout) { > >>>> + const uint32_t tsc_l = (uint32_t)timeout; > >>>> + const uint32_t tsc_h = (uint32_t)(timeout >> 32); > >>>> + /* UMWAIT */ > >>>> + asm volatile(".byte 0xf2, 0x0f, 0xae, 0xf7;" > >>>> + : /* ignore rflags */ > >>>> + : "D"(0), /* enter C0.2 */ > >>>> + "a"(tsc_l), "d"(tsc_h)); } > >>> > >>> question and perhaps Anatoly Burakov can chime in with expertise. > >>> > >>> gcc/clang have built-in intrinsics for umonitor and umwait i believe as per our other > >>> thread of discussion is there a benefit to also providing inline assembly over just > >>> using the intrinsics? I understand that the intrinsics may not exist for the monitorx > >>> and mwaitx below so it is probably necessary for amd. > >>> > >>> so the suggestion here is when they are available just use the intrinsics. > >>> > >>> thanks > >>> > >> The gcc built-in functions __builtin_ia32_monitorx()/__builtin_ia32_mwaitx are available only when -mmwaitx > >> is used specific for AMD platforms. On generic builds, these built-ins are not available and hence inline > >> assembly is required here. > > > > Ok... but we can probably put them into a separate .c file that will be compiled with that specific flag? > > Same thing can be probably done for Intel specific instructions. > > In general, I think it is much more preferable to use built-ins vs inline assembly > > (if possible off-course). > > > > We don't compile different set of files for AMD and Intel, but there are > runtime checks, so putting into separate file is not much different. > > It may be an option to always enable compiler flag (-mmwaitx), I think > it won't hurt other platforms but I am not sure about implications of > this to other platforms (what was the motivation for the compiler guys > to enable these build-ins with specific flag?). > > Also this requires detecting compiler that supports 'mmwaitx' or not, etc.. > This is the biggest reason why we have in the past added support for these instructions via asm bytes rather than intrinsics. It takes a long time for end-user compilers, especially those in LTS releases, to get the necessary intrinsics. Consider a user running e.g. RHEL 8, who wants to take advantages of the latest DPDK features; they should not be required to upgrade their compiler - and possibly binutils/assembler - to do so. /Bruce