From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 70A59430DE; Wed, 23 Aug 2023 11:19:50 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 46A86410FB; Wed, 23 Aug 2023 11:19:50 +0200 (CEST) Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2050.outbound.protection.outlook.com [40.107.243.50]) by mails.dpdk.org (Postfix) with ESMTP id 598B7410F1 for ; Wed, 23 Aug 2023 11:19:48 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=X9TMy+v5/vK0rYlDuP5N7PTtszqrGy1vRpswZsoq1oR9bk5gqkQA0IzEChlHhw0u6NOmshUUvW94uH9uYKrkRalAJVHTFv5YJbUHmGB+aChyVE8Qc4OutNbVet6ouSYJtt95coLGR0tMCciuxgzd9GLqehw6ci0oF4UF9EveBKf3NtfJWlAe3lDsEpkTpsWSXkBQqPCnulYfqsv3MW6znZnXjoLwcgas8+ugdTg5gChz1w3ORSWZYTTI4YyFlI6N06w934g832BUm9kuNrQQvMeTd2KCAHR/wqAZOJS2674voateYoRhdcHR/pdrabBcMK2929DTd+fym9arnejIGw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=r0dVl5+nqv74DsRQ1Hls0nqQq7s6s3oNYmNTxpdfsnA=; b=D14a0qyXuXhJkE2YdgLhvgKRu2A4SF3S49SlD2u5b0EIPIQc3FJ0FaSQY1HTdnzO/BczkZa3isT/W5MF0I+bAzKQQ34JUbyjW6cfK3NtKb6bw4boFAUDAOdb9XWhtlcoBji7G6aG7tpLBsI3xG8CA3Q2KpxuKA31qGNsu7pX5ddU2jc2qUQ9uU5jIWLwbDXKvlVS5zqyrYh9T1B7xp6T2zF7MrhDpf+YYG1M32hesmhRvFBWA9aczR6M/fGC8KbGmK7NDg2I4s/GtgFdKNlbPe4dk75vHexOzl4qcx1WKBf5ItzPegGtpm5F8/e70IaWAm5JKxEpTdndUxcgLn6/NA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=r0dVl5+nqv74DsRQ1Hls0nqQq7s6s3oNYmNTxpdfsnA=; b=m/fwAfPukxn7Xj4KvvphmES3K+RPTkVijngVG+sB8isQ8Ouefvjf3s9BPxJbndkpR6jpEH0IZBMrnlOhqY46AJaGg/lwxnXnS+KrGNNEyGABI0T9X1OPTMg5cLqQciT+dtth0targ2iupmoA9ygEDmGLDnC6QmUz/wWKvk9z+AY= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=amd.com; Received: from CH2PR12MB4294.namprd12.prod.outlook.com (2603:10b6:610:a9::11) by IA1PR12MB8309.namprd12.prod.outlook.com (2603:10b6:208:3fe::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6699.24; Wed, 23 Aug 2023 09:19:45 +0000 Received: from CH2PR12MB4294.namprd12.prod.outlook.com ([fe80::49e9:2bf6:7f06:bbbd]) by CH2PR12MB4294.namprd12.prod.outlook.com ([fe80::49e9:2bf6:7f06:bbbd%3]) with mapi id 15.20.6699.026; Wed, 23 Aug 2023 09:19:45 +0000 Message-ID: <35925f36-ccde-f632-8df2-f7d20e0f95cb@amd.com> Date: Wed, 23 Aug 2023 10:19:39 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.14.0 Content-Language: en-US To: Konstantin Ananyev , Bruce Richardson Cc: Konstantin Ananyev , "Tummala, Sivaprasad" , Tyler Retzlaff , "david.hunt@intel.com" , "anatoly.burakov@intel.com" , "david.marchand@redhat.com" , "thomas@monjalon.net" , "dev@dpdk.org" References: <20230418082529.544777-2-sivaprasad.tummala@amd.com> <20230816185959.1331336-1-sivaprasad.tummala@amd.com> <20230816185959.1331336-3-sivaprasad.tummala@amd.com> <20230816192758.GA12453@linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net> <92e7bd2e-b799-9658-c90e-f50638c6fdbd@amd.com> From: Ferruh Yigit Subject: Re: [PATCH v5 3/3] power: amd power monitor support In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-ClientProxiedBy: LO4P123CA0641.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:296::7) To CH2PR12MB4294.namprd12.prod.outlook.com (2603:10b6:610:a9::11) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR12MB4294:EE_|IA1PR12MB8309:EE_ X-MS-Office365-Filtering-Correlation-Id: 20e49caf-9d88-4fcf-cad1-08dba3ba1756 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: e7+wgxQQ8tefeIPeh3+3+jeXs9CNBFP2Eo0MCSOp0yfrSBvfydeMAdmR87rZbeVOsaVbkKWqBr5sq+HKlxDeiePJnMhgDdPzCiuuFf6Y/FPT9cMvX65IHzL8hnTNYjp7dBobBqukWLI7ls/jES2RR432b/qU0EcTDtLO2zdMJvtameEkgx/ZeO4/EybDmjGJKRpmWD2fHeBuZoWQk5UHSdkL3wKXkUdWOwsUYC3rxb++/qSJ2DFcw2xvHHeYj2S4WVDogLSE2YSJavyh3/uWUgvDVOLYbHbWAnetuEQCQwbcbIcpu/flyd7e6RomsDPeThEUE2jHCqjXl5444QDWnTiWksiAOYuZYhmAjZ1B4N7DGypwG7oVOrOywQhKjLFzLqVFVQDSeNTa2bp5c7fFWgmCUTFxEu0EuXvGTdUgS2ngPwPaezO8vIYW4LtaS51/xJcfF1AE9HcnKdZR63fZpr/Nl592qoBvtyITyLA0aUTUuBt+T6l1QpKdf5tfE2jn3m9JO8scAMFsuRiFxqPdjJp2Gqw6bQZMWpriCDRPr4z9zN0qa9isbCPqspzPpLkb2ZBDnmc+xdHIK9B+nQUh2zIfcjpn/k+8pW8a5VUllYaEorOGqfZABfdADbnRbngUv5lsWz14qfDzpgpLOo8Bcg== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR12MB4294.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(396003)(136003)(39860400002)(376002)(346002)(366004)(1800799009)(186009)(451199024)(66556008)(54906003)(6512007)(66946007)(66476007)(316002)(110136005)(8676002)(8936002)(2616005)(4326008)(36756003)(41300700001)(478600001)(6666004)(6486002)(53546011)(6506007)(38100700002)(2906002)(83380400001)(86362001)(31686004)(31696002)(44832011)(5660300002)(26005)(45980500001)(43740500002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ZDFUTndJczF4ZTI1cjVTTk9RMkRIMVQ0cllGY2hxdjVEMkZCd01uTUs2MG5L?= =?utf-8?B?TTNtTmZqMFk2SG5mWWpmL1RhTlM1aFVtVzUrOHR6Vkp0NXoybi92eVhYU2hz?= =?utf-8?B?VDFvS21lWlh4QmF4aWlVbWFDMFlrZEE5TkVPNks1SzdPMVVsYkt3L2Z5NnZz?= =?utf-8?B?dXI3czBrZ1B3R1dHbzdtcFo4SkZaUWVyS0tlVnJ5ZmcrKzZQWFVzbWVYOGov?= =?utf-8?B?SmVKRjFwblAvQWNydm5MNS9jeURmZFFXRnB0NUpUWUx6bU1xbDEvRFEzWGhW?= =?utf-8?B?NjJDa0VSYlNqNXpuTXRYaHhVRnpPamNua0N6U0YwZ3pJUW1Mbk1GWnlOZkFG?= =?utf-8?B?Vm5FcjE3UXRwdU1CeDMwci8zeUVITFlzVDY2RE5wbVRqc3J5YW9OSlkrTXNF?= =?utf-8?B?dzVsVCtveTJUNGMyMGdjbGdwSEV6em96MnRHR2tKTkFtZzVEN3p4dlZLY3BF?= =?utf-8?B?THVQZWFKandWN3Q3OUtzZEFHZGpuS3hHcHcyNjM4Q3JYQWVybDVzTE1HUWNK?= =?utf-8?B?OTNZaDRlbnovaEV4N1FXT1ovdHpOZW90cjBURXdTMmxLbC8yeFNZdmswajVt?= =?utf-8?B?dUVEQk4rVUE5T3FlK2FTLzVlenpqem4yOC9RekQybjYrUmZhVFZ2ckFnN0pK?= =?utf-8?B?SjVIb0xpVzNWK3k3WDFGQVo2Q0VmU2RaQ0hRRGcwb3ViN0NKcGIwOGJaNVho?= =?utf-8?B?a3NvemlZWTJLWjVlVUVLSnQzYlZIWlFqRjlKa0RyK2RibTNOazEvRE9YZlZY?= =?utf-8?B?Uk53WisvdGZ0NXYrLzB3bnRtVXo3eHF4R3RzVzhKaWxaZ2NjVHdwSVRzS1o5?= =?utf-8?B?cmVCVE1oWFFzTTN5ZnQxMEJhYWFXNFVEb0FVMy95K1ZVeUg0MStZU2VsM3JT?= =?utf-8?B?U1FPQVNjeElMWEFNZWhpQlVSV0NxYlpucVNPaFBQUmZLY3JLT2VOV1QzMXZa?= =?utf-8?B?RmczNVJUUVJZUmg1SWZ0MVRnZjJtejFJMCsrMk54aTd1L2xYTGE0eDg5b0sv?= =?utf-8?B?NHlzYnB2eW9JakY3UlVpdjV6dWU1VHduQVZRWUtIY25IYUl6Q3ByOXBhLzNZ?= =?utf-8?B?WFZDT3puQWdEVnhMYklZdEFCWVBqMjRBcndPUFpoTG9PaHlVY2RkeVBFSDlO?= =?utf-8?B?cittOCt5cDY2eDVCcUxUSDVDNEI5YnZYSEprQlhTK2o5NUdib2RieTJmZFdu?= =?utf-8?B?LzZJR3dtbFBqVlZTNFVCYjluN2tOMDRlTmdXUGRZNmN3VWgxOU9wNTFpc1dH?= =?utf-8?B?eDZmdmtYSlIxK1htUXVzUnNkcG5MMVk4UXZNNEJ6ZmJkZG1qQmg2czk1YUVj?= =?utf-8?B?Q2Z2MFQ5Mm5NQmpadEJrQ2ZNY29kYytndnc4UHd1TE1MU3lZN0U3UUZTMDJn?= =?utf-8?B?NFpWQ29WMGNrVzBMVmpsa1JmWC9pdU0yQ0N1Ky9DK3BrcFBRaEpJOU8zK2pn?= =?utf-8?B?emxacnB3MUovOVFxajcyMUNkV1h5WWIzcm42N2pScWozSSs1L2RqN2w1WEJr?= =?utf-8?B?bCsrY3dieWJ4NjRZa0d2cGlkZXc1SW5YMFVyOVAxakZpMFNwVmdsdC9Na1hW?= =?utf-8?B?RWNTckRXbHRwbmtQRmVrNDFlakJkZmxoanpYM3EyZEIyc3FRR1VBRTV1ekxi?= =?utf-8?B?R1htQ094aTRuV0pwd1F6dDI2ZFNyenJjcnlVWUhXOUI5ZUwwejMzUVhwdkxv?= =?utf-8?B?TXFtZDFhRlRpYnREWmY3c0N5dkVVeWZmbVZJdnJjc2J4aFF6KzBLU2Jzdmw2?= =?utf-8?B?L3RGUHZueitIbUUvd2Z3Mlg1UmNOcWdPdmN5aUZZQTdXelpaSTJSTEJGaFk0?= =?utf-8?B?a2w2YTBxYlNQWmw4dGVyMFVrWTA1VFlLSUdDajFMUFFjRUxlK3NGMDFqQWRH?= =?utf-8?B?ZndWcEtUQVZaUDBwcS9HTlZWQlg5WmxybUdJUFNyMlRXNHZXYmZWYVBneERM?= =?utf-8?B?c2crcmkzMlhlZTNzSjJJQUJ2cWsvVEE0RTJSREk0YkJRMW1YRDROcmlvdmRK?= =?utf-8?B?L1FnTEpBTVNUcjFmME9rU29wT1JUU05Zbi80T1dhb0x5NU12NWh0YlFFbnFG?= =?utf-8?B?MW1FWXVCazlBdGdwTHBTOHhyWTlvcmJVbU9pZnVtbXJwK2c2U2xNcnp1VEpO?= =?utf-8?Q?jPZ8Dfs8IscxOlz6Hk0BbInAS?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 20e49caf-9d88-4fcf-cad1-08dba3ba1756 X-MS-Exchange-CrossTenant-AuthSource: CH2PR12MB4294.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Aug 2023 09:19:45.6250 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: NN5Wpdx4GpE9RRUUbT56D/0lK9H+tFqRosU5QvPVX8PSQwFIzx67eBMFkYxCmttU X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB8309 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 8/22/2023 11:30 PM, Konstantin Ananyev wrote: > 18/08/2023 14:48, Bruce Richardson пишет: >> On Fri, Aug 18, 2023 at 02:25:14PM +0100, Ferruh Yigit wrote: >>> On 8/17/2023 3:18 PM, Konstantin Ananyev wrote: >>>> >>>>>> Caution: This message originated from an External Source. Use >>>>>> proper caution >>>>>> when opening attachments, clicking links, or responding. >>>>>> >>>>>> >>>>>> On Wed, Aug 16, 2023 at 11:59:59AM -0700, Sivaprasad Tummala wrote: >>>>>>> mwaitx allows EPYC processors to enter a implementation dependent >>>>>>> power/performance optimized state (C1 state) for a specific >>>>>>> period or >>>>>>> until a store to the monitored address range. >>>>>>> >>>>>>> Signed-off-by: Sivaprasad Tummala >>>>>>> Acked-by: Anatoly Burakov >>>>>>> --- >>>>>>>   lib/eal/x86/rte_power_intrinsics.c | 77 >>>>>>> +++++++++++++++++++++++++----- >>>>>>>   1 file changed, 66 insertions(+), 11 deletions(-) >>>>>>> >>>>>>> diff --git a/lib/eal/x86/rte_power_intrinsics.c >>>>>>> b/lib/eal/x86/rte_power_intrinsics.c >>>>>>> index 6eb9e50807..b4754e17da 100644 >>>>>>> --- a/lib/eal/x86/rte_power_intrinsics.c >>>>>>> +++ b/lib/eal/x86/rte_power_intrinsics.c >>>>>>> @@ -17,6 +17,60 @@ static struct power_wait_status { >>>>>>>        volatile void *monitor_addr; /**< NULL if not currently >>>>>>> sleeping >>>>>>> */  } __rte_cache_aligned wait_status[RTE_MAX_LCORE]; >>>>>>> >>>>>>> +/** >>>>>>> + * These functions uses UMONITOR/UMWAIT instructions and will >>>>>>> enter C0.2 >>>>>> state. >>>>>>> + * For more information about usage of these instructions, please >>>>>>> +refer to >>>>>>> + * Intel(R) 64 and IA-32 Architectures Software Developer's Manual. >>>>>>> + */ >>>>>>> +static void intel_umonitor(volatile void *addr) { >>>>>>> +     /* UMONITOR */ >>>>>>> +     asm volatile(".byte 0xf3, 0x0f, 0xae, 0xf7;" >>>>>>> +                     : >>>>>>> +                     : "D"(addr)); >>>>>>> +} >>>>>>> + >>>>>>> +static void intel_umwait(const uint64_t timeout) { >>>>>>> +     const uint32_t tsc_l = (uint32_t)timeout; >>>>>>> +     const uint32_t tsc_h = (uint32_t)(timeout >> 32); >>>>>>> +     /* UMWAIT */ >>>>>>> +     asm volatile(".byte 0xf2, 0x0f, 0xae, 0xf7;" >>>>>>> +                     : /* ignore rflags */ >>>>>>> +                     : "D"(0), /* enter C0.2 */ >>>>>>> +                     "a"(tsc_l), "d"(tsc_h)); } >>>>>> >>>>>> question and perhaps Anatoly Burakov can chime in with expertise. >>>>>> >>>>>> gcc/clang have built-in intrinsics for umonitor and umwait i >>>>>> believe as per our other >>>>>> thread of discussion is there a benefit to also providing inline >>>>>> assembly over just >>>>>> using the intrinsics? I understand that the intrinsics may not >>>>>> exist for the monitorx >>>>>> and mwaitx below so it is probably necessary for amd. >>>>>> >>>>>> so the suggestion here is when they are available just use the >>>>>> intrinsics. >>>>>> >>>>>> thanks >>>>>> >>>>> The gcc built-in functions >>>>> __builtin_ia32_monitorx()/__builtin_ia32_mwaitx are available only >>>>> when -mmwaitx >>>>> is used specific for AMD platforms. On generic builds, these >>>>> built-ins are not available and hence inline >>>>> assembly is required here. >>>> >>>> Ok... but we can probably put them into a separate .c file that will >>>> be compiled with that specific flag? >>>> Same thing can be probably done for Intel specific instructions. >>>> In general, I think it is much more preferable to use built-ins vs >>>> inline assembly >>>> (if possible off-course). >>>> >>> >>> We don't compile different set of files for AMD and Intel, but there are >>> runtime checks, so putting into separate file is not much different. > > Well, we probably don't compile .c files for particular vendor, but we > definitely do compile some .c files for particular ISA extensions. > Let say there are files in lib/acl that requires various '-mavx512*' > flags, same for other libs and PMDs. > So still not clear to me why same approach can't be applied to > power_instrincts.c? > >>> >>> It may be an option to always enable compiler flag (-mmwaitx), I think >>> it won't hurt other platforms but I am not sure about implications of >>> this to other platforms (what was the motivation for the compiler guys >>> to enable these build-ins with specific flag?). >>> >>> Also this requires detecting compiler that supports 'mmwaitx' or not, >>> etc.. >>> >> This is the biggest reason why we have in the past added support for >> these >> instructions via asm bytes rather than intrinsics. It takes a long >> time for >> end-user compilers, especially those in LTS releases, to get the >> necessary >> intrinsics. > > Yep, understand. > But why then we can't have both implementations? > Let say if WAITPKG is defined we can use builtins for > umonitor/umwait/tpause, otherwise we fallback to inline asm implementation. > Same story for MWAITX/monitorx. > Yes this can be done, it can be done either as different .c files per implementation, or as #ifdef in same file. But eventually asm implementation is required, as fallback, and if we will rely on asm implementation anyway, does it worth to have the additional checks to be able to use built-in intrinsic? Does it helps to comment name of the built-in function to inline assembly code, to document intention and another possible implementation? >> Consider a user running e.g. RHEL 8, who wants to take >> advantages of the latest DPDK features; they should not be required to >> upgrade their compiler - and possibly binutils/assembler - to do so. >> >> /Bruce >