From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM02-CY1-obe.outbound.protection.outlook.com (mail-cys01nam02on0075.outbound.protection.outlook.com [104.47.37.75]) by dpdk.org (Postfix) with ESMTP id 03A9A6A10 for ; Fri, 19 Aug 2016 13:46:40 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=k1CosXpZoRnNcpnEKt533nEc3JHrvL34BVT8MymA3/o=; b=BGCVlwFTti8mxNsTxWAsvQOfmvlgl1zjQtWu/LHBpR/YXK/6P8YHLAq1yQDU6j1lkXipw5f2E7b/MpGyBVk9wG4PXR++B56peH4hveMdRdCxyPHr4+LbImzJS3ogzpkEbxHN44Z/QaVQAivGwTo64h9j0iuVLSrf0iP/DJExadY= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Jerin.Jacob@cavium.com; Received: from localhost.localdomain (122.171.54.218) by CY1PR0701MB1725.namprd07.prod.outlook.com (10.163.21.14) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384) id 15.1.557.21; Fri, 19 Aug 2016 11:46:33 +0000 Date: Fri, 19 Aug 2016 17:16:12 +0530 From: Jerin Jacob To: Nipun Gupta CC: "dev@dpdk.org" , "thomas.monjalon@6wind.com" , "jianbo.liu@linaro.org" , "viktorin@rehivetech.com" , Hemant Agrawal Message-ID: <20160819114611.GA5510@localhost.localdomain> References: <1471521090-21067-1-git-send-email-jerin.jacob@caviumnetworks.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.6.2 (2016-07-01) X-Originating-IP: [122.171.54.218] X-ClientProxiedBy: MAXPR01CA0048.INDPRD01.PROD.OUTLOOK.COM (10.164.146.148) To CY1PR0701MB1725.namprd07.prod.outlook.com (10.163.21.14) X-MS-Office365-Filtering-Correlation-Id: 64872bf1-20df-47e9-8e0d-08d3c8267972 X-Microsoft-Exchange-Diagnostics: 1; CY1PR0701MB1725; 2:ejR5PEMm9GtRGmCbnxEZe8L9+tv7nGTTDGofcvXEuICpZ2PuGPkxSWUl44lyvYJhePZeHozY7JP5Dpg8tulYoQl092juhYUIXjgf6TwKCy8+CaCjd5tz3sYPE+kqDR4csJRTxHaAdOVe7vzX+wRKqChN7yC2s0LTfkpG+afGu9WYZUP8EPmKOiZW31kPuenK; 3:AiB2WzItD/8j96fD2KaYvtEeBEnc7qFr3vwZhbJdFHr3rT3e+t2lwN/EFZJT7PsEPzbGVAiI0VLWC3ONDDt533PCyOGLi1uayC4jqDCo0uAVfJU26P5sp+cJLwkYxD2j X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:CY1PR0701MB1725; X-Microsoft-Exchange-Diagnostics: 1; CY1PR0701MB1725; 25:0faeijRr63jjTltF8l2hj50sfxnrC84BL8sON27fzb2FNnnDUqLxm9o1Z7xBrj21HsSiAkbiB46kuoMPaTdBeBqGUk2g07m6rpGLqdyTINkRqtZxJbakkD+8uh4ZDdVqEsQrppnjnFEjXoIWob31y0/DGtfWuwPzx1W9QASJU6ZHJ9XMKY99c+huwgOQ8G3sDIJNvdC0IVB8T2xSlb56tcqkaPreEYIq/RWxNqPh5Q2E83J+68cL6gbX0MSxNXtyxlkV+yjgfO9mGDxjO20/DxehZEpiXJwy9xASAsnF+KS8krNS2n107EbvJ0GQLwuR1Pt0DtZxgNblXtc2S2QVJQjRjFRuDcx0aRY/Qbr5XpE7GMZii8qdyRYogw4mBSWC55D43/N5JkGJex6qisMK5myGhcbm19Zy+NwNF15BVloe4bnRELGJo7Gw+F3UlZo8Q3KFKcWBdTtzZ/SvbP+sasU9H0tiYCFpiNxIdfMLRPFN2JHKz7eMNFttQvroaNfVgAIaEycglFCkV2ntYR510k7SpvcT0Ix+4NI18G0goqhcX41jW57HzGX9dMeR4/nU07e8qlzSVNLj9qdXwCMJIOtIK5LibUUaGkAUi3jEqNFH0HOjnljOi5utPNqsYN9jHmsPWUvj/tym7g/AsDXky3bWPUfVsWpdOUJKTGEtG787uQlEF//6FwmAo26bAkxfbMdmew4HJmsQQA9HoH8Fx8LZW6xb8LhbeCcABOaXRqlYMABo2QRg4JJB1rrnMHqRWUQkb/jPHjRC4IkJUnTi8bvccDU3plWzR6P/R9g0vCzeR/CiBpD7MyJGCPY1HIrqXFVlANrof4mUhOpR3322+dCKGaFHGZPLaZrtBIDJKBcNZDhZ6kfOapC19MK+QJ2q X-Microsoft-Exchange-Diagnostics: 1; CY1PR0701MB1725; 31:0WU7lun5s9bBTGsp6lb+qMDST7rS5onT1bTFm5DUunpDwHSzQ5iSpYEvEFggMWhJzNJQVrqBbl8sS5eWXX5ZjoLa2bqPBMjuvsAS81Rfy8RwrcfpmLIvsag7kJrTSQ+rA/uuLHfz6eWALfGRHFl52vUa2q3/pEuDyTKo02y8RbYfOAthSfuyKYUfkdRX2f5qNs8Vq/JC96D9jqLnqFrpx0PzPwVKUucxJ2w3bLCyFcU=; 20:eWLMftmUIqTIX9x1SU+O/DMuiv6pHfRmckKrFOxgwu/eDu4zEhmimV5xMW2725AyuNap0ASUrqOR2/TRdwmltpgZM73QUkF1EZA95gmFqqjfZghqrbF9TRj4QFn97Zw4Hjl2wMGoNJsWAoCS1i6JoHDxl40Qy5akq8Haln8ES4W5OAZn3HTdsmZHH1qPfCc9sbNbu7P43Hsvj6C0sAdeCiWaLjHKlukDGfUUNOGg8dKBHvrIt03juhe5/CgAnOQYEpGy3bvre6jMBoqdlipji3xoHsICRavRoptEKjccyUK0m22MUYLWHHmtp3r8fNwOPta6wa5GX85PNQ60HiTJ5PfajOf6bDmNWR96GCOwlIOnzEjp/QNoXzsQjE6tx00cSuxIf/aJttOAdVUP7ri3P0jpCiULBa6l7P72pt0mDhhM53HEbRPUw4SCFNeWhROEegLis+OjTBlwSTjr4bJY8kgbJhxN7zn8PbjtaoeiAUMnaMqwbZTNFkMsGAopr6k2YPlGdkrYp+cHhphC+XkAsWbROJVjrH5UB7LC1oLtlaiWA4veSuYWnasDtoZSAQwOgmS3iXHfgJQaO+LbD4Ur/EqProCP/+QiIJp0n3FRrAk= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(166708455590820); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040176)(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046); SRVR:CY1PR0701MB1725; BCL:0; PCL:0; RULEID:; SRVR:CY1PR0701MB1725; X-Microsoft-Exchange-Diagnostics: 1; CY1PR0701MB1725; 4:kXnUFlVYVm53SWBugF2UmghKg6AhdZC4EPYPqBzNiXti2viQCM0KlEzDVTNCO05T2HbKUw+yQzM8R2cgwXZdyk5B+OyIacWgFS1h7D18PVqy5SkdkQflkFAk6/r4TBv8u1LMTMM1EJIxfModfrqmlB4/ufZdgely6zjWf5TJ09pP00xDziOsHjJmK8oNTQy5wpQa1DX6rXqqLdsdbI/ahd8/hwCIM+OpeMvE9l3oPf5z+WVjOoC1tvXlcFlmStFfANMgddSMMd+ncyQAJ9fLw51P4p+eKZxqUAUXhVJawp/HkDWlsOq/+wJm6iZXXgqWR7XqyqgOrPPNqBB+niyQVUw0CFXo+x0xTZp+tXe+yqLZbpWTftAEzluMIE+qglTxWvSl7N4N8tzJ2HBojOoZkSaOCCSLGKloOJpyWyFxgsZiQ+OPbjdnhsgwLb7AXKO8 X-Forefront-PRVS: 0039C6E5C5 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6009001)(6069001)(7916002)(24454002)(189002)(199003)(13464003)(305945005)(66066001)(2950100001)(1076002)(50466002)(19580405001)(47776003)(83506001)(4001350100001)(61506002)(6116002)(9686002)(23726003)(19580395003)(101416001)(4326007)(8666005)(3846002)(586003)(76176999)(7736002)(110136002)(97756001)(106356001)(54356999)(97736004)(7846002)(189998001)(33656002)(68736007)(42186005)(8676002)(15975445007)(105586002)(46406003)(81166006)(92566002)(2906002)(81156014)(50986999)(77096005)(18370500001)(7059030); DIR:OUT; SFP:1101; SCL:1; SRVR:CY1PR0701MB1725; H:localhost.localdomain; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CY1PR0701MB1725; 23:wi/z1CBhOD0iv7nlpiMIHKQARWsPXfEwe9EcXMd?= =?us-ascii?Q?Z+2L8gUyiFmKM19zAfiBbGlgwoIbUj+8bT2ZUlcSP90X+ScriW6F9Z7GNh7o?= =?us-ascii?Q?8LzoiN0bTpgI/Vhk31aX1wvtnFzCnHiZ6NuqVDX2ohLK4hyPfwjHLKCBtTRr?= =?us-ascii?Q?U1+cehhXQW+JmN6i9MJK+QgqZniO8Wx28CTOnxCvIC81sHZQ7/8Ola4yMuUZ?= =?us-ascii?Q?HHQQnAiEuP07fYi9lXckezlqfGc97GNc9zkYRmLC5PqJULX8Nq9fwlhaZsYm?= =?us-ascii?Q?RVER9ldVjTqIJRhrYHbX8igFRc1GnT6moS4r1y15HKFGkGHOssr7VrXOkEzU?= =?us-ascii?Q?sotryoRcm9ibeLZHhkKCyLaiyZ5iSgugCyMHM2SjYNgw5g72bOEPZImiZMyl?= =?us-ascii?Q?9hP8Odvk5p/i++dbF2tLx7KVcLywwwwNZ2d1viHXun/WJvcqiG5gx8q8mK+U?= =?us-ascii?Q?A6WatxmD6W3NkBf16QRAjEGoBLfkDL/wBFPaEqv3ldEXGAPZGZwcO+5U4R0g?= =?us-ascii?Q?cWTWViJDAibagPJWSE+Buf92e2ITSCCJJB0WjQzdB/aX0oMstbvpJv9WBDGX?= =?us-ascii?Q?nYrIFu8FaQo7MKM2R7TNAYdSlMfjK0UeYCdKPaDLkZYr8hYN45IsoNdakSKb?= =?us-ascii?Q?aMUfbzPg4qD6McGhkn+R3LX9U+9kul8cQWAYnOTnp2Vq20Om8ouudmhNnwYB?= =?us-ascii?Q?xY/MWtQ/ip071tZiKDB5xa5YC8fEOogPFX21DDZbkD46lCkFCOo0WQozPr1h?= =?us-ascii?Q?z5oe9Kq4pEwFe1hN1Mjl7+V2XcFc7bJZDgOrHgqziz4mJR6t8FNJ49hFeHpw?= =?us-ascii?Q?gOiJIJ6KO91M/muwMKkXfTad+Jdzr/yN8yLbP1hZ3obaxQvhIbzVJ+eldXjB?= =?us-ascii?Q?Y2EvDN/DQ6CS/kSxDjCQjJuI2qnMmGa0QYvNnXei91fmLYpuHGoBuMlqjYRp?= =?us-ascii?Q?H+N3Z7PLytvMqVYSuE5yVTRKI+4SyCu7tkJHCoz29L3mpFFSaERzk5fg2hS2?= =?us-ascii?Q?1sc4fNDLheKXpzI1swIdO4upChD9iUv0Q4/g1v1SyrMwFoAOO9rH3qUKOKmG?= =?us-ascii?Q?eI1SbLJ1qirwCNYAGqZ0ooj8J8nz3C86SYRLgYScxJUa11cTpGqxeg+ZhDLK?= =?us-ascii?Q?gsS9DV50O+2wMopwAM+ninmA8YErgPr6t55L6n3tP6js6ERDdnTDJungqJxc?= =?us-ascii?Q?U2tVdZNIcdodTOUXDpcrcGaREdA0SbL65k2Q7q85ggnKYyFQx7V2TfjGQ4yS?= =?us-ascii?Q?znk2eGkQ0Cilegn8HhbE=3D?= X-Microsoft-Exchange-Diagnostics: 1; CY1PR0701MB1725; 6:jm0voix3Yzb6YYmTpQEfi/I4s2LfPWNCDj+GS5vZYSFmWbho8qQ1AvV1fgXYSnA5thSmTNdRjocLNbTWNzKzzEX5OwJesOMGJ6jg4MOjhUK2G2Wwmkcmq0oTX1pMGfRnWHEleNZe/IU6Rr/ONeC56uMbwyyEkBtz+FLEhGz+pQpTezPq8NCAmYfZ28NL1vF4pqk2aML3527dkWdtfEC2FLy6fUZRyg3dxrC6XKGotT0kVK4TfuBr8ozx7STENOMO6xZtgDuH8AXDZQ/HMpbxyhxHKNY6JtjAi8fCHGrv1ec=; 5:SSdP5glVc6u8WV+4pqkXDfmp0jSb+fKT1QI1XC/1PQlCxE5V9oo4jkRUPQj+7prY0lrY36NsBFa7HI98FZqkkGSTyz+pSKBmyip9bchHSG7XUzl1pjwBWSW+PQfLualvv6vcoAw2KMzIjuqYUPj8nw==; 24:DeZ9zoYWCHkZuZtokdjriAM0/Hj0EV7CRR3YVfT/vGdgpyJhoaEIyKO/JceweivOQVpgfXGCFLqTgqYDXer7GFCtZ2Y/8FfLsKvDI42VmEw=; 7:ARkRzqt/dHiXrYA/0hlhPrJUgvaciarjj3XlE3yxAWIUtCnsWuF5AFJSLE505y3WdxOB8UNnXORNJDj2TC6V2jbb1S6Pp6JT/rYV5vnuu/WMC76vhbBVci2aIPTagDOYpA/LlUKClDvnBFkuB32XtYEexMGn87apWt4VfaoaH3FU1ZZhXhv1ppnKi/L7geaHIwUlRl/USZZYUmLlcQ6IbtDIAIt1Cd8leF2bprh5Al9TFHz5nqaJZcq1dknJiNEZ SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Aug 2016 11:46:33.3252 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR0701MB1725 Subject: Re: [dpdk-dev] [PATCH] eal/armv8: high-resolution cycle counter X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Aug 2016 11:46:40 -0000 On Fri, Aug 19, 2016 at 09:43:36AM +0000, Nipun Gupta wrote: > Hi Jerin, > Hi Nipun, > > -----Original Message----- > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Jerin Jacob > > Sent: Thursday, August 18, 2016 17:22 > > To: dev@dpdk.org > > Cc: thomas.monjalon@6wind.com; jianbo.liu@linaro.org; > > viktorin@rehivetech.com; Jerin Jacob > > Subject: [dpdk-dev] [PATCH] eal/armv8: high-resolution cycle counter > > > > Existing cntvct_el0 based rte_rdtsc() provides portable > > means to get wall clock counter at user space. Typically > > it runs at <= 100MHz. > > > > The alternative method to enable rte_rdtsc() for high resolution > > wall clock counter is through armv8 PMU subsystem. > > The PMU cycle counter runs at CPU frequency, However, > > access to PMU cycle counter from user space is not enabled > > by default in the arm64 linux kernel. > > It is possible to enable cycle counter at user space access > > by configuring the PMU from the privileged mode (kernel space). > > > > by default rte_rdtsc() implementation uses portable > > cntvct_el0 scheme. Application can choose the PMU based > > implementation with CONFIG_RTE_ARM_EAL_RDTSC_USE_PMU > > > > Signed-off-by: Jerin Jacob > > --- > > > > The PMU based scheme useful for high accuracy performance profiling. > > Find below the example steps to configure the PMU based cycle counter on an > > armv8 machine. > > > > # git clone https://github.com/jerinjacobk/armv8_pmu_cycle_counter_el0 > > # cd armv8_pmu_cycle_counter_el0 > > # make > > # sudo insmod pmu_el0_cycle_counter.ko > > # cd $DPDK_DIR > > # make config T=arm64-armv8a-linuxapp-gcc > > # echo "CONFIG_RTE_ARM_EAL_RDTSC_USE_PMU=y" >> build/.config > > # make -j 4 > > Can we make this kernel module also a part of DPDK. May be in the linuxapp so that it is also compiled with DPDK? I thought so, Later I realized it may not be a good idea to add yet another out of tree module in DPDK repo and DPDK tries to get rid of existing out of tree modules. > > > > > --- > > .../common/include/arch/arm/rte_cycles_64.h | 33 > > ++++++++++++++++++++++ > > 1 file changed, 33 insertions(+) > > > > diff --git a/lib/librte_eal/common/include/arch/arm/rte_cycles_64.h > > b/lib/librte_eal/common/include/arch/arm/rte_cycles_64.h > > index 14f2612..867a946 100644 > > --- a/lib/librte_eal/common/include/arch/arm/rte_cycles_64.h > > +++ b/lib/librte_eal/common/include/arch/arm/rte_cycles_64.h > > @@ -45,6 +45,11 @@ extern "C" { > > * @return > > * The time base for this lcore. > > */ > > +#ifndef RTE_ARM_EAL_RDTSC_USE_PMU > > +/** > > + * This call is portable to any ARMv8 architecture, however, typically > > + * cntvct_el0 runs at <= 100MHz and it may be imprecise for some tasks. > > + */ > > static inline uint64_t > > rte_rdtsc(void) > > { > > @@ -53,6 +58,34 @@ rte_rdtsc(void) > > asm volatile("mrs %0, cntvct_el0" : "=r" (tsc)); > > return tsc; > > } > > +#else > > +/** > > + * This is an alternative method to enable rte_rdtsc() with high resolution > > + * PMU cycles counter.The cycle counter runs at cpu frequency and this scheme > > + * uses ARMv8 PMU subsystem to get the cycle counter at userspace, However, > > + * access to PMU cycle counter from user space is not enabled by default in > > + * arm64 linux kernel. > > + * It is possible to enable cycle counter at user space access by configuring > > + * the PMU from the privileged mode (kernel space). > > + * > > + * asm volatile("msr pmintenset_el1, %0" : : "r" ((u64)(0 << 31))); > > + * asm volatile("msr pmcntenset_el0, %0" :: "r" BIT(31)); > > + * asm volatile("msr pmuserenr_el0, %0" : : "r"(BIT(0) | BIT(2))); > > + * asm volatile("mrs %0, pmcr_el0" : "=r" (val)); > > + * val |= (BIT(0) | BIT(2)); > > + * isb(); > > + * asm volatile("msr pmcr_el0, %0" : : "r" (val)); > > In your git repo I see that on cleanup the cycle count register is not disabled (PMCNTENCLR_EL0). It shall be better to disable the cycle count register too at module exit. OK > > > + * > > + */ > > +static inline uint64_t > > +rte_rdtsc(void) > > +{ > > + uint64_t tsc; > > + > > + asm volatile("mrs %0, pmccntr_el0" : "=r"(tsc)); > > + return tsc; > > +} > > +#endif > > > > static inline uint64_t > > rte_rdtsc_precise(void) > > -- > > 2.5.5 > > Do you also plan to support performance monitor event counters? No. This patch was inspired by armv7 PMU scheme and its part of DPDK. The sole reason to add this support to catch any performance regression through app/test application.Other than that, I think cntvct_el0 based existing scheme is good enough for all the use cases. > > Regards, > Nipun >