From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5C487A0032; Fri, 15 Jul 2022 16:19:00 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 50D7342B6D; Fri, 15 Jul 2022 16:19:00 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by mails.dpdk.org (Postfix) with ESMTP id E0CF740696 for ; Fri, 15 Jul 2022 16:18:57 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657894738; x=1689430738; h=message-id:date:subject:to:cc:references:from: in-reply-to:content-transfer-encoding:mime-version; bh=eh5pMd9l7kZsx15wsB2WwWRAewEEV0echg4R0CNcEho=; b=N6b+4GQ0qnnI9UTwd2tu+Ai9hZmbW3v6u0Sr9ad6MRCxi8zrSZvL4mNF N2T5/i0fz0BCfuQ79cdFmEKXObN+hgD/XloD2wyvnMVeK6/NznPshDkzx 6aAI1/9RkgW3K3LPJ9zwb/RGOHFwWWfPHSqaczSP84nxyZmGk4cbUvQZs KwU+lrg7fjrPCKYIFY7dcpX3bjbaPbvVm4r6zrlOQgEMrZHFaLkxWXDfc xfQFiQdpNlXE7O9Z50t5RJXBWPU9BWQHV089u1NSGgjGXSAkuXEHRvET9 4ZeEo0diRjnX1kIMfF/GSVu/f+qmlBahHP+TDkemsrfL8LqDxGNQDP3bW g==; X-IronPort-AV: E=McAfee;i="6400,9594,10408"; a="286543272" X-IronPort-AV: E=Sophos;i="5.92,274,1650956400"; d="scan'208";a="286543272" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Jul 2022 07:18:56 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.92,274,1650956400"; d="scan'208";a="685974797" Received: from orsmsx606.amr.corp.intel.com ([10.22.229.19]) by FMSMGA003.fm.intel.com with ESMTP; 15 Jul 2022 07:18:56 -0700 Received: from orsmsx603.amr.corp.intel.com (10.22.229.16) by ORSMSX606.amr.corp.intel.com (10.22.229.19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27; Fri, 15 Jul 2022 07:18:55 -0700 Received: from orsedg603.ED.cps.intel.com (10.7.248.4) by orsmsx603.amr.corp.intel.com (10.22.229.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27 via Frontend Transport; Fri, 15 Jul 2022 07:18:55 -0700 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (104.47.70.105) by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2308.27; Fri, 15 Jul 2022 07:18:55 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=mrSEQ+reRkkVwwz3earmTVrQ03UxNDKh57IQnvlZqoJRa/hQFbonEvxbrgcf3Ttxy1zzJyGboivTEgvTyTtRiU/p9UX6jfOXmU4w5CPsmgLMTybAlgShtRdPO4SBRUchMinmBozm2fsYk/CYDqwTG1lg7gTNZbc7sxJtZABQWHkNrcZoWn5pSXyjdvxyaIAezejOaRCem6can2JDidK5Agja7aKdtCe4N52qIfWgVXqg9lcWEVrgqDEVlbbJVoiOkiLYTdvLqqYjMr5PyA7wZEf/BgmgynKy5e7bpPANCUJm8PTx+MU4HZsozYmgO5SCVWOeBPYzzCKETG0F1vUcPQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=uHJ+0EQQLDHVXxxnUirCZ9z9ZUzM86ZihAxQI6uWDt4=; b=doyMnbJht3209jyY3ZS/cXrCWmwgJakj9ihnF/GL0KF5m19a7QjRuaniNxYPp1+QwrpPagkeFfnwttKWboR8SCjpi+4F8IK2X/LL6zyV+UdGQYXC7mas5ol6UPRSEGGAkinVXx9RelMTIW/ZaOEcOP9ytUG/cO8Xxf2DWI72F0cAY/0pazYF5KBlVCqy2DdORCCJRHdhvI5o3pMLrhKN6IPwgJ8yD0QI0YJGMMzriPYSQpU5/MMYIw2CN0F/s/84WmI3mOcF/c7z6J/1zJjUUNCbIbibMlmiy/0mI9ExZXGvw3r2TQfHcfGzS2rIszUbdup3MSBnkpnScYWnpiknXA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from BN6PR11MB1251.namprd11.prod.outlook.com (2603:10b6:404:48::10) by SN6PR11MB2591.namprd11.prod.outlook.com (2603:10b6:805:60::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5438.17; Fri, 15 Jul 2022 14:18:51 +0000 Received: from BN6PR11MB1251.namprd11.prod.outlook.com ([fe80::128:8fb:9d0:1a9f]) by BN6PR11MB1251.namprd11.prod.outlook.com ([fe80::128:8fb:9d0:1a9f%8]) with mapi id 15.20.5417.029; Fri, 15 Jul 2022 14:18:50 +0000 Message-ID: Date: Fri, 15 Jul 2022 15:18:44 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Firefox/91.0 Thunderbird/91.11.0 Subject: Re: [PATCH v1 1/2] eal: add lcore busyness telemetry Content-Language: en-US To: Jerin Jacob CC: dpdk-dev , Bruce Richardson , Nicolas Chautru , Fan Zhang , Ashish Gupta , "Akhil Goyal" , David Hunt , Chengwen Feng , Kevin Laatz , Ray Kinsella , Thomas Monjalon , Ferruh Yigit , Andrew Rybchenko , Jerin Jacob , Sachin Saxena , Hemant Agrawal , Ori Kam , "Honnappa Nagarahalli" , Konstantin Ananyev , Conor Walsh References: <24c49429394294cfbf0d9c506b205029bac77c8b.1657890378.git.anatoly.burakov@intel.com> From: "Burakov, Anatoly" In-Reply-To: Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 7bit X-ClientProxiedBy: DB8PR06CA0037.eurprd06.prod.outlook.com (2603:10a6:10:120::11) To BN6PR11MB1251.namprd11.prod.outlook.com (2603:10b6:404:48::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 7ada1fc7-3481-4b0c-ef7e-08da666cf056 X-MS-TrafficTypeDiagnostic: SN6PR11MB2591:EE_ X-LD-Processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 1afjvYpq+nOVviLfwmxkNH/RmVsRCPC2w8eI1k+cEbxZAIz+2v2Pd1OjbYwEgqPpTx6gtVs7nM9uzMr20nqvGl/SzAE0g7Ry2aY6cjHjDQRkCnGbdWLhcPcm6pJOe5jlSotUdbQX1KUzRfXm1x4Lu5sP3zOII11gast7+jwiEcY2FYo75UXFhNcxWABWSbe1tk9MmCTlF1nknKaprlWiWnEWdfdh3DXLqvhKVkZG2oluvN5hvbZG8gePTfjgwCJ75NvtMj4HLPTYQCR32kqtkoi907T9UdkUSX0zukznn20i12rK/NCjAw+jy0zeiV1i27op9eYcVHsi8VsGSKtdHPgvz1v0wuts8bxzdl+9ckdjr5yc0bB3L/T9tRfVpZZVCJt9yV/pWopdIRVLWxHkgFjBLgNyP/P8vuvjgjgoxRvcb5+sAJpIz3crozTzXdpKMFEvQniujrHdcV7VXOjkfE+at4euY+M4574gbV4laSi85a0y6stNdZ51XYF7VmuX2FM3J9RpjAxAY8JxHgLW9mABY8vUSMrFJlYH8PNutxCRR8q6+71/GzcbCEo5auq094BTet58pELkYJsh7fQ4/rPrr1YZeDh7xHyYLkR71ADxr/zBcKzVclctUn5z9mvP66XvYFJe9FkvkDo9NxdsUoLnVY6Hp2PJ3/I7hAbHsoNxW63gp0EydW5FD2MOshwq4ka0UNweKt7zZSvpKKZ4wXpYopQ3SRYddsyxW39u8BKt61bm1Jp8Y5P3uPHdb3BAW9sUbolvdxkg0DEHdq6ZDGDgO5ax+5OcFFmBqn81FPjdIlT6GQB7pNhp7O48jmaKJxRpOCjxmN1xuFQ6s+WB7SHB/4KGdmPthCO8pAxXhMk= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BN6PR11MB1251.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230016)(396003)(39860400002)(136003)(376002)(346002)(366004)(5660300002)(8936002)(7416002)(2906002)(66476007)(4326008)(66556008)(8676002)(66946007)(38100700002)(36756003)(86362001)(31696002)(6512007)(82960400001)(6486002)(478600001)(54906003)(6916009)(316002)(31686004)(186003)(83380400001)(41300700001)(53546011)(107886003)(6506007)(6666004)(2616005)(26005)(45980500001)(43740500002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?dDQ1d1FEclpNcXkyY25iYWR3QkJNZ2g4R3p3ZENZdjNxMWZJcldzcHJtZjh0?= =?utf-8?B?bVZxVGRuL3VjOUJWNFpBR1BuV2JuUzNLZEcyK29jcEV5T0tBWmwxUTE4WkhQ?= =?utf-8?B?Zk9RWWp4amNjWElaZGEveFpGbWViQUZ6NzhVanowVVNnK0hGN20xRm5JZmNo?= =?utf-8?B?UDFPTmJ2SWxtaVdPYzgyaXdnSk5rUHIvWkRCdGtwTi9yM0xCY2lTdkhNb0hy?= =?utf-8?B?c2RyZlViMlVhcjd2QngySUVmejEzRkkzb2djYVFMSnJwWFVhNGdUNzdaemRp?= =?utf-8?B?TlVvRUZYa1FSdE5HcXdIbGNXOTdrR3R3M3lML0huRGJGQTJ5RHBFdXQzR1ly?= =?utf-8?B?RW43bzhKVVlHR2QzNm8yTXlKTWRyVFZxK0hUZUlnQkpPdGIycm96SkNUcVlq?= =?utf-8?B?c2tXZ3VIT0J5bDRpTndwcG1tWi9nOWJBa09XS2tCeEIvRUIrYmxFemk0ay9P?= =?utf-8?B?cVJQSWpmTWNUbXozaU55M052V3JLSVk1STQvcndrSUlqZXYrVFVBZFlQcGF0?= =?utf-8?B?QThvejFsVHdnSnpVbHdYNWY3S0FrZFBrWTUxZzNWZmRTbm5jTnhiK3AwUUpv?= =?utf-8?B?eFNFOEh5SFVtd0NpalpSS25WcTRNQmRvRnpqeXpHUUM1TkhtMDVwZmhkcWRP?= =?utf-8?B?N29WaklhTyt2Qk0raERwelhxSkRZakFQNTJQdnhreUNXTytrV1JIZ2xUVHl3?= =?utf-8?B?Q3R2eVc4aVlyN0svaTEvKzZoUk5pKzZzZjFKSVh4aytManNNZEdCZW5oVVpZ?= =?utf-8?B?MmRNMW1WdzgxdDBDSVl1NVRnNGRmRUwycFY3MFFpYjUwY0NYdm5HNHJqN1Ar?= =?utf-8?B?cXk0RzczSEZybVd4NDh0K0ZVWkNvMFYyT1NEUEhSSXhTMVp4RTJiZkRLYzNE?= =?utf-8?B?cHhRZUtaTHNwUWR5RTA1N0xsbS9OZ3EydHVrYkV3VllPYmhFbllNWVBZQlFh?= =?utf-8?B?ZlpxbjNXMG5NTkNjaXVKcTBEaVpRVjYwdUU2TG1lYWNnczVOOTJ4cGd1KzJJ?= =?utf-8?B?b1REWVRJaVdMVFMxVlgvT1c4N1A0RWdOeG5iTE4wMkliUStPSUJZd0lzM3lt?= =?utf-8?B?THFaSGRBVlljNGkvNDN1Q0YzeW5NNWN5bzc0YjNEMmZ6UWwyLyt3aGZqN0lM?= =?utf-8?B?cjdXSnRFT3JyMHNHc0d5T2lZdlpOWUpSaERkQTlFR1IwakJVZ1JvSE55RjVO?= =?utf-8?B?YkQwaDV5ZlNTeUV0Zm52TGFEM0tuRzI3RmhtU3hmN1FzV1dZdXRjSThJUDVt?= =?utf-8?B?OFdZaW85emg4WTBIdnVtc1R3NDkzR25LQjR5NmxPTkRWeXRSVk5CRys1Yk9W?= =?utf-8?B?Ly9NM1U3YzJmd21LWlgybzNZNUVnTXZoOWc1RFhwUVh1RWNGT2lTODlsalcx?= =?utf-8?B?dE5kZ0dOZVM3dXBIb2pmbFNQc05TTFY0WDJVdmI0STBHZXdwMGlQaTQ2NG8x?= =?utf-8?B?YWc1b0RZOEM2aml0UmVRL2pxSzFVcld1N3owNDVJVmp3ZmVZQW9IQUVLUnBl?= =?utf-8?B?Z1NXRWdUZjRDM3FiaUN2Vk5nU0l1UytCWWw0Z3pLM0lzWllOcW50Z09YRUNz?= =?utf-8?B?bjVwTWR0SHZkTk5lelZJVDVZQS9jUjFZdnZjcFNlL3U0dVBiaVRBbUtTdVk1?= =?utf-8?B?bCttckpBK2REZEdScDljeVhvdUcwMGptcG1iTGFJR3ZjTmtQdjdiV0hndFh1?= =?utf-8?B?ZnRjMzJKMWJhK3laYTFmd3dCU1pHcVVjUTErMEZ1TzRiaXNXUHk3QkF2WnFu?= =?utf-8?B?RnJ6ajFkOEl1aThsdEQ5Q2h6QU9kblVmSEI1N0t5MzdzYTNiOUNTZ1NWUGVO?= =?utf-8?B?OXVGSjVzbVMweUw5cWV1dmpKbFAxZ1ZVNkFlTXNwQi9nR04zVHlwVWZiSWd1?= =?utf-8?B?OEEvUTV0akRKRGgxSEN5WWtMQm9YNks3TVpTY1V6bVBxTHlibVp2UzhqaW5s?= =?utf-8?B?NXV3Z1V6V1NFYTVINFZoYVBQTjRIaHZrV2FXaFJiWml0UzJQbm9ZblpqNlU1?= =?utf-8?B?Vmo2TXduamZTY2tMUjJCY213RkFYTzR1emo2K2dqdmsxYlRSd29wZGdIaXlW?= =?utf-8?B?UkhtaW5VTDZlb2NMS3AwV2RsbGdjZk1JK3kvWmJqbWIxWExhZHJDTzcrSDRD?= =?utf-8?B?cm1MNVNGaEw3b01KK2QvcGE4Ty9DdUU5WHB4b1ZHOVJqZ0hFVWpicmZrb2ZR?= =?utf-8?B?Nnc9PQ==?= X-MS-Exchange-CrossTenant-Network-Message-Id: 7ada1fc7-3481-4b0c-ef7e-08da666cf056 X-MS-Exchange-CrossTenant-AuthSource: BN6PR11MB1251.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Jul 2022 14:18:50.3108 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: YWTqD+1bOCBcs+J1kaLj1rI5YchiXlx1snyejEoFEKrGZ7H6caUBQfXqTqKe1p+51UJdq315FrTwrruGVpoEyrBjejkORAZ++m6zhtqM9uM= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR11MB2591 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 15-Jul-22 2:46 PM, Jerin Jacob wrote: > On Fri, Jul 15, 2022 at 6:42 PM Anatoly Burakov > wrote: >> >> Currently, there is no way to measure lcore busyness in a passive way, >> without any modifications to the application. This patch adds a new EAL >> API that will be able to passively track core busyness. >> >> The busyness is calculated by relying on the fact that most DPDK API's >> will poll for packets. Empty polls can be counted as "idle", while >> non-empty polls can be counted as busy. To measure lcore busyness, we >> simply call the telemetry timestamping function with the number of polls >> a particular code section has processed, and count the number of cycles >> we've spent processing empty bursts. The more empty bursts we encounter, >> the less cycles we spend in "busy" state, and the less core busyness >> will be reported. >> >> In order for all of the above to work without modifications to the >> application, the library code needs to be instrumented with calls to >> the lcore telemetry busyness timestamping function. The following parts >> of DPDK are instrumented with lcore telemetry calls: >> >> - All major driver API's: >> - ethdev >> - cryptodev >> - compressdev >> - regexdev >> - bbdev >> - rawdev >> - eventdev >> - dmadev >> - Some additional libraries: >> - ring >> - distributor >> >> To avoid performance impact from having lcore telemetry support, a >> global variable is exported by EAL, and a call to timestamping function >> is wrapped into a macro, so that whenever telemetry is disabled, it only >> takes one additional branch and no function calls are performed. It is >> also possible to disable it at compile time by commenting out >> RTE_LCORE_BUSYNESS from build config. >> >> This patch also adds a telemetry endpoint to report lcore busyness, as >> well as telemetry endpoints to enable/disable lcore telemetry. >> >> Signed-off-by: Kevin Laatz >> Signed-off-by: Conor Walsh >> Signed-off-by: David Hunt >> Signed-off-by: Anatoly Burakov > > > It is a good feature. Thanks for this work. Hi Jerin, Thanks for your review! Comments below. > > >> --- >> >> Notes: >> We did a couple of quick smoke tests to see if this patch causes any performance >> degradation, and it seemed to have none that we could measure. Telemetry can be >> disabled at compile time via a config option, while at runtime it can be >> disabled, seemingly at a cost of one additional branch. >> >> That said, our benchmarking efforts were admittedly not very rigorous, so >> comments welcome! > >> >> diff --git a/config/rte_config.h b/config/rte_config.h >> index 46549cb062..583cb6f7a5 100644 >> --- a/config/rte_config.h >> +++ b/config/rte_config.h >> @@ -39,6 +39,8 @@ >> #define RTE_LOG_DP_LEVEL RTE_LOG_INFO >> #define RTE_BACKTRACE 1 >> #define RTE_MAX_VFIO_CONTAINERS 64 >> +#define RTE_LCORE_BUSYNESS 1 > > Please don't enable debug features in fastpath as default. It is not meant to be a debug feature. The ability to measure CPU usage in DPDK applications consistently comes up as one of the top asks from all kinds of people working on DPDK, and this is an attempt to address that at a fundamental level. This is more of a quality of life improvement than a debug feature. > >> +#define RTE_LCORE_BUSYNESS_PERIOD 4000000ULL >> >> + >> +#include >> +#include >> +#include >> + >> +#include >> +#include >> +#include >> +#include >> + >> +#ifdef RTE_LCORE_BUSYNESS > > This clutter may not be required. Let it compile all cases. Windows does not have telemetry library (so this define never exists on Windows), and we want to be able to compile this without lcore telemetry enabled as well (by commenting out the config option). We would have to have #ifdef-ery here in any case. Am I missing an obvious way to have this without #ifdef's? > >> +#include >> +#endif >> + >> +int __rte_lcore_telemetry_enabled; >> + >> +#ifdef RTE_LCORE_BUSYNESS >> + >> +struct lcore_telemetry { >> + int busyness; >> + /**< Calculated busyness (gets set/returned by the API) */ >> + int raw_busyness; >> + /**< Calculated busyness times 100. */ >> + uint64_t interval_ts; >> + /**< when previous telemetry interval started */ >> + uint64_t empty_cycles; >> + /**< empty cycle count since last interval */ >> + uint64_t last_poll_ts; >> + /**< last poll timestamp */ >> + bool last_empty; >> + /**< if last poll was empty */ >> + unsigned int contig_poll_cnt; >> + /**< contiguous (always empty/non empty) poll counter */ >> +} __rte_cache_aligned; >> +static struct lcore_telemetry telemetry_data[RTE_MAX_LCORE]; > > Allocate this from hugepage. Good suggestion, probably needs per-socket structures as well to avoid cross-socket accesses. > >> + >> +void __rte_lcore_telemetry_timestamp(uint16_t nb_rx) >> +{ >> + const unsigned int lcore_id = rte_lcore_id(); >> + uint64_t interval_ts, empty_cycles, cur_tsc, last_poll_ts; >> + struct lcore_telemetry *tdata = &telemetry_data[lcore_id]; >> + const bool empty = nb_rx == 0; >> + uint64_t diff_int, diff_last; >> + bool last_empty; >> + >> + last_empty = tdata->last_empty; >> + >> + /* optimization: don't do anything if status hasn't changed */ >> + if (last_empty == empty && tdata->contig_poll_cnt++ < 32) >> + return; >> + /* status changed or we're waiting for too long, reset counter */ >> + tdata->contig_poll_cnt = 0; >> + >> + cur_tsc = rte_rdtsc(); >> + >> + interval_ts = tdata->interval_ts; >> + empty_cycles = tdata->empty_cycles; >> + last_poll_ts = tdata->last_poll_ts; >> + >> + diff_int = cur_tsc - interval_ts; >> + diff_last = cur_tsc - last_poll_ts; >> + >> + /* is this the first time we're here? */ >> + if (interval_ts == 0) { >> + tdata->busyness = LCORE_BUSYNESS_MIN; >> + tdata->raw_busyness = 0; >> + tdata->interval_ts = cur_tsc; >> + tdata->empty_cycles = 0; >> + tdata->contig_poll_cnt = 0; >> + goto end; >> + } >> + >> + /* update the empty counter if we got an empty poll earlier */ >> + if (last_empty) >> + empty_cycles += diff_last; >> + >> + /* have we passed the interval? */ >> + if (diff_int > RTE_LCORE_BUSYNESS_PERIOD) { > > > I think, this function logic can be limited to just updating the > timestamp in the ring buffer, > and another control function of telemetry which runs in control core to do > heavy lifting to reduce the performance impact on fast path, That ring buffer would have to be rather large, because telemetry calls can come in as often as once every couple of hundred cycles (when polls are empty) while the telemetry measuring period is in the millions of cycles. That said, this is an interesting area to explore in further revisions, so I'll think of something along those lines, thanks! > >> + int raw_busyness; >> + >> + /* get updated busyness value */ >> + raw_busyness = calc_raw_busyness(tdata, empty_cycles, diff_int); >> + >> + /* set a new interval, reset empty counter */ >> + tdata->interval_ts = cur_tsc; >> + tdata->empty_cycles = 0; >> + tdata->raw_busyness = raw_busyness; >> + /* bring busyness back to 0..100 range, biased to round up */ >> + tdata->busyness = (raw_busyness + 50) / 100; >> + } else >> + /* we may have updated empty counter */ >> + tdata->empty_cycles = empty_cycles; >> + >> +end: >> + /* update status for next poll */ >> + tdata->last_poll_ts = cur_tsc; >> + tdata->last_empty = empty; >> +} >> + >> +#ifdef RTE_LCORE_BUSYNESS >> +#define RTE_LCORE_TELEMETRY_TIMESTAMP(nb_rx) \ >> + do { \ >> + if (__rte_lcore_telemetry_enabled) \ > > I think, rather than reading memory, Like Linux perf infrastructure, > we can patch up the instruction steam as NOP vs Timestamp capture > function. Would you be so kind as to provide a link to examples of how this can be implemented? > > > Also instead of changing all libraries, Maybe we can use > "-finstrument-functions". > and just mark the function with attribute. This is not meant to be a profiling solution, it is rather a lightweight CPU usage measuring tool, to expose (albeit limited) CPU usage data. I'm by all means not an expert on `-finstrument-functions` flag, but from cursory reading of description in GCC manual it seems to be rather heavyweight as it's captuing function enter/exit and other stuff, which we're not really interested in. This *would* make this feature a debug feature, but this was not the intention here :) -- Thanks, Anatoly