From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C24A9A0555; Fri, 26 Aug 2022 10:07:29 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7402240146; Fri, 26 Aug 2022 10:07:28 +0200 (CEST) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by mails.dpdk.org (Postfix) with ESMTP id E454A40143 for ; Fri, 26 Aug 2022 10:07:26 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1661501247; x=1693037247; h=date:from:to:cc:subject:message-id:references: in-reply-to:mime-version; bh=RQT9C1Yy21jbNPziPC0cxmInXeXCqUhpSrhQVEzk8iI=; b=U0jTYysS/cAAaYpg2/6jyLG/K+iiUB3CjclZBiO59RfVxae0/3/59VlF v6zg4hG3wJ06qSyqA8pND/lQjQZTrafh1VrjtTVrCWHHe23T4/qh1E+4q 3sIePVhYW/5GG/GFHxLoScidPU6H5aU7WaeCL7UIq42QdM0JgU+Fi4/yt 9LrqojY1EXDW7TA4IGA/OlCGbal8IbeEjBlIhjHrkFysURkf1IgdYY4mc ekvKAaLoyW5DAf2mCo8EUS+ZdcYerroNAuLnqOtAKDRw2X/nqVTVxR/Da fkExfSaFX9n4lsRfFMOhuXfJsmMjCKg0ZLATTHL7Sy+j6iWaLCdO2x2Mg w==; X-IronPort-AV: E=McAfee;i="6500,9779,10450"; a="294456933" X-IronPort-AV: E=Sophos;i="5.93,264,1654585200"; d="scan'208";a="294456933" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Aug 2022 01:07:25 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,264,1654585200"; d="scan'208";a="752798014" Received: from fmsmsx601.amr.corp.intel.com ([10.18.126.81]) by fmsmga001.fm.intel.com with ESMTP; 26 Aug 2022 01:07:18 -0700 Received: from fmsmsx607.amr.corp.intel.com (10.18.126.87) by fmsmsx601.amr.corp.intel.com (10.18.126.81) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Fri, 26 Aug 2022 01:07:18 -0700 Received: from fmsmsx612.amr.corp.intel.com (10.18.126.92) by fmsmsx607.amr.corp.intel.com (10.18.126.87) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Fri, 26 Aug 2022 01:07:18 -0700 Received: from fmsedg602.ED.cps.intel.com (10.1.192.136) by fmsmsx612.amr.corp.intel.com (10.18.126.92) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31 via Frontend Transport; Fri, 26 Aug 2022 01:07:18 -0700 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (104.47.55.177) by edgegateway.intel.com (192.55.55.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2375.31; Fri, 26 Aug 2022 01:07:17 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=NDn6ObQdeQgifiosJghVz+uz1ODErKIaXM6DAY+ME6HHA/2zxZd51P3k9nNey3bPWSGUzzgox3juqklXfaybKyUG1eH9OeAK045sLMgcL8KqmWq8wc0VJWvcESWTbJoH6pfo1BFX/hQuKO9Hy/YAg7Mhs+Ohp57VgXO0q+bnD5vZLDrU5mzD0Hsw+/V14mwMV+cNiCoydh8cbZD5EHFTuBccbmUmqJGjLEpj9FHfhZUAPwJ98eXADHQXy3Do5iEw5l6lLou4WMAlP6Zct/ljxEzUc8BeGkjhuxpzXGece9M1LmsWT5vh7u+frPK9yIBZ6Hre0RqKnXqlqrO4JR8a6A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=lJsYxHriISJppMtV4XhLBlXw1MXzZ7POkekv0c+90TY=; b=YNdOc/0zdShGotWEe22rn0XTkaL1NO13pfs3cnPR/Pg+SwY7MNxTsQhpfP7zPQvdRD61IDtSCwMaWL1Tj5/MsLN2yhUWswBLBf+kOLTYXknjHUkGic6oD2pFhYPph9wL2A0mhfBH1o7tGZ3rDMApq5jOiWf6LaesxcvKi6nru4/mNZ7Lv2YxGvjs4g4EoDUjwvl3Xcbp54nGqWE1Y1bWdOI2cpyM/O3jfnCe354rS0pzSustVU+MPpZcpOBiuqzd2IpVexx/oLHokqstLTATRiOQPK0FovEc7KFjo8t4lOn4JHIHm/Y5/M996bpYd411eHMp7Ylyky8i1QeqLn4wiw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from MWHPR11MB1629.namprd11.prod.outlook.com (2603:10b6:301:d::21) by BYAPR11MB3269.namprd11.prod.outlook.com (2603:10b6:a03:75::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5546.19; Fri, 26 Aug 2022 08:07:10 +0000 Received: from MWHPR11MB1629.namprd11.prod.outlook.com ([fe80::13c:8120:d994:16d2]) by MWHPR11MB1629.namprd11.prod.outlook.com ([fe80::13c:8120:d994:16d2%5]) with mapi id 15.20.5546.025; Fri, 26 Aug 2022 08:07:10 +0000 Date: Fri, 26 Aug 2022 09:07:01 +0100 From: Bruce Richardson To: Jerin Jacob CC: Kevin Laatz , dpdk-dev , "Anatoly Burakov" , Conor Walsh , David Hunt , Nicolas Chautru , Fan Zhang , "Ashish Gupta" , Akhil Goyal , Chengwen Feng , Ray Kinsella , Thomas Monjalon , Ferruh Yigit , Andrew Rybchenko , Jerin Jacob , "Sachin Saxena" , Hemant Agrawal , Ori Kam , Honnappa Nagarahalli , Konstantin Ananyev , Morten =?iso-8859-1?Q?Br=F8rup?= Subject: Re: [PATCH v3 1/3] eal: add lcore poll busyness telemetry Message-ID: References: <24c49429394294cfbf0d9c506b205029bac77c8b.1657890378.git.anatoly.burakov@intel.com> <20220825152852.1231849-1-kevin.laatz@intel.com> <20220825152852.1231849-2-kevin.laatz@intel.com> Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: DU2PR04CA0266.eurprd04.prod.outlook.com (2603:10a6:10:28e::31) To MWHPR11MB1629.namprd11.prod.outlook.com (2603:10b6:301:d::21) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 80c5f1a3-f6be-44d9-c47e-08da8739f9b4 X-MS-TrafficTypeDiagnostic: BYAPR11MB3269:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Y3ecVMW/QAJAVlcaqI+4R9LIfe7fxbIc7SIKEUaZAD+IMolcZ3/ST6aCk02UQ/hO/3ywOcPLviKuRJp0OHHgIrg8iWHzDlQmVzpaiG1j9ORQgjFyePXyv1Qzy6iVOhw/EIUhu3XgV5WS1IHMUK/JQgdw+j9VRZjEVJEmLq8fJAGbUkQFKPqFWAC184dDzK46JjvBMUv/5FwKL/6FRWIIWDM2baxXpfk6xo/8PhoaSbqWwjkRregysH83L/ZkcuYVo1pIIsvB5X5shl0buZoER+3xaCa7ungLRkC3Fsd4su+grNpetAJBK74wA37d735C3TkHG0i+EpytOvbMTZND5YFg/MPww78+AA+E7hqr4ozqH2zTNYsUDJZd2xD6Q7QH3M2+4gxk3ZKGBI5RnBcTcCbgcopyhdVskBnfaa6tWLiakmUh3+0nDzltYVh+elaA29HGcFKihkWb9xZKgzlX7eTMqiAj8IDWADfhVdaMsjEjdHtsBtiPGrj6OzGE3FU7sR+nLG6pewol+wn3RsDSAgkv1a5w5YDqyTiV1PxidjEjbY3EC4sJATtvlcZ6IuymzMSCWCCwwbt3G+LmWT9SzLUrDyBYsEMyGm4yIqVL7KqlGC/v9VZAJJyLKmyZVe/rBHTS7Np7KqIfU4hLAXcP5lbPJxt+gVbqYhQolazd0i4aQ4wQDDusWysIMfFl3H94obaWBfpcuh6qfUOfILCsnQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MWHPR11MB1629.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230016)(39860400002)(136003)(376002)(366004)(346002)(396003)(6666004)(41300700001)(86362001)(54906003)(44832011)(316002)(7416002)(478600001)(53546011)(6506007)(26005)(5660300002)(186003)(6512007)(6486002)(6916009)(83380400001)(66556008)(8936002)(8676002)(82960400001)(66946007)(4326008)(66476007)(2906002)(38100700002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?IXUsss5aUA+HjlCsXXp0DmhoPjuK5VowQ3dSiqDMPxt/jfoNeSR+vmARv51a?= =?us-ascii?Q?3kJK8kS9SDcMnaQAZ4uzPJs+laV30iy6H/RGXtj/k9B2ErhPwMFaEKyHtUd3?= =?us-ascii?Q?Ilo4rhAHxnLXm8jE2eB0G+JOoqroEXqVflczAfmnDetnwRzXYF8C2VV3ERwr?= =?us-ascii?Q?zEI4+f1i61aQ/3Sf7UT5JgLti0PVP5ITS5Qntj0St+Yxj/zmqtMg7UFeB0Yp?= =?us-ascii?Q?MtwrNcxbvqXgzMAHkTVxn1UQot4kBP3is0w6xPT3mbunY027YTstoRwX6HyQ?= =?us-ascii?Q?6FXRdW4ww6liyvxYuUTdh31GHWq0MQX6AvJfPvQrFrFWiZkSpuVddwNbWnLs?= =?us-ascii?Q?sc8SFwCQmlOBPVcTLGjLoU14AorvFaPytJv5ELNdCZc1wm5gUnRo3fKBU9ba?= =?us-ascii?Q?WIOfaiGyWGD2DLtNEMi05+7eO+0f3e/I1HxOTtCEAnaRx0LECW967LYlItit?= =?us-ascii?Q?zyTAqb+rtdgoEVki5BI6DqGESsC72Aa7ME1zBUEvnDxTnEotA+5iH9warcg3?= =?us-ascii?Q?cSCLeLio28o57SDZOI/Z81dTFCuy+NmrDhfixcBAMQ2+VybQmz0aCAZcYenE?= =?us-ascii?Q?Ru9KjKflti4EzitZl00chIiKSyNbi2wFhC1ExjG5quSzrYYDp3/j64XM8sQE?= =?us-ascii?Q?qy9Cmx0QZe/gX2zHR6LmQryTbQ3Xkh4JhWM+bNzC/LGSpyY+nR0PT2khHZlB?= =?us-ascii?Q?tRtkn0T9g6uvKwHKXh/nGXuCtC2YSm1Qjw7ZIJvkAuIMQPAV9f4/Ij1XD9SJ?= =?us-ascii?Q?tie571YpEHkn/xS5NRGA+Fk0+OHmmgQp648gRpJyqgkIWJCp+KW4lpaY9XBs?= =?us-ascii?Q?U5oIA0kpsvcXTq5LpH+OQEufFteqK5d6+nTRyGqQzfKgPzKNs3qIGqcY9QKy?= =?us-ascii?Q?QF2W1AW6akbZSvrZhwj7ZZHmBROyNG0iLtzNJ3i09kn85Zflvs7Z+4Z5duJF?= =?us-ascii?Q?3L0MxDOKb1tQRe8ffG4RP8d9musM5N/Bs9uEHLUsmLqvX0INXfb5rizIjCL5?= =?us-ascii?Q?D6CQLE7cof8g00xY8m/J1uYeMOYQBRwIpDlMiGLsVplwb+6we4AnA+BY8dqv?= =?us-ascii?Q?vUzy6LWeNHY6QOnFpwyPawxlOrkwGhNclSlMXywDi+KdLVHG5AHfatjTKa0v?= =?us-ascii?Q?RhVGwIgC7akeQ6Z6ADJVy6UVVwPZYHT0YKMLnFlCE6CfuWHpmvUTCR43akbF?= =?us-ascii?Q?B0wPkqJuzmEZdED6QTIsmg1rGviehm0ALMur4O9x5fdvWr9+5d0HCkOsh9j9?= =?us-ascii?Q?9nIF4/oFlSnh7rdM53VpDIN5OdcgG1WmoODTbKfdoYJGJ8RW4EeIOW0SiYXy?= =?us-ascii?Q?N3N6ExqzY/kgA1ECdFNbDCCxofEagMYSrRstJ6bUSyr0PWwnh2FmnurRyO0H?= =?us-ascii?Q?ywQ9zL5xbObPvtOZBp7D0mmAY9ZvnR0LHBzU2XBYYwGWLNnWtbEOY2QHNDu/?= =?us-ascii?Q?tCFrbjcKI6vkpSBmiHoxmAIMLowltNZEvprPObNBEMAxQmZbxHRgBHnNhI9G?= =?us-ascii?Q?0A24GZvl3U6n9tBta8dvP4VCDHxzJ67Ti77oPRgpoT2vE2UhQRj6bWWBcs5M?= =?us-ascii?Q?soZpt8aX3fa5at2y9SfJXkCRPlIfQsbgHOmi6V3YM2lkIs+7Id36KWnivXP0?= =?us-ascii?Q?ZQ=3D=3D?= X-MS-Exchange-CrossTenant-Network-Message-Id: 80c5f1a3-f6be-44d9-c47e-08da8739f9b4 X-MS-Exchange-CrossTenant-AuthSource: MWHPR11MB1629.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Aug 2022 08:07:10.2199 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: jQw/ndaTv2lcfwrC+HFl2k6XTmIF2w/iviMxXISgfJRsL+ZxyvbWlYhUyJ9nkqyf5ZeTYIdAajCvQrPmmGTELEwAFZuGsIZjCkYbtyxvcKU= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR11MB3269 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Fri, Aug 26, 2022 at 12:35:16PM +0530, Jerin Jacob wrote: > On Thu, Aug 25, 2022 at 8:56 PM Kevin Laatz wrote: > > > > From: Anatoly Burakov > > > > Currently, there is no way to measure lcore poll busyness in a passive way, > > without any modifications to the application. This patch adds a new EAL API > > that will be able to passively track core polling busyness. > > > > The poll busyness is calculated by relying on the fact that most DPDK API's > > will poll for packets. Empty polls can be counted as "idle", while > > non-empty polls can be counted as busy. To measure lcore poll busyness, we > > simply call the telemetry timestamping function with the number of polls a > > particular code section has processed, and count the number of cycles we've > > spent processing empty bursts. The more empty bursts we encounter, the less > > cycles we spend in "busy" state, and the less core poll busyness will be > > reported. > > > > In order for all of the above to work without modifications to the > > application, the library code needs to be instrumented with calls to the > > lcore telemetry busyness timestamping function. The following parts of DPDK > > are instrumented with lcore telemetry calls: > > > > - All major driver API's: > > - ethdev > > - cryptodev > > - compressdev > > - regexdev > > - bbdev > > - rawdev > > - eventdev > > - dmadev > > - Some additional libraries: > > - ring > > - distributor > > > > To avoid performance impact from having lcore telemetry support, a global > > variable is exported by EAL, and a call to timestamping function is wrapped > > into a macro, so that whenever telemetry is disabled, it only takes one > > additional branch and no function calls are performed. It is also possible > > to disable it at compile time by commenting out RTE_LCORE_BUSYNESS from > > build config. > > > > This patch also adds a telemetry endpoint to report lcore poll busyness, as > > well as telemetry endpoints to enable/disable lcore telemetry. A > > documentation entry has been added to the howto guides to explain the usage > > of the new telemetry endpoints and API. > > > > Signed-off-by: Kevin Laatz > > Signed-off-by: Conor Walsh > > Signed-off-by: David Hunt > > Signed-off-by: Anatoly Burakov > > > > --- > > v3: > > * Fix missed renaming to poll busyness > > * Fix clang compilation > > * Fix arm compilation > > > > v2: > > * Use rte_get_tsc_hz() to adjust the telemetry period > > * Rename to reflect polling busyness vs general busyness > > * Fix segfault when calling telemetry timestamp from an unregistered > > non-EAL thread. > > * Minor cleanup > > --- > > > diff --git a/meson_options.txt b/meson_options.txt > > index 7c220ad68d..725b851f69 100644 > > --- a/meson_options.txt > > +++ b/meson_options.txt > > @@ -20,6 +20,8 @@ option('enable_driver_sdk', type: 'boolean', value: false, description: > > 'Install headers to build drivers.') > > option('enable_kmods', type: 'boolean', value: false, description: > > 'build kernel modules') > > +option('enable_lcore_poll_busyness', type: 'boolean', value: true, description: > > + 'enable collection of lcore poll busyness telemetry') > > IMO, All fastpath features should be opt-in. i.e default should be false. > For the trace fastpath related changes, We have done the similar thing > even though it cost additional one cycle for disabled trace points > We do need to consider runtime and build defaults differently, though. Since this has also runtime enabling, I think having build-time enabling true as default is ok, so long as the runtime enabling is false (assuming no noticable overhead when the feature is disabled.) /Bruce