From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 45ABAA0032; Sat, 16 Jul 2022 00:14:00 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 23C4240A87; Sat, 16 Jul 2022 00:14:00 +0200 (CEST) Received: from smartserver.smartsharesystems.com (smartserver.smartsharesystems.com [77.243.40.215]) by mails.dpdk.org (Postfix) with ESMTP id 522BA40141 for ; Sat, 16 Jul 2022 00:13:59 +0200 (CEST) X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Subject: RE: [PATCH v1 1/2] eal: add lcore busyness telemetry Date: Sat, 16 Jul 2022 00:13:52 +0200 Message-ID: <98CBD80474FA8B44BF855DF32C47DC35D871CC@smartserver.smartshare.dk> In-Reply-To: <24c49429394294cfbf0d9c506b205029bac77c8b.1657890378.git.anatoly.burakov@intel.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [PATCH v1 1/2] eal: add lcore busyness telemetry Thread-Index: AdiYTJmZ7BVKfv4oQhKijV2m/y3LPAASDVxw References: <24c49429394294cfbf0d9c506b205029bac77c8b.1657890378.git.anatoly.burakov@intel.com> From: =?iso-8859-1?Q?Morten_Br=F8rup?= To: "Anatoly Burakov" , , "Bruce Richardson" , "Nicolas Chautru" , "Fan Zhang" , "Ashish Gupta" , "Akhil Goyal" , "David Hunt" , "Chengwen Feng" , "Kevin Laatz" , "Ray Kinsella" , "Thomas Monjalon" , "Ferruh Yigit" , "Andrew Rybchenko" , "Jerin Jacob" , "Sachin Saxena" , "Hemant Agrawal" , "Ori Kam" , "Honnappa Nagarahalli" , "Konstantin Ananyev" Cc: "Conor Walsh" X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org > From: Anatoly Burakov [mailto:anatoly.burakov@intel.com] > Sent: Friday, 15 July 2022 15.13 >=20 > Currently, there is no way to measure lcore busyness in a passive way, > without any modifications to the application. This patch adds a new = EAL > API that will be able to passively track core busyness. >=20 > The busyness is calculated by relying on the fact that most DPDK API's > will poll for packets. This is an "alternative fact"! Only run-to-completion applications polls = for RX. Pipelined applications do not poll for packets in every pipeline = stage. > Empty polls can be counted as "idle", while > non-empty polls can be counted as busy. To measure lcore busyness, we > simply call the telemetry timestamping function with the number of > polls > a particular code section has processed, and count the number of = cycles > we've spent processing empty bursts. The more empty bursts we > encounter, > the less cycles we spend in "busy" state, and the less core busyness > will be reported. >=20 > In order for all of the above to work without modifications to the > application, the library code needs to be instrumented with calls to > the lcore telemetry busyness timestamping function. The following = parts > of DPDK are instrumented with lcore telemetry calls: >=20 > - All major driver API's: > - ethdev > - cryptodev > - compressdev > - regexdev > - bbdev > - rawdev > - eventdev > - dmadev > - Some additional libraries: > - ring > - distributor >=20 > To avoid performance impact from having lcore telemetry support, a > global variable is exported by EAL, and a call to timestamping = function > is wrapped into a macro, so that whenever telemetry is disabled, it > only > takes one additional branch and no function calls are performed. It is > also possible to disable it at compile time by commenting out > RTE_LCORE_BUSYNESS from build config. Since all of this can be completely disabled at build time, and thus has = exactly zero performance impact, I will not object to this patch. >=20 > This patch also adds a telemetry endpoint to report lcore busyness, as > well as telemetry endpoints to enable/disable lcore telemetry. >=20 > Signed-off-by: Kevin Laatz > Signed-off-by: Conor Walsh > Signed-off-by: David Hunt > Signed-off-by: Anatoly Burakov > --- >=20 > Notes: > We did a couple of quick smoke tests to see if this patch causes > any performance > degradation, and it seemed to have none that we could measure. > Telemetry can be > disabled at compile time via a config option, while at runtime it > can be > disabled, seemingly at a cost of one additional branch. >=20 > That said, our benchmarking efforts were admittedly not very > rigorous, so > comments welcome! This patch does not reflect lcore business, it reflects some sort of = ingress activity level. All the considerations regarding non-intrusiveness and low overhead are = good, but everything in this patch needs to be renamed to reflect what = it truly does, so it is clear that pipelined applications cannot use = this telemetry for measuring lcore business (except on the ingress = pipeline stage). It's a shame that so much effort clearly has gone into this patch, and = no one stopped to consider pipelined applications. :-(