From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C22DDA0032; Fri, 16 Sep 2022 16:11:00 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A758C4021D; Fri, 16 Sep 2022 16:11:00 +0200 (CEST) Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by mails.dpdk.org (Postfix) with ESMTP id 7364240156 for ; Fri, 16 Sep 2022 16:10:58 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1663337458; x=1694873458; h=message-id:date:subject:from:to:cc:references: in-reply-to:mime-version; bh=Tzl90e2HrGJ0ug4SS93uha+NplJYTJOtcxKOkfdWZqY=; b=L8SJHvXRdDzA2bOt0a7DFjwpMCAtZDe8nxA5EJ5gkP54C9XkTZcdHd7N QhiLEMBfXjJ/M98bHlqKCeJbysxsZ9Wn9hSv80EycsbDAXOsgnID0h7y3 86+iY6zwZX6ssCsUEfjDFb2J3fyTK1b/uhBKTzlZI6nSybRWf1p9nBp9P i7ZR+y4hLif1JGY2OMEsIGNDrpe0WoOz4k8loESSALpLoDcIX8NEzuteX IDeT/Rh8HaHxST66tGjOXDufO/YtWRfGzn+O20BxW2/biwpxHi3rkI1a0 JDubl65wptmDWSrJbX3EG3Mxwiu95CO0AyTUbBWaBvsL/+5K0VIyQoirx A==; X-IronPort-AV: E=McAfee;i="6500,9779,10472"; a="300359950" X-IronPort-AV: E=Sophos;i="5.93,320,1654585200"; d="scan'208,217";a="300359950" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Sep 2022 07:10:37 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,320,1654585200"; d="scan'208,217";a="760061697" Received: from orsmsx603.amr.corp.intel.com ([10.22.229.16]) by fmsmga001.fm.intel.com with ESMTP; 16 Sep 2022 07:10:36 -0700 Received: from orsmsx607.amr.corp.intel.com (10.22.229.20) by ORSMSX603.amr.corp.intel.com (10.22.229.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Fri, 16 Sep 2022 07:10:36 -0700 Received: from orsedg603.ED.cps.intel.com (10.7.248.4) by orsmsx607.amr.corp.intel.com (10.22.229.20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31 via Frontend Transport; Fri, 16 Sep 2022 07:10:36 -0700 Received: from NAM04-MW2-obe.outbound.protection.outlook.com (104.47.73.170) by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2375.31; Fri, 16 Sep 2022 07:10:36 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RxSgowZ7s8PB/xh17NVaeNWFkKf5WBcV0gd759PTF+n35Lspk+sFqzAiZesQHjeMKNZXwYDbbKN1h5dWtwuh1W92YX8TUbazHzI+YsfyL/qX8n05jTUAwz8JMr14vN0FHCZyAypqPzYuEuuggKKrxYjrbLwElgHkQdtDBKJdaSm5my9tDHnt17UEIO6MOBfg4LvbPmzMg2DSjEbWRVJGZLCL8vhWUQ2vyrt1TlaReW6+pa0CF5MHr4S8PpOlRAG3o2D/9EUh5F6E1Hc9RYbuDsvue/ELHaeA8KU/ztdqBNybr0MeG/qYRC3ric8/X3BqUc1VjKTof466vBlV7HpqZQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=HK1Lmd5DVC0wmIvbVYVPBBpjWuJVbr/YJ3BEVaUnqlw=; b=I0nPWjbgP4E4g1KtKgZOSGxJArOVb/9oLUgVgaCdQ//ouFs1XLKOuzXLBlux/ltOzRj3Bl+pSDEMDWK26N8atucXkLF878/KSOiWmyfXaypLPLRzhfZXLT+6ZMtReZ8sygLdXGstTfsfomIIhpRZSAYmwla9u88rSqp2Gzt+Zl4rpqrGvp4lyK1cTqa0FXesUNRqnt4HfT9zce7wEZ49WpOmOW/Qw/5aJ1l7khQ92QiVz7Dycmqs4SbcMn9G59PW10/SmUgc/E0zbiVb6SCuxRtHJE2FiEv/6ot7h/beWuuXVBHFYC3xwAKz8kz6O7ahr+wKEazMIB2cSs3L7EV8kg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from MW4PR11MB5872.namprd11.prod.outlook.com (2603:10b6:303:169::14) by SJ0PR11MB4974.namprd11.prod.outlook.com (2603:10b6:a03:2d6::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5632.15; Fri, 16 Sep 2022 14:10:34 +0000 Received: from MW4PR11MB5872.namprd11.prod.outlook.com ([fe80::c17d:f1c9:e958:b5e]) by MW4PR11MB5872.namprd11.prod.outlook.com ([fe80::c17d:f1c9:e958:b5e%6]) with mapi id 15.20.5632.016; Fri, 16 Sep 2022 14:10:34 +0000 Content-Type: multipart/alternative; boundary="------------c0uLnvq78XQXKZrQPNWezHrH" Message-ID: Date: Fri, 16 Sep 2022 15:10:28 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.2.2 Subject: Re: [PATCH v7 0/4] Add lcore poll busyness telemetry Content-Language: en-US From: Kevin Laatz To: Stephen Hemminger CC: , References: <24c49429394294cfbf0d9c506b205029bac77c8b.1657890378.git.anatoly.burakov@intel.com> <20220914092929.1159773-1-kevin.laatz@intel.com> <20220914073359.6c57fbed@hermes.local> <8646ba9f-a6bc-1bd0-0cf5-349268f4032c@intel.com> In-Reply-To: <8646ba9f-a6bc-1bd0-0cf5-349268f4032c@intel.com> X-ClientProxiedBy: LO4P123CA0322.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:197::21) To MW4PR11MB5872.namprd11.prod.outlook.com (2603:10b6:303:169::14) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MW4PR11MB5872:EE_|SJ0PR11MB4974:EE_ X-MS-Office365-Filtering-Correlation-Id: 850d9d75-00fd-485b-f808-08da97ed38b9 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: vyz8NQAeIg87hIaXP+1+M4/BcgzU2BuxOqGJvj70bHf0JECDQzw6wa59UGcRH5qHwC8UtcRMZrVeoVBWU3Uw4enhVMrcN99zBJvVLeFzxkMf5FU2cGqklfppAC2vMVqeBcknV8zAHIw4um9w42bTicI94ci8YFdSjdKaI2WaTA3mTbEDDYeJzqhTsQ2nwomj+kk3seqItFpGX+Es5u7KscDC/Vg2j6cJMNVJST/jRVZT2bSLayVLnmqY8Dab5LE77vNmdJNfoye6rqgQj+GhYLosLksqRZlcwSO9lgYyDLPCjMVrtK/uleDoTwtBJH4h3vVyXLW0Ong7UDbOei61viBTJ8FdgPwnGXO6IiYcvr+jfhneBuvOofTCXvm5y3aaFVc32nKuKWB9fsN7uDnwkcGNLSM5sxU3IdPPuVjeCUM+J1YYyxKt+olxBuVTITk5WHl5+6TIiswoocnIg5fpXdh3uDXDJ8oRD5SKb1vmtBH62FsI8n3gv//75qryyPTfWsxDfO0AvCgxxfpz+vy4lQrzKSeVdQjHcLDGQfKFkTeAzB3bE6QkqkHzWxYkJbkPqOhwGNLaWTXo7e7wQuTeZsg5Yr1ekino28Sq5rzPAG+eZ+TP+X56RHcmWtUDjsX8HEMsZJxcTa3UWx8tZzUbJd1BoZD+gmoal5zZQrKi6ryKd57cwWbEUld9DwTUbYFLx4Cp7I0yxvtvHVjH6IMEnXpxu+UwZ4i8hERcl9n5/Kmqz2/SywiCOSOIOVDCJNPMjoXgsWWwgC57+XN7VHw5NDH1PvcSvseZo7qsc5arotw= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MW4PR11MB5872.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230022)(346002)(39860400002)(136003)(396003)(376002)(366004)(451199015)(53546011)(33964004)(107886003)(36756003)(186003)(6916009)(86362001)(31696002)(6512007)(41300700001)(478600001)(26005)(2616005)(6486002)(5660300002)(44832011)(8936002)(316002)(83380400001)(82960400001)(6506007)(6666004)(66556008)(66476007)(8676002)(4326008)(38100700002)(31686004)(66946007)(2906002)(45980500001)(43740500002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?MGhjSUVRQ0xzVFc5K08xeWZkZGU0Ung4YjVwb1JDdGcyM2pyL3VpbW5Nbndv?= =?utf-8?B?YmRPRUJOOFJxUmY0SnAwazliTmF6a2Nzd0ZtdmFKNGZsYjRTdSt3R0UrMzJu?= =?utf-8?B?TElnZlJqSXU5Y2F0MVVKR2thZi93RTIyYXBSMklndzNGWndMMzNxKzZuZlpK?= =?utf-8?B?cVkxS3VLdjVtdWF6WmdHTGhYSjlYdnlQOWVzYkJFVmxsWFhBb25iMXg3VEFa?= =?utf-8?B?djR0OFVKWjgzRDUzOXZhYnl6bW1senpmOG9CbGJhbWZRUFdtMndwUlEyRlZO?= =?utf-8?B?T1hWenBjcm43WXlJWXo0QnRqZWF4Y3RJS2lxRHpiZG5jalJ0R3F2bXJ4TGNH?= =?utf-8?B?QlpaMi81WXh3RGxWVnpiTDFkY1JmSDdmMXhnbnFHM3pWQTlOeWNSaHJWYUxB?= =?utf-8?B?VEU4OXJrdVpwek1WZlFUQWRqV1BOcTlhejFsOTE1TEJMNU5hc2dCbDVyWHRZ?= =?utf-8?B?STZmSFNJdUJoM1JYd08vY2JkZWFEUEtYbnluaDdjbi9yTHJtK1A4bkVkVy9t?= =?utf-8?B?YkVQMGU2eVpyOWtkT0xNV3QvYzJtQ2hLZjd0TmJtNU1tVEI0MXd5L2oxRlZH?= =?utf-8?B?VmphdWE1R201NDhKUmRld09jaXRsLzJtbUNmU2tTaDRrK1QwTFk3aFhnUWM3?= =?utf-8?B?UGNNSC91VFF2ZTI3WGtNL3JOWEV6SG1IZ085UkVaVHNHbGhJYjRkR21MNU44?= =?utf-8?B?R296K3JKSjA1bFdybHUrMHZRL1Rsc01xY3YzYktBVzF0VHgydHlVdFdWb29R?= =?utf-8?B?QW1NRG9rcExFY1U0Mk9kWGFZbTAvS1UrZGdtdWdMNWlML0tkTG50S25TdVR1?= =?utf-8?B?VWRkOU53YlNGQTJOTk9EYWhsT1ZkdnRLRUQrSmZlNlBaeWdjeWlEdko1OVhL?= =?utf-8?B?SjBZbThEdnkyMjBGdTBUajYwOGhvR3BSdjJnVUdzaDRSMnhnOVp3S293ZThq?= =?utf-8?B?S0NxM1hybEpNL1oxc2cweHdGVWpZN1pCbk4yWVduQU9TRW5mMXh0cmlsV09I?= =?utf-8?B?NXpOaU82c1JxcjZSeS9TV09QUUp3S3M1c1FLcnRVN2pBOHFPVVA2cFhXcEYw?= =?utf-8?B?VlBLcEJFQW1XRFlsZGtGK3FUcUlwTCsxbk5WYUpNMDdtZVlKOUppV2g0VG56?= =?utf-8?B?eE85YXAyN3VhMlA5VnNaN1hRS3E0c3M2cTRXcktEb3JLNDk0dTkzbGNrekgw?= =?utf-8?B?OEJtTExIbnNQYVZXS3NUT0c5ZCtjNnJlc0VCN0FucXIvcEdPSVg0MmpoeXh0?= =?utf-8?B?VVI1MFJCNU5idEhQa0tkMjloZEI3bzdBaVFFY3VsSkVRMU00VHRzYUpPVjEr?= =?utf-8?B?WFFFdmJRRGJuMkFSeXAyaVdiVVJKT1FkUmxEQm9UVENERFJyRi9LTk1CZ24w?= =?utf-8?B?Vmd1L3k2bE93a1VBU25Rb203Vzd2UWR6K3ZabXF5SW5oS1ZiUlg2WVdhV3Rx?= =?utf-8?B?SmpXOWwrWmpkbzZxbjJXSUVOQ0VjOTN2eVo0UHJrRklHWVVBUm1pZTFOejZh?= =?utf-8?B?OFhic0M4bzVoUGNnNDBrRlBBVnN5WStkN0lVeTRyaFBoYkNDcGwvbS9kSEh1?= =?utf-8?B?U0wzU1BtQmtvZnlEZGR0OU44ZWhyQUtZSTRUTDVtelI3M21wa2ZjbVljR3JX?= =?utf-8?B?RFhGVUk0bFJ0NVFPVVk5YUNoLzVwU0xZU2FSY0NKY2MyT29IVlYwSGlGcjJy?= =?utf-8?B?TE5maHdtL3d3ZXRsMk5qUktGODdxT2MzcUVHZXlVYU5kZWtUR2krNCtteUdS?= =?utf-8?B?T3l2MVkrcDZtZDFNRnplQW1BeDJ5UnYyNThjeWhUTW9od0hYaCsybStkUTh1?= =?utf-8?B?dzY4ZEhNUXZJeFpXWHdBN2NsaVVhNzFmNzhuRGw3cUhldDAwNFZpUkNFbTlZ?= =?utf-8?B?eEkwNXlwSDJNeGJJL1I3MjIrVnQ5Qjk4WGtsMmlocVVrV1FOMlNPS3BDYmZv?= =?utf-8?B?MVUyWnFHdnRWVjdlUlcxMzNWMXFGYStuK214bFN1TE9YcU1QdEdrbitVUytM?= =?utf-8?B?UEJxMEJIY2JEL1VhWE42UHFCWjFkR0I1cUxIQkRBVW5maWZ3V3pLeW5mT2pO?= =?utf-8?B?aWF5ZDVMaXowTXhHVTNSRmEzSk5TU3VEWEdTZ1J4dmdwVkRWekVzSURYNmIx?= =?utf-8?B?NnJ3K2w1bE9uUnl4alZ4OU9uQWhxRDIvR2JhcDZHcjhqQmhaTVJEbWQ1Znky?= =?utf-8?B?MFE9PQ==?= X-MS-Exchange-CrossTenant-Network-Message-Id: 850d9d75-00fd-485b-f808-08da97ed38b9 X-MS-Exchange-CrossTenant-AuthSource: MW4PR11MB5872.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Sep 2022 14:10:34.4060 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: QKT6AVKuNpXg+RpGJN1cgEt27B6aWTNafhJNTTDUbGgigJ4cinEUpGPBgjwaLxRdPGVovysCMNzrJX7A1CKBRA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR11MB4974 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org --------------c0uLnvq78XQXKZrQPNWezHrH Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 7bit On 16/09/2022 13:35, Kevin Laatz wrote: > On 14/09/2022 15:33, Stephen Hemminger wrote: >> On Wed, 14 Sep 2022 10:29:25 +0100 >> Kevin Laatz wrote: >> >>> Currently, there is no way to measure lcore polling busyness in a >>> passive >>> way, without any modifications to the application. This patchset >>> adds a new >>> EAL API that will be able to passively track core polling busyness. >>> As part >>> of the set, new telemetry endpoints are added to read the generate >>> metrics. >> How much does measuring busyness impact performance?? >> >> In the past, calling rte_rdsc() would slow down packet rate >> because it is stops CPU pipeline. Maybe better on more modern >> processors, haven't measured it lately. > > Hi Stephen, > > I've run some 0.001% loss tests using 2x 100G ports, with 64B packets > using testpmd for forwarding. Those tests show a ~2.7% performance > impact when the lcore poll busyness feature is enabled vs compile-time > disabled. > Applications with more compute intensive workloads should see less > performance impact since the proportion of time spent time-stamping > will be smaller. > > In addition, a performance autotest has been added in this patchset > which measures the cycles cost of calling the timestamp macro. Please > feel free to test it on your system (lcore_poll_busyness_perf_autotest). > Worth mentioning as well, is that when lcore poll busyness is enabled at compile-time and disabled at run-time, we see *zero *performance impact. --------------c0uLnvq78XQXKZrQPNWezHrH Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: 7bit
On 16/09/2022 13:35, Kevin Laatz wrote:
On 14/09/2022 15:33, Stephen Hemminger wrote:
On Wed, 14 Sep 2022 10:29:25 +0100
Kevin Laatz <kevin.laatz@intel.com> wrote:

Currently, there is no way to measure lcore polling busyness in a passive
way, without any modifications to the application. This patchset adds a new
EAL API that will be able to passively track core polling busyness. As part
of the set, new telemetry endpoints are added to read the generate metrics.
How much does measuring busyness impact performance??

In the past, calling rte_rdsc() would slow down packet rate
because it is stops CPU pipeline. Maybe better on more modern
processors, haven't measured it lately.

Hi Stephen,

I've run some 0.001% loss tests using 2x 100G ports, with 64B packets using testpmd for forwarding. Those tests show a ~2.7% performance impact when the lcore poll busyness feature is enabled vs compile-time disabled.
Applications with more compute intensive workloads should see less performance impact since the proportion of time spent time-stamping will be smaller.

In addition, a performance autotest has been added in this patchset which measures the cycles cost of calling the timestamp macro. Please feel free to test it on your system (lcore_poll_busyness_perf_autotest).

Worth mentioning as well, is that when lcore poll busyness is enabled at compile-time and disabled at run-time, we see zero performance impact.
--------------c0uLnvq78XQXKZrQPNWezHrH--