From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 36841A0543; Tue, 4 Oct 2022 13:57:34 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0B67E40DFB; Tue, 4 Oct 2022 13:57:34 +0200 (CEST) Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by mails.dpdk.org (Postfix) with ESMTP id 1B09540DDC for ; Tue, 4 Oct 2022 13:57:30 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1664884651; x=1696420651; h=date:from:to:cc:subject:message-id:references: content-transfer-encoding:in-reply-to:mime-version; bh=pYkjj5aFzFhRGJrLlYXsJNk3AFP4AzJYdtWub44RGcA=; b=IzdznQj/d+ZfFz5X9zvlC00wjCCRhICu0T/UzLaZstAvG2touucan3Im iGOY9gVcyCMYk4afaRRO3tKtAnRxgNyY6rSzXVDKHoVQRvJtsvdrZ3077 FGPvaxBfSc65N7D6fzF8vquoYg9pc1b7w6xHJG+Hd/jQd04D0rj4G4A7m 2Q3cZtRnz+MJS61cxEDkinLIKtymZnGrSmwG58wRz6k/6Enw9HtwrIrfT /lnvYQ6KpWpb/i6ZiN5v2UwHNskS2k6fF6PHFEllataUmgPxo7JeNFaZt 1rRi7WeYQ33umuwxhLmU2Ql7y+x5V/E7HOHBoQ+JqIvBwgay7e3XRXo1X w==; X-IronPort-AV: E=McAfee;i="6500,9779,10489"; a="283258526" X-IronPort-AV: E=Sophos;i="5.93,157,1654585200"; d="scan'208";a="283258526" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Oct 2022 04:57:30 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10489"; a="749347135" X-IronPort-AV: E=Sophos;i="5.93,157,1654585200"; d="scan'208";a="749347135" Received: from fmsmsx602.amr.corp.intel.com ([10.18.126.82]) by orsmga004.jf.intel.com with ESMTP; 04 Oct 2022 04:57:29 -0700 Received: from fmsmsx611.amr.corp.intel.com (10.18.126.91) by fmsmsx602.amr.corp.intel.com (10.18.126.82) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Tue, 4 Oct 2022 04:57:28 -0700 Received: from fmsmsx611.amr.corp.intel.com (10.18.126.91) by fmsmsx611.amr.corp.intel.com (10.18.126.91) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Tue, 4 Oct 2022 04:57:28 -0700 Received: from FMSEDG603.ED.cps.intel.com (10.1.192.133) by fmsmsx611.amr.corp.intel.com (10.18.126.91) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31 via Frontend Transport; Tue, 4 Oct 2022 04:57:28 -0700 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (104.47.55.104) by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2375.31; Tue, 4 Oct 2022 04:57:27 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=UEmUBx3Q1qSLI5CvuhhGS0GLHEwb61LMfS+p5mtlopf3QMwi62Jk7g0LDHxcJQnG5+2elG1J35TL5Ll9ScFQtb1I0bqHaMRgSpZB4KnV0ZlWHDWiI59OwjO0QsJfa+kW/81K2/Md3XYlc2YXDLli6M/ROmqk4WR92FnFS2cSeQYOqx8dksDF38v+u9hLupoT8ExhLwH7XPsOnbJ4oxk3JMtEUJlK+c0JwDmtarwTH4rEe2KqHCr90SFqjJRbgtSyB0t0iBSjFR2SoP4KObhRXgMM1ZK6BCLwls+kPyvW/IC4XOYxXW7Grxzp7xYUmUfDc6Uhq1PM6tTU2g6JqKFk0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=1IRyzSP/LRpOgLix9jKM5bXL/mAYf8opo/X0NoRITC0=; b=K3oe+OlGsvqUQ6tzcnDPHdwZYxrkxeshNZIBR544MPKk64i1TB0b6oIp0cquEQDk6vZVtuIW4KBrRCPjQYLWzAxZjY03zZYiINjeilXxbAhl5eMfzbes7LQedySBkBEsHPdOx8TOK8m8nmVQomJzn3nPMgIQGOEZmlHbbIuspr+HCQ69SjRrkRPt3NXjWsq/ENRL66QiqyYYW7d6UEvXq3Ph9dla9/XpW1I9BovLx5pIEHE83vfVZD2j51/Yq+cBAOtos6wgebtXeaOCIX/oYzEkc500C6kVbsCo2Wxv+mCzmjDmBhre58nUnSzO9BQkRu0c86FICqiM39D0mcXrBw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from MWHPR11MB1629.namprd11.prod.outlook.com (2603:10b6:301:d::21) by SA2PR11MB5081.namprd11.prod.outlook.com (2603:10b6:806:118::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5676.24; Tue, 4 Oct 2022 11:57:25 +0000 Received: from MWHPR11MB1629.namprd11.prod.outlook.com ([fe80::5582:9796:3aaa:aa1]) by MWHPR11MB1629.namprd11.prod.outlook.com ([fe80::5582:9796:3aaa:aa1%12]) with mapi id 15.20.5676.032; Tue, 4 Oct 2022 11:57:25 +0000 Date: Tue, 4 Oct 2022 12:57:13 +0100 From: Bruce Richardson To: Morten =?iso-8859-1?Q?Br=F8rup?= CC: Mattias =?iso-8859-1?Q?R=F6nnblom?= , "Konstantin Ananyev" , Kevin Laatz , Konstantin Ananyev , , , Conor Walsh , David Hunt , Nicolas Chautru , Fan Zhang , "Ashish Gupta" , Akhil Goyal , Fengchengwen , Ray Kinsella , "Thomas Monjalon" , Ferruh Yigit , Andrew Rybchenko , Jerin Jacob , Sachin Saxena , "Hemant Agrawal" , Ori Kam , "Honnappa Nagarahalli" , Mattias =?iso-8859-1?Q?R=F6nnblom?= Subject: Re: [PATCH v7 1/4] eal: add lcore poll busyness telemetry Message-ID: References: <24c49429394294cfbf0d9c506b205029bac77c8b.1657890378.git.anatoly.burakov@intel.com> <20220914092929.1159773-1-kevin.laatz@intel.com> <20220914092929.1159773-2-kevin.laatz@intel.com> <9a6fec15f9684d21bb4730596cceacff@huawei.com> <4af4b5d6-57a9-39a4-2197-a2acdc57156b@intel.com> <98CBD80474FA8B44BF855DF32C47DC35D87386@smartserver.smartshare.dk> Content-Type: text/plain; charset="iso-8859-1" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC35D87386@smartserver.smartshare.dk> X-ClientProxiedBy: LO4P123CA0001.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:150::6) To MWHPR11MB1629.namprd11.prod.outlook.com (2603:10b6:301:d::21) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MWHPR11MB1629:EE_|SA2PR11MB5081:EE_ X-MS-Office365-Filtering-Correlation-Id: 1c7d5a3c-f85b-4c8c-a725-08daa5ff9a37 X-LD-Processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: lvsVtumrgyo5zYXYMvcmKt1J3EVtsuLy1X4blp9fGwdcskapsqbLOSTv+HpXKQ5bmC6OBsqVdnfwMPqwzAIo1xB+E8Cl3TDlEq+ToDP6mPe9nup8CyoZ5D26+99dc389kpULU8g/kYBxwWLo3CmQfuF3BDA1ocPFr2BZpYTVGHmDFKp3Gn72cSPiJeoBsdpZhF5H0ACKfALXyzSd35XuWtk6Kv/AHKJnxD9QxfCXCMOhtxYHCjsswQG2i5fTu5wwvGIebLhsvuKR8xtI/HyD9RjerYpDiuS6UFEPbR6mfF1KU/8GcCKbthr2AOtfM2WTM25NBDPZFBzjZYCXQN8EXYO2cHQoI8qv4Hcs1PBY9ZeSJWHcCNYVEWLzVY/eUCKJes9Zss60W5jlG3zCFToxRniueuQJ6BmxgqRizHqtaKMofu7B7HiNgctPy2RxglaMxPBCKohlNG3HCItI53TNdRvrMwi6E6Oiq08CZh/eHoID26G4Zm8o89NkOSzX6DfNDsa+737TW4JBto7GyrW0IUwCHdgEenHo0Xqw7rF8/BCXQO/3qGNnYOqxULkK1VF+UelEWa9qvDopMaPfrVE4sO4l634olaaDW7jnrjzr5bZJXUCpPXS3XzbXaIIojP36G3qfGVXAuzy7Qp2RwcFlPC8zfPyfRKkjomGzfprND6Xd+C9JRlQ8AU+Wb5fI+umCyF9oYJ66Oes6HNkM58r4iw== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MWHPR11MB1629.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230022)(376002)(39860400002)(366004)(396003)(346002)(136003)(451199015)(8936002)(41300700001)(66574015)(2906002)(38100700002)(478600001)(83380400001)(6916009)(66946007)(6666004)(186003)(66556008)(66476007)(5660300002)(54906003)(6512007)(26005)(4326008)(8676002)(6486002)(86362001)(7416002)(316002)(6506007)(44832011)(82960400001); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?iso-8859-1?Q?J34Q/Sqt5cfbd8swwI5coXJ3QFyPx2ddR2le2RukJP+X19mNsuKgkREP6+?= =?iso-8859-1?Q?odhsP9t2gFYB4XzOP8J2v0/Rrk3W/vzEXQDZ2f34+x80+wJcn7Jf5mab2O?= =?iso-8859-1?Q?GncAcfRGRusCxk9U4Q7P9bztZvY6gFNy7bxtjyvVR3bCSyEI9PIN4F6vsf?= =?iso-8859-1?Q?Aha6UZRcPNXoF0WXvlH1bBkxOWAVVSp6j/I96yWIRiraIG/r5EitihZL3+?= =?iso-8859-1?Q?omeVJR0ZuCXmjGreK66oOh4mr1tXZGM8u7ywUw1BxnzUGNn1cuiL/QEJvk?= =?iso-8859-1?Q?PcDbhQifAtbicNvc366kw11fGD+gUutYktYpY6whkAA654412iIypKljZe?= =?iso-8859-1?Q?zT2/XSyg6W46asLM03ngdlvxjKiuB0kKd2NY7hdOTwaYQf3vRbWEBfbJc9?= =?iso-8859-1?Q?5aYP/+ymGpDhssPIRgneceSiZ/uhRkHaI1M8GLrSLpC+XzdqmlE4AjTaVe?= =?iso-8859-1?Q?JlV4lol/uvMEopTUgAnevH6Sf3IGN0CZ5XD2lgouPODWcrvAMBgdbjykfi?= =?iso-8859-1?Q?uBIGF9TqOhStN7pNnEp+l3j79LVswA0WpkQnRTAT8d/vqqaiE56aehhFPl?= =?iso-8859-1?Q?Rk9TrG2zFAykCm+KdbJly5+EIOO4cwvm8RgdBQavIzTtDXrMk92+vD6leX?= =?iso-8859-1?Q?9qdck2UB1uty94Gxr8YPZ1IKKB7bfaqoJ/Nncf1oN9ZwykM1td/B7kjFtQ?= =?iso-8859-1?Q?qXdPd6aZZlVu7QXa5U6CZpnJOvwciTdmfsyJWs4clzfmu9yXhVbMvlr9lj?= =?iso-8859-1?Q?AawyXmg8ZGqbqIyhx7n0ak299+wrQKJiQva7c6/fywKx9jMLCD4/A2wkkU?= =?iso-8859-1?Q?oMBNA0YuNczIoFD7XlXmgX2mNvuEEz6NHmtVQ6odm9L+Zo3f/gkDDIrLhX?= =?iso-8859-1?Q?docEk+8xzNhoyNgFGMRPKt1pn23UlYuDxBB5iubdIJxL7KW0XqGiNmj8W0?= =?iso-8859-1?Q?LloE9lFXTCBm6MZBc+s11B1gjGullBr9jxjRtHg4oLoFN2MfL2aDCOSv46?= =?iso-8859-1?Q?Gb1VyoZ/rO5olsokhBgC3Lns9GyEJhvZU+l/TeGknlATgmxAXspodzbQmA?= =?iso-8859-1?Q?bWCNEJp0H5ZzaL2absG3lYdqmWCoUkLhX1VOnrweGUI3Z72ZKh8UjKF9fT?= =?iso-8859-1?Q?70cKF+bGrVwpF9r3Lslnxutt5fTM70sKEus4RSLh2AAK13jrs1eO4wBaVh?= =?iso-8859-1?Q?dp/SA/VEkBCUTqesw4woLQHBjeG61a/90rnDXK+sG9aNfJpr4MnsEK5KEo?= =?iso-8859-1?Q?HdsBMVM8bn06grqH8rNpd00gRSYQztWgd6rfdkgbXSyH0MgE1fNp1zF/kN?= =?iso-8859-1?Q?zh1cwWajhho1DWHJ5P//HgVmMgOVaXZ1EqWxXCnP5p2Tk2JqkJ/+xpSPCS?= =?iso-8859-1?Q?74MZaAA8+LUlTAGJ9nf80Yii1VSnrNL00Vpt7eCZ9v2FGhBaZq4hPK/aTq?= =?iso-8859-1?Q?CGe9a47YgtefaudedaigAeIfFoatouBC2UJKpAGLH+BAw1JNJOyQ8CI+r9?= =?iso-8859-1?Q?JHkVXlPklmYYVhv5CnQSSF/ofOxbOuIoK0NwhGGDabGcX+GLTVjNlnBmde?= =?iso-8859-1?Q?/P1aGw9lD+9BwVB2d1f8fxR+Kae0TIeB/Nd6ypcr3U3NuNW26oyfFGgHWK?= =?iso-8859-1?Q?8uy9ootvFV7FBGAmmbyN8Obg4BbSd+kXl4rPzscJVWtxHLC/RtolHTROiQ?= =?iso-8859-1?Q?ZUczfq3aqDFxlijfw8Q=3D?= X-MS-Exchange-CrossTenant-Network-Message-Id: 1c7d5a3c-f85b-4c8c-a725-08daa5ff9a37 X-MS-Exchange-CrossTenant-AuthSource: MWHPR11MB1629.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Oct 2022 11:57:25.2834 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: aGUleA/8nJcqi+WyeqP97BeGP9LLPtVb5Ga5cQY55q4kqIYvGLTVDkGGNW/XDogiP89vQB4TEVq9xC+YWuH7zOd3AuJwVpJVBEiflQHdkIM= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA2PR11MB5081 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Tue, Oct 04, 2022 at 11:15:19AM +0200, Morten Brørup wrote: > > From: Mattias Rönnblom [mailto:hofors@lysator.liu.se] > > Sent: Monday, 3 October 2022 22.02 > > [...] > > > The functionality provided is very useful, and the implementation is > > clever in the way it doesn't require any application modifications. > > But, > > a clever, useful brittle hack is still a brittle hack. > > I think that may be a little harsh here. After all, this is a feature which is build-time disabled and runtime disabled by default, so like many other components it's designed for use when it makes sense to do so. Furthermore, I'd just like to point out that the authors, when doing the patches, have left in the hooks so that even apps, for which the "for-free" scheme doesn't work, can still leverage the infrastructure to have the app itself report the busy/free metrics. > > What if there was instead a busyness module, where the application > > would > > explicitly report what it was up to. The new library would hook up to > > telemetry just like this patchset does, plus provide an explicit API to > > retrieve lcore thread load. > > > > The service cores framework (fancy name for rte_service.c) could also > > call the lcore load tracking module, provided all services properly > > reported back on whether or not they were doing anything useful with > > the > > cycles they just spent. > > > > The metrics of such a load tracking module could potentially be used by > > other modules in DPDK, or by the application. It could potentially be > > used for dynamic load balancing of service core services, or for power > > management (e.g, DVFS), or for a potential future deferred-work type > > mechanism more sophisticated than current rte_service, or some green > > threads/coroutines/fiber thingy. The DSW event device could also use it > > to replace its current internal load estimation scheme. > > [...] > > I agree 100 % with everything Mattias wrote above, and I would like to voice my opinion too. > > This patch is full of preconditions and assumptions. Its only true advantage (vs. a generic load tracking library) is that it doesn't require any application modifications, and thus can be deployed with zero effort. > > I my opinion, it would be much better with a well designed generic load tracking library, to be called from the application, so it gets correct information about what the lcores spend their cycles doing. And as Mattias mentions: With the appropriate API for consumption of the collected data, it could also provide actionable statistics for use by the application itself, not just telemetry. ("Actionable statistics": Statistics that is directly usable for decision making.) > > There is also the aspect of time-to-benefit: This patch immediately provides benefits (to the users of the DPDK applications that meet the preconditions/assumptions of the patch), while a generic load tracking library will take years to get integrated into applications before it provides benefits (to the users of the DPDK applications that use the new library). > > So, we should ask ourselves: Do we want an application-specific solution with a short time-to-benefit, or a generic solution with a long time-to-benefit? (I use the term "application specific" because not all applications can be tweaked to provide meaningful data with this patch. You might also label a generic library "application specific", because it requires that the application uses the library - however that is a common requirement of all DPDK libraries.) > > Furthermore, if the proposed patch is primarily for the benefit of OVS, I suppose that calls to a generic load tracking library could be added to OVS within a relatively short time frame (although not as quick as this patch). > > I guess that the developers of this patch initially thought that it was generic and usable for the majority of applications, and it came as somewhat a surprise that it wasn't as generic as expected. The DPDK community has a good review process with open discussions and sharing of thoughts and ideas. Sometimes, an idea doesn't fly, because the corner cases turn out to be more common than expected. I'm sorry to say it, but I think that is the case for this patch. :-( > I'd actually like to question this last statement a little. I think we in the DPDK community are very good at coming up with theoretical examples where things don't work, but are they really cases that occur commonly in the real-world? I accept, for example, that the "for free" approach would not be suitable for something like VPP which does multiple polls to gather packets before processing, but for some of the other cases I'd question their commonality. For example, a number of objections have focused on the case where allocation of buffers fails and so the busyness gets counted wrongly. Are there really (many) apps out there where running out of buffers is not a much more serious problem than incorrectly reported busyness stats? I'd also say that, in my experience, the non-open-source end-user apps tend very much to use DPDK based on the style of operation given in our DPDK examples, rather than trying out new or different ways of working. (Maybe others have different experiences, though, and can comment). I also tend to believe that open-source software using DPDK probably shows more variety in how things are done, which is not representative of a lot of non-OSS users of DPDK. Regards, /Bruce