From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by dpdk.org (Postfix) with ESMTP id 95B89ADF5 for ; Mon, 23 Feb 2015 15:36:20 +0100 (CET) Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga103.jf.intel.com with ESMTP; 23 Feb 2015 06:30:42 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.09,631,1418112000"; d="scan'208";a="531458196" Received: from irsmsx101.ger.corp.intel.com ([163.33.3.153]) by orsmga003.jf.intel.com with ESMTP; 23 Feb 2015 06:27:19 -0800 Received: from irsmsx109.ger.corp.intel.com ([169.254.13.103]) by IRSMSX101.ger.corp.intel.com ([163.33.3.153]) with mapi id 14.03.0195.001; Mon, 23 Feb 2015 14:36:16 +0000 From: "Jastrzebski, MichalX K" To: Thomas Monjalon , "Wodkowski, PawelX" Thread-Topic: [dpdk-dev] [PATCH v5 0/3] new headroom stats library and example application Thread-Index: AQHQTFE89sKNNirwW0yb77uFABjvv5z5r4bQgAR0TYCAAC5q0A== Date: Mon, 23 Feb 2015 14:36:16 +0000 Message-ID: <60ABE07DBB3A454EB7FAD707B4BB1582138EB996@IRSMSX109.ger.corp.intel.com> References: <1424191340-26451-1-git-send-email-pawelx.wodkowski@intel.com> <20150219143334.GH24069@hmsreliant.think-freely.org> <60ABE07DBB3A454EB7FAD707B4BB1582138EB174@IRSMSX109.ger.corp.intel.com> <3091068.GKVt00SAyB@xps13> In-Reply-To: <3091068.GKVt00SAyB@xps13> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [163.33.239.180] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] [PATCH v5 0/3] new headroom stats library and example application X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Feb 2015 14:36:21 -0000 > -----Original Message----- > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] > Sent: Monday, February 23, 2015 12:46 PM > To: Wodkowski, PawelX > Cc: dev@dpdk.org; Jastrzebski, MichalX K; Neil Horman > Subject: Re: [dpdk-dev] [PATCH v5 0/3] new headroom stats library and > example application >=20 > 2015-02-20 15:46, Jastrzebski, MichalX K: > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Neil Horman > > > On Thu, Feb 19, 2015 at 01:18:41PM +0100, Pawel Wodkowski wrote: > > > > Hi community, > > > > I would like to introduce library for measuring load of some arbitr= ary > jobs. > > > > It can be used to profile every kind of job sets on any arbitrary e= xecution > unit > > > > or tasking library. > > > > > > > > In provided l2fwd-headroom example I demonstrate how to use this > library to > > > > select optimal rx burst poll time. Jobs are selected by using exist= ing > rte_timer > > > > library calls. This example does no limit possible schemes on which= this > > > > library can be used. > > > > > > > > Pawel Wodkowski (3): > > > > librte_headroom: New library for checking core/system/app load > > > > examples: introduce new l2fwd-headroom example > > > > MAINTAINERS: claim responsibility for headroom library and exampl= e > app > > > > > > I'm sorry but I still fail to see how this is a particularly useful l= ibrary. It > > > clearly works fine, but it composes an application event loop in its = own > > > terms, > > > and measures stats based on that. While thats ok, any application is > already > > > going to have to write its own event loop, and can makethe same > > > measurements > > > synchnously within that loop, using alot less code to optimize its po= lling > time. > > > > > > In other words, I think this is one of those cases where this library= is > > > probably somewhat useful for anyone who just wants to write an > application > > > in > > > terms the semantics exposed by this library, but not at all useful fo= r > anyone > > > else. I'd personally rather not have the extra code to maintain here= . > > > > > > Stephen just gave a presentation at netdev about some of the > performance > > > optimization measurements Brocade did with DPDK and how they fine > tuned > > > their > > > environment. One of the big take aways for me was that making time > based > > > measurements (especially if it was using the tsc), created cpu stalls= that > > > skewed the measurements, and so the best optimizations they made > avoided > > > time > > > measurements, opting instead for packet count metrics. > > > > > > Neil > > > > Hi Neil, > > > > I think this library offers something quite useful probably not for eve= ryone, > > but for many people that use DPDK, and it is measuring quite accurately= , > > how many spare cycles a CPU have after executing any serial tasks (as y= ou > will know). > > If you look at two places in example application: main_loop() > > and l2fwd_fwd() functions, you will see two possible approach there, bu= t > > this is not limited to that. You can even nest headroom objects and > measure > > process time of particular packets type. > > Of course, this will add an overhead due to the measurements, > > but that time is also measured, so any user can know what is the relati= ve > > time "wasted" for measuring all this. > > If time delays are measured in bigger timestamps, are handled reliably, > > the cost of measuring will be low. > > I find this quite similar to the power library case. I would say that l= ibrary is > not useful > > for every application, but there are several cases where it can be > > (as demonstrated with l3fwd-power app). > > > > About your last bit, not sure if I understood it right, but in case of = the > included sample app, > > the main measurement to see if we are overusing a CPU is the packet cou= nt > > in a queue (in this case RX queue), and I believe this should be used f= or > other apps, > > especially in those that use a pipeline model, where queues and rings a= re > the key part. > > > > As a final point, last week (12th of February), there was a request for= a > tool/library like this > > from a user in the mailing list (Ilan Borenshtein), which indicates tha= t this > would be useful > > (probably not just for him, but for others). It probably could be achie= ved by > the user > > by adding their own code, but I believe this library would be a good-to= - > have, > > in case a user is looking for an easy way to calculate the exposed abov= e. > > Let us give the users an example of this method and we will expand it w= ith > more > > advanced application that may show capabilities of dynamic load scaling > based on headroom library measurement. >=20 Hi Thomas, > I wonder how this library is related to DPDK. > I'm not against its integration, though the question must be asked. > DPDK is a set of libraries. What kind of library fit with DPDK goals and > deserve to be integrated? >=20 I think this library fits into dpdk goals, because it is simple and optimiz= ed for fast packet processing. The library provides an easy way for existing DPDK users to modify their ap= plications to measure available CPU headroom. If we integrate it, users will verify it and decide what else they need and= we will implement this. > I don't know whether it's related but nobody acknowledged this patchset. We were waiting for Neil's final comments. He did not mention anything else= since the last time.=20 When Pawel sends the next version with the copyright date corrected, Pablo = will ack this. >=20 > I also feel that the name of this library is a bit too vague. Some people > were asking first what means "headroom". It's actually for CPU headroom > monitoring. > What about "cpuheadroom", "cpuheadroomstat", "jobstat"? I think we can change the name to "cpuheadroom" as it describes more clear = this library. >=20 > Last comment, less important: as many of your colleagues, you don't pay > attention to the copyright dates. I'm pretty sure this code was not writt= en > in 2010. So why claiming it? Sorry, we will fix this of course and pay more attention now. Best regards Michal