From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by dpdk.org (Postfix) with ESMTP id 446FB9AEA for ; Mon, 23 Feb 2015 16:56:00 +0100 (CET) Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga103.fm.intel.com with ESMTP; 23 Feb 2015 07:48:05 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.09,631,1418112000"; d="scan'208";a="689482158" Received: from irsmsx107.ger.corp.intel.com ([163.33.3.99]) by orsmga002.jf.intel.com with ESMTP; 23 Feb 2015 07:55:58 -0800 Received: from irsmsx109.ger.corp.intel.com ([169.254.13.103]) by IRSMSX107.ger.corp.intel.com ([169.254.10.35]) with mapi id 14.03.0195.001; Mon, 23 Feb 2015 15:55:56 +0000 From: "Jastrzebski, MichalX K" To: Thomas Monjalon Thread-Topic: [dpdk-dev] [PATCH v5 0/3] new headroom stats library and example application Thread-Index: AQHQTFE89sKNNirwW0yb77uFABjvv5z5r4bQgAR0TYCAAC5q0IAABCcAgAAQAWA= Date: Mon, 23 Feb 2015 15:55:55 +0000 Message-ID: <60ABE07DBB3A454EB7FAD707B4BB1582138EBA65@IRSMSX109.ger.corp.intel.com> References: <1424191340-26451-1-git-send-email-pawelx.wodkowski@intel.com> <3091068.GKVt00SAyB@xps13> <60ABE07DBB3A454EB7FAD707B4BB1582138EB996@IRSMSX109.ger.corp.intel.com> <2047616.cJycthpmiM@xps13> In-Reply-To: <2047616.cJycthpmiM@xps13> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [163.33.239.180] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] [PATCH v5 0/3] new headroom stats library and example application X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Feb 2015 15:56:00 -0000 > -----Original Message----- > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] > Sent: Monday, February 23, 2015 3:47 PM > To: Jastrzebski, MichalX K > Cc: Wodkowski, PawelX; dev@dpdk.org; Neil Horman > Subject: Re: [dpdk-dev] [PATCH v5 0/3] new headroom stats library and > example application >=20 > 2015-02-23 14:36, Jastrzebski, MichalX K: > > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] > > > 2015-02-20 15:46, Jastrzebski, MichalX K: > > > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Neil Horman > > > > > On Thu, Feb 19, 2015 at 01:18:41PM +0100, Pawel Wodkowski wrote: > > > > > > Hi community, > > > > > > I would like to introduce library for measuring load of some ar= bitrary > > > jobs. > > > > > > It can be used to profile every kind of job sets on any arbitra= ry > execution > > > unit > > > > > > or tasking library. > > > > > > > > > > > > In provided l2fwd-headroom example I demonstrate how to use thi= s > > > library to > > > > > > select optimal rx burst poll time. Jobs are selected by using e= xisting > > > rte_timer > > > > > > library calls. This example does no limit possible schemes on w= hich > this > > > > > > library can be used. > > > > > > > > > > > > Pawel Wodkowski (3): > > > > > > librte_headroom: New library for checking core/system/app loa= d > > > > > > examples: introduce new l2fwd-headroom example > > > > > > MAINTAINERS: claim responsibility for headroom library and > example > > > app > > > > > > > > > > I'm sorry but I still fail to see how this is a particularly usef= ul library. It > > > > > clearly works fine, but it composes an application event loop in = its > own > > > > > terms, > > > > > and measures stats based on that. While thats ok, any applicatio= n is > > > already > > > > > going to have to write its own event loop, and can makethe same > > > > > measurements > > > > > synchnously within that loop, using alot less code to optimize it= s > polling > > > time. > > > > > > > > > > In other words, I think this is one of those cases where this lib= rary is > > > > > probably somewhat useful for anyone who just wants to write an > > > application > > > > > in > > > > > terms the semantics exposed by this library, but not at all usefu= l for > > > anyone > > > > > else. I'd personally rather not have the extra code to maintain = here. > > > > > > > > > > Stephen just gave a presentation at netdev about some of the > > > performance > > > > > optimization measurements Brocade did with DPDK and how they fine > > > tuned > > > > > their > > > > > environment. One of the big take aways for me was that making ti= me > > > based > > > > > measurements (especially if it was using the tsc), created cpu st= alls > that > > > > > skewed the measurements, and so the best optimizations they made > > > avoided > > > > > time > > > > > measurements, opting instead for packet count metrics. > > > > > > > > > > Neil > > > > > > > > Hi Neil, > > > > > > > > I think this library offers something quite useful probably not for > everyone, > > > > but for many people that use DPDK, and it is measuring quite > accurately, > > > > how many spare cycles a CPU have after executing any serial tasks (= as > you > > > will know). > > > > If you look at two places in example application: main_loop() > > > > and l2fwd_fwd() functions, you will see two possible approach there= , > but > > > > this is not limited to that. You can even nest headroom objects and > > > measure > > > > process time of particular packets type. > > > > Of course, this will add an overhead due to the measurements, > > > > but that time is also measured, so any user can know what is the > relative > > > > time "wasted" for measuring all this. > > > > If time delays are measured in bigger timestamps, are handled relia= bly, > > > > the cost of measuring will be low. > > > > I find this quite similar to the power library case. I would say th= at library > is > > > not useful > > > > for every application, but there are several cases where it can be > > > > (as demonstrated with l3fwd-power app). > > > > > > > > About your last bit, not sure if I understood it right, but in case= of the > > > included sample app, > > > > the main measurement to see if we are overusing a CPU is the packet > count > > > > in a queue (in this case RX queue), and I believe this should be us= ed for > > > other apps, > > > > especially in those that use a pipeline model, where queues and rin= gs > are > > > the key part. > > > > > > > > As a final point, last week (12th of February), there was a request= for a > > > tool/library like this > > > > from a user in the mailing list (Ilan Borenshtein), which indicates= that > this > > > would be useful > > > > (probably not just for him, but for others). It probably could be a= chieved > by > > > the user > > > > by adding their own code, but I believe this library would be a goo= d-to- > > > have, > > > > in case a user is looking for an easy way to calculate the exposed = above. > > > > Let us give the users an example of this method and we will expand = it > with > > > more > > > > advanced application that may show capabilities of dynamic load > scaling > > > based on headroom library measurement. > > > > > Hi Thomas, > > > > > I wonder how this library is related to DPDK. > > > I'm not against its integration, though the question must be asked. > > > DPDK is a set of libraries. What kind of library fit with DPDK goals = and > > > deserve to be integrated? > > > > > I think this library fits into dpdk goals, because it is simple and opt= imized > for fast packet processing. >=20 > I don't have a strong opinion here. If anyone else wants to comment, plea= se > speak now. >=20 > > The library provides an easy way for existing DPDK users to modify thei= r > applications to measure available CPU headroom. > > If we integrate it, users will verify it and decide what else they need= and we > will implement this. >=20 > Do you mean that you plan to add some features to this library? > Is it going to stay at providing some stats or could you make some action= s > like time-sharing helpers? What do you mean here saying time-sharing?=20 >=20 > > > I don't know whether it's related but nobody acknowledged this patchs= et. > > We were waiting for Neil's final comments. He did not mention anything > else since the last time. > > When Pawel sends the next version with the copyright date corrected, > Pablo will ack this. > > > > > I also feel that the name of this library is a bit too vague. Some pe= ople > > > were asking first what means "headroom". It's actually for CPU headro= om > > > monitoring. > > > What about "cpuheadroom", "cpuheadroomstat", "jobstat"? > > > > I think we can change the name to "cpuheadroom" as it describes more > clear this library. >=20 > If you're focusing on CPU usage with possible actions, yes. > If you're focusing on decision helper, jobstat would be better IMHO. >=20 > > > Last comment, less important: as many of your colleagues, you don't p= ay > > > attention to the copyright dates. I'm pretty sure this code was not w= ritten > > > in 2010. So why claiming it? > > Sorry, we will fix this of course and pay more attention now. >=20 > OK, thanks