From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp.tuxdriver.com (charlotte.tuxdriver.com [70.61.120.58]) by dpdk.org (Postfix) with ESMTP id 202AB1F7 for ; Fri, 30 Jan 2015 19:02:21 +0100 (CET) Received: from hmsreliant.think-freely.org ([2001:470:8:a08:7aac:c0ff:fec2:933b] helo=localhost) by smtp.tuxdriver.com with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.63) (envelope-from ) id 1YHFtX-0008E7-40; Fri, 30 Jan 2015 13:02:19 -0500 Date: Fri, 30 Jan 2015 13:02:14 -0500 From: Neil Horman To: "Wodkowski, PawelX" Message-ID: <20150130180214.GB2664@hmsreliant.think-freely.org> References: <1422532206-10662-1-git-send-email-pawelx.wodkowski@intel.com> <20150129132522.GA1999@hmsreliant.think-freely.org> <20150129191326.GF1999@hmsreliant.think-freely.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Score: -2.9 (--) X-Spam-Status: No Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] [PATCH 0/2] new headroom stats library and example application X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Jan 2015 18:02:21 -0000 On Fri, Jan 30, 2015 at 10:47:21AM +0000, Wodkowski, PawelX wrote: > > > > -----Original Message----- > > From: Neil Horman [mailto:nhorman@tuxdriver.com] > > Sent: Thursday, January 29, 2015 8:13 PM > > To: Wodkowski, PawelX > > Cc: dev@dpdk.org > > Subject: Re: [dpdk-dev] [PATCH 0/2] new headroom stats library and example > > application > > > > On Thu, Jan 29, 2015 at 05:10:36PM +0000, Wodkowski, PawelX wrote: > > > > -----Original Message----- > > > > From: Neil Horman [mailto:nhorman@tuxdriver.com] > > > > Sent: Thursday, January 29, 2015 2:25 PM > > > > To: Wodkowski, PawelX > > > > Cc: dev@dpdk.org > > > > Subject: Re: [dpdk-dev] [PATCH 0/2] new headroom stats library and > > example > > > > application > > > > > > > > On Thu, Jan 29, 2015 at 12:50:04PM +0100, Pawel Wodkowski wrote: > > > > > Hi community, > > > > > I would like to introduce library for measuring load of some arbitrary jobs. > > It > > > > > can be used to profile every kind of job sets on any arbitrary execution unit. > > > > > In provided l2fwd-headroom example I demonstrate how to use this library > > to > > > > > profile packet forwarding (job set is froward, flush and stats) on LCores > > > > > (execution unit). This example does no limit possible schemes on which this > > > > > library can be used. > > > > > > > > > > Pawel Wodkowski (2): > > > > > librte_headroom: New library for checking core/system/app load > > > > > examples: introduce new l2fwd-headroom example > > > > > > > > > > config/common_bsdapp | 6 + > > > > > config/common_linuxapp | 6 + > > > > > examples/Makefile | 1 + > > > > > examples/l2fwd-headroom/Makefile | 51 +++ > > > > > examples/l2fwd-headroom/main.c | 875 > > > > ++++++++++++++++++++++++++++++++++++ > > > > > lib/Makefile | 1 + > > > > > lib/librte_headroom/Makefile | 50 +++ > > > > > lib/librte_headroom/rte_headroom.c | 368 +++++++++++++++ > > > > > lib/librte_headroom/rte_headroom.h | 481 ++++++++++++++++++++ > > > > > mk/rte.app.mk | 4 + > > > > > 10 files changed, 1843 insertions(+) > > > > > create mode 100644 examples/l2fwd-headroom/Makefile > > > > > create mode 100644 examples/l2fwd-headroom/main.c > > > > > create mode 100644 lib/librte_headroom/Makefile > > > > > create mode 100644 lib/librte_headroom/rte_headroom.c > > > > > create mode 100644 lib/librte_headroom/rte_headroom.h > > > > > > > > > > -- > > > > > 1.7.9.5 > > > > > > > > > > > > > > > > > > Whats the advantage of this library over the other tools to preform the same > > > > function. > > > > > > Hi Neil, > > > > > > Good point, what is advantage over perf. Answer is: this library does not > > > supposed to be a perf competition and is not for profiling app in the way perf > > does. > > > It is an small and fast extension. It's main task is to manage job list to invoke > > > them exactly when needed and provide some basic stats about application idle > > > time (whatever programmer will consider the idle) and busy time. > > > > > > For example: > > > application might decide to add remove some jobs to/from LCore(s) > > dynamically > > > basing on current idle time (ex: move job from one core to another). > > > > > > Also application might have some information's about traffic type it handles > > > and provide own algorithm to calculate invocation time (it can also > > dynamically > > > switch between those algorithms only replacing handlers). > > > > > > > Perf can provide all the information in this library, and do so > > > > without having to directly modify the source for the execution unit under > > test > > > > > > Yes, perf can provide those information's but it can't handle the case when > > > you are poling for packets too fast or too slow and waist time getting only > > couple > > > of them. Library will adjust time when it execute job basing on value this job > > > returned previously. Code modifications are not so deep, as you can see > > comparing > > > l2wf vs l2fwd-headroom app. > > > > > > For example in application I introduced, when forward job return less than > > > MAX_PKT_BURST execution period will be increased. If it return more it will > > decrease > > > execution period. Stats provided for that can be used to determine if > > application is > > > behaving correctly and if there is a time for handling another port (what did for > > tests). > > > > > You're still re-inventing the wheel here, and I don't see any advantage to doing > > so. If the goal of the library is to profile the run time of a task, then you > > have perf and systemtap for such purposes. If the goal is to create a job > > scheduler that allows you to track multiple parallel tasks, and adjust their > > execution, there are several pre-existing libraries that any application > > programmer can already leverage to do just that (Berkely UPC or libtask to name > > just two examples). Truthfully, on a dedicated cpu, you could just as easily > > create multiple child processes runnnig at SCHED_RR and set their priorities > > accordingly. > > > > I don't see why we need another library to do what several other tools/libraries > > can do quite well. > > > > I am under impression that I am unable to express myself clearly enough. > No, I think you're making yourself clear, you've created a library that allows you to run a set of jobs, measure various metrics about them, and adjust their execution time. Am I correct? > I did not meant to make competition to perf nor reinvent the tasking library. > I understand thats not what your intent was, but it seems to have been your result. > 1. Idle and runtime statistics are provided "by the way" and you can use them > or use perf on linux or libpmc on freebsd or whatever tool you like to get more > sophisticated ones. Ok, so we're in agreement that the idle and runtime statistics in this library are simmilar to, and somewhat less sophisticated that those provided by perf of libpmc. > 2. You can also use tasking library what you like with a headroom object on top > of every task you created with any scheduling you want. This way you can assign > priorities to job sets and adjust sleep time as you like. You can also decide what > is your idle/load time by placing task-dependet-yeld/sleep in idle callback or separate > job callback or loop end callback. > I'm a little unclear on what you're saying here. What I think you're saying is that we can also use other tasking libraries to achieve what you've done with libheadroom, but I might be misreading you. If so, please clarify. > Both of those two points above are no covering one gap - we are running in poll > mode and here is a lack of mutex, semaphores or other synchronization mechanisms > provided by OS/tasking layer. Ok, I'm not sure how thats relevant though. The OS doesn't have to be involved here at all if you don't want it to. Take a look at libtask: http://swtch.com/libtask.tar.gz Its a rudimentary tasking library that makes locking available if need be, but in no way requires it. UPC is simmilar: http://upc.lbl.gov/task.shtml Its got locking, but with proper cpu separation you don't need to use it. This library try to provide facility for estimating optimal > job execution time because every mis-poll is a waste of time and can lead to > impression that core/task is fully loaded but what it does is only polling and getting 1 > or 2 packets instead of 32 (it degrades total NIC throughput). > Yes, I get that you've got a specific use case in mind (optimizing the time spent in a specific task dynamically). My question is: Why did you re-create the code already available in other libraries to do it? Theres no reason you can't use the UPC library to create an example of a job set consisting of 4 jobs: 1) A job to record the start time of job (2) 2) A job to poll an interface 3) A job to measure the end time of job (2) and the number of packets it recorded 4) A job to adjust the run time of job (2) based on the results obtained from stats gathered in job (3) I don't deny that you have a useful use case here, you definately do. My only concern is that you've re-invented the wheel to make it happen. You don't really need the library here, just the example that makes use of an existing library. > You can use perf (or whatever tool you like) to profile your code for performance. If > the code good enough, you use headroom library that will estimate how often or > when your perfect-profiled-and-optimized code need to be executed. When execution > period estimation settles you have your headroom for other jobs on this core/task. > You can also provide your own algorithm for estimating execution period. You are also > able to drop using this headroom library at runtime and switch to tasking one when > you decide that you have enough data make decision how many jobs you can execute > on lcore/task basing on current throughput. > > I deliberately did not used 'task' but 'job' to not make an impression that I am > reinventing another task library. This code is to be simple and fast. Of course you can > do all those things in every application you make but this library is provided to not > reinvent this logic all the time. > All user space task libraries are simple and fast, its the only reason they exist :). Theres no need to create another one here. Neil