Re: [dpdk-dev] [PATCH v5 0/3] new headroom stats library and example application

["Jastrzebski, MichalX K" <michalx.k.jastrzebski@intel.com>]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Neil Horman
> Sent: Thursday, February 19, 2015 3:34 PM
> To: Wodkowski, PawelX
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH v5 0/3] new headroom stats library and
> example application
> 
> On Thu, Feb 19, 2015 at 01:18:41PM +0100, Pawel Wodkowski wrote:
> > Hi community,
> > I would like to introduce library for measuring load of some arbitrary jobs.
> It
> > can be used to profile every kind of job sets on any arbitrary execution unit
> or
> > tasking library.
> >
> > In provided l2fwd-headroom example I demonstrate how to use this library
> to
> > select optimal rx burst poll time. Jobs are selected by using existing
> rte_timer
> > library calls. This example does no limit possible schemes on which this
> library
> > can be used.
> >
> > PATCH v5 changes:
> >  - Fix spelling and checkpatch.pl errors.
> >  - Add maintainer claim for library and example app.
> >
> > PATCH v4 changes:
> >  - use proper branch for generating patch.
> >
> > PATCH v3 changes:
> >  - Fix spelling.
> >
> > PATCH v2 changes:
> >  - Remove jobs management/callback from library to not duplicate tasking
> library
> >    behaviour.
> >  - Cleenup/remove useless statistics.
> >  - Rework example application to use rte_timer library for jobs selection.
> >  - Introduce new app parameter '-l' for automatic thousands separating in
> stats.
> >  - More readable statistics format.
> >
> >
> >
> > Pawel Wodkowski (3):
> >   librte_headroom: New library for checking core/system/app load
> >   examples: introduce new l2fwd-headroom example
> >   MAINTAINERS: claim responsibility for headroom library and example app
> >
> >  MAINTAINERS                                  |    4 +
> >  config/common_bsdapp                         |    5 +
> >  config/common_linuxapp                       |    5 +
> >  examples/Makefile                            |    1 +
> >  examples/l2fwd-headroom/Makefile             |   51 ++
> >  examples/l2fwd-headroom/main.c               | 1040
> ++++++++++++++++++++++++++
> >  lib/Makefile                                 |    1 +
> >  lib/librte_headroom/Makefile                 |   54 ++
> >  lib/librte_headroom/rte_headroom.c           |  271 +++++++
> >  lib/librte_headroom/rte_headroom.h           |  324 ++++++++
> >  lib/librte_headroom/rte_headroom_version.map |   19 +
> >  mk/rte.app.mk                                |    4 +
> >  12 files changed, 1779 insertions(+)
> >  create mode 100644 examples/l2fwd-headroom/Makefile
> >  create mode 100644 examples/l2fwd-headroom/main.c
> >  create mode 100644 lib/librte_headroom/Makefile
> >  create mode 100644 lib/librte_headroom/rte_headroom.c
> >  create mode 100644 lib/librte_headroom/rte_headroom.h
> >  create mode 100644 lib/librte_headroom/rte_headroom_version.map
> >
> > --
> > 1.9.1
> >
> >
> I'm sorry but I still fail to see how this is a particularly useful library.  It
> clearly works fine, but it composes an application event loop in its own
> terms,
> and measures stats based on that.  While thats ok, any application is already
> going to have to write its own event loop, and can makethe same
> measurements
> synchnously within that loop, using alot less code to optimize its polling time.
> 
> In other words, I think this is one of those cases where this library is
> probably somewhat useful for anyone who just wants to write an application
> in
> terms the semantics exposed by this library, but not at all useful for anyone
> else.  I'd personally rather not have the extra code to maintain here.
> 
> Stephen just gave a presentation at netdev about some of the performance
> optimization measurements Brocade did with DPDK and how they fine tuned
> their
> environment.  One of the big take aways for me was that making time based
> measurements (especially if it was using the tsc), created cpu stalls that
> skewed the measurements, and so the best optimizations they made avoided
> time
> measurements, opting instead for packet count metrics.
> 
> Neil

Hi Neil,

I think this library offers something quite useful probably not for everyone, 
but for many people that use DPDK, and it is measuring quite accurately,
how many spare cycles a CPU have after executing any serial tasks (as you will know). 
If you look at two places in example application: main_loop()
and l2fwd_fwd() functions, you will see two possible approach there, but
this is not limited to that. You can even nest headroom objects and measure
process time of particular packets type.
Of course, this will add an overhead due to the measurements, 
but that time is also measured, so any user can know what is the relative
time "wasted" for measuring all this.
If time delays are measured in bigger timestamps, are handled reliably, 
the cost of measuring will be low.    
I find this quite similar to the power library case. I would say that library is not useful
for every application, but there are several cases where it can be 
(as demonstrated with l3fwd-power app).

About your last bit, not sure if I understood it right, but in case of the included sample app,
the main measurement to see if we are overusing a CPU is the packet count
in a queue (in this case RX queue), and I believe this should be used for other apps,
especially in those that use a pipeline model, where queues and rings are the key part.

As a final point, last week (12th of February), there was a request for a tool/library like this
from a user in the mailing list (Ilan Borenshtein), which indicates that this would be useful
(probably not just for him, but for others). It probably could be achieved by the user
by adding their own code, but I believe this library would be a good-to-have,
in case a user is looking for an easy way to calculate the exposed above.
Let us give the users an example of this method and we will expand it with more 
advanced application that may show capabilities of dynamic load scaling based on headroom library measurement.

Best regards
Michal