Re: [dpdk-dev] Future Direction for rte_eth_stats_get()

["Van Haaren, Harry" <harry.van.haaren@intel.com>]

+Jay, (@all, please keep everybody in the CCs :)

> From: David Harton (dharton) [mailto:dharton@cisco.com]
> To: Van Haaren, Harry <harry.van.haaren@intel.com>; Thomas Monjalon
> > > xstats are driver agnostic and have a well-defined naming scheme.
> >
> > Indeed, described here:
> > http://dpdk.org/doc/guides/prog_guide/poll_mode_drv.html#extended-
> > statistics-api
> 
> Thanks for sharing.  I do think what is in the link is well thought out but it also may
> not match how the application would choose to represent that stat (capitalization,
> abbreviation, etc).

Welcome. Sure, an application can match any xstats string to its "Display" counterpart,
or a few simple translations actually make (almost) all of them human readable. Translating
an _ to space, capitalize first letter of description items, and remove the unit at the end).
This use-case was considered when detailing the naming scheme.


> > The "rules" of encoding a statistic as an xstats string are pretty simple,
> > and if any PMD breaks the rules, it should be considered broken and fixed.
> 
> I understand it may be broken, but that doesn't help finding them and ensuring they aren't
> broken to begin with.  What regression/automation is in place to ensure drivers aren't
> broken?

Currently nothing automated - patch review on the mailing list. I'm open to ideas on this,
if you have suggestions.


> > Do you see a fundamental problem with parsing the strings to retrieve
> > values?
> 
> I think parsing strings is expensive CPU wise

Parsing the strings can be done once at startup, and the index of the statistics in the
array cached. When actually reading statistics, access of the array of statistics at the
previously cached index will return the same statistic. It is not needed to do strcmp()
per statistic per read.


> That's why I was wondering if a binary id could be generated instead.

I've worked with such a system before, it dynamically registered string->int mappings at
runtime, and allowed "reverse" mapping them too. Its workable, and I'm not opposed to it
if the conclusion is that it really is necessary, but I'm not sure about that.


> The API
> would be very similar to the current xstats API except it would pass id/value pairs
> instead of string/value pairs.  This avoids string comparisons to figure out while still
> allowing flexibility between drivers.

Apart from a once-off scan at startup, xstats achieves this. 


>  It would also 100% guarantee that any strings
> returned by "const char* rte_eth_xstats_str_get(uint32_t stat_id)" are consistent across
> devices.

Yes - that is a nice feature that xstats (in its current form) doesn't have.
But what price is there to pay for this?
We need to map an ID for each stat that exists.

How and where will this mapping happen? Perhaps would you expand a little on how
you see this working? 

> I also think there is less chance for error if drivers assign their stats to ID versus
> constructing a string. 

Have a look in how the mapping is done in the xstats implementation for ixgbe for example:
http://dpdk.org/browse/dpdk/tree/drivers/net/ixgbe/ixgbe_ethdev.c#n540

It's a pretty simple  {"string stat name" -> offsetof( stuct hw, register_var_name )}


> I haven't checked my application, but I wonder if the binary size
> would actually be smaller if all the stats strings were centrally located versus
> distributed across binaries (highly dependent on the linker and optimization level).

Sounds like optimizing the wrong thing to me.


> > If you like I could code up a minimal sample-app that only pulls
> > statistics, and you can show "interest" in various statistics for printing
> > / monitoring?
> 
> Appreciate the offer.  I actually understand what's is available.  And, BTW, I apologize
> for being late to the game (looks like this was discussed last summer) but I'm just now
> getting looped in and following the mailer but I'm just wondering if something that is
> performance friendly for the user/application and flexible for the drivers is possible.

We're all looking for the same thing - just different approaches that's all :)


RE: Thomas asking about performance numbers:
I can scrape together some raw tsc data on Monday and post to list, and we can discuss it more then.


Regards, -Harry