Re: [dpdk-dev] Future Direction for rte_eth_stats_get()

["Tahhan, Maryam" <maryam.tahhan@intel.com>]

Hi David

+ Olivier and Harry as contributors and advisors in the past on the stats and stats API.

So I pulled the rtnl_link_stats64 from http://lxr.free-electrons.com/source/include/uapi/linux/if_link.h#L41 and crossed them with http://dpdk.org/dev/patchwork/patch/5842/ and http://dpdk.org/browse/dpdk/tree/lib/librte_ether/rte_ethdev.h 

40 /* The main device statistics structure */
 41 struct rtnl_link_stats64 {
 42         __u64   rx_packets;             /* total packets received       */             <MT> is in struct rte_eth_stats
 43         __u64   tx_packets;             /* total packets transmitted    */          <MT> is in struct rte_eth_stats
 44         __u64   rx_bytes;               /* total bytes received         */                  <MT> is in struct rte_eth_stats
 45         __u64   tx_bytes;               /* total bytes transmitted      */               <MT> is in struct rte_eth_stats
 46         __u64   rx_errors;              /* bad packets received         */                <MT> I'm working on a patch to distinguish between error and drops for struct rte_eth_stats at the moment only drops are reported through ierrors and oerrors and we are tied to those field name for backward compatibility.
 47         __u64   tx_errors;              /* packet transmit problems     */           <MT> Same comment as rx_errors.
 48         __u64   rx_dropped;             /* no space in linux buffers    */          <MT> is in struct rte_eth_stats
 49         __u64   tx_dropped;             /* no space available in linux  */         <MT> is in struct rte_eth_stats
 50         __u64   multicast;              /* multicast packets received   */           <MT> was deprecated in struct rte_eth_stats - but we can remove the notice and expose again from drivers we modified
 51         __u64   collisions; <MT> Not in struct rte_eth_stats - not sure it's in xstats either... 
 52 
 53         /* detailed rx_errors: */
 54         __u64   rx_length_errors;                                                                            <MT> was deprecated in struct rte_eth_stats, is in xstats - but we can remove the notice and expose again from drivers we modified
 55         __u64   rx_over_errors;         /* receiver ring buff overflow  */    <MT> was never exposed to struct rte_eth_stats  - but is in xstats.
 56         __u64   rx_crc_errors;          /* recved pkt with crc error    */         <MT> was deprecated in struct rte_eth_stats, is in xstats - but we can remove the notice and expose again from drivers we modified
 57         __u64   rx_frame_errors;        /* recv'd frame alignment error */ <MT> was never exposed to struct rte_eth_stats - but is in xstats.
 58         __u64   rx_fifo_errors;         /* recv'r fifo overrun          */               <MT> was never in struct rte_eth_stats. Not exposed by all drivers
 59         __u64   rx_missed_errors;       /* receiver missed packet       */   <MT> was deprecated in struct rte_eth_stats, is in xstats - but we can remove the notice
 60 
 61         /* detailed tx_errors */
 62         __u64   tx_aborted_errors;                                                                         <MT> was never in struct rte_eth_stats. Not exposed by all drivers
 63         __u64   tx_carrier_errors;                                                                            <MT> was never in struct rte_eth_stats. Not exposed by all drivers
 64         __u64   tx_fifo_errors;                                                                                  <MT> was never in struct rte_eth_stats. Not exposed by all drivers
 65         __u64   tx_heartbeat_errors;                                                                     <MT> was never in struct rte_eth_stats. Not exposed by all drivers
 66         __u64   tx_window_errors;                                                                         <MT> was never in struct rte_eth_stats. Not exposed by all drivers
 67 
 68         /* for cslip etc */
 69         __u64   rx_compressed;                                                                               <MT> was never in struct rte_eth_stats. Not exposed by all drivers
 70         __u64   tx_compressed;                                                                               <MT> was never in struct rte_eth_stats. Not exposed by all drivers
 71 };

So what can be enabled again in struct rte_eth_stats  from what was already there is the equivalent of: 
* rx_length_errors
* rx_crc_errors
* rx_missed_errors - the deprecation notice was removed for this field.
* multicast

What should be added in to distinguish between errors and drops. struct rte_eth_stats :
 * rx_errors
 * tx_errors

As for the detailed rx errors and tx errors I'm open to feedback from you folks as to what should go in and what is too detailed. These weren't in struct rte_eth_stats previously, they are available through xstats and are uniformly named across the drivers. Oliver + Harry any thoughts?

David I assume you are looking for all the missing fields to be added?

Best Regards, 
Maryam


> -----Original Message-----
> From: David Harton (dharton) [mailto:dharton@cisco.com]
> Sent: Friday, January 22, 2016 1:41 PM
> To: Tahhan, Maryam <maryam.tahhan@intel.com>; dev@dpdk.org
> Subject: RE: Future Direction for rte_eth_stats_get()
> 
> Hi Maryam,
> 
> Thanks for the pointer.  I'll review the convo's.
> 
> Consider I have an application that uses many(all?) of the DPDK drivers
> and their netmap counterparts depending on configuration.  The user
> interface provides a set of I/O stats to help debug I/O issues and that
> set is the same regardless of driver type.  The set of stats provided
> matches what linux provides today since netmap existed before dpdk.
> 
> What I want to avoid is having an application that is driver independent
> having to become driver dependent interpreting a bunch of strings
> (from xstats) or worse the layer running at the data plane core that is
> advertising the API needed by the application parsing those strings
> because the application cannot change the upper layer.
> 
> What if instead of passing strings and values a set of stat ids and value
> are returned.  At least this way the application can remain driver
> agnostic versus having to parse a free form string that likely isn't the
> same across driver types.
> 
> Also, why wouldn't rte_eth_stats_get() align with linux which is the
> defacto standard?  I understand that not every driver may not support
> every stat but that's ok.  Just return 0 (pause frames, crc errors, etc).
> It's not like the list of linux stats is ever growing or advertising an
> absurd number of stats that aren't applicable to all drivers.
> 
> Thanks again,
> Dave
> 
> > -----Original Message-----
> > From: Tahhan, Maryam [mailto:maryam.tahhan@intel.com]
> > Sent: Friday, January 22, 2016 6:08 AM
> > To: David Harton (dharton) <dharton@cisco.com>; dev@dpdk.org
> > Subject: RE: Future Direction for rte_eth_stats_get()
> >
> > Hi David
> > Some of the stats were HW specific rather than generic stats that
> > should be exposed through rte_eth_stats and were migrated to the
> xstats API.
> > http://dpdk.org/ml/archives/dev/2015-June/019915.html. The
> naming was
> > also not generic enough to cover some of the drivers and in some
> cases
> > what was exposed was only a partial view of the relevant stats.
> >
> > They were marked as deprecated to deter people from using them
> in the
> > future, but haven't been removed from all the driver
> implementations yet.
> > The Registers that remain undeprecated are those considered to be
> generic.
> >
> > Which registers are you particularly interested in that have been
> > deprecated? Can you elaborate on what you mean by " scenarios
> where a
> > static view is expected "
> >
> > Thanks in advance.
> >
> > Best Regards,
> > Maryam
> >
> >
> > > -----Original Message-----
> > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of David
> Harton
> > > (dharton)
> > > Sent: Wednesday, January 20, 2016 5:19 PM
> > > To: dev@dpdk.org
> > > Subject: [dpdk-dev] Future Direction for rte_eth_stats_get()
> > >
> > > I see that some of the rte_eth_stats have been marked
> deprecated in
> > > 2.2 that are returned by rte_eth_stats_get().  Applications that
> > > utilize any number of device types rely on functions like this one
> > > to debug I/O issues.
> > >
> > > Is there a reason the stats have been deprecated?  Why not keep
> the
> > > stats in line with the standard linux practices such as
> > rtnl_link_stats64?
> > >
> > > Note, using rte_eth_xstats_get() does not help for this particular
> > scenario
> > > because a common binary API is needed to communicate through
> various
> > > layers and also provide a consistent view/meaning to users.  The
> > > xstats is excellent for debugging device specific scenarios but
> > > can't
> > help
> > > in scenarios where a static view is expected.
> > >
> > > Thanks,
> > > Dave