DPDK patches and discussions
 help / color / mirror / Atom feed
* [dpdk-dev] [PATCH v12 0/6] Expanded statistics reporting
@ 2017-03-27 20:21 Remy Horton
  2017-03-27 20:21 ` [dpdk-dev] [PATCH v12 1/6] lib: add information metrics library Remy Horton
                   ` (7 more replies)
  0 siblings, 8 replies; 41+ messages in thread
From: Remy Horton @ 2017-03-27 20:21 UTC (permalink / raw)
  To: dev; +Cc: Thomas Monjalon

This patchset consists of three libraries: A Metrics library for
distributing device information, a library that calculates bit-rate
statistics, and a library that calculates latency statistics. The
latter two libraries make use of the first library.

Metrics Library
---------------
The Metrics library implements a mechanism by which producers can
publish numeric information for later querying by consumers.
In practice producers will typically be other libraries or
primary processes, whereas consumers will typically be applications.

Metrics themselves are statistics that are not generated by PMDs.
Metric information is populated using a push model, where producers
update the values contained within the metric library by calling an
update function on the relevant metrics. Consumers receive metric
information by querying the central metric data, which is held
in shared memory so it can also be accessed by secondary processes.

For each metric, a separate value is maintained for each port id,
and when publishing metric values the producers need to specify
which port is being updated. In addition there is a special id
RTE_METRICS_GLOBAL that is intended for global statistics that are
not associated with any individual device. Since the metrics
library is self-contained, the only restriction on port numbers is
that they are less than RTE_MAX_ETHPORTS - there is no requirement
for the ports to actually exist.

Metrics must first be registered, which is the way producers declare
the names of the metrics they will be publishing. Registration can
either be done individually, or as a group as a metric set. The
intention is for all metrics in a set to be updated in one go,
although it is also possible for metrics within a set to be updated
individually. It is up to the producers to update metrics as required.

Bit-rate statistics library
---------------------------
The bit-rate library calculates the mean, the exponentially-weighted
moving average, and peak bit-rates for each active port (i.e. network
device). These statistics are then reported via the metrics library
using the following names: mean_bits_in, mean_bits_out, ewma_bits_in,
ewma_bits_out, peak_bits_in, and peak_bits_out. The sampling window 
used for calculation is decided by the application requiring the statistics.

Latency statistics library
--------------------------
The latency statistics library calculates the port-to-port latency of
packet processing by a DPDK application, reporting the minimum, average,
and maximum nano-seconds that packet processing takes, as well as the
jitter in processing delay. These statistics are then reported via the
metrics library using the following names: min_latency_ns, avg_latency_ns,
mac_latency_ns, and jitter_ns.

This is dependent on http://dpdk.org/dev/patchwork/patch/21607/ for the
timestamp MBUF field.

For more details on the metrics library and the Bit-rate and Latency
components see the Programmer's Guide updates in the patchset.
--

v12 changes:
* Rebased to latest master
* Misspellings in MAINTAINERS corrected
* Removed MBUF modifications (use Olivier's MBUF rework)

v11 changes:
* Rebased
* .map references to 17.02 changed to 17.05
* Release ntoes moved to release_17_05.rst
* Bit-rate library now also gives unfiltered average

v10 changes:
* Rebased
* Relocated some config-related directives.
* Removed incorrect capitalisations in API docs.
* Formatting & detail corrections in release notes.
* Moved around struct member descriptions.
* Rewritten rte_metrics.h file description.
* Rewritten description of RTE_METRICS_GLOBAL.
* Used 'producers' and 'consumers' as terms.
* Removed markup (bold text) in Doxygen tags.
* Added programming guide section.

v9 changes:
* Updated .map files to reflect function changes in v8
* Fixed rte_malloc() of zero bytes in proc_info when no metrics exist
* Fixed rte_metrics_init not being called explicitly in testpmd

v8 changes:
* Release notes correction
* Updated copyright years
* rte_metric_init() takes socket id & must be explicitly called
* rte_metrics_reg_metric renamed to rte_metrics_reg_name()
* rte_metrics_update_metric() renamed to rte_metrics_update_value()
* Doxygen updates
* Changed malloc()/free() to rte_malloc()/rte_free()
* Removed redundant memset()
* rte_stats_bitrates_s renamed to rte_stats_bitrates_s
* Split mbuf change to own patch for visibility
* CYCLES_PER_NS now a static inline function
* latency: "hidden" pthread creation now has polling API instead.
* Struct declarations and variable definitions cleaned up
* Double initialization of the latency library now returns -EEXIST
* MAINTAINERS entry for layenctstats in correct section

v7 changes:
* RTE_METRICS_NONPORT renamed to RTE_METRICS_GLOBAL
* Multiple changes to rte_metrics.h doxygen documentation
* Split apart latency patch into lib, test-pmd, & proc_info parts
* Reordered patches by functionality
* Insufficent capacity return value changed from -ERANGE to actual size
* Cache alignment in bitrate library
* Tightened up const usage to avoid STATIC_CONST_CHAR_ARRAY warning
* Reshma reinstated as author for (now split) latency patch
* Rebase to master

v6 changes:
* Metrics display now has "Non port specific" rather than "port -1"
* Fixed sign issue in EWMA delta calculation
* Rebased to master

v5 changes:
* Updated Shared Library Versions in release notes
* Merged in Reshma's latencystats library

v4 changes:
* References to 16.11 changed to 17.02
* Fetching of non-port values was broken
* Added sanity checks to value fetching
* rte_stat_value renamed to rte_metric_value
* Corrected doxygen descriptions
* Added MAINTAINERS entries
* Added #ifdef directives to bitrate code in test-pmd

v3 changes:
* Marked rte_stats_bitrate_s as internal
* Minor integer roundoff correction
* Coding style corrections
* Removed spurious object allocation
* Changes to rte_metrics.[ch] moved from Patch 2/3 to 1/3.
* Reintroduced non-port values (RTE_METRICS_NONPORT)
* Added spinlocks to metric library
* Removed spurious test registration/update
* Added release notes entries

v2 changes:
* Uses a new metrics library rather than being part of ethdev


Harry van Haaren (2):
  lib: added new library for latency stats
  app/test-pmd: add latency statistics calculation

Remy Horton (3):
  lib: add information metrics library
  lib: add bitrate statistics library
  app/test-pmd: add bitrate statistics calculation

Reshma Pattan (1):
  app/proc_info: add metrics displaying

 MAINTAINERS                                        |  12 +
 app/proc_info/main.c                               |  79 ++++-
 app/test-pmd/parameters.c                          |  20 +-
 app/test-pmd/testpmd.c                             |  78 ++++-
 app/test-pmd/testpmd.h                             |   6 +-
 config/common_base                                 |  15 +
 doc/api/doxy-api-index.md                          |   3 +
 doc/api/doxy-api.conf                              |   3 +
 doc/guides/prog_guide/index.rst                    |   1 +
 doc/guides/prog_guide/metrics_lib.rst              | 299 +++++++++++++++++
 doc/guides/rel_notes/release_17_02.rst             |   3 +
 doc/guides/rel_notes/release_17_05.rst             |  18 +
 lib/Makefile                                       |   3 +
 lib/librte_bitratestats/Makefile                   |  53 +++
 lib/librte_bitratestats/rte_bitrate.c              | 141 ++++++++
 lib/librte_bitratestats/rte_bitrate.h              |  80 +++++
 .../rte_bitratestats_version.map                   |   9 +
 lib/librte_latencystats/Makefile                   |  56 ++++
 lib/librte_latencystats/rte_latencystats.c         | 366 +++++++++++++++++++++
 lib/librte_latencystats/rte_latencystats.h         | 154 +++++++++
 .../rte_latencystats_version.map                   |  11 +
 lib/librte_metrics/Makefile                        |  51 +++
 lib/librte_metrics/rte_metrics.c                   | 299 +++++++++++++++++
 lib/librte_metrics/rte_metrics.h                   | 240 ++++++++++++++
 lib/librte_metrics/rte_metrics_version.map         |  13 +
 mk/rte.app.mk                                      |   3 +
 26 files changed, 2012 insertions(+), 4 deletions(-)
 create mode 100644 doc/guides/prog_guide/metrics_lib.rst
 create mode 100644 lib/librte_bitratestats/Makefile
 create mode 100644 lib/librte_bitratestats/rte_bitrate.c
 create mode 100644 lib/librte_bitratestats/rte_bitrate.h
 create mode 100644 lib/librte_bitratestats/rte_bitratestats_version.map
 create mode 100644 lib/librte_latencystats/Makefile
 create mode 100644 lib/librte_latencystats/rte_latencystats.c
 create mode 100644 lib/librte_latencystats/rte_latencystats.h
 create mode 100644 lib/librte_latencystats/rte_latencystats_version.map
 create mode 100644 lib/librte_metrics/Makefile
 create mode 100644 lib/librte_metrics/rte_metrics.c
 create mode 100644 lib/librte_metrics/rte_metrics.h
 create mode 100644 lib/librte_metrics/rte_metrics_version.map

-- 
2.5.5

^ permalink raw reply	[flat|nested] 41+ messages in thread

end of thread, other threads:[~2017-04-05 16:02 UTC | newest]

Thread overview: 41+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-03-27 20:21 [dpdk-dev] [PATCH v12 0/6] Expanded statistics reporting Remy Horton
2017-03-27 20:21 ` [dpdk-dev] [PATCH v12 1/6] lib: add information metrics library Remy Horton
2017-03-28 14:14   ` Pattan, Reshma
2017-03-28 15:00     ` Remy Horton
2017-03-27 20:21 ` [dpdk-dev] [PATCH v12 2/6] app/proc_info: add metrics displaying Remy Horton
2017-03-27 20:21 ` [dpdk-dev] [PATCH v12 3/6] lib: add bitrate statistics library Remy Horton
2017-03-28 15:30   ` Pattan, Reshma
2017-03-28 15:45     ` Remy Horton
2017-03-27 20:21 ` [dpdk-dev] [PATCH v12 4/6] app/test-pmd: add bitrate statistics calculation Remy Horton
2017-03-28 15:49   ` Pattan, Reshma
2017-03-27 20:21 ` [dpdk-dev] [PATCH v12 5/6] lib: added new library for latency stats Remy Horton
2017-03-27 20:21 ` [dpdk-dev] [PATCH v12 6/6] app/test-pmd: add latency statistics calculation Remy Horton
2017-03-27 21:03 ` [dpdk-dev] [PATCH v12 0/6] Expanded statistics reporting Stephen Hemminger
2017-03-28 11:21   ` Mcnamara, John
2017-03-29 18:28 ` [dpdk-dev] [PATCH v13 " Remy Horton
2017-03-29 18:28   ` [dpdk-dev] [PATCH v13 1/6] lib: add information metrics library Remy Horton
2017-03-30 10:01     ` Pattan, Reshma
2017-03-29 18:28   ` [dpdk-dev] [PATCH v13 2/6] app/proc_info: add metrics displaying Remy Horton
2017-03-29 18:28   ` [dpdk-dev] [PATCH v13 3/6] lib: add bitrate statistics library Remy Horton
2017-03-30 10:04     ` Pattan, Reshma
2017-03-30 10:08     ` Pattan, Reshma
2017-03-29 18:28   ` [dpdk-dev] [PATCH v13 4/6] app/test-pmd: add bitrate statistics calculation Remy Horton
2017-03-29 18:28   ` [dpdk-dev] [PATCH v13 5/6] lib: added new library for latency stats Remy Horton
2017-03-29 18:28   ` [dpdk-dev] [PATCH v13 6/6] app/test-pmd: add latency statistics calculation Remy Horton
2017-03-30 10:22     ` Pattan, Reshma
2017-03-30 13:18   ` [dpdk-dev] [PATCH v13 0/6] Expanded statistics reporting Pattan, Reshma
2017-03-30 21:00   ` [dpdk-dev] [PATCH v14 " Remy Horton
2017-03-30 21:00     ` [dpdk-dev] [PATCH v14 1/6] lib: add information metrics library Remy Horton
2017-04-05 13:39       ` Thomas Monjalon
2017-04-05 14:46         ` Remy Horton
2017-04-05 15:06           ` Thomas Monjalon
2017-04-05 15:16             ` Remy Horton
2017-04-05 15:23               ` Thomas Monjalon
2017-03-30 21:00     ` [dpdk-dev] [PATCH v14 2/6] app/proc_info: add metrics displaying Remy Horton
2017-03-30 21:00     ` [dpdk-dev] [PATCH v14 3/6] lib: add bitrate statistics library Remy Horton
2017-03-30 21:01     ` [dpdk-dev] [PATCH v14 4/6] app/test-pmd: add bitrate statistics calculation Remy Horton
2017-03-30 21:01     ` [dpdk-dev] [PATCH v14 5/6] lib: added new library for latency stats Remy Horton
2017-04-05 14:24       ` Thomas Monjalon
2017-04-05 14:34         ` Remy Horton
2017-03-30 21:01     ` [dpdk-dev] [PATCH v14 6/6] app/test-pmd: add latency statistics calculation Remy Horton
2017-04-05 16:01     ` [dpdk-dev] [PATCH v14 0/6] Expanded statistics reporting Thomas Monjalon

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).