From: "Pattan, Reshma" <reshma.pattan@intel.com>
To: "Horton, Remy" <remy.horton@intel.com>, "dev@dpdk.org" <dev@dpdk.org>
Cc: Thomas Monjalon <thomas.monjalon@6wind.com>
Subject: Re: [dpdk-dev] [PATCH v13 0/6] Expanded statistics reporting
Date: Thu, 30 Mar 2017 13:18:52 +0000 [thread overview]
Message-ID: <3AEA2BF9852C6F48A459DA490692831F0112F4B3@irsmsx110.ger.corp.intel.com> (raw)
In-Reply-To: <1490812128-4172-1-git-send-email-remy.horton@intel.com>
> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Remy Horton
> Sent: Wednesday, March 29, 2017 7:29 PM
> To: dev@dpdk.org
> Cc: Thomas Monjalon <thomas.monjalon@6wind.com>
> Subject: [dpdk-dev] [PATCH v13 0/6] Expanded statistics reporting
>
> This patchset consists of three libraries: A Metrics library for distributing
> device information, a library that calculates bit-rate statistics, and a library
> that calculates latency statistics. The latter two libraries make use of the first
> library.
>
> Metrics Library
> ---------------
> The Metrics library implements a mechanism by which producers can publish
> numeric information for later querying by consumers.
> In practice producers will typically be other libraries or primary processes,
> whereas consumers will typically be applications.
>
> Metrics themselves are statistics that are not generated by PMDs.
> Metric information is populated using a push model, where producers
> update the values contained within the metric library by calling an update
> function on the relevant metrics. Consumers receive metric information by
> querying the central metric data, which is held in shared memory so it can
> also be accessed by secondary processes.
>
> For each metric, a separate value is maintained for each port id, and when
> publishing metric values the producers need to specify which port is being
> updated. In addition there is a special id RTE_METRICS_GLOBAL that is
> intended for global statistics that are not associated with any individual
> device. Since the metrics library is self-contained, the only restriction on port
> numbers is that they are less than RTE_MAX_ETHPORTS - there is no
> requirement for the ports to actually exist.
>
> Metrics must first be registered, which is the way producers declare the
> names of the metrics they will be publishing. Registration can either be done
> individually, or as a group as a metric set. The intention is for all metrics in a
> set to be updated in one go, although it is also possible for metrics within a
> set to be updated individually. It is up to the producers to update metrics as
> required.
>
> Bit-rate statistics library
> ---------------------------
> The bit-rate library calculates the mean, the exponentially-weighted moving
> average, and peak bit-rates for each active port (i.e. network device). These
> statistics are then reported via the metrics library using the following names:
> mean_bits_in, mean_bits_out, ewma_bits_in, ewma_bits_out, peak_bits_in,
> and peak_bits_out. The sampling window used for calculation is decided by
> the application requiring the statistics.
>
> Latency statistics library
> --------------------------
> The latency statistics library calculates the port-to-port latency of packet
> processing by a DPDK application, reporting the minimum, average, and
> maximum nano-seconds that packet processing takes, as well as the jitter in
> processing delay. These statistics are then reported via the metrics library
> using the following names: min_latency_ns, avg_latency_ns, mac_latency_ns,
> and jitter_ns.
>
> This is dependent on http://dpdk.org/dev/patchwork/patch/21607/ for the
> timestamp MBUF field.
>
> For more details on the metrics library and the Bit-rate and Latency
> components see the Programmer's Guide updates in the patchset.
> --
> v13 changes:
> * Rebased to latest master (b88161be4227)
> * Updated copyright years to 2017
> * Some release notes were in wrong file (17.02 rather than 17.05)
> * Added a missing NULL check
> * Doxygen & comment spelling corrections
> * Authorship of some patches changed to reflect main contributer
> * EWMA calculation for inbound and outbound bitrates were subtly
> different
> * Use sizeof() rather than magic numbers for array sizes
> * The recent feb9f680cd2c ("mk: optimize directory dependencies") required
> DEPDIRS-librte_ dependency information to be added to lib/Makefile
>
> v12 changes:
> * Rebased to latest master
> * Misspellings in MAINTAINERS corrected
> * Removed MBUF modifications (use Olivier's MBUF rework)
>
> v11 changes:
> * Rebased
> * .map references to 17.02 changed to 17.05
> * Release ntoes moved to release_17_05.rst
> * Bit-rate library now also gives unfiltered average
>
> v10 changes:
> * Rebased
> * Relocated some config-related directives.
> * Removed incorrect capitalisations in API docs.
> * Formatting & detail corrections in release notes.
> * Moved around struct member descriptions.
> * Rewritten rte_metrics.h file description.
> * Rewritten description of RTE_METRICS_GLOBAL.
> * Used 'producers' and 'consumers' as terms.
> * Removed markup (bold text) in Doxygen tags.
> * Added programming guide section.
>
> v9 changes:
> * Updated .map files to reflect function changes in v8
> * Fixed rte_malloc() of zero bytes in proc_info when no metrics exist
> * Fixed rte_metrics_init not being called explicitly in testpmd
>
> v8 changes:
> * Release notes correction
> * Updated copyright years
> * rte_metric_init() takes socket id & must be explicitly called
> * rte_metrics_reg_metric renamed to rte_metrics_reg_name()
> * rte_metrics_update_metric() renamed to rte_metrics_update_value()
> * Doxygen updates
> * Changed malloc()/free() to rte_malloc()/rte_free()
> * Removed redundant memset()
> * rte_stats_bitrates_s renamed to rte_stats_bitrates_s
> * Split mbuf change to own patch for visibility
> * CYCLES_PER_NS now a static inline function
> * latency: "hidden" pthread creation now has polling API instead.
> * Struct declarations and variable definitions cleaned up
> * Double initialization of the latency library now returns -EEXIST
> * MAINTAINERS entry for layenctstats in correct section
>
> v7 changes:
> * RTE_METRICS_NONPORT renamed to RTE_METRICS_GLOBAL
> * Multiple changes to rte_metrics.h doxygen documentation
> * Split apart latency patch into lib, test-pmd, & proc_info parts
> * Reordered patches by functionality
> * Insufficent capacity return value changed from -ERANGE to actual size
> * Cache alignment in bitrate library
> * Tightened up const usage to avoid STATIC_CONST_CHAR_ARRAY warning
> * Reshma reinstated as author for (now split) latency patch
> * Rebase to master
>
> v6 changes:
> * Metrics display now has "Non port specific" rather than "port -1"
> * Fixed sign issue in EWMA delta calculation
> * Rebased to master
>
> v5 changes:
> * Updated Shared Library Versions in release notes
> * Merged in Reshma's latencystats library
>
> v4 changes:
> * References to 16.11 changed to 17.02
> * Fetching of non-port values was broken
> * Added sanity checks to value fetching
> * rte_stat_value renamed to rte_metric_value
> * Corrected doxygen descriptions
> * Added MAINTAINERS entries
> * Added #ifdef directives to bitrate code in test-pmd
>
> v3 changes:
> * Marked rte_stats_bitrate_s as internal
> * Minor integer roundoff correction
> * Coding style corrections
> * Removed spurious object allocation
> * Changes to rte_metrics.[ch] moved from Patch 2/3 to 1/3.
> * Reintroduced non-port values (RTE_METRICS_NONPORT)
> * Added spinlocks to metric library
> * Removed spurious test registration/update
> * Added release notes entries
>
> v2 changes:
> * Uses a new metrics library rather than being part of ethdev
>
>
> Remy Horton (3):
> lib: add information metrics library
> lib: add bitrate statistics library
> app/test-pmd: add bitrate statistics calculation
>
> Reshma Pattan (3):
> app/proc_info: add metrics displaying
> lib: added new library for latency stats
> app/test-pmd: add latency statistics calculation
>
> MAINTAINERS | 12 +
> app/proc_info/main.c | 79 ++++-
> app/test-pmd/parameters.c | 20 +-
> app/test-pmd/testpmd.c | 78 ++++-
> app/test-pmd/testpmd.h | 6 +-
> config/common_base | 15 +
> doc/api/doxy-api-index.md | 5 +-
> doc/api/doxy-api.conf | 5 +-
> doc/guides/prog_guide/index.rst | 3 +-
> doc/guides/prog_guide/metrics_lib.rst | 299 +++++++++++++++++
> doc/guides/rel_notes/release_17_05.rst | 21 ++
> lib/Makefile | 6 +
> lib/librte_bitratestats/Makefile | 53 +++
> lib/librte_bitratestats/rte_bitrate.c | 145 +++++++++
> lib/librte_bitratestats/rte_bitrate.h | 84 +++++
> .../rte_bitratestats_version.map | 9 +
> lib/librte_latencystats/Makefile | 56 ++++
> lib/librte_latencystats/rte_latencystats.c | 361
> +++++++++++++++++++++
> lib/librte_latencystats/rte_latencystats.h | 154 +++++++++
> .../rte_latencystats_version.map | 11 +
> lib/librte_metrics/Makefile | 51 +++
> lib/librte_metrics/rte_metrics.c | 302 +++++++++++++++++
> lib/librte_metrics/rte_metrics.h | 240 ++++++++++++++
> lib/librte_metrics/rte_metrics_version.map | 13 +
> mk/rte.app.mk | 3 +
> 25 files changed, 2023 insertions(+), 8 deletions(-) create mode 100644
> doc/guides/prog_guide/metrics_lib.rst
> create mode 100644 lib/librte_bitratestats/Makefile create mode 100644
> lib/librte_bitratestats/rte_bitrate.c
> create mode 100644 lib/librte_bitratestats/rte_bitrate.h
> create mode 100644 lib/librte_bitratestats/rte_bitratestats_version.map
> create mode 100644 lib/librte_latencystats/Makefile create mode 100644
> lib/librte_latencystats/rte_latencystats.c
> create mode 100644 lib/librte_latencystats/rte_latencystats.h
> create mode 100644 lib/librte_latencystats/rte_latencystats_version.map
> create mode 100644 lib/librte_metrics/Makefile create mode 100644
> lib/librte_metrics/rte_metrics.c create mode 100644
> lib/librte_metrics/rte_metrics.h create mode 100644
> lib/librte_metrics/rte_metrics_version.map
>
> --
> 2.5.5
Since I have only very few minor comments on this patch set , I will ack this patch, please feel free to add my ack for next patch set.
Series Acked-by: Reshma Pattan <reshma.pattan@intel.com>
next prev parent reply other threads:[~2017-03-30 13:18 UTC|newest]
Thread overview: 41+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-03-27 20:21 [dpdk-dev] [PATCH v12 " Remy Horton
2017-03-27 20:21 ` [dpdk-dev] [PATCH v12 1/6] lib: add information metrics library Remy Horton
2017-03-28 14:14 ` Pattan, Reshma
2017-03-28 15:00 ` Remy Horton
2017-03-27 20:21 ` [dpdk-dev] [PATCH v12 2/6] app/proc_info: add metrics displaying Remy Horton
2017-03-27 20:21 ` [dpdk-dev] [PATCH v12 3/6] lib: add bitrate statistics library Remy Horton
2017-03-28 15:30 ` Pattan, Reshma
2017-03-28 15:45 ` Remy Horton
2017-03-27 20:21 ` [dpdk-dev] [PATCH v12 4/6] app/test-pmd: add bitrate statistics calculation Remy Horton
2017-03-28 15:49 ` Pattan, Reshma
2017-03-27 20:21 ` [dpdk-dev] [PATCH v12 5/6] lib: added new library for latency stats Remy Horton
2017-03-27 20:21 ` [dpdk-dev] [PATCH v12 6/6] app/test-pmd: add latency statistics calculation Remy Horton
2017-03-27 21:03 ` [dpdk-dev] [PATCH v12 0/6] Expanded statistics reporting Stephen Hemminger
2017-03-28 11:21 ` Mcnamara, John
2017-03-29 18:28 ` [dpdk-dev] [PATCH v13 " Remy Horton
2017-03-29 18:28 ` [dpdk-dev] [PATCH v13 1/6] lib: add information metrics library Remy Horton
2017-03-30 10:01 ` Pattan, Reshma
2017-03-29 18:28 ` [dpdk-dev] [PATCH v13 2/6] app/proc_info: add metrics displaying Remy Horton
2017-03-29 18:28 ` [dpdk-dev] [PATCH v13 3/6] lib: add bitrate statistics library Remy Horton
2017-03-30 10:04 ` Pattan, Reshma
2017-03-30 10:08 ` Pattan, Reshma
2017-03-29 18:28 ` [dpdk-dev] [PATCH v13 4/6] app/test-pmd: add bitrate statistics calculation Remy Horton
2017-03-29 18:28 ` [dpdk-dev] [PATCH v13 5/6] lib: added new library for latency stats Remy Horton
2017-03-29 18:28 ` [dpdk-dev] [PATCH v13 6/6] app/test-pmd: add latency statistics calculation Remy Horton
2017-03-30 10:22 ` Pattan, Reshma
2017-03-30 13:18 ` Pattan, Reshma [this message]
2017-03-30 21:00 ` [dpdk-dev] [PATCH v14 0/6] Expanded statistics reporting Remy Horton
2017-03-30 21:00 ` [dpdk-dev] [PATCH v14 1/6] lib: add information metrics library Remy Horton
2017-04-05 13:39 ` Thomas Monjalon
2017-04-05 14:46 ` Remy Horton
2017-04-05 15:06 ` Thomas Monjalon
2017-04-05 15:16 ` Remy Horton
2017-04-05 15:23 ` Thomas Monjalon
2017-03-30 21:00 ` [dpdk-dev] [PATCH v14 2/6] app/proc_info: add metrics displaying Remy Horton
2017-03-30 21:00 ` [dpdk-dev] [PATCH v14 3/6] lib: add bitrate statistics library Remy Horton
2017-03-30 21:01 ` [dpdk-dev] [PATCH v14 4/6] app/test-pmd: add bitrate statistics calculation Remy Horton
2017-03-30 21:01 ` [dpdk-dev] [PATCH v14 5/6] lib: added new library for latency stats Remy Horton
2017-04-05 14:24 ` Thomas Monjalon
2017-04-05 14:34 ` Remy Horton
2017-03-30 21:01 ` [dpdk-dev] [PATCH v14 6/6] app/test-pmd: add latency statistics calculation Remy Horton
2017-04-05 16:01 ` [dpdk-dev] [PATCH v14 0/6] Expanded statistics reporting Thomas Monjalon
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=3AEA2BF9852C6F48A459DA490692831F0112F4B3@irsmsx110.ger.corp.intel.com \
--to=reshma.pattan@intel.com \
--cc=dev@dpdk.org \
--cc=remy.horton@intel.com \
--cc=thomas.monjalon@6wind.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).