From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by dpdk.org (Postfix) with ESMTP id 5C7381E2F for ; Thu, 30 Mar 2017 23:01:10 +0200 (CEST) Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga105.jf.intel.com with ESMTP; 30 Mar 2017 14:01:09 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.36,248,1486454400"; d="scan'208";a="81136789" Received: from mprzygod-mobl.ger.corp.intel.com (HELO FC23.ger.corp.intel.com) ([10.252.23.81]) by orsmga005.jf.intel.com with ESMTP; 30 Mar 2017 14:01:08 -0700 From: Remy Horton To: dev@dpdk.org Cc: Thomas Monjalon Date: Thu, 30 Mar 2017 22:00:59 +0100 Message-Id: <1490907662-27666-4-git-send-email-remy.horton@intel.com> X-Mailer: git-send-email 2.5.5 In-Reply-To: <1490907662-27666-1-git-send-email-remy.horton@intel.com> References: <1490907662-27666-1-git-send-email-remy.horton@intel.com> Subject: [dpdk-dev] [PATCH v14 3/6] lib: add bitrate statistics library X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Mar 2017 21:01:11 -0000 This patch adds a library that calculates peak and average data-rate statistics. For ethernet devices. These statistics are reported using the metrics library. Signed-off-by: Remy Horton --- MAINTAINERS | 4 + config/common_base | 5 + doc/api/doxy-api-index.md | 1 + doc/api/doxy-api.conf | 1 + doc/guides/prog_guide/metrics_lib.rst | 65 +++++++++ doc/guides/rel_notes/release_17_05.rst | 6 + lib/Makefile | 2 + lib/librte_bitratestats/Makefile | 53 ++++++++ lib/librte_bitratestats/rte_bitrate.c | 146 +++++++++++++++++++++ lib/librte_bitratestats/rte_bitrate.h | 84 ++++++++++++ .../rte_bitratestats_version.map | 9 ++ mk/rte.app.mk | 1 + 12 files changed, 377 insertions(+) create mode 100644 lib/librte_bitratestats/Makefile create mode 100644 lib/librte_bitratestats/rte_bitrate.c create mode 100644 lib/librte_bitratestats/rte_bitrate.h create mode 100644 lib/librte_bitratestats/rte_bitratestats_version.map diff --git a/MAINTAINERS b/MAINTAINERS index 4eede38..09afcd3 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -630,6 +630,10 @@ Metrics M: Remy Horton F: lib/librte_metrics/ +Bit-rate statistics +M: Remy Horton +F: lib/librte_bitratestats/ + Test Applications ----------------- diff --git a/config/common_base b/config/common_base index 52394d9..1b59174 100644 --- a/config/common_base +++ b/config/common_base @@ -632,3 +632,8 @@ CONFIG_RTE_TEST_PMD_RECORD_BURST_STATS=n # Compile the crypto performance application # CONFIG_RTE_APP_CRYPTO_PERF=y + +# +# Compile the bitrate statistics library +# +CONFIG_RTE_LIBRTE_BITRATE=y diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md index 1cbe128..18c8676 100644 --- a/doc/api/doxy-api-index.md +++ b/doc/api/doxy-api-index.md @@ -157,4 +157,5 @@ There are many libraries, so their headers may be grouped by topics: [ABI compat] (@ref rte_compat.h), [keepalive] (@ref rte_keepalive.h), [device metrics] (@ref rte_metrics.h), + [bitrate statistics] (@ref rte_bitrate.h), [version] (@ref rte_version.h) diff --git a/doc/api/doxy-api.conf b/doc/api/doxy-api.conf index 0562814..7a971b6 100644 --- a/doc/api/doxy-api.conf +++ b/doc/api/doxy-api.conf @@ -36,6 +36,7 @@ INPUT = doc/api/doxy-api-index.md \ lib/librte_eal/common/include \ lib/librte_eal/common/include/generic \ lib/librte_acl \ + lib/librte_bitratestats \ lib/librte_cfgfile \ lib/librte_cmdline \ lib/librte_compat \ diff --git a/doc/guides/prog_guide/metrics_lib.rst b/doc/guides/prog_guide/metrics_lib.rst index 87f806d..71c3a1b 100644 --- a/doc/guides/prog_guide/metrics_lib.rst +++ b/doc/guides/prog_guide/metrics_lib.rst @@ -178,3 +178,68 @@ print out all metrics for a given port: free(metrics); free(names); } + + +Bit-rate statistics library +--------------------------- + +The bit-rate library calculates the exponentially-weighted moving +average and peak bit-rates for each active port (i.e. network device). +These statistics are reported via the metrics library using the +following names: + + - ``mean_bits_in``: Average inbound bit-rate + - ``mean_bits_out``: Average outbound bit-rate + - ``ewma_bits_in``: Average inbound bit-rate (EWMA smoothed) + - ``ewma_bits_out``: Average outbound bit-rate (EWMA smoothed) + - ``peak_bits_in``: Peak inbound bit-rate + - ``peak_bits_out``: Peak outbound bit-rate + +Once initialised and clocked at the appropriate frequency, these +statistics can be obtained by querying the metrics library. + +Initialization +~~~~~~~~~~~~~~ + +Before the library can be used, it has to be initialised by calling +``rte_stats_bitrate_create()``, which will return a bit-rate +calculation object. Since the bit-rate library uses the metrics library +to report the calculated statistics, the bit-rate library then needs to +register the calculated statistics with the metrics library. This is +done using the helper function ``rte_stats_bitrate_reg()``. + +.. code-block:: c + + struct rte_stats_bitrates *bitrate_data; + + bitrate_data = rte_stats_bitrate_create(); + if (bitrate_data == NULL) + rte_exit(EXIT_FAILURE, "Could not allocate bit-rate data.\n"); + rte_stats_bitrate_reg(bitrate_data); + +Controlling the sampling rate +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Since the library works by periodic sampling but does not use an +internal thread, the application has to periodically call +``rte_stats_bitrate_calc()``. The frequency at which this function +is called should be the intended sampling rate required for the +calculated statistics. For instance if per-second statistics are +desired, this function should be called once a second. + +.. code-block:: c + + tics_datum = rte_rdtsc(); + tics_per_1sec = rte_get_timer_hz(); + + while( 1 ) { + /* ... */ + tics_current = rte_rdtsc(); + if (tics_current - tics_datum >= tics_per_1sec) { + /* Periodic bitrate calculation */ + for (idx_port = 0; idx_port < cnt_ports; idx_port++) + rte_stats_bitrate_calc(bitrate_data, idx_port); + tics_datum = tics_current; + } + /* ... */ + } diff --git a/doc/guides/rel_notes/release_17_05.rst b/doc/guides/rel_notes/release_17_05.rst index e8695b2..9699541 100644 --- a/doc/guides/rel_notes/release_17_05.rst +++ b/doc/guides/rel_notes/release_17_05.rst @@ -95,6 +95,11 @@ Resolved Issues reporting mechanism that is independent of other libraries such as ethdev. +* **Added bit-rate calculation library.** + + A library that can be used to calculate device bit-rates. Calculated + bitrates are reported using the metrics library. + EAL ~~~ @@ -228,6 +233,7 @@ The libraries prepended with a plus sign were incremented in this version. .. code-block:: diff librte_acl.so.2 + + librte_bitratestats.so.1 librte_cfgfile.so.2 librte_cmdline.so.2 librte_cryptodev.so.2 diff --git a/lib/Makefile b/lib/Makefile index 4da6daf..90ab559 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -70,6 +70,8 @@ DIRS-$(CONFIG_RTE_LIBRTE_JOBSTATS) += librte_jobstats DEPDIRS-librte_jobstats := librte_eal DIRS-$(CONFIG_RTE_LIBRTE_METRICS) += librte_metrics DEPDIRS-librte_metrics := librte_eal +DIRS-$(CONFIG_RTE_LIBRTE_BITRATE) += librte_bitratestats +DEPDIRS-librte_bitratestats := librte_eal librte_metrics librte_ether DIRS-$(CONFIG_RTE_LIBRTE_POWER) += librte_power DEPDIRS-librte_power := librte_eal DIRS-$(CONFIG_RTE_LIBRTE_METER) += librte_meter diff --git a/lib/librte_bitratestats/Makefile b/lib/librte_bitratestats/Makefile new file mode 100644 index 0000000..74fd75a --- /dev/null +++ b/lib/librte_bitratestats/Makefile @@ -0,0 +1,53 @@ +# BSD LICENSE +# +# Copyright(c) 2017 Intel Corporation. All rights reserved. +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in +# the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Intel Corporation nor the names of its +# contributors may be used to endorse or promote products derived +# from this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +include $(RTE_SDK)/mk/rte.vars.mk + +# library name +LIB = librte_bitratestats.a + +CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3 + +EXPORT_MAP := rte_bitratestats_version.map + +LIBABIVER := 1 + +# all source are stored in SRCS-y +SRCS-$(CONFIG_RTE_LIBRTE_BITRATE) := rte_bitrate.c + +# Install header file +SYMLINK-$(CONFIG_RTE_LIBRTE_BITRATE)-include += rte_bitrate.h + +DEPDIRS-$(CONFIG_RTE_LIBRTE_BITRATE) += lib/librte_eal +DEPDIRS-$(CONFIG_RTE_LIBRTE_BITRATE) += lib/librte_ether +DEPDIRS-$(CONFIG_RTE_LIBRTE_BITRATE) += lib/librte_metrics + +include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/librte_bitratestats/rte_bitrate.c b/lib/librte_bitratestats/rte_bitrate.c new file mode 100644 index 0000000..260750f --- /dev/null +++ b/lib/librte_bitratestats/rte_bitrate.c @@ -0,0 +1,146 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2017 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#include +#include +#include +#include +#include + +#define ARRAY_SIZE(x) (sizeof(x) / sizeof((x)[0])) + +/* + * Persistent bit-rate data. + * @internal + */ +struct rte_stats_bitrate { + uint64_t last_ibytes; + uint64_t last_obytes; + uint64_t peak_ibits; + uint64_t peak_obits; + uint64_t mean_ibits; + uint64_t mean_obits; + uint64_t ewma_ibits; + uint64_t ewma_obits; +}; + +struct rte_stats_bitrates { + struct rte_stats_bitrate port_stats[RTE_MAX_ETHPORTS]; + uint16_t id_stats_set; +}; + +struct rte_stats_bitrates * +rte_stats_bitrate_create(void) +{ + return rte_zmalloc(NULL, sizeof(struct rte_stats_bitrates), + RTE_CACHE_LINE_SIZE); +} + +int +rte_stats_bitrate_reg(struct rte_stats_bitrates *bitrate_data) +{ + const char * const names[] = { + "ewma_bits_in", "ewma_bits_out", + "mean_bits_in", "mean_bits_out", + "peak_bits_in", "peak_bits_out", + }; + int return_value; + + return_value = rte_metrics_reg_names(&names[0], ARRAY_SIZE(names)); + if (return_value >= 0) + bitrate_data->id_stats_set = return_value; + return return_value; +} + +int +rte_stats_bitrate_calc(struct rte_stats_bitrates *bitrate_data, + uint8_t port_id) +{ + struct rte_stats_bitrate *port_data; + struct rte_eth_stats eth_stats; + int ret_code; + uint64_t cnt_bits; + int64_t delta; + const int64_t alpha_percent = 20; + uint64_t values[6]; + + ret_code = rte_eth_stats_get(port_id, ð_stats); + if (ret_code != 0) + return ret_code; + + port_data = &bitrate_data->port_stats[port_id]; + + /* Incoming bitrate. This is an iteratively calculated EWMA + * (Exponentially Weighted Moving Average) that uses a + * weighting factor of alpha_percent. An unsmoothed mean + * for just the current time delta is also calculated for the + * benefit of people who don't understand signal processing. + */ + cnt_bits = (eth_stats.ibytes - port_data->last_ibytes) << 3; + port_data->last_ibytes = eth_stats.ibytes; + if (cnt_bits > port_data->peak_ibits) + port_data->peak_ibits = cnt_bits; + delta = cnt_bits; + delta -= port_data->ewma_ibits; + /* The +-50 fixes integer rounding during divison */ + if (delta > 0) + delta = (delta * alpha_percent + 50) / 100; + else + delta = (delta * alpha_percent - 50) / 100; + port_data->ewma_ibits += delta; + port_data->mean_ibits = cnt_bits; + + /* Outgoing bitrate (also EWMA) */ + cnt_bits = (eth_stats.obytes - port_data->last_obytes) << 3; + port_data->last_obytes = eth_stats.obytes; + if (cnt_bits > port_data->peak_obits) + port_data->peak_obits = cnt_bits; + delta = cnt_bits; + delta -= port_data->ewma_obits; + if (delta > 0) + delta = (delta * alpha_percent + 50) / 100; + else + delta = (delta * alpha_percent - 50) / 100; + port_data->ewma_obits += delta; + port_data->mean_obits = cnt_bits; + + values[0] = port_data->ewma_ibits; + values[1] = port_data->ewma_obits; + values[2] = port_data->mean_ibits; + values[3] = port_data->mean_obits; + values[4] = port_data->peak_ibits; + values[5] = port_data->peak_obits; + rte_metrics_update_values(port_id, bitrate_data->id_stats_set, + values, ARRAY_SIZE(values)); + return 0; +} diff --git a/lib/librte_bitratestats/rte_bitrate.h b/lib/librte_bitratestats/rte_bitrate.h new file mode 100644 index 0000000..b9f11bd --- /dev/null +++ b/lib/librte_bitratestats/rte_bitrate.h @@ -0,0 +1,84 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2017 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#ifndef _RTE_BITRATE_H_ +#define _RTE_BITRATE_H_ + +/** + * Bitrate statistics data structure. + * This data structure is intentionally opaque. + */ +struct rte_stats_bitrates; + + +/** + * Allocate a bitrate statistics structure + * + * @return + * - Pointer to structure on success + * - NULL on error (zmalloc failure) + */ +struct rte_stats_bitrates *rte_stats_bitrate_create(void); + + +/** + * Register bitrate statistics with the metric library. + * + * @param bitrate_data + * Pointer allocated by rte_stats_create() + * + * @return + * Zero on success + * Negative on error + */ +int rte_stats_bitrate_reg(struct rte_stats_bitrates *bitrate_data); + + +/** + * Calculate statistics for current time window. The period with which + * this function is called should be the intended sampling window width. + * + * @param bitrate_data + * Bitrate statistics data pointer + * + * @param port_id + * Port id to calculate statistics for + * + * @return + * - Zero on success + * - Negative value on error + */ +int rte_stats_bitrate_calc(struct rte_stats_bitrates *bitrate_data, + uint8_t port_id); + +#endif /* _RTE_BITRATE_H_ */ diff --git a/lib/librte_bitratestats/rte_bitratestats_version.map b/lib/librte_bitratestats/rte_bitratestats_version.map new file mode 100644 index 0000000..fe74544 --- /dev/null +++ b/lib/librte_bitratestats/rte_bitratestats_version.map @@ -0,0 +1,9 @@ +DPDK_17.05 { + global: + + rte_stats_bitrate_calc; + rte_stats_bitrate_create; + rte_stats_bitrate_reg; + + local: *; +}; diff --git a/mk/rte.app.mk b/mk/rte.app.mk index ed3315c..7104a8b 100644 --- a/mk/rte.app.mk +++ b/mk/rte.app.mk @@ -99,6 +99,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_EAL) += -lrte_eal _LDLIBS-$(CONFIG_RTE_LIBRTE_CMDLINE) += -lrte_cmdline _LDLIBS-$(CONFIG_RTE_LIBRTE_REORDER) += -lrte_reorder _LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += -lrte_metrics +_LDLIBS-$(CONFIG_RTE_LIBRTE_BITRATE) += -lrte_bitratestats ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),n) # plugins (link only if static libraries) -- 2.5.5