From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by dpdk.org (Postfix) with ESMTP id EF689298F for ; Thu, 27 Apr 2017 16:45:53 +0200 (CEST) Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 27 Apr 2017 07:45:30 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.37,384,1488873600"; d="scan'208";a="79640268" Received: from unknown ([10.103.103.28]) by orsmga002.jf.intel.com with SMTP; 27 Apr 2017 07:45:26 -0700 Received: by (sSMTP sendmail emulation); Thu, 27 Apr 2017 16:44:44 +0200 From: Michal Jastrzebski To: dev@dpdk.org Cc: harry.van.haaren@intel.com, deepak.k.jain@intel.com, Kuba Kozak Date: Thu, 27 Apr 2017 16:42:36 +0200 Message-Id: <20170427144240.10188-3-michalx.k.jastrzebski@intel.com> X-Mailer: git-send-email 2.12.2 In-Reply-To: <20170427144240.10188-1-michalx.k.jastrzebski@intel.com> References: <20170427144240.10188-1-michalx.k.jastrzebski@intel.com> Subject: [dpdk-dev] [PATCH v1 2/6] ethdev: retrieve xstats by ID X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Apr 2017 14:45:55 -0000 From: Kuba Kozak Extended xstats API in ethdev library to allow grouping of stats logically so they can be retrieved per logical grouping managed by the application. Added new functions rte_eth_xstats_get_names_by_id and rte_eth_xstats_get_by_id using additional arguments (in compare to rte_eth_xstats_get_names and rte_eth_xstats_get) - array of ids and array of values. doc: add description for modified xstats API Documentation change for new extended statistics API functions. The old API only allows retrieval of *all* of the NIC statistics at once. Given this requires a MMIO read PCI transaction per statistic it is an inefficient way of retrieving just a few key statistics. Often a monitoring agent only has an interest in a few key statistics, and the old API forces wasting CPU time and PCIe bandwidth in retrieving *all* statistics; even those that the application didn't explicitly show an interest in. The new, more flexible API allow retrieval of statistics per ID. If a PMD wishes, it can be implemented to read just the required NIC registers. As a result, the monitoring application no longer wastes PCIe bandwidth and CPU time. Signed-off-by: Kuba Kozak --- doc/guides/prog_guide/poll_mode_drv.rst | 167 ++++++++++++++++++--- doc/guides/rel_notes/release_17_05.rst | 4 + lib/librte_ether/rte_ethdev.c | 250 ++++++++++++++++++++++++++++++++ lib/librte_ether/rte_ethdev.h | 70 +++++++++ lib/librte_ether/rte_ether_version.map | 2 + 5 files changed, 476 insertions(+), 17 deletions(-) diff --git a/doc/guides/prog_guide/poll_mode_drv.rst b/doc/guides/prog_guide/poll_mode_drv.rst index e48c121..4987f70 100644 --- a/doc/guides/prog_guide/poll_mode_drv.rst +++ b/doc/guides/prog_guide/poll_mode_drv.rst @@ -334,24 +334,21 @@ The Ethernet device API exported by the Ethernet PMDs is described in the *DPDK Extended Statistics API ~~~~~~~~~~~~~~~~~~~~~~~ -The extended statistics API allows each individual PMD to expose a unique set -of statistics. Accessing these from application programs is done via two -functions: - -* ``rte_eth_xstats_get``: Fills in an array of ``struct rte_eth_xstat`` - with extended statistics. -* ``rte_eth_xstats_get_names``: Fills in an array of - ``struct rte_eth_xstat_name`` with extended statistic name lookup - information. - -Each ``struct rte_eth_xstat`` contains an identifier and value pair, and -each ``struct rte_eth_xstat_name`` contains a string. Each identifier -within the ``struct rte_eth_xstat`` lookup array must have a corresponding -entry in the ``struct rte_eth_xstat_name`` lookup array. Within the latter -the index of the entry is the identifier the string is associated with. -These identifiers, as well as the number of extended statistic exposed, must -remain constant during runtime. Note that extended statistic identifiers are +The extended statistics API allows a PMD to expose all statistics that are +available to it, including statistics that are unique to the device. +Each statistic has three properties ``name``, ``id`` and ``value``: + +* ``name``: A human readable string formatted by the scheme detailed below. +* ``id``: An integer that represents only that statistic. +* ``value``: A unsigned 64-bit integer that is the value of the statistic. + +Note that extended statistic identifiers are driver-specific, and hence might not be the same for different ports. +The API consists of various ``rte_eth_xstats_*()`` functions, and allows an +application to be flexible in how it retrieves statistics. + +Scheme for Human Readable Names +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ A naming scheme exists for the strings exposed to clients of the API. This is to allow scraping of the API for statistics of interest. The naming scheme uses @@ -392,3 +389,139 @@ Some additions in the metadata scheme are as follows: An example where queue numbers are used is as follows: ``tx_q7_bytes`` which indicates this statistic applies to queue number 7, and represents the number of transmitted bytes on that queue. + +API Design +^^^^^^^^^^ + +The xstats API uses the ``name``, ``id``, and ``value`` to allow performant +lookup of specific statistics. Performant lookup means two things; + +* No string comparisons with the ``name`` of the statistic in fast-path +* Allow requesting of only the statistics of interest + +The API ensures these requirements are met by mapping the ``name`` of the +statistic to a unique ``id``, which is used as a key for lookup in the fast-path. +The API allows applications to request an array of ``id`` values, so that the +PMD only performs the required calculations. Expected usage is that the +application scans the ``name`` of each statistic, and caches the ``id`` +if it has an interest in that statistic. On the fast-path, the integer can be used +to retrieve the actual ``value`` of the statistic that the ``id`` represents. + +API Functions +^^^^^^^^^^^^^ + +The API is built out of a small number of functions, which can be used to +retrieve the number of statistics and the names, IDs and values of those +statistics. + +* ``rte_eth_xstats_get_names_by_id()``: returns the names of the statistics. When given a + ``NULL`` parameter the function returns the number of statistics that are available. + +* ``rte_eth_xstats_get_id_by_name()``: Searches for the statistic ID that matches + ``xstat_name``. If found, the ``id`` integer is set. + +* ``rte_eth_xstats_get_by_id()``: Fills in an array of ``uint64_t`` values + with matching the provided ``ids`` array. If the ``ids`` array is NULL, it + returns all statistics that are available. + + +Application Usage +^^^^^^^^^^^^^^^^^ + +Imagine an application that wants to view the dropped packet count. If no +packets are dropped, the application does not read any other metrics for +performance reasons. If packets are dropped, the application has a particular +set of statistics that it requests. This "set" of statistics allows the app to +decide what next steps to perform. The following code-snippets show how the +xstats API can be used to achieve this goal. + +First step is to get all statistics names and list them: + +.. code-block:: c + + struct rte_eth_xstat_name *xstats_names; + uint64_t *values; + int len, i; + + /* Get number of stats */ + len = rte_eth_xstats_get_names_by_id(port_id, NULL, NULL, 0); + if (len < 0) { + printf("Cannot get xstats count\n"); + goto err; + } + + xstats_names = malloc(sizeof(struct rte_eth_xstat_name) * len); + if (xstats_names == NULL) { + printf("Cannot allocate memory for xstat names\n"); + goto err; + } + + /* Retrieve xstats names, passing NULL for IDs to return all statistics */ + if (len != rte_eth_xstats_get_names_by_id(port_id, xstats_names, NULL, len)) { + printf("Cannot get xstat names\n"); + goto err; + } + + values = malloc(sizeof(values) * len); + if (values == NULL) { + printf("Cannot allocate memory for xstats\n"); + goto err; + } + + /* Getting xstats values */ + if (len != rte_eth_xstats_get_by_id(port_id, NULL, values, len)) { + printf("Cannot get xstat values\n"); + goto err; + } + + /* Print all xstats names and values */ + for (i = 0; i < len; i++) { + printf("%s: %"PRIu64"\n", xstats_names[i].name, values[i]); + } + +The application has access to the names of all of the statistics that the PMD +exposes. The application can decide which statistics are of interest, cache the +ids of those statistics by looking up the name as follows: + +.. code-block:: c + + uint64_t id; + uint64_t value; + const char *xstat_name = "rx_errors"; + + if(!rte_eth_xstats_get_id_by_name(port_id, xstat_name, &id)) { + rte_eth_xstats_get_by_id(port_id, &id, &value, 1); + printf("%s: %"PRIu64"\n", xstat_name, value); + } + else { + printf("Cannot find xstats with a given name\n"); + goto err; + } + +The API provides flexibility to the application so that it can look up multiple +statistics using an array containing multiple ``id`` numbers. This reduces the +function call overhead of retrieving statistics, and makes lookup of multiple +statistics simpler for the application. + +.. code-block:: c + + #define APP_NUM_STATS 4 + /* application cached these ids previously; see above */ + uint64_t ids_array[APP_NUM_STATS] = {3,4,7,21}; + uint64_t value_array[APP_NUM_STATS]; + + /* Getting multiple xstats values from array of IDs */ + rte_eth_xstats_get_by_id(port_id, ids_array, value_array, APP_NUM_STATS); + + uint32_t i; + for(i = 0; i < APP_NUM_STATS; i++) { + printf("%d: %"PRIu64"\n", ids_array[i], value_array[i]); + } + + +This array lookup API for xstats allows the application create multiple +"groups" of statistics, and look up the values of those IDs using a single API +call. As an end result, the application is able to achieve its goal of +monitoring a single statistic ("rx_errors" in this case), and if that shows +packets being dropped, it can easily retrieve a "set" of statistics using the +IDs array parameter to ``rte_eth_xstats_get_by_id`` function. diff --git a/doc/guides/rel_notes/release_17_05.rst b/doc/guides/rel_notes/release_17_05.rst index dcd55ff..12d6b49 100644 --- a/doc/guides/rel_notes/release_17_05.rst +++ b/doc/guides/rel_notes/release_17_05.rst @@ -450,6 +450,10 @@ API Changes * The vhost public header file ``rte_virtio_net.h`` is renamed to ``rte_vhost.h`` +* **Reworked rte_ethdev library** + + * Added new functions ``rte_eth_xstats_get_by_id`` and ``rte_eth_xstats_get_names_by_id`` + ABI Changes ----------- diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 9922430..331d2e3 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -1360,12 +1360,19 @@ get_xstats_count(uint8_t port_id) RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL); dev = &rte_eth_devices[port_id]; + if (dev->dev_ops->xstats_get_names_by_id != NULL) { + count = (*dev->dev_ops->xstats_get_names_by_id)(dev, NULL, + NULL, 0); + if (count < 0) + return count; + } if (dev->dev_ops->xstats_get_names != NULL) { count = (*dev->dev_ops->xstats_get_names)(dev, NULL, 0); if (count < 0) return count; } else count = 0; + count += RTE_NB_STATS; count += RTE_MIN(dev->data->nb_rx_queues, RTE_ETHDEV_QUEUE_STAT_CNTRS) * RTE_NB_RXQ_STATS; @@ -1375,6 +1382,123 @@ get_xstats_count(uint8_t port_id) } int +rte_eth_xstats_get_names_by_id(uint8_t port_id, + struct rte_eth_xstat_name *xstats_names, unsigned int size, + uint64_t *ids) +{ + /* Get all xstats */ + if (!ids) { + struct rte_eth_dev *dev; + int cnt_used_entries; + int cnt_expected_entries; + int cnt_driver_entries; + uint32_t idx, id_queue; + uint16_t num_q; + + cnt_expected_entries = get_xstats_count(port_id); + if (xstats_names == NULL || cnt_expected_entries < 0 || + (int)size < cnt_expected_entries) + return cnt_expected_entries; + + /* port_id checked in get_xstats_count() */ + dev = &rte_eth_devices[port_id]; + cnt_used_entries = 0; + + for (idx = 0; idx < RTE_NB_STATS; idx++) { + snprintf(xstats_names[cnt_used_entries].name, + sizeof(xstats_names[0].name), + "%s", rte_stats_strings[idx].name); + cnt_used_entries++; + } + num_q = RTE_MIN(dev->data->nb_rx_queues, + RTE_ETHDEV_QUEUE_STAT_CNTRS); + for (id_queue = 0; id_queue < num_q; id_queue++) { + for (idx = 0; idx < RTE_NB_RXQ_STATS; idx++) { + snprintf(xstats_names[cnt_used_entries].name, + sizeof(xstats_names[0].name), + "rx_q%u%s", + id_queue, + rte_rxq_stats_strings[idx].name); + cnt_used_entries++; + } + + } + num_q = RTE_MIN(dev->data->nb_tx_queues, + RTE_ETHDEV_QUEUE_STAT_CNTRS); + for (id_queue = 0; id_queue < num_q; id_queue++) { + for (idx = 0; idx < RTE_NB_TXQ_STATS; idx++) { + snprintf(xstats_names[cnt_used_entries].name, + sizeof(xstats_names[0].name), + "tx_q%u%s", + id_queue, + rte_txq_stats_strings[idx].name); + cnt_used_entries++; + } + } + + if (dev->dev_ops->xstats_get_names_by_id != NULL) { + /* If there are any driver-specific xstats, append them + * to end of list. + */ + cnt_driver_entries = + (*dev->dev_ops->xstats_get_names_by_id)( + dev, + xstats_names + cnt_used_entries, + NULL, + size - cnt_used_entries); + if (cnt_driver_entries < 0) + return cnt_driver_entries; + cnt_used_entries += cnt_driver_entries; + + } else if (dev->dev_ops->xstats_get_names != NULL) { + /* If there are any driver-specific xstats, append them + * to end of list. + */ + cnt_driver_entries = (*dev->dev_ops->xstats_get_names)( + dev, + xstats_names + cnt_used_entries, + size - cnt_used_entries); + if (cnt_driver_entries < 0) + return cnt_driver_entries; + cnt_used_entries += cnt_driver_entries; + } + + return cnt_used_entries; + } + /* Get only xstats given by IDS */ + else { + uint16_t len, i; + struct rte_eth_xstat_name *xstats_names_copy; + + len = rte_eth_xstats_get_names_by_id(port_id, NULL, 0, NULL); + + xstats_names_copy = + malloc(sizeof(struct rte_eth_xstat_name) * len); + if (!xstats_names_copy) { + RTE_PMD_DEBUG_TRACE( + "ERROR: can't allocate memory for values_copy\n"); + free(xstats_names_copy); + return -1; + } + + rte_eth_xstats_get_names_by_id(port_id, xstats_names_copy, + len, NULL); + + for (i = 0; i < size; i++) { + if (ids[i] >= len) { + RTE_PMD_DEBUG_TRACE( + "ERROR: id value isn't valid\n"); + return -1; + } + strcpy(xstats_names[i].name, + xstats_names_copy[ids[i]].name); + } + free(xstats_names_copy); + return size; + } +} + +int rte_eth_xstats_get_names(uint8_t port_id, struct rte_eth_xstat_name *xstats_names, unsigned int size) @@ -1441,6 +1565,132 @@ rte_eth_xstats_get_names(uint8_t port_id, /* retrieve ethdev extended statistics */ int +rte_eth_xstats_get_by_id(uint8_t port_id, const uint64_t *ids, uint64_t *values, + unsigned int n) +{ + /* If need all xstats */ + if (!ids) { + struct rte_eth_stats eth_stats; + struct rte_eth_dev *dev; + unsigned int count = 0, i, q; + signed int xcount = 0; + uint64_t val, *stats_ptr; + uint16_t nb_rxqs, nb_txqs; + + RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV); + dev = &rte_eth_devices[port_id]; + + nb_rxqs = RTE_MIN(dev->data->nb_rx_queues, + RTE_ETHDEV_QUEUE_STAT_CNTRS); + nb_txqs = RTE_MIN(dev->data->nb_tx_queues, + RTE_ETHDEV_QUEUE_STAT_CNTRS); + + /* Return generic statistics */ + count = RTE_NB_STATS + (nb_rxqs * RTE_NB_RXQ_STATS) + + (nb_txqs * RTE_NB_TXQ_STATS); + + + /* implemented by the driver */ + if (dev->dev_ops->xstats_get_by_id != NULL) { + /* Retrieve the xstats from the driver at the end of the + * xstats struct. Retrieve all xstats. + */ + xcount = (*dev->dev_ops->xstats_get_by_id)(dev, + NULL, + values ? values + count : NULL, + (n > count) ? n - count : 0); + + if (xcount < 0) + return xcount; + /* implemented by the driver */ + } else if (dev->dev_ops->xstats_get != NULL) { + /* Retrieve the xstats from the driver at the end of the + * xstats struct. Retrieve all xstats. + * Compatibility for PMD without xstats_get_by_ids + */ + unsigned int size = (n > count) ? n - count : 1; + struct rte_eth_xstat xstats[size]; + + xcount = (*dev->dev_ops->xstats_get)(dev, + values ? xstats : NULL, size); + + if (xcount < 0) + return xcount; + + if (values != NULL) + for (i = 0 ; i < (unsigned int)xcount; i++) + values[i + count] = xstats[i].value; + } + + if (n < count + xcount || values == NULL) + return count + xcount; + + /* now fill the xstats structure */ + count = 0; + rte_eth_stats_get(port_id, ð_stats); + + /* global stats */ + for (i = 0; i < RTE_NB_STATS; i++) { + stats_ptr = RTE_PTR_ADD(ð_stats, + rte_stats_strings[i].offset); + val = *stats_ptr; + values[count++] = val; + } + + /* per-rxq stats */ + for (q = 0; q < nb_rxqs; q++) { + for (i = 0; i < RTE_NB_RXQ_STATS; i++) { + stats_ptr = RTE_PTR_ADD(ð_stats, + rte_rxq_stats_strings[i].offset + + q * sizeof(uint64_t)); + val = *stats_ptr; + values[count++] = val; + } + } + + /* per-txq stats */ + for (q = 0; q < nb_txqs; q++) { + for (i = 0; i < RTE_NB_TXQ_STATS; i++) { + stats_ptr = RTE_PTR_ADD(ð_stats, + rte_txq_stats_strings[i].offset + + q * sizeof(uint64_t)); + val = *stats_ptr; + values[count++] = val; + } + } + + return count + xcount; + } + /* Need only xstats given by IDS array */ + else { + uint16_t i, size; + uint64_t *values_copy; + + size = rte_eth_xstats_get_by_id(port_id, NULL, NULL, 0); + + values_copy = malloc(sizeof(values_copy) * size); + if (!values_copy) { + RTE_PMD_DEBUG_TRACE( + "ERROR: can't allocate memory for values_copy\n"); + return -1; + } + + rte_eth_xstats_get_by_id(port_id, NULL, values_copy, size); + + for (i = 0; i < n; i++) { + if (ids[i] >= size) { + RTE_PMD_DEBUG_TRACE( + "ERROR: id value isn't valid\n"); + return -1; + } + values[i] = values_copy[ids[i]]; + } + free(values_copy); + return n; + } +} + +int rte_eth_xstats_get(uint8_t port_id, struct rte_eth_xstat *xstats, unsigned int n) { diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h index a46290c..8594416 100644 --- a/lib/librte_ether/rte_ethdev.h +++ b/lib/librte_ether/rte_ethdev.h @@ -185,6 +185,7 @@ extern "C" { #include "rte_ether.h" #include "rte_eth_ctrl.h" #include "rte_dev_info.h" +#include "rte_compat.h" struct rte_mbuf; @@ -1121,6 +1122,12 @@ typedef int (*eth_xstats_get_t)(struct rte_eth_dev *dev, struct rte_eth_xstat *stats, unsigned n); /**< @internal Get extended stats of an Ethernet device. */ +typedef int (*eth_xstats_get_by_id_t)(struct rte_eth_dev *dev, + const uint64_t *ids, + uint64_t *values, + unsigned int n); +/**< @internal Get extended stats of an Ethernet device. */ + typedef void (*eth_xstats_reset_t)(struct rte_eth_dev *dev); /**< @internal Reset extended stats of an Ethernet device. */ @@ -1128,6 +1135,11 @@ typedef int (*eth_xstats_get_names_t)(struct rte_eth_dev *dev, struct rte_eth_xstat_name *xstats_names, unsigned size); /**< @internal Get names of extended stats of an Ethernet device. */ +typedef int (*eth_xstats_get_names_by_id_t)(struct rte_eth_dev *dev, + struct rte_eth_xstat_name *xstats_names, const uint64_t *ids, + unsigned int size); +/**< @internal Get names of extended stats of an Ethernet device. */ + typedef int (*eth_queue_stats_mapping_set_t)(struct rte_eth_dev *dev, uint16_t queue_id, uint8_t stat_idx, @@ -1566,6 +1578,11 @@ struct eth_dev_ops { eth_timesync_adjust_time timesync_adjust_time; /** Adjust the device clock. */ eth_timesync_read_time timesync_read_time; /** Get the device clock time. */ eth_timesync_write_time timesync_write_time; /** Set the device clock time. */ + + eth_xstats_get_by_id_t xstats_get_by_id; + /**< Get extended device statistic values by ID. */ + eth_xstats_get_names_by_id_t xstats_get_names_by_id; + /**< Get name of extended device statistics by ID. */ }; /** @@ -2319,6 +2336,59 @@ int rte_eth_xstats_get(uint8_t port_id, struct rte_eth_xstat *xstats, unsigned int n); /** + * Retrieve names of extended statistics of an Ethernet device. + * + * @param port_id + * The port identifier of the Ethernet device. + * @param xstats_names + * An rte_eth_xstat_name array of at least *size* elements to + * be filled. If set to NULL, the function returns the required number + * of elements. + * @param ids + * IDs array given by app to retrieve specific statistics + * @param size + * The size of the xstats_names array (number of elements). + * @return + * - A positive value lower or equal to size: success. The return value + * is the number of entries filled in the stats table. + * - A positive value higher than size: error, the given statistics table + * is too small. The return value corresponds to the size that should + * be given to succeed. The entries in the table are not valid and + * shall not be used by the caller. + * - A negative value on error (invalid port id). + */ +int +rte_eth_xstats_get_names_by_id(uint8_t port_id, + struct rte_eth_xstat_name *xstats_names, unsigned int size, + uint64_t *ids); + +/** + * Retrieve extended statistics of an Ethernet device. + * + * @param port_id + * The port identifier of the Ethernet device. + * @param ids + * A pointer to an ids array passed by application. This tells wich + * statistics values function should retrieve. This parameter + * can be set to NULL if n is 0. In this case function will retrieve + * all avalible statistics. + * @param values + * A pointer to a table to be filled with device statistics values. + * @param n + * The size of the ids array (number of elements). + * @return + * - A positive value lower or equal to n: success. The return value + * is the number of entries filled in the stats table. + * - A positive value higher than n: error, the given statistics table + * is too small. The return value corresponds to the size that should + * be given to succeed. The entries in the table are not valid and + * shall not be used by the caller. + * - A negative value on error (invalid port id). + */ +int rte_eth_xstats_get_by_id(uint8_t port_id, const uint64_t *ids, + uint64_t *values, unsigned int n); + +/** * Reset extended statistics of an Ethernet device. * * @param port_id diff --git a/lib/librte_ether/rte_ether_version.map b/lib/librte_ether/rte_ether_version.map index 238c2a1..1abd717 100644 --- a/lib/librte_ether/rte_ether_version.map +++ b/lib/librte_ether/rte_ether_version.map @@ -151,5 +151,7 @@ DPDK_17.05 { rte_eth_dev_attach_secondary; rte_eth_find_next; + rte_eth_xstats_get_by_id; + rte_eth_xstats_get_names_by_id; } DPDK_17.02; -- 2.7.4